Big model inference optimization method and device based on background information fusion, equipment and medium

Abstract

The application discloses a large model inference optimization method and device based on background information fusion, equipment and medium, relates to the field of artificial intelligence, including: determining the background information related to the to-be-processed task; the background information includes user interaction record information, task environment information and domain knowledge information related to the to-be-processed task; the background information and the input data of the large model are fused to obtain target information; the input data is the data input by the user side to the large model based on the to-be-processed task; the target inference path is determined based on the target information, the background information and the task demand of the to-be-processed task, so that the to-be-processed task is executed by using the large model and the target inference path, and the corresponding inference result is obtained; after the to-be-processed task is executed, the feedback information corresponding to the inference result is determined, and the parameters of the large model are adjusted based on the feedback information, so that the inference process of the large model is optimized. Therefore, the application can improve the inference efficiency and accuracy of the large model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method, apparatus, device, and medium for optimizing large-scale model reasoning based on background information fusion. Background Technology

[0002] With the rapid development of deep learning and artificial intelligence technologies, large models have been widely applied in many fields such as natural language processing and computer vision.

[0003] Currently, these models, due to their large parameter scale, require significant computational resources to achieve high-precision inference and have significant shortcomings. First, existing inference methods generally lack utilization of context and background information. Most models infer solely from input data, failing to fully utilize user interaction records, environmental information, domain knowledge, and other background content. This limits the model's ability to handle complex tasks, making it difficult to understand the task's environment and accurately complete complex inference. Second, traditional large-scale models have fixed inference paths, relying on a computational path involving the entire model regardless of task complexity or computational resource usage. This approach leads to overcomputation and resource waste when handling simple tasks; while for complex tasks, it struggles to guarantee accurate inference.

[0004] Therefore, improving the inference efficiency and accuracy of large models in different task scenarios is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for optimizing large model inference based on background information fusion, which can improve the inference efficiency and accuracy of large models in different task scenarios. The specific solution is as follows:

[0006] Firstly, this application provides a large-scale model inference optimization method based on background information fusion, including:

[0007] When the large model performs inference based on the task to be processed, it determines the background information related to the task to be processed; the background information includes user interaction record information, task environment information and domain knowledge information related to the task to be processed.

[0008] The background information and the input data of the large model are fused to obtain the target information; the input data is the data input by the user terminal to the large model based on the task to be processed.

[0009] Based on the target information, the background information, and the task requirements of the task to be processed, a target inference path is determined so as to use the large model and the target inference path to execute the task to be processed and obtain the corresponding inference results.

[0010] Once the task to be processed is completed, the feedback information corresponding to the reasoning result is determined, and the parameters of the large model are adjusted based on the feedback information to optimize the reasoning process of the large model.

[0011] Optionally, determining the background information related to the task to be processed includes:

[0012] Extract the task type, task domain, and task keywords of the task to be processed to determine the corresponding task characteristics;

[0013] The correlation between each preset data source and the task to be processed is evaluated based on the task characteristics, and the corresponding evaluation results are obtained.

[0014] Based on the evaluation results, a target data source is determined from each of the preset data sources, and information is retrieved from the target data source using the task characteristics to obtain background information related to the task to be processed.

[0015] Optionally, fusing the background information and the input data of the large model to obtain the target information includes:

[0016] Based on the background information, a background information vector is determined, and the background information vector is input into a pre-trained embedding model to transform the background information vector into a first embedding vector using the first preset activation function, the first preset bias term, and the preset embedding transformation matrix in the pre-trained embedding model.

[0017] The input data vector is determined based on the input data of the large model, and the second embedding vector is determined based on the input data vector.

[0018] The first embedding vector and the second embedding vector are fused using a preset attention mechanism and the dimension of the first embedding vector, and the resulting fused vector is used as the target information.

[0019] Optionally, determining the target inference path based on the target information, the background information, and the task requirements of the task to be processed includes:

[0020] Determine the task requirements of the task to be processed, and determine the task requirement vector based on the task requirements;

[0021] Select one inference path from the various inference paths of the large model as the current inference path;

[0022] The path activation probability of the current inference path is determined based on the second preset activation function, the target information, the first embedding vector, and the second preset bias term.

[0023] Jump to the step of selecting one inference path from the inference paths of the large model as the current inference path, until all inference paths of the large model have been selected and the activation probability of each path is obtained.

[0024] In each inference path of the large model, the inference path whose activation probability is greater than a preset probability threshold is determined as the initial inference path;

[0025] The accuracy and computational cost of the initial inference path are determined, and based on the accuracy, computational cost, and a preset path loss algorithm, the initial inference path with the minimum path loss value is determined, and the initial inference path with the minimum path loss value is determined as the target inference path.

[0026] Optionally, the step of using the large model and the target inference path to execute the task to be processed and obtain the corresponding inference result includes:

[0027] The target reasoning path is decomposed into reasoning steps, and the large model is driven to perform reasoning step by step based on the reasoning steps and the task to be processed.

[0028] During the stepwise reasoning process, the output result of the current reasoning step of the large model is determined, and it is judged whether the output result is greater than the preset confidence threshold to obtain the judgment result;

[0029] If the judgment result is greater than, then the output result of the current reasoning step is used as the input data for the next reasoning step;

[0030] If the judgment result is not greater than, then a local inference fragment is generated based on the task to be processed, and the target content in the current inference step is replaced by the local inference fragment to obtain the replaced inference step, and the large model is driven to execute the replaced inference step.

[0031] Once all the reasoning steps have been completed, the corresponding reasoning result is determined.

[0032] Optionally, adjusting the parameters of the large model based on the feedback information includes:

[0033] The quantity of feedback information is determined, the feedback analysis result is determined based on the feedback information and the quantity, and the loss value is determined using the current parameters of the large model, the feedback information, and a preset loss function.

[0034] The feedback analysis result and the loss value are added together, and the resulting sum is multiplied by the learning rate of the large model to obtain the corresponding product value.

[0035] The target model parameters are determined using the product value and the current parameters of the large model, and the current parameters of the large model are adjusted using the target model parameters.

[0036] Secondly, this application provides a large-scale model inference optimization device based on background information fusion, comprising:

[0037] The information determination module is used to determine background information related to the task to be processed when the large model performs inference based on the task to be processed; the background information includes user interaction record information, task environment information and domain knowledge information related to the task to be processed.

[0038] The information fusion module is used to fuse the background information and the input data of the large model to obtain the target information; the input data is the data input by the user terminal to the large model based on the task to be processed.

[0039] The task reasoning module is used to determine the target reasoning path based on the target information, the background information, and the task requirements of the task to be processed, so as to execute the task to be processed using the large model and the target reasoning path and obtain the corresponding reasoning results.

[0040] The inference optimization module is used to determine the feedback information corresponding to the inference result after the task to be processed is completed, and adjust the parameters of the large model based on the feedback information to optimize the inference process of the large model.

[0041] Thirdly, this application provides an electronic device, comprising:

[0042] Memory, used to store computer programs;

[0043] A processor is used to execute the computer program to implement the aforementioned large model inference optimization method based on background information fusion.

[0044] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned large model inference optimization method based on background information fusion.

[0045] In this application, when the large model performs inference based on the task to be processed, background information related to the task to be processed is determined; the background information includes user interaction record information, task environment information, and domain knowledge information related to the task to be processed; the background information and the input data of the large model are fused to obtain target information; the input data is the data input by the user to the large model based on the task to be processed; a target inference path is determined based on the target information, the background information, and the task requirements of the task to be processed, so as to use the large model and the target inference path to execute the task to be processed and obtain the corresponding inference result; when the task to be processed is completed, feedback information corresponding to the inference result is determined, and the parameters of the large model are adjusted based on the feedback information to optimize the inference process of the large model. As can be seen from the above, in this application, during inference, the large model first determines the background information related to the task to be processed; it then integrates this background information with the input data from the user terminal to the large model for the task to obtain the target information; based on the target information, background information, and task requirements, it determines the target inference path, and uses the large model and this target inference path to execute the task to be processed and obtain the inference result; after the task is completed, it determines the feedback information corresponding to the inference result, and adjusts the parameters of the large model accordingly to optimize the inference process. In this way, this application enables the inference of the large model to adapt to the task scenario, thereby improving the inference accuracy of the large model. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0047] Figure 1 This is a flowchart of a large model reasoning optimization method based on background information fusion disclosed in this application;

[0048] Figure 2 This application discloses a specific flowchart of a large model inference optimization method based on background information fusion.

[0049] Figure 3 This is a schematic diagram of the structure of a large model inference optimization device based on background information fusion disclosed in this application;

[0050] Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Currently, large-scale models, due to their massive parameter size, require significant computational resources to achieve high-precision inference, and exhibit significant shortcomings. Firstly, existing inference methods generally lack utilization of context and background information. Most models rely solely on input data for inference, failing to fully utilize user interaction records, environmental information, domain knowledge, and other background content. This limits the model's ability to handle complex tasks, making it difficult to understand the task's environment and accurately complete complex inference. Secondly, traditional large-scale model inference paths are fixed, relying on a computational path involving the entire model regardless of task complexity or computational resource usage. This approach leads to overcomputation and resource waste when handling simple tasks; while for complex tasks, it struggles to guarantee accurate inference. Therefore, this application provides a large-scale model inference optimization method, apparatus, device, and medium based on background information fusion, which can improve the inference efficiency and accuracy of large-scale models.

[0053] See Figure 1 As shown, this embodiment of the invention discloses a large model inference optimization method based on background information fusion, including:

[0054] Step S11: When the large model performs inference based on the task to be processed, determine the background information related to the task to be processed; the background information includes user interaction record information, task environment information and domain knowledge information related to the task to be processed.

[0055] It should be noted that the large model in this embodiment can be applied to the fields of natural language processing, computer vision, and recommendation systems, and improve the processing effect of tasks in various fields through background information fusion technology.

[0056] Specifically, in the field of natural language processing, this large model can be applied to intelligent customer service scenarios. For example, when a user inquires about the after-sales service policy of an e-commerce platform, the large model, by integrating the user's historical order information, domain knowledge of the platform's after-sales rules, and the context of the current inquiry, can not only accurately understand the user's needs but also recommend suitable solutions based on the user's past purchase records, such as a fast return and exchange process or repair service discounts, making customer service responses more targeted and efficient.

[0057] In the field of computer vision, large-scale models can be used for intelligent security monitoring. Taking airport security checks as an example, when the system detects suspected prohibited items in luggage, the large-scale model integrates the item's visual features, such as prohibited item lists, airport security domain knowledge of size specifications, and real-time environmental information such as passenger flow in security lanes and equipment operating status. This not only enables accurate identification of item categories but also automatically prompts subsequent processing steps according to security procedures, such as guiding luggage for secondary inspection or notifying manual verification, thereby improving the accuracy and efficiency of security checks.

[0058] In the field of recommendation systems, this large-scale model can be applied to content recommendation on short video platforms. When users browse short videos, the model combines their historical viewing history, likes, favorites, and other interaction information with knowledge of short video tag classification, trending topics, and other areas, while also considering the user's current usage environment, such as network status, to recommend content that matches their interests and preferences and fits the current context. For example, during a user's evening leisure time, based on their past interest in technology content and the current good network environment, the model prioritizes recommending the latest technology news short videos, improving the user's viewing experience and recommendation satisfaction.

[0059] In this embodiment, specifically, after receiving a task to be processed, the large model first parses and processes the task. Natural language processing techniques are used to extract the task type, such as question-answering, text generation, or logical reasoning; simultaneously, the domain to which the task belongs is determined, such as the medical, financial, or educational fields. Furthermore, key terms and phrases are extracted from the task to form a more comprehensive description of its features.

[0060] Next, based on the extracted task features, the relevance of each preset data source to the task to be processed is evaluated. Preset data sources include, but are not limited to, historical user interaction databases, real-time environmental parameter acquisition systems, and professional domain knowledge bases. During the evaluation process, the degree of matching between information in each preset data source and task features is analyzed, or whether there are key environmental variables in the real-time environmental parameters that affect task reasoning, thereby obtaining the corresponding evaluation results.

[0061] Furthermore, based on the evaluation results, target data sources with high relevance to the task to be processed are selected from various preset data sources. For each target data source, relevant information is retrieved using task characteristics. For example, if the target data source is a user interaction record database, and the task characteristics indicate an intelligent customer service task in the field of natural language processing, then the database is retrieved for user history consultation records, order information, and other interaction records. If the target data source is an environmental information collection system, and the task characteristics involve intelligent security monitoring tasks in the field of computer vision, then real-time environmental parameters such as light intensity and camera angle in the monitoring scene are retrieved. If the target data source is a domain knowledge base, and the task belongs to short video content recommendation in the field of recommendation systems, then knowledge information in areas such as video tag classification and user behavior preference models is retrieved, ultimately obtaining background information closely related to the task to be processed.

[0062] Step S12: Fuse the background information and the input data of the large model to obtain target information; the input data is the data input by the user terminal to the large model based on the task to be processed.

[0063] In this embodiment, after obtaining background information related to the task to be processed, the background information needs to be vectorized and embedded. First, based on the semantic content and structural features of the background information, natural language processing techniques such as word embedding and sentence vector generation are used to determine the background information vector. Then, the background information vector is input into a pre-trained embedding model, and a first preset activation function in the embedding model is used to perform a nonlinear transformation on the background information vector. Combined with a first preset bias term, the vector distribution is adjusted, and then a preset embedding transformation matrix is ​​used to map and transform the vector dimension and feature space. Finally, the background information vector is transformed into a first embedding vector adapted to the large model inference architecture.

[0064] Simultaneously, vector processing is performed on the input data of the large model. The input data includes task data in the form of text, images, and speech input from the user. Based on the type and format of the input data, the input data vector can be determined through corresponding encoding methods. After obtaining the input data vector, a second embedding vector is determined through linear transformation, dimension alignment, and other operations, based on the inference requirements of the large model and the features of the input layer, so that this vector is in the same feature space dimension as the first embedding vector.

[0065] Finally, a pre-defined attention mechanism is used to fuse the first and second embedding vectors. This pre-defined attention mechanism can employ classic attention architectures such as self-attention or cross-attention, determining the matrix dimension and weight allocation rules for attention calculation based on the dimension of the first embedding vector. During the fusion process, the attention mechanism automatically focuses on information features more valuable to the reasoning task by calculating the semantic association weights between background information and input data, suppressing the influence of irrelevant or redundant information. For example, in the intelligent customer service scenario of natural language processing, the attention mechanism can strengthen the association feature weights between the user's historical consultation records and the current input question; in the security monitoring scenario of computer vision, it can highlight the key correspondence between environmental parameters and image features. Through the weighted fusion operation of the attention mechanism, the first and second embedding vectors are integrated into a fused vector. This fused vector contains both the core task features of the input data and incorporates contextual associations and domain knowledge from the background information. Finally, the fused vector is used as the target information input to the large model, providing information support for subsequent reasoning optimization.

[0066] Step S13: Determine the target inference path based on the target information, the background information, and the task requirements of the task to be processed, so as to use the large model and the target inference path to execute the task to be processed and obtain the corresponding inference results.

[0067] In this embodiment, after obtaining the fused target information, the specific requirements of the task to be processed must first be clarified. By parsing the semantic content and parameter indicators of the task input, the task requirements are determined, such as the required level of detail in the intelligent customer service scenario, the real-time requirements for target recognition in the computer vision scenario, and the personalized dimensions that need to be met in the recommendation system. These requirements are then transformed into a task requirement vector.

[0068] Next, one of the multiple inference paths pre-defined by the large model is selected as the current evaluation object. The inference paths of the large model are constructed based on different algorithmic logics and knowledge retrieval strategies. For example, natural language processing may include rule-based inference paths and deep learning-based end-to-end inference paths, while computer vision may include inference paths based on traditional feature engineering and inference paths based on novel architectures. When selecting the current inference path, heuristic strategies such as polling, random selection, or based on historical performance can be used to ensure coverage of all inference paths.

[0069] After determining the current inference path, the activation probability of the current inference path is calculated. This process uses a second preset activation function to jointly process the target information and the first embedding vector, and adjusts the probability distribution by incorporating a second preset bias term. The task features in the target information and the contextual relationships in the background information directly affect the adaptability of the inference path: when the target information contains complex domain knowledge, the inference path that is deeply integrated with the domain knowledge base will obtain a higher activation probability; if the real-time environmental parameters account for a large proportion of the target information, the inference path that focuses on real-time data processing will be more advantageous.

[0070] The process of selecting inference paths and calculating activation probabilities is repeated until all inference paths have been evaluated, forming a set of activation probabilities for each path. Then, a probability threshold is set, and paths with activation probabilities higher than the threshold are selected as initial inference paths; these paths are considered to have high suitability for the current task.

[0071] Two key metrics are further evaluated for the initial inference path. The first is accuracy, measured by the inference results on validation data, such as the accuracy of classification tasks and the error magnitude of regression tasks. The second is computational cost, which involves statistically analyzing resource consumption during the inference process, including computational load, memory usage, and inference time. Based on these two metrics, a pre-defined loss algorithm is used to calculate the loss value of each path. This algorithm balances the weights of accuracy and efficiency, avoiding the selection of paths with high accuracy but excessive computational cost, or paths with low computational cost but insufficient accuracy. Finally, the initial inference path with the smallest loss value is selected as the target inference path, ensuring that inference efficiency is maximized while meeting the task's accuracy requirements.

[0072] After determining the target inference path, it is broken down into a series of ordered inference steps. Then, the large model is driven to execute the inference task according to these steps. After each step, the output result is obtained and its confidence level is evaluated. If the confidence level is higher than a preset confidence threshold, the result of the current step is considered reliable and is directly used as the input for the next step. If the confidence level is insufficient, a local inference fragment is generated based on task requirements and background information to replace the uncertain content in the current step, and the step is re-executed until the result meets the confidence requirement.

[0073] After all inference steps are completed, the outputs of each stage are integrated to form the final inference result. This process ensures that large models achieve accurate and efficient inference decisions in complex tasks through dynamic optimization of the target inference path and confidence control of the inference steps, avoiding inference biases caused by unreasonable path selection or unreliable intermediate results.

[0074] Step S14: After the task to be processed is completed, determine the feedback information corresponding to the reasoning result, and adjust the parameters of the large model based on the feedback information to optimize the reasoning process of the large model.

[0075] In this embodiment, after the task to be processed is completed and the reasoning result is obtained, it is first necessary to collect feedback information related to the reasoning result. Feedback information sources include, but are not limited to, direct user evaluations of the reasoning result, the degree of matching between the reasoning result and the actual scenario, and user interaction behavior with recommended content in the recommendation system.

[0076] After obtaining feedback information, the first step is to count the number of feedback messages. The number of feedback messages reflects the richness of the data sample. When the number of feedback messages is small, it is necessary to consider whether insufficient samples may lead to analytical bias; when the number is sufficient, a more comprehensive statistical analysis can be performed. Based on the specific content and quantity of the feedback information, corresponding feedback analysis results are generated.

[0077] Simultaneously, the loss value is calculated using the current parameters of the large model, feedback information, and a preset loss function. That is, the expected result implied in the feedback information is compared with the inference result generated by the large model based on the current parameters, and the deviation is converted into a numerical loss value through the loss function. The larger this value, the lower the match between the current model parameters and the task requirements.

[0078] The feedback analysis results are fused with the loss value. The feedback analysis results qualitatively indicate the direction of the model's inference deficiencies, while the loss value quantitatively measures the degree of bias. The two are added together to obtain a sum, which is then multiplied by the learning rate of the larger model to obtain the parameter update product. The learning rate controls the step size of each parameter adjustment; if the learning rate is too large, it may lead to over-updating and deviation from the optimal solution; if it is too small, the optimization efficiency will decrease. The calculation of the product essentially determines a reasonable parameter adjustment range based on the degree of bias and the direction of optimization.

[0079] Finally, the target model parameters are calculated using the product value and the current parameters of the large model. The current parameters of the large model are then replaced with the target model parameters to complete the model parameter adjustment. This adjustment process dynamically optimizes the model's parameter configuration in areas such as background information fusion, inference path selection, and inference step execution, addressing problems exposed during inference. This allows the large model to more accurately fuse background information and more efficiently select inference paths when processing similar tasks in the future, thereby improving the overall inference performance of the large model.

[0080] As can be seen from the above, in this application, during inference, the large model first determines the background information related to the task to be processed; it then integrates this background information with the input data from the user terminal to the large model for the task to obtain the target information; based on the target information, background information, and task requirements, it determines the target inference path, and uses the large model and this target inference path to execute the task to be processed and obtain the inference result; after the task is completed, it determines the feedback information corresponding to the inference result, and adjusts the parameters of the large model accordingly to optimize the inference process. In this way, this application enables the inference of the large model to adapt to the task scenario, thereby improving the inference accuracy of the large model.

[0081] The following is combined with Figure 2 The schematic diagram shown illustrates the technical solution of the embodiments of this application in detail.

[0082] Specifically, firstly, the background information acquisition module obtains task-related (i.e., the task to be processed) background information from multiple data sources, providing richer contextual support for large-scale model inference. Based on the different characteristics of the task, suitable background information sources are determined. Task-related background information is extracted from the selected data sources. For example, in natural language processing tasks, relevant concept definitions or contextual interaction records can be extracted from a knowledge base. Furthermore, the extracted background information is standardized using the formula shown below:

[0083] ;

[0084] In the formula, It contains background information. yes The average value, yes The standard deviation.

[0085] Next, the background information fusion module fuses the preprocessed background information with the original input data of the large model to ensure that the large model can effectively understand the context required for the current task.

[0086] Using a pre-trained embedding model, background information is transformed into a model-compatible embedding representation. First, the background information is converted into a background information vector. Then, the embedding transformation process converts the background information vector... Convert to first embedding vector The formula for embedding transformation is: In this formula, It is the first preset activation function. It is a preset embedded transformation matrix. It is the first preset bias term. Furthermore, the second embedding vector is determined based on the input data.

[0087] An attention mechanism is introduced into the background information fusion to highlight the information most relevant to the task requirements. The background information embedding layer assigns different weights to different pieces of information, causing the large model to focus more on information that is more relevant to the current task, thereby improving the accuracy of inference.

[0088] The attention mechanism fuses background information and original input data by calculating attention weights; the fused vector represents the target information. In other words, given the background information embedding vector (i.e., the first embedding vector) and the original input data embedding vector (i.e., the second embedding vector), the fused representation calculated by the attention mechanism can be expressed by the following formula:

[0089] ;

[0090] In the formula, The dimension of the vector embedded for background information. Embed vectors for the original input data. It is a normalized exponential function.

[0091] Furthermore, the embedded Represented as Furthermore, based on background information and the fused vectors, the dynamic inference optimization module adjusts the model's inference paths in real time to optimize computational efficiency. By utilizing background information and task requirements, the complexity of the current task is determined, thereby selectively activating certain inference paths of the model. In this embodiment, the activation probability of the inference path is calculated using the following formula, and an inference path is selected based on each activation probability.

[0092] ;

[0093] In the formula, This indicates the second preset activation function. , as well as For relative to , as well as The weight parameters, It is a vector representing the task requirements. It is the second bias term.

[0094] The inference path optimization algorithm is used to monitor and select paths in real time. The following formula can be used:

[0095] ;

[0096] In the formula, It is the weight of accuracy. It is the weight of the computational quantity. It is the accuracy of the reasoning path. It is the computational cost corresponding to the reasoning path. This indicates the choice of the reasoning path that minimizes the loss.

[0097] Then, the initial inference path with the minimum path loss value is determined as the target inference path. This target inference path is then decomposed into inference steps, driving the large model to infer the inference steps and the task at hand, thereby obtaining the corresponding inference results. The inference results are then displayed on the user's device, and the user can provide feedback on these results.

[0098] Finally, after the task is completed, collect feedback information on the reasoning results. Analyze the feedback data to evaluate the effectiveness of background information fusion and the rationality of the current reasoning path selection. The impact of the feedback information can be evaluated using the following formula:

[0099] ;

[0100] In the formula, It is the amount of feedback information. It is the first One piece of feedback information.

[0101] Based on feedback information, large models can utilize adaptive learning to automatically adjust their inference strategies in similar tasks, enabling continuous improvement. Adaptive learning can be expressed by the formula:

[0102] ;

[0103] In the formula, These are the old parameters of the large model. These are the updated parameters. It's the learning rate. It is the loss function calculated based on feedback information under the old parameters of the large model. It is the result of feedback analysis. It represents the weight of the feedback analysis results.

[0104] Accordingly, see Figure 3 As shown, this application provides a large model inference optimization device based on background information fusion, including:

[0105] The information determination module 11 is used to determine background information related to the task to be processed when the large model performs inference based on the task to be processed; the background information includes user interaction record information, task environment information and domain knowledge information related to the task to be processed.

[0106] The information fusion module 12 is used to fuse the background information and the input data of the large model to obtain the target information; the input data is the data input by the user terminal to the large model based on the task to be processed;

[0107] The task reasoning module 13 is used to determine the target reasoning path based on the target information, the background information and the task requirements of the task to be processed, so as to use the large model and the target reasoning path to execute the task to be processed and obtain the corresponding reasoning results.

[0108] The inference optimization module 14 is used to determine the feedback information corresponding to the inference result after the task to be processed is completed, and adjust the parameters of the large model based on the feedback information to optimize the inference process of the large model.

[0109] As can be seen from the above, in this application, during inference, the large model first determines the background information related to the task to be processed; it then integrates this background information with the input data from the user terminal to the large model for the task to obtain the target information; based on the target information, background information, and task requirements, it determines the target inference path, and uses the large model and this target inference path to execute the task to be processed and obtain the inference result; after the task is completed, it determines the feedback information corresponding to the inference result, and adjusts the parameters of the large model accordingly to optimize the inference process. In this way, this application enables the inference of the large model to adapt to the task scenario, thereby improving the inference accuracy of the large model.

[0110] In some specific embodiments, the information determination module 11 specifically includes:

[0111] The feature determination unit is used to extract the task type, task domain, and task keywords of the task to be processed in order to determine the corresponding task features.

[0112] The first result determination unit is used to evaluate the correlation between each preset data source and the task to be processed based on the task characteristics, and obtain the corresponding evaluation results.

[0113] The information determination unit is used to determine the target data source from each of the preset data sources based on the evaluation results, and to perform information retrieval in the target data source using the task characteristics to obtain background information related to the task to be processed.

[0114] In some specific embodiments, the information fusion module 12 specifically includes:

[0115] The vector transformation unit is used to determine a background information vector based on the background information, and input the background information vector into a pre-trained embedding model to transform the background information vector into a first embedding vector using the first preset activation function, the first preset bias term and the preset embedding transformation matrix in the pre-trained embedding model.

[0116] The first vector determination unit is used to determine an input data vector based on the input data of the large model, and to determine a second embedding vector based on the input data vector;

[0117] The vector fusion unit is used to fuse the first embedding vector and the second embedding vector using a preset attention mechanism and the dimension of the first embedding vector, and use the fused vector as the target information.

[0118] In some specific embodiments, the task reasoning module 13 specifically includes:

[0119] The second vector determination unit is used to determine the task requirements of the task to be processed, and to determine the task requirement vector based on the task requirements.

[0120] The first path determination unit is used to select one inference path from the various inference paths of the large model as the current inference path;

[0121] The first probability determination unit is used to determine the path activation probability of the current inference path based on the second preset activation function, the target information, the first embedding vector, and the second preset bias term.

[0122] The second probability determination unit is used to jump to the step of selecting one inference path from each inference path of the large model as the current inference path, until all inference paths of the large model have been selected and the activation probability of each path is obtained.

[0123] The second path determination unit is used to determine the inference path with an activation probability greater than a preset probability threshold as the initial inference path among the inference paths of the large model.

[0124] The third path determination unit is used to determine the accuracy and computational cost of the initial inference path, and based on the accuracy, computational cost and a preset path loss algorithm, determine the initial inference path with the minimum path loss value among the initial inference paths, and determine the initial inference path with the minimum path loss value as the target inference path.

[0125] In some specific embodiments, the task reasoning module 13 specifically includes:

[0126] The task reasoning unit is used to decompose the target reasoning path into reasoning steps and drive the large model to perform reasoning step by step based on the reasoning steps and the task to be processed.

[0127] The result judgment unit is used to determine the output result of the large model executing the current inference step during the stepwise inference process, and to determine whether the output result is greater than a preset confidence threshold, so as to obtain the judgment result.

[0128] An input data determination unit is used to take the output result of the current inference step as the input data for the next inference step if the judgment result is greater than 1.

[0129] The step execution unit is used to generate a local inference fragment based on the task to be processed if the judgment result is not greater than the target content in the current inference step, and use the local inference fragment to replace the target content in the current inference step to obtain the replaced inference step, and drive the large model to execute the replaced inference step.

[0130] The second result determination unit is used to determine the corresponding reasoning result after all the reasoning steps have been executed.

[0131] In some specific embodiments, the inference optimization module 14 specifically includes:

[0132] The loss value determination unit is used to determine the quantity of the feedback information, determine the feedback analysis result based on the feedback information and the quantity, and determine the loss value using the current parameters of the large model, the feedback information, and a preset loss function.

[0133] The product value determination unit is used to add the feedback analysis result and the loss value, and multiply the resulting sum by the learning rate of the large model to obtain the corresponding product value;

[0134] The model adjustment unit is used to determine the target model parameters using the product value and the current parameters of the large model, and to adjust the current parameters of the large model using the target model parameters.

[0135] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the large model inference optimization method based on background information fusion disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0136] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0137] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0138] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the large model inference optimization method based on background information fusion executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0139] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed large model inference optimization method based on background information fusion. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0140] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0141] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0143] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0144] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A large-scale model inference optimization method based on background information fusion, characterized in that, include: When the large model used for content recommendation on short video platforms performs inference based on the task to be processed, background information related to the task to be processed is determined. The background information includes user interaction records, task environment information, and domain knowledge information related to the task to be processed; the user interaction records include user viewing records and user likes and favorites on the short video platform; the domain knowledge information includes the tag classification of the short video. The background information and the input data of the large model are fused to obtain the target information; the input data is the data input by the user terminal to the large model based on the task to be processed. Based on the target information, the background information, and the task requirements of the task to be processed, a target inference path is determined so as to use the large model and the target inference path to execute the task to be processed and obtain the corresponding inference results. After the task to be processed is completed, the feedback information corresponding to the reasoning result is determined, and the parameters of the large model are adjusted based on the feedback information to optimize the reasoning process of the large model. The step of determining the target inference path based on the target information, the background information, and the task requirements of the task to be processed includes: The task requirements of the task to be processed are determined, and a task requirement vector is determined based on the task requirements; an inference path is selected from the inference paths of the large model as the current inference path; the path activation probability of the current inference path is determined based on a second preset activation function, the target information, a first embedding vector, and a second preset bias term; wherein, the first embedding vector is a background information vector determined based on the background information, and the background information vector is input into a pre-trained embedding model to utilize the first preset activation function, the first preset bias term, and the preset embedding transformation matrix in the pre-trained embedding model to transform the background information vector. The transformed vector is then used to proceed to the step of selecting one inference path from the various inference paths of the large model as the current inference path, until all inference paths of the large model have been selected and the activation probability of each path is obtained. Among the various inference paths of the large model, the inference path with an activation probability greater than a preset probability threshold is determined as the initial inference path. The accuracy and computational complexity of the initial inference path are determined, and based on the accuracy, computational complexity, and a preset path loss algorithm, the initial inference path with the smallest path loss value is determined, and the initial inference path with the smallest path loss value is determined as the target inference path. The step of using the large model and the target inference path to execute the task to be processed and obtain the corresponding inference result includes: The target inference path is decomposed into inference steps, and the large model is driven to perform inference step by step based on the inference steps and the task to be processed. During the step-by-step inference process, the output result of the large model executing the current inference step is determined, and it is judged whether the output result is greater than a preset confidence threshold to obtain a judgment result. If the judgment result is greater than, the output result of the current inference step is used as the input data for the next inference step. If the judgment result is not greater than, a local inference fragment is generated based on the task to be processed, and the target content in the current inference step is replaced by the local inference fragment to obtain a replaced inference step, and the large model is driven to execute the replaced inference step. When all the inference steps have been executed, the corresponding inference result is determined. The step of adjusting the parameters of the large model based on the feedback information includes: The quantity of feedback information is determined, and a feedback analysis result is determined based on the feedback information and the quantity. A loss value is determined using the current parameters of the large model, the feedback information, and a preset loss function. The feedback analysis result and the loss value are added together, and the resulting sum is multiplied by the learning rate of the large model to obtain a corresponding product value. The target model parameters are determined using the product value and the current parameters of the large model, and the current parameters of the large model are adjusted using the target model parameters.

2. The large model inference optimization method based on background information fusion according to claim 1, characterized in that, The determination of background information related to the task to be processed includes: Extract the task type, task domain, and task keywords of the task to be processed to determine the corresponding task characteristics; The correlation between each preset data source and the task to be processed is evaluated based on the task characteristics, and the corresponding evaluation results are obtained. Based on the evaluation results, a target data source is determined from each of the preset data sources, and information is retrieved from the target data source using the task characteristics to obtain background information related to the task to be processed.

3. The large model inference optimization method based on background information fusion according to claim 1, characterized in that, The process of fusing the background information and the input data of the large model to obtain target information includes: The input data vector is determined based on the input data of the large model, and the second embedding vector is determined based on the input data vector. The first embedding vector and the second embedding vector are fused using a preset attention mechanism and the dimension of the first embedding vector, and the resulting fused vector is used as the target information.

4. A large-scale model inference optimization device based on background information fusion, characterized in that, include: The information determination module is used to determine background information related to the task to be processed when the large model for content recommendation on the short video platform makes inferences based on the task to be processed. The background information includes user interaction records, task environment information, and domain knowledge information related to the task to be processed; the user interaction records include user viewing records and user likes and favorites on the short video platform; the domain knowledge information includes the tag classification of the short video. The information fusion module is used to fuse the background information and the input data of the large model to obtain the target information; the input data is the data input by the user terminal to the large model based on the task to be processed. The task reasoning module is used to determine the target reasoning path based on the target information, the background information, and the task requirements of the task to be processed, so as to execute the task to be processed using the large model and the target reasoning path and obtain the corresponding reasoning results. The inference optimization module is used to determine the feedback information corresponding to the inference result after the task to be processed is completed, and adjust the parameters of the large model based on the feedback information to optimize the inference process of the large model. The task reasoning module includes: The second vector determination unit is used to determine the task requirements of the task to be processed, and to determine the task requirement vector based on the task requirements. The first path determination unit is used to select one inference path from the various inference paths of the large model as the current inference path; The first probability determination unit is used to determine the path activation probability of the current inference path based on the second preset activation function, the target information, the first embedding vector, and the second preset bias term; wherein, the first embedding vector is a background information vector determined based on the background information, and the background information vector is input into a pre-trained embedding model to transform the background information vector using the first preset activation function, the first preset bias term, and the preset embedding transformation matrix in the pre-trained embedding model. The second probability determination unit is used to jump to the step of selecting one inference path from each inference path of the large model as the current inference path, until all inference paths of the large model have been selected and the activation probability of each path is obtained. The second path determination unit is used to determine the inference path with an activation probability greater than a preset probability threshold as the initial inference path among the inference paths of the large model. The third path determination unit is used to determine the accuracy and computational cost of the initial inference path, and based on the accuracy, computational cost and a preset path loss algorithm, determine the initial inference path with the minimum path loss value among the initial inference paths, and determine the initial inference path with the minimum path loss value as the target inference path. The task reasoning module includes: The task reasoning unit is used to decompose the target reasoning path into reasoning steps and drive the large model to perform reasoning step by step based on the reasoning steps and the task to be processed. The result judgment unit is used to determine the output result of the large model executing the current inference step during the stepwise inference process, and to determine whether the output result is greater than a preset confidence threshold, so as to obtain the judgment result. An input data determination unit is used to take the output result of the current inference step as the input data for the next inference step if the judgment result is greater than 1. The step execution unit is used to generate a local inference fragment based on the task to be processed if the judgment result is not greater than the target content in the current inference step, and use the local inference fragment to replace the target content in the current inference step to obtain the replaced inference step, and drive the large model to execute the replaced inference step. The second result determination unit is used to determine the corresponding reasoning result after all the reasoning steps have been executed. The inference optimization module includes: The loss value determination unit is used to determine the quantity of the feedback information, determine the feedback analysis result based on the feedback information and the quantity, and determine the loss value using the current parameters of the large model, the feedback information, and a preset loss function. The product value determination unit is used to add the feedback analysis result and the loss value, and multiply the resulting sum by the learning rate of the large model to obtain the corresponding product value; The model adjustment unit is used to determine the target model parameters using the product value and the current parameters of the large model, and to adjust the current parameters of the large model using the target model parameters.

5. The large model inference optimization device based on background information fusion according to claim 4, characterized in that, The inference optimization module includes: The loss value determination unit is used to determine the quantity of the feedback information, determine the feedback analysis result based on the feedback information and the quantity, and determine the loss value using the current parameters of the large model, the feedback information, and a preset loss function. The product value determination unit is used to add the feedback analysis result and the loss value, and multiply the resulting sum by the learning rate of the large model to obtain the corresponding product value; The model adjustment unit is used to determine the target model parameters using the product value and the current parameters of the large model, and to adjust the current parameters of the large model using the target model parameters.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the large model inference optimization method based on background information fusion as described in any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the large model inference optimization method based on background information fusion as described in any one of claims 1 to 3.

Images