An optimization method, system, terminal device and storage medium for AI large model

Abstract

The present invention discloses an optimization method, system, terminal device and storage medium for an AI large model. The method predicts the real-time load data of each sub-model through a resource prediction model, predicts the resource demand of each sub-model in a future period of time, and then determines the resource allocation strategy, and predicts the real-time user behavior data through a behavior prediction model to generate a sub-model request prediction result, and determines the preloading strategy for the sub-model request prediction result through a cache consistency protocol and a cache strategy. Finally, the resource allocation strategy is used to dynamically and reasonably adjust the resource allocation to reduce the loss of computing resources, and the preloading strategy is used to preload each sub-model of the AI ​​large model to reduce the model loading time, thereby improving the response speed and throughput of each sub-model in the AI ​​large model, and solving the problem that many sub-models currently have a poor user experience due to slow response speed and low throughput.

Description

Technical Field

[0001] The present invention relates to the field of optimization of AI large models, and in particular to an optimization method, system, terminal device and storage medium for AI large models. Background Art

[0002] In the current AI technology ecosystem, it has become a trend to integrate multiple sub-models into a large model to provide comprehensive services. The AI ​​large model integrating multiple sub-models can be widely used in many different fields to improve people's work efficiency and quality of life. Different sub-models can provide different functions. For example, the question-and-answer sub-model can provide question-and-answer functions, and the painting sub-model can provide painting functions. However, many sub-models currently have slow response speeds and low throughput, resulting in poor user experience. Summary of the invention

[0003] The present invention provides an optimization method and system for a large AI model, which solves the problem that many sub-models currently have poor user experience due to slow response speed and low throughput.

[0004] In order to solve the above technical problems, an embodiment of the present invention provides an optimization method for a large AI model, including:

[0005] Obtain real-time user behavior data and real-time load data of each sub-model of the AI ​​model;

[0006] Input the real-time load data of each sub-model of the AI ​​big model into the trained resource prediction model, so that the trained resource prediction model can make predictions based on the real-time load data of each sub-model to obtain the resource requirements of each sub-model in the future;

[0007] Determine a resource allocation strategy according to the resource requirements of each sub-model in the future period of time;

[0008] Input the real-time user behavior data into the trained behavior prediction model, so that the trained behavior prediction model makes predictions based on the real-time user behavior data to obtain a sub-model request prediction result;

[0009] According to the prediction result of the sub-model request, the preloading strategy is determined through the cache consistency protocol and cache strategy;

[0010] Optimize large AI models based on resource allocation strategies and preloading strategies.

[0011] Furthermore, the model training of the resource prediction model includes:

[0012] Obtain historical load data of each sub-model of the AI ​​large model;

[0013] Performing data preprocessing on the historical load data of each sub-model of the AI ​​large model to obtain processed historical load data of each sub-model, and dividing the processed historical load data of each sub-model into a first training set and a first validation set;

[0014] By using the LSTM layer of the resource prediction model to be trained, time series feature extraction is performed on the first training set to generate time series load feature data;

[0015] Through the self-attention layer of the resource prediction model to be trained, the time series load feature data is enhanced to obtain the enhanced time series load feature data;

[0016] The first fully connected layer of the resource prediction model to be trained is used to integrate and nonlinearly transform the enhanced time series load characteristic data to generate the resource requirements of each sub-model in the future period.

[0017] The loss value is calculated by comparing the resource demand of each sub-model in the future with the resource demand of each sub-model in the actual period of time, and the parameters of the resource prediction model to be trained are optimized according to the loss value until the loss value converges to obtain a preliminarily trained resource prediction model;

[0018] The performance of the initially trained resource prediction model is evaluated through the first validation set, and the model hyperparameters are adjusted according to the performance evaluation results to obtain a trained resource prediction model.

[0019] Furthermore, determining the resource allocation strategy according to the resource requirements of each sub-model in the future period of time includes:

[0020] Determine the priority of resource requirements of each sub-model according to the resource requirements of each sub-model in the future period of time;

[0021] According to the total running time and failure time of the resource, combined with the availability percentage evaluation index calculation formula, the availability percentage evaluation index value is calculated, and the resource availability is judged according to the availability percentage evaluation index value to determine the available resources;

[0022] Dynamically adjust resource allocation for each sub-model based on the priority of each sub-model's resource requirements and available resources.

[0023] Furthermore, the model training of the behavior prediction model includes:

[0024] Obtain historical user behavior data;

[0025] Performing data preprocessing on the historical user behavior data to obtain processed historical user behavior data, and dividing the processed historical user behavior data into a second training set and a second validation set;

[0026] Extracting features from the second training set through the TCN layer in the behavior prediction model to be trained to obtain first time series behavior feature data;

[0027] Extracting features from the first time series behavior feature data through a GRU layer in the behavior prediction model to be trained to obtain second time series behavior feature data;

[0028] Performing feature enhancement on the second time series behavior feature data through a self-attention layer in the behavior prediction model to be trained to obtain third time series behavior feature data;

[0029] Performing data integration and nonlinear transformation on the third time series behavior feature data through the second fully connected layer in the behavior prediction model to be trained, and generating a sub-model request prediction result;

[0030] The loss value is calculated by comparing the prediction result of the sub-model request with the actual sub-model request result, and the parameters of the behavior prediction model to be trained are optimized according to the loss value until the loss value converges to obtain a preliminarily trained behavior prediction model;

[0031] The performance of the initially trained behavior prediction model is evaluated through the second validation set, and the model hyperparameters are adjusted according to the performance evaluation results to obtain a trained behavior prediction model.

[0032] Furthermore, the method of determining the preloading strategy through the cache consistency protocol and the cache strategy according to the prediction result of the sub-model request includes:

[0033] Determine the frequency of each sub-model request according to the prediction result of the sub-model request, and determine the priority of each sub-model request according to the frequency of each sub-model request;

[0034] According to the priority of each sub-model request and the preset cache capacity, an active cache loading strategy is adopted to load the sub-model data with high request priority into the cache;

[0035] When the sub-model data with high request priority is loaded into the cache, the medium and low priority requests are processed in order of priority according to the remaining cache capacity, and the corresponding sub-model data is loaded into the cache;

[0036] When the sub-model data stored in the cache is applied, the sub-model data that has been applied is eliminated and replaced through the cache elimination strategy;

[0037] During the process of loading or eliminating sub-model data, the state of the sub-model data in the cache is managed through the cache consistency protocol.

[0038] Furthermore, the GRU layer in the behavior prediction model to be trained includes an update gate, a reset gate and a hidden state update; wherein the hidden state update includes a candidate hidden state update and a current hidden state update;

[0039] The calculation formula of the update gate is as follows: ;in, is the weight matrix of the update gate; is the bias vector of the update gate; is the hidden state of the previous time step; is the input feature vector of the current time step; is the sigmoid activation function; Indicates the value of the update gate;

[0040] The calculation formula of the reset gate is as follows: ;in, is the weight matrix of the reset gate; is the bias vector for the reset gate; is the input feature vector of the current time step; Indicates the value of the reset gate;

[0041] The calculation formula for updating the candidate hidden state is as follows: ;in, is the weight matrix of the candidate hidden states; is the bias vector of the candidate hidden state; represents element-wise multiplication; represents the hyperbolic tangent activation function; represents the value of the candidate hidden state;

[0042] The calculation formula for updating the current hidden state is as follows: ;in, Represents the value of the update gate; Indicates the value of the reset gate; represents the candidate hidden state; represents element-wise multiplication; The value representing the current hidden state.

[0043] Based on the above method embodiment, the present invention provides a corresponding system embodiment;

[0044] An embodiment of the present invention provides an AI large model optimization system, including a data acquisition module, a resource demand prediction module, a resource allocation strategy determination module, a sub-model request prediction module, a preloading strategy determination module and an AI large model optimization module;

[0045] The data acquisition module is used to obtain real-time user behavior data and real-time load data of each sub-model of the AI ​​large model;

[0046] The resource demand prediction module is used to input the real-time load data of each sub-model of the AI ​​large model into the trained resource prediction model, so that the trained resource prediction model can make predictions based on the real-time load data of each sub-model to obtain the resource demand of each sub-model in the future period of time;

[0047] The resource allocation strategy determination module is used to determine the resource allocation strategy according to the resource requirements of each sub-model in the future period of time;

[0048] The sub-model request prediction module is used to input the real-time user behavior data into the trained behavior prediction model, so that the trained behavior prediction model makes predictions based on the real-time user behavior data to obtain the sub-model request prediction results;

[0049] The preloading strategy determination module is used to determine the preloading strategy through the cache consistency protocol and cache strategy according to the prediction result of the sub-model request;

[0050] The AI ​​large model optimization module is used to optimize the AI ​​large model according to the resource allocation strategy and the preloading strategy.

[0051] Furthermore, the optimization system of the AI ​​large model further includes: a model training module; wherein the model training module includes a resource prediction model training unit and a behavior prediction model training unit;

[0052] The resource prediction model training unit is used for model training of the resource prediction model;

[0053] The behavior prediction model training unit is used for model training of the behavior prediction model.

[0054] Based on the above-mentioned method embodiment, the present invention provides a corresponding terminal device embodiment, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements an optimization method for an AI large model as described in the present invention.

[0055] Based on the above-mentioned method embodiment, the present invention provides a corresponding computer-readable storage medium embodiment, including: a stored computer program, which controls the device where the computer-readable storage medium is located to execute an optimization method for an AI large model as described in the present invention when the computer program is running.

[0056] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0057] The present invention predicts the real-time load data of each sub-model through a resource prediction model, generates the resource requirements of each sub-model in the future, and then determines the resource allocation strategy, and predicts the real-time user behavior data through a behavior prediction model to generate a sub-model request prediction result, and determines the preloading strategy for the sub-model request prediction result through a cache consistency protocol and a cache strategy. Finally, the AI ​​large model is optimized according to the resource allocation strategy and the preloading strategy, that is, the resource allocation strategy is used to dynamically and reasonably adjust the resource allocation of the AI ​​large model to reduce the loss of computing resources, and the preloading strategy is used to preload each sub-model of the AI ​​large model to reduce the model loading time and improve the response speed and throughput of each sub-model in the AI ​​large model, which solves the problem that many sub-models currently have poor user experience due to slow response speed and low throughput. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A flowchart of a method for optimizing a large AI model provided by an embodiment of the present invention;

[0059] Figure 2 A system structure diagram of an AI large model optimization system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] In the description of the present invention, it should be understood that the terms “first” and “second” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0062] Embodiment 1:

[0063] Please refer to Figure 1 , is a flowchart of a method for optimizing an AI large model provided by an embodiment of the present invention, the method comprising at least the following steps:

[0064] Step S1: Obtain real-time user behavior data and real-time load data of each sub-model of the AI ​​large model;

[0065] In this embodiment, real-time user behavior data is obtained by technical means including but not limited to tracking technology, data collection tools, log analysis tools and real-time data processing platforms, and real-time load data of each sub-model of the AI ​​big model is obtained by technical means including but not limited to monitoring tools, log analysis tools, resource managers and API interfaces; wherein the tracking technology is to implant data collection code in key positions or functions of an application (such as an APP) or a website to capture and report information about user interaction with the application or website; the data collection tools include but are not limited to Google Analytics and Baidu Statistics; the log analysis tools include ELK Stack, Splunk, Graylog, Sumo Logic, Fluentd and Loggly; the real-time data processing platform includes but is not limited to Apache Kafka and Apache Storm; the monitoring tools include but are not limited to Prometheus and Grafana; the resource managers include but are not limited to AWS's CloudWatch and Azure's Monitor; the API interface is to obtain the real-time load data of each sub-model of the AI ​​big model by calling the API interface of the AI ​​big model.

[0066] Step S2: input the real-time load data of each sub-model of the AI ​​big model into the trained resource prediction model, so that the trained resource prediction model makes predictions based on the real-time load data of each sub-model to obtain the resource requirements of each sub-model in the future;

[0067] In this embodiment, the resource prediction model is constructed by combining a long short-term memory neural network LSTM and a self-attention mechanism; the model structure of the resource prediction model includes an input layer, an LSTM layer, a self-attention layer, a first fully connected layer and an output layer.

[0068] In this embodiment, the model training of the resource prediction model includes:

[0069] Obtain historical load data of each sub-model in the AI ​​large model;

[0070] In this embodiment, the real-time load data of each sub-model of the AI ​​large model is obtained through technical means including but not limited to monitoring tools, log analysis tools, resource managers and API interfaces.

[0071] Performing data preprocessing on the historical load data of each sub-model of the AI ​​large model to obtain processed historical load data of each sub-model, and dividing the processed historical load data of each sub-model into a first training set and a first validation set;

[0072] In this embodiment, the data preprocessing includes but is not limited to data cleaning, data smoothing, data conversion and data dimension reduction; wherein, the data cleaning is to process missing values, outliers and error values ​​of the data; the data smoothing is to smooth the data by smoothing technology, and the smoothing technology includes but is not limited to moving average, weighted moving average and exponential smoothing; the data includes but is not limited to logarithmic transformation, normalization and standardization; the data dimension reduction method includes principal component analysis and linear discriminant analysis.

[0073] By using the LSTM layer of the resource prediction model to be trained, time series feature extraction is performed on the first training set to generate time series load feature data;

[0074] In this embodiment, the LSTM layer of the resource prediction model to be trained is composed of multiple LSTM units; each LSTM unit processes input data at each time step of the time series and passes the state information to the next time step; each LSTM unit includes a forget gate, an input gate, a unit state update and an output gate; the calculation formula of the forget gate is ; The calculation formula of the input gate is: and ; The calculation formula for updating the unit state is: ; The calculation formula of the output gate is and ;in, is the sigmoid function; is the hyperbolic tangent activation function, represents element-by-element multiplication, W represents weight, b represents bias vector, and are the hidden state vectors of the previous time step and the current time step, respectively, is the input data of the current time step, and are the cell states at the previous and current time steps, respectively.

[0075] Through the self-attention layer of the resource prediction model to be trained, the time series load feature data is enhanced to obtain the enhanced time series load feature data;

[0076] In this embodiment, the dependencies between different time steps of the time series load feature data are identified through the self-attention layer, and attention weights are assigned to the time series load feature data according to the dependencies to obtain enhanced time series load feature data.

[0077] The first fully connected layer of the resource prediction model to be trained is used to integrate and nonlinearly transform the enhanced time series load characteristic data to generate the resource requirements of each sub-model in the future period.

[0078] In this embodiment, the enhanced time series load characteristic data is integrated through the first fully connected layer of the resource prediction model to be trained, and nonlinear transformation processing is performed through the activation function to generate the resource requirements of each sub-model in the future period of time; wherein the activation function includes but is not limited to the Sigmoid function, the Tanh function and the ReLU function.

[0079] The loss value is calculated by comparing the resource demand of each sub-model in the future with the resource demand of each sub-model in the actual period of time, and the parameters of the resource prediction model to be trained are optimized according to the loss value until the loss value converges to obtain a preliminarily trained resource prediction model;

[0080] In this embodiment, according to the resource requirements of each sub-model in a future period of time and the resource requirements of each sub-model in an actual period of time, the loss value is calculated through the loss function, and the parameters of the resource prediction model to be trained are optimized and adjusted according to the loss value through the back propagation algorithm or the optimizer until the loss value converges to obtain a preliminary trained resource prediction model; wherein, the loss function includes but is not limited to the mean square error function, the mean absolute error function and the smoothed L1 loss function; the optimizer includes but is not limited to SGD, Adam and RMSprop.

[0081] The performance of the initially trained resource prediction model is evaluated through the first validation set, and the model hyperparameters are adjusted according to the performance evaluation results to obtain a trained resource prediction model;

[0082] In this embodiment, the performance evaluation includes but is not limited to accuracy, recall and F1 value.

[0083] Step S3: determining a resource allocation strategy according to the resource requirements of each sub-model in the future period of time;

[0084] In this embodiment, determining the resource allocation strategy according to the resource requirements of each sub-model in the future period of time includes:

[0085] Determine the priority of resource requirements of each sub-model according to the resource requirements of each sub-model in the future period of time;

[0086] According to the total operation data and fault data of the resources, combined with the availability percentage evaluation index calculation formula, the availability percentage evaluation index value is calculated, and the resource availability is judged according to the availability percentage evaluation index value to determine the available resources;

[0087] In this embodiment, the availability percentage evaluation index value is calculated based on the total operation data and fault data of the resource in combination with the availability percentage evaluation index calculation formula, and the resource availability is judged by comparing the availability percentage evaluation index value with a preset threshold value, thereby determining the available resources; wherein, the availability percentage evaluation index calculation formula is: availability percentage evaluation index value = (total operation time - fault time) / total operation time * 100%.

[0088] Dynamically adjust resource allocation for each sub-model based on the priority of each sub-model's resource requirements and available resources.

[0089] Step S4: input the real-time user behavior data into the trained behavior prediction model, so that the trained behavior prediction model makes predictions based on the real-time user behavior data to obtain a sub-model request prediction result;

[0090] In this embodiment, the model structure of the behavior prediction model is a TCN-GRU-Attention model.

[0091] In this embodiment, the model training of the behavior prediction model includes:

[0092] Obtain historical user behavior data;

[0093] In this embodiment, real-time user behavior data is obtained through technical means including but not limited to tracking technology, data collection tools, log analysis tools and real-time data processing platforms;

[0094] Performing data preprocessing on the historical user behavior data to obtain processed historical user behavior data, and dividing the processed historical user behavior data into a second training set and a second validation set;

[0095] In this embodiment, the data preprocessing includes but is not limited to data cleaning, data smoothing, data conversion and data dimension reduction; wherein, the data cleaning is to process missing values, outliers and error values ​​of the data; the data smoothing is to smooth the data by smoothing technology, and the smoothing technology includes but is not limited to moving average, weighted moving average and exponential smoothing; the data includes but is not limited to logarithmic transformation, normalization and standardization; the data dimension reduction method includes principal component analysis and linear discriminant analysis.

[0096] Extracting features from the second training set through the TCN layer in the behavior prediction model to be trained to obtain first time series behavior feature data;

[0097] In this embodiment, the second training set is feature extracted through several convolutional layers in the TCN layer to obtain several behavior feature data, the several behavior feature data are nonlinearly transformed through a nonlinear activation function to obtain several transformed behavior feature data, and finally the several transformed behavior feature data are residually connected through a residual connection to obtain the first time series behavior feature data;

[0098] Extracting features from the first time series behavior feature data through a GRU layer in the behavior prediction model to be trained to obtain second time series behavior feature data;

[0099] Performing feature enhancement on the second time series behavior feature data through a self-attention layer in the behavior prediction model to be trained to obtain third time series behavior feature data;

[0100] Performing data integration and nonlinear transformation on the third time series behavior feature data through the second fully connected layer in the behavior prediction model to be trained, and generating a sub-model request prediction result;

[0101] The loss value is calculated by comparing the prediction result of the sub-model request with the actual sub-model request result, and the parameters of the behavior prediction model to be trained are optimized according to the loss value until the loss value converges to obtain a preliminarily trained behavior prediction model;

[0102] The performance of the initially trained behavior prediction model is evaluated through the second validation set, and the model hyperparameters are adjusted according to the performance evaluation results to obtain a trained behavior prediction model.

[0103] In this embodiment, the GRU layer in the behavior prediction model to be trained includes an update gate, a reset gate and a hidden state update; wherein the hidden state update includes a candidate hidden state update and a current hidden state update;

[0104] The calculation formula of the update gate is as follows: ;in, is the weight matrix of the update gate; is the bias vector of the update gate; is the hidden state of the previous time step; is the input feature vector of the current time step; is the sigmoid activation function; Indicates the value of the update gate;

[0105] The calculation formula of the reset gate is as follows: ;in, is the weight matrix of the reset gate; is the bias vector for the reset gate; is the input feature vector of the current time step; Indicates the value of the reset gate;

[0106] The calculation formula for updating the candidate hidden state is as follows: ;in, is the weight matrix of the candidate hidden states; is the bias vector of the candidate hidden state; represents element-wise multiplication; represents the hyperbolic tangent activation function; represents the value of the candidate hidden state;

[0107] The calculation formula for updating the current hidden state is as follows: ;in, Indicates the value of the update gate; Indicates the value of the reset gate; represents the candidate hidden state; represents element-wise multiplication; The value representing the current hidden state.

[0108] Step S5: According to the prediction result of the sub-model request, the preloading strategy is determined through the cache consistency protocol and the cache strategy;

[0109] In this embodiment, the prediction result of the sub-model request is determined based on the cache consistency protocol and the cache strategy, including:

[0110] Determine the frequency of each sub-model request according to the prediction result of the sub-model request, and determine the priority of each sub-model request according to the frequency of each sub-model request;

[0111] According to the priority of each sub-model request and the preset cache capacity, an active cache loading strategy is adopted to load the sub-model data with high request priority into the cache;

[0112] When the sub-model data with high request priority is loaded into the cache, the medium and low priority requests are processed in order of priority according to the remaining cache capacity, and the corresponding sub-model data is loaded into the cache;

[0113] When the sub-model data stored in the cache is applied, the sub-model data that has been applied is eliminated and replaced through the cache elimination strategy;

[0114] In this embodiment, the cache elimination strategy includes but is not limited to the least recently used strategy, the least frequently used strategy, the first-in-first-out elimination strategy, the elimination strategy based on the cache size, and the elimination strategy based on the cache item life cycle;

[0115] During the process of loading or eliminating sub-model data, the state of the sub-model data in the cache is managed through the cache consistency protocol.

[0116] Step S6: Optimize the AI ​​large model according to the resource allocation strategy and preloading strategy.

[0117] In this embodiment, resources are dynamically adjusted and allocated to the AI ​​big model through the resource allocation strategy, and the sub-models in the AI ​​big model are preloaded according to priority through the preloading strategy, so as to improve the response speed and throughput of each sub-model in the AI ​​big model.

[0118] Embodiment 2:

[0119] Please refer to Figure 2 , which is a system structure diagram of an AI large model optimization system provided by an embodiment of the present invention, the system includes: a data acquisition module, a resource demand prediction module, a resource allocation strategy determination module, a sub-model request prediction module, a preloading strategy determination module and an AI large model optimization module;

[0120] The data acquisition module is used to obtain real-time user behavior data and real-time load data of each sub-model of the AI ​​large model;

[0121] The resource demand prediction module is used to input the real-time load data of each sub-model of the AI ​​large model into the trained resource prediction model, so that the trained resource prediction model can make predictions based on the real-time load data of each sub-model to obtain the resource demand of each sub-model in the future period of time;

[0122] The resource allocation strategy determination module is used to determine the resource allocation strategy according to the resource requirements of each sub-model in the future period of time;

[0123] The sub-model request prediction module is used to input the real-time user behavior data into the trained behavior prediction model, so that the trained behavior prediction model makes predictions based on the real-time user behavior data to obtain the sub-model request prediction results;

[0124] The preloading strategy determination module is used to determine the preloading strategy through the cache consistency protocol and cache strategy according to the prediction result of the sub-model request;

[0125] The AI ​​large model optimization module is used to optimize the AI ​​large model according to the resource allocation strategy and the preloading strategy.

[0126] In this embodiment, the optimization system of the AI ​​large model further includes: a model training module; wherein the model training module includes a resource prediction model training unit and a behavior prediction model training unit;

[0127] The resource prediction model training unit is used for model training of the resource prediction model;

[0128] The behavior prediction model training unit is used for model training of the behavior prediction model.

[0129] Based on the above method embodiment, another embodiment is provided;

[0130] Another embodiment of the present invention provides an AI large model optimization terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements an AI large model optimization method described in any one of the above method embodiments of the present invention.

[0131] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the optimization terminal device of the AI ​​large model.

[0132] The optimization terminal device of the AI ​​large model can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The optimization terminal device of the AI ​​large model can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that, for example, the optimization terminal device of the AI ​​large model can also include input and output devices, network access devices, buses, etc.

[0133] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the optimization terminal device of the AI ​​large model, and uses various interfaces and lines to connect the various parts of the optimization terminal device of the entire AI large model.

[0134] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the optimization terminal device of the AI ​​large model by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0135] Based on the above method embodiment, another embodiment is provided;

[0136] A storage medium provided by another embodiment of the present invention includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute an optimization method for an AI large model described in any one of the above method embodiments of the present invention.

[0137] Wherein, the above-mentioned storage medium is a computer-readable storage medium, and the module / unit integrated in the optimization system / terminal device of the AI ​​large model, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0138] It should be noted that the above-mentioned terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the above-mentioned terminal device is merely an example and does not constitute a limitation on the terminal device. It may include more or fewer components, or a combination of certain components, or different components.

[0139] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. An optimization method for a large AI model, characterized in that: include: Obtain real-time user behavior data and real-time load data of each sub-model of the AI ​​model; Input the real-time load data of each sub-model of the AI ​​big model into the trained resource prediction model, so that the trained resource prediction model can make predictions based on the real-time load data of each sub-model to obtain the resource requirements of each sub-model in the future; Determine a resource allocation strategy according to the resource requirements of each sub-model in the future period of time; Input the real-time user behavior data into the trained behavior prediction model, so that the trained behavior prediction model makes predictions based on the real-time user behavior data to obtain a sub-model request prediction result; According to the prediction result of the sub-model request, the preloading strategy is determined through the cache consistency protocol and cache strategy; Optimize large AI models based on resource allocation and preloading strategies; The determining of the resource allocation strategy according to the resource requirements of each sub-model in the future period of time includes: Determine the priority of resource requirements of each sub-model according to the resource requirements of each sub-model in the future period of time; According to the total running time and failure time of the resource, combined with the availability percentage evaluation index calculation formula, the availability percentage evaluation index value is calculated, and the resource availability is judged according to the availability percentage evaluation index value to determine the available resources; Dynamically adjust resource allocation for each sub-model based on the priority of resource requirements of each sub-model and available resources; The model training of the behavior prediction model includes: Obtain historical user behavior data; Performing data preprocessing on the historical user behavior data to obtain processed historical user behavior data, and dividing the processed historical user behavior data into a second training set and a second validation set; Extracting features from the second training set through the TCN layer in the behavior prediction model to be trained to obtain first time series behavior feature data; Extracting features from the first time series behavior feature data through a GRU layer in the behavior prediction model to be trained to obtain second time series behavior feature data; Performing feature enhancement on the second time series behavior feature data through a self-attention layer in the behavior prediction model to be trained to obtain third time series behavior feature data; Performing data integration and nonlinear transformation on the third time series behavior feature data through the second fully connected layer in the behavior prediction model to be trained, and generating a sub-model request prediction result; The loss value is calculated by comparing the prediction result of the sub-model request with the actual sub-model request result, and the parameters of the behavior prediction model to be trained are optimized according to the loss value until the loss value converges to obtain a preliminarily trained behavior prediction model; The performance of the initially trained behavior prediction model is evaluated through the second validation set, and the model hyperparameters are adjusted according to the performance evaluation results to obtain a trained behavior prediction model; The method of requesting prediction results based on the sub-model and determining the preloading strategy through a cache consistency protocol and a cache strategy includes: Determine the frequency of each sub-model request according to the prediction result of the sub-model request, and determine the priority of each sub-model request according to the frequency of each sub-model request; According to the priority of each sub-model request and the preset cache capacity, an active cache loading strategy is adopted to load the sub-model data with high request priority into the cache; When the sub-model data with high request priority is loaded into the cache, the medium and low priority requests are processed in order of priority according to the remaining cache capacity, and the corresponding sub-model data is loaded into the cache; When the sub-model data stored in the cache is applied, the sub-model data that has been applied is eliminated and replaced through the cache elimination strategy; During the process of loading or eliminating sub-model data, the state of the sub-model data in the cache is managed through the cache consistency protocol.

2. The optimization method of an AI large model according to claim 1, characterized in that: The model training of the resource prediction model includes: Obtain historical load data of each sub-model of the AI ​​large model; Performing data preprocessing on the historical load data of each sub-model of the AI ​​large model to obtain processed historical load data of each sub-model, and dividing the processed historical load data of each sub-model into a first training set and a first validation set; By using the LSTM layer of the resource prediction model to be trained, time series feature extraction is performed on the first training set to generate time series load feature data; Through the self-attention layer of the resource prediction model to be trained, the time series load feature data is enhanced to obtain the enhanced time series load feature data; The first fully connected layer of the resource prediction model to be trained is used to integrate and nonlinearly transform the enhanced time series load characteristic data to generate the resource requirements of each sub-model in the future period. The loss value is calculated by comparing the resource demand of each sub-model in the future period with the resource demand of each sub-model in the actual period, and the parameters of the resource prediction model to be trained are optimized according to the loss value until the loss value converges to obtain a preliminarily trained resource prediction model; The performance of the initially trained resource prediction model is evaluated through the first validation set, and the model hyperparameters are adjusted according to the performance evaluation results to obtain a trained resource prediction model.

3. The optimization method of an AI large model according to claim 1, characterized in that: The GRU layer in the behavior prediction model to be trained includes an update gate, a reset gate and a hidden state update; wherein the hidden state update includes a candidate hidden state update and a current hidden state update; The calculation formula of the update gate is as follows: ;in, is the weight matrix of the update gate; is the bias vector of the update gate; is the hidden state of the previous time step; is the input feature vector of the current time step; is the sigmoid activation function; Indicates the value of the update gate; The calculation formula of the reset gate is as follows: ;in, is the weight matrix of the reset gate; is the bias vector for the reset gate; is the input feature vector of the current time step; Represents the value of the reset gate; The calculation formula for updating the candidate hidden state is as follows: ;in, is the weight matrix of the candidate hidden states; is the bias vector of the candidate hidden state; represents element-wise multiplication; represents the hyperbolic tangent activation function; represents the value of the candidate hidden state; The calculation formula for updating the current hidden state is as follows: ;in, Indicates the value of the update gate; Represents the value of the reset gate; represents the candidate hidden state; represents element-wise multiplication; The value representing the current hidden state.

4. An optimization system for a large AI model, characterized in that: include: Data collection module, resource demand prediction module, resource allocation strategy determination module, sub-model request prediction module, preloading strategy determination module and AI large model optimization module; The data acquisition module is used to obtain real-time user behavior data and real-time load data of each sub-model of the AI ​​large model; The resource demand prediction module is used to input the real-time load data of each sub-model of the AI ​​large model into the trained resource prediction model, so that the trained resource prediction model can make predictions based on the real-time load data of each sub-model to obtain the resource demand of each sub-model in the future period of time; The resource allocation strategy determination module is used to determine the resource allocation strategy according to the resource requirements of each sub-model in the future period of time; The sub-model request prediction module is used to input the real-time user behavior data into the trained behavior prediction model, so that the trained behavior prediction model makes predictions based on the real-time user behavior data to obtain the sub-model request prediction results; The preloading strategy determination module is used to determine the preloading strategy through the cache consistency protocol and the cache strategy according to the prediction result of the sub-model request; The AI ​​large model optimization module is used to optimize the AI ​​large model according to the resource allocation strategy and the preloading strategy; The determining of the resource allocation strategy according to the resource requirements of each sub-model in the future period of time includes: Determine the priority of resource requirements of each sub-model according to the resource requirements of each sub-model in the future period of time; According to the total running time and failure time of the resource, combined with the availability percentage evaluation index calculation formula, the availability percentage evaluation index value is calculated, and the resource availability is judged according to the availability percentage evaluation index value to determine the available resources; Dynamically adjust resource allocation for each sub-model based on the priority of resource requirements of each sub-model and available resources; The model training of the behavior prediction model includes: Obtain historical user behavior data; Performing data preprocessing on the historical user behavior data to obtain processed historical user behavior data, and dividing the processed historical user behavior data into a second training set and a second validation set; Extracting features from the second training set through the TCN layer in the behavior prediction model to be trained to obtain first time series behavior feature data; Extracting features from the first time series behavior feature data through a GRU layer in the behavior prediction model to be trained to obtain second time series behavior feature data; Performing feature enhancement on the second time series behavior feature data through a self-attention layer in the behavior prediction model to be trained to obtain third time series behavior feature data; Performing data integration and nonlinear transformation on the third time series behavior feature data through the second fully connected layer in the behavior prediction model to be trained, and generating a sub-model request prediction result; The loss value is calculated by comparing the prediction result of the sub-model request with the actual sub-model request result, and the parameters of the behavior prediction model to be trained are optimized according to the loss value until the loss value converges to obtain a preliminarily trained behavior prediction model; The performance of the initially trained behavior prediction model is evaluated through the second validation set, and the model hyperparameters are adjusted according to the performance evaluation results to obtain a trained behavior prediction model; The method of requesting prediction results based on the sub-model and determining the preloading strategy through a cache consistency protocol and a cache strategy includes: Determine the frequency of each sub-model request according to the prediction result of the sub-model request, and determine the priority of each sub-model request according to the frequency of each sub-model request; According to the priority of each sub-model request and the preset cache capacity, an active cache loading strategy is adopted to load the sub-model data with high request priority into the cache; When the sub-model data with high request priority is loaded into the cache, the medium and low priority requests are processed in order of priority according to the remaining cache capacity, and the corresponding sub-model data is loaded into the cache; When the sub-model data stored in the cache is applied, the sub-model data that has been applied is eliminated and replaced through the cache elimination strategy; During the process of loading or eliminating sub-model data, the state of the sub-model data in the cache is managed through the cache consistency protocol.

5. The optimization system of an AI large model according to claim 4, characterized in that: Also includes: Model training module; wherein the model training module includes a resource prediction model training unit and a behavior prediction model training unit; The resource prediction model training unit is used for model training of the resource prediction model; The behavior prediction model training unit is used for model training of the behavior prediction model.

6. An optimization terminal device for an AI large model, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements an optimization method for an AI large model as described in any one of claims 1 to 3.

7. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute an optimization method for an AI large model as described in any one of claims 1 to 3.

Images