Regression type large model training method and device
Through the regression-based large-model training method, the input data is directly converted into feature vectors for training, which solves the problem of large-model training method consumes a lot of GPU memory in the existing technology, and realizes a wider application scenario and a lower model illusion.
Patent Information
- Application Number
- CN202411443778.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-03-21
AI Technical Summary
The training method of existing large models requires the pre-quantitative model to convert the original data into tokens, resulting in large GPU memory consumption, limited application scenarios, and increasing the illusion of the model.
The regression-based big model training method is adopted. By obtaining the input data and converting it into the input feature vector, the big model is directly input to the big model for prediction, the regression loss of the predicted feature vector and the actual feature vector is calculated, and the model is trained based on the loss result, and the token quantization process is removed.
It reduces the illusion of the big model, saves GPU memory, and expands the application scenarios, avoiding the performance losses caused by token quantization.
Smart Images
Figure CN119670874A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of model training technology, and more specifically, to a regression-type large model training method and device. Background Art
[0002] Large models, especially those based on deep learning and artificial intelligence, such as models under the Transformer architecture (such as GPT-3, BERT, etc.), have made significant progress in natural language processing, computer vision and other fields. These models are trained with large amounts of data and parameters, and can achieve performance close to or even exceeding human levels on a variety of tasks. With development, the application of large models is no longer limited to a single task, but has expanded to multimodal and multitask learning, covering a wide range of fields such as natural language processing, image generation, machine translation, and dialogue systems. These models have stronger generalization and migration capabilities, and can share knowledge between different tasks and improve the efficiency of collaboration between tasks.
[0003] The fundamental principle of the existing large model is to use neural networks to train token classification tasks using the training method of classification tasks (for example: softmax activation function + cross entropy); however, the existing training method requires a pre-quantization model to convert the original data into tokens (positive integer type), which consumes a lot of GPU memory and has limited application scenarios. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide a regression-based large model training method and device to solve the drawbacks of classification-based large models.
[0005] In a first aspect, an embodiment of the present application provides a regression large model training method, comprising:
[0006] Get input data and convert it into input feature vector;
[0007] Input the input feature vector into the large model for prediction to obtain the predicted feature vector;
[0008] The predicted feature vector is combined with the actual feature vector to calculate the regression loss, and the model is trained based on the calculation results.
[0009] In the above implementation process, the input data is obtained and converted into an input feature vector; the input feature vector is input into the large model for prediction to obtain a predicted feature vector; the predicted feature vector is combined with the actual feature vector to calculate the regression loss, and the model is trained according to the calculation results; the idea of regression tasks is used to train the large model, and the token quantization process is removed, which can avoid the loss caused by token quantization, thereby reducing the illusion of the large model, saving GPU memory, and having a wide range of application scenarios.
[0010] Furthermore, the step of inputting the input feature vector into the large model for prediction to obtain the predicted feature vector includes:
[0011] Calculate the eigenvector sequence through the characteristic function:
[0012] V=f(X,ψ)=v 1 ,v 2 ,…v t-1 ;
[0013] Among them, ψ is the parameter of the feature function f; X is the input feature, X = x 1 ,x 2 ,x 3 …x t-1 ; v is a unit eigenvector;
[0014] Compute the predicted feature vector using the generative large model:
[0015]
[0016] Among them, g is the generative large model, and θ is the parameter of the generative large model.
[0017] In the above implementation process, the feature vector of t is predicted by the feature vector sequence from 1 to t-1, so that the regression loss calculation is calculated by the predicted feature vector and the actual feature vector.
[0018] Furthermore, the step of calculating the regression loss by combining the predicted feature vector with the actual feature vector includes:
[0019] Compute the current actual eigenvector:
[0020]
[0021] in, is the current input feature, y is the actual eigenvector at time t;
[0022] Combine the predicted feature vector with the actual feature vector to calculate the mean square loss function:
[0023]
[0024] Where N is the dimension of the actual feature vector.
[0025] In the above implementation process, the predicted feature vector is combined with the actual feature vector to calculate the regression loss, and the model training process is guided by the regression loss calculation result to help the model continuously optimize its prediction ability.
[0026] Furthermore, the step of obtaining input data and converting the input data into an input feature vector includes:
[0027] Input data is obtained, and an input feature vector is extracted from the input data through a feature extraction algorithm.
[0028] In the above implementation process, the input data is directly converted into input feature vectors, the token embedding layer and feature layer of the large model are discarded, the tokenizer quantization process is removed, the performance loss caused by quantization is avoided, the hallucination problem of the model can be reduced, and a wider range of application scenarios is ensured.
[0029] Furthermore, the step of obtaining input data and extracting an input feature vector from the input data by a feature extraction algorithm includes:
[0030] Input text data is obtained, and feature vectors are extracted from the input text data through a text vector extraction model.
[0031] In the above implementation process, the feature vector of the text is extracted to realize model training in the text scenario.
[0032] Furthermore, the step of obtaining input data and extracting an input feature vector from the input data by a feature extraction algorithm includes:
[0033] The input image data is obtained, and the feature vector in the input image data is extracted through the image recognition model.
[0034] In the above implementation process, the feature vector of the image is extracted to realize model training in the image scene.
[0035] Furthermore, the step of obtaining input data and extracting an input feature vector from the input data by a feature extraction algorithm includes:
[0036] The input speech data is obtained, and a feature vector in the input speech data is extracted by a speech feature extraction algorithm.
[0037] In the above implementation process, the feature vector of speech is extracted to realize model training in the speech scene.
[0038] In a second aspect, an embodiment of the present application provides a large regression model training device, comprising:
[0039] A feature acquisition module, used to acquire input data and convert the input data into an input feature vector;
[0040] A feature prediction module is used to input the input feature vector into the large model for prediction to obtain a predicted feature vector;
[0041] The regression loss module is used to calculate the regression loss by combining the predicted feature vector with the actual feature vector, and train the model based on the calculation results.
[0042] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0043] A processor, a memory and a bus, wherein the processor is connected to the memory via the bus, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, they are used to implement the regression large model training method as described above.
[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a server, the regression large model training method as described above is implemented.
[0045] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions, and when the instructions are executed by a computer, the computer implements the regression large model training method as described above. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0047] Figure 1 A flowchart of a large regression model training method provided in an embodiment of the present application;
[0048] Figure 2 It is a structural schematic diagram of a regression large model training device provided in an embodiment of the present application;
[0049] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0050] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0051] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0052] Regression: In the field of machine learning and deep learning, tasks can be divided into regression tasks and classification tasks. Classification tasks use models to classify samples of a limited number of types, such as using image recognition models to distinguish between "cats" and "dogs". Regression tasks use models to fit an infinite number of features, such as house price prediction (1 million, 1.02 million, 1.08 million).
[0053] GPT: A generative pre-trained large language model that can automatically generate the following content based on the previous context.
[0054] Softmax: The Softmax function is a commonly used activation function, which is mainly used in multi-classification problems to convert a real number vector into a probability distribution.
[0055] Cross-entropy loss: The cross-entropy loss function is often used to measure the difference between two probability distributions, especially in classification problems, to measure the difference between the probability distribution predicted by the model and the probability distribution of the true label. In multi-classification problems, the softmax function is often used together with the cross-entropy loss function. After the model output is converted to a probability distribution by the softmax function, the cross-entropy loss function is used to calculate the loss, thereby optimizing the model parameters.
[0056] Tokenizer: It is an important tool in natural language processing (NLP). Its main function is to convert text data into a numerical form that the model can process. This process usually includes the following steps: word segmentation, vocabulary building, encoding into integer token id, etc.
[0057] Token: A discrete integer value after quantization encoding by Tokenizer, such as 1, 2, 3..., which constitutes the vocabulary of the large model. The number is limited and is used for softmax+cross entropy to train classification tasks.
[0058] The fundamental principle of existing large models is to use neural networks to learn the relationship between tokens using the training method of classification tasks (for example: softmax + cross entropy). This method has the following disadvantages:
[0059] Disadvantage 1: A pre-quantization model is required to convert the original data into tokens (positive integer type).
[0060] For example, text uses Tokenizer; speech uses HuBERT to extract the acoustic sequence of speech and then encodes and aligns it into tokens, or uses VQ-GAN and other methods to quantize audio. This pre-quantization model needs to be trained separately, and its performance will affect the effect of the subsequent large model. For large model technology in the image field, it is also necessary to use technologies similar to CLIP Tokenizer to convert images into tokens.
[0061] Disadvantage 2: Consumes a lot of GPU memory.
[0062] For example, the dimension of token embedding in GPT3 is 12288, which is much higher than the dimension of speech short-time Fourier feature: 256-512, semantic sentenceBert feature dimension: 512, and image clip feature dimension: 768. Using the regression training method, the vector features of the current batch of training data can be loaded to the GPU in real time on demand, while the classification training method must load the entire token embedding list to the GPU at one time, which will take up a lot of GPU space.
[0063] Disadvantage 3: Using tokens will increase the illusion of a large model.
[0064] Tokens are obtained by extracting data features and quantizing and compressing them. The algorithm of quantizing and compressing them into finite integers will inevitably have losses, which will increase the illusion of a large model.
[0065] Disadvantage 4: The application scenarios are limited.
[0066] For example, in the field of voiceprint recognition, it is impossible to convert voice into token because different people have different voices, and tokens must be a finite set, which cannot meet the requirements of different tokens for different people.
[0067] Based on the above problems, an embodiment of the present application provides a regression large model training method.
[0068] Please see Figure 1 , Figure 1 A flow chart of a regression-type large model training method provided in an embodiment of the present application. The regression-type large model training method includes:
[0069] 100. Obtain input data and convert the input data into an input feature vector.
[0070] Specifically, the input data is obtained, and the input feature vector is extracted from the input data through the feature extraction algorithm; wherein, the input data is directly converted into the input feature vector, the token embedding layer and feature layer of the large model are discarded, the tokenizer quantization process is removed, the token embedding is removed, the performance loss caused by quantization is avoided, the hallucination problem of the model can be reduced, and a wider range of application scenarios is ensured.
[0071] Exemplarily, input text data is obtained, and feature vectors in the input text data are extracted through a text vector extraction model; thus, the feature vectors of the text are extracted, and model training can be implemented in a text scenario.
[0072] Exemplarily, input image data is obtained, and feature vectors in the input image data are extracted through an image recognition model; thus, the feature vectors of the image are extracted, and model training can be implemented in the image scene.
[0073] Exemplarily, input speech data is obtained, and feature vectors in the input speech data are extracted through a speech feature extraction algorithm; thus, the feature vectors of the speech are extracted, and model training can be implemented in a speech scenario.
[0074] It is understandable that the input data may also be of other types, and feature vectors of the data may be extracted according to a feature extraction algorithm for model training.
[0075] 200. Input the input feature vector into the large model for prediction to obtain a predicted feature vector;
[0076] Specifically, the input feature vector is input into the large model, and the feature vector sequence is calculated through the feature function:
[0077] V=f(X,ψ)=v 1 ,v 2 ,…v t-1 ;
[0078] Where ψ is the parameter of the feature function f; v is the feature vector of a unit (such as a frame); X is the input feature, X = x 1 ,x 2 ,x 3 …x t-1 .
[0079] Compute the predicted feature vector using the generative large model:
[0080]
[0081] Among them, g is the generative large model, θ is the parameter of the generative large model, is the predicted feature vector.
[0082] Thus, the feature vector of t is predicted by the sequence of feature vectors from 1 to t-1, so that the regression loss calculation is calculated by the predicted feature vector and the actual feature vector of t.
[0083] For example, by inputting feature x 1 ,x 2 ,x 3 …x t-1 , calculate the feature vector sequence v 1 ,v 2 ,…v t-1 , to predict x t and perform regression tasks with the actual feature vector.
[0084] Understandably, large models usually refer to deep learning models with a large number of parameters and complex structures, such as Transformer, BERT, GPT series, etc. These models are optimized by training on large-scale data sets.
[0085] 300. Calculate the regression loss based on the predicted feature vector and the actual feature vector, and train the model based on the calculation results.
[0086] Specifically, calculate the current actual eigenvector:
[0087]
[0088] in, is the current input feature, y is the actual eigenvector at time t.
[0089] Combine the predicted feature vector with the actual feature vector to calculate the mean square loss MSE function:
[0090]
[0091] Among them, MSE Loss is a mean square loss function in the regression task loss function; N is the dimension of the actual feature vector.
[0092] It is understandable that during the training process, the predicted feature vector needs to be compared with the actual feature vector and the loss between them needs to be calculated. This calculation process is usually performed in batches, that is, the loss of a batch of samples is calculated each time, and the parameters of the model are updated based on this loss.
[0093] Therefore, the predicted feature vector is combined with the actual feature vector to calculate the regression loss, and the model training process is guided by the regression loss calculation result to help the model continuously optimize its prediction ability. The idea of regression tasks is used to train large models, and the token quantization process is removed, which can avoid the loss caused by token quantization and reduce the illusion of large models.
[0094] As described above, the embodiment of the present application obtains input data and converts the input data into an input feature vector; inputs the input feature vector into the large model for prediction to obtain a predicted feature vector; calculates the regression loss based on the predicted feature vector and the actual feature vector, and trains the model based on the calculation results; uses the idea of regression tasks to train the large model, removes the token quantization process, avoids the loss caused by token quantization, and thus reduces the illusion of the large model, saves GPU memory, and has a wide range of application scenarios.
[0095] Therefore, the embodiment of the present application simplifies the algorithm flow. Through the regression task, the large model can directly learn the relationship between features, rather than the relationship between tokens. That is, tokenizer is no longer needed for quantification. For example: speech uses the features of short-time Fourier transform, text uses the features extracted by sentence Bert, and images use the features extracted by clip.
[0096] The embodiment of the present application can save GPU memory. The dimension of token embedding in GPT3 is 12288, which is much higher than the short-time Fourier feature dimension of speech: 256-512, the semantic sentence Bert feature dimension: 512, and the image clip feature dimension: 768. The embodiment of the present application uses the feature dimension to input a large model, which is less than one-tenth of the GPT3 token dimension.
[0097] The embodiment of the present application removes the tokenizer quantization process, avoids the performance loss caused by quantization, and can reduce the hallucination problem of the model.
[0098] The embodiment of the present application removes the tokenizer quantization process, which can help the embodiment of the present application be applied in scenarios where it is difficult to train a tokenizer model, such as voiceprint-related scenarios.
[0099] The above steps are not to be performed in a strict order as described in the numbers, but should be understood as an overall solution.
[0100] In the second aspect, based on the above embodiments, Figure 2 A schematic diagram of the structure of a large regression model training device provided in an embodiment of the present application. Figure 2The regression large model training device provided in this embodiment specifically includes: a feature acquisition module 201, a feature prediction module 202 and a regression loss module 203.
[0101] Among them, the feature acquisition module 201 is used to obtain input data and convert the input data into an input feature vector; the feature prediction module 202 is used to input the input feature vector into the large model for prediction to obtain a predicted feature vector; the regression loss module 203 is used to combine the predicted feature vector with the actual feature vector to calculate the regression loss, and train the model according to the calculation results.
[0102] As described above, the embodiment of the present application obtains input data and converts the input data into an input feature vector; inputs the input feature vector into the large model for prediction to obtain a predicted feature vector; calculates the regression loss based on the predicted feature vector and the actual feature vector, and trains the model based on the calculation results; uses the idea of regression tasks to train the large model, removes the token quantization process, avoids the loss caused by token quantization, and thus reduces the illusion of the large model, saves GPU memory, and has a wide range of application scenarios.
[0103] The regression large model training device provided in the embodiment of the present application can be used to execute the regression large model training method provided in the above embodiment, and has corresponding functions and beneficial effects.
[0104] In a third aspect, an embodiment of the present application further provides an electronic device that can integrate the regression large model training device provided in an embodiment of the present application. Figure 3 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 3 The electronic device includes: an input device 33, an output device 34, a memory 32 and one or more processors 31; the memory 32 is used to store one or more programs; when the one or more programs are executed by the one or more processors 31, the one or more processors 31 implement the regression large model training method provided in the above embodiment. The input device 33, the output device 34, the memory 32 and the processor 31 can be connected by a bus or other means. Figure 3 The example of connecting through bus is taken in the following.
[0105] The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 32, that is, implements the above-mentioned regression large model training method.
[0106] The electronic device provided above can be used to execute the regression large model training method provided in the above embodiment, and has corresponding functions and beneficial effects.
[0107] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the regression large model training method as described above, and can achieve the same beneficial effects.
[0108] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present application is not limited to the regression large model training method described above, and the computer executable instructions can also execute related operations in the regression large model training method provided in any embodiment of the present application.
[0109] In a fifth aspect, the embodiments of the present application also provide a computer program product, and the methods described in the various embodiments of the present application can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the processes or functions described in the various embodiments of the present application are executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer grid, a grid device, a user device, a core network device, an OAM (Open Application Model) or other programmable device.
[0110] The computer program or instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.
[0111] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0112] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0113] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for enabling an electronic device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0114] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0116] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
Claims
1. A regression large model training method, characterized in that: include: Get input data and convert it into input feature vector; Input the input feature vector into the large model for prediction to obtain the predicted feature vector; The predicted feature vector is combined with the actual feature vector to calculate the regression loss, and the model is trained based on the calculation results.
2. The large regression model training method according to claim 1, characterized in that: The step of inputting the input feature vector into the large model for prediction to obtain the predicted feature vector includes: Input the input feature vector into the large model and calculate the feature vector sequence through the feature function: V=f(X,ψ)=v1,v2,…v t-1 ; Among them, ψ is the parameter of the feature function f; X is the input feature, X = x1, x2, x3…x t-1 ; v is a unit eigenvector; Compute the predicted feature vector using the generative large model: Among them, g is the generative large model, θ is the parameter of the generative large model, is the predicted feature vector.
3. The large regression model training method according to claim 2, characterized in that: The step of combining the predicted feature vector with the actual feature vector to calculate the regression loss includes: Compute the current actual eigenvector: in, is the current input feature, y is the actual eigenvector at time t; Combine the predicted feature vector with the actual feature vector to calculate the mean square loss function: Where N is the dimension of the actual feature vector.
4. The large regression model training method according to claim 1, characterized in that: The step of obtaining input data and converting the input data into an input feature vector includes: Input data is obtained, and an input feature vector is extracted from the input data through a feature extraction algorithm.
5. The large regression model training method according to claim 4, characterized in that: The step of obtaining input data and extracting an input feature vector from the input data by using a feature extraction algorithm includes: Input text data is obtained, and feature vectors are extracted from the input text data through a text vector extraction model.
6. The large regression model training method according to claim 4, characterized in that: The step of obtaining input data and extracting an input feature vector from the input data by using a feature extraction algorithm includes: The input image data is obtained, and the feature vector in the input image data is extracted through the image recognition model.
7. The large regression model training method according to claim 4, characterized in that: The step of obtaining input data and extracting an input feature vector from the input data by using a feature extraction algorithm includes: The input speech data is obtained, and a feature vector in the input speech data is extracted by a speech feature extraction algorithm.
8. A large regression model training device, characterized in that: include: A feature acquisition module, used to acquire input data and convert the input data into an input feature vector; A feature prediction module is used to input the input feature vector into the large model for prediction to obtain a predicted feature vector; The regression loss module is used to calculate the regression loss by combining the predicted feature vector with the actual feature vector, and train the model based on the calculation results.
9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the processor is connected to the memory via the bus, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, they are used to implement the regression large model training method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a server, implements the large regression model training method as described in any one of claims 1-7.