Large language model training method, natural language processing method and device
By calculating the cosine and sine function values of large language model parameters, the gradient descent process is optimized, and the problem of low training efficiency of large language model in the existing technology is solved, and a more efficient training process is achieved.
Patent Information
- Application Number
- CN202411853944.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-16
- Publication Date
- 2025-04-25
AI Technical Summary
In the prior art, the efficiency is low when using gradient descent method to train large language models.
By calculating the cosine function values and sine function values of the first and second parameters of the parameters of the large language model, combining these values to optimize the gradient descent process and adjust the parameter values to improve training efficiency.
Effectively amplify the step size, improve the training efficiency of large models, and reduce the overhead of computing resources.
Smart Images
Figure CN119884736A_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of artificial intelligence technology, and in particular to a large language model training method, a natural language processing method and a device. Background Art
[0002] A large language model (LLM) is a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text.
[0003] Gradient descent is an iterative method that can be used to solve least squares problems. Gradient descent can gradually approach the minimum value by iteratively adjusting parameters, and can be used to minimize functions, such as minimizing the loss function in model training. When solving the minimum value of the loss function, the gradient descent method can be used to iterate step by step to obtain the minimized loss function and the corresponding model parameters.
[0004] The existing technology is inefficient when using gradient descent to train large language models. Summary of the invention
[0005] To overcome the problems existing in the related art, this specification provides a large language model training method, a natural language processing method, an apparatus, an electronic device and a medium.
[0006] According to a first aspect of an embodiment of the present specification, a training method for a large language model is provided, the method comprising: performing natural language processing on text data using a large language model to obtain a processing result, the parameters of the large language model comprising a first parameter and a second parameter, and the text data comprising at least one of text, an image, a video, and an audio; calculating a loss function value according to the processing result; calculating a first gradient of the first parameter and a second gradient of the second parameter according to the loss function value; calculating a cosine function value and a sine function value according to the first parameter and the second parameter; adjusting the value of the first parameter according to the cosine function value and the first gradient, and adjusting the value of the second parameter according to the sine function value and the second gradient; returning to the operation of performing natural language processing on the text data using the large language model to obtain a processing result until a training end condition is met.
[0007] According to a second aspect of an embodiment of this specification, a natural language processing method is provided, the method comprising: obtaining a text to be processed; performing natural language processing on the text to be processed using a large language model to obtain a processing result, wherein the large language model is trained according to the large language model training method of the first aspect or any corresponding implementation method thereof.
[0008] According to a third aspect of an embodiment of this specification, a training device for a large language model is provided, comprising: a first processing module, used to perform natural language processing on text data using a large language model to obtain a processing result, the parameters of the large language model include a first parameter and a second parameter, and the text data includes at least one of text, image, video, and audio; a loss calculation module, used to calculate a loss function value according to the processing result; a gradient calculation module, used to calculate a first gradient of the first parameter and a second gradient of the second parameter according to the loss function value; a trigonometric function calculation module, used to calculate a cosine function value and a sine function value according to the first parameter and the second parameter; an adjustment module, used to adjust the value of the first parameter according to the cosine function value and the first gradient, and adjust the value of the second parameter according to the sine function value and the second gradient; and returning to the operation of performing natural language processing on the text data using the large language model to obtain a processing result until a training end condition is met.
[0009] According to a fourth aspect of the embodiments of this specification, a natural language processing device is provided, the device comprising: a text acquisition module, used to acquire a text to be processed; a second processing module, used to perform natural language processing on the text to be processed using a large language model to obtain a processing result, wherein the large language model is trained according to the large language model training method according to the above-mentioned first aspect or any corresponding implementation method thereof.
[0010] According to a fifth aspect of the embodiments of this specification, an electronic device is provided, including:
[0011] processor;
[0012] a memory for storing processor-executable instructions;
[0013] The processor is configured to execute the method of the first aspect, the second aspect or any corresponding implementation method thereof.
[0014] According to the sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the method of the above-mentioned first aspect, second aspect or any corresponding implementation method.
[0015] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:
[0016] In the embodiments of the present specification, the cosine function value and the sine function value are calculated according to the first parameter and the second parameter, and the gradient descent process is optimized by combining the cosine function value and the sine function value, which can effectively amplify the step size and improve the training efficiency of large models.
[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.
[0019] Figure 1 It is a schematic diagram of a system architecture shown in this specification according to an exemplary embodiment.
[0020] Figure 2 It is a flowchart of a large language model training method shown in this specification according to an exemplary embodiment.
[0021] Figure 3 It is a schematic diagram of the relationship between step length and parameters according to an exemplary embodiment of this specification.
[0022] Figure 4 It is a hardware structure diagram of the computer device where the training device of the large language model in the embodiment of this specification is located.
[0023] Figure 5 It is a block diagram of a large language model training device shown in this specification according to an exemplary embodiment.
[0024] Figure 6 It is a block diagram of a natural language processing device shown in this specification according to an exemplary embodiment. DETAILED DESCRIPTION
[0025] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.
[0026] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0027] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0028] Next, the embodiments of this specification are described in detail.
[0029] The following combination Figure 1 The system architecture of the training method and device that can apply the large language model in the embodiments of this specification is described. It should be noted that Figure 1 What is shown is merely an example of a system architecture to which the embodiments of this specification can be applied, to help those skilled in the art understand the technical content of this specification, but it does not mean that the embodiments of this specification cannot be used in other devices, systems, environments or scenarios.
[0030] Figure 1 It is a schematic diagram of a system architecture shown in this specification according to an exemplary embodiment.
[0031] like Figure 1 As shown, the system architecture may include, for example, a terminal device, a network and a server. The network is used to provide a medium for a communication link between the terminal device and the server. The network may include various connection types, such as wired and / or wireless communication links, etc.
[0032] Users can use terminal devices to interact with servers through the network to receive or send messages, etc. Various communication client applications can be installed on the terminal devices, such as artificial intelligence applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software.
[0033] The terminal device may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.
[0034] The server may be a server that provides various services, such as a backend server that provides artificial intelligence services for the content browsed by the user using the terminal device. The backend server may use a large language model to analyze and process the received user request and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device.
[0035] The following is a detailed description of the training method of the large language model provided in the embodiments of this specification. Figure 2 As shown, Figure 2 This is a flowchart of a large language model training method shown in this specification according to an exemplary embodiment. The method can be applied to a server, for example. The large language model training method provided in the embodiment of this specification may include the following steps.
[0036] In step 210, natural language processing is performed on the text data using a large language model to obtain a processing result, and the parameters of the large language model include a first parameter and a second parameter.
[0037] According to an embodiment of the present specification, a large language model can process natural language tasks. The input of the large language model can include text data.
[0038] Optionally, the large language model may also be a multimodal large language model, based on which text data and other types of data may be simultaneously used as inputs of the large language model. Other types of data may include, for example, image, video, audio and other data.
[0039] In step 220, the loss function value is calculated according to the processing result.
[0040] According to an embodiment of the present specification, the loss function may include, for example, mean square error, mean absolute error, cross entropy loss, logarithmic loss, hinge loss, etc.
[0041] In step 230, it is determined whether the training end condition is met. If the training end condition is not met, then step 240 is continued. If the training end condition is met, then the training ends.
[0042] According to an embodiment of the present specification, the training end condition may include, for example, the loss function value converges or the number of iterations reaches an iteration number threshold. The iteration number threshold may be set according to actual needs.
[0043] In step 240, a first gradient of the first parameter and a second gradient of the second parameter are calculated according to the loss function value.
[0044] According to an embodiment of the present specification, for example, a partial derivative of the loss function with respect to a first parameter may be calculated to obtain a first gradient, and a partial derivative of the loss function with respect to a second parameter may be calculated to obtain a second gradient.
[0045] In step 250, the cosine function value and the sine function value are calculated according to the first parameter and the second parameter.
[0046] According to the embodiments of the present specification, for example, the sum of the square of the first parameter and the bisection of the second parameter can be calculated to obtain a first intermediate value; the square root operation of the first value can be calculated to obtain a second intermediate value; then, the ratio of the first parameter to the second intermediate value can be calculated to obtain a cosine function value and a pre-function value, and the ratio of the second parameter to the second intermediate value can be calculated to obtain a sine function value.
[0047] In step 260 , the value of the first parameter is adjusted according to the cosine function value and the first gradient, and the value of the second parameter is adjusted according to the sine function value and the second gradient. Then, the process returns to step 210 .
[0048] According to an embodiment of the present specification, the value of the first parameter may be adjusted by, for example, calculating the product of the cosine function value, the first gradient and the preset step length to obtain a first adjustment amount, and then subtracting the first adjustment amount from the value of the first parameter.
[0049] According to an embodiment of the present specification, the value of the second parameter may be adjusted by, for example, calculating the product of the sine function value, the second gradient and the preset step length to obtain the second adjustment amount, and then subtracting the second adjustment amount from the value of the second parameter.
[0050] According to the embodiments of the present specification, the cosine function value and the sine function value are calculated according to the first parameter and the second parameter, and the cosine function value and the sine function value are combined to optimize the gradient descent process, which can effectively amplify the step size and improve the training efficiency of large models.
[0051] The large language model training method according to the embodiments of the present specification can be applied to scenarios such as pre-training and incremental pre-training of large models.
[0052] According to the embodiments of the present specification, a trained large language model can be used to process natural language tasks, such as text classification, question and answer, dialogue, etc. For example, a text to be processed can be obtained. The trained large language model is used to perform natural language processing on the text to be processed to obtain a processing result. Optionally, the large language model can also be a multimodal large language model, based on which multimodal data containing both text data and other types of data can be input into the large language model so that the large language model analyzes and processes the multimodal data to obtain a processing result.
[0053] According to the embodiments of this specification, it can be assumed that η is the step size, and x and y are parameters of the large model. (x, y) can be regarded as a vector, x can be used as the x-axis of the xy coordinate system, y can be used as the y-axis of the xy coordinate system, and η has an angle with the x-axis and y-axis of the xy coordinate system, so Δx and Δy can be calculated according to trigonometric functions. Figure 3 is a schematic diagram of the relationship between step length and parameters according to an exemplary embodiment of this specification, such as Figure 3As shown, in order to conform to the traditional vector representation habits, the unit vector l=η, Δx=l*cosα, Δy=l*sinα can be set.
[0054] Based on this, the calculation method of the trigonometric function cosα (cosine function) can be:
[0055]
[0056] The calculation method of the trigonometric function sinα (sine function) can be:
[0057]
[0058] For example, if the known parameters include x and y, (x, y) can be regarded as a vector, and the partial derivative of the loss function f(x, y) with respect to x is The partial derivative of f(x,y) with respect to y is The step length is known to be η.
[0059] Based on this, cosα and sinα can be calculated.
[0060]
[0061] Then, the adjustment amount in the x-direction and the y-direction can be calculated. The adjustment amount in the x-direction can be The adjustment amount in the y direction can be
[0062] Next, the value of (x,y) can be adjusted according to the following formula:
[0063]
[0064] The above process is iterated repeatedly until the function f(x, y) converges or the number of iterations reaches a predetermined threshold.
[0065] The following describes a method for training a large language model in conjunction with a specific embodiment.
[0066] For example, in this embodiment, η=0.1. Parameters and gradients may be as shown in Table 1. (x, y) may be (3, 2). i is a positive integer and may represent the number of iterations.
[0067]
[0068] Table 1
[0069] Based on this, cosα and sinα can be calculated as:
[0070]
[0071] Then Δx and Δy can be calculated:
[0072] Δx=η*cosα=0.1*0.83=0.083
[0073] Δy=η*sinα=0.1*0.56=0.056
[0074] Next, it can be determined that the adjustment amount in the x direction is 6*0.083=0.498, and the adjustment amount in the y direction is 4*0.055=0.22.
[0075] Therefore, the adjusted x i+1 =3-0.495=2.505, adjusted y i+1 =2-0.22=1.78.
[0076] In the related art, when training a model, the step length η is fixed and must be set to a very small number. However, the training method of a large language model using the embodiment of this specification can effectively enlarge the step length. In the training of a large language model, when calculating parameters such as neural network weights, a very large amount of calculation is involved. Therefore, according to the training method of a large language model according to the embodiment of this specification, by enlarging the step length, the overhead of computing resources can be reduced.
[0077] According to the embodiments of this specification, for example, the following program code may be used to adjust the parameters of the large model:
[0078]
[0079]
[0080]
[0081] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of a training device for a large language model and a terminal to which it is applied.
[0082] The embodiments of the training device for the large language model in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, such as Figure 4 As shown, it is a hardware structure diagram of the computer device where the training device of the large language model in the embodiment of this specification is located, except Figure 4In addition to the processor 410, memory 430, network interface 420, and non-volatile memory 440 shown, the server or electronic device where the device 431 is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.
[0083] like Figure 5 As shown, Figure 5 is a block diagram of a large language model training device according to an exemplary embodiment of the present specification, the device comprising:
[0084] A first processing module 510 is used to perform natural language processing on the text data using a large language model to obtain a processing result, wherein the parameters of the large language model include a first parameter and a second parameter, and the text data includes at least one of text, image, video, and audio;
[0085] A loss calculation module 520, used to calculate a loss function value according to the processing result;
[0086] A gradient calculation module 530, used to calculate a first gradient of a first parameter and a second gradient of a second parameter according to a loss function value;
[0087] A trigonometric function calculation module 540, configured to calculate a cosine function value and a sine function value according to a first parameter and a second parameter;
[0088] The adjustment module 550 is used to adjust the value of the first parameter according to the cosine function value and the first gradient, and to adjust the value of the second parameter according to the sine function value and the second gradient; and return to the operation of performing natural language processing on the text data using the large language model to obtain the processing result until the training end condition is met.
[0089] Optionally, the gradient calculation module may include:
[0090] A first partial derivative calculation submodule, used to calculate the partial derivative of the loss function with respect to the first parameter to obtain a first gradient;
[0091] The second partial derivative calculation submodule is used to calculate the partial derivative of the loss function with respect to the second parameter to obtain a second gradient.
[0092] Optionally, the trigonometric function calculation module may include:
[0093] A summation submodule, used for calculating the sum of the square of the first parameter and the equal division of the second parameter to obtain a first intermediate value;
[0094] A square root submodule, used for performing a square root operation on the first value to obtain a second intermediate value;
[0095] A cosine calculation submodule, used to calculate the ratio of the first parameter to the second intermediate value to obtain a cosine function value;
[0096] The sine calculation submodule is used to calculate the ratio of the second parameter to the second intermediate value to obtain a sine function value.
[0097] Optionally, the adjustment module may include:
[0098] A first product submodule, used for calculating the product of the cosine function value, the first gradient and the preset step length to obtain a first adjustment amount;
[0099] The first subtraction submodule is used to subtract the first adjustment amount from the value of the first parameter.
[0100] Optionally, the adjustment module may further include:
[0101] A second product submodule, used for calculating the product of the sine function value, the second gradient and the preset step length to obtain a second adjustment amount;
[0102] The second subtraction submodule is used to subtract the second adjustment amount from the value of the second parameter.
[0103] According to the embodiments of the present specification, the cosine function value and the sine function value are calculated according to the first parameter and the second parameter, and the cosine function value and the sine function value are combined to optimize the gradient descent process, which can effectively amplify the step size and improve the training efficiency of large models.
[0104] Accordingly, the present specification also provides an electronic device, which includes a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: perform natural language processing on text data using a large language model to obtain a processing result, the parameters of the large language model include a first parameter and a second parameter, and the text data includes at least one of text, image, video, and audio; calculate a loss function value based on the processing result; calculate a first gradient of the first parameter and a second gradient of the second parameter based on the loss function value; calculate a cosine function value and a sine function value based on the first parameter and the second parameter; adjust the value of the first parameter based on the cosine function value and the first gradient, and adjust the value of the second parameter based on the sine function value and the second gradient; return to the operation of performing natural language processing on the text data using the large language model to obtain a processing result until the training end condition is met.
[0105] like Figure 6 As shown, Figure 6 is a block diagram of a natural language processing device according to an exemplary embodiment of the present specification, the device comprising:
[0106] A text acquisition module 610 is used to acquire the text to be processed;
[0107] The second processing module 620 is used to perform natural language processing on the text to be processed by using a large language model to obtain a processing result, wherein the large language model is trained according to the large language model training method of any embodiment of this specification.
[0108] Correspondingly, the present specification also provides an electronic device, which includes a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: obtain a text to be processed; use a large language model to perform natural language processing on the text to be processed to obtain a processing result, wherein the large language model is trained according to the large language model training method of any embodiment of the present specification.
[0109] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.
[0110] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.
[0111] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0112] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.
[0113] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.
[0114] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.
Claims
1. A method for training a large language model, characterized in that: The method comprises: Performing natural language processing on the text data using a large language model to obtain a processing result, wherein the parameters of the large language model include a first parameter and a second parameter; According to the processing results, calculate the loss function value; Calculating a first gradient of the first parameter and a second gradient of the second parameter according to the loss function value; Calculate a cosine function value and a sine function value according to the first parameter and the second parameter; adjusting the value of the first parameter according to the cosine function value and the first gradient, and adjusting the value of the second parameter according to the sine function value and the second gradient; Return to the operation of performing natural language processing on text data using the large language model to obtain processing results until the training end condition is met.
2. The method according to claim 1, characterized in that The calculating, according to the loss function value, a first gradient of the first parameter and a second gradient of the second parameter comprises: Take the partial derivative of the loss function with respect to the first parameter to obtain the first gradient; The partial derivative of the loss function with respect to the second parameter is calculated to obtain the second gradient.
3. The method according to claim 1, characterized in that The step of calculating the cosine function value and the sine function value according to the first parameter and the second parameter comprises: Calculate the sum of the square of the first parameter and the equal division of the second parameter to obtain a first intermediate value; Performing a square root operation on the first value to obtain a second intermediate value; Calculating a ratio of the first parameter to the second intermediate value to obtain the cosine function value; The ratio of the second parameter to the second intermediate value is calculated to obtain the sine function value.
4. The method according to claim 1, characterized in that: The adjusting the value of the first parameter according to the cosine function value and the first gradient includes: Calculating the product of the cosine function value, the first gradient and a preset step length to obtain a first adjustment amount; The first adjustment amount is subtracted from the value of the first parameter.
5. The method according to claim 1, characterized in that The step of adjusting the value of the second parameter according to the sine function value and the second gradient includes: Calculating the product of the sine function value, the second gradient and a preset step length to obtain a second adjustment amount; The second adjustment amount is subtracted from the value of the second parameter.
6. The method according to claim 1, characterized in that The training end condition includes: the loss function value converges or the number of iterations reaches an iteration number threshold.
7. A natural language processing method, characterized in that: The method comprises: Get the text to be processed; The large language model is used to perform natural language processing on the text to be processed to obtain a processing result, wherein the large language model is trained according to the large language model training method according to any one of claims 1 to 6.
8. A large language model training device, characterized in that: The device comprises: A first processing module, configured to perform natural language processing on text data using a large language model to obtain a processing result, wherein the parameters of the large language model include a first parameter and a second parameter, and the text data includes at least one of text, image, video, and audio; The loss calculation module is used to calculate the loss function value according to the processing results; A gradient calculation module, used for calculating a first gradient of the first parameter and a second gradient of the second parameter according to the loss function value; A trigonometric function calculation module, used for calculating a cosine function value and a sine function value according to the first parameter and the second parameter; An adjustment module is used to adjust the value of the first parameter according to the cosine function value and the first gradient, and to adjust the value of the second parameter according to the sine function value and the second gradient; and return to the operation of performing natural language processing on text data using a large language model to obtain a processing result until a training end condition is met.
9. A natural language processing device, characterized in that: The device comprises: A text acquisition module is used to acquire the text to be processed; The second processing module is used to perform natural language processing on the text to be processed by using a large language model to obtain a processing result, wherein the large language model is trained according to the large language model training method according to any one of claims 1 to 6.
10. An electronic device comprising: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 7.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.