Model optimization method, business processing method, server and terminal equipment

Abstract

The embodiments of the present application provide a model optimization method, a business processing method, a server and a terminal device. In the above-mentioned model optimization method, after the server obtains the large language model to be adjusted, it traverses the operators in the large language model to obtain the operators to be quantized, and then performs simulated quantization according to the set hardware-friendly quantization configuration to obtain a simulated quantization model. Afterwards, based on the input vertical domain business data and the labels of the vertical domain business data, the weights of the above-mentioned simulated quantization model are optimized for the purpose of converging the cross entropy loss function between the output of the simulated quantization model and the labels of the vertical domain business data, so that simulated quantization can be performed on the deployed hardware during the fine-tuning of the large language model, combining quantization with model fine-tuning, and shortening the quantitative analysis and deployment cycle of the large language model. And the large language model can be adaptively adjusted to the quantization loss in a data-driven manner, so that the accuracy of the quantized model can be restored.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent terminal technology, and in particular to a model optimization method, a business processing method, a server and a terminal device. Background Art

[0002] Figure 1 This is a schematic diagram of deploying a large language model to the client side. Figure 1 As shown, after the large language model is trained, it must first be converted into the original model format and then converted into the quantized model format before it can be deployed to the end side and used for reasoning on the end side. In the existing related technology, when quantizing a large language model, it is necessary to use a corresponding quantization algorithm for specific deployment hardware (for example, graphics processing unit (GPU) / neural-network processing unit (NPU)) to reduce the quantization error of the large language model and restore the accuracy of the quantized model to the maximum extent possible before quantization when it is deployed. However, this process has problems such as difficult analysis and / or long deployment cycle. Summary of the invention

[0003] The embodiments of the present application provide a model optimization method, a business processing method, a server and a terminal device. The embodiments of the present application also provide a computer-readable storage medium to enable a large language model to fit training data under the condition of perceiving a quantization error of a hardware platform, thereby ensuring the accuracy of the model obtained by quantization, and the trained model can be directly deployed to the hardware platform, shortening the deployment cycle.

[0004] In the first aspect, an embodiment of the present application provides a model optimization method, including: obtaining a large language model to be adjusted; traversing the operators in the large language model to obtain the operators to be quantized in the large language model; simulating quantization of the operators to be quantized according to a set hardware-friendly quantization configuration to obtain a simulated quantization model; obtaining vertical business data and labels of the vertical business data; processing the vertical business data through the simulated quantization model to obtain the output of the simulated quantization model, optimizing the weights of the simulated quantization model with the purpose of converging the cross entropy loss function between the output of the simulated quantization model and the label of the vertical business data; saving the optimized weights of the simulated quantization model, and saving the quantization parameters corresponding to the optimized weights to obtain an optimized simulated quantization model.

[0005] In one possible implementation, the quantization parameters corresponding to the optimized weights include: a quantization scaling factor and a quantization offset.

[0006] In one possible implementation, the set hardware-friendly quantization configuration includes a set hardware-friendly quantization grouping method and a set number of quantization bits; thus, the simulated quantization of the operator to be quantized according to the set hardware-friendly quantization configuration to obtain a simulated quantization model may be: obtaining the weight of the operator to be quantized; grouping the weights of the operator to be quantized according to the quantization grouping method; for each group of weights obtained by the grouping, counting the maximum and minimum values ​​in each group of weights; calculating the quantization scaling factor according to the maximum and minimum values ​​in each group of weights and the number of quantization bits; calculating the quantization offset according to the minimum value in each group of weights and the quantization scaling factor; quantizing each group of weights according to the quantization scaling factor and the quantization offset to obtain simulated quantization weights.

[0007] In one possible implementation, the weight of the operator to be quantized includes the weight matrix of the operator to be quantized; thus, according to the quantization grouping method, the weights of the operator to be quantized can be grouped as follows: when the hardware to be deployed is an end-side graphics processor, the weight matrix of the operator to be quantized is grouped according to a predetermined block size; when the hardware to be deployed is an end-side neural network processor, each row of the weight matrix of the operator to be quantized is grouped.

[0008] In one possible implementation manner, optimizing the weight of the analog quantization model may be: optimizing the weight of the analog quantization model by using a gradient descent method.

[0009] In the above model optimization method, after the server obtains the large language model to be adjusted, it traverses the operators in the large language model to obtain the operators to be quantized in the large language model, and then simulates and quantizes the operators to be quantized according to the set hardware-friendly quantization configuration to obtain a simulated quantization model. After that, the vertical domain business data and the labels of the vertical domain business data are obtained, and the vertical domain business data is processed through the simulated quantization model to obtain the output of the simulated quantization model, and then the weights of the simulated quantization model are optimized for the purpose of convergence of the cross entropy loss function between the output of the simulated quantization model and the labels of the vertical domain business data, and the optimized weights of the simulated quantization model are saved, and the quantization parameters corresponding to the optimized weights are saved to obtain the optimized simulated quantization model, so that the weights of the large language model can be simulated and quantized for the deployed hardware during the fine-tuning of the large language model, and quantization is combined with model fine-tuning, thereby shortening the quantitative analysis and deployment cycle of the large language model. And the large language model can be adaptively adjusted to the quantization loss in the process of fitting the vertical domain data in a data-driven manner, so that the accuracy of the quantized model can be restored.

[0010] In a second aspect, an embodiment of the present application provides a business processing method, including: obtaining business data; calling a vertical domain large language model to process the business data; wherein the vertical domain large language model includes a simulation quantization model optimized by the method provided in the first aspect; and displaying the processing results corresponding to the above business data.

[0011] In one possible implementation, the business data includes: a request for generating a video input by a user, wherein the request for generating a video includes information of images and / or videos required to generate the video; in this way, calling the vertical domain large language model to process the business data may be: calling the vertical domain large language model to obtain the information of images and / or videos required to generate the video included in the request.

[0012] In one possible implementation, the business data includes: the user's schedule information; thus, calling the vertical domain large language model to process the business data may be: calling the vertical domain large language model to generate corresponding schedule data according to the user's schedule information.

[0013] In one possible implementation, obtaining business data may be: obtaining the user's ticket booking information; thus, calling the vertical domain big language model to process the business data may be: calling the vertical domain big language model to generate corresponding notification data according to the user's ticket booking information.

[0014] In the above-mentioned business processing method, after the terminal device obtains the business data, it can call the vertical domain large language model to process the above-mentioned business data, and then display the processing results corresponding to the above-mentioned business data. Since in the process of fine-tuning the large language model to obtain the above-mentioned vertical domain large language model, the large language model adaptively adjusts the quantization loss in the process of fitting the vertical domain data in a data-driven manner, so that the accuracy of the quantized model can be restored, calling the vertical domain large language model to process the business data can obtain a higher quantization accuracy and improve the user experience.

[0015] In a third aspect, an embodiment of the present application provides a server, comprising: one or more processors; a memory; multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the server, enable the server to execute the method provided in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a terminal device, comprising: one or more processors; a memory; multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the terminal device, enable the terminal device to execute the method provided in the second aspect.

[0017] It should be understood that the third aspect of the embodiment of the present application is consistent with the technical solution of the first aspect of the embodiment of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here.

[0018] It should be understood that the fourth aspect of the embodiment of the present application is consistent with the technical solution of the second aspect of the embodiment of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here.

[0019] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method provided in the first aspect or the second aspect.

[0020] In a sixth aspect, an embodiment of the present application provides a computer program, which, when executed by a computer, is used to execute the method provided in the first aspect or the second aspect.

[0021] In one possible design, the program in the sixth aspect may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of deploying a large language model on the client side;

[0023] Figure 2 A schematic diagram of vertical domain fine-tuning of a large language model provided by existing related technologies;

[0024] Figure 3 A schematic diagram of schedule data extracted by a vertical domain large language model in the prior art;

[0025] Figure 4 A schematic diagram of the structure of a server provided for one embodiment of the present application;

[0026] Figure 5 A flowchart of a model optimization method provided for one embodiment of the present application;

[0027] Figure 6 A schematic diagram of block quantization;

[0028] Figure 7 It is a schematic diagram of quantification by channel;

[0029] Figure 8 A schematic diagram of the graph structure of a model before and after simulation quantization provided by an embodiment of the present application;

[0030] Fig. 9 A schematic diagram of a system for calculating quantization scaling and quantization offset provided in accordance with an embodiment of the present application;

[0031] Fig.10 A schematic diagram of vertical domain fine-tuning of a large language model provided by one embodiment of the present application;

[0032] Fig.11 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application;

[0033] Fig.12 A flowchart of a business processing method provided by an embodiment of the present application;

[0034] Fig.13 A schematic diagram of processing business data provided by an embodiment of the present application;

[0035] Fig.14 A schematic diagram of processing business data provided by another embodiment of the present application;

[0036] Fig.15 A schematic diagram of a schedule reminder provided by an embodiment of the present application;

[0037] Fig.16 A schematic diagram of an interface for displaying notification messages provided by an embodiment of the present application;

[0038] Fig.17 A schematic diagram of the structure of a server provided in another embodiment of the present application;

[0039] Fig.18 A schematic diagram of the structure of a terminal device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0040] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0041] Figure 2 A schematic diagram of vertical domain fine-tuning of a large language model provided by existing related technologies, such as Figure 2 As shown, in the existing related technology, after obtaining a large language model, the large language model can be fine-tuned and trained using a vertical domain dataset to obtain a vertical domain large language model. Figure 2The solution shown in the figure has a low quantization accuracy of the vertical domain large language model. The vertical domain refers to a specific industry or professional field, usually with a clear business scope and a specific user group. It refers to applications, platforms or services that focus on a specific field, as opposed to a general platform that covers multiple fields.

[0042] For example, for the schedule extraction task, Figure 3 As shown in 301, when the user inputs the sentence "There is a lunch party at 12 o'clock\nNext Wednesday, I will attend the department staff meeting\nAt 3:30, I will invite the marked friends to have a meeting together" (as shown in 301), the call is made according to Figure 2 The vertical domain large language model obtained by the scheme shown in the figure processes the sentence input by the user, and the extracted schedule data can be shown in 302. It can be seen that in the existing scheme, when the vertical domain large language model performs the schedule extraction task, it simply parses the sentence input by the user and generates the schedule data, which has low quantization accuracy and poor user experience. Figure 3 A schematic diagram of schedule data extracted by a vertical large language model in the prior art.

[0043] Based on the above problems, the embodiment of the present application provides a model optimization method, which can simulate and quantize the weights of the large language model for the deployed hardware during the fine-tuning of the large language model, combine quantization with model fine-tuning, and thus shorten the quantitative analysis and deployment cycle of the large language model. And in a data-driven way, the large language model can adaptively adjust the quantization loss in the process of fitting vertical domain data, so that the accuracy of the quantized model can be restored.

[0044] The model optimization method provided in the embodiment of the present application can be applied to a server, wherein the above-mentioned server can perform simulation quantization and fine-tuning training on a large language model, and the above-mentioned server can be set in the cloud. The embodiment of the present application does not impose any restrictions on the specific type of the server.

[0045] For example, Figure 4 A schematic diagram of the structure of a server provided in one embodiment of the present application, such as Figure 4 As shown, the server 400 includes a processor 410 and a communication interface 420. Optionally, the server 400 may also include a memory 430. The processor 410, the communication interface 420 and the memory 430 may communicate with each other through an internal connection path to transmit control and / or data signals, the memory 430 is used to store a computer program, and the processor 410 is used to call and run the computer program from the memory 430.

[0046] The processor 410 and the memory 430 may be combined into a processing device, or more commonly, they are independent components, and the processor 410 is used to execute the program code stored in the memory 430. In specific implementation, the memory 430 may also be integrated into the processor 410, or may be independent of the processor 410.

[0047] In addition, in order to make the functions of the server 400 more complete, the server 400 may further include one or more of an input unit 460 and a display unit 470 .

[0048] Optionally, the server 400 may further include a power supply 450 for providing power to various devices or circuits in the server 400 .

[0049] It should be understood that Figure 4 The processor 410 in the server 400 shown may be a system on chip SOC, and the processor 410 may include a central processing unit (CPU), and may further include other types of processors, such as a GPU.

[0050] The following has Figure 4 Taking the server with the shown structure as an example, the model optimization method provided in the embodiment of the present application is specifically explained in combination with the accompanying drawings and application scenarios.

[0051] Figure 5 A flowchart of a model optimization method provided for one embodiment of the present application, which model optimization method can be executed by server 400.

[0052] like Figure 5 As shown, the above model optimization method may include:

[0053] Step 501: Obtain a large language model to be adjusted.

[0054] Step 502: traverse the operators in the large language model to obtain the operators to be quantized in the large language model.

[0055] Specifically, after obtaining the large language model to be adjusted, a depth-first search traversal can be performed on the operator nodes in the graph structure of the large language model to obtain the operator to be quantized in the large language model. , for example: fully connected layer operators and / or convolution operators, etc.

[0056] Step 503 , according to the set hardware-friendly quantization configuration, simulate quantization is performed on the above-mentioned operator to be quantized to obtain a simulated quantization model.

[0057] Hardware-friendly quantization configuration refers to the selection and configuration of quantization parameters and strategies during the model quantization process to ensure that the quantized model can run efficiently on a specific hardware platform.

[0058] In this embodiment, the hardware-friendly quantization configuration set above may include a hardware-friendly quantization grouping method and a quantization bit number set; thus, step 503 may be: obtaining the weight of the operator to be quantized, grouping the weight of the operator to be quantized according to the quantization grouping method, and for each group of weights obtained by grouping, counting the maximum and minimum values ​​in each group of weights; then, calculating the quantization scaling factor according to the maximum and minimum values ​​in each group of weights and the quantization bit number, and then calculating the quantization offset according to the minimum value in each group of weights and the quantization scaling factor. Finally, quantize each group of weights according to the quantization scaling factor and the quantization offset to obtain the analog quantization weights.

[0059] Among them, the weight of the operator to be quantized includes the weight matrix of the operator to be quantized; therefore, according to the above quantization grouping method, the weight of the operator to be quantized can be grouped as follows: when the hardware to be deployed is an end-side GPU, the weight matrix of the operator to be quantized is grouped according to a predetermined block size; when the hardware to be deployed is an end-side NPU, each row of the weight matrix of the operator to be quantized is grouped. Among them, the above predetermined block size can be set by itself during the specific implementation. This embodiment does not limit the above predetermined block size. For example, the predetermined block size can be 4 rows and 32 columns.

[0060] Specifically, in this embodiment, the server 400 can set the hardware-friendly quantization configuration (quantization bit number) and quantization grouping method, etc.), in each operator to be quantized Insert the analog quantization operator before , the input of the simulated quantization operator is the weight of the operator to be quantized , which is used to simulate the quantization of the weight W of the operator to be quantized and output the weight after simulated quantization .

[0061] Specifically, first, for the weight of the operator to be quantized , can be grouped according to the set hardware-friendly quantization grouping method.

[0062] If the hardware to be deployed is the end-side , then the weight matrix Every 128 elements of 4 rows and 32 columns form a group, that is, the weight W of the operator to be quantized is block-quantized, and the weights belonging to the same block share the quantization parameters. Figure 6 This is a schematic diagram of block quantization. Figure 6 In the figure, a 4×4 matrix is ​​taken as an example to illustrate block quantization. Figure 6 In , the 4×4 matrix is ​​divided into 4 groups, with 4 weights in each group consisting of 2 rows and 2 columns. Figure 6 The four weights in each dotted box are grouped together, and the weights belonging to the same block share quantization parameters.

[0063] If the hardware to be deployed is the end-side , then the weight matrix Each row of elements is a group, that is, the weight matrix is ​​per channel (per-channel) Quantization is performed, and weights belonging to the same row share the quantization parameters. Figure 7 This is a schematic diagram of channel quantization. Figure 7 Still taking a 4×4 matrix as an example, channel-based quantization is explained. Figure 7 In , the 4×4 matrix is ​​divided into 4 groups, with 4 weights in each row as one group. Figure 7 The four weights in each dotted box are a group, and the weights belonging to the same group share the quantization parameters.

[0064] Secondly, for the weight of the operator to be quantized Each group in , count the maximum value in the group weight With minimum , and then according to the set quantization bit number , the quantization scaling factor is calculated by equation (1), and then the quantization offset is calculated by equation (2).

[0065]

[0066] In formula (1) and formula (2), s k is the quantization scaling factor, z k To quantize the offset, round(·) represents the rounding function.

[0067] Finally, for the weight of the operator to be quantized Each group in , we can perform simulation quantization according to formula (3) to obtain the weight after simulation quantization .

[0068]

[0069] In formula (3), represents the rounding function, Represents the truncation function.

[0070] Figure 8 A schematic diagram of the graph structure of a model before and after simulation quantization provided by an embodiment of the present application, Figure 8 The figure shows a schematic diagram comparing the graph structure of the model before and after the simulated quantization of the operators to be quantized in the large language model in the above manner.

[0071] The following is an example to illustrate the process of calculating the quantization scaling factor and the quantization offset. Fig. 9 A schematic diagram of a system for calculating quantization scaling and quantization offset provided for one embodiment of the present application.

[0072] See also Fig. 9 , for the weight of the operator to be quantized whose data type is a floating point number , assuming that it is quantized into an unsigned integer with a data type of 4 bits (i.e. N = 4). Since the representation range of a 4-bit unsigned integer is 0~15, we need to first Scale so that The scaled value range is aligned with the value range of 4-bit unsigned integers. Specifically, we need to count The maximum value W max With the minimum value W min , according to W max and W min Calculate quantization scaling factor , get the quantization scaling factor s k Then calculate the quantization offset z k , so that Each value in is scaled and offset by the quantization scaling factor and falls in the interval [0, 15], then it can be deduced that .

[0073] Step 504, obtain vertical domain business data and labels of the vertical domain business data.

[0074] Step 505, the vertical business data is processed through the simulation quantization model to obtain the output of the simulation quantization model, and the weight of the simulation quantization model is optimized with the purpose of converging the cross entropy loss function between the output of the simulation quantization model and the label of the vertical business data.

[0075] The weight of the simulation quantization model may be optimized by using a gradient descent method.

[0076] Specifically, after obtaining the vertical business data x and the label y of the vertical business data, the corresponding hyperparameters (such as learning rate, input batch size and / or weight decay parameters, etc.) can be set, and then the vertical business data is used to train the simulation quantization model as a whole, and the cross entropy loss function is used. All weights of the above simulated quantization model are fine-tuned and updated using the gradient descent method until the output of the simulated quantization model is With label The cross entropy loss convergence.

[0077] In the specific implementation, during the gradient back propagation process, for the non-differentiable rounding function round(·) in the analog quantization operator, the gradient of the function can be approximated to 1, that is, , then the output of the analog quantization operator About input Gradient , loss function About weight The gradient of , so in the process of gradient back propagation, the weights can be updated according to the gradient calculation formula .

[0078] Step 506, saving the optimized weights of the simulated quantization model, and saving the quantization parameters corresponding to the optimized weights, to obtain an optimized simulated quantization model.

[0079] In this embodiment, the quantization parameters corresponding to the optimized weights may include: quantization scaling coefficients and quantization offsets. That is to say, after the cross entropy loss function between the output of the simulation quantization model and the label of the vertical domain service data converges, it can be determined that the simulation quantization model has been optimized. At this time, the weights optimized by the simulation quantization model and the quantization parameters corresponding to the optimized weights can be saved, so that the optimized simulation quantization model is obtained. The optimized simulation quantization model can be deployed on the terminal device later for the terminal device to call when processing vertical domain service data.

[0080] In the above model optimization method, after the server 400 obtains the large language model to be adjusted, it traverses the operators in the large language model to obtain the operators to be quantized in the large language model, and then simulates and quantizes the operators to be quantized according to the set hardware-friendly quantization configuration to obtain a simulated quantization model. After that, the vertical domain business data and the labels of the vertical domain business data are obtained, and the vertical domain business data is processed through the simulated quantization model to obtain the output of the simulated quantization model, and then the weight of the simulated quantization model is optimized for the purpose of convergence of the cross entropy loss function between the output of the simulated quantization model and the label of the vertical domain business data, and the optimized weight of the simulated quantization model is saved, and the quantization parameters corresponding to the optimized weight are saved to obtain the optimized simulated quantization model, so that the weight of the large language model can be simulated and quantized for the deployment hardware during the fine-tuning of the large language model, and quantization is combined with model fine-tuning, thereby shortening the quantitative analysis and deployment cycle of the large language model. And the large language model can be adaptively adjusted to the quantization loss in the process of fitting the vertical domain data in a data-driven manner, so that the accuracy of the quantized model can be restored.

[0081] use Figure 5 The schematic diagram of fine-tuning a large language model using the method provided in the illustrated embodiment can be as follows: Fig.10 As shown, Fig.10 A schematic diagram of vertical fine-tuning of a large language model provided by an embodiment of the present application. Fig.10 In this embodiment, after obtaining the large language model to be adjusted, the operators in the large language model are traversed to obtain the operators to be quantized in the large language model, and then the operators to be quantized are simulated and quantized according to the set hardware-friendly quantization configuration to obtain a simulated quantization model. After that, the vertical domain business data and the labels of the vertical domain business data are obtained, and the vertical domain business data are processed through the simulated quantization model to obtain the output of the simulated quantization model, and then the weight of the simulated quantization model is optimized according to the input business data and the labels of the business data, with the purpose of convergence of the cross entropy loss function between the output of the simulated quantization model and the labels of the vertical domain business data.

[0082] In addition, the embodiment of the present application also provides a business processing method, which can call according to Figure 5~Figure 10The simulated quantization model optimized by the method provided in the illustrated embodiment processes the vertical domain business data. The business processing method can be applied to terminal devices, wherein the terminal devices can be smart phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks or personal digital assistants (PDA) and other devices; the embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0083] For example, Fig.11 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application is shown in FIG. Fig.11 As shown, the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0084] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0085] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0086] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0087] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0088] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0089] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0090] Below Fig.11 Taking the terminal device 100 shown as an example, the service processing method provided in the embodiment of the present application is explained.

[0091] Fig.12 This is a flowchart of a service processing method provided by an embodiment of the present application, and the service processing method can be executed by the terminal device 100.

[0092] like Fig.12 As shown, the above business processing method may include:

[0093] Step 1201, obtaining business data.

[0094] Step 1202, call the vertical domain large language model to process the above business data.

[0095] Among them, the above vertical domain large language model can be Figure 5~Figure 10 The method provided by the illustrated embodiment completes the optimized simulation quantization model.

[0096] Step 1203: Display the processing result corresponding to the above business data.

[0097] In one implementation of this embodiment, the above-mentioned business data may be: the request for generating a video input by the above-mentioned user, and the above-mentioned request for generating a video includes the information of the image and / or video required to generate the above-mentioned video; thus, step 1202 may be: calling the above-mentioned vertical domain large language model to obtain the information of the image and / or video required to generate the above-mentioned video included in the above-mentioned request.

[0098] That is to say, the above business processing method can be applied in the "smart film" scenario. For details, see Fig.13 , Fig.13 A schematic diagram of processing business data provided for an embodiment of the present application. The user can enter "Please help me generate a video log (video log, vlog) of my travel in Beijing last week" in the interface of the target application, as shown in 1301. After obtaining the request shown in 1301, the terminal device 100 can call the vertical domain large language model to obtain the information of the images and / or videos required to generate the video included in the request shown in 1301. In 1301, "Playing in Beijing last week" can be regarded as the information of the images and / or videos required to generate the video, and the information includes the shooting time and location of the images and / or videos. Among them, the above-mentioned target application can be an artificial intelligence (AI) type application, and this embodiment does not limit the name and / or interface display form of the above-mentioned target application.

[0099] After obtaining the information of the images and / or videos required to generate the video, the terminal device 100 can determine the shooting time and shooting location of the images and / or videos saved by the user. If the shooting time and shooting location of the images and / or videos saved by the user are clear and unique, for example, the user only went to Beijing once last week, then the terminal device 100 can determine that the images and / or videos saved by the user only show that the user was in Beijing once. The terminal device 100 can directly generate a vlog of the user's travel in Beijing last week and display it in the interface of the target application, as shown in 1302; if the terminal device 100 determines that the images and / or videos saved by the user show that the user has multiple images and / or videos in Beijing, the terminal device 100 can call the vertical domain large language model to communicate with the user, allowing the user to clarify the shooting time, for example: "Which time in Beijing do you want to generate a video?" Wait for the user to reply until the logic is completely clear, and then the terminal device 100 generates the video again and displays it in the interface of the target application, as shown in 1302.

[0100] The images and / or videos saved by the user may include not only the images and / or videos saved locally by the terminal device 100, but also the images and / or videos stored in the cloud by the user account of the user. In addition, it should be noted that the terminal device 100 determines the shooting time and shooting location, and generates the video, which is not achieved by calling the vertical domain large language model, but needs to call other models.

[0101] In another implementation of this embodiment, the above-mentioned business data may be: the user's schedule information; thus, step 1202 may be: calling the vertical domain large language model to generate corresponding schedule data according to the above-mentioned user's schedule information.

[0102] That is to say, the above business processing method can be applied in the "one-delay schedule" scenario. For details, see Fig.14 , Fig.14 A schematic diagram of processing business data provided for another embodiment of the present application. The user can enter "There will be a dinner party at 12 noon\nAttend the department staff meeting next Wednesday\nAt 3:30, invite the marked friends to have a meeting together" in the interface of the above-mentioned target application, as shown in 1401. After obtaining the schedule information shown in 1401, the terminal device 100 can call the vertical domain large language model to generate corresponding schedule data according to the schedule information shown in 1401, as shown in 1402. It should be noted that after generating the schedule data shown in 1402, the terminal device 100 can display the schedule data 1402 in the interface of the target application, or it may not display the schedule data 1402 in the interface of the target application.

[0103] In a specific implementation, after generating the schedule data shown in 1402, the terminal device 100 can display the above schedule in the form of a system message or a pop-up box at the first moment before the start time of each schedule to remind the user, such as Fig.15 The time interval between the first moment and the start time of each schedule can be set by the user, and this embodiment does not limit this. For example, the time interval can be half an hour or 10 minutes.

[0104] Fig.15 A schematic diagram of a schedule reminder provided by an embodiment of the present application, Fig.15 Take the pop-up window as an example, showing the start time of the user's dinner party to remind the user. Fig.15 The interface shown is only an example and does not constitute a limitation of this embodiment.

[0105] In another implementation of this embodiment, the above-mentioned acquisition of business data may be: acquiring the user's ticket booking information; thus, step 1202 may be: calling the vertical domain large language model to generate corresponding notification data according to the above-mentioned user's ticket booking information.

[0106] Among them, the above-mentioned user's ticket booking information can be the user's ticket booking information for transportation such as train tickets, bus tickets or airplane tickets, or it can be the user's ticket booking information for movie tickets, drama tickets, concert tickets or park tickets. This embodiment does not limit the user's ticket booking information.

[0107] The application scenario of this implementation method can be regarded as a "notification parsing" scenario. Specifically, after the user books a ticket, as long as the user uses the mobile phone number corresponding to the terminal device 100 to book the ticket, the terminal device 100 can obtain the user's ticket booking information. Then, the terminal device 100 can call the vertical domain large language model to generate corresponding notification data based on the user's ticket booking information. Then, at the second moment before the start time, a notification message is displayed in the form of a system message or a pop-up box to remind the user. Among them, the above-mentioned start time can be the start time on the ticket, such as: the opening time of a movie or the departure time of a train, etc. The time interval between the second moment and the start time can also be set by the user. This embodiment is not limited to this. For example, the above-mentioned time interval can be 3 hours.

[0108] Taking the user booking a train ticket as an example, after the terminal device 100 obtains the user's train ticket booking information, a notification message can be displayed in a pop-up box 3 hours before the train departure time, such as Fig.16 As shown, it reminds the user that the train will depart in 3 hours, and the pop-up interface also displays information such as the train number and the boarding station. Fig.16 A schematic diagram of an interface for displaying a notification message provided by an embodiment of the present application, of course Fig.16 The interface shown is only an example and does not constitute a limitation of the present embodiment. In addition, when it is detected that the user arrives at the boarding station, the terminal device 100 can also display information such as the ticket gate and / or the boarding platform in a pop-up window.

[0109] In the above-mentioned business processing method, after the terminal device 100 obtains the business data, it can call the vertical domain large language model to process the above-mentioned business data, and then display the processing results corresponding to the above-mentioned business data. Since in the process of fine-tuning the large language model to obtain the above-mentioned vertical domain large language model, the large language model adaptively adjusts the quantization loss in the process of fitting the vertical domain data in a data-driven manner, so that the accuracy of the quantized model can be restored, calling the vertical domain large language model to process the business data can obtain a higher quantization accuracy and improve the user experience.

[0110] It is to be understood that some or all of the steps or operations in the above embodiments are merely examples, and the present application embodiments may also perform other operations or variations of various operations. In addition, the various steps may be performed in different orders presented in the above embodiments, and it is possible that not all of the operations in the above embodiments need to be performed.

[0111] It is understandable that, in order to implement the above functions, the server includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0112] This embodiment can divide the server into functional modules according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0113] Fig.17 A schematic diagram of the structure of a server provided in another embodiment of the present application, in which each functional module is divided into corresponding functional modules, Fig.17 A possible composition diagram of the server 1700 involved in the above embodiment is shown. Fig.17 As shown, the server 1700 may include: an operator matching module 1701, a simulation quantization module 1702, a fine-tuning training module 1703 and a model export module 1704;

[0114] The operator matching module 1701 may be used to support the server 1700 in executing step 501 and step 502, and / or other processes of the technical solution described in the embodiment of the present application;

[0115] The simulation quantization module 1702 may be used to support the server 1700 in executing step 503, and / or other processes of the technical solution described in the embodiments of the present application;

[0116] The fine-tuning training module 1703 may be used to support the server 1700 in executing step 504 and step 505, and / or other processes of the technical solution described in the embodiment of the present application;

[0117] The model export module 1704 may be used to support the server 1700 in executing step 506 and / or other processes of the technical solution described in the embodiments of the present application;

[0118] It should be noted that this application Figure 5~Figure 10All relevant contents of each step involved in the illustrated method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0119] The server 1700 provided in this embodiment is used to execute Figure 5~Figure 10 The model optimization method provided by the illustrated embodiment can therefore achieve the same effect as the above-mentioned method.

[0120] It should be understood that the server 1700 may correspond to Figure 4 The functions of the operator matching module 1701, the simulation quantization module 1702, the fine-tuning training module 1703 and the model export module 1704 can be Figure 4 The processor 410, the communication interface 420 and the memory 430 in the server 400 are shown to be implemented.

[0121] In the case of adopting an integrated unit, the server 1700 may include a processing unit, a storage unit, and a communication unit.

[0122] The processing unit may be used to control and manage the actions of the server 1700, for example, to support the server 1700 in executing the steps performed by the operator matching module 1701, the simulation quantization module 1702, the fine-tuning training module 1703, and the model export module 1704. The storage unit may be used to support the server 1700 in storing program codes and data, etc. The communication unit may be used to support the communication between the server 1700 and other devices.

[0123] The processing unit may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and the like. The storage unit may be a memory. The communication unit may specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip and / or a Wi-Fi chip.

[0124] In one embodiment, when the processing unit is a processor and the storage unit is a memory, the server 1700 involved in this embodiment can be a server having Figure 4 Device of the structure shown.

[0125] It is understandable that, in order to implement the above functions, the terminal device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0126] This embodiment can divide the terminal device into functional modules according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0127] Fig.18 A schematic diagram of the structure of a terminal device provided in another embodiment of the present application, in which each functional module is divided according to each function, Fig.18 A possible schematic diagram of the composition of the terminal device 1800 involved in the above embodiment is shown. Fig.18 As shown, the terminal device 1800 may include: a receiving unit 1801, a processing unit 1802 and a sending unit 1803;

[0128] Among them, the processing unit 1802 can be used to support the terminal device 1800 to execute steps 1201 to 1203, and / or other processes of the technical solution described in the embodiments of the present application.

[0129] It should be noted that this application Figure 12~Figure 16 All relevant contents of each step involved in the illustrated method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0130] The terminal device 1800 provided in this embodiment is used to execute Figure 12~Figure 16 The service processing method provided by the illustrated embodiment can therefore achieve the same effect as the above method.

[0131] It should be understood that the terminal device 1800 may correspond to Fig.11 The terminal device 100 shown in FIG. The functions of the receiving unit 1801 and the sending unit 1803 can be Fig.11The processor 110, antenna 1 and mobile communication module 150 in the terminal device 100 shown in the figure, and / or, are implemented by the processor 110, antenna 2 and wireless communication module 160; the function of the processing unit 1802 can be implemented by Figure 1 The processor 110 and the display screen 194 in the terminal device 100 are shown.

[0132] In the case of adopting an integrated unit, the terminal device 1800 may include a processing module, a storage module, and a communication module.

[0133] The processing module can be used to control and manage the actions of the terminal device 1800, for example, it can be used to support the terminal device 1800 to execute the steps performed by the above-mentioned receiving unit 1801, processing unit 1802 and sending unit 1803. The storage module can be used to support the terminal device 1800 to store program codes and data, etc. The communication module can be used to support the communication between the terminal device 1800 and other devices.

[0134] Among them, the processing module can be a processor or a controller, which can implement or execute various exemplary logic boxes, modules and circuits described in combination with the contents disclosed in this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip and / or a Wi-Fi chip.

[0135] In one embodiment, when the processing module is a processor and the storage module is a memory, the terminal device 1800 involved in this embodiment may be a Fig.11 Device of the structure shown.

[0136] The present application also provides a computer-readable storage medium in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the present application. Figure 5~Figure 10 The method provided in the embodiment shown, or the computer is enabled to execute the present application Figure 12~Figure 16 The method provided by the illustrated embodiment.

[0137] The present application also provides a computer program product, which includes a computer program, which, when executed on a computer, enables the computer to execute the present application. Figure 5~Figure 10 The method provided in the embodiment shown, or the computer is enabled to execute the present application Figure 12~Figure 16 The method provided by the illustrated embodiment.

[0138] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0139] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0141] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0142] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model optimization method, characterized in that: include: Get the large language model to be adjusted; Traversing the operators in the large language model to obtain operators to be quantized in the large language model; According to the set hardware-friendly quantization configuration, simulate quantization is performed on the operator to be quantized to obtain a simulated quantization model; Obtain vertical domain service data and a label of the vertical domain service data; The vertical domain service data is processed by the simulation quantization model to obtain the output of the simulation quantization model, and the weight of the simulation quantization model is optimized with the purpose of converging the cross entropy loss function between the output of the simulation quantization model and the label of the vertical domain service data; Saving the optimized weights of the simulated quantization model and saving the quantization parameters corresponding to the optimized weights to obtain an optimized simulated quantization model; The set hardware-friendly quantization configuration includes a set hardware-friendly quantization grouping method and a quantization bit number; The step of performing simulated quantization on the operator to be quantized according to the set hardware-friendly quantization configuration to obtain a simulated quantization model includes: Obtaining the weight of the operator to be quantized; According to the quantization grouping method, the weights of the operators to be quantized are grouped; For each group of weights obtained by grouping, counting the maximum and minimum values ​​in each group of weights; Calculate and obtain a quantization scaling factor according to the maximum value and the minimum value in each group of weights and the number of quantization bits; Calculate and obtain a quantization offset according to the minimum value in each group of weights and the quantization scaling coefficient; Each group of weights is quantized according to the quantization scaling coefficient and the quantization offset to obtain analog quantization weights.

2. The method according to claim 1, characterized in that The quantization parameters corresponding to the optimized weights include: a quantization scaling coefficient and a quantization offset.

3. The method according to claim 1, characterized in that The weight of the operator to be quantized includes a weight matrix of the operator to be quantized; The grouping of the weights of the operators to be quantized according to the quantization grouping method includes: When the hardware to be deployed is a terminal-side graphics processor, the weight matrices of the operators to be quantized are grouped according to a predetermined block size; When the hardware to be deployed is a terminal-side neural network processor, each row of the weight matrix of the operator to be quantized is formed as a group.

4. The method according to claim 1, characterized in that: The optimizing the weight of the simulation quantization model includes: The weights of the simulated quantization model are optimized using a gradient descent method.

5. A business processing method, characterized in that: include: Get business data; Calling a vertical domain large language model to process the business data; wherein the vertical domain large language model includes a simulated quantization model optimized by the method described in any one of claims 1 to 4; Display the processing results corresponding to the business data.

6. The method according to claim 5, characterized in that The business data includes: a request for generating a video input by a user, wherein the request for generating a video includes information of images and / or videos required for generating the video; The calling of the vertical domain large language model to process the business data includes: The vertical domain large language model is called to obtain the image and / or video information required to generate the video included in the request.

7. The method according to claim 5, characterized in that The business data includes: user's schedule information; The calling of the vertical domain large language model to process the business data includes: The vertical domain large language model is called to generate corresponding schedule data according to the schedule information of the user.

8. The method according to claim 5, characterized in that The obtaining of business data includes: Get the user's booking information; The calling of the vertical domain large language model to process the business data includes: The vertical domain language model is called to generate corresponding notification data according to the user's ticket booking information.

9. A server, characterized in that: include: one or more processors; Memory; Multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the server, enable the server to perform the method according to any one of claims 1 to 4.

10. A terminal device, characterized in that: include: one or more processors; Memory; Multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the terminal device, enable the terminal device to execute the method according to any one of claims 5 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 4, or enables the computer to execute the method according to any one of claims 5 to 8.

Images