Model processing method, device, equipment and storage medium

Abstract

The present disclosure provides a model processing method, device, equipment and storage medium, which relate to the field of data processing, especially to the field of artificial intelligence and intelligent search. The specific implementation scheme is: input the retrieval data pair into the semantic model, the retrieval data pair at least includes the target retrieval information, and the retrieval result corresponding to the target retrieval information; using the semantic model, the parameters to be processed required to characterize the semantic features are at least quantized as follows: the first parameter to be processed in the first parameter group after the first normalization process through the first processing branch is quantized, and the second parameter to be processed in the second parameter group of the second processing branch after the first normalization process is quantized; the target semantic model after the parameter quantization process is obtained; the target semantic model is used to extract semantic features to obtain the matching degree between the target retrieval information and the retrieval result corresponding to the target retrieval information.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to the field of artificial intelligence and intelligent search technology. Background Art

[0002] The models used in online search services are large in scale and consume a lot of resources. Therefore, model compression is needed to reduce resource consumption and costs. Therefore, how to improve processing efficiency while ensuring controllable accuracy is an urgent problem to be solved in large-scale models. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device and storage medium for model processing.

[0004] According to one aspect of the present disclosure, a model processing method is provided, comprising:

[0005] Inputting a retrieval data pair into a semantic model, wherein the retrieval data pair includes at least target retrieval information and a retrieval result corresponding to the target retrieval information; the semantic model includes N parameters to be processed, wherein the parameters to be processed among the N parameters to be processed are parameters required for characterizing the semantic features of the target retrieval information, or are parameters required for characterizing the semantic features of the retrieval result corresponding to the target retrieval information;

[0006] Using the semantic model, at least the following quantization processing is performed on the parameters to be processed required to characterize the semantic features, wherein the quantization processing includes: quantizing the first parameter to be processed in the first parameter group after the first normalization processing and the second parameter to be processed in the second parameter group of the second processing branch after the first normalization processing; the first parameter to be processed is one of the N parameters to be processed, and the second parameter to be processed is one of the N parameters to be processed;

[0007] A target semantic model is obtained after parameter quantization processing; the target semantic model is used to extract semantic features of the target retrieval information and the retrieval results corresponding to the target retrieval information in the retrieval data pair to obtain the matching degree between the target retrieval information and the retrieval results corresponding to the target retrieval information.

[0008] According to another aspect of the present disclosure, there is provided a model processing device, comprising:

[0009] An input unit, used to input a retrieval data pair into a semantic model, wherein the retrieval data pair includes at least target retrieval information and a retrieval result corresponding to the target retrieval information; the semantic model includes N parameters to be processed, wherein the parameters to be processed among the N parameters to be processed are parameters required for characterizing the semantic features of the target retrieval information, or are parameters required for characterizing the semantic features of the retrieval result corresponding to the target retrieval information;

[0010] A processing unit is used to use a semantic model to perform at least the following quantization processing on the parameters to be processed required to represent the semantic features, wherein the quantization processing includes: quantizing the first parameter to be processed in the first parameter group via the first processing branch after the first normalization processing, and quantizing the second parameter to be processed in the second parameter group of the second processing branch after the first normalization processing; the first parameter to be processed is one of the N parameters to be processed, and the second parameter to be processed is one of the N parameters to be processed; obtaining a target semantic model after parameter quantization processing; the target semantic model is used to extract the semantic features of the target retrieval information and the retrieval results corresponding to the target retrieval information in the retrieval data pair, so as to obtain the matching degree between the target retrieval information and the retrieval results corresponding to the target retrieval information.

[0011] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0012] at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0015] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0017] In this way, by quantizing the parameters to be processed corresponding to the two processing branches after normalization, the model can be compressed and the processing efficiency of the model can be improved while ensuring controllable accuracy, laying the foundation for subsequent prediction using the quantized model; moreover, the solution is simple and easy to implement, has a low threshold for use, and is both practical.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 This is a schematic flow chart of a model processing method according to an embodiment of the present application. Figure 1 ;

[0021] Figure 2 It is a schematic diagram of the application scenario of the disclosed solution in the search business scenario;

[0022] Figure 3 This is a schematic diagram of the structure of the semantic model according to an embodiment of the present application. Figure 1 ;

[0023] FIG4( a ) is a schematic diagram showing the distribution effects of activation parameters before and after smoothing according to an embodiment of the present application;

[0024] FIG4( b ) is a schematic diagram showing the distribution effect of weight parameters before and after smoothing according to an embodiment of the present application;

[0025] FIG. 4( c ) is a schematic diagram of the structure of a semantic model according to an embodiment of the present application. Figure 2 ;

[0026] FIG. 5( a ) is a schematic diagram of the structure of a semantic model according to an embodiment of the present application. Figure 3 ;

[0027] FIG5( b ) is a fourth structural diagram of a semantic model according to an embodiment of the present application;

[0028] Figure 6 is a structural diagram 5 of a semantic model according to an embodiment of the present application;

[0029] Figure 7 is a structural schematic diagram of a model processing device according to an embodiment of the present application;

[0030] Figure 8 It is a block diagram of an electronic device used to implement the model processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0032] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0033] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0034] Based on this, the disclosed solution proposes a full quantization method for semantic models, such as post-layernorm models, so that the accuracy can be maintained without loss while ensuring acceleration benefits.

[0035] Specifically, Figure 1 This is a schematic flow chart of a model processing method according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices with classical computing capabilities.

[0036] The method includes at least part of the following contents. Figure 1 As shown, including:

[0037] Step S101: input the search data pair into the semantic model.

[0038] Here, the search data pair includes at least target search information and a search result corresponding to the target search information; the semantic model includes N parameters to be processed.

[0039] In a specific example, the target retrieval information may be specifically related data such as text, image, or audio used for retrieval; accordingly, the retrieval results corresponding to the target retrieval information may also be specifically related data such as text, image, or audio used for retrieval.

[0040] Step S102: using the semantic model, at least perform the following quantization processing on the parameters to be processed required to represent the semantic features.

[0041] Here, the quantization process includes:

[0042] The first parameter to be processed in the first parameter group of the first processing branch after the first normalization processing is quantized, and the second parameter to be processed in the second parameter group of the second processing branch after the first normalization processing is quantized.

[0043] Here, the first parameter to be processed is one of the N parameters to be processed, and the first parameter group contains at least one first parameter to be processed. Correspondingly, the second parameter to be processed is one of the N parameters to be processed, and the second parameter group contains at least one second parameter to be processed. In other words, the first parameter to be processed is a parameter required to characterize the semantic features of the target retrieval information, or a parameter required to characterize the semantic features of the retrieval results corresponding to the target retrieval information, and accordingly, the second parameter to be processed is a parameter required to characterize the semantic features of the target retrieval information, or a parameter required to characterize the semantic features of the retrieval results corresponding to the target retrieval information.

[0044] It should be noted that the above two quantization processing processes may be executed synchronously or asynchronously, and the disclosed solution does not limit the specific execution order of the above two quantization processing processes.

[0045] Step S103: Obtain the target semantic model after parameter quantization processing.

[0046] Here, the target semantic model is used to extract semantic features of the target retrieval information and the retrieval result corresponding to the target retrieval information in the retrieval data pair, so as to obtain the matching degree between the target retrieval information and the retrieval result corresponding to the target retrieval information.

[0047] In this way, the disclosed scheme quantizes the parameters to be processed corresponding to the two processing branches after normalization, thereby compressing the model and improving the processing efficiency of the model, laying the foundation for subsequent prediction using the quantized model; moreover, the scheme is simple and easy to implement, has a low threshold for use, and is both practical.

[0048] It should be pointed out that the semantic model can specifically be a pre-trained model; that is, the disclosed solution, based on the pre-trained model, quantizes the parameters to be processed (also referred to as parameters to be quantized, which are parameters required to characterize semantic features) to obtain the target semantic model that can ultimately be used for prediction; in this way, the computing time is greatly reduced and a large amount of computing resources are saved.

[0049] In a specific example, the semantic model described above may be a post-layernorm model. Further, the first normalization process may be implemented by the first normalization (layernorm) layer in the post-layernorm model. Further, in the post-layernorm model, after the first normalization (layernorm) layer, there are two processing branches, for example, a first quantization layer may be provided on the first processing branch, at which time the first quantization layer may be used to quantize the first parameter to be processed in the first parameter group after the first normalization process through the first processing branch, and accordingly, a second quantization layer may be provided on the second processing branch, at which time the second quantization layer may be used to quantize the second parameter to be processed in the second parameter group of the second processing branch after the first normalization process.

[0050] In a specific example, both the first quantization layer and the second quantization layer can use linear quantization (quantizelinear) to quantize the processing parameters. Further, it can be understood that in practical applications, the quantization layer can also use other quantization methods, for example, it can be set according to the actual model requirements, and the present disclosure does not limit this.

[0051] In a specific example, Figure 2 The schematic diagram is an application scenario of the disclosed solution in a search business scenario, which involves a user 201, a server (or a server cluster) 202, and a target semantic model 203 for prediction obtained through the disclosed solution; the target semantic model 203 runs in the server (or server cluster) 202; specifically, the user 202 inputs a search term, the server 202 performs a quick query, and obtains multiple search results corresponding to the search term, then the server 202 calls the target semantic model 203, inputs the search term and the corresponding multiple search results into the target semantic model 203, to obtain the similarity between the search term and each search result, and the server 202 outputs a search result with a high similarity (for example, the similarity is greater than a threshold); in this way, in the online scenario of the search business, the target semantic model obtained by the disclosed solution can output search results that meet the accuracy requirements while ensuring acceleration benefits, thereby effectively improving the user experience.

[0052] It is understandable that the above Figure 2 This is only a specific example. In actual applications, the target semantic model 203 may also be stored on a server other than the server (or server cluster) 202, and the disclosed solution does not impose any limitation on this.

[0053] In a specific example of the disclosed solution, the quantization process may be performed in the following manner; specifically, the semantic model described above is used to perform at least the following quantization process on the parameters to be processed required to represent the semantic features (i.e., step S102 described above), specifically including:

[0054] Using the target coding unit in the semantic model, at least the following quantization processing is performed on the parameters to be processed required to represent the semantic features; the semantic model includes multiple coding units connected in series, and the target coding unit is one of the multiple coding units.

[0055] Here, the encoding unit in the plurality of encoding units is used to extract the semantic features of the target retrieval information in the retrieval data pair and the retrieval result corresponding to the target retrieval information.

[0056] That is to say, in one example, the semantic model may include several encoding units (ENCODER), and the several encoding units may be connected in series; at this time, for the current encoding unit (such as the target encoding unit), the input of the current encoding unit is the output of the previous encoding unit, and the output of the current encoding unit is the input of the next encoding unit; here, for the first encoding unit, the input of the first encoding unit is the input of the semantic model, that is, the retrieval data pair; and for the last encoding unit, the output of the last encoding unit is the output of the semantic model.

[0057] For example, if Figure 3 As shown, a semantic model including 12 coding units is taken as an example, wherein the 12 coding units are connected in series, for example, for the first coding unit, the input is the input of the semantic model, such as the retrieval data pair, and the output is the input of the second coding unit; for any coding unit from the second to the eleventh coding unit, the input is the output of the previous coding unit, and the output is the input of the next target coding unit; for the last (i.e. the twelfth) coding unit, the output is the output of the semantic model.

[0058] It is understandable that in a specific example, the processing flow of different coding units can be the same. For example, in the process of processing data by the semantic model, different coding units can all execute the processing method described in the disclosed solution, but due to the serial connection, the data processed by different coding units are different. In addition, the processing flow of different coding units can also be different. In this case, it can be adjusted according to the actual model needs, and the disclosed solution does not limit this.

[0059] In this way, the disclosed solution provides a specific quantization solution based on the semantic model, which is compatible with the existing model and lays the foundation for subsequent prediction using the quantized model; moreover, the solution is simple and easy to implement, has a low threshold for use, and is practical.

[0060] In a specific example of the disclosed solution, specifically, the above-mentioned quantization processing of the first parameter to be processed in the first parameter group after the first normalization processing through the first processing branch specifically includes:

[0061] The first parameter to be processed in the first parameter group after the first normalization processing and the processing by the feedforward neural network layer of the first processing branch is quantized.

[0062] That is to say, the first processing branch of the semantic model also includes a feedforward neural network (FFN) layer, for example, the feedforward neural network layer is used to extract the semantic features of the target retrieval information in the retrieval data pair after the first normalization process, and / or the semantic features of the retrieval results corresponding to the target retrieval information in the retrieval data pair. In this way, the feedforward neural network layer is used to improve the accuracy of the output results.

[0063] Furthermore, a first quantization layer is provided after the feedforward neural network layer on the first processing branch of the semantic model to quantize the first parameter to be processed after the feedforward neural network layer is processed.

[0064] In this way, the disclosed scheme provides a specific quantization process, so that the model can be compressed while ensuring controllable accuracy, thereby laying the foundation for subsequent prediction using the quantized target semantic model; moreover, the scheme is simple and easy to implement, has a low threshold for use, and is both practical.

[0065] In a specific example of the present disclosure, the feedforward neural network layer is at least used to smooth the first target parameter to be processed after the first normalization processing, and to quantize it after the smoothing processing, and to smooth the acquired second target parameter to be processed, and to quantize it after the smoothing processing.

[0066] Here, the first target parameter to be processed is one of N parameters to be processed, and correspondingly, the second target parameter to be processed is one of N parameters to be processed.

[0067] In a specific example, the first target parameter to be processed can be specifically an activation parameter. For example, there are multiple activation parameters after the first normalization process, as shown in FIG4(a). If there are many outliers in the multiple activation parameters, resulting in poor distribution of the activation parameters and difficulty in quantization, the multiple activation parameters can be smoothed first, for example, divided by a smoothing factor, so that the smoothed activation parameters are easy to quantize. After quantization, the processing accuracy can be effectively ensured, and the effect of lossless accuracy can be achieved while ensuring the acceleration benefit.

[0068] In another specific example, the second target parameter to be processed is a weight parameter. For example, there are multiple weight parameters, as shown in Figure 4(b). The distribution of the multiple weight parameters is smooth and easy to quantify. However, in order to ensure the equivalence of the processing flow, the multiple weight parameters also need to be smoothed, for example, multiplied by a smoothing factor. It should be pointed out that although the distribution of the weight parameters after smoothing is slightly worse than that of the weight parameters before processing, it is still easy to quantify. At the same time, this processing can greatly improve the smoothness of the distribution of the activation parameters as shown in Figure 4(a), thereby making the overall effect better, thereby achieving the effect of lossless accuracy while ensuring the acceleration benefit.

[0069] For example, as shown in Figure 4(c), a feedforward neural network layer is provided on the first processing branch of the semantic model, and the feedforward neural network layer may specifically include a third quantization layer, a fourth quantization layer, and a first fully connected layer (FullConnection, FC); wherein the third quantization layer is used to smooth the first target parameter to be processed after the first normalization processing, and perform quantization processing after smoothing, and input the result of the quantization processing to the first fully connected layer; the fourth quantization layer is used to smooth the acquired second target parameter to be processed, and perform quantization processing after smoothing, and input the result of the quantization processing to the first fully connected layer.

[0070] Furthermore, the feedforward neural network layer may also include a fifth quantization layer, a sixth quantization layer, and a second fully connected layer; in this case, the fifth quantization layer and the sixth quantization layer may be used to further quantize the processing parameters, and then output to the next processing layer after being processed by the second fully connected layer. It should be noted that, as can be seen from Figure 4(c), the output of the first fully connected layer is used as the input of the fifth quantization layer, and then output to the next processing layer via the second fully connected layer.

[0071] In this way, the disclosed solution provides a specific solution for implementing quantization processing using a feedforward neural network layer, thereby implementing quantization processing of the first target parameter to be processed and the second target parameter to be processed, thereby further improving the efficiency of model reasoning to further improve the user experience.

[0072] In a specific example of the disclosed solution, before quantizing the first parameter to be processed, the method further includes:

[0073] The first parameter to be processed is subjected to first element-wise data processing (for example, processed by a first element processing layer in a semantic model) to smooth the first parameter to be processed.

[0074] In this way, before the quantization process is performed, the first parameter to be processed is smoothed to facilitate quantization, thereby effectively ensuring the acceleration benefit while achieving the effect of lossless accuracy.

[0075] For example, a first element processing layer is provided on the first processing branch and before the first quantization layer, and the first element processing layer is used to perform first element data processing on the first parameter to be processed, so as to smooth the first parameter to be processed. Then, after the first parameter to be processed is smoothed by the first element processing layer, it is input to the first quantization layer for quantization processing, thus laying the foundation for realizing full quantization processing, and further laying the foundation for improving the efficiency of model reasoning and further improving user experience.

[0076] In a specific example, as shown in FIG5(a), after the feedforward neural network layer on the first processing branch, there is also a first element processing layer (for example), and further, after the first element processing layer, the first quantization layer is provided, and then the first parameter to be processed after the first normalization processing (for example, the first normalization layer processing) and after being processed by the feedforward neural network layer of the first processing branch is smoothed by the first element processing layer and then input into the first quantization layer for quantization processing, thus laying the foundation for realizing full quantization processing, and at the same time, laying the foundation for ensuring acceleration benefits while ensuring lossless accuracy, and thus laying the foundation for improving the efficiency of model reasoning and further improving user experience.

[0077] Further, in a specific example, the first element-wise data processing may be performed in the following manner: Specifically, the first element-wise data processing of the first parameter to be processed described above specifically includes: dividing the first parameter to be processed by a first smoothing factor. That is, the first element-wise data processing may specifically be an elementwise division (Elementwise div) processing.

[0078] In this way, the disclosed solution provides a specific solution for quantizing the first parameter to be processed via the first processing branch, thereby laying the foundation for achieving full quantization processing. At the same time, it also lays the foundation for ensuring acceleration benefits while ensuring lossless accuracy, thereby laying the foundation for improving the efficiency of model reasoning and further improving user experience.

[0079] In a specific example of the disclosed solution, after quantizing the first parameter to be processed in the first parameter group and quantizing the second parameter to be processed in the second parameter group using the semantic model, the method further includes:

[0080] Performing data processing in terms of a second element on a first processing result corresponding to the first processing branch after the quantization processing and a second processing result corresponding to the second processing branch after the quantization processing (for example, processing through a second element processing layer in a semantic model);

[0081] After the second element aspect data processing (for example, processing by the second element processing layer in the semantic model), the third parameter to be processed in the third parameter group is subjected to third element aspect data processing (for example, processing by the third element processing layer in the semantic model).

[0082] For example, a second element processing layer is set on the first processing branch and after the first quantization layer, and the second element processing layer is used to perform second element-wise data processing on the first processing result corresponding to the first processing branch after quantization processing and the second processing result corresponding to the second processing branch after quantization processing, such as performing elementwise addition (Elementwise add) processing on the first processing result and the second processing result to merge the data of the first processing branch and the second processing branch. Then, a third element processing layer is set after the second element processing layer, and the third element processing layer is used to perform third element-wise data processing on the third parameter to be processed in the third parameter group after the second element-wise data processing to ensure the equivalence of the processing flow, so as to achieve full quantization processing of the parameters, thereby ensuring acceleration benefits while ensuring lossless accuracy, so as to effectively improve the efficiency of model reasoning and further improve user experience.

[0083] Here, it should be noted that the third parameter group may include the first parameter to be processed and / or the second parameter to be processed. In this case, when the third parameter group includes the first parameter to be processed, the third parameter to be processed may be the first parameter to be processed. Correspondingly, when the third parameter group includes the second parameter to be processed, the third parameter to be processed may be the second parameter to be processed. Furthermore, the third parameter group may also include other parameters to be processed except the first parameter to be processed and the second parameter to be processed, and the disclosed solution does not limit this.

[0084] In a specific example, as shown in FIG5(b), after the feedforward neural network layer on the first processing branch, there is also a first element processing layer, and the first quantization layer is arranged after the first element processing layer; and after the first quantization layer and the second quantization layer, a second element processing layer is arranged, and the output of the first quantization layer and the output of the second quantization layer are both used as inputs of the second element processing layer, so that the second element processing layer is used to summarize the first processing result of the first processing branch and the second processing result of the second processing branch. Further, a third element processing layer is arranged after the second element processing layer. That is, in order to avoid a significant decrease in accuracy due to direct quantization, in this example, the input of the second element processing layer (such as the Elementwise add layer) is pre-smoothed to ensure that the output is in a smooth state (such as the semantic features of the output are in a smooth state), for example, the parameters to be processed of the first processing branch are smoothed, such as divided by a smoothing factor, to ensure that the distribution is relatively smooth and easy to quantize; at the same time, in order to ensure equivalence, processing is performed again after the second element processing layer, such as multiplying by the smoothing factor, so as to achieve full quantization of the parameters, thereby ensuring acceleration benefits while ensuring lossless accuracy, so as to effectively improve the efficiency of model reasoning and further improve user experience.

[0085] Furthermore, after the second element processing layer in the semantic model, a second normalization layer may be set, and then the output result is obtained after processing by the second normalization layer.

[0086] Further, in a specific example, the third element data processing can be performed in the following manner: Specifically, the third element data processing is performed on the third parameter to be processed after the second element data processing, including: multiplying the third parameter to be processed after the second element data processing by the first smoothing factor. For example, the third element data processing is performed on the third parameter to be processed after the second element processing layer is summarized, for example, elementwise multiplication (Elementwise mul) processing is performed.

[0087] In this way, the two types of parameters to be processed that are input into the second element processing layer are quantized, thereby achieving full quantization processing. Moreover, in order to ensure the equivalence of the processing flow, a third element processing layer is added after the second element processing layer. In this way, the acceleration benefits are guaranteed while the accuracy is intact, thereby effectively improving the efficiency of model reasoning and further improving the user experience.

[0088] In a specific example, the target coding unit described above may be specifically a structure as shown in FIG5(b), and accordingly, the processing flow described in the disclosed scheme may be executed in the target coding unit; further, in the case where the semantic model includes multiple coding units connected in series, each of the multiple coding units may also be a structure as shown in FIG5(b), except that due to the serial connection, different coding units process different specific data.

[0089] In a specific example, if Figure 6 As shown, it is a schematic diagram of the structure of a semantic model including multiple coding units, and a schematic diagram of the structure of each coding unit in the multiple coding units; here, the coding unit can be used to extract the target retrieval information in the retrieval data pair, and / or the semantic features of the retrieval results corresponding to the target retrieval information. It should be noted that Figure 6 The structure shown in the dotted box can be specifically the structure shown in Figure 5(b). Furthermore, in this example, the encoding unit can also specifically include a start normalization layer, a third fully connected layer (for example, it can be specifically a fully connected layer based on query key values ​​(Query Key Value Full Connection, QKV FC)), a self-attention mechanism layer, and a fourth fully connected layer. The output of the fourth fully connected layer is used as the input of the first normalization layer to implement the full quantization processing flow of the disclosed solution. In this way, the full quantization processing of the parameters is realized, ensuring acceleration benefits while ensuring lossless accuracy, thereby effectively improving the efficiency of model reasoning and further improving user experience.

[0090] Furthermore, it should be noted that the quantization layers described in the present disclosure, such as the first quantization layer, the second quantization layer, or the third to sixth quantization layers contained in the feedforward neural network layer, can be specifically linear quantization (quantize linear); or, further, they can also be specifically Wie quantization and dequantization (quantize dequantize, QDQ) layers, and the present disclosure does not impose any restrictions on this.

[0091] In summary, the disclosed solution realizes full quantization. For example, in the feedforward neural network layer, the quantization of activation parameters and weight parameters is realized. Moreover, since the quantization process is performed after the smoothing process, it is possible to ensure the acceleration gain while ensuring the accuracy is intact. For example, the acceleration gain can reach about 30%. Moreover, the quantization of other parameters except the activation parameters and weight parameters is also realized. For example, the quantization of other parameters is performed using the first quantization layer and the second quantization layer. Moreover, the quantization process is also performed after the smoothing process. Therefore, the accuracy is further ensured while the acceleration gain is intact. For example, on the basis of the acceleration gain of about 30%, the performance improvement of about 20% can be obtained.

[0092] It should be noted that the quantization process described above in the disclosed solution can be executed online or offline, and the disclosed solution does not impose any limitation on this.

[0093] In a specific example of the disclosed solution, after obtaining the target semantic model, prediction may be performed in the following manner, such as online prediction. Specifically, after obtaining the target semantic model, the method further includes:

[0094] Inputting the to-be-reasoned retrieval data pair into the target semantic model, wherein the to-be-reasoned retrieval data pair at least includes the to-be-reasoned retrieval information and the target retrieval result corresponding to the to-be-reasoned retrieval information;

[0095] A target reasoning result of the target semantic model is obtained, wherein the target reasoning result represents a matching degree between the retrieval information to be reasoned and the target retrieval result.

[0096] It is understandable that in the online prediction scenario, the to-be-reasoned retrieval data pair can be specifically online data. For example, in the online retrieval scenario, the to-be-reasoned retrieval information can be specifically the search term input by the user, and the target retrieval result corresponding to the to-be-reasoned retrieval information can be specifically the retrieval result of the search term. In this way, the target semantic model can be used to improve the online retrieval efficiency, thereby improving the user experience.

[0097] It should be pointed out that in the model reasoning stage, the relevant plug-ins (such as the skip layernorm plugin) of the reasoning library (such as TensorRT) required to be called in the reasoning process can be adapted and modified based on the first smoothing factor to adapt to the target semantic model after the above-mentioned full quantization processing, thereby improving the processing efficiency of the reasoning stage.

[0098] In this way, since the target semantic model used for model prediction in the disclosed solution is a fully quantized model, the model reasoning efficiency can be effectively improved, thereby effectively improving the user experience.

[0099] In summary, the disclosed solution has the following advantages:

[0100] First, it has acceleration benefits and lossless accuracy. The disclosed solution overcomes the problem that acceleration benefits can be achieved after full quantization but lossless accuracy cannot be guaranteed.

[0101] Second, it runs fast and occupies less computing resources. The disclosed solution fully quantizes the parameters on the pre-trained semantic model, which reduces the calculation of complex links. Therefore, it greatly reduces the occupancy of computing resources and runs faster.

[0102] Third, it improves user experience. In online search business scenarios, the target semantic model obtained by using the disclosed solution can quickly provide users with more accurate search results, further improving the user experience.

[0103] The effectiveness of the disclosed solution is illustrated below with reference to specific data; as shown in Table 1, in the search business, the model quantization accuracy of the disclosed solution is greatly improved to lossless.

[0104] Table 1

[0105]

[0106]

[0107] As shown in Figure 2, the fully quantized target semantic model of the disclosed solution can be accelerated by about 20% compared to the existing partially quantized models (such as FP16, FC, FC+Matmul, FC+Skipin).

[0108] Table 2

[0109]

[0110] The disclosed solution also provides a model processing device, such as Figure 7 As shown, including:

[0111] An input unit 701 is used to input a search data pair into a semantic model, wherein the search data pair includes at least target search information and a search result corresponding to the target search information; the semantic model includes N parameters to be processed, wherein the parameters to be processed among the N parameters to be processed are parameters required for characterizing the semantic features of the target search information, or are parameters required for characterizing the semantic features of the search result corresponding to the target search information;

[0112] The processing unit 702 is used to use the semantic model to perform at least the following quantization processing on the parameters to be processed required to represent the semantic features, wherein the quantization processing includes: quantizing the first parameter to be processed in the first parameter group via the first processing branch after the first normalization processing, and quantizing the second parameter to be processed in the second parameter group of the second processing branch after the first normalization processing; the first parameter to be processed is one of the N parameters to be processed, and the second parameter to be processed is one of the N parameters to be processed; obtaining a target semantic model after the parameter quantization processing; the target semantic model is used to extract the semantic features of the target retrieval information and the retrieval results corresponding to the target retrieval information in the retrieval data pair, so as to obtain the matching degree between the target retrieval information and the retrieval results corresponding to the target retrieval information.

[0113] In a specific example of the disclosed solution, the processing unit 702 is specifically configured to:

[0114] Using the target coding unit in the semantic model, the parameters to be processed required to represent the semantic features are quantized at least as follows;

[0115] Among them, the semantic model includes multiple encoding units connected in series, and the encoding units in the multiple encoding units are used to extract semantic features of target retrieval information and retrieval results corresponding to the target retrieval information in the retrieval data pair; the target encoding unit is one of the multiple encoding units.

[0116] In a specific example of the disclosed solution, the processing unit 702 is specifically configured to:

[0117] quantizing a first parameter to be processed in the first parameter group after the first normalization processing and after the feedforward neural network layer of the first processing branch has been processed;

[0118] The feedforward neural network layer is used to extract semantic features of target retrieval information in the retrieval data pair after the first normalization process, and / or semantic features of retrieval results corresponding to the target retrieval information in the retrieval data pair.

[0119] In a specific example of the present disclosure, the feedforward neural network layer is at least used to smooth the first target parameter to be processed after the first normalization processing, and to quantize it after the smoothing processing, and to smooth the acquired second target parameter to be processed, and to quantize it after the smoothing processing.

[0120] In a specific example of the disclosed solution, the processing unit 702 is further configured to:

[0121] Performing first element-wise data processing on the first parameter to be processed to smooth the first parameter to be processed.

[0122] In a specific example of the disclosed solution, the processing unit 702 is specifically configured to:

[0123] The first parameter to be processed is divided by the first smoothing factor.

[0124] In a specific example of the disclosed solution, the processing unit 702 is further configured to:

[0125] Performing data processing in terms of a second element on a first processing result corresponding to the first processing branch after the quantization processing and a second processing result corresponding to the second processing branch after the quantization processing;

[0126] The third parameter to be processed in the third parameter group after the second element aspect data processing is performed on the third element aspect data processing.

[0127] In a specific example of the disclosed solution, the processing unit 702 is specifically configured to:

[0128] The third parameter to be processed after the second element-wise data processing is multiplied by the first smoothing factor.

[0129] In a specific example of the disclosed solution, it also includes: a prediction unit; wherein,

[0130] The prediction unit is used to input the retrieval data pair to be inferred into the target semantic model, wherein the retrieval data pair to be inferred includes at least the retrieval information to be inferred and the target retrieval result corresponding to the retrieval information to be inferred; and obtain the target reasoning result of the target semantic model, wherein the target reasoning result represents the degree of matching between the retrieval information to be inferred and the target retrieval result.

[0131] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0132] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0133] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0134] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0135] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0136] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0137] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the model processing method. For example, in some embodiments, the model processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the model processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the model processing method in any other appropriate manner (e.g., by means of firmware).

[0138] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0140] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0143] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0144] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0145] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A model processing method, comprising: Inputting a search data pair into a semantic model, the search data pair at least including target search information and a search result corresponding to the target search information; The semantic model includes N parameters to be processed, wherein the parameters to be processed among the N parameters to be processed are parameters required for characterizing the semantic features of the target search information, or are parameters required for characterizing the semantic features of the search results corresponding to the target search information; The parameters to be processed required to characterize the semantic features in the semantic model are subjected to at least the following quantization processing to obtain a target semantic model after parameter quantization processing; wherein the semantic model comprises a first normalization layer for normalization processing, and a first processing branch and a second processing branch provided after the first normalization layer, and the quantization processing comprises: quantizing a first parameter to be processed in a first parameter group processed by the first normalization layer and passing through the first processing branch, and quantizing a second parameter to be processed in a second parameter group processed by the second normalization layer and passing through the second processing branch; the first parameter group comprises at least one first parameter to be processed, which is one of the N parameters to be processed; the second parameter group comprises at least one second parameter to be processed, and the first parameter to be processed and the second parameter to be processed are both one of the N parameters to be processed; The target semantic model is used to extract semantic features of the target retrieval information and the retrieval results corresponding to the target retrieval information in the retrieval data pair, so as to obtain the matching degree between the target retrieval information and the retrieval results corresponding to the target retrieval information.

2. The method according to claim 1, wherein: The at least following quantization processing is performed on the parameters to be processed required to characterize the semantic features in the semantic model, including: Using the target coding unit in the semantic model, the parameters to be processed required to represent the semantic features are quantized at least as follows; Among them, the semantic model includes multiple encoding units connected in series, and the encoding units in the multiple encoding units are used to extract semantic features of target retrieval information and retrieval results corresponding to the target retrieval information in the retrieval data pair; the target encoding unit is one of the multiple encoding units.

3. The method according to claim 1, wherein: The quantizing process of the first parameter to be processed in the first parameter group processed by the first normalization layer and passed through the first processing branch comprises: quantizing a first parameter to be processed in the first parameter group processed by the first normalization layer and by the feedforward neural network layer of the first processing branch; Among them, the feedforward neural network layer is used to extract the semantic features of the target retrieval information in the retrieval data pair processed by the first normalization layer, and / or the semantic features of the retrieval results corresponding to the target retrieval information in the retrieval data pair.

4. The method according to claim 3, wherein: The feedforward neural network layer is at least used to smooth the first target parameter to be processed after being processed by the first normalization layer, and to quantize it after smoothing, and to smooth the acquired second target parameter to be processed, and to quantize it after smoothing.

5. The method according to any one of claims 1 to 4, before quantizing the first parameter to be processed, the method further comprises: Performing first element-wise data processing on the first parameter to be processed to smooth the first parameter to be processed.

6. The method according to claim 5, wherein: The performing first element-wise data processing on the first parameter to be processed includes: The first parameter to be processed is divided by the first smoothing factor.

7. The method according to claim 6, after at least performing the following quantization processing on the parameters to be processed required to characterize the semantic features in the semantic model, the method further comprises: Performing second element-wise data processing on a first processing result corresponding to the first processing branch after the quantization processing and a second processing result corresponding to the second processing branch after the quantization processing, so as to merge data of the first processing branch and the second processing branch; The third parameter to be processed in the third parameter group after the second element aspect data processing is performed with respect to the third element, so as to ensure the equivalence before and after the smoothing processing.

8. The method according to claim 7, wherein: The step of performing third element-wise data processing on the third parameter to be processed after the second element-wise data processing comprises: The third parameter to be processed after the second element-wise data processing is multiplied by the first smoothing factor.

9. The method according to any one of claims 1 to 4, further comprising: Inputting the to-be-reasoned retrieval data pair into the target semantic model, wherein the to-be-reasoned retrieval data pair at least includes the to-be-reasoned retrieval information and the target retrieval result corresponding to the to-be-reasoned retrieval information; A target reasoning result of the target semantic model is obtained, wherein the target reasoning result represents a matching degree between the retrieval information to be reasoned and the target retrieval result.

10. A model processing device, comprising: An input unit, used to input a retrieval data pair into a semantic model, wherein the retrieval data pair includes at least target retrieval information and a retrieval result corresponding to the target retrieval information; the semantic model includes N parameters to be processed, wherein the parameters to be processed among the N parameters to be processed are parameters required for characterizing the semantic features of the target retrieval information, or are parameters required for characterizing the semantic features of the retrieval result corresponding to the target retrieval information; A processing unit, used to perform at least the following quantization processing on the parameters to be processed required to represent the semantic features in the semantic model, so as to obtain a target semantic model after the parameter quantization processing; wherein the semantic model includes a first normalization layer for normalization processing, and a first processing branch and a second processing branch set after the first normalization layer, and the quantization processing includes: quantizing the first parameter to be processed in the first parameter group after the first normalization layer is processed through the first processing branch, and quantizing the second parameter to be processed in the second parameter group after the first normalization layer is processed through the second processing branch; the first parameter group includes at least one first parameter to be processed; the second parameter group includes at least one second parameter to be processed, and the first parameter to be processed and the second parameter to be processed are both one of the N parameters to be processed; wherein the target semantic model is used to extract the semantic features of the target retrieval information and the retrieval results corresponding to the target retrieval information in the retrieval data pair, so as to obtain the matching degree between the target retrieval information and the retrieval results corresponding to the target retrieval information.

11. The device according to claim 10, wherein: The processing unit is specifically used for: Using the target coding unit in the semantic model, the parameters to be processed required to represent the semantic features are quantized at least as follows; Among them, the semantic model includes multiple encoding units connected in series, and the encoding units in the multiple encoding units are used to extract semantic features of target retrieval information and retrieval results corresponding to the target retrieval information in the retrieval data pair; the target encoding unit is one of the multiple encoding units.

12. The device according to claim 10, wherein: The processing unit is specifically used for: quantizing a first parameter to be processed in the first parameter group processed by the first normalization layer and by the feedforward neural network layer of the first processing branch; Among them, the feedforward neural network layer is used to extract the semantic features of the target retrieval information in the retrieval data pair processed by the first normalization layer, and / or the semantic features of the retrieval results corresponding to the target retrieval information in the retrieval data pair.

13. The device according to claim 12, wherein: The feedforward neural network layer is at least used to smooth the first target parameter to be processed after being processed by the first normalization layer, and to quantize it after smoothing, and to smooth the acquired second target parameter to be processed, and to quantize it after smoothing.

14. The device according to any one of claims 10 to 13, wherein: The processing unit is further used for: Performing first element-wise data processing on the first parameter to be processed to smooth the first parameter to be processed.

15. The device according to claim 14, wherein: The processing unit is specifically used for: The first parameter to be processed is divided by the first smoothing factor.

16. The device according to claim 15, wherein: The processing unit is further used for: Performing second element-wise data processing on a first processing result corresponding to the first processing branch after the quantization processing and a second processing result corresponding to the second processing branch after the quantization processing, so as to merge data of the first processing branch and the second processing branch; The third parameter to be processed in the third parameter group after the second element aspect data processing is performed with respect to the third element, so as to ensure the equivalence before and after the smoothing processing.

17. The device according to claim 16, wherein: The processing unit is specifically used for: The third parameter to be processed after the second element-wise data processing is multiplied by the first smoothing factor.

18. The device according to any one of claims 10 to 13, further comprising: Prediction unit; wherein, The prediction unit is used to input the retrieval data pair to be inferred into the target semantic model, wherein the retrieval data pair to be inferred includes at least the retrieval information to be inferred and the target retrieval result corresponding to the retrieval information to be inferred; and obtain the target reasoning result of the target semantic model, wherein the target reasoning result represents the degree of matching between the retrieval information to be inferred and the target retrieval result.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images