A content recommendation method and related device based on quantized network model

Abstract

The present application discloses a model quantization method and related devices, which can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, assisted driving, and in-vehicle scenarios. When performing model training, training data is obtained, including feature data and corresponding labels collected for different sample objects, and the neural network model is trained according to the feature data and the corresponding labels to obtain an original network model including an embedding layer and a network layer. The storage format of the model parameters of the original network model is the original format. The model parameters of the embedding layer are quantized according to the determined target scaling factor to convert the storage format of the model parameters of the embedding layer from the original format to the target format, and then the neural network model composed of the network layer and the quantized embedding layer is determined as the target network model for online launch. This solution only performs post-training quantization on the embedding layer, compresses the model volume without reducing the model indicators, and reduces the impact on the online prediction effect of the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a model quantization method and related devices. Background Art

[0002] With the development of computer technology, neural network models are gaining more and more attention and applications, for example, in information recognition, content recommendation, and object interaction behavior prediction. The size of the model ranges from several GB to several TB, and the larger the model, the higher the upper limit of the model effect. It takes a certain amount of service resources to launch a super-large-scale model. As the model continues to expand, service memory gradually becomes a resource bottleneck for online machines. Model compression is an important means to reduce the size of the model and use less service memory at runtime.

[0003] A common technology for model compression is model quantization. The currently used model quantization technologies include eight-byte integer (INT8) quantization. Compared with the original single-precision floating-point (FP32) model, the INT8 quantized model can reduce the model size by 4 times and the memory bandwidth requirement by 4 times. However, it will cause model indicators such as the area under the curve (AUC) to drop significantly, which will greatly affect the online prediction effect of the model. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a model quantization method and related devices, which can greatly compress the model volume and reduce the impact on the online prediction effect of the model without reducing model indicators such as AUC indicators.

[0005] The embodiments of the present application disclose the following technical solutions:

[0006] On the one hand, an embodiment of the present application provides a model quantization method, the method comprising:

[0007] Acquire training data, wherein the training data includes feature data collected for different sample objects and corresponding labels;

[0008] The constructed neural network model is trained according to the feature data and the corresponding labels to obtain an original network model, wherein the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer;

[0009] determining a target scaling factor, the target scaling factor being used to convert a storage format of the model parameters from the original format to a target format;

[0010] quantizing the model parameters of the embedding layer according to the target scaling factor so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format;

[0011] A neural network model consisting of the network layer and the quantized embedding layer is determined as a target network model, and the target network model is launched.

[0012] On the one hand, an embodiment of the present application provides a model quantization device, the device comprising an acquisition unit, a training unit, a determination unit and a quantization unit:

[0013] The acquisition unit is used to acquire training data, wherein the training data includes feature data collected for different sample objects and corresponding labels;

[0014] The training unit is used to train the constructed neural network model according to the feature data and the corresponding labels to obtain an original network model, the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer;

[0015] The determining unit is used to determine a target scaling factor, wherein the target scaling factor is used to convert a storage format of the model parameters from the original format to a target format;

[0016] The quantization unit is used to quantize the model parameters of the embedding layer according to the target scaling factor, so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format;

[0017] The determination unit is further used to determine the neural network model composed of the network layer and the quantized embedding layer as the target network model, and launch the target network model.

[0018] On the one hand, an embodiment of the present application provides a device for model quantization, the device comprising a processor and a memory:

[0019] The memory is used to store program code and transmit the program code to the processor;

[0020] The processor is used to execute the model quantization method described in the above aspects according to the instructions in the program code.

[0021] On the one hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the model quantization method described in the above aspects.

[0022] On the one hand, an embodiment of the present application provides a computer program product, including a computer program, which implements the model quantization method described in the above aspects when executed by a processor.

[0023] It can be seen from the above technical solution that when performing model training, first obtain the training data, which includes the feature data and corresponding labels collected for different sample objects, and then train the constructed neural network model according to the feature data and the corresponding labels to obtain the original network model. At this time, the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer. Next, in order to compress the model volume, the original network model can be quantized. Since the embedding layer may account for more than 99% of the model parameters in the original network model, and the loss that may be caused by quantization can be compensated by the subsequent network layer, based on this, the present application can only quantize the embedding layer. Specifically, determine the target scaling factor for converting the original format to the target format, and then quantize the model parameters of the embedding layer according to the target scaling factor to convert the storage format of the model parameters of the embedding layer from the original format to the target format, and finally determine the neural network model composed of the network layer and the quantized embedding layer as the target network model, and launch the target network model. Since the embedding layer accounts for more than 99% of the model parameters in the original network model, quantizing the embedding layer can greatly reduce the model size, thereby reducing the deployment cost of ultra-large-scale models and reducing the occupation of service resources. At the same time, since the network layer can compensate for the losses that may be caused by quantization, it is possible to greatly compress the model size without reducing model indicators such as the AUC indicator, thereby reducing the impact on the model's online prediction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technical members in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 An architectural diagram of an application scenario of a model quantization method provided in an embodiment of the present application;

[0026] Figure 2 A flowchart of a model quantization method provided in an embodiment of the present application;

[0027] Figure 3 A structural flow chart of a model quantization method provided in an embodiment of the present application;

[0028] Figure 4A structural diagram of a model quantization device provided in an embodiment of the present application;

[0029] Figure 5 A structural diagram of a terminal provided in an embodiment of the present application;

[0030] Figure 6 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0032] Neural network models are widely used in various fields, such as information recognition, content recommendation, and object interaction behavior prediction. The size of the model ranges from several GB to several TB. The larger the model, the higher the upper limit of the model effect. It takes a certain amount of service resources to launch a super-large-scale model. As the model continues to expand, the service memory gradually becomes the resource bottleneck of the online machine. Model compression is an important means to reduce the size of the model and use less service memory at runtime.

[0033] There are several main methods for model compression, including quantization, pruning, and distillation. They mainly reduce the memory and computing requirements by reducing the parameters and number or number of bits of the original model. Among them, quantization is the most stable model compression method. At present, the quantization method for neural network models mainly includes TFLite (a lightweight inference library) solution.

[0034] The quantization-aware training (QAT) method of the TFLite solution refers to adding model quantization (MQ) only in the forward propagation, that is, linearly quantizing the model parameters, and then dequantizing them into floating-point numbers before performing matrix operations. The specific method is to add pseudo-quantization nodes to the operation nodes that need to be quantized in the neural network model, so that the forward accuracy during training is consistent with the final hardware implementation.

[0035] However, although this method can reduce the size of the model by 4 times and the memory bandwidth requirement by 4 times, quantization-aware training will bring irreversible convergence problems to the model, causing a significant drop in model indicators such as the AUC indicator, which will greatly affect the model's online prediction effect.

[0036] In order to solve the above technical problems, an embodiment of the present application provides a model quantization method, which analyzes the network structure of a neural network model. The neural network model is usually composed of an embedding layer, a hidden layer, and an output layer. Since the number of sparse features can reach hundreds of billions, the embedding layer accounts for more than 99% of the model parameters in the original network model. Since the embedding layer accounts for more than 99% of the model parameters in the original network model, quantizing the embedding layer can greatly reduce the model volume, thereby reducing the deployment cost of ultra-large-scale models and reducing the occupancy of service resources. At the same time, since the network layer can compensate for the losses that may be caused by quantization, it is possible to greatly compress the model volume without reducing model indicators such as the AUC indicator, thereby greatly reducing the impact on the online prediction effect of the model.

[0037] It should be noted that the method provided in the embodiment of the present application can be applied to various scenarios using models, such as information recognition, content recommendation, and object interaction behavior prediction, especially in the content recommendation scenario. The content recommendation scenario is extremely sensitive to online effects. It is not feasible to directly apply quantification technology to the recommendation scenario, which will lead to a decrease in the AUC index, and then bring about a significant negative online prediction effect. For this reason, it is necessary to use the model quantization method provided in the embodiment of the present application to achieve a significant compression of the model volume without reducing model indicators such as the AUC index, and reduce the impact on the online prediction effect of the model.

[0038] Among them, content recommendation scenarios can include online services for all recommendation scenarios, such as advertising, e-commerce shopping, video (or music) recommendations, and news information flow recommendations. This can achieve lossless offline and online effects before and after model quantization, thereby achieving cost reduction and efficiency improvement for the platform when providing services. While reducing online service memory, the real-time delivery of the model can also be improved, thereby increasing user satisfaction with the model. In fact, this technology is applicable to most recommendation and advertising models, and can bring direct service cost savings to the company.

[0039] Next, we will combine Figure 1 The application scenarios of the model quantization method provided in the embodiments of the present application are introduced. Figure 1 A schematic diagram of the architecture of an application scenario of a model quantization method is shown. The application scenario may include a server 101. The server 101 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0040] When performing model training, the server 101 first obtains training data, which includes feature data and corresponding labels collected for different sample objects. In different scenarios, the feature data collected and the corresponding labels are also different. In the content recommendation scenario, the feature data may include sample object features and sample content features, and the labels may include historical feedback information of the sample object on the sample content. In the content recommendation scenario, the historical feedback information may include, for example, click-through rate, conversion rate, rating, etc., and the content recommendation scenario may include, for example, recommendation scenarios such as advertising, video, audio, and news information. For example, in the video recommendation scenario, the sample object may be a sample user, the feature data collected may be sample object features and sample video features, and the label may be the historical rating of the sample object for the sample video. In the content recommendation scenario, the sample object features may include, for example, the age, region, preferences, etc. of the sample object.

[0041] It should be noted that, in the embodiment of the present application, the acquisition of sample object features and other data related to the sample object is authorized by the user.

[0042] In the object recognition scenario, feature data may include sample object features, and labels may include the categories of sample objects. Sample objects may be people, plants, animals, and various objects. For example, if the sample object is a vehicle among various objects, that is, in the vehicle recognition scenario, the collected feature data may be vehicle features, which may include shape, structure, color, and the like.

[0043] Then, the server 101 trains the constructed neural network model according to the feature data and the corresponding labels to obtain the original network model. At this time, the model parameters of the original network model are in the original format, and the original network model includes an embedding layer and a network layer. When training the neural network model, a gradient descent algorithm can be used, for example, by calculating the gradient through a machine operation unit, and then returning the calculated gradient to the parameter service unit (for example Figure 1 The location of the middle network layer + embedding layer, Figure 1 (not shown) so that the model parameters of the network layer and the embedding layer can be adjusted through the parameter service unit, and then the adjusted model parameters are broadcast to the machine computing unit for the next iterative training.

[0044] Next, in order to compress the model volume, the original network model can be quantized. Since the embedding layer may account for more than 99% of the model parameters in the original network model, and the loss caused by quantization can be compensated by the subsequent network layer, based on this, the present application can only quantize the embedding layer. Specifically, the server 101 determines the target scaling factor for converting the original format to the target format, and quantizes the model parameters of the embedding layer according to the target scaling factor to convert the storage format of the model parameters of the embedding layer from the original format to the target format, and finally determines the neural network model composed of the network layer and the quantized embedding layer as the target network model, and launches the target network model.

[0045] Since the embedding layer accounts for more than 99% of the model parameters in the original network model, quantizing the embedding layer can greatly reduce the model size, thereby reducing the deployment cost of ultra-large-scale models and reducing the occupation of service resources. At the same time, since the network layer can compensate for the losses that may be caused by quantization, it is possible to greatly compress the model size without reducing model indicators such as the AUC indicator, thereby reducing the impact on the online prediction effect of the model.

[0046] It should be noted that Figure 1 Taking the server executing the model quantization method as an example, in some cases, the model quantization method can also be executed by the terminal. The terminal can be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc., but is not limited to this. It should be noted that the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, vehicle scenarios, etc.

[0047] It should be noted that the method provided in the embodiments of the present application may involve artificial intelligence (AI), which is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0048] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. For example, the embodiment of the present application can select a suitable target scaling factor to quantize the embedding layer, and greatly compress the model volume without reducing model indicators such as the AUC indicator.

[0049] The embodiments of the present application are particularly related to machine learning, which is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. For example, the embodiments of the present application train a neural network model through machine learning to obtain an original network model.

[0050] It should be noted that the embodiments of the present application may also involve blockchain. For example, the training data used in the embodiments of the present application, such as the feature data and corresponding labels of different sample objects, may be stored on the blockchain.

[0051] Next, the model quantization method provided by the embodiment of the present application will be introduced in detail by taking the server executing the model quantization method as an example in combination with the accompanying drawings.

[0052] See also Figure 2 , Figure 2 A flow chart of a model quantization method is shown, the method comprising:

[0053] S201: Acquire training data, where the training data includes feature data collected for different sample objects and corresponding labels.

[0054] S202. The constructed neural network model is trained according to the feature data and the corresponding labels to obtain an original network model, wherein the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer.

[0055] Quantization technology is usually divided into post-training quantization that does not require training model parameter correction and quantization-aware training that is directly oriented to quantization. Although quantization-aware training has a lower effect loss, it is still worse than the original network model and has convergence problems; post-training quantization refers to the quantization operation performed in the export process after the model training is completed, and only the model parameters that need to be quantized are adjusted. Its goal is usually to make the distribution of the quantized model parameters as close as possible to the data distribution of the original format (such as floating point space). Because it is difficult to configure a calibration data set in many scenarios, especially content recommendation scenarios, in an embodiment of the present application, you can choose to perform model quantization by post-training quantization.

[0056] Among them, the original format is usually a 32-bit floating point format, and model compression mainly reduces the model parameters and quantity or number of bits of the original network model to reduce memory and computing requirements. Therefore, the target format of model quantization can be any data format with fewer bits than the original format, such as INT8 (8-byte integer), INT4 (4-byte integer), INT3 (3-byte integer), INT2 (2-byte integer), and so on.

[0057] When using the post-training quantization method, the server needs to first train to obtain the original network model. Therefore, the server first obtains the training data, which includes the feature data and corresponding labels collected for different sample objects. Among them, the data format of the feature data is the required input format, and the feature data can be divided into sparse feature data and continuous feature data for configuration, and input into the neural network model for training. The infinite training framework is used here. If the original format is FP32 format, the exported original network model can be called the FP32 original network model.

[0058] See also Figure 3 As shown, Figure 3 The structural flow chart of the model quantization is shown. When the server trains the neural network model, it can use the gradient descent algorithm, for example, by calculating the gradient through the machine operation unit, and then returning the calculated gradient to the parameter service unit (for example Figure 3 The location of the middle network layer D + embedded layer E, Figure 3 (not shown) so that the model parameters of the network layer and the embedding layer can be adjusted by the parameter service unit, and then the adjusted model parameters are broadcast to the machine operation unit for the next iterative training. When the iterative training stops, the original network model can be obtained.

[0059] It should be noted that training data from different data sets can be selected according to different application scenarios. For example, in the movie recommendation scenario, the data set corresponding to the training data can be a movie lens (MovieLens) data set. The MovieLens data set is a collection of historical rating records of sample objects (such as users) for movies. It has data volumes of different sizes, named 1M, 10M, and 20M, representing 10, 100,000, and 200,000 ratings, respectively. The 1M data set is used here. The way to divide the training and test data sets is consistent with others. All collected sample files are randomly divided into training set, validation set, and test set in a ratio of 8:1:1. Use the training data in the training set to train the neural network model.

[0060] When training the model, there are no restrictions on the network structure, loss function, and optimizer selection. (1) In terms of network structure, different network structures can be selected according to the application scenario. For example, in the advertising application scenario, click-through rate prediction models such as Factorization Machines (FM) and Deep Neural Networks (DNN) or other multi-objective models can be selected; (2) In terms of loss function, the cross entropy loss function or the mean square error loss function can be used to complete the training of different model optimization objectives; (3) In terms of optimizer, the adaptive (Adam) optimizer, stochastic gradient descent (SGD) optimizer, and adaptive gradient (AdaGrad) optimizer can be used. After training is completed, the original network model stored in the original format (such as FP32 format) is obtained, and its size can range from several GB to several TB.

[0061] It should be noted that the training data obtained in the embodiment of the present application is authorized by the user.

[0062] S203: Determine a target scaling factor, where the target scaling factor is used to convert the storage format of the model parameters from the original format to the target format.

[0063] S204: quantize the model parameters of the embedding layer according to the target scaling factor, so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format.

[0064] Since the size of the original network model can reach several GB to several TB, it requires certain service resources to launch the original network model. Therefore, in order to reduce the occupation of service resources, the volume of the model can be compressed by model quantization. In general, model quantization requires quantizing the model parameters according to a certain scaling factor, so as to convert the model parameters that need to be converted from the original format to the target format and compress the volume of the model. Therefore, the server can first determine the target scaling factor (scale) for converting the original format to the target format.

[0065] By analyzing the network structure of the neural network model, since the number of sparse features can reach hundreds of billions, the embedding layer accounts for more than 99% of the model parameters in the original network model. Since the embedding layer accounts for more than 99% of the model parameters in the original network model, quantizing the embedding layer can greatly reduce the model volume, thereby reducing the deployment cost of ultra-large-scale models and reducing the occupancy of service resources. In addition, by testing the impact of quantization on the model prediction effect, it is found that when the network layer and the embedding layer are quantized at the same time, model indicators such as the AUC indicator have a significant thousandth drop, while only quantizing the embedding layer can align the model indicators at the ten-thousandth. Therefore, the embodiment of the present application chooses to quantize only the embedding layer, that is, after obtaining the target scaling factor, the model parameters of the embedding layer are quantized according to the target scaling factor, and the model parameters of the embedding layer are converted from the original format to the target format.

[0066] The recommended model quantization scheme proposed in the embodiment of the present application can be integrated into the export process of the training framework (such as the quantized framework), directly reducing the model parameter volume of the embedding layer of the original network model, and greatly reducing the deployment cost of ultra-large-scale models while achieving lossless online and offline effects before and after model quantization, thereby achieving cost reduction and efficiency improvement for the platform when providing services.

[0067] It should be noted that, in one possible implementation, all model parameters of the embedding layer can correspond to a target scaling factor. In another possible implementation, the feature data in the training data belongs to N sample objects, the model parameters of the embedding layer are N×K, K is the feature dimension, N and K are positive integers, and each sample object can be used as an index (Key) of the feature data in the embedding layer, that is, one feature data contains N Keys, and there are differences between the Keys due to the frequency of occurrence and other reasons. If only the same scale, such as the target scaling factor, is used to quantize the parameter amount that may reach hundreds of millions, it is easy to cause instability when an abnormal Key occurs. Therefore, the embodiment of the present application adopts a row-wise operation mode to refine the quantization granularity from the feature level to the Key level, and each Key of length E has an exclusive quantization scale, that is, each Key corresponds to a target scaling factor.

[0068] In this case, the target scaling factor can be determined by determining the corresponding target scaling factor for each model parameter corresponding to each sample object according to the original format and the target format, and obtaining N target scaling factors, where the target scaling factors of the model parameters corresponding to the same sample object are the same. Accordingly, the model parameters of the embedding layer are quantized according to the target scaling factors, so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format. The model parameters corresponding to the same sample object in the embedding layer can be used as target model parameters respectively, and the target scaling factors corresponding to the target model parameters are determined. The target model parameters are quantized using the target scaling factors corresponding to the target model parameters, so that the storage format of the target model parameters is converted from the original format to the target format.

[0069] It should be noted that, generally speaking, quantization can be divided into linear asymmetric quantization and linear symmetric quantization according to different methods of the target scaling factor. Taking linear asymmetric quantization as an example, the conversion formula involved in the quantization process can be:

[0070]

[0071] Among them, x int is the model parameter in the target format, x is the model parameter in the original format, x max is the maximum value of the model parameter in the original format, x min is the minimum value of the model parameter in the original format, round() is the rounding function, scale is the scaling factor, x float is the model parameter of the original format obtained after dequantization, bias is the offset, and n is the number of bytes in the target format.

[0072] In some cases, the necessity of using linear asymmetric quantization on the model is considered. Since the model parameters of the embedding layer are symmetrical, the target format (such as 8 bits under INT8 quantization) itself has higher precision. This can also be proved by experiments. When determining the target scaling factor, the two values ​​of linear asymmetric quantization are changed to one value of linear symmetric quantization. Each Key can save 4 bytes of scale space and maintain the quantization accuracy.

[0073] Based on this, in some cases, such as the INT8 quantization scheme, the embedding layer can be quantized in a linear symmetric quantization manner. In this case, the target scaling factor can be determined by determining the maximum target absolute value of the model parameter in the original format, and then determining the target scaling factor based on the maximum target absolute value. Specifically, the maximum target absolute value can be determined as the target scaling factor, or the maximum target absolute value can be multiplied by (2 n-1 -1) is determined as the target scaling factor.

[0074] To compare the maximum absolute value of the target with (2 n-1 -1) as the target scaling factor, the conversion formula involved in the quantization process may be, for example:

[0075]

[0076] x float =scale*x int (4)

[0077]

[0078] Among them, x int are the model parameters in the target format, x are the model parameters in the original format, |x| max is the maximum absolute value of the target of the model parameter in the original format, round() is the rounding function, scale is the scaling factor, x float are the model parameters of the original format obtained after dequantization, and n is the number of bytes in the target format.

[0079] Taking the content recommendation scenario as an example, the original network model can be a content recommendation model. Here, the compression ratio of the quantization function of the content recommendation model is explained. The storage of each Key on Wuliang consists of three parts, namely the index number, the Key index and the weight value. The INT8 quantization method acts on the weight value in a Row-wise manner, and the keys do not affect each other. The size of a single embedded layer Key is N. Taking the original format of FP32 as an example, when exporting, each Key is loaded into the service memory and occupies (N*32+64*2) bits, where the first 64*2 represents the index number and Key index of a single Key; INT8 function, loaded into the service memory is (N*8+32+64*2) bits, where the first 32 represents the scale value taken for linear symmetric quantization of a single Key. The corresponding compression ratio is:

[0080]

[0081] Wherein, rate represents the compression ratio. When N=64, the compression ratio is 31%. When the size of the embedding layer continues to increase, the compression ratio tends to 25%.

[0082] It can be seen that the INT8 quantization method can achieve lossless effects and enable business migration without additional costs; it compresses the model size and further speeds up model export; and it saves an average of about 50% of service memory.

[0083] In the embodiment of the present application, when using the INT8 quantization method, the consistency of the model accuracy of the INT8 format is ensured by only quantizing the model parameters of the embedding layer, wherein the Row-wise method and linear symmetric quantization are adopted when quantizing the model parameters of the embedding layer, and high-proportion compression is achieved under the premise that the model offline and online model indicators are lossless, without increasing additional migration costs.

[0084] Based on the different target formats, the embodiments of the present application can provide a variety of quantization methods. The INT8 quantization has been introduced above. In some cases, the above method can be directly reused in the INT4 quantization method to improve the model compression rate, that is, the target format is INT4 at this time. Since the model compression rate is higher after using the INT4 quantization method, in order to avoid the AUC index having a certain effect decrease in the ten thousandth place due to the increase in the model compression rate, the embodiments of the present application can reduce the effect loss caused by quantization by selecting the best target scaling factor. By visualizing the distribution of model parameters of the embedding layer, it is found that the model parameters of the embedding layer itself are normally symmetrically distributed, but there will be some outlier large values, which may have a certain impact on the model accuracy when selecting the target scaling factor. Therefore, a greedy algorithm is introduced to optimize this part of the loss. Since the target scaling factor can be determined according to the maximum value of the target absolute value, how to select the target scaling factor can be converted into how to select the maximum value of the target absolute value.

[0085] In this case, the maximum value of the target absolute value of the model parameter in the original format can be determined by determining the current scaling factor according to the maximum value of the current absolute value, determining the current quantized model parameters according to the current scaling factor and the model parameters of the embedding layer, and then calculating the current mean square error loss according to the current quantized model parameters and the model parameters of the embedding layer. If the current mean square error loss meets the optimization stop condition, the maximum value of the current absolute value is determined as the maximum value of the target absolute value.

[0086] Here, a mean square error (MSE) loss is introduced to evaluate the degree of deviation of model parameters before and after quantization. The formula of MSE loss is as follows:

[0087]

[0088] Among them, Q() represents the current quantized model parameters, X is the model parameters in the original format, and f(x min , x max ) is the corresponding current MSE loss, equivalent to loss, x max is the maximum value of the model parameters in the original format obtained during the iteration, x min is the maximum value of the model parameter in the original format obtained during the iteration process. Since linear symmetric quantization is used, xmin Can be equal to x max .

[0089] With the goal of reducing the MSE loss, a greedy algorithm is used to optimize the scale value, reducing the model effect loss caused by model quantization. The steps are as follows:

[0090] Initialization parameters, including Xmax, loss and stepsize, where Xmax is the maximum value of the model parameters in the original format, loss represents the MSE loss, and stepsize represents the truncation interval;

[0091] Define the exit condition of the loop judgment, which is related to min_steps. For example, if the difference between the current maximum absolute value and stepsize is greater than min_steps, the loop judgment is executed. min_steps indicates the minimum number of truncation.

[0092] The MSE loss is optimized using a greedy algorithm until the MSE loss meets the optimization stop condition, such as the minimum MSE loss. The maximum value of the current absolute value is determined as the maximum value of the target absolute value, and the optimal scale value is obtained.

[0093] The pseudo code flow is as follows:

[0094] loss = compute_loss(X, Q(X, -Xmax, Xmax)); represents the current mean square error loss calculated after initializing the parameters. Xmax is the maximum value of the current absolute value. When using Q() to calculate the current quantized model parameters, the current scaling factor can be determined first using the maximum value of the current absolute value.

[0095] The maximum value of the current absolute value is obtained by adjusting the maximum value of the initial absolute value according to the preset step size. The preset step size can be expressed as stepsize, stepsize = Xmax / b, b is a constant, for example, b = 200. min_steps = b*(1-r)*stepsize, r is a constant, for example, r = 0.16.

[0096] When it is determined that the current mean square error loss does not meet the optimization stop condition, the current maximum absolute value is adjusted according to the preset step size, and the current mean square error loss is recalculated. Each time the adjustment is made, the preset step size is subtracted from the current maximum absolute value until it is determined that the current mean square error loss does not meet the optimization stop condition, and the current maximum absolute value is determined as the maximum target absolute value.

[0097] The INT4 quantization method can achieve a higher compression rate, and combined with the greedy algorithm to optimize the scale value to reduce the quantization loss, the accuracy loss caused by the INT4 quantization method is reduced by an average of 50%. This greatly compresses the model export volume without reducing the model indicators.

[0098] The embodiments of the present application are suitable for model quantization applied to various application scenarios, such as the quantization of NLP models in natural language processing scenarios and the quantization of content recommendation models in content recommendation scenarios. Since the embodiments of the present application fully consider the limitations of the model compression method provided by the relevant technology in directly applying it to the content recommendation model, the method has superior ability in maintaining the effect of the content recommendation model, improves the accuracy and stability of the model after quantization, and can reduce the model size and thus reduce online memory usage while ensuring that the model's offline and online effects are intact. It is universal in terms of content recommendation models and can provide relatively significant service cost savings and a better user experience.

[0099] S205. Determine the neural network model composed of the network layer and the quantized embedding layer as the target network model, and launch the target network model.

[0100] After quantizing the model parameters of the embedding layer, the neural network model composed of the network layer and the quantized embedding layer can be determined as the target network model, and the target network model can be launched. Figure 3 As shown, the network layer D is not changed, and the embedding layer E is quantized to obtain the quantized embedding layer E'. The quantization of the model parameters of the embedding layer can be achieved by combining the greedy algorithm introduced above.

[0101] After the target network model is online, services can be provided through the target network model. When providing services through the target network model, the model parameters of the embedding layer in the target network model can be dequantized according to the target scaling factor, and the dequantized embedding layer can be used to encode the input data to obtain a feature vector, and then the feature vector is input into the network layer in the target network model for prediction to obtain a prediction result.

[0102] The target network model can provide corresponding services according to different scenarios. If in the content recommendation scenario, the neural network model based on which the training is carried out is a content recommendation model, the trained target network model can be called a target content recommendation model, and the service provided by the target network model can be a content recommendation service, that is, recommending corresponding content to the current object based on the object characteristics of the current object. The content can be video, audio, advertisement, news information, etc. Taking the video as an example, when a user opens the video software, the target content recommendation model can predict the videos that the user may like based on the user's characteristic information, and thus recommend the corresponding video to the user.

[0103] If, in the object recognition scenario, the neural network model used for training is an object recognition model, the target network model obtained through training can be called a target object recognition model, and the service provided by the target network model can be an object recognition service, that is, identifying the category of an object or identifying the location of an object of a specific category. The object can be a person, an animal, a plant, or various objects. For example, when the object is a vehicle among various objects, when the image / video to be recognized is obtained, feature data is extracted from the image / video to be recognized through the target object recognition model, and then the vehicle included therein is recognized based on the feature data.

[0104] It can be seen from the above technical solution that when performing model training, first obtain the training data, which includes the feature data and corresponding labels collected for different sample objects, and then train the constructed neural network model according to the feature data and the corresponding labels to obtain the original network model. At this time, the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer. Next, in order to compress the model volume, the original network model can be quantized. Since the embedding layer may account for more than 99% of the model parameters in the original network model, and the loss that may be caused by quantization can be compensated by the subsequent network layer, based on this, the present application can only quantize the embedding layer. Specifically, determine the target scaling factor for converting the original format to the target format, and then quantize the model parameters of the embedding layer according to the target scaling factor to convert the storage format of the model parameters of the embedding layer from the original format to the target format, and finally determine the neural network model composed of the network layer and the quantized embedding layer as the target network model, and launch the target network model. Since the embedding layer accounts for more than 99% of the model parameters in the original network model, quantizing the embedding layer can greatly reduce the model size, thereby reducing the deployment cost of ultra-large-scale models and reducing the occupation of service resources. At the same time, since the network layer can compensate for the losses that may be caused by quantization, it is possible to greatly compress the model size without reducing model indicators such as the AUC indicator, thereby reducing the impact on the model's online prediction effect.

[0105] The embodiment of the present application also provides a model quantization method, which is mainly applied to the content recommendation scenario. In the content recommendation scenario, the neural network model based on the training is a content recommendation model, and the target network model obtained by training can be called a target content recommendation model. The model quantization method mainly includes the following steps: obtaining the historical feedback information of the sample object on the sample content, constructing the training data according to the historical feedback information of the sample object on the sample content, and the constructed training data includes the sample object features, the sample content features and the historical feedback information. Among them, the sample object features and the sample content features are the feature data collected for different sample objects, and the historical feedback information is the corresponding label. Then, the initial content recommendation model is trained according to the sample object features, the sample content features and the historical feedback information to obtain the original network model, and the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer. Then, the target scaling factor is determined, and the model parameters of the embedding layer are quantized using the target scaling factor, so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format, and finally the neural network model composed of the network layer and the quantized embedding layer is determined as the target content recommendation model, and the target content recommendation model is launched. It should be noted that, based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods.

[0106] based on Figure 2 Corresponding to the model quantization method provided in the embodiment, the embodiment of the present application also provides a model quantization device 400. Figure 4 , the device 400 includes an acquisition unit 401, a training unit 402, a determination unit 403 and a quantization unit 404:

[0107] The acquisition unit 401 is used to acquire training data, wherein the training data includes feature data collected for different sample objects and corresponding labels;

[0108] The training unit 402 is used to train the constructed neural network model according to the feature data and the corresponding labels to obtain an original network model, the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer;

[0109] The determining unit 403 is used to determine a target scaling factor, where the target scaling factor is used to convert the storage format of the model parameters from the original format to the target format;

[0110] The quantization unit 404 is used to quantize the model parameters of the embedding layer according to the target scaling factor, so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format;

[0111] The determining unit 403 is further used to determine the neural network model composed of the network layer and the quantized embedding layer as the target network model, and launch the target network model.

[0112] In a possible implementation, the feature data in the training data belongs to N sample objects, the model parameter of the embedding layer is N×K, K is the feature dimension, N and K are positive integers, and the determining unit 403 is specifically used to:

[0113] For each model parameter corresponding to the sample object, determine a corresponding target scaling factor according to the original format and the target format to obtain N target scaling factors, and the target scaling factors corresponding to the model parameters of the same sample object are the same;

[0114] The quantization unit 404 is specifically used for:

[0115] Using the model parameters corresponding to the same sample object in the embedding layer as target model parameters respectively, and determining the target scaling factors corresponding to the target model parameters;

[0116] The target model parameters are quantized using the target scaling factors corresponding to the target model parameters, so that the storage format of the target model parameters is converted from the original format to the target format.

[0117] In a possible implementation manner, the determining unit 403 is specifically configured to:

[0118] Determining the maximum target absolute value of the model parameter in the original format;

[0119] The target scaling factor is determined according to the maximum value of the target absolute values.

[0120] In a possible implementation, the determining unit 403 is specifically configured to:

[0121] Determine the current scaling factor according to the maximum current absolute value;

[0122] Determine current quantized model parameters according to the current scaling factor and the model parameters of the embedding layer;

[0123] Calculate the current mean square error loss according to the current quantized model parameters and the model parameters of the embedding layer;

[0124] If the current mean square error loss satisfies the optimization stop condition, the maximum value of the current absolute value is determined as the maximum value of the target absolute value.

[0125] In a possible implementation, the maximum value of the current absolute value is obtained by adjusting the maximum value of the initialized absolute value according to a preset step size.

[0126] In a possible implementation, the device further includes a dequantization unit, a coding unit, and a prediction unit:

[0127] The dequantization unit is used to perform dequantization processing on the model parameters of the embedding layer in the target network model according to the target scaling factor;

[0128] The encoding unit is used to perform feature encoding on the input data using the embedding layer after the inverse quantization process to obtain a feature vector;

[0129] The prediction unit is used to input the feature vector into the network layer in the target network model for prediction to obtain a prediction result.

[0130] In a possible implementation, the neural network model is a content recommendation model, the feature data includes sample object features and sample content features, and the label includes feedback information of the sample object on the sample content.

[0131] The device for model quantization provided in the embodiment of the present application may be a terminal, and taking a smart phone as an example:

[0132] Figure 5 The block diagram shows a partial structure of a smart phone provided in an embodiment of the present application. Figure 5 , the smartphone includes: a radio frequency (RF) circuit 510, a memory 520, an input unit 530, a display unit 540, a sensor 550, an audio circuit 560, a wireless fidelity (WiFi) module 570, a processor 580, and a power supply 590. The input unit 530 may include a touch panel 531 and other input devices 532, the display unit 540 may include a display panel 541, and the audio circuit 560 may include a speaker 561 and a microphone 562. It is understood that, Figure 5 The structure of the smartphone shown in the figure does not constitute a limitation of the smartphone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0133] The memory 520 can be used to store software programs and modules. The processor 580 executes various functional applications and data processing of the smartphone by running the software programs and modules stored in the memory 520. The memory 520 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the smartphone (such as audio data, a phone book, etc.), etc. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0134] The processor 580 is the control center of the smartphone, which uses various interfaces and lines to connect various parts of the entire smartphone, and executes various functions of the smartphone and processes data by running or executing software programs and / or modules stored in the memory 520, and calling data stored in the memory 520. Optionally, the processor 580 may include one or more processing units; preferably, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 580.

[0135] In this embodiment, the processor 580 in the smartphone may perform the following steps:

[0136] Acquire training data, wherein the training data includes feature data collected for different sample objects and corresponding labels;

[0137] The constructed neural network model is trained according to the feature data and the corresponding labels to obtain an original network model, wherein the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer;

[0138] determining a target scaling factor, the target scaling factor being used to convert a storage format of the model parameters from the original format to a target format;

[0139] quantizing the model parameters of the embedding layer according to the target scaling factor so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format;

[0140] A neural network model consisting of the network layer and the quantized embedding layer is determined as a target network model, and the target network model is launched.

[0141] The device for model quantization provided in the embodiment of the present application may also be a server, see Figure 6 As shown, Figure 6 The structural diagram of the server 600 provided in the embodiment of the present application, the server 600 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 622 (for example, one or more processors) and a memory 632, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 642 or data 644. Among them, the memory 632 and the storage medium 630 can be short-term storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 622 can be configured to communicate with the storage medium 630 and execute a series of instruction operations in the storage medium 630 on the server 600.

[0142] The server 600 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input and output interfaces 658, and / or one or more operating systems 641, such as Windows Server 2000 or Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0143] In this embodiment, the CPU 622 in the server 600 may perform the following steps:

[0144] Acquire training data, wherein the training data includes feature data collected for different sample objects and corresponding labels;

[0145] The constructed neural network model is trained according to the feature data and the corresponding labels to obtain an original network model, wherein the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer;

[0146] determining a target scaling factor, the target scaling factor being used to convert a storage format of the model parameters from the original format to a target format;

[0147] quantizing the model parameters of the embedding layer according to the target scaling factor so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format;

[0148] A neural network model consisting of the network layer and the quantized embedding layer is determined as a target network model, and the target network model is launched.

[0149] According to one aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the model quantization method described in the aforementioned embodiments.

[0150] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations of the above embodiments.

[0151] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0152] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0153] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0157] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, ordinary technical members in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A content recommendation method based on a quantized network model, characterized in that: The method comprises: Obtain user feature information; Predicting the content that the user is interested in through a target content recommendation model according to the characteristic information of the user, and recommending the content that the user is interested in to the user; The online method of the target content recommendation model includes: Acquire training data, wherein the training data includes feature data collected for different sample objects and corresponding labels; The constructed neural network model is trained according to the feature data and the corresponding labels to obtain an original network model, wherein the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer; determining a target scaling factor, the target scaling factor being used to convert a storage format of the model parameters from the original format to a target format; quantizing the model parameters of the embedding layer according to the target scaling factor so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format; Determine a neural network model consisting of the network layer and the quantized embedding layer as the target content recommendation model, and launch the target content recommendation model; Wherein, determining the target scaling factor includes: Determine the maximum value of the target absolute value of the model parameter in the original format; the maximum value of the target absolute value is the maximum value of the absolute value of the model parameter when the mean square error loss of the model parameter before and after quantization meets the optimization stop condition; The target scaling factor is determined according to the maximum value of the target absolute value, so that the target scaling factor is changed from two values ​​of linear asymmetric quantization to one value of linear symmetric quantization.

2. The method according to claim 1, characterized in that: The feature data in the training data belongs to N sample objects, the model parameter of the embedding layer is N×K, K is the feature dimension, N and K are positive integers, and the determining of the target scaling factor includes: For each model parameter corresponding to the sample object, determine a corresponding target scaling factor according to the original format and the target format to obtain N target scaling factors, where the target scaling factors corresponding to the model parameters of the same sample object are the same; The quantizing the model parameters of the embedding layer according to the target scaling factor so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format includes: Using the model parameters corresponding to the same sample object in the embedding layer as target model parameters respectively, and determining the target scaling factors corresponding to the target model parameters; The target model parameters are quantized using the target scaling factors corresponding to the target model parameters, so that the storage format of the target model parameters is converted from the original format to the target format.

3. The method according to claim 1, characterized in that The determining the maximum target absolute value of the model parameter in the original format comprises: Determine the current scaling factor according to the maximum current absolute value; Determine current quantized model parameters according to the current scaling factor and the model parameters of the embedding layer; Calculate the current mean square error loss according to the current quantized model parameters and the model parameters of the embedding layer; If the current mean square error loss satisfies the optimization stop condition, determining the maximum value of the current absolute value as the maximum value of the target absolute value; The maximum value of the current absolute value is obtained by adjusting according to a preset step size during the iterative calculation of the current mean square error loss based on the maximum value of the initialization absolute value; the maximum value of the initialization absolute value is the maximum value of the absolute value when the model parameters are initialized.

4. The method according to any one of claims 1 to 3, characterized in that: After the target content recommendation model is launched, the method further includes: According to the target scaling factor, dequantizing the model parameters of the embedding layer in the target content recommendation model; The input data is feature encoded using the embedding layer after dequantization to obtain a feature vector; The feature vector is input into the network layer of the target content recommendation model for prediction to obtain a prediction result.

5. The method according to any one of claims 1 to 3, characterized in that: The neural network model is a content recommendation model, the feature data includes sample object features and sample content features, and the label includes historical feedback information of the sample object on the sample content.

6. A content recommendation device based on a quantized network model, characterized in that: The device is used to obtain characteristic information of a user; predict the content that the user is interested in through a target content recommendation model according to the characteristic information of the user, and recommend the content that the user is interested in to the user; the device includes an acquisition unit, a training unit, a determination unit and a quantification unit: The acquisition unit is used to acquire training data, wherein the training data includes feature data collected for different sample objects and corresponding labels; The training unit is used to train the constructed neural network model according to the feature data and the corresponding labels to obtain an original network model, the storage format of the model parameters of the original network model is the original format, and the original network model includes an embedding layer and a network layer; The determining unit is used to determine a target scaling factor, wherein the target scaling factor is used to convert a storage format of the model parameters from the original format to a target format; The quantization unit is used to quantize the model parameters of the embedding layer according to the target scaling factor, so that the storage format of the model parameters of the embedding layer is converted from the original format to the target format; The determining unit is further used to determine the neural network model composed of the network layer and the quantized embedding layer as the target content recommendation model, and launch the target content recommendation model; Wherein, the determining unit is specifically used for: Determine the maximum value of the target absolute value of the model parameter in the original format; the maximum value of the target absolute value is the maximum value of the absolute value of the model parameter when the mean square error loss of the model parameter before and after quantization meets the optimization stop condition; The target scaling factor is determined according to the maximum value of the target absolute value, so that the target scaling factor is changed from two values ​​of linear asymmetric quantization to one value of linear symmetric quantization.

7. The device according to claim 6, characterized in that The feature data in the training data belongs to N sample objects, the model parameter of the embedding layer is N×K, K is the feature dimension, and N and K are positive integers; The determining unit is specifically configured to determine, for each model parameter corresponding to each sample object, a corresponding target scaling factor according to the original format and the target format, to obtain N target scaling factors, wherein the target scaling factors corresponding to the model parameters of the same sample object are the same; The quantization unit is specifically used for: Using the model parameters corresponding to the same sample object in the embedding layer as target model parameters respectively, and determining the target scaling factors corresponding to the target model parameters; The target model parameters are quantized using the target scaling factors corresponding to the target model parameters, so that the storage format of the target model parameters is converted from the original format to the target format.

8. The device according to claim 6, characterized in that The determining unit is specifically configured to: Determine the current scaling factor according to the maximum current absolute value; Determine current quantized model parameters according to the current scaling factor and the model parameters of the embedding layer; Calculate the current mean square error loss according to the current quantized model parameters and the model parameters of the embedding layer; If the current mean square error loss satisfies the optimization stop condition, determining the maximum value of the current absolute value as the maximum value of the target absolute value; The maximum value of the current absolute value is obtained by adjusting according to a preset step size during the iterative calculation of the current mean square error loss based on the maximum value of the initialization absolute value; the maximum value of the initialization absolute value is the maximum value of the absolute value when the model parameters are initialized.

9. The device according to any one of claims 6 to 8, characterized in that: After the target content recommendation model is put online, the device further includes: a dequantization unit, a coding unit and a prediction unit; The dequantization unit is used to perform dequantization processing on the model parameters of the embedding layer in the target content recommendation model according to the target scaling factor; The encoding unit is used to perform feature encoding on the input data using the embedding layer after the inverse quantization process to obtain a feature vector; The prediction unit is used to input the feature vector into the network layer in the target content recommendation model for prediction to obtain a prediction result.

10. The device according to any one of claims 6 to 8, characterized in that: The neural network model is a content recommendation model, the feature data includes sample object features and sample content features, and the label includes historical feedback information of the sample object on the sample content.

11. A device for model quantization, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1 to 5 according to the instructions in the program code.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and when the program codes are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 5.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 5 when being executed by a processor.

Images