Model parameter compression method and device

Abstract

The invention provides a model parameter compression method and device. The method comprises the steps that after a model parameter compression device obtains a first model parameter belonging to an Embedding layer in a first model, the first model parameter is firstly subjected to vector compression, then when the compression rate of the first model parameter is smaller than or equal to a target compression rate, the first model parameter is subjected to domain-level compression, and then the first model parameter is subjected to domain-level compression; and updating each compressed feature domain vector group to an Embedding layer, and training the first model by taking the service data of the recommended service as a training sample. According to the method, two stages of compression are performed on the first model parameter, structured compression of the first model parameter can be realized, manual prior frequency information is not needed, the labor cost is effectively reduced, zero padding operation does not need to be performed on the compressed position of the model parameter in the reasoning stage of the first model, and the compression efficiency is improved. The calculation amount can be effectively reduced, so that the reasoning acceleration of the first model is realized.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model parameter compression method and device. Background Art

[0002] In the Internet age of information overload, recommendation systems are widely used in many fields, such as e-commerce, social media, news information, music or movie playback platforms, etc. However, the models involved in the recommendation system are often accompanied by high-dimensional sparse features, and the parameter scale of the model is extremely large. Among them, the Embedding table parameters (including Embedding vectors corresponding to multiple sparse features) account for more than 90%, and the memory demand is extremely high. At the same time, the application scenarios of the recommendation system require good real-time performance and high latency requirements, so good reasoning performance is needed to meet the low latency requirements. In view of this, how to effectively compress the Embedding table parameters to achieve lossless reasoning acceleration of the model accuracy has become an urgent problem to be solved.

[0003] To address the above issues, the industry has provided the following solutions:

[0004] Solution 1: For each sparse feature, the computing device determines the compression breakpoint of the Embedding vector based on the Embedding vector corresponding to the sparse feature and the prior frequency information of the sparse feature. Afterwards, the Embedding vector is compressed based on the compression breakpoint of the Embedding vector. However, this solution requires the use of artificial prior frequency information, which incurs a large manpower cost. In addition, this solution needs to fill the compressed position of the Embedding vector with 0 during the model inference stage to restore the original dimension of the Embedding vector, which will cause additional performance loss and result in poor model inference acceleration effect.

[0005] Solution 2: The computing device assigns different Embedding dimensions to the eigenvalues ​​of different frequency stages (i.e., the Embedding vectors corresponding to the sparse features) based on artificial a priori frequency information. Afterwards, when training the model, the computing device restores the Embedding dimensions assigned to the eigenvalues ​​to the original dimensions through the conversion module. The computing device then inputs the restored eigenvalues ​​into the dense parameter network for calculation. However, this solution requires the use of artificial a priori frequency information, which incurs a large manpower cost. In addition, since the Embedding vector compression of this solution is a rough semi-structured compression, it is necessary to rely on efficient adaptation of the framework to ensure performance. Summary of the invention

[0006] The present application provides a model parameter compression method and device to effectively reduce the amount of calculation, achieve a better model reasoning acceleration effect, and effectively reduce labor costs.

[0007] In a first aspect, the present application provides a model parameter compression method, which can be performed by a model parameter compression device or a component (such as a chip, a chip system or a circuit, etc.) that can support the model parameter compression device to implement the functions required by the method. Exemplarily, the model parameter compression device can be a service device (such as a computing device) having the functions required to implement the method, or it can also be any other device (such as a vehicle-mounted device, etc.) having the functions required to implement the method, or it can also be a functional element (such as a plug-in, component, chip or circuit, etc.) in a service device having the functions required to implement the method, or it can also be a functional element set in any other device, and the functional element has the function of implementing the model parameter compression method. Optionally, taking the model parameter compression device executing the model parameter compression method as an example, the method may include the following steps:

[0008] After obtaining the first model parameter belonging to the Embedding layer in the first model, the model parameter compression device can perform the following compression processing on the multiple Embedding vectors included in the first model parameter: determine the compression dimension corresponding to each of the multiple Embedding vectors included in the first model parameter; compress each Embedding vector according to the compression dimension corresponding to each Embedding vector; wherein the first model parameter includes multiple feature domain vector groups, any feature domain vector group includes at least one Embedding vector, and then the model parameter compression device can determine the compression rate of the multiple Embedding vectors included in the first model parameter after compression, and when the compression rate is greater than the target compression rate (or it can be said that the compression rate does not reach the target compression rate) , the model parameter compression device can re-compress the multiple Embedding vectors included in the first model parameters until the compression rate is less than or equal to the target compression rate (or it can be called that the compression rate reaches the target compression rate); or when the compression rate is less than or equal to the target compression rate, the model parameter compression device can determine the compression dimension corresponding to each feature domain vector group according to the compression dimension corresponding to at least one Embedding vector included in each feature domain vector group in the multiple feature domain vector groups, and can compress each feature domain vector group according to the compression dimension corresponding to each feature domain vector group, and then, the model parameter compression device can update each feature domain vector group after compression to the Embedding layer of the first model, and can use the business data of the recommended business as a training sample to train the first model.

[0009] In this method, the model parameter compression device can achieve structural compression of the first model parameters by performing two-stage compression on the first model parameters (i.e., one stage is vector-level compression of each Embedding vector included in the first model parameters, and the other stage is domain-level compression of each feature domain vector group included in the first model parameters), without the need for artificial a priori frequency information, so it can effectively reduce the manpower cost caused by artificial a priori frequency information, and can reduce the additional consumption caused by the acquisition of frequency prior information. Furthermore, this method can be implemented in the reasoning stage of the first model (or can be called the use stage or application stage), without the need to perform zero padding operation on the compressed position of the model parameters (such as the first model parameters) of the model layer included in the first model, so it can effectively reduce the amount of calculation, thereby realizing the reasoning acceleration of the first model, so that the acceleration effect of the first model in the reasoning stage is better. In addition, this method does not need to save the position information of the compressed elements, so that the method is easy to deploy and can be applied without framework customization and adaptation.

[0010] In one possible design, a model parameter compression device determines the amount of compression dimension corresponding to each of a plurality of Embedding vectors included in a first model parameter, including: the model parameter compression device may first obtain an importance index value corresponding to the first Embedding vector, wherein the importance index value is used to indicate the importance of a data object corresponding to the first Embedding vector, and the first Embedding vector is any one of the plurality of Embedding vectors; thereafter, the model parameter compression device may determine the amount of compression dimension corresponding to the first Embedding vector based on a relationship between the importance index value corresponding to the first Embedding vector and an importance threshold; wherein the importance threshold may be the importance index value corresponding to the plurality of Embedding vectors, respectively, sorted in order from large to small, and then the importance index value corresponding to the i-th Embedding vector; i may be determined based on a preset coefficient and the total number of the plurality of Embedding vectors.

[0011] In the above design, the model parameter compression device determines the compression dimension amounts corresponding to multiple Embedding vectors respectively according to the importance index values ​​corresponding to the multiple Embedding vectors. The compression dimension amount of each Embedding vector calculated in this way is more accurate and can more accurately reflect the compression degree of each Embedding vector. At the same time, it is also more in line with the actual compression situation of each Embedding vector, thereby realizing refined semi-structured compression of each Embedding vector (or vector-level compression).

[0012] In one possible design, the model parameter compression device obtains the importance index value corresponding to the first Embedding vector, including: the model parameter compression device can first determine the absolute value of the numerical value of the element located at the last position in the first Embedding vector, and then the model parameter compression device can use the absolute value as the importance index value corresponding to the first Embedding vector.

[0013] In the above design, the model parameter compression device uniformly selects the absolute value of the numerical element at the last position in each Embedding vector as the index value reflecting the importance of the Embedding vector, so that each Embedding vector can be effectively semi-structured compressed according to the importance index value determined by a unified standard.

[0014] In a possible design, the model parameter compression device determines the compression dimension corresponding to the first Embedding vector according to the relationship between the importance index value corresponding to the first Embedding vector and the importance threshold. There are several possible implementation methods.

[0015] Method 1: When the importance index value corresponding to the first Embedding vector is greater than or equal to the importance threshold, the model parameter compression device may add the first vector dimension of the first Embedding vector to a preset value to obtain a second vector dimension, wherein the first vector dimension is used to indicate the current remaining dimension of the first Embedding vector. Then, when the second vector dimension is within the set vector dimension range, the model parameter compression device may use the difference between the maximum vector dimension and the second vector dimension within the set vector dimension range as the compressed dimension amount corresponding to the first Embedding vector, or when the second vector dimension is not within the set vector dimension range, the model parameter compression device may adjust the second vector dimension to obtain a third vector dimension, and then, when the third vector dimension is within the set vector dimension range, the model parameter compression device may use the difference between the maximum vector dimension and the third vector dimension as the compressed dimension amount corresponding to the first Embedding vector.

[0016] Method 2: When the importance index value corresponding to the first Embedding vector is less than the importance threshold, the model parameter compression device may subtract the first vector dimension of the first Embedding vector from the preset value to obtain a fourth vector dimension. Then, when the fourth vector dimension is within the set vector dimension range, the model parameter compression device may use the difference between the maximum vector dimension and the fourth vector dimension as the compression dimension amount corresponding to the first Embedding vector, or when the fourth vector dimension is not within the set vector dimension range, the model parameter compression device may adjust the fourth vector dimension to obtain a fifth vector dimension. Then, when the fifth vector dimension is within the set vector dimension range, the model parameter compression device may use the difference between the maximum vector dimension and the fifth vector dimension as the compression dimension amount corresponding to the first Embedding vector.

[0017] In the above design, for each Embedding vector, the model parameter compression device can accurately increase or decrease the dimension of the Embedding vector according to the relationship between the importance index value corresponding to the Embedding vector and the importance threshold, thereby achieving refined semi-structured compression of each Embedding vector.

[0018] In one possible design, the model parameter compression device compresses each Embedding vector separately according to the compression dimension amount corresponding to each Embedding vector, including: for any Embedding vector among the multiple Embedding vectors (such as the second Embedding vector), the model parameter compression device can first determine the number of compressed elements equal to the compression dimension amount from the second Embedding vector according to the compression dimension amount corresponding to the second Embedding vector, and then, the model parameter compression device can adjust the value of the compressed element determined in the second Embedding vector to 0.

[0019] In the above design, the model parameter compression device can achieve the effect of semi-structured compression of each Embedding vector by masking and setting zero to the numerical values ​​of the compressed elements determined in each Embedding vector, and the multiple Embedding vectors after the masking and setting zero to the values ​​can be used as new first model parameters to participate in the subsequent training of the first model until the compression rate of the first model parameters is less than or equal to the target compression rate.

[0020] In one possible design, the model parameter compression device determines the compression rate of multiple Embedding vectors included in the first model parameter after compression, including: the model parameter compression device can first determine the number of at least one non-zero elements included in the multiple Embedding vectors, and then the model parameter compression device can use the ratio of the number of at least one non-zero element to the total number of multiple elements included in the multiple Embedding vectors as the compression rate.

[0021] In the above design, the model parameter compression device can determine whether the compression effect of the first model parameter meets the requirements by calculating the compression rate of the first model parameter, thereby further determining whether to continue compressing the first model parameter or enter the stage of domain-level compression of the first model parameter.

[0022] In one possible design, the model parameter compression device determines the compression dimension amount corresponding to each feature domain vector group according to the compression dimension amount corresponding to at least one Embedding vector included in each feature domain vector group in multiple feature domain vector groups, including: for any feature domain vector group among the multiple feature domain vector groups (such as the first feature domain vector group), the model parameter compression device can first determine the average value or weighted average value of the compression dimension amount corresponding to at least one Embedding vector included in the first feature domain vector group, and then the model parameter compression device can use the average value or weighted average value as the compression dimension amount corresponding to the first feature domain vector group.

[0023] In the above design, for each feature domain vector group, the model parameter compression device determines a unified compression dimension for realizing domain-level compression of the feature domain vector group by comprehensively considering the compression dimension corresponding to each Embedding vector included in the feature domain vector group. In this way, the method can accurately characterize the comprehensive compression degree of the feature domain vector group by taking the average value or weighted average value of the compression dimension corresponding to each Embedding vector included in the feature domain vector group as the compression dimension corresponding to the feature domain vector group. The calculated compression dimension corresponding to the feature domain vector group is also more in line with the actual compression situation of the feature domain vector group, and the Embedding vectors included in the feature domain vector group can be compressed according to the same compression dimension, so that domain-level structured compression (or can be called domain-level compression) for the feature domain vector group can be realized.

[0024] In one possible design, the model parameter compression device compresses each feature domain vector group separately according to the compression dimension amount corresponding to each feature domain vector group, including: for any feature domain vector group among the multiple feature domain vector groups (such as the second feature domain vector group), the model parameter compression device can first determine, according to the compression dimension amount corresponding to the second feature domain vector group, a number of compressed elements equal to the compression dimension amount from each Embedding vector included in the second feature domain vector group, and then, the model parameter compression device can adjust the value of the compressed element determined in each Embedding vector included in the second feature domain vector group to 0.

[0025] In the above design, for each feature domain vector group, the model parameter compression device can achieve the effect of structured compression of the feature domain vector group by performing a mask-to-zero operation on the numerical values ​​of the compressed elements determined in each Embedding vector included in the feature domain vector group, thereby facilitating the trimming of the Embedding vectors included in the feature domain vector group according to the same compression dimension, and can provide effective support for improving the inference speed of the first model.

[0026] In a possible design, after the model parameter compression device uses the business data of the recommended business as training samples to train the first model, when the first model meets the training termination condition, the model parameter compression device can trim the second model parameters of the Embedding layer in the first model according to the compression dimensions corresponding to the multiple feature domain vector groups, to obtain the trimmed second model parameters, wherein the trimmed second model parameters are used as the model parameters used by the first model during inference, and the training termination condition is one of the following: the number of training times of the first model meets the preset number of training times, the prediction accuracy of the first model is greater than or equal to the target accuracy, or the loss function value of the first model is less than or equal to the target loss function value.

[0027] In the above design, the model parameter compression device can achieve effective compression of the model parameters by cutting off the compressed elements of the first model parameters. In this way, during the inference stage of the first model, there is no need to perform zero-padding operations on the compressed positions of the first model parameters, thereby effectively reducing the amount of calculation and achieving inference acceleration of the first model.

[0028] Optionally, after the model parameter compression device trains the first model using the business data of the recommended business as a training sample, when the first model meets the training termination condition, the model parameter compression device can also trim the model parameters of the model layer after the Embedding layer in the first model (such as the dense parameters of the fully connected layer) according to the compression dimension quantities corresponding to the multiple feature domain vector groups, to obtain the trimmed model parameters. In this way, the method can realize the joint compression of the model parameters of the Embedding layer and the model parameters of the model layer after the Embedding layer by trimming the elements at the corresponding positions of the model parameters of the model layer after the Embedding layer according to the compressed positions (i.e. the positions of the compressed elements) corresponding to the multiple feature domain vector groups, thereby more effectively improving the inference speed of the first model.

[0029] In the second aspect, the present application provides a possible model parameter compression device. Optionally, the model parameter compression device may be a service device (such as a computing device) having the functions required to implement the method, or may be any other device (such as a vehicle-mounted device, etc.) having the functions required to implement the method, or may be a functional element (such as a plug-in, component, chip or circuit, etc.) in a service device having the functions required to implement the method, or may be a functional element set in any other device, which has the function of implementing the model parameter compression device method.

[0030] Taking the model parameter compression device as a chip set in a computing device as an example, when the model parameter compression device is a chip, the model parameter compression device includes a transceiver and a processor, but does not include a memory. Among them, the transceiver exists as an input and output interface, and the input and output interface is used for the chip to realize the transceiver of the computing device. The input and output interface may include an input interface and / or an output interface, the input interface can realize the reception of the computing device, and the output interface can be used to realize the sending of the computing device. The processor is used to read and execute the corresponding computer program or instruction, so that the corresponding function of the model parameter compression device is realized.

[0031] When the model parameter compression device is used to implement the method in any possible design of the first aspect, the beneficial effects can be found in the corresponding description of the first aspect, which will not be repeated here. The model parameter compression device has the function of implementing any possible design of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the model parameter compression device includes an acquisition module and a processing module. The acquisition module is used to acquire the first model parameter belonging to the Embedding layer in the first model, wherein the first model parameter includes a plurality of feature domain vector groups, and any feature domain vector group includes at least one Embedding vector. The processing module is used to perform the following compression processing on the plurality of Embedding vectors included in the first model parameter: determine the compression dimension corresponding to each Embedding vector in the plurality of Embedding vectors included in the first model parameter; compress each Embedding vector according to the compression dimension corresponding to each Embedding vector; determine the compression rate of the plurality of Embedding vectors included in the first model parameter after compression. The processing module is also used to, when the compression rate is greater than the target compression rate, re-compress the multiple Embedding vectors included in the first model parameters until the compression rate is less than or equal to the target compression rate; or when the compression rate is less than or equal to the target compression rate, determine the compression dimension corresponding to each feature domain vector group according to the compression dimension corresponding to at least one Embedding vector included in each feature domain vector group in the multiple feature domain vector groups, and perform compression processing on each feature domain vector group according to the compression dimension corresponding to each feature domain vector group. The processing module is also used to update each feature domain vector group after compression processing to the Embedding layer of the first model, and train the first model using the business data of the recommended business as training samples. These modules can perform the corresponding functions in any possible design of the first aspect mentioned above. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0032] In a third aspect, the present application provides a possible computing device, which has the function of implementing the behavior of the model parameter compression device in the method example of the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The computing device includes a memory and a processor. Optionally, the computing device may also include a communication interface. The communication interface is used to communicate with other devices, the memory is used to store computer programs or instructions, and the processor is coupled to the memory and the communication interface. When the processor executes the computer program or instruction, the computing device executes the method in any possible design of the first aspect above.

[0033] In a fourth aspect, the present application provides a possible computing device cluster, which has the function of implementing the behavior of the model parameter compression device in the method example of the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The computing device cluster includes at least one computing device, any computing device includes a processor and a memory, and the processor in any computing device is configured to support the model parameter compression device to perform part or all of the functions of the method in any possible design of the first aspect above. The memory is coupled to the processor, which stores the necessary computer program instructions and data for the model parameter compression device. Optionally, any computing device may also include a communication interface.

[0034] In a fifth aspect, the present application provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method in any possible design of the first aspect above.

[0035] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computing device, enables the computing device to execute a method in any possible design of the first aspect described above.

[0036] In a seventh aspect, the present application further provides a computer chip, which is coupled to a memory, and is used to read a computer program stored in the memory to execute a method in any possible design of the first aspect above.

[0037] In an eighth aspect, the present application further provides a chip system, the chip system comprising a processor for supporting a computer device to implement the method in any possible design of the first aspect. In one possible design, the chip system further comprises a memory, the memory being used to store the necessary programs and data of the computer device. The chip system may be composed of a computer chip, or may include a computer chip and other discrete devices.

[0038] Based on the implementations provided in the above aspects, the present application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a possible application scenario provided by an embodiment of the present application is exemplified;

[0040] Figure 2 A schematic diagram showing a flow chart of a model parameter compression method provided in an embodiment of the present application;

[0041] Figure 3aA schematic diagram showing an example of calculating an importance index value provided by an embodiment of the present application;

[0042] Figure 3b A schematic diagram showing, by way of example, a method of calculating a compression dimension corresponding to a feature domain vector group provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram exemplarily illustrates a flow chart of another model parameter compression method provided in an embodiment of the present application;

[0044] Figure 5 A schematic diagram showing a possible structure of a model parameter compression device provided in an embodiment of the present application is exemplified;

[0045] Figure 6 A schematic diagram showing a possible structure of a computing device provided by an embodiment of the present application is exemplified;

[0046] Figure 7 A schematic diagram showing a possible computing device cluster structure provided by an embodiment of the present application is exemplified;

[0047] Figure 8 A schematic diagram illustrating the structure of another possible computing device cluster provided in an embodiment of the present application is exemplified. DETAILED DESCRIPTION

[0048] Before introducing the technical solution provided by the present application, some of the terms involved in the present application are first explained to facilitate understanding by those skilled in the art.

[0049] (1) Embedding layer: In the embodiment of the present application, sparse features are embedded into word vectors. Specifically, Embedding is a method of using a numerical vector to represent an object. The object here can be a word, an object, a movie, or a song.

[0050] (2) Recommender system: It is an information filtering system that can automatically predict the user's preference for a specific product or service and provide personalized recommendations to the user. It is usually based on the user's historical behavior, personal preferences, interests and preferences, etc., and generates personalized recommendation content with the support of big data through data mining and machine learning algorithms, thereby improving user purchase rate and satisfaction.

[0051] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0052] The following introduces possible application scenarios to which the model parameter compression method provided in this application is applicable. It should be noted that these introductions are for the convenience of understanding by those skilled in the art and do not constitute a limitation on the scope of protection claimed by this application.

[0053] Figure 1 The following is a schematic diagram showing a possible application scenario to which the present application embodiment is applicable. Figure 1 As shown, the application scenario may include a terminal device 100 and a service device 200. Optionally, the terminal device 100 and the service device 200 may be connected to each other through one or more networks. The network may be a wired network or a wireless network, for example, the wireless network may be a wireless fidelity (WIREless-Fidelity, WIFI) network, or a mobile cellular network, or other forms of networks, which are not limited in the embodiments of the present application.

[0054] Among them, a target application for assisting in compressing the model parameters of the model to be compressed is installed on the terminal device 100. Among them, the user can edit and submit the model parameter compression request of the model to be compressed (such as a recommended model in the recommendation system, such as a deep cross network (DCN)) to the service device 200 through the target application. Among them, the model parameter compression request of the model to be compressed may include the model information of the model to be compressed, such as the initial model parameters (such as the initial Embedding parameters or dense parameters, etc.), the initial model or the training sample data set, etc., or may also include the model parameters before each round (or each time) of training before the model to be compressed meets the training termination condition, such as the Embedding parameters or dense parameters of the model to be compressed before each round of training. Exemplarily, the target application can be a client, a web application, or a small program embedded in other applications. Optionally, the terminal device 100 may include a display for presenting a configuration interface provided by the service device 200 (such as a configuration interface for assisting users to upload model information of the model to be compressed) to achieve human-computer interaction. The configuration interface displays information edited by the user or information provided to the user.

[0055] Exemplarily, the terminal device 100 may be a smart phone, a tablet computer, a desktop computer, a computer with wireless transceiver function (such as a laptop computer, etc.), a PPD (Personal Digital Assistant), a mobile internet device (MID), a vehicle terminal, a wearable device with wireless communication function (such as a smart watch, a smart bracelet, smart glasses or a smart helmet, etc.), a virtual reality (VR) device, an augmented reality (AR) device or a smart home device (such as a smart speaker or a smart TV, etc.), etc. It should be understood that the present application does not limit the specific device form of the terminal device.

[0056] The service device 200 is a device (such as a server or computing device) used to provide services for users to perform corresponding operations on the target application installed on the terminal device 100. For example, the service device 200 can be a model parameter compression system, or it can also be a computing device (such as a computer), a computing device cluster or other arbitrary device with a model parameter compression function, or it can also be a functional element (such as a model parameter compression device or a chip with a model parameter compression function, etc.) set in a computing device, a computing device cluster or other arbitrary device. Exemplarily, the model parameter compression device can be a chip or a circuit, etc.; the model parameter compression system can be an independent physical server, or it can also be a server cluster or a distributed system composed of multiple physical servers, or it can also be a cloud server for providing cloud services, cloud computing, cloud storage, cloud communications, network services, security services, big data and other cloud computing services.

[0057] For example, the terminal device 100 is a laptop computer, the target application is a web browser installed on the laptop computer for assisting users to upload model information of the model to be compressed, and the service device 200 is a computing device with a model parameter compression function. When a user (such as user A) needs to compress the model information of a model to be compressed (such as model A), user A can log in to the configuration interface provided by the service device 200 for assisting users to upload model information of the model to be compressed through the web browser installed on the laptop computer used, and can edit the model information of model A on the configuration interface. After that, after user A has edited the model information of model A, a model parameter compression request for model A can be generated according to the edited model information of model A. When the computing device obtains the model parameter compression request for model A uploaded by user A through the web browser, the model parameter compression process for model A can be executed.

[0058] Optionally, the model parameter compression method provided in the embodiment of the present application can be applied to the recommendation system model reasoning acceleration scenario, and can also be applied to any model reasoning scenario using Embedding technology (such as natural language processing, large models, etc.).

[0059] It should be noted that Figure 1 Only one possible application scenario is provided schematically, and this schematic application scenario is intended to more clearly illustrate the technical solution of the embodiment of the present application, and does not limit the application scenario of the model parameter compression method provided in the present application. Figure 1 The form and quantity of each device in the application scenario shown are only for example and do not constitute a limitation of the present application. Figure 1 The names of the various devices in the application scenarios shown are only examples, and the names of the various devices in the specific implementation may also be other names, which are not specifically limited in this application. In addition, it is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0060] The following is based on Figure 1 The application scenario shown is used to introduce the specific implementation of the model parameter compression method in the embodiment of the present application in detail.

[0061] Figure 2 A flow chart of a model parameter compression method provided in an embodiment of the present application is exemplarily shown. The method can be but is not limited to being applicable to Figure 1 The method flow may be executed by a model parameter compression device or a component (such as a chip, a chip system or a circuit, etc.) that can support the model parameter compression device to implement the functions required by the method. For example, the model parameter compression device may be Figure 1 The service device 200 shown in FIG. 200 (such as a computing device with a model parameter compression function or a computing device cluster with a model parameter compression function, etc.), or it may also be Figure 1 The functional element (such as a plug-in, component, chip or circuit, etc.) in the service device 200 shown in the figure has the function of implementing the model parameter compression method. For example, taking the service device 200 as a computing device with a model parameter compression function, the model parameter compression device can be the computing device with the model parameter compression function, or it can also be an independent unit embedded in the computing device for implementing the model parameter compression function, or it can also be a functional component (such as a chip) encapsulated in the computing device for implementing the model parameter compression function. In order to facilitate the introduction of the technical solution provided in the embodiment of the present application, the following is an introduction taking the model parameter compression method executed by the model parameter compression device as an example. Figure 2 As shown, the method includes:

[0062] Step 201: The model parameter compression device obtains a first model parameter belonging to an Embedding layer in a first model.

[0063] Exemplarily, the first model parameters may be initial model parameters of the first model, such as initial Embedding parameters belonging to the Embedding layer in the first model or model parameters of other model layers located after the Embedding layer (such as dense parameters), or may also be model parameters of the first model before any round of training starts before the training termination condition is met (such as Embedding parameters or dense parameters). Optionally, the first model parameters may also be model parameters of the first model after any round of training is completed before the training termination condition is met (such as Embedding parameters or dense parameters). For example, when the model parameter compression device obtains the model parameters of the first model after a certain round of training, the implementation method of compressing the model parameters of the first model after the end of the round of training can refer to the compression processing method provided in the following steps 202 to 206, which will not be repeated here.

[0064] For example, taking the first model parameter as the Embedding parameter of the first model before any round of training starts before the training termination condition is met, the first model parameter may include multiple feature domain vector groups, and any feature domain vector group may include at least one Embedding vector. Exemplarily, taking the first model as a product recommendation model as an example, it is assumed that the first model parameter includes 2 feature domain vector groups, such as one feature domain vector group is a user domain vector group, and the other feature domain vector group is a product domain vector group. Among them, the user domain vector group may include Embedding vectors corresponding to multiple users (such as Embedding vectors corresponding to user ID1, Embedding vectors corresponding to user ID2, Embedding vectors corresponding to user ID3, etc.); the product domain vector group may include Embedding vectors corresponding to multiple products (such as Embedding vectors corresponding to product ID1, Embedding vectors corresponding to product ID2, Embedding vectors corresponding to product ID3, etc.).

[0065] In one example, the first model parameter may be sent by a user to the model parameter compression device through a terminal device. For example, before each round of training of the first model starts, the user may send a model parameter compression request to the model parameter compression device through a terminal device. The model parameter compression request is used to request compression of the first model parameters of the first model before the start of the round of training. Exemplarily, the model parameter compression request may include the first model parameters (such as Embedding parameters) of the first model before each round of training starts. In another example, the first model parameter may also be obtained by the model parameter compression device from a device for training the first model (such as a computing device for training the first model or a functional element for training the first model).

[0066] After the model parameter compression device obtains the first model parameters (such as Embedding parameters) of the first model before the start of the i-th round (or i-th time) of training, the first model parameters of the first model before the start of the i-th round of training can be compressed (or referred to as vector-level compression processing, or vector-level semi-structured compression processing) through the following steps (i.e., steps 202 to 203). When the compression rate of the model parameters of the first model before the start of the i-th round of training is less than or equal to the target compression rate, the model parameter compression device can compress the first model parameters of the first model through the following steps (i.e., steps 205 to 206) (or referred to as domain-level compression processing, or referred to as domain-level structured compression processing).

[0067] When the compression rate of the first model parameters of the first model before the start of the i-th round of training is greater than the target compression rate, the first model is trained for the i+1th round according to the compressed first model parameters and the training sample data set, and the first model parameters of the first model before the start of the i+1th round of training are once again subjected to vector-level compression processing (or can be called ID-level compression processing, or can be called ID-level semi-structured compression processing) through the following steps (i.e., steps 202 to 203) until the compression rate of the first model parameters of the first model is less than or equal to the target compression rate. When the compression rate of the first model parameters of the first model before a certain round of training is less than or equal to the target compression rate, the model parameter compression device can perform domain-level compression processing on the first model parameters of the first model through the following steps (i.e., steps 205 to 206). Exemplarily, when the first model is a product recommendation model, the training sample data set may include but is not limited to product data, user data, location data of the user's area, location data of the product's area, weather data of the user's area, data of each user purchasing one or more products, or user's interests and hobbies, etc.

[0068] Step 202: The model parameter compression device determines the compression dimension corresponding to each Embedding vector in the multiple Embedding vectors included in the first model parameter.

[0069] For any Embedding vector (such as the first Embedding vector) among the multiple Embedding vectors, after the model parameter compression device obtains the importance index value corresponding to the first Embedding vector, it can determine the compression dimension corresponding to the first Embedding vector according to the size relationship between the importance index value corresponding to the first Embedding vector and the importance threshold. Among them, the importance index value corresponding to the first Embedding vector can be used to indicate the importance of the data object corresponding to the first Embedding vector; the importance threshold is the importance index value corresponding to the Embedding vector located at the i-th position after the importance index values ​​corresponding to the multiple Embedding vectors are sorted in order from large to small; i is determined according to the preset coefficient and the total number of multiple Embedding vectors.

[0070] Optionally, the model parameter compression device may maintain a record table tensor_vector, which is used to record the real-time compression dimension corresponding to each Embedding vector. For example, taking the first Embedding vector as an example, the model parameter compression device may record the compression dimension corresponding to the first Embedding vector in each compression process in the record table tensor_vector.

[0071] Based on the above content, the following describes the implementation process of the model parameter compression device obtaining the importance index value corresponding to the first Embedding vector through the following possible examples.

[0072] Example 1: The model parameter compression device may first query the value of the element at the end position (such as the last dimension position, or the last column) in the first Embedding vector. Afterwards, the model parameter compression device may use the absolute value of the value of the element as the importance index value corresponding to the first Embedding vector.

[0073] Figure 3a The following is a schematic diagram showing a method for calculating an importance index value provided by an embodiment of the present application. Figure 3aIn the example, after obtaining the tables of sizes V1*DIM, V2*DIM, ..., VN*DIM corresponding to the multiple feature domains of the first model, the model parameter compression device splices the tables corresponding to the multiple feature domains to obtain the total table Total_Table. Among them, V1 is used to represent the number of vectors included in the table, and DIM is used to represent the dimension of each vector in the table. Among them, the table of size V1*DIM is table1 corresponding to feature domain 1, the table of size V2*DIM is table2 corresponding to feature domain 2, ..., and the table of size VN*DIM is tableN corresponding to feature domain N. Afterwards, the model parameter compression device can select the absolute value of the current last element of each Embedding vector in Total_Table as the importance index value corresponding to the Embedding vector. Then, after calculating the importance index value corresponding to each Embedding vector, the model parameter compression device splices the importance index values ​​corresponding to the multiple Embedding vectors included in Total_Table into an importance tensor.

[0074] Optionally, after calculating the importance index value corresponding to each Embedding vector, the model parameter compression device can sort the importance index values ​​corresponding to the multiple Embedding vectors included in Total_Table in descending order. Afterwards, the model parameter compression device can select the importance index value corresponding to the Embedding vector ranked at the kth * total number (i.e., the total number of multiple Embedding vectors included in Total_Table) as the importance threshold. Among them, the preset coefficient k can be flexibly selected according to actual needs, 0<k<1.

[0075] Example 2: The model parameter compression device may first query the values ​​of the elements located in the last few dimensions in the first Embedding vector (such as the elements in the last 2 dimensions or the elements in the last 2 or more dimensions). Afterwards, the model parameter compression device may use the average absolute value or weighted average absolute value of the values ​​of the elements in the last few dimensions as the importance index value corresponding to the first Embedding vector.

[0076] In addition, the following describes the implementation process of the model parameter compression device determining the compression dimension corresponding to the first Embedding vector through the following possible implementation methods.

[0077] Implementation method 1: When the importance index value corresponding to the first Embedding vector is greater than or equal to the importance threshold, the model parameter compression device can add the first vector dimension of the first Embedding vector to a preset value (such as value 1) to obtain a second vector dimension. Afterwards, the model parameter compression device can determine the compression dimension amount corresponding to the first Embedding vector based on the second vector dimension and the set vector dimension range.

[0078] The first vector dimension is used to indicate the current remaining dimension of the first Embedding vector.

[0079] Exemplarily, taking the original dimension of the first Embedding vector as 64 as an example, if the first Embedding vector is not compressed, the first vector dimension of the first Embedding vector (i.e., the current remaining dimension) is 64. If the first Embedding vector is compressed, the first vector dimension of the first Embedding vector is the current remaining dimension after compression, for example, the current remaining dimension after compression is 61. For example, taking the first vector dimension of the first Embedding vector as 61, the importance index value corresponding to the first Embedding vector as 20, the importance threshold as 18, and the preset value as 1 as an example, when the importance index value 20 corresponding to the first Embedding vector is greater than the importance threshold as 18, the model parameter compression device may add the first vector dimension 61 and the preset value 1 to obtain a second vector dimension of 62. For another example, taking the first vector dimension of the first Embedding vector as 64, the importance index value corresponding to the first Embedding vector as 20, the importance threshold as 18, and the preset value as 1, when the importance index value 20 corresponding to the first Embedding vector is greater than the importance threshold as 18, the model parameter compression device can add the first vector dimension 64 and the preset value 1 to obtain a second vector dimension of 65.

[0080] In one example, when the second vector dimension is within the set vector dimension range, the model parameter compression device may use the difference between the maximum vector dimension and the second vector dimension within the set vector dimension range as the compression dimension quantity corresponding to the first Embedding vector. For example, taking the set vector dimension range as (1,64) and the second vector dimension as 62, the model parameter compression device may use the difference between the maximum vector dimension 64 and the second vector dimension 62 as the compression dimension quantity corresponding to the first Embedding vector. In other words, the value 2 is the number of elements in the first Embedding vector that need to be masked to zero.

[0081] In another example, when the second vector dimension is not within the set vector dimension range, the model parameter compression device may adjust the second vector dimension to obtain a third vector dimension. Thereafter, when the third vector dimension is within the set vector dimension range, the model parameter compression device may use the difference between the maximum vector dimension and the third vector dimension as the compression dimension amount corresponding to the first Embedding vector. For example, taking the set vector dimension range as (1,64) and the second vector dimension as 65 as an example, when the second vector dimension 65 exceeds the set vector dimension range, the model parameter compression device may adjust or modify the second vector dimension 65 to 64 to limit the second vector dimension within the set vector dimension range, wherein the value 64 is the third vector dimension. Thereafter, the model parameter compression device may use the difference between the maximum vector dimension 64 and the third vector dimension 64 as 0 as the compression dimension amount corresponding to the first Embedding vector.

[0082] Implementation method 2: When the importance index value corresponding to the first Embedding vector is less than the importance threshold, the model parameter compression device can perform a difference process between the first vector dimension of the first Embedding vector and a preset value (such as value 1) to obtain a fourth vector dimension. Afterwards, the model parameter compression device can determine the compression dimension amount corresponding to the first Embedding vector based on the fourth vector dimension and the set vector dimension range.

[0083] For example, taking the first vector dimension of the first Embedding vector as 62, the importance index value corresponding to the first Embedding vector as 15, the importance threshold as 18, and the preset value as 1 as an example, when the importance index value 15 corresponding to the first Embedding vector is less than the importance threshold as 18, the model parameter compression device may perform a difference processing between the first vector dimension 62 and the preset value as 1, and obtain a fourth vector dimension of 61. For another example, taking the first vector dimension of the first Embedding vector as 60, the importance index value corresponding to the first Embedding vector as 15, the importance threshold as 18, and the preset value as 1 as an example, when the importance index value 15 corresponding to the first Embedding vector is less than the importance threshold as 18, the model parameter compression device may perform a difference processing between the first vector dimension 60 and the preset value as 1, and obtain a fourth vector dimension of 59.

[0084] For another example, taking the first vector dimension of the first Embedding vector as 1, the importance index value corresponding to the first Embedding vector as 15, the importance threshold as 18, and the preset value as 1, when the importance index value 15 corresponding to the first Embedding vector is less than the importance threshold as 18, the model parameter compression device may perform a difference processing on the first vector dimension 1 and the preset value 1 to obtain a fourth vector dimension of 0.

[0085] In one example, when the fourth vector dimension is within a set vector dimension range, the model parameter compression device may use the difference between the maximum vector dimension and the fourth vector dimension as the compression dimension quantity corresponding to the first Embedding vector.

[0086] For example, taking the vector dimension range as (1, 64) and the fourth vector dimension as 61, the model parameter compression device can use the difference 3 between the maximum vector dimension 64 and the fourth vector dimension 61 as the compression dimension corresponding to the first Embedding vector.

[0087] In another example, when the fourth vector dimension is not within the set vector dimension range, the model parameter compression device may adjust the fourth vector dimension to obtain a fifth vector dimension. Thereafter, when the fifth vector dimension is within the set vector dimension range, the model parameter compression device may use the difference between the maximum vector dimension and the fifth vector dimension as the compressed dimension amount corresponding to the first Embedding vector.

[0088] For example, taking the setting vector dimension range as (1,64) and the fourth vector dimension as 0, when the fourth vector dimension 0 exceeds the set vector dimension range, the model parameter compression device can adjust or modify the fourth vector dimension 0 to 1, and the value 1 is the fifth vector dimension, so as to limit the fourth vector dimension to the set vector dimension range. Afterwards, the model parameter compression device can use the difference 63 between the maximum vector dimension 64 and the fifth vector dimension 1 as the compression dimension amount corresponding to the first Embedding vector.

[0089] Step 203: The model parameter compression device compresses each Embedding vector according to the compression dimension corresponding to each Embedding vector.

[0090] After determining the compression dimension amount corresponding to each Embedding vector in the multiple Embedding vectors, the model parameter compression device can determine the number of compressed elements equal to the compression dimension amount from the second Embedding vector for any Embedding vector in the multiple Embedding vectors (such as the second Embedding vector) according to the compression dimension amount corresponding to the second Embedding vector. Afterwards, the model parameter compression device can perform a mask-to-zero operation on the compressed elements determined in the second Embedding vector, for example, the model parameter compression device can adjust the values ​​of the compressed elements equal to the compression dimension amount in the second Embedding vector to 0.

[0091] Exemplarily, take the first model parameter including 5 Embedding vectors (such as Embedding vector 1, Embedding vector 2, Embedding vector 3, Embedding vector 4 and Embedding vector 5) as an example. Among them, Embedding vector 1, Embedding vector 2, Embedding vector 3 belong to characteristic vector group a, and Embedding vector 4 and Embedding vector 5 belong to characteristic domain vector group b. Assume that the original dimension (or initial dimension) of Embedding vector 1, Embedding vector 2, Embedding vector 3, Embedding vector 4 and Embedding vector 5 is 64, and assume that the compressed dimension corresponding to Embedding vector 1 is 15, the compressed dimension corresponding to Embedding vector 2 is 25, the compressed dimension corresponding to Embedding vector 3 is 35, the compressed dimension corresponding to Embedding vector 4 is 40, and the compressed dimension corresponding to Embedding vector 5 is 50.

[0092] For Embedding vector 1, the model parameter compression device may select the elements of the last 15 dimensional positions (such as the last 15 columns) in Embedding vector 1 as compressed elements. Thereafter, the model parameter compression device may adjust the values ​​of the compressed elements of the 15 dimensional positions to 0. For Embedding vector 2, the model parameter compression device may select the elements of the last 25 dimensional positions (such as the last 25 columns) in Embedding vector 2 as compressed elements. Thereafter, the model parameter compression device may adjust the values ​​of the compressed elements of the 25 dimensional positions to 0. For Embedding vector 3, the model parameter compression device may select the elements of the last 35 dimensional positions (such as the last 35 columns) in Embedding vector 3 as compressed elements. Thereafter, the model parameter compression device may adjust the values ​​of the compressed elements of the 35 dimensional positions to 0. For Embedding vector 4, the model parameter compression device may select the elements of the last 40 dimensional positions (such as the last 40 columns) in Embedding vector 4 as compressed elements. Afterwards, the model parameter compression device may adjust the values ​​of the compressed elements in the 40 dimensional positions to 0. For Embedding vector 5, the model parameter compression device may select the elements in the last 50 dimensional positions (such as the last 50 columns) in Embedding vector 5 as compressed elements. Afterwards, the model parameter compression device may adjust the values ​​of the compressed elements in the 50 dimensional positions to 0.

[0093] Based on the above content, the implementation process of the model parameter compression device performing the mask zeroing operation on the compressed elements determined in the second Embedding vector is introduced through the following possible implementation methods.

[0094] Implementation method 1: The model parameter compression device adjusts (or can be called modifying) the numerical value of one or more compressed elements determined in the second Embedding vector to 0.

[0095] For example, taking the second Embedding vector as [1,1,1,1,1], and there are compressed elements in 2 dimensional positions in the second Embedding vector, the model parameter compression device can use the elements in the last 2 dimensional positions (i.e., the last 2 columns) in the second Embedding vector [1,1,1,1,1] as compressed elements, and adjust the values ​​of the compressed elements in the 2 dimensional positions to 0. In this way, the second Embedding vector [1,1,1,1,1] becomes [1,1,1,0,0].

[0096] Implementation method 2: The model parameter compression device sets a preset Embedding vector. The dimension of the preset Embedding vector is the same as the initial dimension of the second Embedding vector, and the value of the element in the preset Embedding vector corresponding to the dimensional position of the compressed element included in the second Embedding vector is set to 0. Afterwards, the model parameter compression device can multiply the second Embedding vector with the preset Embedding vector to obtain a third Embedding vector, thereby realizing a mask zeroing operation to hide (or cover) one or more compressed elements in the second Embedding vector. The third Embedding vector is the second Embedding vector after the mask zeroing operation.

[0097] For example, let's continue with the example where the second Embedding vector is [1,1,1,1,1] and there are 2 compressed elements in the second Embedding vector. The model parameter compression device first sets a 5*5 preset Embedding vector matrix Afterwards, the model parameter compression device can multiply the second Embedding vector [1,1,1,1,1] with the preset Embedding vector matrix to obtain a third Embedding vector [1,1,1,0,0].

[0098] Step 204: The model parameter compression device determines the compression ratio of the multiple Embedding vectors included in the first model parameter after compression.

[0099] After compressing the multiple Embedding vectors respectively, the model parameter compression device can calculate the first number of at least one non-zero element included in the multiple Embedding vectors, and can calculate the total number of multiple elements included in the multiple Embedding vectors. Afterwards, the model parameter compression device divides the first number by the total number to obtain a ratio, which can be used as the compression rate of the first model parameter.

[0100] Optionally, after determining the compression rate of the first model parameter, the model parameter compression device may compare the compression rate with the target compression rate. When the compression rate is less than or equal to the target compression rate, the model parameter compression device determines that the first model parameter can be compressed at the next stage of domain level. When the compression rate is greater than the target compression rate, the model parameter compression device determines that the first model parameter still needs to undergo one or more vector level compressions.

[0101] When the compression rate of the first model parameters is greater than the target compression rate, the model parameter compression device updates the first model parameters after the vector-level compression processing to the Embedding layer of the first model, and uses the business data of the recommended business as a training sample to perform one or more subsequent rounds of training on the first model. Afterwards, the model parameter compression device executes the above steps 202 to 203 according to the first model parameters of the first model before the start of a subsequent round of training, until the compression rate of the first model parameters is less than or equal to the target compression rate. In one example, when the compression rate of the first model parameters is greater than the target compression rate, the model parameter compression device can execute the above steps 202 to 203 on the first model parameters of the first model before the start of each subsequent round of training, until the compression rate of the first model parameters is less than or equal to the target compression rate. In another example, when the compression rate of the first model parameters is greater than the target compression rate, the model parameter compression device may select the first model parameters of the first model after multiple rounds (such as every 2 rounds, every 3 rounds, or every 3 rounds or more) of training at each interval (or can be called after), and execute the above steps 202 to 203 until the compression rate of the first model parameters is less than or equal to the target compression rate.

[0102] When the compression rate of the first model parameter is less than or equal to the target compression rate, the model parameter compression device executes the following steps 205 to 207.

[0103] Exemplarily, let's continue to take the example that the first model parameter includes 5 Embedding vectors (such as Embedding vector 1, Embedding vector 2, Embedding vector 3, Embedding vector 4 and Embedding vector 5). Among them, Embedding vector 1, Embedding vector 2, Embedding vector 3 belong to characteristic vector group a, and Embedding vector 4 and Embedding vector 5 belong to characteristic domain vector group b. Assume that the total number of elements contained in Embedding vector 1, Embedding vector 2, Embedding vector 3, Embedding vector 4 and Embedding vector 5 is 64, and assume that the compressed dimension corresponding to Embedding vector 1 is 15, the compressed dimension corresponding to Embedding vector 2 is 25, the compressed dimension corresponding to Embedding vector 3 is 35, the compressed dimension corresponding to Embedding vector 4 is 40, and the compressed dimension corresponding to Embedding vector 5 is 50.

[0104] For Embedding vector 1, the model parameter compression device can determine that there are 15 elements with a value of 0 in Embedding vector 1 according to the compression dimension 15 corresponding to Embedding vector 1. For Embedding vector 2, the model parameter compression device can determine that there are 25 elements with a value of 0 in Embedding vector 2 according to the compression dimension 25 corresponding to Embedding vector 2. For Embedding vector 3, the model parameter compression device can determine that there are 35 elements with a value of 0 in Embedding vector 3 according to the compression dimension 35 corresponding to Embedding vector 3. For Embedding vector 4, the model parameter compression device can determine that there are 40 elements with a value of 0 in Embedding vector 4 according to the compression dimension 40 corresponding to Embedding vector 4. For Embedding vector 5, the model parameter compression device can determine that there are 50 elements with a value of 0 in Embedding vector 5 according to the compression dimension 50 corresponding to Embedding vector 5.

[0105] In one example, the model parameter compression device can calculate the number of elements with a value of 0 in the five Embedding vectors, that is, the number of elements with a value of 0 = (15 + 25 + 35 + 40 + 50) = 165, and can calculate the total number of elements included in the five Embedding vectors as (64 + 64 + 64 + 64 + 64) = 320. Afterwards, the model parameter compression device can calculate the compression rate of the first model parameter according to the number of elements with a value of 0 in the five Embedding vectors and the total number of elements included in the five Embedding vectors, that is, compression rate = (320-165) / 320 = 0.48.

[0106] In another example, the model parameter compression device can calculate the number of elements whose values ​​are not 0 in the three Embedding vectors, that is, the number of elements whose values ​​are not 0 is (64-15+64-25+64-35+64-40+64-50)=155, and can calculate the total number of elements included in the three Embedding vectors as (64+64+64+64+64)=320. Afterwards, the model parameter compression device can calculate the compression rate of the first model parameter according to the number of elements whose values ​​are not 0 in the five Embedding vectors and the total number of elements included in the five Embedding vectors, that is, compression rate=155 / 320=0.48.

[0107] For example, taking the target compression rate as 0.5, after calculating the compression rate of the first model parameter as 0.48, the model parameter compression device can compare the compression rate of the first model parameter as 0.48 with the target compression rate as 0.5. When determining that the compression rate as 0.48 is less than the target compression rate as 0.5, the model parameter compression device can perform the following steps 205 to 207.

[0108] Step 205: When the compression rate of the first model parameter is less than or equal to the target compression rate, the model parameter compression device determines the compression dimension corresponding to each feature domain vector group according to the compression dimension corresponding to at least one Embedding vector included in each feature domain vector group in the multiple feature domain vector groups.

[0109] After calculating the compressed dimension quantity corresponding to at least one Embedding vector included in each of the multiple feature domain vector groups, the model parameter compression device can calculate the average value or weighted average value of the compressed dimension quantity corresponding to at least one Embedding vector included in the first feature domain vector group for any one of the multiple feature domain vector groups (such as the first feature domain vector group). Afterwards, the model parameter compression device can use the average value or the weighted average value as the compressed dimension quantity corresponding to the first feature domain vector group. It should be understood that the current compressed dimension quantity corresponding to at least one Embedding vector included in the first feature domain vector group becomes the average value or the weighted average value.

[0110] Figure 3b A schematic diagram showing an exemplary method of calculating the compression dimension corresponding to the feature domain vector group provided in an embodiment of the present application is shown. Figure 3b In the example, the average value is taken as the compression dimension corresponding to the feature domain vector group. After the compression rate of the first model parameter is less than or equal to the target compression rate, the model parameter compression device obtains the compression dimension corresponding to at least one Embedding vector included in each feature domain in the multiple feature domains, such as the compression dimension corresponding to at least one Embedding vector included in table1 corresponding to feature domain 1, the compression dimension corresponding to at least one Embedding vector included in table2 corresponding to feature domain 2, ..., the compression dimension corresponding to at least one Embedding vector included in tableN corresponding to feature domain N. Afterwards, the model parameter compression device can average the compressed dimensional quantities corresponding to at least one Embedding vector included in table1 to obtain the average value dim1, and use dim1 as the compressed dimensional quantity corresponding to feature domain 1; average the compressed dimensional quantities corresponding to at least one Embedding vector included in table2 to obtain the average value dim2, and use dim2 as the compressed dimensional quantity corresponding to feature domain 2;…, average the compressed dimensional quantities corresponding to at least one Embedding vector included in tableN to obtain the average value dimN, and use dimN as the compressed dimensional quantity corresponding to feature domain N.

[0111] For example, take the first feature domain vector group as feature domain vector group a. Feature domain vector group a includes Embedding vector 1, Embedding vector 2 and Embedding vector 3, and the total number of elements contained in Embedding vector 1, Embedding vector 2 and Embedding vector 3 is 64, the compression dimension corresponding to Embedding vector 1 is 15, the compression dimension corresponding to Embedding vector 2 is 25, and the compression dimension corresponding to Embedding vector 3 is 35. In one example, the model parameter compression device can calculate the average value of the compression dimensions corresponding to the three Embedding vectors, that is, the average value = (15+25+35) / 3 = 25, and the average value 25 can be used as the compression dimension corresponding to feature domain vector group a. It should be understood that the average value 25 is the compression dimension required for feature domain vector group a when performing domain-level compression. In another example, the model parameter compression device can assign weight values ​​to the compressed dimensional quantities corresponding to the three Embedding vectors, for example, the weight value of the compressed dimensional quantity corresponding to Embedding vector 1 is 0.2, the weight value of the compressed dimensional quantity corresponding to Embedding vector 2 is 0.3, and the weight value of the compressed dimensional quantity corresponding to Embedding vector 3 is 0.5. Afterwards, the model parameter compression device can calculate the weighted average value of the compressed dimensional quantities corresponding to the three Embedding vectors, that is, the weighted average value = (15*0.2+25*0.3+35*0.5) = 28, and can use the weighted average value 28 as the compressed dimensional quantity corresponding to the feature domain vector group a. It should be understood that the weighted average value 28 is the compressed dimensional quantity required for the feature domain vector group a when performing domain-level compression.

[0112] For another example, take the first feature domain vector group as feature domain vector group b. Feature domain vector group b includes Embedding vector 4 and Embedding vector 5, and the total number of elements contained in Embedding vector 4 and Embedding vector 5 is 64, respectively. The compression dimension corresponding to Embedding vector 4 is 40, and the compression dimension corresponding to Embedding vector 5 is 50. In one example, the model parameter compression device can calculate the average value of the compression dimension corresponding to the two Embedding vectors, that is, the average value = (40 + 50) / 2 = 45, and the average value 45 can be used as the compression dimension corresponding to feature domain vector group b. It should be understood that the average value 45 is the compression dimension required for feature domain vector group b when performing domain-level compression. In another example, the model parameter compression device can assign weight values ​​to the compression dimension corresponding to the two Embedding vectors, for example, the weight value of the compression dimension corresponding to Embedding vector 4 is 0.3, and the weight value of the compression dimension corresponding to Embedding vector 5 is 0.7. Afterwards, the model parameter compression device can calculate the weighted average of the compression dimensions corresponding to the two Embedding vectors, that is, the weighted average value = (40*0.3+50*0.7) = 47, and can use the weighted average value 47 as the compression dimension corresponding to the feature domain vector group b. It should be understood that the average value 45 is the compression dimension required for the feature domain vector group b when performing domain-level compression.

[0113] Step 206: The model parameter compression device compresses each feature domain vector group according to the compression dimension corresponding to each feature domain vector group.

[0114] After determining the compression dimension amount corresponding to each feature domain vector group in the multiple feature domain vector groups, the model parameter compression device can determine, for any feature domain vector group in the multiple feature domain vector groups (such as the second feature domain vector group), the number of compressed elements equal to the compression dimension amount from each Embedding vector included in the second feature domain vector group according to the compression dimension amount corresponding to the second feature domain vector group. Afterwards, the model parameter compression device can perform a mask-to-zero operation on the compressed elements determined in each Embedding vector included in the second feature domain vector group, for example, the model parameter compression device can adjust the values ​​of the compressed elements equal to the compression dimension amount in each Embedding vector to 0. It should be understood that the model parameter compression device can achieve the effect of compressing the feature domain vector group by performing a mask-to-zero operation on the compressed elements determined in each Embedding vector included in any feature domain vector group. Among them, the compressed elements determined by each Embedding vector included in the feature domain vector group are only masked to zero, and the original dimension of each Embedding vector included in the feature domain vector group does not change during the compression process or the training process. Therefore, when the first model meets the training termination condition, the model parameter compression device needs to trim the model parameters in the first model (such as the first model parameters) to achieve effective compression of the model parameters, and in the inference stage of the first model, there is no need to perform zero padding operations on the compressed positions of the model parameters, so it can effectively reduce the amount of calculation, help to significantly reduce latency, greatly speed up the response speed of the first model in related application scenarios, and improve the overall user experience.

[0115] For example, take the second feature domain vector group as feature domain vector group a, and the compressed dimension corresponding to feature domain vector group a is 25. Among them, feature domain vector group a includes Embedding vector 1, Embedding vector 2 and Embedding vector 3, and the total number of elements contained in Embedding vector 1, Embedding vector 2 and Embedding vector 3 is 64. For Embedding vector 1, the model parameter compression device can select the elements of the last 25 dimensional positions (such as the last 25 columns) in Embedding vector 1 as compressed elements. Afterwards, the model parameter compression device can adjust the values ​​of the compressed elements of the 25 dimensional positions to 0. For Embedding vector 2, the model parameter compression device can select the elements of the last 25 dimensional positions (such as the last 25 columns) in Embedding vector 2 as compressed elements. Afterwards, the model parameter compression device can adjust the values ​​of the compressed elements of the 25 dimensional positions to 0. For Embedding vector 3, the model parameter compression device may select the elements in the last 25 dimensional positions (such as the last 25 columns) in Embedding vector 3 as compressed elements. Afterwards, the model parameter compression device may adjust the values ​​of the compressed elements in the 25 dimensional positions to 0.

[0116] For another example, take the second feature domain vector group as feature domain vector group b, and the compressed dimension corresponding to feature domain vector group b is 45. Among them, feature domain vector group b includes Embedding vector 4 and Embedding vector 5, and the total number of elements contained in Embedding vector 4 and Embedding vector 5 is 64. For Embedding vector 4, the model parameter compression device can select the elements of the last 45 dimensional positions (such as the last 45 columns) in Embedding vector 4 as compressed elements. Afterwards, the model parameter compression device can adjust the values ​​of the compressed elements of the 45 dimensional positions to 0. For Embedding vector 5, the model parameter compression device can select the elements of the last 45 dimensional positions (such as the last 45 columns) in Embedding vector 5 as compressed elements. Afterwards, the model parameter compression device can adjust the values ​​of the compressed elements of the 45 dimensional positions to 0.

[0117] Based on the above content, the implementation process of the model parameter compression device performing a mask zeroing operation on the compressed elements determined in each Embedding vector included in the second feature domain vector group is introduced below through the following possible implementation methods.

[0118] Implementation method 1: The model parameter compression device adjusts the values ​​of one or more compressed elements determined in each Embedding vector included in the second feature domain vector group to 0.

[0119] For example, take the case where the second feature domain vector group includes two Embedding vectors (such as Embedding vector 4 and Embedding vector 5), and the compressed dimension corresponding to the second feature domain vector group is 45. Assume that Embedding vector 4 is [1,1,…,1] and Embedding vector 5 is [2,2,…,2]. The model parameter compression device can use the elements of the last 45 dimensional positions (i.e., the last 45 columns) in Embedding vector 4 [1,1,…,1] as compressed elements, and adjust the values ​​of the compressed elements of the 45 dimensional positions to 0, so that Embedding vector 4 [1,1,…,1] becomes [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,…,0]. In addition, the model parameter compression device can use the elements of the last 45 dimensional positions (i.e., the last 45 columns) in the Embedding vector 5[2,2,…,2] as compressed elements, and adjust the values ​​of the compressed elements in the 45 dimensional positions to 0. In this way, the Embedding vector 5[2,2,…,2] becomes [2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,0,…,0].

[0120] Implementation method 2: The model parameter compression device sets a preset Embedding vector. The dimension of the preset Embedding vector is the same as the initial dimension of the multiple Embedding vectors included in the second feature domain vector group, and the value of the element in the preset Embedding vector corresponding to the dimensional position of the compressed element included in the second feature domain vector group is set to 0. Afterwards, the model parameter compression device can multiply each Embedding vector included in the second feature domain vector group with the preset Embedding vector to obtain the processed Embedding vector, thereby realizing the masking and zeroing operation of the compressed elements in each Embedding vector included in the second feature domain vector group.

[0121] For example, let's take the second feature domain vector group including two Embedding vectors (such as Embedding vector 4 and Embedding vector 5), and the compression dimension corresponding to the second feature domain vector group is 45. It is assumed that Embedding vector 4 is [1,1,…,1], and Embedding vector 5 is [2,2,…,2]. The model parameter compression device first sets a 64*64 preset Embedding vector matrix Among them, the values ​​of the first 19 columns of the preset Embedding vector are all 1, and the values ​​of the last 45 columns of the elements are all 0. Afterwards, the model parameter compression device can multiply the Embedding vector 4 [1,1,…,1] with the preset Embedding vector matrix to obtain the processed Embedding vector [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,…,0], and multiply the Embedding vector 5 [2,2,…,2] with the preset Embedding vector to obtain the processed Embedding vector [2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,0,…,0].

[0122] Step 207: The model parameter compression device updates each feature domain vector group after compression processing to the Embedding layer of the first model, and trains the first model using the business data of the recommended business as training samples.

[0123] After the model parameter compression device performs domain-level compression processing on the Embedding vector included in each feature domain vector group in the multiple feature domain vector groups, the model parameter compression device obtains the Embedding vector included in each feature domain vector group after domain-level compression processing. Afterwards, the model parameter compression device can update the Embedding vector included in each feature domain vector group after domain-level compression processing to the Embedding layer of the first model. It can be understood that the model parameter compression device updates the first model parameter of the Embedding layer. Then, the model parameter compression device uses the business data of the recommended business as a training sample to train the updated first model, and stops training when the first model meets the training termination condition to obtain the trained first model. It should be understood that the first model parameters belonging to the Embedding layer in the first model are constantly changing during the training process of the first model. Among them, the trained first model (such as a trained recommendation model) can be used to infer or predict some information. For example, in a recommendation application scenario, the trained recommendation model can be used to predict the user's preference for a specific product or service, and can provide personalized recommendations to the user. Optionally, the training termination conditions may include but are not limited to the number of training times (or training rounds) of the first model meeting a preset number of training times (or training rounds), the prediction accuracy of the first model being greater than or equal to a target accuracy, or the loss function value of the first model being less than or equal to a target loss function value.

[0124] Based on the above content, the implementation process of the model parameter compression device clipping the model parameters in the trained first model is introduced below through the following possible implementation methods.

[0125] Implementation method 1: When the first model meets the training termination condition, the model parameter compression device can trim the second model parameters of the Embedding layer in the trained first model according to the compression dimension quantities corresponding to the multiple feature domain vector groups, and obtain the trimmed second model parameters. Among them, the trimmed second model parameters can be used as the model parameters used by the Embedding layer in the first model during reasoning. It can be understood that the trimmed second model parameters are used to update the model parameters corresponding to the Embedding layer in the first model. Optionally, after the model parameter compression device trims the second model parameters, a model parameter file in PB (Protocol buffer) format can be output. Among them, the model parameter file in PB format includes the trimmed second model parameters. Afterwards, the model parameter compression device can convert the model parameter file in PB format into a model parameter file in OM (omicron) format, wherein the model parameter file in OM format is used as the file used by the first model in the reasoning stage.

[0126] Exemplarily, the first model parameter includes feature domain vector group a and feature domain vector group b. Feature domain vector group a includes Embedding vector 1, Embedding vector 2 and Embedding vector 3, and each Embedding vector included in feature domain vector group a performs a mask-to-zero operation on the elements in the last 25 dimensional positions; feature domain vector group b includes Embedding vector 4 and Embedding vector 5, and each Embedding vector included in feature domain vector group b performs a mask-to-zero operation on the elements in the last 45 dimensional positions.

[0127] The model parameter compression device updates the feature domain vector group a and the feature domain vector group b after the mask zeroing operation as the new first model parameters (such as model parameter 1) to the Embedding layer of the first model, and uses the recommended business data to train the first model that updates the first model parameters. It should be understood that the compressed elements included in the model parameter 1 are subjected to the mask zeroing operation, so the values ​​of the compressed elements are kept as 0 during the training process. For example, taking the feature domain vector group a as an example, the values ​​of the elements in the last 25 dimensional positions of each Embedding vector included in the feature domain vector group a are kept as 0 during the training process. For another example, taking the feature domain vector group b as an example, the values ​​of the elements in the last 45 dimensional positions of each Embedding vector included in the feature domain vector group b are kept as 0 during the training process. However, the values ​​of the other elements (i.e.) included in the model parameter 1 except the compressed elements are constantly changing during the training process until the first model meets the training termination condition, and the second model parameter (such as model parameter 2) is obtained. Among them, model parameter 2 includes the changed feature domain vector group a and the changed feature domain vector group b. For example, taking feature domain vector group a as an example, the values ​​of the elements in the first 39 dimensional positions of each Embedding vector included in feature domain vector group a are constantly changing during the training process. For another example, taking feature domain vector group b as an example, the values ​​of the elements in the first 19 dimensional positions of each Embedding vector included in feature domain vector group b are constantly changing during the training process.

[0128] When the first model meets the training termination condition, for the changed feature domain vector group a included in the model parameter 2, the model parameter compression device can trim the elements of the last 25 dimensional positions of each Embedding vector included in the changed feature domain vector group a according to the compression dimension 25 corresponding to the feature domain vector group a, and obtain the trimmed feature domain vector group a. For the changed feature domain vector group b included in the model parameter 2, the model parameter compression device can trim the elements of the last 45 dimensional positions of each Embedding vector included in the changed feature domain vector group b according to the compression dimension 45 corresponding to the feature domain vector group b, and obtain the trimmed feature domain vector group b.

[0129] Implementation method 2: When the first model meets the training termination condition, in addition to trimming the second model parameters of the Embedding layer, the model parameter compression device can also trim the dense parameters of the model layer (such as the fully connected layer) located after the Embedding layer in the trained first model according to the compression dimension quantities corresponding to the multiple feature domain vector groups, to obtain the trimmed dense parameters. Among them, the trimmed second model parameters are used to update the model parameters corresponding to the Embedding layer in the first model, and the trimmed dense parameters are used to update the dense parameters corresponding to the model layer located after the Embedding layer in the first model. Exemplarily, the updated first model can be used for reasoning of recommendation business scenarios (such as providing personalized product recommendations to users by predicting which products users are more interested in). Optionally, after the model parameter compression device trims the second model parameters, a model parameter file in PB format can be output. Among them, the model parameter file in PB format includes the trimmed second model parameters and the trimmed dense parameters. Afterwards, the model parameter compression device can convert the model parameter file in PB format into a model parameter file in OM format, wherein the model parameter file in OM format is used as a file used by the first model in the inference stage. Optionally, the dense parameters of the model layer located after the Embedding layer include multiple parameters, and the multiple parameters correspond one-to-one to multiple feature domain vector groups. For example, taking the first model parameter including two feature domain vector groups (such as a user domain vector group and a product domain vector group) as an example, the dense parameters include two corresponding parameters, one parameter corresponds to the user domain vector group, and the multiple vectors included in the parameter are related to the user data and can reflect user attributes, and the other parameter corresponds to the product domain vector group, and the multiple vectors included in the parameter are related to the product data and can reflect product attributes.

[0130] Optionally, the implementation method of the model parameter compression device in the second implementation method cropping the second model parameter of the Embedding layer can refer to the implementation method of the model parameter compression device in the above-mentioned implementation method one cropping the second model parameter of the Embedding layer, which will not be repeated here.

[0131] Exemplarily, the first model parameter includes a feature domain vector group a and a feature domain vector group b, and the dense parameter includes two parameters (such as parameter a and parameter b). Among them, the feature domain vector group a includes Embedding vector 1, Embedding vector 2 and Embedding vector 3, and each Embedding vector included in the feature domain vector group a performs a mask-setting operation on the elements of the last 25 dimensional positions; the feature domain vector group b includes Embedding vector 4 and Embedding vector 5, and each Embedding vector included in the feature domain vector group b performs a mask-setting operation on the elements of the last 45 dimensional positions; the original dimension of the vector included in parameter a is the same as the original dimension of the Embedding vector included in the feature domain vector group a, and the original dimension of the vector included in parameter b is the same as the original dimension of the Embedding vector included in the feature domain vector group b.

[0132] The model parameter compression device updates the feature domain vector group a and the feature domain vector group b after the mask zeroing operation as the new first model parameters (such as model parameter 1) to the Embedding layer of the first model, and uses the recommended business data to train the first model with the updated first model parameters. During the first model training process, the new first model parameters are constantly changing, and the dense parameters of the model layer after the Embedding layer (such as dense parameter 1) are also constantly changing until the first model meets the training termination condition, and the second model parameters (such as model parameter 2) and the new dense parameters (such as dense parameter 2) are obtained.

[0133] When the first model meets the training termination condition, the implementation process of the model parameter compression device trimming model parameter 2 can refer to the implementation process of the model parameter compression device trimming model parameter 2 in the above-mentioned implementation method 1, which will not be repeated here.

[0134] When the first model meets the training termination condition, for the changed parameter a included in the dense parameter 2, the model parameter compression device can clip the elements of the last 25 dimensional positions of each vector included in the changed parameter a according to the compression dimension 25 corresponding to the feature domain vector group a, and obtain the clipped parameter a. For the changed parameter b included in the dense parameter 2, the model parameter compression device can clip the elements of the last 45 dimensional positions of each vector included in the changed parameter b according to the compression dimension 45 corresponding to the feature domain vector group b, and obtain the clipped parameter b.

[0135] Optionally, when the terminal device 100 has the function of implementing the model parameter compression method, the technical solutions provided in the above steps 201 to 207 can also be executed locally by the terminal device 100, and the structured compression of the first model parameters belonging to the Embedding layer in the first model can also be achieved, so that the first model can achieve a better inference acceleration effect.

[0136] It can be seen from the above steps 201 to 207 that after the model parameter compression device obtains the first model parameter belonging to the Embedding layer in the first model, it compresses the first model parameter in two stages, that is, the first stage is to perform vector-level compression on each Embedding vector included in the first model parameter, and the second stage is to perform domain-level compression on each feature domain vector group included in the first model parameter, so as to achieve structural compression of the first model parameter without the need for artificial a priori frequency information, thereby effectively reducing the manpower cost caused by artificial a priori frequency information and reducing the additional consumption caused by the acquisition of frequency a priori information. Furthermore, the method can be implemented in the reasoning stage of the first model without the need to perform zero padding operation on the compressed position of the model parameter (such as the first model parameter) of the model layer included in the first model, thereby effectively reducing the amount of calculation, thereby achieving the reasoning acceleration of the first model, so that the acceleration effect of the first model in the reasoning stage is better. In addition, the method does not need to save the position information of the compressed elements, so that the method is easy to deploy and can be applied without framework customization and adaptation.

[0137] Based on the technical solutions provided in steps 201 to 207 above, the following takes the first model as a DCN model, in which the model parameters belonging to the Embedding layer in the DCN model include 39 feature domain vector groups, each feature domain vector group includes at least one Embedding vector as an example to introduce the implementation process of compressing the model parameters of the DCN model by the model parameter compression device.

[0138] Figure 4 A schematic diagram showing another model parameter compression method provided in an embodiment of the present application is shown as an example. The method is applicable to Figure 1 The method flow can be executed by a model parameter compression device or a component (such as a chip, a chip system or a circuit, etc.) that can support the model parameter compression device to implement the functions required by the method. In order to facilitate the introduction of the technical solution provided by the embodiment of the present application, the following is an example of a model parameter compression method executed by a model parameter compression device. Figure 4 As shown, the method includes:

[0139] Step 401: The model parameter compression device determines whether the interval training times of the DCN model between the i-th vector level compression and the i-1-th vector level compression meet the preset times. If yes, step 402 is executed; if no, the DCN model continues to be trained.

[0140] Exemplarily, taking the preset number of times as 2 as an example, it is assumed that the model parameter compression device performs the first vector-level compression on the model parameters belonging to the Embedding layer in the DCN model before the first training of the DCN model begins. If the compression rate of the model parameters belonging to the Embedding layer in the DCN model after the first vector-level compression is greater than the target compression rate, the model parameter compression device may perform the second vector-level compression on the model parameters belonging to the Embedding layer after the second training of the DCN model after determining that the DCN model has been trained twice (that is, before the third training of the DCN model begins, or it can be understood that the DCN model is after the second training). If the compression rate of the model parameters belonging to the Embedding layer in the DCN model after the second vector-level compression is greater than the target compression rate, the model parameter compression device can perform a third vector-level compression on the model parameters belonging to the Embedding layer of the DCN model after the fourth training, after determining that the DCN model has been trained twice (that is, before the fifth training of the DCN model starts, or it can be understood as after the fourth training is completed), until the compression rate of the model parameters belonging to the Embedding layer in the DCN model after the vector-level compression is less than or equal to the target compression rate.

[0141] Step 402: The model parameter compression device performs the i-th vector-level compression process on the model parameters belonging to the Embedding layer in the DCN model.

[0142] Optionally, the implementation process of the model compression device in step 402 performing the i-th vector-level compression processing on the multiple Embedding vectors included in the 39 feature domain vector groups can refer to the implementation process of the model compression device in the above steps 202 to 203 performing vector-level compression processing on each Embedding vector in the multiple Embedding vectors included in the first model parameters, which will not be repeated here.

[0143] Step 403: The model parameter compression device determines whether the compression rate of the model parameters belonging to the Embedding layer after the i-th vector level compression is less than or equal to the target compression rate. If yes, execute step 404; if not, execute step 401.

[0144] Optionally, the implementation process of the model parameter compression device in step 403 determining the compression rate of the model parameters belonging to the Embedding layer after the i-th vector level compression can refer to the implementation process of the model parameter compression device in the above step 204 determining the compression rate of the first model parameter after compression, which will not be repeated here.

[0145] Step 404: The model parameter compression device performs domain-level compression processing on the model parameters belonging to the Embedding layer in the DCN model.

[0146] Optionally, the implementation process of the model parameter compression device in step 404 performing domain-level compression on 39 feature domain vector groups respectively can refer to the implementation process of the model parameter compression device in the above steps 205 to 206 performing domain-level compression on multiple feature domain vector groups included in the first model parameters respectively, which will not be repeated here.

[0147] Optionally, after executing step 404, the model parameter compression device may also execute steps 405 to 406. By executing steps 405 to 406, effective compression of the model parameters can be achieved, and in the reasoning stage of the first model, there is no need to perform zero padding on the compressed positions of the model parameters, thereby effectively reducing the amount of calculation, thereby achieving acceleration of model reasoning.

[0148] Step 405: the model parameter compression device updates the model parameters belonging to the Embedding layer that have undergone domain-level compression processing to the Embedding layer of the DCN model, and trains the DCN model using the service data of the recommended service as training samples.

[0149] Optionally, the implementation process of the model parameter compression device in step 405 training the DCN model based on the model parameters belonging to the Embedding layer that have been compressed at the domain level can refer to the implementation process of the model parameter compression device in the above step 207 training the first model based on the first model parameters that have been compressed at the domain level, which will not be repeated here.

[0150] Step 406: When the DCN model meets the training termination condition, the model parameter compression device clips the model parameters of the model layer included in the DCN model according to the compression dimension quantities corresponding to the 39 feature domain vector groups, and obtains a model parameter file. The model parameter file includes at least one model parameter (such as clipped Embedding parameters, clipped dense parameters, etc.).

[0151] Optionally, the implementation process of the model parameter compression device in step 406 cropping the model parameters of the model layer included in the DCN model based on the compression dimensions corresponding to the 39 feature domain vector groups can be referred to the implementation process of the model parameter compression device in the above step 207 cropping the model parameters of the model layer included in the first model based on the compression dimensions corresponding to the multiple feature domain vector groups, which will not be repeated here.

[0152] Optionally, when the terminal device 100 has the function of implementing the model parameter compression method, the technical solutions provided in the above steps 401 to 406 can also be executed locally by the terminal device 100.

[0153] It can be seen from the above steps 401 to 406 that after the model parameter compression device obtains the model parameters belonging to the Embedding layer in the DCN model, it compresses the model parameters belonging to the Embedding layer in two stages, that is, the first stage is to perform vector-level compression on each Embedding vector included in the model parameters belonging to the Embedding layer, and the second stage is to perform domain-level compression on each of the 39 feature domain vector groups included in the model parameters belonging to the Embedding layer, so as to achieve structural compression of the model parameters belonging to the Embedding layer without the need for artificial a priori frequency information, thereby effectively reducing the labor cost caused by artificial a priori frequency information and reducing the additional consumption caused by the acquisition of frequency a priori information. Furthermore, the method can be implemented in the inference stage of the DCN model without the need to perform zero padding operation on the compressed position of the model parameters of the model layer included in the DCN model, thereby effectively reducing the amount of calculation, thereby achieving inference acceleration of the DCN model, so that the acceleration effect of the DCN model in the inference stage is better. In addition, the method does not need to save the position information of the compressed elements, so that the method is easy to deploy and can be applied without framework customization and adaptation.

[0154] It should be noted that in the description of the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there may be three relationships, for example, A and / or B, which can represent: the situation where A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are an "or" relationship. "At least one of the following (individuals)" or its similar expression refers to any combination of these items, including any combination of single items (individuals) or plural items (individuals). For example, "at least one of A, B and C" includes A, B, C, AB, AC, BC or ABC. And, unless otherwise specified, the ordinal numbers such as "first", "second", and "third" mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. In addition, the terms "including", "comprising", "having" and their variations appearing in the present application all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0155] In addition, it should be noted that each step involved in the above embodiments can be performed by a corresponding device, or by a chip, processor, or chip system in the device, and the embodiments of the present application do not limit them. The above embodiments are only described by taking the corresponding device as an example.

[0156] It should be noted that in the above embodiments, some steps can be selected for implementation, and the order of the steps in the diagram can be adjusted for implementation, and this application does not limit this. It should be understood that executing some steps in the diagram, adjusting the order of the steps, or combining them for specific implementation all fall within the scope of protection of this application.

[0157] It is understandable that, in order to implement the functions in the above embodiments, the various devices involved in the above embodiments include hardware structures and / or software modules corresponding to the execution of the various functions. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of the various examples described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0158] It should be noted that the "steps" in the embodiments of the present application are only for illustration, and are a method of expression used to better understand the embodiments, and do not constitute a substantial limitation on the execution of the scheme of the present application. For example, the "steps" can also be understood as "features". In addition, the steps do not constitute any limitation on the execution order of the scheme of the present application. Any changes in the order of steps, step merging, or step splitting made on this basis that do not affect the implementation of the overall scheme, and the resulting new technical solutions are also within the scope of the disclosure of the present application.

[0159] Based on the same concept, the embodiment of the present application also provides a possible model parameter compression device, which is suitable for Figure 1 In the application scenario shown in FIG. 1 , the model parameter compression device may be Figure 1 The service device 200 shown in the figure (such as a computing device with the function of the model parameter compression device or a computing device cluster with the function of the model parameter compression device, etc.), or it can also be any other device with the function required to implement the method (such as a vehicle-mounted device, etc.), or it can also be Figure 1 The functional element (such as a plug-in, component, chip or circuit, etc.) in the service device 200 shown in the figure, or it can also be a functional element set in any other device, and the functional element has the function of implementing the model parameter compression device method. Optionally, the model parameter compression device is used to implement the model parameter compression device method provided in the above embodiment, or the module (such as a chip) of the model parameter compression device is used to implement the model parameter compression device method provided in the above embodiment, so the beneficial effects of the above embodiment can also be achieved. Exemplarily, taking the model parameter compression device as a chip set in a computing device as an example, when the model parameter compression device is a chip, the model parameter compression device includes a transceiver and a processor, but does not include a memory. Among them, the transceiver exists as an input and output interface, and the input and output interface is used for the chip to realize the transceiver of the computing device. The input and output interface may include an input interface and / or an output interface, the input interface can realize the reception of the computing device, and the output interface can be used to realize the sending of the computing device. The processor is used to read and execute corresponding computer programs or instructions so that the corresponding functions of the model parameter compression device are realized. Optionally, when the chip implements the corresponding functions of the model parameter compression device in the above embodiment, the input and output interface can implement the transceiver operation performed by the model parameter compression device in the above embodiment; the processor can implement other operations except the transceiver operation performed by the model parameter compression device in the above embodiment. For specific related descriptions, please refer to the related descriptions in the above embodiment, which will not be described in detail here.

[0160] See also Figure 5The model parameter compression device 500 includes an acquisition module 501 (or may be called a transceiver module or a transceiver unit, used to send and receive data) and a processing module 502.

[0161] Optionally, the acquisition module 501 may include a receiving module and / or a sending module. The receiving module may be used for the model parameter compression device 500 to receive a signal (information or data, etc.); the sending module may be used for the model parameter compression device 500 to send a signal (information or data, etc.). The sending module may send a signal (information or data, etc.) under the control of the processing module 502, and the receiving module may receive a signal (information or data, etc.) under the control of the processing module 502.

[0162] Optionally, the processing module 502 may include one or more of an Embedding vector importance index value calculation module, an Embedding vector dimension increase or decrease module, a feature domain length calculation module, or a model parameter pruning module (or may be referred to as a model parameter pruning module).

[0163] Exemplarily, the processing module 502 includes an Embedding vector importance index value calculation module, an Embedding vector dimension increase and decrease module, a feature domain length calculation module and a model parameter clipping module. Among them, the Embedding vector importance index value calculation module is used to evaluate (or can be called calculation or determination) the importance index value of each Embedding vector. Specifically, the implementation process of the Embedding vector importance index value calculation module evaluating the importance index value of each Embedding vector can refer to the implementation process of the model parameter compression device in the above method embodiment to obtain the importance index value corresponding to each Embedding vector, which will not be repeated here.

[0164] The Embedding vector dimension increase / decrease module is used to increase / decrease the current remaining dimensions of each Embedding vector according to the importance index value of each Embedding vector, and can perform vector-level compression on each Embedding vector according to the increase / decrease adjustment of each Embedding vector. Specifically, the implementation process of the Embedding vector dimension increase / decrease module performing vector-level compression on each Embedding vector can refer to the implementation process of the model parameter compression device performing vector-level compression on each Embedding vector in the above method embodiment, which will not be repeated here.

[0165] The feature domain length calculation module is used to calculate the compression dimension corresponding to each feature domain vector group, and can perform domain-level compression on each feature domain vector group according to the compression dimension corresponding to each feature domain vector group. Specifically, the implementation process of the feature domain length calculation module performing domain-level compression on each feature domain vector group can refer to the implementation process of the model parameter compression device performing domain-level compression on each feature domain vector group in the above method embodiment, which will not be repeated here.

[0166] The model parameter clipping module is used to clip the model parameters of the model layer included in the trained first model according to the compression dimension quantities corresponding to the multiple feature domain vector groups. Specifically, the implementation process of the model parameter clipping module clipping the model parameters of the model layer included in the trained first model can refer to the implementation process of the model parameter compression device clipping the model parameters of the model layer included in the trained first model in the above method embodiment, which will not be repeated here.

[0167] When the model parameter compression device 500 is used to implement Figure 2 or Figure 4 The functions of the model parameter compression device in the method embodiment shown are as follows: an acquisition module 501 is used to acquire a first model parameter belonging to an Embedding layer in a first model, wherein the first model parameter includes a plurality of feature domain vector groups, and any feature domain vector group includes at least one Embedding vector. A processing module 502 is used to perform the following compression processing on the plurality of Embedding vectors included in the first model parameter: determining the compression dimension corresponding to each Embedding vector in the plurality of Embedding vectors included in the first model parameter; compressing each Embedding vector separately according to the compression dimension corresponding to each Embedding vector; and determining the compression ratio of the plurality of Embedding vectors included in the first model parameter after compression. The processing module 502 is also used to, when the compression rate does not meet the target compression rate, re-compress the multiple Embedding vectors included in the first model parameters until the compression rate meets the target compression rate; or when the compression rate meets the target compression rate, determine the compression dimension corresponding to each feature domain vector group according to the compression dimension corresponding to at least one Embedding vector included in each feature domain vector group in the multiple feature domain vector groups, and perform compression processing on each feature domain vector group according to the compression dimension corresponding to each feature domain vector group. The processing module 502 is also used to update each feature domain vector group after compression processing to the Embedding layer of the first model, and train the first model using the business data of the recommended business as a training sample.

[0168] Wherein, when the model parameter compression device 500 is used to implement Figure 2 or Figure 4 For a more detailed description of the acquisition module 501 and the processing module 502, please refer to the above Figure 2 or Figure 4 The relevant description about the model parameter compression device in the method embodiment shown is not repeated here.

[0169] Optional, above Figure 5 The acquisition module and processing module (including one or more of the Embedding vector importance index value calculation module, the Embedding vector dimension increase and decrease module, the feature domain length calculation module or the model parameter clipping module) shown in the figure can be implemented by software, or can be implemented by hardware. Exemplarily, the implementation method of the processing module is introduced below by taking the processing module as an example. Similarly, the implementation method of the acquisition module can refer to the implementation method of the processing module, which will not be repeated here.

[0170] When implemented by software, the processing module is used as an example of a software functional unit, and the processing module may be an application or a code block running on a computer device. Among them, the computer device may be at least one of a physical host, a virtual machine, a container and other computing devices. Furthermore, the above-mentioned computer device may be one or more. For example, the processing module may be an application running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application may be distributed in the same availability zone (AZ) or in different AZs. The multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. Among them, usually a region may include multiple AZs.

[0171] Similarly, multiple hosts / virtual machines / containers for running the application can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Typically, a region can include multiple VPCs, and a VPC can include multiple AZs.

[0172] When implemented by hardware, the processing module is used as an example of a hardware functional unit, and the processing module may include at least one computing device, such as a server, etc. Alternatively, the processing module may also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0173] The multiple computing devices included in the processing module can be distributed in the same AZ or in different AZs. The multiple computing devices included in the processing module can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the processing module can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0174] It should be noted that the steps that the acquisition module and the processing module (including one or more of the Embedding vector importance index value calculation module, the Embedding vector dimension increase and decrease module, the feature domain length calculation module or the model parameter trimming module) are responsible for implementing can be specified as needed. The full functions of the model parameter compression device can be realized by respectively implementing different execution steps of the model parameter compression method through the acquisition module and the processing module.

[0175] In addition, it should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The functional modules in the embodiments of the present application can be integrated into one module, or each module can exist physically separately, or two or more modules can be integrated into one module. For example, the acquisition module and the processing module can be integrated into one module, or the acquisition module and the processing module are the same module. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0176] Based on the same concept, the present application embodiment also provides a possible computing device for executing the model parameter compression method shown in the above method embodiment. The relevant features can be found in the above method embodiment and will not be described in detail here. Figure 6 As shown, the computing device 600 includes: a bus 601, a processor 602 and a memory 604. Optionally, the computing device 600 may also include a communication interface 603. The processor 602, the communication interface 603 and the memory 604 communicate through the bus 601. The computing device 600 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 600.

[0177] The bus 601 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus 601 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 601 may include a path for transmitting information between various components of the computing device 600 (eg, the memory 604, the processor 602, and the communication interface 603).

[0178] The processor 602 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0179] The memory 604 may include a volatile memory, such as a random access memory (RAM). The processor 602 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0180] The memory 604 stores executable program codes, and the processor 602 executes the executable program codes to respectively implement the functions of the aforementioned model parameter compression device (such as the acquisition module, the processing module (including one or more of the Embedding vector importance index value calculation module, the Embedding vector dimension increase and decrease module, the feature domain length calculation module or the model parameter clipping module)), thereby implementing the model parameter compression method provided in the embodiment of the present application. That is, the memory 604 stores computer program instructions for executing the model parameter compression method.

[0181] Alternatively, the memory 604 stores executable code, and the processor 602 executes the executable code to implement the functions of the aforementioned model parameter compression device (such as an acquisition module, a processing module (including one or more of an Embedding vector importance index value calculation module, an Embedding vector dimension increase and decrease module, a feature domain length calculation module, or a model parameter clipping module)), thereby implementing the model parameter compression method provided in the embodiment of the present application. That is, the memory 604 stores computer program instructions for the model parameter compression device to execute the model parameter compression method provided in the embodiment of the present application.

[0182] The communication interface 603 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or a communication network.

[0183] Based on the same concept, the embodiment of the present application also provides a possible computing device cluster for executing the method shown in the above method embodiment. The relevant features can be found in the above method embodiment, which will not be repeated here. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop, a laptop, or a smart phone.

[0184] like Figure 7 As shown, the computing device cluster 700 includes at least one computing device 600. The memory 604 in one or more computing devices 600 in the computing device cluster 700 may store the same instructions for executing the model parameter compression method provided in the embodiment of the present application.

[0185] In some possible implementations, part of the instructions for executing the model parameter compression method may also be stored in the memory 604 of one or more computing devices 600 in the computing device cluster 700. In other words, the combination of one or more computing devices 600 may jointly execute the instructions for executing the model parameter compression method.

[0186] It should be noted that the memory 604 in different computing devices 600 in the computing device cluster 700 can store different instructions, which are respectively used to execute part of the functions of the model parameter compression device. That is, the instructions stored in the memory 604 in different computing devices 600 can realize the functions of one or more modules in the acquisition module, the processing module (including one or more of the Embedding vector importance index value calculation module, the Embedding vector dimension increase and decrease module, the feature domain length calculation module or the model parameter clipping module).

[0187] In some possible implementations, one or more computing devices in computing device cluster 700 may be connected via a network, which may be a wide area network or a local area network. Figure 8 A possible implementation is shown. Figure 8 As shown, two computing devices 600A and 600B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 604 in the computing device 600A stores instructions for executing the functions of the acquisition module 501, the Embedding vector importance index value calculation module, and the Embedding vector dimension increase and decrease module. At the same time, the memory 604 in the computing device 600B stores instructions for executing the functions of the feature domain length calculation module and the model parameter clipping module. In some possible implementations, the memory 604 of one or more computing devices 600 in the computing device cluster may also respectively store some instructions for executing the model parameter compression method. In other words, a combination of one or more computing devices 600 can jointly execute instructions for executing the model parameter compression method.

[0188] It should be understood that Figure 8 The functions of the computing device 600A shown in FIG. 6A may also be completed by multiple computing devices 600. Similarly, the functions of the computing device 600B may also be completed by multiple computing devices 600.

[0189] Based on the same concept, the embodiment of the present application also provides a computer program product comprising instructions. The computer program product may be a software or program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the model parameter compression method provided in the embodiment of the present application.

[0190] Based on the same concept, an embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the model parameter compression method provided in the embodiment of the present application.

[0191] Based on the same concept, an embodiment of the present application also provides a computer chip, which is coupled to a memory and is used to read a computer program stored in the memory to execute the model parameter compression method provided in the embodiment of the present application.

[0192] Based on the same concept, an embodiment of the present application also provides a chip system, which includes a processor for supporting a computer device to implement the model parameter compression method provided in an embodiment of the present application. In a possible design, the chip system also includes a memory, which is used to store the necessary programs and data of the computer device. The chip system can be composed of a computer chip, or it can include a computer chip and other discrete devices.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A model parameter compression method, characterized in that: include: Acquire a first model parameter belonging to an Embedding layer in the first model, wherein the first model parameter includes a plurality of feature domain vector groups, and any feature domain vector group includes at least one Embedding vector; The multiple Embedding vectors included in the first model parameters are compressed as follows: determining a compression dimension corresponding to each Embedding vector in the multiple Embedding vectors included in the first model parameters; and compressing each Embedding vector according to the compression dimension corresponding to each Embedding vector; Determine a compression ratio of the plurality of Embedding vectors included in the first model parameters after compression; When the compression rate is greater than the target compression rate, re-performing the above compression process on the multiple Embedding vectors included in the first model parameters until the compression rate is less than or equal to the target compression rate; or When the compression rate is less than or equal to the target compression rate, determining the compression dimension corresponding to each feature domain vector group according to the compression dimension corresponding to at least one Embedding vector included in each feature domain vector group in the multiple feature domain vector groups; and performing compression processing on each feature domain vector group according to the compression dimension corresponding to each feature domain vector group; Each feature domain vector group after compression is updated to the Embedding layer of the first model, and the business data of the recommended business is used as a training sample to train the first model.

2. The method according to claim 1, characterized in that Determining a compression dimension corresponding to each of the multiple Embedding vectors included in the first model parameters includes: Obtaining an importance index value corresponding to a first Embedding vector, where the importance index value is used to indicate the importance of a data object corresponding to the first Embedding vector, and the first Embedding vector is any one of the multiple Embedding vectors; According to the relationship between the importance index value corresponding to the first Embedding vector and the importance threshold, the compression dimension corresponding to the first Embedding vector is determined; wherein the importance threshold is the importance index value corresponding to the multiple Embedding vectors, which is sorted in order from large to small and then the importance index value corresponding to the i-th Embedding vector; i is determined based on a preset coefficient and the total number of the multiple Embedding vectors.

3. The method according to claim 2, characterized in that Get the importance index value corresponding to the first Embedding vector, including: Determine the absolute value of the element at the end of the first Embedding vector; The absolute value is used as the importance index value corresponding to the first Embedding vector.

4. The method according to claim 2 or 3, characterized in that Determining the compression dimension corresponding to the first Embedding vector according to the relationship between the importance index value corresponding to the first Embedding vector and the importance threshold, including: When the importance index value corresponding to the first Embedding vector is greater than or equal to the importance threshold, the first vector dimension of the first Embedding vector is added to a preset value to obtain a second vector dimension, where the first vector dimension is used to indicate the current remaining dimension of the first Embedding vector; when the second vector dimension is within a set vector dimension range, the difference between the maximum vector dimension within the set vector dimension range and the second vector dimension is used as the compression dimension amount corresponding to the first Embedding vector; when the second vector dimension is not within the set vector dimension range, the second vector dimension is adjusted to obtain a third vector dimension; when the third vector dimension is within the set vector dimension range, the difference between the maximum vector dimension and the third vector dimension is used as the compression dimension amount corresponding to the first Embedding vector; or When the importance index value corresponding to the first Embedding vector is less than the importance threshold, the first vector dimension of the first Embedding vector is subtracted from the preset value to obtain a fourth vector dimension; when the fourth vector dimension is within the set vector dimension range, the difference between the maximum vector dimension and the fourth vector dimension is used as the compression dimension amount corresponding to the first Embedding vector; when the fourth vector dimension is not within the set vector dimension range, the fourth vector dimension is adjusted to obtain a fifth vector dimension; when the fifth vector dimension is within the set vector dimension range, the difference between the maximum vector dimension and the fifth vector dimension is used as the compression dimension amount corresponding to the first Embedding vector.

5. The method according to any one of claims 1 to 4, characterized in that: According to the compression dimension corresponding to each Embedding vector, each Embedding vector is compressed separately, including: According to the compressed dimension amount corresponding to the second Embedding vector, determining from the second Embedding vector a number of compressed elements equal to the compressed dimension amount, the second Embedding vector being any one of the multiple Embedding vectors; The value of the compressed element determined in the second Embedding vector is adjusted to 0.

6. The method according to any one of claims 1 to 5, characterized in that: Determining the compression ratio of the plurality of Embedding vectors included in the first model parameters after compression includes: Determining the number of at least one non-zero element included in the plurality of Embedding vectors; The ratio of the number of the at least one non-zero element to the total number of elements included in the multiple Embedding vectors is used as the compression ratio.

7. The method according to any one of claims 1 to 6, characterized in that: According to the compression dimension amount corresponding to at least one Embedding vector included in each of the plurality of feature domain vector groups, respectively determining the compression dimension amount corresponding to each feature domain vector group, including: Determine an average value or a weighted average value of a compressed dimension quantity corresponding to at least one Embedding vector included in a first feature domain vector group, wherein the first feature domain vector group is any one of the multiple feature domain vector groups; The average value or the weighted average value is used as the compression dimension corresponding to the first feature domain vector group.

8. The method according to any one of claims 1 to 7, characterized in that: According to the compression dimension corresponding to each feature domain vector group, each feature domain vector group is compressed respectively, including: According to the compressed dimension amount corresponding to the second feature domain vector group, determining the number of compressed elements equal to the compressed dimension amount from each Embedding vector included in the second feature domain vector group, wherein the second feature domain vector group is any one of the multiple feature domain vector groups; The value of the compressed element determined in each Embedding vector included in the second feature domain vector group is adjusted to 0.

9. The method according to any one of claims 1 to 8, characterized in that: After training the first model using the business data of the recommended business as training samples, the method further includes: When the first model meets the training termination condition, the second model parameters of the Embedding layer in the first model are pruned according to the compressed dimensions respectively corresponding to the multiple feature domain vector groups to obtain pruned second model parameters, and the pruned second model parameters are used as model parameters used by the first model during inference; Among them, the training termination condition is at least one of the following: the training times of the first model meet the preset training times, the prediction accuracy of the first model is greater than or equal to the target accuracy, or the loss function value of the first model is less than or equal to the target loss function value.

10. A model parameter compression device, characterized in that: include: An acquisition module, used to acquire first model parameters belonging to an Embedding layer in a first model, wherein the first model parameters include a plurality of feature domain vector groups, and any feature domain vector group includes at least one Embedding vector; Processing modules for: The multiple Embedding vectors included in the first model parameters are compressed as follows: determining a compression dimension corresponding to each Embedding vector in the multiple Embedding vectors included in the first model parameters; and compressing each Embedding vector according to the compression dimension corresponding to each Embedding vector; Determine a compression ratio of the plurality of Embedding vectors included in the first model parameters after compression; When the compression rate is greater than the target compression rate, re-compressing the multiple Embedding vectors included in the first model parameters until the compression rate is less than or equal to the target compression rate; or When the compression rate is less than or equal to the target compression rate, determining the compression dimension corresponding to each feature domain vector group according to the compression dimension corresponding to at least one Embedding vector included in each feature domain vector group in the multiple feature domain vector groups; and performing compression processing on each feature domain vector group according to the compression dimension corresponding to each feature domain vector group; Each feature domain vector group after compression is updated to the Embedding layer of the first model, and the business data of the recommended business is used as a training sample to train the first model.

11. The device according to claim 10, characterized in that When determining the amount of compressed dimension corresponding to each of the multiple Embedding vectors included in the first model parameters, the processing module is specifically used to: Obtaining an importance index value corresponding to a first Embedding vector, where the importance index value is used to indicate the importance of a data object corresponding to the first Embedding vector, and the first Embedding vector is any one of the multiple Embedding vectors; Determine the compressed dimension corresponding to the first Embedding vector according to the relationship between the importance index value corresponding to the first Embedding vector and the importance threshold; wherein the importance threshold is the importance index value corresponding to the i-th Embedding vector after the importance index values ​​corresponding to the multiple Embedding vectors are sorted in descending order; The i is determined according to a preset coefficient and the total number of the multiple Embedding vectors.

12. The device according to claim 11, characterized in that When obtaining the importance index value corresponding to the first Embedding vector, the processing module is specifically used to: Determine the absolute value of the element at the end of the first Embedding vector; The absolute value is used as the importance index value corresponding to the first Embedding vector.

13. The device according to claim 11 or 12, characterized in that When the processing module determines the compressed dimension corresponding to the first Embedding vector according to the magnitude relationship between the importance index value corresponding to the first Embedding vector and the importance threshold, it is specifically used to: When the importance index value corresponding to the first Embedding vector is greater than or equal to the importance threshold, the first vector dimension of the first Embedding vector is added to a preset value to obtain a second vector dimension, where the first vector dimension is used to indicate the current remaining dimension of the first Embedding vector; when the second vector dimension is within a set vector dimension range, the difference between the maximum vector dimension within the set vector dimension range and the second vector dimension is used as the compression dimension amount corresponding to the first Embedding vector; when the second vector dimension is not within the set vector dimension range, the second vector dimension is adjusted to obtain a third vector dimension; when the third vector dimension is within the set vector dimension range, the difference between the maximum vector dimension and the third vector dimension is used as the compression dimension amount corresponding to the first Embedding vector; or When the importance index value corresponding to the first Embedding vector is less than the importance threshold, the first vector dimension of the first Embedding vector is subtracted from the preset value to obtain a fourth vector dimension; when the fourth vector dimension is within the set vector dimension range, the difference between the maximum vector dimension and the fourth vector dimension is used as the compression dimension amount corresponding to the first Embedding vector; when the fourth vector dimension is not within the set vector dimension range, the fourth vector dimension is adjusted to obtain a fifth vector dimension; when the fifth vector dimension is within the set vector dimension range, the difference between the maximum vector dimension and the fifth vector dimension is used as the compression dimension amount corresponding to the first Embedding vector.

14. The device according to any one of claims 10 to 13, characterized in that: When the processing module compresses each Embedding vector according to the compression dimension corresponding to each Embedding vector, it is specifically used to: According to the compressed dimension amount corresponding to the second Embedding vector, determining from the second Embedding vector a number of compressed elements equal to the compressed dimension amount, the second Embedding vector being any one of the multiple Embedding vectors; The value of the compressed element determined in the second Embedding vector is adjusted to 0.

15. The device according to any one of claims 10 to 14, characterized in that: When determining the compression ratio of the plurality of Embedding vectors included in the first model parameters after compression, the processing module is specifically used to: Determining the number of at least one non-zero element included in the plurality of Embedding vectors; The ratio of the number of the at least one non-zero element to the total number of elements included in the multiple Embedding vectors is used as the compression ratio.

16. The device according to any one of claims 10 to 15, characterized in that The processing module is specifically used to determine the compression dimension amount corresponding to each feature domain vector group according to the compression dimension amount corresponding to at least one Embedding vector included in each feature domain vector group in the multiple feature domain vector groups: Determine an average value or a weighted average value of a compressed dimension quantity corresponding to at least one Embedding vector included in a first feature domain vector group, wherein the first feature domain vector group is any one of the multiple feature domain vector groups; The average value or the weighted average value is used as the compression dimension corresponding to the first feature domain vector group.

17. The device according to any one of claims 10 to 16, characterized in that: When the processing module performs compression processing on each feature domain vector group according to the compression dimension corresponding to each feature domain vector group, the processing module is specifically used to: According to the compressed dimension amount corresponding to the second feature domain vector group, determining the number of compressed elements equal to the compressed dimension amount from each Embedding vector included in the second feature domain vector group, wherein the second feature domain vector group is any one of the multiple feature domain vector groups; The value of the compressed element determined in each Embedding vector included in the second feature domain vector group is adjusted to 0.

18. The device according to any one of claims 10 to 17, characterized in that After the processing module trains the first model using the business data of the recommended business as training samples, the processing module is further configured to: When the first model meets the training termination condition, the second model parameters of the Embedding layer in the first model are pruned according to the compressed dimensions respectively corresponding to the multiple feature domain vector groups to obtain pruned second model parameters, and the pruned second model parameters are used as model parameters used by the first model during inference; Among them, the training termination condition is one of the following: the training times of the first model meet the preset training times, the prediction accuracy of the first model is greater than or equal to the target accuracy, or the loss function value of the first model is less than or equal to the target loss function value.

19. A computing device, characterized in that The computing device includes a processor and a memory; The memory is used to store computer program instructions; The processor calls the computer program instructions stored in the memory to execute the method according to any one of claims 1 to 9.

20. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device includes a processor and a memory; the memory in the at least one computing device is used to store computer program instructions; The processor in the at least one computing device calls the computer program instructions stored in the memory to execute the method according to any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 9.

22. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method as claimed in any one of claims 1 to 9.

Images