A data processing method, device, apparatus, and storage medium

Abstract

The application belongs to the technical field of artificial intelligence, and particularly relates to a data processing method and device, equipment and a storage medium. The method comprises the following steps: obtaining a preset operator model and a candidate compressed operator model corresponding to the preset operator model from a storage; the candidate compressed operator model is obtained by compressing the preset operator model based on at least two compression strategies through a processor; inputting multimedia data into the preset operator model and each candidate compressed operator model for prediction processing, to obtain a first prediction result output by the preset operator model and a second prediction result output by each compressed operator model; and determining a target compressed operator model corresponding to the preset operator model from the candidate compressed operator model based on the first prediction result and each second prediction result. The scheme can improve the compression accuracy of each preset operator model in the preset operator model, thereby improving the service processing efficiency and the computing resource utilization rate of an end-side device.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, neural network models are widely used in various fields of life. However, with the proliferation of edge smart devices, more and more neural network models need to be deployed and run on edge devices with relatively small storage capacities and limited computing power. Therefore, how to efficiently apply neural network models on devices with limited computing power is an urgent problem to be solved.

[0003] In related technologies, the common approach is to compress neural network models and then deploy the compressed models onto edge devices with limited computing power. However, when compressing neural network models, the entire model is often compressed to a fixed bit width. On one hand, during model compression, the bit width used by the neural network model is fixed, meaning the compressed model uses the same computing resources for processing, resulting in low utilization of computing resources. On the other hand, this method does not rely on input feature data during model compression, leading to significant accuracy loss in the resulting neural network model. After deploying the compressed neural network model onto edge devices, the devices cannot output accurate prediction results, reducing processing efficiency and affecting the normal operation of the edge devices. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a data processing method, apparatus, device, and storage medium.

[0005] On one hand, embodiments of this application provide a data processing method, the method comprising:

[0006] Retrieve a preset operator model and a corresponding candidate compression operator model from memory; wherein, the preset operator model is any one of the multiple operator models contained in the preset model; the candidate compression operator model is obtained by the processor compressing the preset operator model based on at least two compression strategies;

[0007] Multimedia data is input into a preset operator model and each candidate compression operator model for prediction processing, resulting in a first prediction result output by the preset operator model and a second prediction result output by each compression operator model.

[0008] Based on the first prediction result and each second prediction result, the target compression operator model corresponding to the preset operator model is determined among the candidate compression operator models.

[0009] In some optional embodiments, the method for determining the candidate compression operator model includes:

[0010] Obtain the preset operator model and at least two compression strategies;

[0011] The processor compresses the weight parameters and / or activation parameters in the preset operator model using at least two compression strategies to obtain candidate compression operator models.

[0012] In some optional embodiments, based on the first prediction result and each second prediction result, a target compression operator model corresponding to the preset operator model is determined from the candidate compression operator models, including:

[0013] Based on the first prediction result, a target second prediction result that meets the preset conditions is determined in each second prediction result;

[0014] Among the candidate compression operator models, the candidate compression operator model that outputs the second prediction result of the target is determined, and the target compression operator model corresponding to the preset operator model is obtained.

[0015] In some optional embodiments, based on the first prediction result, a target second prediction result that satisfies preset conditions is determined in each second prediction result, including:

[0016] The processor calculates the similarity between each second prediction result and the first prediction result to obtain the first similarity data corresponding to each second prediction result.

[0017] Based on the first similarity data corresponding to each second prediction result, the target second prediction result is determined from the second prediction results; the first similarity data corresponding to the target second prediction result is greater than the similarity threshold.

[0018] In some optional embodiments, the processor calculates the similarity between each second prediction result and the first prediction result to obtain first similarity data corresponding to each second prediction result, including:

[0019] The processor performs a transformation process on each second prediction result to obtain the transformed second prediction result corresponding to each second prediction result.

[0020] The processor calculates the similarity between each second prediction result and the first prediction result, and obtains the first similarity data corresponding to each second prediction result.

[0021] In some optional embodiments, the method further includes:

[0022] Retrieve the target compression operator model corresponding to each operator model contained in the preset model from the memory;

[0023] The initial compression model is obtained by combining each target compression operator model.

[0024] Multimedia data is input into a preset model and an initial compression model for prediction processing, respectively, to obtain the original prediction results output by the preset model and the initial compressed prediction results output by the initial compression model.

[0025] The processor calculates the similarity between the original prediction results and the initial compressed prediction results to obtain the second similarity data;

[0026] If the second similarity data meets the preset conditions, the initial compression model is determined as the target compression model corresponding to the preset model.

[0027] In some optional embodiments, after obtaining second similarity data by calculating the similarity between the original prediction result and the initial compressed prediction result using a processor, the method further includes:

[0028] If the second similarity data does not meet the preset conditions, a preset target compression operator model is determined in the initial compression model; the preset target compression operator model is the target compression operator model in the initial compression model that meets the preset conditions.

[0029] In the initial compression model, the preset target compression operator model is replaced with the preset operator model corresponding to the preset target compression operator model to obtain the updated compression model;

[0030] Multimedia data is input into a preset model and an updated compression model for prediction processing, respectively, to obtain the original prediction results output by the preset model and the updated compression prediction results output by the updated compression model.

[0031] The processor calculates the similarity between the original prediction results and the updated compressed prediction results to obtain the third similarity data;

[0032] If the third similarity data meets the preset conditions, the updated compression model will be determined as the target compression model corresponding to the preset model.

[0033] On the other hand, embodiments of this application also provide a data processing apparatus, the apparatus comprising:

[0034] The acquisition module is used to acquire a preset operator model and a candidate compression operator model corresponding to the preset operator model from the memory; wherein, the preset operator model is any one of the multiple operator models contained in the preset model; the candidate compression operator model is obtained by the processor compressing the preset operator model based on at least two compression strategies;

[0035] The prediction processing module is used to input multimedia data into a preset operator model and each candidate compression operator model for prediction processing, and obtain the first prediction result output by the preset operator model and the second prediction result output by each compression operator model.

[0036] The target compression operator model determination module is used to determine the target compression operator model corresponding to the preset operator model from among the candidate compression operator models based on the first prediction result and each second prediction result.

[0037] In some optional embodiments, the apparatus further includes a candidate compression operator model determination module, the candidate compression operator model determination module comprising:

[0038] The acquisition submodule is used to acquire the preset operator model and at least two compression strategies;

[0039] The compression processing submodule is used to compress the weight parameters and / or activation parameters in the preset operator model using at least two compression strategies to obtain candidate compressed operator models.

[0040] In some optional embodiments, the target compression operator model determination module includes:

[0041] The second prediction result determination submodule is used to determine the second prediction result of the target that meets the preset conditions in each second prediction result based on the first prediction result.

[0042] The target compression operator model determination submodule is used to determine the candidate compression operator model that outputs the second prediction result of the target from the candidate compression operator models, and obtain the target compression operator model corresponding to the preset operator model.

[0043] In some optional embodiments, the target second prediction result determination submodule includes:

[0044] The first similarity data determination unit is used to calculate the similarity between each second prediction result and the first prediction result through the processor, and obtain the first similarity data corresponding to each second prediction result;

[0045] The target second prediction result determination unit is used to determine the target second prediction result from the second prediction results based on the first similarity data corresponding to each second prediction result; the first similarity data corresponding to the target second prediction result is greater than the similarity threshold.

[0046] In some optional embodiments, the first similarity data determination unit includes:

[0047] The transformation processing subunit is used to perform transformation processing on each second prediction result by the processor to obtain the transformed second prediction result corresponding to each second prediction result;

[0048] The first similarity data determination subunit is used by the processor to calculate the similarity between each transformed second prediction result and the first prediction result, so as to obtain the first similarity data corresponding to each second prediction result.

[0049] In some alternative embodiments, the apparatus further includes:

[0050] The model acquisition module is used to retrieve the target compression operator model corresponding to each operator model contained in the preset model from the memory;

[0051] The initial compression model determination module is used to combine each target compression operator model to obtain the initial compression model;

[0052] The initial compression prediction result determination module is used to input multimedia data into the preset model and the initial compression model respectively for prediction processing, and obtain the original prediction result output by the preset model and the initial compression prediction result output by the initial compression model.

[0053] The second similarity data determination module is used to calculate the similarity between the original prediction result and the initial compressed prediction result through the processor to obtain the second similarity data;

[0054] The target compression model determination module is used to determine the initial compression model as the target compression model corresponding to the preset model when the second similarity data meets the preset conditions.

[0055] In some alternative embodiments, the apparatus further includes:

[0056] The preset target compression operator model determination module is used to determine the preset target compression operator model in the initial compression model when the second similarity data does not meet the preset conditions; the preset target compression operator model is the target compression operator model in the initial compression model that meets the preset conditions.

[0057] The update compression model determination module is used to replace the preset target compression operator model in the initial compression model with the preset operator model corresponding to the preset target compression operator model, so as to obtain the updated compression model;

[0058] The module for determining the updated compression prediction results is used to input multimedia data into the preset model and the updated compression model respectively for prediction processing, and obtain the original prediction results output by the preset model and the updated compression prediction results output by the updated compression model.

[0059] The third similarity data determination module is used to calculate the similarity between the original prediction result and the updated compressed prediction result through the processor to obtain the third similarity data.

[0060] The target compression model determination module is used to determine the updated compression model as the target compression model corresponding to the preset model when the third similarity data meets the preset conditions.

[0061] On the other hand, embodiments of this application also provide an electronic device for data processing, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded by the processor and executed by the data processing method described above.

[0062] On the other hand, embodiments of this application also provide a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the data processing method described above.

[0063] The data processing method, apparatus, device, and storage medium proposed in this application retrieve a preset operator model and its corresponding candidate compression operator model from a preset model in a memory. Multimedia data is then input into both the preset operator model and each candidate compression operator model for prediction processing. This yields a first prediction result output by the preset operator model and a second prediction result output by each compression operator model. Based on the first and second prediction results, a target compression operator model corresponding to the preset operator model is determined from the candidate compression operator models. This improves the compression accuracy of each preset operator model within the preset model, reduces the accuracy loss of the compressed model, and thus improves the service processing efficiency of the terminal device. Furthermore, the candidate compression operator models are obtained by the processor compressing the preset operator model using at least two compression strategies, which improves the utilization of computing resources and the efficiency of model compression processing, thereby reducing system operating costs. Attached Figure Description

[0064] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a schematic diagram illustrating the implementation environment of a data processing method according to an exemplary embodiment.

[0066] Figure 2 This is a flowchart illustrating a data processing method according to an exemplary embodiment.

[0067] Figure 3This is a block diagram of a data processing apparatus according to an exemplary embodiment.

[0068] Figure 4 This is a hardware structure block diagram of a server for a data processing method provided according to an exemplary embodiment. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0070] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the present application described herein can be implemented in orders other than those illustrated or described herein. Thus, features defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments, unless otherwise stated, "a plurality of" means two or more. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0071] To make the objectives, technical solutions, and advantages disclosed in the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application.

[0072] With the development of deep learning, neural networks have been widely applied in various fields. While improving model performance, this has also introduced a huge number of parameters and computational demands. Applications require substantial memory and incur enormous computational overhead, thus limiting their use on devices with limited computing power. Model compression can accelerate inference and reduce hardware requirements. However, traditional model compression methods typically compress the entire neural network model to a fixed number of bits. This compression method fails to achieve a good balance between accuracy and performance, resulting in significant accuracy loss in the compressed neural network model. This leads to inaccurate predictions when applied to edge devices, reducing processing efficiency and wasting computational resources.

[0073] In view of this, embodiments of this application propose a data processing method, apparatus, device, and storage medium. By retrieving a preset operator model and its corresponding candidate compression operator model from a preset model in a memory, and inputting multimedia data into both the preset operator model and each candidate compression operator model for prediction processing, a first prediction result output by the preset operator model and a second prediction result output by each compression operator model are obtained. Then, based on the first prediction result and each second prediction result, a target compression operator model corresponding to the preset operator model is determined from the candidate compression operator models. This improves the compression accuracy of each preset operator model within the preset operator model, reduces the accuracy loss of the compressed model, and thus improves the service processing efficiency of the end-side device. Furthermore, the candidate compression operator model is obtained by the processor compressing the preset operator model based on at least two compression strategies, which can improve the utilization of computing resources and increase the efficiency of model compression processing.

[0074] Figure 1 This is a schematic diagram illustrating the implementation environment of a data processing method according to an exemplary embodiment. For example... Figure 1 As shown, the implementation environment may include at least a client 01 and a server 02. The client 01 and the server 02 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0075] Specifically, the server 02 can be used to retrieve a preset operator model and a corresponding candidate compression operator model from memory; wherein, the preset operator model is any one of the multiple operator models included in the preset model; the candidate compression operator model is obtained by the processor compressing the preset operator model based on at least two compression strategies. It is also used to input multimedia data into the preset operator model and each candidate compression operator model respectively for prediction processing, obtaining a first prediction result output by the preset operator model and a second prediction result output by each compression operator model. Furthermore, it is used to determine the target compression operator model corresponding to the preset operator model among the candidate compression operator models based on the first prediction result and each second prediction result. Optionally, the server 02 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0076] Specifically, the client 01 can be used to obtain a compressed model corresponding to a preset model from the server 02 and use the compressed model for business processing. Optionally, the client 01 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smart voice interaction device, smart home appliance, smartwatch, vehicle terminal, aircraft, etc., but is not limited to these. This application embodiment can be applied to various scenarios, including but not limited to rendering and displaying interactive information involved in various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0077] It should be noted that, Figure 1 This is just one example. Other implementation environments may also be included in other scenarios.

[0078] Figure 2 This is a flowchart illustrating a data processing method according to an exemplary embodiment. The method can be used for... Figure 1 In the implementation environment described herein, the method operation steps are provided as shown in the embodiments or flowcharts. However, based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server products, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or figures. This method can be applied to servers, specifically as follows: Figure 2 As shown, it includes:

[0079] S101: Retrieve the preset operator model and the candidate compression operator model corresponding to the preset operator model from the memory.

[0080] In this embodiment, the preset operator model is any one of the multiple operator models included in the preset model. The preset model is obtained by training the model using sample multimedia data. Optionally, the sample multimedia data can be any type of data such as text data, audio data, and image data. The preset model can be a model used for prediction based on multimedia data. As an example, the preset model can be an autonomous driving control model. By inputting environmental perception data collected by onboard sensors into the preset model for prediction processing, the preset model can output corresponding autonomous driving parameters. As another example, the preset model can be a generative language model. By inputting text data into the preset model for prediction processing, the preset model can output corresponding generated text.

[0081] In order to enable the preset model to be deployed on edge devices with limited computing power, after the preset model is trained, it can be compressed to obtain the target compressed model corresponding to the preset model.

[0082] Specifically, the preset model may include a preset number of neural network layers. Optionally, the preset number may be one layer, two layers, or more layers; this embodiment does not impose excessive restrictions on this. For each neural network layer, it can be compressed as a separate operator model, thereby improving the model compression ratio.

[0083] For any given operator model, there may be two or more candidate compression operator models. These candidate compression operator models are obtained by the processor compressing the operator model based on at least two compression strategies.

[0084] Specifically, the method for determining candidate compression operator models may include: obtaining a preset operator model and at least two compression strategies; and using a processor to compress the weight parameters and / or activation parameters in the preset operator model based on at least two compression strategies to obtain candidate compression operator models.

[0085] In this embodiment, when compressing a preset operator model, a compression strategy can be obtained first, and then the preset operator model can be compressed according to the determined compression strategy. Each compression strategy can include a compression type and a compression algorithm. The compression type includes, but is not limited to, symmetric compression, asymmetric compression, layer-by-layer compression, and channel-by-channel compression. The compression algorithm includes, but is not limited to, maximum-minimum algorithm, moving average maximum-minimum algorithm, and relative entropy (Kullback-Leibler divergence, KL distance) algorithm. When determining the compression strategy, a compression type can be selected from a pre-built compression type information table, and a compression algorithm can be selected from a compression algorithm information table, thereby combining them into a compression strategy. Following the above method, one or more compression strategies can be obtained, and these compression strategies can be combined to form a compression strategy set.

[0086] In this embodiment, when compressing a preset operator model, at least two candidate compression factors can be determined based on at least two compression strategies. Each compression strategy corresponds to one candidate compression factor. Then, the preset operator model is compressed based on each candidate compression factor. Specifically, each compression strategy yields a candidate compressed operator model after compressing the preset operator model. When compressing the preset operator model, the weight parameters and / or activation parameters corresponding to that preset operator model can be compressed. The weight parameters and activation parameters in the preset operator model are obtained based on model training. After compressing the preset operator model using each candidate compression factor, at least two candidate compressed operator models corresponding to the preset operator model can be obtained.

[0087] In this embodiment, when compressing a single operator model, one or more compression strategies can be selected from a set of compression strategies. Then, the selected compression strategies are used to compress the weights and activation parameters corresponding to the single operator model. Different compression strategies can produce different compressed operator models when compressing the single operator model. These different compressed operator models constitute the candidate compressed operator models corresponding to the preset operator model.

[0088] S103: Input the multimedia data into the preset operator model and each candidate compression operator model respectively for prediction processing to obtain the first prediction result output by the preset operator model and the second prediction result output by each compression operator model.

[0089] In this embodiment of the application, after obtaining the preset operator model and its corresponding candidate operator model, multimedia data can be used to detect the prediction accuracy of each candidate operator model, so that the candidate operator model whose prediction accuracy meets the requirements can be used as the compressed operator model corresponding to the preset operator model.

[0090] Specifically, multimedia data is first acquired, and then input into a preset operator model and each candidate operator model for prediction processing. Optionally, the multimedia data can be the model training data used when training the preset model, or data that can be input into the preset model for prediction. Before inputting the multimedia data into the preset operator model and each candidate compression operator model, the multimedia data needs to be embedded and represented to convert it into feature data that the model can process, such as feature vectors, and then the feature data is input into the preset operator model and each candidate compression operator model for prediction.

[0091] As an example, for the first-layer operator model in the preset model, after obtaining the feature data, the feature data is input into the operator model. The operator model uses its corresponding weight parameters to calculate the input feature data, thereby outputting the corresponding first prediction result. Similarly, for any candidate operator model, after obtaining the feature data, the feature data is input into the candidate operator model. The candidate operator model uses its corresponding weight parameters to calculate the input feature data, thereby outputting the corresponding second prediction result.

[0092] As another example, for a non-first-layer operator model in the preset model, after obtaining the feature data, the feature data is input into the first-layer operator model in the preset model to obtain the activation value output by the first-layer operator model. This activation value is then input into the next-layer operator model to obtain the activation value output by the next-layer operator model. This process is repeated until the activation value output by the previous layer of the current operator model is obtained. Then, the activation value output by the previous layer of the current operator model is input into the current operator model to obtain the first prediction result. For any candidate operator model, the activation value output by the previous layer of the current operator model is input into the candidate operator model to obtain the second prediction result output by the candidate operator model. For example, the current operator model is the third-layer neural network in the preset model. After obtaining the feature data, the feature data is input into the first-layer operator model in the preset model to obtain the activation value output by the first-layer operator model. This activation value is then input into the second-layer operator model to obtain the activation value output by the second-layer operator model. The activation values ​​output by the second-layer operator model are then input into the current operator model for prediction processing to obtain the first prediction result. Similarly, the activation values ​​output by the second-layer operator model are input into each candidate operator model for prediction processing to obtain the second prediction result output by each candidate operator model.

[0093] S105: Based on the first prediction result and each second prediction result, determine the target compression operator model corresponding to the preset operator model among the candidate compression operator models.

[0094] In this embodiment, among multiple candidate compression operator models corresponding to a preset operator model, the candidate compression operator model with high accuracy is selected as the target compression operator model corresponding to the preset operator model by screening these candidate compression operator models. Specifically, based on the first prediction result, a target second prediction result that meets the preset conditions is determined in each second prediction result. Then, the candidate compression operator model that outputs the target second prediction result is determined from the candidate compression operator models, thus obtaining the target compression operator model corresponding to the preset operator model.

[0095] When determining the target second prediction result that meets the preset conditions in each second prediction result, the processor can calculate the similarity between each second prediction result and the first prediction result to obtain the first similarity data corresponding to each second prediction result. Then, based on the first similarity data corresponding to each second prediction result, the target second prediction result is determined from the second prediction results. The first similarity data corresponding to the target second prediction result is greater than the similarity threshold. That is, when determining the target compression operator model corresponding to the preset operator model, the first prediction result output by the preset operator model and the second prediction result output by each candidate operator model are used to calculate the similarity, resulting in multiple first similarity data. Thus, among the candidate compression operator models, the candidate compression operator model whose first similarity data meets the preset conditions is determined as the target compression operator model corresponding to the preset operator model.

[0096] In this embodiment of the application, when screening candidate compression operator models according to preset conditions, the screening can be based on the first similarity data corresponding to each second prediction result to obtain the target compression operator model. Optionally, the preset condition can be that the first similarity data is greater than a threshold, or that the first similarity data is at its maximum. Screening based on the first similarity data corresponding to each second prediction result can ensure the compression accuracy of each layer of operator models, thereby improving the accuracy of the target compression model.

[0097] In this embodiment, the second prediction result is integer data, while the first prediction result is typically floating-point data. Furthermore, when compressing the first prediction result, it may be numerically mapped, meaning the second and first prediction results may not necessarily reside in the same numerical space. Therefore, to calculate the first similarity data corresponding to the second prediction result, the second prediction result needs to be decompressed before the first similarity data can be calculated. Specifically, the processor performs a transformation process on each second prediction result to obtain a transformed second prediction result. Then, the processor calculates the similarity between each transformed second prediction result and the first prediction result to obtain the first similarity data corresponding to each second prediction result.

[0098] When calculating the similarity between the first prediction result output by the preset operator model and the second prediction result output by each candidate operator model, it is necessary to first perform inverse compression on each second prediction result to obtain the inverse compressed prediction result corresponding to each second prediction result. Then, similarity calculation is performed based on the first prediction result output by the preset operator model and the inverse compressed prediction result corresponding to each second prediction result to obtain the first similarity data corresponding to each second prediction result. The similarity calculation for the second prediction result and the first prediction result can be performed according to the following formula:

[0099]

[0100] Where S is the first similarity data, x i Let y be the i-th component of the N-dimensional vector X. i Let Y be the i-th component of the N-dimensional vector Y, let X be the inverse compression prediction result corresponding to the second prediction result, and let Y be the first prediction result output by the preset operator model.

[0101] In this embodiment of the application, after obtaining the target compression operator model corresponding to each operator model in the preset model, the target compression operator model corresponding to each operator model can be stored in the memory so as to facilitate subsequent compression or decompression of each operator model in the preset model.

[0102] In this embodiment, after obtaining the target compression operator models corresponding to all operator models in the preset model, the target compression operator models can be combined according to their positions in the preset model to obtain the initial compression model corresponding to the preset model. To ensure the accuracy of the compressed model, the accuracy of the initial compression model can be judged. If the accuracy meets the requirements, the initial compression model is used as the target compression model of the preset model.

[0103] Specifically, the target compression operator model corresponding to each operator model contained in the preset model is retrieved from memory. Then, each target compression operator model is combined to obtain an initial compression model. Multimedia data is input into the preset model and the initial compression model respectively for prediction processing, yielding the original prediction result output by the preset model and the initial compressed prediction result output by the initial compression model. Then, the processor calculates the similarity between the original prediction result and the initial compressed prediction result to obtain second similarity data. If the second similarity data meets preset conditions, the initial compression model is determined as the target compression model corresponding to the preset model.

[0104] Specifically, after obtaining the target compression operator model corresponding to each operator model in the preset model, the target compression operator models corresponding to each operator model can be connected according to the connection order of each layer in the preset model to obtain the initial compression model. To ensure compression accuracy, multimedia data can be used to determine the accuracy of the initial compression model. That is, multimedia data is input into the initial compression model and the preset model for processing. After obtaining the original prediction result and the initial compressed prediction result, the accuracy of the initial compression model is judged by calculating the similarity between the two. The calculation of the similarity between the original prediction result and the initial compressed prediction result can refer to the similarity calculation method of the first prediction result and the second prediction result output by the preset operator model mentioned above, and will not be repeated here.

[0105] In this embodiment, after obtaining the second similarity data, the accuracy can be judged by using the condition that the second similarity data is greater than the similarity threshold. If the second similarity data is greater than the similarity threshold, the initial compression model can be determined as the target compression model corresponding to the preset model.

[0106] In this embodiment of the application, if the second similarity data does not meet the preset conditions, that is, if the second similarity data is less than or equal to the similarity threshold, the initial compressed model can be updated according to the first similarity data in order to ensure the accuracy of model compression.

[0107] Specifically, if the second similarity data does not meet the preset conditions, a preset target compression operator model is determined in the initial compression model. The preset target compression operator model is the target compression operator model in the initial compression model that meets the preset conditions. Then, the preset target compression operator model in the initial compression model is replaced with the preset operator model corresponding to the preset target compression operator model, resulting in an updated compression model. Next, multimedia data is input into the preset model and the updated compression model respectively for prediction processing, obtaining the original prediction result output by the preset model and the updated compressed prediction result output by the updated compression model. The similarity between the original prediction result and the updated compressed prediction result is calculated by the processor to obtain the third similarity data. If the third similarity data meets the preset conditions, the updated compression model is determined as the target compression model corresponding to the preset model.

[0108] In other words, if the accuracy of the initial compression model does not meet the accuracy requirements, the first similarity data corresponding to the target compression operator model for each layer in the initial compression model is determined. Then, the layer with the first similarity data less than the similarity threshold or the smallest first similarity data is identified and replaced from the target compression operator model to the original operator model, thus obtaining the updated compression model. Multimedia data is also used to test the accuracy of the updated compression model. If the accuracy of the updated compression model meets the requirements, it is used as the target compression model. If the accuracy of the updated compression model does not meet the requirements, the process of replacing the layer with the first similarity data less than the similarity threshold or the smallest first similarity data with the original operator model is repeated until the accuracy of the updated compression model meets the requirements or all layers of the updated compression model are replaced with the original operator model.

[0109] The data processing method described in this application compresses each layer of the neural network in the preset model individually, determines the target compression strategy with the highest first similarity data, and compresses the corresponding operator model, ensuring the compression accuracy of each operator model. For layers with low first similarity data, compression is canceled to maintain the original accuracy. Through a hybrid accuracy strategy, the final accuracy of the compressed model is ensured to meet the target. This method achieves model compression while ensuring the accuracy of the target compressed model, and can automatically adjust the compression strategy of each operator model, avoiding the resource consumption caused by manual search, thus improving model compression efficiency and reducing model compression costs. Furthermore, since the final target compressed model has high prediction accuracy, it can be applied to terminal devices such as smartphones, tablets, laptops, desktop computers, smart speakers, smart voice interaction devices, smart home appliances, smartwatches, in-vehicle terminals, and aircraft, enabling these devices to better perform multimedia-related services, such as content generation, audio-visual recommendation, and voice and video optimization.

[0110] This application also provides a data processing apparatus. Figure 3 This is a block diagram of a data processing apparatus according to an exemplary embodiment. Figure 3 As shown, the device may include at least:

[0111] The acquisition module 201 is used to acquire a preset operator model and a candidate compression operator model corresponding to the preset operator model from the memory; wherein, the preset operator model is any one of the multiple operator models included in the preset model; the candidate compression operator model is obtained by the processor compressing the preset operator model based on at least two compression strategies;

[0112] The prediction processing module 203 is used to input multimedia data into a preset operator model and each candidate compression operator model respectively for prediction processing, and obtain the first prediction result output by the preset operator model and the second prediction result output by each compression operator model.

[0113] The target compression operator model determination module 205 is used to determine the target compression operator model corresponding to the preset operator model from among the candidate compression operator models based on the first prediction result and each second prediction result.

[0114] In some optional embodiments, the apparatus further includes a candidate compression operator model determination module, the candidate compression operator model determination module comprising:

[0115] The acquisition submodule is used to acquire the preset operator model and at least two compression strategies;

[0116] The compression processing submodule is used to compress the weight parameters and / or activation parameters in the preset operator model using at least two compression strategies to obtain candidate compressed operator models.

[0117] In some optional embodiments, the target compression operator model determination module includes:

[0118] The second prediction result determination submodule is used to determine the second prediction result of the target that meets the preset conditions in each second prediction result based on the first prediction result.

[0119] The target compression operator model determination submodule is used to determine the candidate compression operator model that outputs the second prediction result of the target from the candidate compression operator models, and obtain the target compression operator model corresponding to the preset operator model.

[0120] In some optional embodiments, the target second prediction result determination submodule includes:

[0121] The first similarity data determination unit is used to calculate the similarity between each second prediction result and the first prediction result through the processor, and obtain the first similarity data corresponding to each second prediction result;

[0122] The target second prediction result determination unit is used to determine the target second prediction result from the second prediction results based on the first similarity data corresponding to each second prediction result; the first similarity data corresponding to the target second prediction result is greater than the similarity threshold.

[0123] In some optional embodiments, the first similarity data determination unit includes:

[0124] The transformation processing subunit is used to perform transformation processing on each second prediction result by the processor to obtain the transformed second prediction result corresponding to each second prediction result;

[0125] The first similarity data determination subunit is used by the processor to calculate the similarity between each transformed second prediction result and the first prediction result, so as to obtain the first similarity data corresponding to each second prediction result.

[0126] In some alternative embodiments, the apparatus further includes:

[0127] The model acquisition module is used to retrieve the target compression operator model corresponding to each operator model contained in the preset model from the memory;

[0128] The initial compression model determination module is used to combine each target compression operator model to obtain the initial compression model;

[0129] The initial compression prediction result determination module is used to input multimedia data into the preset model and the initial compression model respectively for prediction processing, and obtain the original prediction result output by the preset model and the initial compression prediction result output by the initial compression model.

[0130] The second similarity data determination module is used to calculate the similarity between the original prediction result and the initial compressed prediction result through the processor to obtain the second similarity data;

[0131] The target compression model determination module is used to determine the initial compression model as the target compression model corresponding to the preset model when the second similarity data meets the preset conditions.

[0132] In some alternative embodiments, the apparatus further includes:

[0133] The preset target compression operator model determination module is used to determine the preset target compression operator model in the initial compression model when the second similarity data does not meet the preset conditions; the preset target compression operator model is the target compression operator model in the initial compression model that meets the preset conditions.

[0134] The update compression model determination module is used to replace the preset target compression operator model in the initial compression model with the preset operator model corresponding to the preset target compression operator model, so as to obtain the updated compression model;

[0135] The module for determining the updated compression prediction results is used to input multimedia data into the preset model and the updated compression model respectively for prediction processing, and obtain the original prediction results output by the preset model and the updated compression prediction results output by the updated compression model.

[0136] The third similarity data determination module is used to calculate the similarity between the original prediction result and the updated compressed prediction result through the processor to obtain the third similarity data.

[0137] The target compression model determination module is used to determine the updated compression model as the target compression model corresponding to the preset model when the third similarity data meets the preset conditions.

[0138] It should be noted that the data processing device embodiments provided in this application are based on the same inventive concept as the data processing method embodiments described above.

[0139] This application also provides an electronic device for data processing, which includes a processor and a memory. The memory stores at least one instruction or at least one program. The processor loads and executes the at least one instruction or at least one program to implement the data processing method provided in any of the above embodiments.

[0140] Embodiments of this application also provide a computer-readable storage medium that can be disposed in a terminal to store at least one instruction or at least one program for implementing a data processing method in the method embodiments, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the data processing method provided in the above method embodiments.

[0141] Optionally, in the embodiments of this specification, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0142] The memory described in this specification can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for functions, etc.; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0143] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing method provided in the above-described method embodiments.

[0144] The methods and embodiments provided in this application can be executed on a terminal, computer terminal, server, or similar computing device. Taking running on a server as an example, Figure 4 This is a hardware structure block diagram of a server for a data processing method provided according to an exemplary embodiment. For example... Figure 4As shown, the server 300 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 310 (CPUs 310 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 330 for storing data, and one or more storage media 320 (e.g., one or more mass storage devices) for storing application programs 323 or data 322. The memory 330 and storage media 320 may be temporary or persistent storage. The program stored in the storage media 320 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 310 may be configured to communicate with the storage media 320 and execute the series of instruction operations stored in the storage media 320 on the server 300. Server 300 may also include one or more power supplies 360, one or more wired or wireless network interfaces 350, one or more input / output interfaces 340, and / or one or more operating systems 321, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0145] The input / output interface 340 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 300. In one example, the input / output interface 340 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 340 may be a radio frequency (RF) module for wireless communication with the Internet.

[0146] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 300 may also include... Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown.

[0147] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and server embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0149] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0150] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A data processing method, characterized in that, The method includes: Retrieve a preset operator model and a candidate compression operator model corresponding to the preset operator model from the memory; wherein the preset operator model is any one of the multiple operator models included in the preset model; the candidate compression operator model is obtained by the processor compressing the weight parameters and / or activation parameters corresponding to the preset operator model based on at least two compression strategies; Multimedia data is input into the preset operator model and each candidate compression operator model for prediction processing, resulting in a first prediction result output by the preset operator model and a second prediction result output by each candidate compression operator model; wherein, the first prediction result is floating-point data and the second prediction result is integer data; the multimedia data is one of text data, audio data, and image data; The processor calculates the similarity between each second prediction result and the first prediction result to obtain the first similarity data corresponding to each second prediction result. Based on the first similarity data corresponding to each second prediction result, a target second prediction result is determined from the second prediction results; the first similarity data corresponding to the target second prediction result is greater than a similarity threshold; Among the candidate compression operator models, a candidate compression operator model that outputs the second prediction result of the target is determined, and the target compression operator model corresponding to the preset operator model is obtained.

2. The method according to claim 1, characterized in that, The method for determining the candidate compression operator model includes: Obtain the preset operator model and at least two compression strategies; The processor compresses the weight parameters and / or activation parameters in the preset operator model based on at least two of the compression strategies to obtain the candidate compression operator model.

3. The method according to claim 1, characterized in that, The step of calculating the similarity between each second prediction result and the first prediction result using a processor to obtain first similarity data corresponding to each second prediction result includes: The processor performs a transformation process on each second prediction result to obtain a transformed second prediction result corresponding to each second prediction result. The processor calculates the similarity between each of the transformed second prediction results and the first prediction result to obtain the first similarity data corresponding to each second prediction result.

4. The method according to claim 1, characterized in that, After obtaining the target compression operator model corresponding to the preset operator model, the method further includes: Retrieve the target compression operator model corresponding to each operator model contained in the preset model from the memory; The initial compression model is obtained by combining each of the target compression operator models. Multimedia data is input into the preset model and the initial compression model respectively for prediction processing to obtain the original prediction result output by the preset model and the initial compression prediction result output by the initial compression model. The processor calculates the similarity between the original prediction result and the initial compressed prediction result to obtain second similarity data; If the second similarity data meets the preset conditions, the initial compression model is determined as the target compression model corresponding to the preset model.

5. The method according to claim 4, characterized in that, After calculating the similarity between the original prediction result and the initial compressed prediction result using a processor to obtain second similarity data, the method further includes: If the second similarity data does not meet the preset conditions, a preset target compression operator model is determined in the initial compression model; the preset target compression operator model is the target compression operator model in the initial compression model that meets the preset conditions. In the initial compression model, the preset target compression operator model is replaced with the corresponding preset operator model to obtain the updated compression model; Multimedia data is input into the preset model and the updated compression model respectively for prediction processing to obtain the original prediction result output by the preset model and the updated compression prediction result output by the updated compression model. The processor calculates the similarity between the original prediction result and the updated compressed prediction result to obtain third similarity data; If the third similarity data meets the preset conditions, the updated compression model is determined as the target compression model corresponding to the preset model.

6. A data processing apparatus, characterized in that, The device includes: An acquisition module is used to acquire a preset operator model and a candidate compressed operator model corresponding to the preset operator model from a memory; wherein, the preset operator model is any one of the multiple operator models included in the preset model; the candidate compressed operator model is obtained by the processor compressing the weight parameters and / or activation parameters corresponding to the preset operator model based on at least two compression strategies; The prediction processing module is used to input multimedia data into the preset operator model and each candidate compression operator model respectively for prediction processing, to obtain a first prediction result output by the preset operator model and a second prediction result output by each candidate compression operator model; wherein, the first prediction result is floating-point data and the second prediction result is integer data; the multimedia data is at least one of text data, audio data, and image data; The target compression operator model determination module is used to calculate the similarity between each second prediction result and the first prediction result through the processor, and obtain the first similarity data corresponding to each second prediction result; Based on the first similarity data corresponding to each second prediction result, a target second prediction result is determined from the second prediction results; the first similarity data corresponding to the target second prediction result is greater than a similarity threshold; Among the candidate compression operator models, a candidate compression operator model that outputs the second prediction result of the target is determined, and the target compression operator model corresponding to the preset operator model is obtained.

7. An electronic device for data processing, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded by the processor and executed as the data processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the data processing method as described in any one of claims 1 to 5.

Images