Data processing method and electronic device

Abstract

The embodiment of the present application is applied to the field of artificial intelligence, and provides a data processing method and electronic device, the data processing method comprising: based on the initial data, determining the input data of the first processing unit through a data processing model, the data processing model comprising the first processing unit, the first processing unit comprising a bit width selector and a first operator; through the bit width selector, based on the input data, determining the target bit width of the first operator from multiple candidate bit widths of the first operator; mapping the input data to first input adjustment data with a bit width of a target bit width through the data processing model; outputting the processing result corresponding to the first processing unit through the first operator of the first processing unit based on the target parameter of the target bit width in multiple parameters and the first input adjustment data; determining the target processing result based on the processing result corresponding to the first processing unit through the data processing model. Based on the technical method of the present application, the accuracy of reasoning can be improved and labor costs can be reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a data processing method and an electronic device. Background Art

[0002] In digital signal processing, quantization is the process of mapping input values ​​from a large set (usually a continuous set) to a smaller set (usually with a finite number of elements). In the field of machine learning, especially neural networks, this mapping is usually from floating point values ​​to integer values.

[0003] Neural network quantization is an effective technique that can significantly reduce model size, improve computational efficiency, and reduce power consumption while maintaining model performance. However, in the process of neural network quantization, the bit width of each operator in the neural network after quantization is usually determined by manual analysis.

[0004] The method of determining the bit width through manual analysis is inefficient, and the reasoning accuracy of the quantized neural network model is low. Summary of the invention

[0005] The present application provides a data processing method that can improve the accuracy of data processing model reasoning and reduce labor costs.

[0006] In a first aspect, a data processing method is provided, comprising: determining input data of a first processing unit through a data processing model based on initial data, the data processing model comprising a first processing unit, the first processing unit comprising a bit width selector and a first operator; determining a target bit width of the first operator from multiple candidate bit widths of the first operator based on the input data through the bit width selector of the first processing unit; selecting a target parameter whose bit width is the target bit width from multiple parameters of the first operator through the data processing model, different parameters of the first operator corresponding to different candidate bit widths of the first operator; mapping the input data to first input adjustment data through the data processing model, the bit width of the first input adjustment data being the target bit width; outputting a processing result corresponding to the first processing unit through the first operator of the first processing unit based on the target parameter and the first input adjustment data; determining a target processing result based on the processing result corresponding to the first processing unit through the data processing model.

[0007] In the data processing method provided by the embodiment of the present application, the bit width selector in the first processing unit processes the input data of the first processing unit to determine the target bit width from the multiple candidate bit widths of the first operator in the first processing unit, and maps the input data of the first processing unit to the first input adjustment data with the target bit width through the data processing model, so that the first operator can determine the processing result corresponding to the first processing unit based on the first input adjustment data and the target parameters in the multiple parameters corresponding to the multiple candidate bit widths one by one, and the processing result corresponding to the first processing unit is used for the data processing model to determine the target processing result, so that the target bit width of the first operator can be set according to the input data of the first processing unit, and the target bit width of the first operator is more suitable for processing the input data of the first processing unit, thereby improving the accuracy of the data processing model reasoning. In addition, the bit width of the first operator is no longer set based on manual experience, which reduces the dependence on manual labor and reduces labor costs.

[0008] In some possible implementations, the data processing model also includes a second operator, and the output of the second operator serves as the input of the first processing unit; determining the input data of the first processing unit through the data processing model based on the initial data includes: quantizing the initial data through the data processing model to obtain second input adjustment data, the bit width of the second input adjustment data being the first bit width, and the first bit width being greater than or equal to the maximum candidate bit width of the first operator in the first processing unit; and determining the input data of the first processing unit based on the second input adjustment data through the second operator.

[0009] In the data processing model, the second operator can be understood as the first operator. The features extracted by the first operator in the data processing model are more important, so the parameters of the second operator in the data processing model can be quantized to a larger bit width, that is, the first bit width. The second operator can process data with a larger bit width, that is, the second input adjustment data of the first bit width, thereby improving the accuracy of the data processing model reasoning, that is, improving the accuracy of the target processing result.

[0010] In some possible implementations, the data processing model also includes a second processing unit and a third processing unit, the second processing unit includes a third operator, the third processing unit includes at least one fourth operator, the output of the first processing unit is used as the input of the second processing unit, and the output of the second processing unit is used as the input of the third processing unit; the target processing result is determined based on the processing result corresponding to the first processing unit through the data processing model, including: mapping the processing result corresponding to the first processing unit to third input adjustment data through the second processing unit, the bit width of the third input adjustment data is the first bit width; processing the third input adjustment data through the third operator in the second processing unit; dequantizing the output data of the third operator through the second processing unit to obtain first dequantized data; determining the target processing result based on the first dequantized data through the at least one fourth operator, the bit width of the input of each fourth operator is equal to the bit width of the first dequantized data.

[0011] The third operator can be understood as the last quantized operator in the data processing model. The features extracted by the last quantized operator have a greater impact on the accuracy of the target processing result. Therefore, the parameters of the third operator can be quantized to a larger bit width, that is, the first bit width. The third operator can process data with a larger bit width, that is, the third input adjustment data of the first bit width, thereby improving the accuracy of the data processing model reasoning, that is, improving the accuracy of the target processing result.

[0012] In some possible implementations, the bit width selector of the first processing unit determines the target bit width of the first operator from multiple candidate bit widths of the first operator based on the input data, including: performing global pooling processing based on the input data to determine global information through the bit width selector; performing feature extraction on the global information to obtain input data features through the bit width selector; and determining the target bit width according to the input data features through the bit width selector.

[0013] By globally pooling the input data and performing feature extraction and classification on the results of global pooling, the occupation of processing resources by the data processing process of the bit width selector is reduced, thereby reducing the total occupation of processing resources by the data processing model for processing the initial data and improving operating efficiency.

[0014] In some possible implementations, the performing of global average pooling processing based on the input data through the bit width selector to determine the global average information also includes: performing inverse quantization on the input data to obtain second inverse quantized data; performing global pooling processing based on the input data through the bit width selector to determine the global information, including: performing global pooling processing on the second inverse quantized data through the bit width selector to obtain the global information.

[0015] By processing the inversely quantized input data, that is, the second inversely quantized data, the bit width selector can more conveniently process the input data.

[0016] In some possible implementations, mapping the input data to first input adjustment data through the data processing model includes: quantizing the second dequantized data based on the target bit width through the data processing model to obtain the first input adjustment data.

[0017] When the bit width selector dequantizes the input data to obtain second dequantized data, the data processing model can quantize the second dequantized data to obtain first input adjustment data, thereby making it easier to implement the mapping of the input data to the first input adjustment data of the target bit width.

[0018] In some possible implementations, there are multiple first processing units in the data processing model, and the processing results corresponding to each of the multiple first processing units serve as input data for the next first processing unit; determining the target processing result based on the processing results corresponding to the first processing units through the data processing model includes: after determining the processing result corresponding to the first first processing unit, continuing to determine the processing results corresponding to other first processing units through other first processing units among the multiple first processing units; until the last first processing unit is passed, the processing result corresponding to the last first processing unit is determined; and determining the target processing result based on the processing result corresponding to the last first processing unit through the data processing model.

[0019] In the data processing model, the bit widths of the parameters of the consecutive multiple first operators can be flexibly adjusted according to the input data of the first processing unit where the first operator is located, so that the reasoning process of the data processing model occupies less processing resources and the reasoning results of the data processing model are more accurate. In addition, the bit widths of the multiple first operators do not need to be set manually, reducing labor costs.

[0020] In some possible implementations, the data processing model is obtained by adjusting the model parameters in the initial data processing model according to the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, as well as the operator training information of the first initial operator; the initial data processing model includes a first initial processing unit, and the first initial processing unit includes an initial bit width selector and the first initial operator; the training information corresponding to the training sample is determined by the initial data processing model based on the processing result output by the first initial processing unit, and the processing result output by the first initial processing unit is the training input corresponding to the training bit width of the first initial operator based on the training target parameter of the first initial operator The training input adjustment data is processed and determined, the training input adjustment data is obtained by mapping the training input data of the first initial processing unit according to the training bit width of the first initial operator; the training bit width of the first initial operator is determined by the initial bit width selector based on the training input data of the first initial processing unit, among multiple candidate bit widths of the first initial operator, the training target parameter of the first initial operator is the parameter corresponding to the training bit width among the multiple initial parameters of the first initial operator, and different initial parameters of the first initial operator correspond to different candidate bit widths; the training input data of the first initial processing unit is determined according to the training samples, and the operator training information of the first initial operator is positively correlated with the training bit width of the first initial operator.

[0021] During the training of the data processing model, the operator training information of the first initial operator which is positively correlated with the training bit width of the first initial operator is considered, so that the trained data processing model can better balance the contradiction between the reasoning accuracy and the occupation of processing resources by the reasoning process.

[0022] In some possible implementations, the operator training information of the first initial operator includes the training bit width of the first initial operator and / or the computational complexity of the first initial operator processing the training input adjustment data.

[0023] In some possible implementations, there are multiple first initial processing units in the initial data processing model, and the processing results output by each of the multiple first initial processing units serve as input data for the next first initial processing unit. The training information corresponding to the training sample is determined based on the processing results output by the last first initial processing unit.

[0024] In some possible implementations, the operator training information of the first initial operator includes the training bit width of the first initial operator and the computational amount of the first initial operator processing the training input adjustment data; the data processing model is obtained by adjusting the model parameters in the initial data processing model according to the difference between the operator training information corresponding to the training sample and the label information corresponding to the training sample, the difference between the sum of the training bit widths of multiple first initial operators and the second bit width, and the difference between the sum of the computational amount of multiple first initial operators and the first computational amount.

[0025] In the initial data processing model, when there are multiple first initial processing units, the adjustment of the model parameters of the initial data processing model, in addition to considering that the data processing model is based on the difference between the operator training information corresponding to the training samples and the label information corresponding to the training samples, also considers the difference between the sum of the training bit widths of the first initial operators in the multiple first initial units and the second bit width, and the difference between the sum of the computational amount of the multiple first initial operators and the first computational amount, so that the trained data processing model can better balance the contradiction between the inference accuracy and the occupation of processing resources by the inference process.

[0026] In a second aspect, a training method for a data processing model is provided, comprising: determining input data of a first initial processing unit based on a training sample through an initial data processing model, the initial data processing model comprising a first initial processing unit, the first initial processing unit comprising an initial bit width selector and a first initial operator; determining a training bit width of the first initial operator from a plurality of candidate bit widths of the first initial operator based on the training input data of the first initial processing unit through the initial bit width selector in the first initial processing unit; mapping the training input data of the first initial operator to first training input adjustment data corresponding to the training bit width of the first initial operator through the initial data processing model; mapping the training input data of the first initial operator to first training input adjustment data corresponding to the training bit width of the first initial operator based on the first initial operator The first training input adjustment data corresponding to the training bit width of the first initial operator is processed to determine the processing result output by the first initial processing unit, the training target parameter of the first initial operator is the parameter corresponding to the training bit width among the multiple initial parameters of the first initial operator, and different initial parameters of the first initial operator correspond to different candidate bit widths; through the initial data processing model, based on the processing result output by the first initial processing unit, the training information corresponding to the training sample is determined; according to the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, as well as the operator training information of the first initial operator, the model parameters in the initial data processing model are adjusted to obtain the data processing model.

[0027] In a third aspect, a data processing device is provided, comprising a unit for executing the method of the first aspect and / or the second aspect. The device may be a terminal device or a chip in the terminal device.

[0028] In a fourth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method of the first aspect and / or the second aspect.

[0029] In a fifth aspect, a chip is provided, comprising a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to implement the method of the first aspect and / or the second aspect.

[0030] In a sixth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code, and the computer program code is used to implement the method of the first aspect and / or the second aspect.

[0031] In a seventh aspect, a computer program product is provided, the computer program product comprising: a computer program code, wherein the computer program code is used to implement the method of the first aspect and / or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic structural diagram of a data processing model provided in an embodiment of the present application;

[0033] Figure 2 is a schematic flow chart of a method for training a data processing model provided in an embodiment of the present application;

[0034] Figure 3 It is a structural schematic diagram of an initial model;

[0035] Figure 4 and Figure 5 is a schematic diagram of the processing performed by the data processing model and the initial data processing model;

[0036] Figure 6 is a schematic flow chart of a data processing method provided in an embodiment of the present application;

[0037] Figure 7 is a schematic structural diagram of a bit width selector provided in an embodiment of the present application;

[0038] Figure 8 is a schematic diagram of a hardware system of an electronic device applicable to the present application;

[0039] Fig. 9 is a schematic diagram of a software system for an electronic device applicable to the present application;

[0040] Fig.10 It is a schematic structural diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0042] It should be understood that the "at least one" mentioned in this application refers to one or more, and "multiple" refers to two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, the words "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not limit them to be different.

[0043] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0044] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties (for example, the user's explicit consent, effective notification to the user, etc.), and the collection, use and processing of relevant data must comply with the relevant regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0045] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0046] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0047] Machine learning is an important branch of artificial intelligence, and deep learning is an important branch of machine learning. Deep learning refers to the use of multi-layer neural network structures to learn from big data the representation of various things in the real world that can be directly used for computer calculations (for example, things in images, sounds in audio, etc.).

[0048] The neural network model is a mathematical method that simulates the actual human neural network. The neural network model is a complex network system formed by a large number of simple processing units (called neurons or neural units) that are widely connected to each other. The neural network model is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neural units.

[0049] A deep neural network (DNN) can also be called a multi-layer neural network, which can be understood as a neural network with multiple hidden layers. According to the position of different layers, the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. In a deep neural network, more hidden layers allow the network to better depict complex situations in the real world. In theory, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by many layers of vectors).

[0050] Neural network models can be applied in image recognition, computer vision, natural language processing, etc. For example, images can be recognized and classified through neural network models, thereby realizing face recognition, object detection, handwriting font recognition, etc. Video data can be processed through neural network models to realize behavior analysis, video monitoring, etc. Neural networks can process human speech, realize speech-to-text conversion, and recognize and execute voice commands. Using neural networks for machine translation can realize automatic translation between multiple languages ​​and improve the accuracy and fluency of translation. Neural networks can generate natural and fluent text, which is applied to intelligent writing, intelligent customer service and other fields.

[0051] In digital signal processing, quantization refers to the process of mapping input values ​​from a large set (usually a continuous set) to a smaller set (usually with a finite number of elements). In the field of machine learning, especially neural networks, this mapping is usually from floating point values ​​to integer values. The data obtained by mapping floating point data can also be called fixed point data or integer data.

[0052] Quantization of neural networks is an effective technique that can significantly reduce model size, improve computational efficiency, and reduce power consumption while maintaining model performance.

[0053] Deep neural network models usually contain a large number of parameters, which take up a lot of storage space when stored as floating point numbers. Quantization can significantly reduce the size of the model by reducing the number of bits per parameter.

[0054] Fixed-point operations are generally faster and use less energy than floating-point operations, especially on mobile devices and embedded systems. Quantized models can achieve faster inference speeds on these platforms.

[0055] Operators in neural networks refer to functions or classes used to perform various mathematical operations and operations. These operators are usually used to build various layers and components of neural networks to achieve data transmission, conversion and calculation. They are the basic building blocks of deep learning models and define the structure and operation process of the model, including input, output and intermediate calculations.

[0056] In the process of neural network quantization, the bit width of each operator in the neural network after quantization is usually determined through manual analysis.

[0057] The method of determining the bit width by manual analysis is inefficient, and the reasoning accuracy of the quantized neural network model is low. Especially for mixed precision quantization, the manual analysis method of determining the bit width is labor-intensive and labor-intensive, and the labor cost is high.

[0058] Mixed-precision quantization (MPQ) can be applied in machine learning, especially in the field of deep learning. Mixed-precision quantization means that in a neural network model, different layers or parameters are represented and calculated using data types of different precision (such as 8-bit integers, 16-bit half-precision floating-point numbers, etc.). The purpose of this technology is to reduce the storage occupancy and computational complexity of the model while maintaining the accuracy of the model, thereby improving the operating efficiency of the model, reducing resource consumption, and achieving a balance between model size, operating efficiency, and inference accuracy. Mixed-precision quantization is a commonly used model compression method.

[0059] In digital signal processing, quantization refers to the process of approximating the continuous value (or a large number of possible discrete values) of a signal to a finite number (or fewer) of discrete values. Model quantization can significantly reduce the size and computational complexity of the model, thereby improving operational efficiency.

[0060] In the implementation process of mixed-precision quantization, determining the quantization bit width, that is, determining the accuracy of the data, is a very important step.

[0061] The quantization bit width can be determined through manual analysis. That is, by using deployment experience and hardware support, the weight distribution range and quantization sensitivity of the model are analyzed layer by layer, and the impact of different bit width combinations on model performance and accuracy is analyzed to determine the bit width corresponding to each layer. This requires a lot of work and has high labor costs.

[0062] Moreover, the quantized neural network model cannot be well adapted to process data of different complexity.

[0063] For example, for a neural network model that recognizes objects in images, the brightness of the images input to the neural network model is inconsistent, resulting in different levels of complexity in object recognition. Static mixed-precision quantization methods are difficult to achieve the best trade-off between efficiency and accuracy in a variety of scenarios. The quantized neural network model obtained by the static mixed-precision quantization method has low inference accuracy.

[0064] In order to solve the above problems, an embodiment of the present application provides a data processing model. Figure 1 It is a schematic structural diagram of a data processing model provided in an embodiment of the present application.

[0065] The data processing model 300 can be used to process initial data to obtain a target processing result.

[0066] The data processing model 300 includes N+M operators and N-2 bit width selectors, where N is a positive integer greater than or equal to 3, and M is a positive integer.

[0067] Among the first N operators of the data processing model 300, the bit width of the parameters of the first operator 311 and the parameters of the Nth operator 314 may be 8 bits (binary digit, bit). Other operators (such as the second operator 312 and the N-1th operator 313) may be set with parameters of multiple candidate bit widths. The multiple candidate bit widths may be 2 bits, 4 bits, and 8 bits, respectively.

[0068] The initial data can be floating point data, and the bit width can be 32 bits. Figure 1 and Figure 4 As shown in (b) in FIG. 3 , the data processing model 300 quantizes the initial data to obtain the first input data. The bit width of the first input data is 8 bits.

[0069] The first operator 311 is used to process the first input adjustment information obtained by quantization. The bit width of the parameter in the first operator 311 is 8 bits, and the bit width of the first output information of the first operator 311 is also 8 bits.

[0070] like Figure 1 and Figure 5 As shown in (b), different operators from the 2nd operator to the N-1th operator correspond to different bit width selectors. When n is an integer greater than 1 and less than N, that is, n≥2 and n≤N-1, the nth operator corresponds to the nth bit width selector.

[0071] The data processing model 300 performs inverse quantization on the n-1th output information output by the n-1th operator. The inverse quantized n-1th output information may be 32 bits.

[0072] The nth bit width selector is used to process the inverse quantized (n-1)th output information to obtain the nth bit width.

[0073] The data processing model 300 quantizes the inversely quantized (n-1)th output information to the nth bit width to obtain the nth input data of the nth bit width, and selects the nth bit width parameter from a plurality of candidate bit width parameters of the nth operator.

[0074] For the nth operator, a parameter corresponding to each candidate bit width may be set in the data processing model 300. The bit width of the parameter corresponding to each candidate bit width is equal to the candidate bit width.

[0075] The data processing model may determine a parameter of the nth bit width from parameters corresponding to a plurality of candidate bit widths of the nth operator.

[0076] The nth operator is used to process the nth input data of the nth bit width based on the nth bit width parameter of the nth operator to obtain the nth output information of the nth bit width.

[0077] When n=2, the n-th bit width selector is the second bit width selector 322, and the n-th operator is the second operator 312. When n=N-1, the n-th bit width selector is the N-1-th bit width selector 323, and the n-th operator is the N-1-th operator 313.

[0078] Each bit width selector in the 2nd bit width selector to the N-1th bit width selector may have the same or different structures. In the case where the structure of each bit width selector is the same, the parameters of different bit width selectors may be the same or different.

[0079] Each bit width selector can be understood as a classifier. A classifier (classnfner) is a neural network model used to map input data into predefined categories. For the nth bit width selector, the predefined categories can be multiple candidate bit widths. The output of the bit width selector is a probability vector, and the length of the probability vector, that is, the number of elements contained in the probability vector, is equal to the number of candidate bit widths. Each element in the probability vector represents the probability of the candidate bit width corresponding to the element.

[0080] Figure 1 The vector length of each bit width selector shown is 3. The candidate bit width corresponding to the element with the highest probability represented in the probability vector output by the nth bit width selector is the nth bit width.

[0081] like Figure 1 As shown, the data processing model 300 dequantizes the N-1th output information of the N-1th operator, and quantizes the dequantized N-1th output information to 8 bits to obtain 8-bit Nth input data.

[0082] The Nth operator 314 is used to process the Nth input data to obtain 8-bit Nth output information.

[0083] The processing performed by each of the first N operators in the data processing model 300 may be a linear operation.

[0084] The data processing model 300 performs inverse quantization on the Nth output information, and the bit width of the inverse quantized Nth output information may be 32 bits. Through the N+1th operator 315 to the N+Mth operator 316, based on the inverse quantized Nth output information with a bit width of 32 bits, the target processing result may be obtained. The target processing result may be understood as the processing result of the initial sample by the data processing model 300. The parameters of each operator in the N+1th operator 315 to the N+Mth operator 316 may be unquantized, for example, the bit width of each operator may be 32 bits.

[0085] The N+1th operator 315 can process the Nth output information after inverse quantization. In the N+1th operator 315 to the N+Mth operator 316, the output of each operator can be used as the input of the next operator, and the output of the last operator is the target processing result.

[0086] In the data processing model 300, the bit width of each operator from the second operator to the N-1th operator is dynamically adjusted during the processing of the input initial data by the data processing model 300, so that the bit width of each operator is more suitable for the initial data currently being processed. Therefore, for initial data of different complexity, the bit widths of multiple operators in the data processing model 300 can be dynamically selected, so that the reasoning accuracy of the data processing model 300, that is, the balance between the accuracy of the reasoning result and the model performance, can be dynamically adjusted. The data processing model 300 can judge the importance of each operator from the second operator to the N-1th operator according to the complexity of the input initial data, and select the quantization bit width online and dynamically, thereby achieving the best balance between operating efficiency and accuracy.

[0087] The data processing model 300 may be obtained through training. Figures 2 to 5 , the training process of the data processing model is explained.

[0088] Figure 2 It is a schematic flowchart of a training method for a data processing model provided in an embodiment of the present application. Figure 2 The training method shown includes steps S210 to S260.

[0089] Before performing step S210, training data may be obtained, including training samples and label information corresponding to the training samples.

[0090] The label information corresponding to the training sample can also be called the true value (ground truth, GT) corresponding to the training sample.

[0091] Step S210, determining input data of a first initial processing unit based on training samples through an initial data processing model, wherein the initial data processing model includes a first initial processing unit, and the first initial processing unit includes an initial bit width selector and a first initial operator.

[0092] Before the first initial processing unit, the initial data processing model may or may not include other initial operators.

[0093] In the case that the initial data processing model does not include an initial operator before the first processing unit, the initial data processing model may use the training sample as input data of the first initial processing unit.

[0094] When the initial data processing model includes other initial operators before the first initial processing unit, the other processing units in the initial data processing model located before the first initial processing unit can process the training samples to obtain input data of the first initial unit.

[0095] Before the first initial processing unit, the number of other initial operators in the initial data processing model can be one or more. The one or more initial operators can process the training samples, and the initial data processing model can use the processing results of the one or more initial operators on the initial data as the training input data of the first initial processing unit.

[0096] The initial data processing unit may include a second initial operator. The second initial operator may be the first operator in the initial data processing unit that needs to be quantized. The operator that needs to be quantized may be preset, for example, may be manually determined.

[0097] When the second initial operator is the first operator in the initial processing unit, the initial data processing unit may perform quantization processing on the training sample to determine the second training input adjustment data corresponding to the first bit width. The first bit width may be preset.

[0098] The input data of the first initial processing unit can be determined by the second initial operator based on the second training input adjustment data. The second initial operator can be used to process the second training input adjustment data.

[0099] The output of the second initial operator can be used as input data of the first initial processing unit.

[0100] Alternatively, before the second initial operator, the initial data processing model may further include other operators, and quantization processing may be performed on the processing results of the other operators on the initial samples to determine the second training input adjustment data corresponding to the first bit width.

[0101] Step S220 , determining, by an initial bit width selector in the first initial processing unit, a training bit width of the first initial operator from a plurality of candidate bit widths of the first initial operator based on the training input data of the first initial processing unit.

[0102] The initial bit width selector can process the training input data of the first initial processing unit to obtain the training bit width of the first initial operator.

[0103] The initial bit width selector may be used to extract features from the training input data of the first initial processing unit, and to classify the features extracted from the training input data to obtain a training bit width from among a plurality of candidate bit widths.

[0104] The initial bit width selector can also perform global pooling on the training input data to obtain global training information, perform feature extraction on the global training information to obtain features of the training input data, and classify the features of the training input data to determine the training bit width.

[0105] Global pooling can be used to generate compact feature representations. Global pooling can be global max pooling or global average pooling. Global max pooling extracts the maximum value of each channel of the global pooling input data to obtain a one-dimensional vector with the same number of channels. Global average pooling calculates the average value of each channel of the global pooling input data to obtain a one-dimensional vector with the same number of channels.

[0106] By globally pooling the input data and extracting and classifying the features of the global pooling results, the occupation of processing resources by the data processing process of the bit width selector is reduced, that is, the operating efficiency is improved.

[0107] The initial bit width selector may include a global average pooling layer, at least one fully connected layer, and an activation function. The global average pooling layer is used to perform global average pooling on the training input data of the first initial processing unit. At least one fully connected layer is used to extract features of the training global information output by the global average pooling layer. The activation function is used to classify the features extracted from the training input data to obtain a training bit width from a plurality of candidate bit widths.

[0108] Step S230: Mapping the training input data of the first initial operator to first training input adjustment data corresponding to the training bit width of the first initial operator through the initial data processing model.

[0109] The mapping of the training input data of the first initial operator may include quantizing the training input data of the first initial operator.

[0110] Step S240, processing the first training input adjustment data corresponding to the training bit width of the first initial operator based on the training target parameters of the first initial operator through the first initial operator to determine the processing result output by the first initial processing unit, the training target parameters of the first initial operator are the parameters corresponding to the training bit width among the multiple initial parameters of the first initial operator, and different initial parameters of the first initial operator correspond to different candidate bit widths.

[0111] Before step S240, a training target parameter whose bit width is the training bit width may be selected from among the multiple initial parameters of the first initial operator, and different initial parameters of the first initial operator correspond to different candidate bit widths of the first initial operator. That is, after determining the training bit width of the first initial operator, a parameter corresponding to the training bit width is selected from among the corresponding candidate bit widths in the multiple initial parameters of the first initial operator as the training target parameter.

[0112] The output of the first initial operator in the first initial processing unit can be understood as the processing result output by the first initial processing unit.

[0113] Step S250, determining the training information corresponding to the training sample based on the processing result output by the first initial processing unit through the initial data processing model.

[0114] In the initial data processing model, the number of the first initial processing units is one or more.

[0115] When there are multiple first initial processing units, the processing result output by the first initial operator of each first initial processing unit can be used as input data of the next first initial processing unit. The training information can be determined according to the processing result output by the last first initial processing unit.

[0116] Thus, each first initial processing unit can determine the training bit width corresponding to the first initial processing unit from among the multiple candidate bit widths of the first initial operator in the first initial processing unit according to the training input data of the first initial processing unit. The initial data processing unit can determine the training target parameter corresponding to the training bit width from among the multiple initial parameters corresponding to the multiple candidate bit widths of the first initial operator. The initial data processing model can map the training input data of the first initial processing unit to the first training input adjustment data corresponding to the training bit width. The first initial operator can process the first training input adjustment data corresponding to the training bit width based on the training target parameter corresponding to the training bit width to obtain the processing result corresponding to the first initial processing unit.

[0117] The training input data of the first initial processing unit determined based on the training sample in step S210 can be understood as the training input data of the first first initial processing unit. Steps S220 and S240 can be understood as the processing performed by the first first initial processing unit. In step S240, the processing result corresponding to the first first initial processing unit is determined. And in step S250, the processing results corresponding to the other initial first processing units in the multiple first initial processing units can be continued to be determined until the last first initial processing unit is passed and the processing result corresponding to the last first initial processing unit is determined. Based on the processing result corresponding to the last first initial processing unit, the training information corresponding to the training sample can be determined.

[0118] It should be understood that the multiple candidate bit widths of different first initial operators may be the same or different.

[0119] In the case that there are multiple first initial processing units, the bit width of the parameter of the second initial operator located before the first initial processing unit may be greater than or equal to the maximum candidate bit width of each first initial operator in the multiple first initial processing units.

[0120] The processing result output by the first initial processing unit can be used as the training information corresponding to the training sample. That is, the first initial operator in the first initial processing can be the last operator in the initial data processing model. In the case where there are multiple first initial processing units, the processing result of the last first initial processing unit can be used as the training information corresponding to the training sample.

[0121] Alternatively, the initial data processing model may further include a third initial processing unit. The processing result corresponding to the first initial processing unit may be used as an input of the third initial processing unit. The training information corresponding to the training sample may be determined according to the processing result of the third initial processing unit. The third initial processing unit may include at least one fourth initial operator that is not quantized.

[0122] That is to say, the first operator in the first initial processing unit can be understood as the last quantized operator in the initial data processing model.

[0123] In the case that there are multiple first initial processing units, the processing result corresponding to the last first initial processing unit can be used as the input of the third initial processing unit.

[0124] Alternatively, the initial data processing model may further include a second initial processing unit and a third initial processing unit. The second initial processing unit may include a third initial operator. The third initial processing unit may include at least one fourth initial operator.

[0125] The output of the first initial processing unit can be used as the input of the second initial processing unit. The output of the second initial processing unit can be used as the input of the third initial processing unit.

[0126] Through the second initial processing unit, the processing result output by the last first initial processing unit can be mapped to the third training input adjustment data corresponding to the first bit width.

[0127] The third training input adjustment data is processed by the third initial operator in the second initial processing unit.

[0128] By processing the processing result of the third training input adjustment data output by at least one fourth initial operator in the third initial processing unit, training information corresponding to the training sample can be obtained. The bit width corresponding to the input of each fourth initial operator is equal to the bit width corresponding to the training sample. That is to say, during the processing of the third initial processing unit, the initial data processing model can no longer be quantized.

[0129] Step S260, adjusting the model parameters in the initial data processing model according to the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, and the operator training information of the first initial operator, so as to obtain the data processing model.

[0130] Adjusting the model parameters in the initial data processing model may include adjusting the parameters of the initial bit width selector. In some embodiments, adjusting the model parameters in the initial data processing model may also include adjusting the parameters of one or more initial operators in the first initial operator, the second initial operator, the third initial operator, and the fourth initial operator.

[0131] When the initial bit width processor includes a global average pooling layer, at least one fully connected layer and an activation function, the adjustment of the parameters of the initial bit width selector can be understood as the adjustment of the parameters of the at least one fully connected layer.

[0132] The difference between the training information corresponding to the training sample and the label information corresponding to the training sample, and the operator training information of the first initial operator can be expressed as a loss value.

[0133] The loss value can be calculated according to the loss function. In other words, the loss value is the output value of the loss function.

[0134] In the process of training deep neural networks, because we hope that the output of the deep neural network is as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the target value we really want, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the predicted value of the network is high, adjust the weight vector to make it predict lower, and keep adjusting until the deep neural network can predict the target value we really want or a value very close to the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0135] The loss value can be expressed as the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, and the sum of the operator training information of the first initial operator.

[0136] The operator training information of the first initial operator includes the training bit width of the first initial operator and / or the computation amount of the first initial operator processing the first training input adjustment data.

[0137] In the case where the number of the first initial processing units in the initial data processing model is plural, the loss value may be determined according to the first difference, and the second difference and / or the third difference.

[0138] The first difference is the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, which can also be called the original loss value.

[0139] The second difference is determined according to the difference between the sum of the training bit widths of the plurality of first initial operators and the second bit width, and the second bit width may be preset. The second bit width may also be referred to as a model target parameter (bitParam, BP).

[0140] The third difference is determined based on the difference between the sum of the calculation amounts of the plurality of first initial operators and the first calculation amount, and the first calculation amount may be preset. The first calculation amount may also be referred to as the model target calculation amount.

[0141] Exemplarily, the second difference may be a difference between a sum of training bit widths of the plurality of first initial operators and the second bit width, and the third difference may be a difference between a sum of calculation amounts of the plurality of first initial operators and the first calculation amount.

[0142] The loss value can be expressed as the sum of the first difference, the second difference, and the third difference. As shown in formula (1):

[0143] (1)

[0144] in, represents the first difference, and are all preset coefficients, P is the number of the first initial processing units in the initial data processing model, and P is an integer greater than 1. represents the second difference, Indicates the third difference, is the second bit width, is the first calculation quantity, represents the training bit width of the first initial operator in the i-th initial processing unit among the P initial processing units, represents the computational load (operations, OPs) of the i-th initial processing unit among P initial processing units, Indicated in and Take the maximum value.

[0145] That is, when the sum of the training bit widths of the P first initial operators is greater than or equal to the second bit width, the second difference is the difference between the sum of the training bit widths of the P first initial operators and the second bit width. When the sum of the training bit widths of the P first initial operators is less than the second bit width, the second difference is 0.

[0146] When the sum of the calculation amounts of the P first initial operators is greater than or equal to the first calculation amount, the third difference is the difference between the sum of the calculation amounts of the P first initial operators and the first calculation amount. When the sum of the calculation amounts of the P first initial operators is less than the first calculation amount, the third difference is 0.

[0147] The computation amount of the ith initial processing unit is positively correlated with the training bit width of the first initial operator in the ith initial processing unit. The computation amount of the ith initial processing unit may also be correlated with the operation performed by the first initial operator in the ith initial processing unit.

[0148] The adjustment of the parameters of the initial data processing model can be achieved through the back propagation (BP) algorithm. The neural network model can use the error BP algorithm to correct the size of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the parameters in the initial neural network model are updated by back propagating the error loss information, so that the error loss converges. The back propagation algorithm is a back propagation movement dominated by error loss, which aims to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0149] According to the first initial operator after parameter adjustment, the first operator can be determined. According to the initial parameters corresponding to the multiple candidate bit widths of the adjusted first initial operator, the parameters corresponding to the multiple candidate bit widths of the first operator can be determined. According to the first initial processing unit after parameter adjustment, the first processing unit can be determined.

[0150] The second operator can be determined according to the second initial operator after the parameter adjustment, the third operator can be determined according to the third initial operator after the parameter adjustment, and the fourth operator can be determined according to the fourth initial operator after the parameter adjustment.

[0151] Exemplarily, in the initial data processing model, the initial operator after parameter adjustment can be used as the operator in the data processing model. For example, the first initial operator after parameter adjustment can be used as the first operator in the data processing model, and the first initial processing unit after parameter adjustment can be used as the first processing unit.

[0152] In the case where the initial operator includes the Conv layer, the BN layer, and the ReLU layer, in order to reduce the amount of calculation, the operation of the BN layer, or the BN layer and the ReLU layer can be fused into the Conv layer to form an equivalent single layer. This fusion essentially modifies the parameters of the Conv layer, so that the amount of calculation of the BN layer is omitted without increasing the amount of calculation of the Conv layer.

[0153] In other words, the operator in the data processing model can also be obtained by fusing multiple layers of the initial operator after parameter adjustment.

[0154] It should be noted that this fusion is only performed in the model inference phase. The training phase still needs to keep the three layers of Conv, BN and ReLU separate for back propagation and parameter update. That is, in the data processing unit, the processing unit can only include the Conv layer, while the initial processing unit can include the three layers of Conv, BN and ReLU.

[0155] The multiple candidate bit widths of each first initial operator and the number of the multiple candidate bit widths, the first bit width of the second initial operator, and the first bit width of the third initial operator can all be hyperparameters. Hyperparameter is an important concept in machine learning. It refers to the parameters that need to be manually set before the algorithm starts the learning process, rather than the parameters that are automatically learned through the training process. The selection and optimization of these hyperparameters have an important impact on the performance of the model.

[0156] The multiple candidate bit widths of each first initial operator may be artificially set according to hardware conditions and other conditions during actual deployment.

[0157] The initial data processing model may be determined according to the initial model. The initial model may be an initialized model or a pre-trained model. The initial data processing model may include operators and initial bit width selectors in the initial model. The initial bit width selector may be an initialized model. Initialization of a neural network model is an important step in deep learning, which involves setting parameters in the neural network, such as initial values ​​of weights and biases. For example, random initialization may be used for initialization. The parameters in the neural network may be randomly set by random initialization.

[0158] The initial operator in the initial model may include the first initial operator, and may also include one or more of the second initial operator, the third initial operator, and the fourth initial operator. That is, in the initial data processing model, the first initial operator, the second initial operator, the third initial operator, and the fourth initial operator may all be operators in the initial model.

[0159] The parameters of the first initial operator, the second initial operator, and the third initial operator in the initial data processing model may all be quantized parameters, and the parameters of the fourth initial operator may be unquantized parameters. The initial parameters corresponding to the multiple candidate bit widths of the first initial operator in the initial data processing model may be determined by quantizing the parameters of the first initial operator in the initial model according to the multiple candidate bit widths. The parameters of the second initial operator in the initial data processing model may be determined by quantizing the parameters of the second initial operator in the initial model according to the first bit width. The parameters of the third initial operator in the initial data processing model may be determined by quantizing the parameters of the third initial operator in the initial model according to the first bit width. The parameters of the fourth initial operator in the initial data processing model may be the parameters of the fourth initial operator in the initial model.

[0160] When the initial data processing model includes multiple initial operators, the sequence of the multiple initial operators is the same as the sequence of the multiple initial operators in the initial model.

[0161] The initial data processing model can be obtained by quantizing the parameters of the first initial operator, the second initial operator, and the third initial operator in the initial model, and inserting an initial bit width selector before the first initial operator after the parameter quantization. The structure of the initial model can be found in Figure 3 Description.

[0162] Figure 3 It is a structural diagram of an initial model.

[0163] like Figure 3 As shown in (a), the initial model may include N+M initial operators, wherein the first initial operator 411 is the second initial operator, the second initial operator 413 to the N-1th initial operator are all the first initial operator, the Nth initial operator 414 is the third initial operator, and the N+1th initial operator 415 to the N+Mth initial operator 416 are all the fourth initial operators. The initial model may be a pre-trained neural network model.

[0164] Each of the initial operators 411 to 416 in the initial model may include one or more layers. The number of layers in different initial operators may be the same or different. Each of all or part of the operators in the first initial operator 411 to the Nth initial operator 414 may include: Figure 3 The convolutional layer (Conv) layer 401, the batch normalization layer (BN) layer 402 and the rectified linear unit layer (ReLU) layer 403 shown in (b) of FIG.

[0165] The initial model can be understood as a convolutional neural network (CNN). CNN is a deep neural network with a convolutional structure. The convolutional neural network contains a feature extractor consisting of a convolution layer and a subsampling layer. The feature extractor can be regarded as a filter, and the convolution process can be regarded as using a trainable filter to convolve with an input image or convolution feature plane (feature map).

[0166] A convolutional layer refers to a neuron layer in a convolutional neural network that performs convolution processing on the input signal. In a convolutional layer of a convolutional neural network, a neuron can only be connected to some neurons in adjacent layers. A convolutional layer usually contains several feature planes, and each feature plane can be composed of some neural units arranged in a rectangular shape. The neural units in the same feature plane share weights, and the shared weights here are convolution kernels. Shared weights can be understood as the way to extract image information is independent of position. The implicit principle is that the statistical information of a certain part of the image is the same as that of other parts. This means that the image information learned in a certain part can also be used in another part. Therefore, the same learned image information can be used for all positions on the image. In the same convolutional layer, multiple convolution kernels can be used to extract different image information. Generally speaking, the more convolution kernels there are, the richer the image information reflected by the convolution operation.

[0167] When a convolutional neural network has multiple convolutional layers, the initial convolutional layer often extracts more general features, which can also be called low-level features. As the depth of the convolutional neural network increases, the features extracted by the subsequent convolutional layers become more and more complex, such as high-level semantic features. Features with higher levels are more suitable for the problem to be solved.

[0168] The BN layer is a technique used to improve the speed and stability of network training. It normalizes the input data of each mini-batch so that the input of each layer follows the same distribution. Specifically, the BN layer normalizes each feature of each mini-batch data to have zero mean and unit variance, and then scales and translates it through a learnable scaling parameter (scale) and offset parameter (shift) to maintain the network's expressiveness.

[0169] The ReLU layer is a nonlinear activation function layer that sets all negative values ​​to zero while keeping positive values ​​unchanged. This simple nonlinear operation helps increase the nonlinearity of the network while avoiding the gradient vanishing problem (a problem that easily occurs when using activation functions such as Sigmoid or Tanh in deep networks).

[0170] The N+1th initial operator 415 to the N+M-1th initial operator may all be fully connected layers, and the N+Mth initial operator 416 may be an activation function softmax. Thus, the initial model may be used to classify data input into the initial model.

[0171] All or part of the operators from the first initial operator 411 to the N+Mth initial operator 416 may also include other layers, so that the initial model can realize functions such as object recognition, object segmentation, and semantic recognition.

[0172] exist Figure 3 In the initial model shown in (a), before each initial operator from the second initial operator 412 to the N-1th initial operator 413, an initial bit width selector corresponding to the initial operator is inserted, and the parameters of the first N initial operators are quantized, and the initial data processing model can be obtained. In other words, the initial bit width selector can be inserted layer by layer. In the initial data processing model, when n≥2 and n≤N-1, the nth initial operator corresponds to the nth initial bit width selector. The parameters of different initial bit width selectors can be the same or different. Figure 1 The data processing model 300 shown is obtained by training an initial data processing model based on training data.

[0173] In some embodiments, when the initial model is a pre-trained model, adjusting the model parameters in the initial data processing model may not include adjusting the parameters of other units in the initial data processing model except the initial bit width selector.

[0174] During the training process, the data processed by the first initial operator, the second initial operator, and the third initial operator may be quantized data. The parameters of each of the first initial operator, the second initial operator, and the third initial operator in the initial data processing model are also obtained through quantization.

[0175] The parameters of the first initial operator in the initial model are quantized according to the multiple candidate bit widths of the first initial operator, and the initial parameters of each candidate bit width of the first initial operator can be obtained. The parameters of the second initial operator and the third operator in the initial model are quantized according to the first bit width, and the parameters of the first bit width of the second initial operator and the first bit width of the third initial operator can be obtained.

[0176] In the case where the training sample is used as the training input data of the first initial processing unit, mapping the training input data of the first initial operator to the first training input adjustment data corresponding to the training bit width of the first initial operator may be quantizing the training input data of the first initial processing unit to obtain the first training input adjustment data. The bit width of the first training input adjustment data may be the training bit width of the first initial operator.

[0177] In the case where the initial data processing model includes the second initial operator, the initial data processing model can quantize the training sample to obtain the second training input adjustment data corresponding to the first bit width. The bit width of the second training input adjustment data can be the first bit width. The second initial operator processes the second training input adjustment data based on the parameter of the first bit width of the second initial operator. The output of the second initial operator can be used as the training input data of the first initial processing unit. The first training input adjustment data corresponding to the training bit width of the first initial operator can be obtained by dequantizing the training input data of the first initial processing unit and quantizing it again. The bit width of the first training input adjustment data can be the training bit width of the first initial operator.

[0178] In the case where the initial data processing model includes the third initial operator, the processing result corresponding to the first initial processing unit is dequantized and quantized again to obtain the third training input adjustment data corresponding to the first bit width. The bit width of the third training input adjustment data corresponding to the first bit width is the first bit width. The third initial operator can process the third training input adjustment data based on the parameter of the first bit width of the third initial operator, thereby determining the training information corresponding to the training sample.

[0179] The quantization parameter used to quantize the parameters of each initial operator of the initial model as much as possible can be determined according to the range of the parameters of the initial operator.

[0180] The quantization parameters used by the initial data processing model to quantize the training samples can be determined according to the data range of the training samples. The quantization parameters used by the initial data processing model to quantize the inverse quantization results of the training input data of the first initial processing unit can be determined according to the range of the inverse quantization results of the training input data of the first initial processing unit.

[0181] The number of training samples may be one or more. In the case where the number of training samples is multiple, a sample quantization parameter may be determined based on the data range of the multiple training samples, and the sample quantization parameter is a quantization parameter used to quantize the training samples. In the case where the number of training samples is multiple, a quantization parameter corresponding to the first initial processing unit may be determined based on the range of the training input data of the first initial processing unit in the process of processing each training sample according to the initial data processing model, and the quantization parameter corresponding to the first initial processing unit is used to quantize the inverse quantization result of the training input data of the first initial processing unit.

[0182] The input of each initial bit width selector may be the training input data of the first initial processing unit where the initial bit width selector is located, or may be the dequantization result of the training input data of the first initial processing unit where the initial bit width selector is located.

[0183] In other embodiments, the adjustment of model parameters in the initial data processing model includes the adjustment of parameters in the initial bit width selector, and may also include the adjustment of parameters of one or more initial operators among the first initial operator, the second initial operator, the third initial operator, and the fourth initial operator.

[0184] The quantization process often leads to a certain loss of precision, especially in the case of extremely low bits (such as the bit width of the quantized parameter is less than or equal to 4 bits), the loss of precision is more obvious, that is, the reasoning accuracy of the quantized neural network model is low. Adjusting the parameters of the initial operators such as the first initial operator during the training process helps to alleviate the precision loss caused by quantization and improve the performance of the model.

[0185] During the training process, the input processed by each of the first initial operator, the second initial operator, and the third initial operator may be obtained by quantizing and then dequantizing.

[0186] The parameters of each of the first initial operator, the second initial operator, and the third initial operator in the initial data processing model may also be obtained by quantizing and then dequantizing.

[0187] The parameters of the second initial operator in the initial model are quantized to the first bit width, and the quantized parameters of the second initial operator are dequantized. The dequantized results can be used as the parameters of the second initial operator in the initial data processing model.

[0188] The parameters of the third initial operator in the initial model are quantized to the first bit width, and the quantized parameters of the third initial operator are dequantized. The dequantized results can be used as the parameters of the third initial operator in the initial data processing model.

[0189] The parameters of the first initial operator in the initial model are quantized to multiple candidate bit widths, and then the quantized parameters corresponding to each candidate bit width are dequantized, so that the initial parameters corresponding to each candidate bit width of the first initial operator in the initial data processing model can be obtained. The initial parameters corresponding to each candidate bit width of the first initial operator can also be called the parameters corresponding to each candidate bit width of the first initial operator.

[0190] The adjustment of the parameters of the initial operator obtained after quantization and dequantization may be the adjustment of the parameters of the initial operator with the scaling factor used in the quantization process as the granularity. In other words, the adjustment of the parameters of a certain initial operator may be the adjustment of the parameters of the certain initial operator with the scaling factor as the minimum adjustment unit.

[0191] In the process of processing the training samples using the initial data processing model, the first training input adjustment data corresponding to the training bit width of the first initial operator may be obtained by quantizing the training input data of the first initial operator according to the training bit width of the first initial operator and then performing dequantization. The second training input adjustment data corresponding to the first bit width may be obtained by quantizing the training sample according to the first bit width and then performing dequantization. The third training input adjustment data corresponding to the first bit width may be obtained by quantizing the processing result output by the first initial unit according to the first bit width and then performing dequantization.

[0192] The input of each bit width selector may be the training input data of the first initial processing unit where it is located.

[0193] For according to Figure 2 The initial data processing model determined by the initial model shown in the figure is used during the training process. Figure 4 As shown in (a) in FIG. 1 , the training sample can be quantized according to 8 bits to obtain an 8-bit quantized training sample. The quantized training sample can be dequantized again, and the dequantized result can be used as the input of the first initial operator 411. The first initial operator 411 is the second initial operator. The result of dequantizing the quantized training sample can be understood as the second training input adjustment data, that is, the quantized and dequantized training sample.

[0194] In the initial model, the parameter of each initial operator can be floating point data, that is, the parameter of each initial operator can be 32 bits. The parameter of the first initial operator 411 in the initial model is quantized to 8 bits, and the 8-bit quantized parameter of the first initial operator 411 can be obtained. The quantized parameter of the first initial operator 411 is dequantized to obtain the dequantized parameter of the first initial operator 411. The dequantized parameter of the first initial operator 411 is 32 bits.

[0195] The first initial operator 411 is used to process the quantized and dequantized training samples based on the dequantized parameters of the first initial operator 411 to obtain the first training output data. The first training output data is 32 bits.

[0196] like Figure 5 As shown in (a), for the nth initial bit width selector where n≥2 and n≤N-1, the nth initial bit width selector is used to process the n-1th training output to obtain the nth training bit width. The nth training bit width belongs to multiple candidate bit widths of the nth initial operator.

[0197] The n-1th training output data is quantized according to the nth training bit width, that is, the n-1th training output data is quantized to the nth training bit width, and the quantized n-1th training output data is dequantized, and the obtained n-1th training input adjustment data can be used as the input of the nth initial operator. The n-1th training output after quantization and dequantization, that is, the n-1th training input adjustment data, has a bit width of 32 bits. The bit width corresponding to the n-1th training input adjustment data can be understood as the n-1th training bit width. When n≥2 and n≤N-1, the n-1th initial operator can be understood as the n-1th first initial operator, and each n-1th training input adjustment data can be understood as the first training input adjustment data corresponding to the n-1th first initial operator.

[0198] The parameters of the nth initial operator in the initial model are quantized to multiple candidate bit widths, that is, quantized to 2 bits, 4 bits and 8 bits, respectively, and the quantized parameters of 2 bits, 4 bits and 8 bits can be obtained. The quantized parameters of 2 bits, 4 bits and 8 bits are dequantized, respectively, and the initial parameters of the nth initial operator corresponding to 2 bits, 4 bits and 8 bits can be obtained. The initial parameters of the nth initial operator corresponding to 2 bits, 4 bits and 8 bits are all 32 bits.

[0199] The nth initial operator is used to process the nth training input adjustment data based on the training target parameters corresponding to the nth training bit width in the initial parameters of the nth initial operator corresponding to 2 bits, 4 bits and 8 bits respectively, and can obtain 32-bit nth training output data.

[0200] For the Nth initial operator 414, the parameters of the Nth initial operator 414 in the initial model may be quantized to 8 bits to obtain the quantized parameters of the Nth initial operator 414. The quantized parameters of the Nth initial operator are dequantized to obtain the dequantized parameters of the Nth initial operator 414. The bit width of the quantized parameters of the Nth initial operator 414 obtained after quantization and dequantization is 32 bits.

[0201] The bit width of the N-1th training output data is 32 bits. The N-1th training output data is quantized according to 8 bits, and the quantized N-1th training output data is dequantized to obtain the quantized and dequantized N-1th training output data. The bit width of the quantized and dequantized N-1th training output data is 32 bits.

[0202] The Nth initial operator 414 can be used to process the quantized and dequantized N-1th training output data based on the dequantized parameters of the Nth initial operator 414 to obtain the Nth training output data. The Nth training output data is 32 bits. The Nth initial operator 414 is the third operator, and the quantized and dequantized N-1th training output data can be understood as the third training input adjustment data.

[0203] The third initial processing unit processes the Nth output data to obtain training information corresponding to the training sample. The third initial processing unit includes the N+1th initial operator 415 to the N+Mth initial operator 416.

[0204] For the second initial operator 411 to the N-1th initial operator 414, the initial parameters corresponding to different candidate bit widths are adjusted respectively, so that the adjusted parameters are more suitable for processing the candidate bit width, thereby improving the inference accuracy of the data processing model.

[0205] The training process involves the quantization of the parameters of the initial operator, as well as the quantization of the training samples and training output data. The quantization of training data can also be called activation quantization. Activation quantization refers to the process of converting the activation value (i.e. neuron output) in the neural network from continuous floating point data to discrete integer data.

[0206] The quantization result q obtained by quantizing the value r is shown in formula (2):

[0207] (2)

[0208] Among them, round means rounding to an integer. , , , , are all quantitative parameters, among which, Indicates the scaling factor (scale), Indicates the offset. , It can be preset, for example, it can represent the maximum and minimum values ​​represented by a certain bit width. and As shown in formula (3):

[0209] (3)

[0210] in, and It can be determined based on a numerical range.

[0211] The training sample can include multiple values. For the quantization of the training sample, the and They can be the maximum and minimum values ​​of the range of multiple numerical values ​​in the training samples, respectively.

[0212] The parameters of each initial operator in the initial model can include multiple values. and They may be the maximum value and the minimum value of the range of multiple numerical values ​​of the parameters of the certain initial operator in the initial model, respectively.

[0213] Each training output data may include multiple values. For the quantization of each training output data, the and They may be the maximum value and the minimum value of the range of multiple numerical values ​​of the certain training output data respectively.

[0214] like Figure 4 As shown, the quantization parameters used by the initial data processing model to quantize the training samples can be used as the quantization parameters used by the data processing model to quantize the initial data.

[0215] like Figure 5 As shown, when n≥2 and n≤N, the quantization parameter used by the initial data processing model to quantize the n-1th training output data can be used as the quantization parameter used by the data processing model to quantize the n-1th output data.

[0216] The quantization parameter used by the initial data processing model to quantize the Nth training output data may be used as the quantization parameter used by the data processing model to quantize the Nth output data.

[0217] During the training process, the number of training data can be multiple and can be divided into multiple batches. In other words, batch processing can be performed during the training process. Batch processing divides the training data into multiple smaller subsets (i.e., "batches"), and adjusts the parameters of the initial data processing model according to the loss values ​​obtained by the initial neural network model processing multiple training samples in a batch. The parameter adjustment of the initial data processing model includes the adjustment of the parameters of the initial operator.

[0218] When the initial data processing model processes the next batch of training samples, the latest parameters of each initial operator are used for calculation and processing. For example, for each first initial operator, when processing the next batch of training samples, the training target parameter corresponding to the training bit width of each first initial operator may change, resulting in a change in the range of the training output data of the first initial operator.

[0219] For the quantization of training samples, the and It can be determined based on the numerical range of the training samples currently processed by the initial data processing model, or it can be determined based on the numerical range of the training samples that have been processed by the initial data processing model. and It can be determined based on the numerical range of a batch of training samples processed by the initial data processing model.

[0220] When n≥1 and n≤N, for the quantization of any n-th training output data, the and It can be determined based on the numerical range of the nth training output data obtained from the training samples currently processed by the initial data processing model, or based on the numerical range of the nth training output data obtained from the training samples that have been processed by the initial data processing model, such as the training samples processed in the last batch.

[0221] The quantization parameter used by the data processing model to quantize the initial data may be a quantization parameter used by the initial data processing model to quantize the last or last batch of training samples during the training process.

[0222] When n≥1 and n≤N, the quantization parameter used by the data processing model to quantize the nth output data may be the quantization parameter used by the initial data processing model to quantize the nth training output data when processing the last or last batch of training samples during the training process.

[0223] Training equipment can be Figure 2 The training method shown is based on training data to obtain a data processing model. The data processing model obtained by training with a training device can be applied to an execution device. The execution device can perform data processing on the initial data based on the data processing model. The initial data can be data collected or stored in the execution device, or data sent by a client device received by the execution device. A user can send the initial data to the execution device through a client device.

[0224] It should be understood that the training device, the execution device, and the client device may be the same or different electronic devices. For example, the training device may be a server or a cloud, and the execution device may be a terminal, such as a mobile terminal, a tablet computer, a laptop computer, an augmented reality (AR) device, a virtual reality (VR) device, a vehicle-mounted terminal, etc.

[0225] A lightweight model submodule, namely the initial bit width selector, is inserted on the basis of the initial model. During training, the initial bit width selector can update parameters in back propagation to obtain the bit width selector in the data processing model. During inference, the bit width selector can dynamically output a probability vector based on the input data of the first processing unit, specify the bit width of the first operator in the first processing unit, and realize dynamic adjustment of the quantization bit width.

[0226] Combine the following Figure 6 The data processing method provided in the embodiment of the present application is described in detail. The execution subject of the method provided in the present application can be an electronic device, or a software / hardware module capable of data processing in the electronic device. For the sake of convenience, the following embodiments are described by taking the electronic device as an example.

[0227] Figure 6 is a schematic flow chart of a data processing method provided in an embodiment of the present application. The method may include steps S610 to S660, and these steps are described in detail below.

[0228] Steps S610 to S660 can be understood as a process of processing the initial data through a data processing model to determine a target processing result, that is, an inference process of the data processing model.

[0229] Step S610: Based on the initial data, determine the input data of the first processing unit through the data processing model, the data processing model includes the first processing unit, and the first processing unit includes a bit width selector and a first operator.

[0230] The bit width of the initial data is the initial bit width. The initial bit width may be preset.

[0231] The operator in the first processing unit may be the first operator in the data processing unit, or may be another operator other than the first operator.

[0232] When the data processing model is located before the first processing unit and does not include other operators, the data processing model can use the initial data as input data of the first processing unit.

[0233] In the case where the data processing model includes other operators before the first processing unit, the number of the other operators may be one or more. The one or more operators may process the initial data, and the data processing model may use the processing results of the one or more operators on the initial data as input data of the first processing unit.

[0234] Step S620 : determining, by a bit width selector of the first processing unit, a target bit width of the first operator from a plurality of candidate bit widths of the first operator based on the input data.

[0235] The bit width selector may be used to extract features from input data, and to classify the features extracted from the input data to obtain a target bit width from among a plurality of candidate bit widths.

[0236] In order to reduce the resource occupation of the data processing process of the bit width selector, the bit width selector can also perform global pooling based on the input data to obtain global information, perform feature extraction on the global information to obtain input data features, and classify the input data features to determine the target bit width.

[0237] Global pooling can be used to generate compact feature representations. Global pooling can effectively reduce the number of model parameters and computational complexity, and improve the generalization and robustness of the model. Global pooling can be global max pooling or global average pooling. Global max pooling extracts the maximum value of each channel in the feature map to obtain a one-dimensional vector with the same number of channels. Global average pooling calculates the average value of each channel in the feature map to obtain a one-dimensional vector with the same number of channels.

[0238] The global pooling method used by the data processing model during the inference process and the global pooling method used during the training process are both global maximum pooling or global average pooling.

[0239] By globally pooling the input data and extracting and classifying the features of the global pooling results, the occupation of processing resources by the data processing process of the bit width selector is reduced.

[0240] like Figure 7 As shown, each bit width selector may include a global average pooling layer 701, at least one fully connected layer, and an activation function 704. The at least one fully connected layer may include a first fully connected layer 702 to a second fully connected layer 703.

[0241] The global average pooling layer 701 is used to perform global average pooling on the input data. Based on at least one fully connected layer, feature extraction can be performed on the result of global pooling. The activation function 704 is used to classify the features extracted based on at least one fully connected layer.

[0242] The advantage of global average pooling is that it can extract the overall information of the feature map and average the information in the channel, which helps to reduce the model's sensitivity to specific locations. In addition, the calculation of global average pooling comprehensively considers all the information in the channel, making the processing result of the bit width selector more accurate.

[0243] The activation function 704 may be an activation function softmax or an activation function gumbel softmax or the like.

[0244] The activation function softmax is an activation function used for multi-class classification problems, usually used in the output layer of a neural network. It converts the original scores (also called logits) into numerical values ​​representing probability distribution, so that the probability value of each category is between 0 and 1, and the sum of the probabilities of all categories is equal to 1.

[0245] The activation function gumbel softmax adds Gumbel distribution (or Gumbel distribution, gumbel) noise to the logits (i.e., original scores) of the neural network and processes the result through the softmax function to obtain a vector that is close to a single-bit effective encoding (or one-hot encoding, one-hot) but is still continuous. This vector can be used for gradient calculation and can approximate discrete distributions. The activation function gumbel softmax can be seen as an extension of the classic softmax function, combined with gumbel noise, which is used to approximate discrete one-hot vectors while maintaining the computability of gradients.

[0246] The candidate bit width corresponding to the maximum probability output by the activation function 704 is the target bit width.

[0247] The data processing model may further include a second operator, and the output of the second operator may be used as the input of the first processing unit. The second operator may be understood as other operators located before the first processing unit in the data processing model. The second operator may be the first operator in the data processing model. Alternatively, the second operator may be the first quantized operator in the data processing model.

[0248] The quantized operator may be understood as an operator that has been quantized. The parameters of the quantized operator may be obtained through quantization.

[0249] The initial data can be quantized by the data processing model to obtain the second input adjustment data. The bit width of the second input adjustment data is the first bit width, and the first bit width is greater than or equal to the maximum candidate bit width of the first operator in the first processing unit. The first bit width can be preset. The maximum candidate bit width of the first operator can be understood as the maximum value among the multiple candidate bit widths of the first operator.

[0250] By means of the second operator, input data of the first processing unit may be determined based on the second input adjustment data.

[0251] The second operator may process the second input adjustment data to obtain input data of the first processing unit.

[0252] That is, before the first processing unit, the data processing model may include one other operator, and the one other operator may be the second operator.

[0253] Each operator can use the parameters of the operator to process data having a bit width equal to the bit width of the parameters of the operator.

[0254] That is to say, the bit width of the parameter of the second operator is also the first bit width.

[0255] In the data processing model, the first operator and other operators before the first operator can be used to extract features from the initial data. In order to reduce the storage space occupied by the data processing model and reduce the processing resource occupied by the data processing model in the process of processing the initial data, the parameters of the first operator and other operators before the first operator can all be quantized parameters. Operators using quantized parameters can process the quantized data.

[0256] In the data processing model, the features extracted by the first operator are more important, so the parameters of the first operator in the data processing model can be quantized to a larger bit width, that is, the first bit width. The first operator can process data with a larger bit width, that is, the second input adjustment data of the first bit width, thereby improving the accuracy of the data processing model reasoning, that is, improving the accuracy of the target processing result.

[0257] The first operator in the data processing model is the second operator.

[0258] The bit width of the first operator in the first processing unit after the second operator can be determined by processing the input data of the first processing unit by the bit width selector in the first processing unit. Thus, while making the accuracy of the data processing model inference higher, the occupation of the processing resources of the data processing of the data processing model is reduced as much as possible, achieving a balance between the accuracy of the data processing model inference and the processing resources occupied by the data processing of the data processing model.

[0259] Step S630: Select a target parameter whose bit width is the target bit width from among multiple parameters of the first operator through the data processing model, and different parameters of the first operator correspond to different candidate bit widths of the first operator.

[0260] Step S640: Mapping the input data to first input adjustment data through the data processing model, wherein the bit width of the first input adjustment data is the target bit width.

[0261] Mapping the input data to the first input adjustment data can be understood as adjusting the bit width of the input data.

[0262] According to the mapping relationship of the first processing unit, the input data can be mapped into the first input adjustment data.

[0263] Alternatively, the input data may be dequantized and then the dequantized input data may be quantized, thereby achieving mapping from the input data to the first input adjustment data.

[0264] Dequantizing the input data may make the bit width of the dequantized input data equal to the initial bit width. Quantizing the dequantized input data may make the bit width of the quantized data equal to the target bit width.

[0265] Step S650: determining a processing result corresponding to the first processing unit based on the target parameter and the first input adjustment data through the first operator of the first processing unit.

[0266] The first operator of the first processing unit can process the first input adjustment data with the target bit width based on the target parameter with the target bit width to obtain the processing result corresponding to the first processing unit. The processing result corresponding to the first processing unit can be understood as the output of the first operator.

[0267] Step S660, determining the target processing result based on the processing result corresponding to the first processing unit through the data processing model.

[0268] In the data processing model, the number of the first processing unit can be one or more. In the case where the number of the first processing unit is multiple, the processing result corresponding to each first processing unit can be used as input data of the next first processing unit.

[0269] Thus, each first processing unit can determine the target bit width corresponding to the first processing unit from among the multiple candidate bit widths of the first operator in the first processing unit according to the input data of the first processing unit. The first operator can process the first input adjustment data mapped from the input data of the first processing unit to the target bit width corresponding to the first processing unit based on the target parameter whose bit width is the target bit width corresponding to the first processing unit, and obtain the processing result corresponding to the first processing unit.

[0270] It should be understood that when the bit width of the input data of the first processing unit is equal to the target bit width corresponding to the first processing unit, the first input data can also be mapped to obtain the first input adjustment data. The mapping of the first input data can make the mapped first input adjustment data more suitable for subsequent processing of the data processing model.

[0271] That is to say, in step S610, the input data of the first processing unit determined based on the initial data can be understood as the input data of the first first processing unit. Steps S620 and S650 can be understood as the processing performed by the first first processing unit. In step S650, the processing result corresponding to the first first processing unit is determined. And in step S660, the processing results corresponding to other first processing units in the plurality of first processing units can be continued to be determined until the last first processing unit is passed and the processing result corresponding to the last first processing unit is determined. Based on the processing result corresponding to the last first processing unit, the target processing result can be determined.

[0272] In the data processing model, the bit widths of the parameters of the consecutive multiple first operators can be flexibly adjusted according to the input data of the first processing unit where the first operator is located, so that the reasoning process of the data processing model occupies less processing resources and the reasoning results of the data processing model are more accurate. In addition, the bit widths of the multiple first operators do not need to be set manually, reducing labor costs.

[0273] In the case where there are multiple first processing units, the bit width of the parameter of the second operator located before the first processing unit may be greater than or equal to the maximum candidate bit width of each first operator in the multiple first processing units. It should be understood that the multiple candidate bit widths of different first operators may be the same or different.

[0274] In the data processing model, the parameters of other operators after the first operator may be unquantized parameters. Thus, the data processing model may dequantize the output result corresponding to the first processing unit where the first operator is located, and process the dequantized output result corresponding to the first processing unit through other operators after the first operator to obtain the target processing result.

[0275] The first operator in the last first processing unit in the data processing model may be the last operator of the data processing model. In other words, the processing result corresponding to the first processing unit may be used as the target processing result.

[0276] Alternatively, other operators may be included after the last first operator in the data processing model. The other operators may be unquantized operators. The processing result of the last first processing unit may be dequantized, and the dequantized result of the processing result of the last processing unit may be processed by other operators after the last first processing unit to determine the target processing result.

[0277] Alternatively, a third operator may be included. The third operator may be understood as the last quantized operator in the data processing model. In other words, the parameters of the third operator are obtained through quantization.

[0278] The data processing model may further include a second processing unit and a third processing unit. The output of the first processing unit may be used as an input of the second processing unit. The second processing unit includes a third operator. The output of the first processing unit is the processing result corresponding to the first processing unit. The output of the second processing unit may be used as an input of the third processing unit. The third processing unit may include at least one fourth operator. The third operator and the at least one fourth operator are other operators after the first operator in the data processing model.

[0279] The processing result corresponding to the last first processing unit may be mapped to third input adjustment data.

[0280] The bit width of the third input adjustment data may be preset. The bit width of the third input adjustment data may be greater than or equal to the maximum candidate bit width of the first operator in the first processing unit. The bit width of the third input adjustment data may be equal to or unequal to the bit width of the second input adjustment data. In other words, the bit width of the third input adjustment data may be the first bit width or another bit width that is unequal to the first bit width.

[0281] In the case where there are multiple first processing units, the bit width of the third input adjustment data may be greater than or equal to the maximum candidate bit width of each first operator. The output of the last first processing unit may be used as the input of the second processing unit.

[0282] The third operator pair can process the third input adjustment data. The output data of the third operator can be dequantized to obtain the first dequantized data. The third processing unit can process the first dequantized data to obtain the target processing result.

[0283] The processing of the first inverse quantized data by the third processing unit can be understood as determining the target processing result based on the first inverse quantized data by at least one fourth operator in the third processing unit. The bit width of the input of each fourth operator is equal to the bit width of the first inverse quantized data. In other words, the bit width of the parameter of each fourth operator is equal to the bit width of the first inverse quantized data.

[0284] The parameters of each fourth operator and the bit width of the data processed by each fourth operator are equal to the bit width of the first dequantized data; and the bit width of the parameters of the third operator and the bit width of the third input adjustment data processed by the third operator can be understood as the quantized bit width. In other words, the third operator can be understood as the last quantized operator in the data processing model.

[0285] In the data processing model, the output of the last quantized operator has an important influence on the accuracy of the inference result of the data processing model. Therefore, for the last quantized third operator in the data processing model, the bit width of the parameter of the third operator and the bit width of the input of the third operator are both greater than or equal to the maximum candidate bit width of the first operator in the first processing unit, which can improve the accuracy of the inference result of the data processing model.

[0286] For the input data of the first processing unit, the bit width selector can perform subsequent global pooling processing, feature extraction and classification on the input data.

[0287] Alternatively, the bit width selector may map the input data into mapping data having a bit width of a third bit width, and perform subsequent global pooling processing, feature extraction, classification, and other processing on the mapping data.

[0288] Alternatively, the bit width selector may perform inverse quantization on the input data to obtain second inverse quantized data, and perform subsequent global pooling processing, feature extraction, classification, and other processing on the second inverse quantized data.

[0289] Compared with mapping the input data to the third bit width, it is easier to perform dequantization on the input data. By processing the dequantized input data, that is, the second dequantized data, the bit width selector can more easily process the input data.

[0290] The mapping process from the input data to the first input adjustment data may be to first dequantize the input data and then quantize the dequantization result based on the target bit width. Alternatively, the mapping process from the input data to the first input adjustment data may also be to map the input data according to the target mapping relationship corresponding to the combination of the bit width of the first input data and the target bit width. Different combinations of the bit width of the input data and the candidate bit widths may correspond to different mapping relationships. The mapping relationship corresponding to the combination of the bit width of each input data and the candidate bit width may be determined based on the parameters for dequantizing the input data and the parameters for quantizing the dequantization result based on the target bit width.

[0291] When the bit width selector performs inverse quantization on the input data to obtain second inverse quantized data, the second inverse quantized data may be quantized based on the target bit width through the data processing model to obtain the first input adjusted data.

[0292] Therefore, the mapping of the input data to the first input adjustment data of the target bit width is more convenient.

[0293] Alternatively, the mapping process from the input data to the first input adjustment data may also be a mapping of the input data according to a target mapping relationship corresponding to a combination of the bit width of the input data and the target bit width. Combinations of different bit widths of the input data and different candidate bit widths may correspond to different mapping relationships. That is, among the multiple mapping relationships of the first processing unit, a target mapping relationship corresponding to a combination of the bit width of the input data and the target bit width may be determined, so that the input data is mapped according to the target mapping relationship to obtain the first input adjustment data of the target bit width.

[0294] The mapping relationship between the bit width of each input data and the combination of candidate bit widths may be determined based on the parameters used for dequantizing the input data of the bit width and the quantization parameters used for quantizing the dequantized result according to the candidate bit width.

[0295] Through steps S610 to S660, the bit width selector in the first processing unit processes the input data of the first processing unit to determine the target bit width from the multiple candidate bit widths of the first operator in the first processing unit, and maps the input data of the first processing unit to the first input adjustment data with the target bit width through the data processing model, so that the first operator can determine the processing result corresponding to the first processing unit based on the first input adjustment data and the target parameters in the multiple parameters corresponding to the multiple candidate bit widths one by one, and the processing result corresponding to the first processing unit is used for the data processing model to determine the target processing result, so that the target bit width of the first operator can be set according to the input data of the first processing unit, and the target bit width of the first operator is more suitable for processing the input data of the first processing unit, thereby improving the accuracy of the data processing model reasoning. In addition, the bit width of the first operator is no longer set based on manual experience, which reduces the dependence on manual labor and reduces labor costs.

[0296] It should be understood that Figure 6 The method shown can be processed by a CPU, or by a CPU and a GPU together, or by using other processors suitable for neural network calculations without using a GPU, and this application does not impose any restrictions.

[0297] Figure 6 The method described above can be specifically performed by an execution device. Figure 6 The initial data in the method described above may be input data given by a client device.

[0298] The execution device 1110 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, AR / VR, a vehicle-mounted terminal, etc., or it can be a server or a cloud, etc.

[0299] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0300] Combination of the above Figures 1 to 7 , describes in detail the data processing model, the training method of the data processing model and the data processing method of the embodiment of the present application, and will be combined with Figures 8 to 10 , describes the device embodiment of the present application in detail. It should be understood that the data processing device in the embodiment of the present application can execute the various data processing methods and data processing model training methods in the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0301] Figure 8 A hardware system of an electronic device suitable for the present application is shown.

[0302] The method provided in the embodiments of the present application can be applied to various electronic devices capable of image acquisition, such as mobile phones, tablet computers, wearable devices, laptop computers, netbooks, personal digital assistants (PDAs), and vehicle-mounted devices. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0303] Figure 8 The structure diagram of the electronic device 100 is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0304] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0305] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0306] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0307] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0308] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input data and can also continuously self-learn. NPU can realize applications such as intelligent cognition of electronic device 100, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0309] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0310] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0311] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0312] Fig. 9 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the system library of the Android runtime (Android runtime), and the kernel layer. The application layer can include a series of application packages.

[0313] like Fig. 9 As shown, the application layer may include applications such as camera, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, wallpaper, gallery, multimedia editor, etc.

[0314] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0315] like Fig. 9As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0316] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon can include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for the electronic device 100. The resource manager provides various resources for applications. The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction.

[0317] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0318] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0319] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0320] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc. For example, the system library may also include data processing models, etc.

[0321] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0322] The kernel layer is the layer between hardware and software. The kernel layer can include driver modules such as display driver, camera driver, audio driver and sensor driver.

[0323] Fig.10 It is a schematic diagram of a data processing device provided in an embodiment of the present application.

[0324] The data processing device 1000 includes a processing unit 1010 .

[0325] In some embodiments, the data processing device 1000 may be used to perform Figure 6 The data processing method shown.

[0326] The processing unit 1010 is used to determine input data of a first processing unit through a data processing model based on initial data, wherein the data processing model includes a first processing unit, and the first processing unit includes a bit width selector and a first operator.

[0327] The processing unit 1010 is further configured to determine, through the bit width selector of the first processing unit, a target bit width of the first operator from a plurality of candidate bit widths of the first operator based on the input data.

[0328] The processing unit 1010 is further configured to select, from among multiple parameters of the first operator through the data processing model, a target parameter whose bit width is the target bit width, wherein different parameters of the first operator correspond to different candidate bit widths of the first operator.

[0329] The processing unit 1010 is further configured to map the input data into first input adjustment data through the data processing model, where the bit width of the first input adjustment data is the target bit width.

[0330] The processing unit 1010 is further configured to output a processing result corresponding to the first processing unit based on the target parameter and the first input adjustment data through the first operator of the first processing unit.

[0331] The processing unit 1010 is further configured to determine a target processing result based on a processing result corresponding to the first processing unit through the data processing model.

[0332] In some other embodiments, the data processing device 1000 may be used to execute Figure 2 The training method of the data processing model shown.

[0333] The processing unit 1010 is used to determine input data of a first initial processing unit based on training samples through an initial data processing model, wherein the initial data processing model includes a first initial processing unit, and the first initial processing unit includes an initial bit width selector and a first initial operator.

[0334] The processing unit 1010 is further configured to determine, through an initial bit width selector in the first initial processing unit, a training bit width of the first initial operator from among a plurality of candidate bit widths of the first initial operator based on the training input data of the first initial processing unit.

[0335] The processing unit 1010 is further configured to map the training input data of the first initial operator to first training input adjustment data corresponding to the training bit width of the first initial operator through an initial data processing model.

[0336] The processing unit 1010 is also used to process the first training input adjustment data corresponding to the training bit width of the first initial operator based on the training target parameters of the first initial operator through the first initial operator to determine the processing result output by the first initial processing unit. The training target parameters of the first initial operator are the parameters corresponding to the training bit width among the multiple initial parameters of the first initial operator, and different initial parameters of the first initial operator correspond to different candidate bit widths.

[0337] The processing unit 1010 is further configured to determine, through the initial data processing model and based on the processing result output by the first initial processing unit, the training information corresponding to the training sample.

[0338] The processing unit 1010 is also used to adjust the model parameters in the initial data processing model according to the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, and the operator training information of the first initial operator to obtain the data processing model.

[0339] It should be noted that the above data processing device 1000 is embodied in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0340] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0341] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0342] The present application also provides a chip, the chip comprising a data interface and one or more processors. When the one or more processors execute instructions, the one or more processors read instructions stored in a memory through the data interface to implement the data processing method and / or data processing model training method described in the above method embodiment.

[0343] The one or more processors may be general-purpose processors or special-purpose processors. For example, the one or more processors may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices such as discrete gates, transistor logic devices, or discrete hardware components.

[0344] The chip may be used as a component of a terminal device or other electronic devices. For example, the chip may be located in the electronic device 100 .

[0345] The processor and the memory may be provided separately or integrated together. For example, the processor and the memory may be integrated on a system on chip (SOC) of a terminal device. That is, the chip may also include a memory.

[0346] A program may be stored in the memory, and the program may be executed by the processor to generate instructions, so that the processor executes the data processing method and / or data processing model training method described in the above method embodiment according to the instructions.

[0347] Optionally, data may be stored in the memory. Optionally, the processor may read data stored in the memory, and the data may be stored at the same storage address as the program, or may be stored at a different storage address than the program.

[0348] Exemplarily, the memory can be used to store relevant programs of the data processing method provided in the embodiments of the present application, and the processor can be used to call relevant programs of the data processing method stored in the memory to implement the data processing method of the embodiments of the present application.

[0349] For example, based on initial data, input data of a first processing unit is determined through a data processing model, the data processing model includes a first processing unit, and the first processing unit includes a bit width selector and a first operator; based on the input data, a target bit width of the first operator is determined from multiple candidate bit widths of the first operator through the bit width selector of the first processing unit; a target parameter whose bit width is the target bit width is selected from multiple parameters of the first operator through the data processing model, and different parameters of the first operator correspond to different candidate bit widths of the first operator; the input data is mapped to first input adjustment data through the data processing model, and the bit width of the first input adjustment data is the target bit width; based on the target parameter and the first input adjustment data, the first operator of the first processing unit outputs a processing result corresponding to the first processing unit; based on the processing result corresponding to the first processing unit, a target processing result is determined through the data processing model.

[0350] Exemplarily, the memory can be used to store relevant programs of the data processing model training method provided in the embodiments of the present application, and the processor can be used to call relevant programs of the data processing model training method stored in the memory to implement the data processing model training method in the embodiments of the present application.

[0351] For example, based on the training sample, the input data of the first initial processing unit is determined through the initial data processing model, the initial data processing model includes the first initial processing unit, and the first initial processing unit includes an initial bit width selector and a first initial operator; through the initial bit width selector, based on the training input data of the first initial processing unit, the training bit width of the first initial operator is determined from multiple candidate bit widths of the first initial operator; through the initial data processing model, the training input data of the first initial operator is mapped to the first training input adjustment data corresponding to the training bit width of the first initial operator; through the first initial operator based on the training target parameter of the first initial operator, the first initial operator is adjusted. The training input adjustment data corresponding to the training bit width of the operator is processed to determine the processing result output by the first initial processing unit, the training target parameter of the first initial operator is the parameter corresponding to the training bit width among the multiple initial parameters of the first initial operator, and different initial parameters of the first initial operator correspond to different candidate bit widths; through the initial data processing model, the training information corresponding to the training sample is determined based on the processing result output by the first initial processing unit; according to the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, as well as the operator training information of the first initial operator, the model parameters in the initial data processing model are adjusted to obtain the data processing model.

[0352] The chip can be arranged in an electronic device.

[0353] The present application also provides a computer program product, which, when executed by a processor, implements the data processing method and / or data processing model training method described in any method embodiment of the present application.

[0354] The computer program product may be stored in a memory, for example, a program, which is finally converted into an executable target file that can be executed by a processor after preprocessing, compiling, assembling and linking.

[0355] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the data processing method and / or data processing model training method described in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0356] The computer-readable storage medium is, for example, a memory. The memory may be a volatile memory or a nonvolatile memory, or the memory may include both a volatile memory and a nonvolatile memory. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0357] In the description of this application, the terms "first", "second", etc. are used only for descriptive purposes and should not be understood as indicating or implying relative importance, or a specific order or sequence. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0358] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0359] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0360] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0361] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0362] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0363] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0364] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: include: In digital signal processing, Based on the initial data, determine input data of a first processing unit through a data processing model, wherein the data processing model includes a first processing unit, and the first processing unit includes a bit width selector and a first operator; Determining, by the bit width selector of the first processing unit, a target bit width of the first operator from a plurality of candidate bit widths of the first operator based on the input data; Selecting, by the data processing model, a target parameter whose bit width is the target bit width from among a plurality of parameters of the first operator, wherein different parameters of the first operator correspond to different candidate bit widths of the first operator; Mapping the input data into first input adjustment data through the data processing model, wherein the bit width of the first input adjustment data is the target bit width; Outputting, by the first operator of the first processing unit, a processing result corresponding to the first processing unit based on the target parameter and the first input adjustment data; Determining a target processing result based on a processing result corresponding to the first processing unit by using the data processing model; The data processing model is obtained by adjusting the model parameters in the initial data processing model according to the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, and the operator training information of the first initial operator; The initial data processing model comprises a first initial processing unit, wherein the first initial processing unit comprises an initial bit width selector and the first initial operator; The training information corresponding to the training sample is determined by the initial data processing model based on a processing result output by the first initial processing unit, the processing result output by the first initial processing unit is determined by the first initial operator processing the training input adjustment data corresponding to the training bit width of the first initial operator based on the training target parameter of the first initial operator, and the training input adjustment data is obtained by mapping the training input data of the first initial processing unit according to the training bit width of the first initial operator; The training bit width of the first initial operator is determined by the initial bit width selector from multiple candidate bit widths of the first initial operator based on the training input data of the first initial processing unit, the training target parameter of the first initial operator is a parameter corresponding to the training bit width among the multiple initial parameters of the first initial operator, and different initial parameters of the first initial operator correspond to different candidate bit widths; The training input data of the first initial processing unit is determined according to the training sample, and the operator training information of the first initial operator is positively correlated with the training bit width of the first initial operator.

2. The method according to claim 1, characterized in that The data processing model further includes a second operator, the output of which serves as an input of the first processing unit; The step of determining input data of the first processing unit by using a data processing model based on the initial data includes: quantizing the initial data through the data processing model to obtain second input adjusted data, where the bit width of the second input adjusted data is the first bit width, and the first bit width is greater than or equal to the maximum candidate bit width of the first operator in the first processing unit; The input data of the first processing unit is determined based on the second input adjustment data by the second operator.

3. The method according to claim 2, characterized in that The data processing model further includes a second processing unit and a third processing unit, the second processing unit includes a third operator, the third processing unit includes at least one fourth operator, the output of the first processing unit serves as the input of the second processing unit, and the output of the second processing unit serves as the input of the third processing unit; The determining a target processing result based on the processing result corresponding to the first processing unit by using the data processing model includes: Mapping the processing result corresponding to the first processing unit to third input adjustment data by the second processing unit, wherein the bit width of the third input adjustment data is the first bit width; Processing the third input adjustment data by the third operator in the second processing unit; Dequantizing the output data of the third operator by the second processing unit to obtain first dequantized data; The target processing result is determined based on the first inverse quantized data by the at least one fourth operator, and the bit width of the input of each fourth operator is equal to the bit width of the first inverse quantized data.

4. The method according to any one of claims 1 to 3, characterized in that The step of determining, by the bit width selector of the first processing unit, a target bit width of the first operator from a plurality of candidate bit widths of the first operator based on the input data, comprises: Performing global pooling processing based on the input data through the bit width selector to determine global information; By means of the bit width selector, feature extraction is performed on the global information to obtain input data features; The target bit width is determined by the bit width selector according to the input data characteristics.

5. The method according to claim 4, characterized in that The bit width selector is used to perform global pooling processing based on the input data to determine the global information, and the method further includes: performing inverse quantization on the input data to obtain second inverse quantized data; Performing global pooling processing based on the input data through the bit width selector to determine global information includes: performing global pooling processing on the second dequantized data through the bit width selector to obtain the global information.

6. The method according to claim 5, characterized in that Mapping the input data to first input adjustment data through the data processing model includes: quantizing the second dequantized data based on the target bit width through the data processing model to obtain the first input adjustment data.

7. The method according to any one of claims 1 to 3, 5 and 6, characterized in that: There are multiple first processing units in the data processing model, and the processing results corresponding to each of the multiple first processing units serve as input data for the next first processing unit; The determining a target processing result based on the processing result corresponding to the first processing unit by using the data processing model includes: After determining the processing result corresponding to the first first processing unit, continue to determine the processing results corresponding to other first processing units through other first processing units among the plurality of first processing units; until the last first processing unit is passed, and a processing result corresponding to the last first processing unit is determined; The target processing result is determined by the data processing model based on the processing result corresponding to the last first processing unit.

8. The method according to any one of claims 1-3, 5 and 6, characterized in that: The operator training information of the first initial operator includes a training bit width of the first initial operator and / or a computational complexity of the first initial operator processing the training input adjustment data.

9. The method according to any one of claims 1 to 3, 5 and 6, characterized in that: There are multiple first initial processing units in the initial data processing model, and the processing results output by each of the multiple first initial processing units serve as input data for the next first initial processing unit. The training information corresponding to the training sample is determined based on the processing results output by the last first initial processing unit.

10. The method according to claim 9, characterized in that The operator training information of the first initial operator includes a training bit width of the first initial operator and a computation amount of the first initial operator processing the training input adjustment data; The data processing model is obtained by adjusting the model parameters in the initial data processing model according to the difference between the training information corresponding to the training sample and the label information corresponding to the training sample, the difference between the sum of the training bit widths of multiple first initial operators and the second bit width, and the difference between the sum of the calculation amounts of multiple first initial operators and the first calculation amount.

11. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method according to any one of claims 1 to 10.

12. A chip, characterized in that: The method comprises a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to implement the method according to any one of claims 1 to 10.

Images