Data processing method, device, medium and program product

Abstract

This application relates to the field of data processing technology, specifically to a data processing method, device, medium, and program product. The method includes: acquiring input data of a model, and based on the input data, acquiring first input data of the i-th computational layer among the M computational layers of the model, wherein the data type of the first input data is a first data type, and i is an integer from 1 to M; when the first data type does not satisfy N preset data types, determining a target data type from the N preset data types based on the matching degree between the first data type and the N preset data types, wherein the N preset data types are the data types of input data supported by the operators in the i-th computational layer, and N is a positive integer; converting the data type of the first input data to the target data type to obtain second input data; and inputting the second input data into the i-th computational layer to obtain the output result of the i-th computational layer. In this way, the model can support input of more types of initial data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a data processing method, device, medium, and program product. Background Technology

[0002] With the development of artificial intelligence (AI) technology, model sizes are becoming increasingly large. To accelerate model inference and improve operational efficiency, it is usually necessary to quantize the model. During the quantization process, the input and output data types of operators need to be dynamically determined according to the algorithm logic and data statistical range. Therefore, the underlying operator library needs to pre-support all potential type combinations.

[0003] Taking 8-bit quantization as an example, assume that the operator is compatible with both 8-bit integer (int8) and 8-bit unsigned integer (uint8) data types. For an operator with 3 inputs, since each input can be of data type int8 or uint8, at this precision alone, it is necessary to adapt to 8 combinations of input data types, which corresponds to 8 operator versions needing to be pre-deployed in the operator library.

[0004] However, actual quantization strategies are not singular or fixed: low-bit quantization such as 8-bit, 16-bit, and 4-bit are commonly used schemes, and cross-precision hybrid quantization is a common strategy for model accuracy tuning. This leads to an exponential increase in the number of type combinations that a single operator needs to adapt to, resulting in a surge in the workload of development, testing, and maintenance during the quantization process; moreover, most type combinations are called very infrequently during deployment, which can easily lead to resource redundancy and waste. Summary of the Invention

[0005] The purpose of this application is to provide a data processing method, apparatus, medium, and program product.

[0006] In a first aspect of this application, a data processing method is provided, applied to an electronic device. The electronic device includes a first model, which includes M computational layers, where M is a positive integer greater than or equal to 1. The method includes: acquiring input data of the first model, and acquiring first input data of the i-th computational layer among the M computational layers based on the input data, wherein the data type of the first input data is a first data type, and i is an integer from 1 to M; when the first data type does not satisfy N preset data types, determining a target data type from the N preset data types based on the matching degree between the first data type and the N preset data types, wherein the N preset data types are the data types of input data supported by operators in the i-th computational layer, and N is a positive integer greater than or equal to 1; converting the data type of the first input data from the first data type to the target data type to obtain second input data; and inputting the second input data into the i-th computational layer to obtain the output result of the i-th computational layer.

[0007] In this embodiment, it is not necessary to extend the operator implementation version for all potential input data types of the first model. Adaptation can be completed using the N preset data types already supported by the operator, which can reduce the development workload of the operator and reduce maintenance redundancy. At the same time, by filtering the target data type based on the matching degree, a scheme that is more suitable for the first data type can be selected within the operator compatibility range, effectively balancing model inference accuracy and hardware computing performance. In addition, the data type matching and conversion process can be automated without manual intervention, simplifying the deployment difficulty of the model in complex scenarios such as mixed quantization and improving the flexibility and efficiency of model adaptation to different hardware platforms.

[0008] In one possible implementation of the first aspect, the preset data type is determined from the N preset data types based on the matching degree between the first data type and the N preset data types, including: determining the matching value of each preset data type based on preset rules, the first data type, and the N preset data types, wherein the matching value is positively correlated with the matching degree; and determining the preset data type based on the matching value of each preset data type.

[0009] In one possible implementation of the first aspect, the matching values ​​include a first score, a second score, a third score, and a fourth score, all of which are initially 0. The data range corresponding to the first data type is from a first lower limit to a first upper limit, and the data range corresponding to the j-th preset data type is from the j-th preset lower limit to the j-th preset upper limit, where j is an integer ranging from 1 to N. Based on the preset rules, the first data type, and the N preset data types, the matching values ​​of each preset data type are determined, including: determining the j-th matching value corresponding to the j-th preset data type: when the first lower limit is equal to the j-th preset lower limit and the first upper limit is equal to the j-th preset upper limit, the first score of the j-th matching value is updated to: the first score plus the difference between the j-th preset upper limit and the j-th preset lower limit; when the first lower limit is greater than or equal to the j-th preset lower limit, ... Furthermore, if the first upper limit is less than or equal to the j-th preset upper limit, the second score of the j-th matching value is updated as follows: the second score plus the first difference and the second difference, where the first difference is the difference between the first lower limit and the j-th preset lower limit, and the second difference is the difference between the j-th preset upper limit and the first upper limit; if both the first data type and the j-th preset data type contain a sign bit or neither contains a sign bit, the third score of the j-th matching value is updated as the third score plus 2; otherwise, the third score of the j-th input matching value is updated as the third score plus 1; if the first lower limit is less than the j-th preset lower limit, the fourth score of the j-th matching value is updated as follows: the fourth score plus the difference between the j-th preset lower limit and the first lower limit; if the first upper limit is greater than the j-th preset upper limit, the fourth score of the j-th matching value is updated as follows: the fourth score plus the difference between the first upper limit and the j-th preset upper limit.

[0010] In this embodiment of the application, by using the preset matching value scoring rules, combinations that are more suitable for the quantized type can be selected from the preset data types supported by the operator library, thereby reducing the accuracy loss caused by data type conversion.

[0011] In one possible implementation of the first aspect, determining the target data type from N preset data types includes: sorting the N matching values ​​based on preset sorting rules to obtain the preset data type with the highest matching value; and determining the preset data type with the highest matching value as the target data type.

[0012] In one possible implementation of the first aspect, the preset sorting rules are as follows: sort the N matching values ​​based on the size of the first score; if the first scores are the same, sort the N matching values ​​based on the third score; if the first score and the third score are the same, sort the N matching values ​​based on the second score; if the first score, the third score, and the second score are all the same, sort the N matching values ​​based on the fourth score.

[0013] In one possible implementation of the first aspect, the preset sorting rules are as follows: sort the N matching values ​​based on the size of the first score; if the first scores are the same, sort the N matching values ​​based on the third score; if the first score and the third score are the same, sort the N matching values ​​based on the fourth score; if the first score, the third score, and the fourth score are all the same, sort the N matching values ​​based on the second score.

[0014] In one possible implementation of the first aspect, the first input data includes K first sub-data, and the input data corresponding to the target data type includes K second sub-data, where K is an integer greater than or equal to 1; converting the data type of the first input data from the first data type to the target data type to obtain the second input data includes: determining a conversion operator, the conversion operator including K sub-operators, wherein the g-th sub-operator is used to receive the g-th first sub-data and output the g-th first sub-data after data type conversion, the data type of the converted g-th first sub-data is consistent with the data type of the g-th second sub-data, and g is an integer taking values ​​sequentially from 1 to K; inputting the first input data into the conversion operator to obtain the second input data of the target data type output by the conversion operator.

[0015] In this embodiment, a split-type data type conversion is performed for multi-input operators. If the data type of the first sub-data input is consistent with the target data type, redundant conversion can be automatically omitted. This ensures the compatibility of hardware operators and avoids unnecessary conversion overhead, thereby improving the inference efficiency of the model.

[0016] In one possible implementation of the first aspect, the method further includes: when the data type of the first sub-data corresponding to the g-th sub-operator remains unchanged after conversion, updating the conversion operator, wherein the updated conversion operator does not include the g-th sub-operator, and g is an integer taking values ​​sequentially from 1 to K.

[0017] In one possible implementation of the first aspect, the method further includes: quantizing the first input data based on a preset dataset to obtain the data type of the first input data, as well as a first lower limit and a first upper limit.

[0018] In one possible implementation of the first aspect, the quantization process is post-training quantization.

[0019] In one possible implementation of the first aspect, when i is 1, the first input data is the input data of the first model; when i is not 1, the first input data is the data output by the (i-1)th computational layer of the first model.

[0020] In one possible implementation of the first aspect, the data type of the input data of the first model includes at least one of the following: text, image, and speech.

[0021] In one possible implementation of the first aspect, the method further includes: using the data output from the Mth computational layer of the first model as the processing result of the input data of the first model, wherein the first model is used to implement at least one of the following functions: classification, generation, recognition, object detection, and semantic segmentation.

[0022] In a second aspect of this application, an electronic device is provided, including a processor, a memory, and a communication interface. The memory is used to store instructions, and the processor is used to call and execute the instructions to implement the methods described in the first aspect and any possible implementation thereof.

[0023] In a third aspect of this application, a computer-readable medium is provided, on which instructions are stored, which, when executed on an electronic device, cause the electronic device to perform the methods of the first aspect and any possible implementation thereof.

[0024] In a fourth aspect of this application, a computer program product is provided, the program product including computer instructions, which, when executed by an electronic device, cause the electronic device to perform the method of the first aspect and any possible implementation thereof.

[0025] The beneficial effects of the second to fourth aspects mentioned above can be found in the beneficial effects of the first aspect, and will not be repeated here. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings are described below.

[0027] Figure 1 According to some embodiments of this application, a schematic flowchart of a data processing method is shown;

[0028] Figure 2 A schematic diagram of a first model is shown according to some embodiments of this application;

[0029] Figure 3 According to some embodiments of this application, a flowchart illustrating a method for determining a matching value is shown;

[0030] Figure 4 According to some embodiments of this application, a flowchart of a sorting rule is shown;

[0031] Figure 5 According to some embodiments of this application, a flowchart of another sorting rule is shown;

[0032] Figure 6 According to some embodiments of this application, a schematic diagram of data type conversion is shown;

[0033] Figure 7 According to some embodiments of this application, a schematic diagram of the structure of an electronic device 100 is shown. Detailed Implementation

[0034] The illustrative embodiments of this application include, but are not limited to, a data processing method, apparatus, medium, and program product.

[0035] Model quantization: Model quantization is a technique that optimizes a neural network model by reducing the precision of its numerical representations. The core idea of ​​quantization is to convert the model weights and activation values, which were originally represented using high-precision floating-point numbers (such as 32-bit floating-point numbers float32), into a low-precision format (such as 8-bit integers int8), thereby significantly reducing the size of the model and computational requirements while maintaining the model's accuracy as much as possible.

[0036] During the quantization process of the model, the data types of the input and output data of the operator need to be dynamically determined based on factors such as the specific algorithm and the range of statistical data. Therefore, the underlying operator library must pre-support all possible combinations of data types involved in the operator.

[0037] Taking 8-bit quantization as an example, suppose the operator library needs to support both 8-bit integers (int8) and 8-bit unsigned integers (uint8) data types. For an operator with multiple inputs, the number of supported data type combinations grows exponentially. For example, an operator with 3 input data, if each input can be either an 8-bit integer (int8) or an 8-bit unsigned integer (uint8), theoretically needs to support 2... 3 There are eight possible combinations of input data types. In other words, the operator library needs to implement corresponding operator versions for each of these eight different combinations.

[0038] However, in actual model quantization processes, 16-bit quantization, 8-bit quantization, 4-bit quantization, and even lower bit quantization may occur, and mixed-precision quantization between different bit widths may be involved (for example, some computational layers of the model use int8, while others use int16). Requiring the operator library to provide native hardware implementations for all possible data type combinations would lead to a surge in the number of implementations, increasing the burden on development, testing, and maintenance. Furthermore, many of these data type combinations are rarely used in actual deployments, resulting in wasted resources.

[0039] Therefore, to solve the above problems, this application provides a data processing method applied to an electronic device. The method includes: acquiring first input data for a certain computational layer of a model, wherein the data type of the first input data is a first data type; when the first data type does not satisfy a preset data type for input data in the model's operator library, determining a target data type based on the matching degree between the first data type and the preset data type; converting the data type of the first input data from the first data type to the target data type to obtain second input data; and inputting the second input data into the computational layer to obtain the computation result of the computational layer.

[0040] The data processing method provided in this application automatically matches and converts data types not supported by the model to those already supported, eliminating the need to extend operator implementations for all potential input types and reducing development and maintenance costs. Furthermore, the method can filter out suitable preset data types, improving the flexibility of model adaptation across hardware platforms while maintaining model inference accuracy.

[0041] To more clearly illustrate the technical solutions provided in the embodiments of this application, the technical solutions of this application will be described below in conjunction with the accompanying drawings.

[0042] Figure 1An embodiment of this application illustrates a schematic flowchart of a data processing method. It is understood that the electronic device 100 is the executing entity for each step of the data processing method in this application. For simplicity, the executing entity for each step will not be described again in the following description of the method flow. Figure 1 As shown, the data processing method may include the following steps:

[0043] S101: Obtain the input data of the first model, and based on the input data, obtain the first input data of the i-th computational layer among the M computational layers. The data type of the first input data is the first data type.

[0044] Figure 2 This is a schematic diagram of a first model in some embodiments of this application. For example... Figure 2 As shown in the embodiment of this application, the first model includes M computation layers, where M is a positive integer and i is an integer from 1 to M. For example, M can be 1, 2, 5, 10, etc., and correspondingly, i can be 1, 2, 3, 7, etc.

[0045] Each computational layer includes several operators. When i is 1, the first input data is the input data of the first computational layer, and also the input data of the first model. When i is not 1, the first input data is the output data of the (i-1)th computational layer of the first model, that is, the operation result output by the previous computational layer is the input data of the next computational layer.

[0046] In some embodiments of this application, the data type of the input data for the first model includes at least one of the following: text, image, and speech. The first model is used to implement at least one of the following functions: classification, generation, recognition, object detection, and semantic segmentation.

[0047] For example, the input data for the first model includes text. The functions implemented by the first model include text classification, text generation, machine translation, sentiment analysis, and / or semantic segmentation.

[0048] For example, the input data for the first model can be images. The functions implemented by the first model include image classification, object detection, image generation, and face detection and recognition.

[0049] For example, the data type of the input data for the first model includes speech. The functions implemented by the first model include speech recognition, speech synthesis, speaker recognition, speech emotion analysis, speech translation, and / or speech enhancement and noise reduction.

[0050] It is understood that the specific type and form of the processing result of the input data of the first model are determined based on the function of the first model and the content of the input data. This application does not limit the specific type and form of the processing result of the input data of the first model.

[0051] It is understood that the first model may include, but is not limited to, convolutional neural networks (CNN), feedforward neural networks (FNN), recurrent neural networks (RNN), and other neural network models and variations thereof.

[0052] In this embodiment, the first data type refers to the data type after quantization, such as fixed-point data types like int8, uint8, and int16. During model quantization, different computational layers may require data representations with different precisions; therefore, the input data types for each computational layer may differ and need to be configured according to hardware support and precision requirements.

[0053] In some embodiments of this application, the first input data can be quantized based on a preset dataset to obtain the data type of the first input data, i.e., the first data type, and the data range of the first data type. Exemplarily, the quantization process can be post-training quantization or other quantization methods, and this application does not limit it in this regard.

[0054] Assume the data type of the first input data after quantization is [uint8, uint8, uint8]. That is, the first input data contains three sub-inputs. Assume the data range of the first sub-input is between the first lower limit and the first upper limit. Since the data type of the first sub-input is uint8, the first lower limit is 0 and the first upper limit is 255.

[0055] S102: When the first data type does not satisfy the N preset data types, the target data type is determined from the N preset data types based on the matching degree between the first data type and the N preset data types.

[0056] In this embodiment, the N preset data types are the data types of input data supported by the operators in the i-th computation layer, that is, the N preset data types are the data types of operators supported in the operator library of the i-th computation layer. N is a positive integer greater than or equal to 1, such as 1, 2, 5, 10, 15, 80, 100, etc.

[0057] For example, suppose there is an op operator in the i-th computation layer, which has three inputs, denoted as op_inp0, op_inp1, and op_inp2, and one output, op_out0, and the input / output data types supported by the hardware operator implementation are:

[0058] Op_inp_out_dtypes={#idx:[inp_dtypes_list, out_dtypes_list]

[0059] Id0:[[int8,int8,uint8],[int8]],

[0060] Id1:[[int8,uint8,uint8],[uint8]],

[0061] Id2:[[int16,int16,int16],[int16]]

[0062] }

[0063] For ease of subsequent description, the data types of input data supported by this op operator are denoted as op_inp_dtypes=[[int8,int8,uint8], [int8,uint8,uint8], [int16,int16,int16]], and the data types of output data supported by this op operator are denoted as op_out_dtypes=[int8,uint8,int16].

[0064] Based on the data type of the first input data, determine whether the first input data is among the data types supported by the operator library in the i-th computation layer, that is, whether the first data type is in the op_inp_dtypes table. If it is in the table, it means that the first input data type is already supported in the operator library, and no additional data type conversion operation is required. If it is not in the table, it means that the current first input data is not supported by the operator library, and the first data type needs to be converted to a data type already supported by the operator library.

[0065] In some embodiments of this application, the matching value of each preset data type can be determined based on preset rules, a first data type, and N preset data types. The matching value is positively correlated with the matching degree, and the target data type is determined based on the matching value of each preset data type. The target data type is the data type with the highest matching degree with the first data type among the input data data types of operators currently supported in the operator library. Converting the first data type to the target data type can reduce the accuracy loss caused by data type conversion, make full use of the operator library, and improve the hardware compatibility of the model.

[0066] In some embodiments of this application, the matching value includes a first score, a second score, a third score, and a fourth score, all of which are initially 0. For ease of explanation, the matching value can be denoted as score=[0.0, 0.0, 0.0, 0.0], where score[0], score[1], score[2], and score[3] correspond to the first score, the second score, the third score, and the fourth score, respectively.

[0067] For example, the data range corresponding to the first data type is from the first lower limit to the first upper limit, and the data range corresponding to the j-th preset data type is from the j-th preset lower limit to the j-th preset upper limit, where j is an integer that takes values ​​sequentially from 1 to N.

[0068] The following description, in conjunction with the accompanying drawings, uses the determination of the j-th matching value corresponding to the j-th preset data type as an example to illustrate the method for determining the matching value in the embodiments of this application.

[0069] Figure 3 This is a flowchart illustrating a method for determining a matching value in some embodiments of this application. For example... Figure 3 As shown, in some embodiments of this application, determining the matching value of each preset data type based on preset rules, a first data type, and N preset data types may include the following steps S201 to S205:

[0070] S201: When the first lower limit is equal to the j-th preset lower limit and the first upper limit is equal to the j-th preset upper limit, the first score of the j-th matching value is updated to be: the first score plus the difference between the j-th preset upper limit and the j-th preset lower limit.

[0071] In the embodiments of this application, when the first lower limit is equal to the j-th preset lower limit and the first upper limit is equal to the j-th preset upper limit, it can be considered that the first data type and the j-th preset data type are completely matched.

[0072] For example, suppose the j-th matching value corresponding to the j-th preset data type is score=[0.0, 0.0,0.0, 0.0], where score[0], score[1], score[2], and score[3] correspond to the first, second, third, and fourth scores of the j-th matching value, respectively.

[0073] For example, suppose the first input data 'a' includes three inputs, denoted as a = [a0, a1, a2]. Then the first lower limit and the first upper limit of the first input data include: the first lower limit of a0 is a0_min, and the first upper limit of a0 is a0_max; the first lower limit of a1 is a1_min, and the first upper limit of a1 is a1_max; the first lower limit of a2 is a2_min, and the first upper limit of a2 is a2_max.

[0074] For example, suppose the j-th preset data type bj includes three inputs, denoted as bj=[bj0, bj1, bj2]. Then the j-th preset upper limit and j-th preset lower limit of the j-th preset data type bj include: the j-th preset lower limit bj0_min, the j-th preset upper limit bj0_max; the j-th preset lower limit bj1_min, the j-th preset upper limit bj1_max; the j-th preset lower limit bj2_min, the j-th preset upper limit bj2_max.

[0075] If a0_min = bj0_min and a0_max = bj0_max, it can be considered that the data type bj0 that completely corresponds to the input data a0 is found. Then score[0] = score[0] + (bj0_max - bj0_min). Similarly, if a1_min = bj1_min and a1_max = bj1_max, then score[0] = score[0] + (bj1_max - bj1_min). If a2_min = bj2_min and a2_max = bj2_max, then score[0] = score[0] + (bj2_max - bj2_min).

[0076] S202: When the first lower limit is greater than or equal to the j-th preset lower limit, and the first upper limit is less than or equal to the j-th preset upper limit, the second score of the j-th matching value is updated as follows: the second score is added to the first difference and the second difference, where the first difference is the difference between the first lower limit and the j-th preset lower limit, and the second difference is the difference between the j-th preset upper limit and the first upper limit.

[0077] In this embodiment of the application, when the first lower limit is greater than or equal to the j-th preset lower limit, and the first upper limit is less than or equal to the j-th preset upper limit, it can be considered that the data range of the first data type is less than the data range of the j-th preset data type.

[0078] For example, if a0_min>=bj0_min and a0_max<=bj0_max, then score[1]=score[1]+(a0_min-bj0_min)+(bj0_amx-a0_max). Similarly, if a1_min>=bj1_min and a1_max<=bj1_max, then score[1]=score[1]+(a1_min-bj1_min)+(bj1_amx-a1_max). If a2_min>=bj2_min and a2_max<=bj2_max, then score[1]=score[1]+(a2_min-bj2_min)+(bj2_amx-a2_max).

[0079] S203: If both the first data type and the j-th preset data type contain a sign bit or neither contains a sign bit, update the third score of the j-th matching value to the third score plus 2; otherwise, update the third score of the j-th input matching value to the third score plus 1.

[0080] In the embodiments of this application, when both the first data type and the j-th preset data type contain a sign bit or neither contains a sign bit, it indicates that the sign bits of the first data type and the j-th preset data type are consistent.

[0081] For example, if the data types of a0 and bj0 are both unsigned, such as unsigned integers, then score[2] = score[2] + 2. If a1 is signed, such as int, and the data type of bj1 is unsigned, such as uint, then score[2] = score[2] + 1. If the data types of a2 and bj2 are both signed, such as signed floating-point data, then score[2] = score[2] + 2.

[0082] S204: When the first lower limit is less than the j-th preset lower limit, update the fourth score of the j-th matching value to: the fourth score plus the difference between the j-th preset lower limit and the first lower limit.

[0083] In this embodiment of the application, when the first lower limit is less than the j-th preset lower limit, it is considered that the data range of the first data type is greater than the data range of the j-th preset data type.

[0084] Exemplarily, if a0_min < bj0_min, then score[3] = score[3] + (bj0_min - a0_min). Similarly, if a1_min < bj1_min, then score[3] = score[3] + (bj1_min - a1_min), and if a2_min < bj2_min, then score[3] = score[3] + (bj2_min - a2_min).

[0085] S205: When the first upper limit is greater than the j-th preset upper limit, update the fourth score of the j-th matching value to: the fourth score plus the difference between the first upper limit and the j-th preset upper limit.

[0086] In the embodiments of the present application, when the first upper limit is greater than the j-th preset upper limit, it is considered that the data range of the first data type is greater than the data range of the j-th preset data type.

[0087] Exemplarily, if a0_max > bj0_max, then score[3] = score[3] + (a0_max - bj0_max). Similarly, if a1_max > bj1_max, then score[3] = score[3] + (a1_max - bj1_max), and if a2_max > bj2_max, then score[3] = score[3] + (a2_max - bj2_max).

[0088] In some embodiments of the present application, determining the target data type from N preset data types includes: sorting the N matching values based on a preset sorting rule to obtain the preset data type with the highest matching value; determining the preset data type with the highest matching value as the target data type.

[0089] The present application screens the preset data type with the highest matching degree through a hierarchical priority sorting rule, and controls the precision loss caused by type conversion at a relatively low level.

[0090] Figure 4 For a sorting rule of matching values provided in some embodiments of the present application, as Figure 4 shown, in some embodiments of the present application, the preset sorting rule may be:[[]]

[0091] S301: Sort the N matching values based on the magnitude of the first score;

[0092] S302: When the magnitudes of the first scores are the same, sort the N matching values based on the third score;

[0093] S303a: When the first score and the third score are both the same, sort the N matching values based on the second score;

[0094] S304a: If the first score, third score, and second score are all the same, sort the N matching values ​​based on the fourth score.

[0095] For example, a sorting function can be used to sort the matches of the preset data types. The custom sorting function is key=lambda x: (score[0], score[2], score[1], score[3]), which sorts the matching values ​​according to the priority of the first score, the third score, the second score, and the fourth score. Suppose there is a set of matching values ​​of the preset data type [[0,128,128,128],[255,16384,128,-128]]. The result of sorting according to the above sorting function is [[0,128,128,128],[255,16384,128,-128]]. The matching value at the last position of the sorted sequence is the highest, and the preset data type corresponding to this matching value is the target data type.

[0096] Figure 4 The hierarchical priority sorting logic shown prioritizes perfect matching (first score) to minimize the accuracy loss of type conversion, followed by sign consistency (third score) to reduce cross-sign conversion error, then range coverage (second score) to filter hardware compatibility types, and finally data range out of bounds (fourth score) to exclude high-risk options. This prioritizes ensuring that the range of data after conversion is not lost, thereby improving the efficiency of model deployment while ensuring the accuracy of the converted data.

[0097] Figure 5 Another sorting rule for matching values ​​provided in some embodiments of this application, such as Figure 5 As shown, in some embodiments of this application, the preset sorting rule may also be:

[0098] S301: Sort the N matching values ​​based on the size of the first score;

[0099] S302: If the first scores are the same, sort the N matching values ​​based on the third score;

[0100] S303b: If the first score and the third score are the same, sort the N matching values ​​based on the fourth score;

[0101] S304b: If the first, third, and fourth scores are all the same, sort the N matching values ​​based on the second score.

[0102] Figure 5The hierarchical priority sorting logic shown prioritizes data types that are closer to the preset data type as the target data type to optimize computational efficiency, giving priority to computational performance. Figure 4 The precision-priority sorting rules shown are complementary, which can flexibly meet the precision control requirements of different businesses and further improve the hardware adaptation efficiency of the model.

[0103] It is understandable that after sorting by the above sorting function, there may be preset data types with the same matching value. In some embodiments, the preset data type located at the end of the sorting sequence can be selected as the target data type.

[0104] It is understood that this application does not limit the preset sorting rules, and different matching value sorting rules can be customized according to the actual needs of the model application.

[0105] S103: Convert the data type of the first input data from the first data type to the target data type to obtain the second input data.

[0106] Figure 6 Provided for some embodiments of this application, such as Figure 6 As shown, in some embodiments of this application, the first input data includes K first sub-data. The input data corresponding to the target data type includes K second sub-data, where K is an integer greater than or equal to 1, such as 1, 2, 3, 8, 15, etc.

[0107] Converting the data type of the first input data from a first data type to a target data type to obtain the second input data includes: determining the conversion operator, such as... Figure 6 As shown, the conversion operator includes K sub-operators, where the g-th sub-operator is used to receive the g-th first sub-data and output the g-th first sub-data after data type conversion. The data type of the converted g-th first sub-data is consistent with the data type of the g-th second sub-data, and g is an integer that takes values ​​from 1 to K.

[0108] The first input data is input into the transformation operator, that is, the K first sub-data are input into the K sub-operators respectively, and the second input data output by the transformation operator is consistent with the target data type.

[0109] In some embodiments of this application, when the data type of the first sub-data corresponding to the g-th sub-operator remains unchanged after conversion, the conversion operator is updated. The updated conversion operator does not include the g-th sub-operator, where g is an integer that takes values ​​sequentially from 1 to K.

[0110] In some specific embodiments, the above sub-operator can be a cast operator.

[0111] This application, by splitting the conversion operator into sub-operators that correspond one-to-one with the input sub-data, enables data type conversion for each input port of a multi-input operator, ensuring that the input data matches the combinations of operator input types already supported by the hardware. It also supports automatic removal of redundant sub-operators with no type change, reducing invalid computation nodes in the inference graph and lowering additional conversion overhead. This avoids redundant development of all types of multi-input operator combinations and improves the inference efficiency of the quantization model while maintaining accuracy. It should be noted that the above description only mentions the data type conversion of the operator's input data; in fact, the operator's output data can also be converted into data types already supported by the hardware operator library using the matching degree method provided in this application.

[0112] In this embodiment, there is no need to manually configure the input data types of each computational layer of the model, enabling faster data quantization while maintaining a certain level of model accuracy. Using the data type conversion method of this embodiment, the operator library only needs to implement a small number of core data type combinations. When the input data of an operator is not in the operator library, the quantization configuration of the input data can be completed simply by inserting the conversion operator provided in this application into the input side of the operator, significantly reducing development and maintenance workload. Furthermore, the data type conversion supports both quantized and non-quantized fixed-point data types, making it applicable to a wider range of scenarios. Moreover, the data type conversion is not dependent on specific hardware architectures, allowing for flexible adaptation to various computing platforms and improving model deployment efficiency.

[0113] S104: Input the second input data into the i-th computation layer to obtain the output result of the i-th computation layer.

[0114] Understandable. Figure 1 The process shown is merely an illustrative example; in other embodiments, Figure 1 The process shown may include more or fewer steps, and this application does not limit this.

[0115] For example, in some embodiments, the electronic device 100 may also use the data output by the Mth computational layer of the first model as the processing result of the input data of the first model.

[0116] It is understood that the data processing method provided in this application can be applied to any electronic device capable of running neural network models, including but not limited to mobile phones, wearable devices (such as smartwatches), tablets, desktops, laptops, handheld computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, etc. This application does not impose any restrictions on the specific type of electronic device.

[0117] To facilitate understanding of the technical solutions of the embodiments of this application, the structure of the electronic device to which the instruction processing method provided in the embodiments of this application is applicable will be described using an electronic device 100 as an example.

[0118] For example, Figure 7 A schematic diagram of the structure of an electronic device 100 is shown according to some embodiments of this application.

[0119] like Figure 7 As shown, the electronic device 100 includes one or more processors 101, system memory 102, non-volatile memory (NVM) 103, communication interface 104, input / output (I / O) device 105, and system control logic 106 for coupling the processor 101, system memory 102, non-volatile memory 103, communication interface 104 and input / output device 105.

[0120] Processor 101 can be used to control electronic device 100 to execute the data processing method of this application. Processor 101 may include one or more processing units, such as a central processing unit (CPU), graphics processing unit (GPU), digital signal processor, microprocessor, artificial intelligence (AI) processor, or a processing module or circuit of a field programmable gate array (FPGA). Processor 101 may include one or more single-core or multi-core processors.

[0121] In some embodiments, the processor 101 can also be used to perform addition, multiplication and other operations corresponding to each computational layer of the first model through the arithmetic units in the processor (such as the multiply-add (MAC) unit, arithmetic logic unit (ALU).

[0122] System memory 102 is volatile memory, such as random-access memory (RAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc. System memory is used for temporary storage of data and / or instructions.

[0123] The non-volatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), a solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 103 may also be a removable storage medium, such as a secure digital (SD) memory card, etc.

[0124] Specifically, system memory 102 and non-volatile memory 103 may each include a temporary copy and a permanent copy of instructions 107. Instructions 107 may include instructions executed by processor 101, such as instructions for implementing the data processing methods provided in the embodiments of this application.

[0125] The communication interface 104 may include a transceiver for providing a wired or wireless communication interface for the electronic device 100, thereby enabling communication with any other suitable device via one or more networks. In some embodiments, the communication interface 104 may be integrated into other components of the electronic device 100, for example, the communication interface 104 may be integrated into the processor 101.

[0126] Input / output devices 105 may include input devices such as keyboards and mice, and output devices such as monitors.

[0127] System control logic 106 may include any suitable interface controller to provide any suitable interface for other modules of electronic device 100. For example, in some embodiments, system control logic 106 may include one or more memory controllers to provide an interface to system memory 102 and non-volatile memory 103.

[0128] In some embodiments, at least one of the processors 101 may be packaged together with the logic of one or more controllers for the system control logic unit 106 to form a system in package (SiP). In other embodiments, at least one of the processors 101 may also be integrated on the same chip with the logic of one or more controllers for the system control logic unit 106 to form a system on chip (SoC).

[0129] Understandable. Figure 7 The structure of the electronic device 100 shown is merely an example. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0130] In some embodiments, this application also provides a readable storage medium storing a program or instructions that, when executed on an electronic device, cause the electronic device to perform the data processing method described in the above embodiments.

[0131] In some embodiments, this application also provides a program product, including: a program or instructions, which, when run on an electronic device, cause the electronic device to perform the data processing method described in the above embodiments.

[0132] It is understood that, as used herein, the term “module” may refer to or include, or be part of, an application-specific integrated circuit (ASIC), electronic circuitry, a processor (shared, dedicated, or grouped) and / or memory that executes one or more software or firmware programs, combinational logic circuitry, and / or other suitable hardware components that provide the described functionality.

[0133] It is understood that in the various embodiments of this application, the processor may be a microprocessor, a digital signal processor, a microcontroller, etc., and / or any combination thereof. According to another aspect, the processor may be a single-core processor, a multi-core processor, etc., and / or any combination thereof.

[0134] The embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0135] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0136] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0137] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0138] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0139] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0140] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0141] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made thereto without departing from the scope of this application.

Claims

1. A data processing method applied to electronic devices, characterized in that, The electronic device includes a first model, which includes M computing layers, where M is a positive integer greater than or equal to 1; The method includes: Obtain the input data of the first model, and based on the input data, obtain the first input data of the i-th computational layer among the M computational layers. The data type of the first input data is a first data type, and i is an integer from 1 to M. If the first data type does not satisfy the N preset data types, the target data type is determined from the N preset data types based on the matching degree between the first data type and the N preset data types. Wherein, the N preset data types are the data types of input data supported by the operators in the i-th computation layer, and N is a positive integer greater than or equal to 1; The data type of the first input data is converted from the first data type to the target data type to obtain the second input data; The second input data is input into the i-th computation layer to obtain the output result of the i-th computation layer.

2. The method according to claim 1, characterized in that, The step of determining the preset data type from the N preset data types based on the matching degree between the first data type and the N preset data types includes: Based on preset rules, the first data type, and the N preset data types, a matching value for each preset data type is determined, and the matching value is positively correlated with the matching degree. The preset data type is determined based on the matching value of each preset data type.

3. The method according to claim 2, characterized in that, The matching value includes a first score, a second score, a third score, and a fourth score, and the initial values ​​of the first score, the second score, the third score, and the fourth score are all 0; The data range corresponding to the first data type is from the first lower limit to the first upper limit, and the data range corresponding to the j-th preset data type is from the j-th preset lower limit to the j-th preset upper limit, where j is an integer that takes values ​​sequentially from 1 to N; The step of determining the matching value of each preset data type based on preset rules, the first data type, and the N preset data types includes: Determine the j-th matching value corresponding to the j-th preset data type: When the first lower limit is equal to the j-th preset lower limit and the first upper limit is equal to the j-th preset upper limit, the first score of the j-th matching value is updated to be: the first score plus the difference between the j-th preset upper limit and the j-th preset lower limit; When the first lower limit is greater than or equal to the j-th preset lower limit, and the first upper limit is less than or equal to the j-th preset upper limit, the second score of the j-th matching value is updated as follows: the second score is added to the first difference and the second difference, where the first difference is the difference between the first lower limit and the j-th preset lower limit, and the second difference is the difference between the j-th preset upper limit and the first upper limit; When both the first data type and the j-th preset data type contain a sign bit or neither contains a sign bit, the third score of the j-th matching value is updated to the third score plus 2; otherwise, the third score of the j-th input matching value is updated to the third score plus 1. When the first lower limit is less than the j-th preset lower limit, the fourth score of the j-th matching value is updated as follows: the fourth score is added to the difference between the j-th preset lower limit and the first lower limit; When the first upper limit is greater than the j-th preset upper limit, the fourth score of the j-th matching value is updated as follows: the fourth score plus the difference between the first upper limit and the j-th preset upper limit.

4. The method according to claim 3, characterized in that, The step of determining the target data type from the N preset data types includes: The N matching values ​​are sorted according to a preset sorting rule to obtain the preset data type with the highest matching value; The preset data type with the highest matching value is determined as the target data type.

5. The method according to claim 4, characterized in that, The preset sorting rule is as follows: Sort the N matching values ​​based on the size of the first score; If the first scores are all the same, the N matching values ​​are sorted based on the third score; If the first score and the third score are both the same, the N matching values ​​are sorted based on the second score; If the first score, the third score, and the second score are all the same, the N matching values ​​are sorted based on the fourth score.

6. The method according to claim 4, characterized in that, The preset sorting rule is as follows: Sort the N matching values ​​based on the size of the first score; If the first scores are all the same, the N matching values ​​are sorted based on the third score; If the first score and the third score are both the same, the N matching values ​​are sorted based on the fourth score; If the first score, the third score, and the fourth score are all the same, the N matching values ​​are sorted based on the second score.

7. The method according to claim 1, characterized in that, The first input data includes K first sub-data, and the input data corresponding to the target data type includes K second sub-data, where K is an integer greater than or equal to 1; The step of converting the data type of the first input data from the first data type to the target data type to obtain the second input data includes: A conversion operator is determined, which includes K sub-operators. The g-th sub-operator is used to receive the g-th first sub-data and output the g-th first sub-data after data type conversion. The data type of the converted g-th first sub-data is consistent with the data type of the g-th second sub-data, where g is an integer that takes values ​​from 1 to K. The first input data is input into the conversion operator to obtain the second input data of the target data type output by the conversion operator.

8. The method according to claim 7, characterized in that, The method further includes: If the data type of the first sub-data corresponding to the g-th sub-operator remains unchanged after conversion, the conversion operator is updated. The updated conversion operator does not include the g-th sub-operator, where g is an integer that takes values ​​sequentially from 1 to K.

9. The method according to claim 3, characterized in that, The method further includes: The first input data is quantized based on a preset dataset to obtain the data type of the first input data, as well as the first lower limit and the first upper limit.

10. The method according to claim 9, characterized in that, The quantization process is a post-training quantization process.

11. The method according to any one of claims 1 to 10, characterized in that, When i is 1, the first input data is the input data of the first model; When i is not 1, the first input data is the data output by the (i-1)th computational layer of the first model.

12. The method according to any one of claims 1 to 10, characterized in that, The data type of the input data for the first model includes at least one of the following: text, image, and voice.

13. The method according to any one of claims 1 to 10, characterized in that, The method further includes: using the data output by the Mth computational layer of the first model as the processing result of the input data of the first model, wherein the first model is used to implement at least one of the following functions: classification, generation, recognition, object detection, and semantic segmentation.

14. An electronic device, comprising a processor, a memory, and a communication interface, characterized in that, The memory is used to store instructions, and the processor is used to call and execute the instructions to implement the method as described in any one of claims 1 to 13.

15. A computer-readable medium, characterized in that, The readable medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 13.

16. A computer program product, characterized in that, The program product includes computer instructions that, when executed by an electronic device, perform the method of any one of claims 1 to 13.

Images