A method for data processing and related devices

Abstract

A method for data processing and related devices, which relate to the field of artificial intelligence and are used to fill the gap that the target arrangement feature of target data cannot be determined when the rank is unknown or the rank is known but the dimension values of at least one dimension are unknown. The method includes: obtaining at least one input information, each input information is used to characterize the arrangement feature of the target data, the arrangement feature includes a first feature, the value of the first feature is a first value or the value of the first feature is a second value, the first value is used to represent the rank of the unknown target data, and the second value is used to represent the rank of the known target data and the dimension values corresponding to at least one dimension are unknown; obtaining the constraint conditions corresponding to the target operator; processing the at least one input information based on the constraint conditions to obtain at least one output information, each output information is used to characterize the target arrangement feature of the target data, and the value of the target arrangement feature includes the target value of the first feature.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a data processing method and related equipment. Background Art

[0002] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, basic AI theory, etc. With the rapid development of AI and the further expansion of deep learning applications, it is necessary not only to implement some customized operators, but also to implement a set of derivation methods for the shape of the output data of the operator, so that the derivation method can be called during image compilation or image running.

[0003] In the related art, only -1 is used to represent unknown dimension values, and the maximum and minimum values ​​are used to represent the range of the dimension values ​​to complete the derivation of the shape of the output data. However, in a neural network (NN) model, the dimension value and rank value of the input data are often unknown, which poses a new challenge to how to better derive the shape of the output data. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and related equipment, aiming to fill the gap of being unable to determine the target arrangement characteristics of the target data when the rank is unknown or the rank is known but the dimension value of at least one dimension is unknown.

[0005] The first aspect of the embodiment of the present application provides a method for data processing, which may include: obtaining at least one input information, wherein each input information is used to characterize the arrangement characteristics of the target data, the arrangement characteristics include a first feature, the value of the first feature is a first value or the value of the first feature is a second value, the first value is used to represent the rank of the unknown target data, and the second value is used to represent the rank of the known target data and the dimension value corresponding to at least one dimension is unknown; obtaining the constraint condition corresponding to the target operator, the constraint condition is used to characterize the execution logic of the target operator; processing at least one input information based on the constraint condition to obtain at least one output information, each output information is used to characterize the target arrangement characteristics of the target data, and the value of the target arrangement characteristics includes the target value of the first feature. In the above manner, when the value of the first feature in the arrangement characteristics reflects the rank of the unknown target data or the value of the target data is known but the dimension value corresponding to at least one dimension is unknown, the at least one input information is processed by the constraint condition of the target operator, so that the at least one output information obtained can reflect the target arrangement characteristics of the target data, filling the blank that the target arrangement characteristics of the target data cannot be determined when the rank is unknown or the rank is known but the dimension value of at least one dimension is unknown.

[0006] In some embodiments, the arrangement feature also includes a second feature and a third feature, the value of the second feature is a third value, the third value is used to indicate the range of the rank, the value of the third feature is a fourth value or the value of the third feature is a fifth value, the fourth value is used to indicate the total dimensional range of the target data when the value of the first feature is the first value, and the fifth value is used to indicate the dimensional range corresponding to each dimension when the value of the first feature is the second value; at least one input information is processed based on the constraint condition to obtain at least one output information, including: processing the value of the second feature in at least one input information based on the constraint condition to obtain a first target value, the first target value is used to reflect the target range of the rank in the target data; processing the value of the first feature and the first target value in at least one input information based on the constraint condition to obtain a second target value, the second target value is used to reflect the target value of the rank; processing the value of the third feature and the second target value in at least one input information based on the constraint condition to obtain a third target value, the third target value is used to reflect the target dimensional range of the target data; obtaining at least one output information based on the first target value, the second target value and the third target value.

[0007] In some embodiments, after processing at least one input information based on the constraint condition and obtaining at least one output information, the method may further include: determining the pre-allocated memory range of the corresponding output information based on the first target value and the third target value. In the above manner, the required memory range can be pre-allocated in advance based on the first target value and the third target value during the image compilation period. Compared with determining the memory range based on the known rank and dimension range during the image operation period, the memory allocation can be better optimized before operation, so as to reasonably allocate memory and reduce unnecessary memory waste.

[0008] In some embodiments, the method may further include: selecting a target template based on the first target value, the target template being used to optimize the target operator. In the above manner, a target template matching the first target value can be selected from the database, and unmatched templates can be eliminated, so that the target operator can be optimized and compiled based on the target template to reduce the processing branches of the target operator, thereby selecting a suitable execution algorithm to improve the compilation and execution performance of the target operator.

[0009] In other embodiments, the method may further include: obtaining a target arrangement feature of the target data based on any one of at least one output information, wherein the target arrangement feature includes a first feature, a second feature and a third feature, the value of the first feature in the target arrangement feature is the second target value, the value of the second feature in the target arrangement feature is the first target value, and the value of the third feature in the target arrangement feature is the third target value.

[0010] In other embodiments, the first value is a symbol or a numerical value used to indicate that the rank is unknown.

[0011] A second aspect of an embodiment of the present application provides a data processing device, which may include: a programming interface module, used to obtain at least one input information, wherein each input information is used to characterize the arrangement characteristics of the target data, the arrangement characteristics include a first feature, the value of the first feature is a first value or the value of the first feature is a second value, the first value is used to represent the rank of unknown target data, and the second value is used to represent the rank of known target data and the dimension value corresponding to at least one unknown dimension; a programming interface module, used to obtain constraints corresponding to a target operator, the constraints are used to characterize the execution logic of the target operator; a processing module, used to process at least one input information based on the constraints to obtain at least one output information, each output information is used to characterize the target arrangement characteristics of the target data, and the values ​​of the target arrangement characteristics include the target value of the first feature.

[0012] In some embodiments, the arrangement feature also includes a second feature and a third feature, the value of the second feature is the third value, the third value is used to indicate the range of the rank, the value of the third feature is the fourth value or the value of the third feature is the fifth value, the fourth value is used to indicate the total dimensional range of the target data when the value of the first feature is the first value, and the fifth value is used to indicate the dimensional range corresponding to each dimension when the value of the first feature is the second value; a processing module is specifically used to: process the value of the second feature in at least one input information according to a constraint condition to obtain a first target value, the first target value is used to reflect the target range of the rank in the target data; process the value of the first feature and the first target value in at least one input information according to the constraint condition to obtain a second target value, the second target value is used to reflect the target value of the rank; process the value of the third feature and the second target value in at least one input information according to the constraint condition to obtain a third target value, the third target value is used to range the target dimensional range of the target data; obtain at least one output information according to the first target value, the second target value and the third target value.

[0013] In other embodiments, the processing module is further specifically used to determine a pre-allocated memory range corresponding to the output information based on the first target value and the third target value after processing at least one information based on the constraint condition to obtain at least one output information.

[0014] In other embodiments, the processing module is specifically used to select a target template according to the first target value, and the target template is used to optimize the target operator.

[0015] In other embodiments, the processing module is further specifically used to obtain a target arrangement feature of the target data based on any one of at least one output information, wherein the target arrangement feature includes a first feature, a second feature and a third feature, the value of the first feature in the target arrangement feature is the second target value, the value of the second feature in the target arrangement feature is the first target value, and the value of the third feature in the target arrangement feature is the third target value.

[0016] In other embodiments, the first value is a symbol or a numerical value used to indicate that the rank is unknown.

[0017] The third aspect of the present application provides a data processing device, which may include: a memory for storing computer-readable instructions. It may also include a processor coupled to the memory for executing the computer-readable instructions in the memory to perform the method described in the first aspect or any possible implementation of the first aspect.

[0018] A fourth aspect of the present application provides a computer-readable storage medium, which, when instructions are executed on a computer device, enables the computer device to execute the method described in the first aspect or any possible implementation of the first aspect.

[0019] A fifth aspect of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0020] In a sixth aspect, the present application provides a chip system, which may include a processor for supporting a terminal device or a server to implement the functions involved in the method described in the first aspect or any possible implementation manner of the first aspect.

[0021] Optionally, in combination with the sixth aspect above, in a first possible implementation, the chip system may further include a memory, which is used to store program instructions and data necessary for the terminal device. The chip system may be composed of a chip, or may include a chip and other discrete devices. Among them, the chip system may include an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices. Furthermore, the chip system may also include an interface circuit, etc.

[0022] It should be noted that the beneficial effects brought about by the implementation methods of the second to sixth aspects of the present application can be understood with reference to the implementation method of the first aspect, and will not be repeated here.

[0023] In the technical solution provided in the embodiments of the present application, since the value of the first feature in the arrangement feature can reflect the rank of unknown target data or the value of the target data is known but the dimension value corresponding to at least one dimension is unknown, at least one input information is processed through the constraint conditions of the target operator so that at least one output information obtained can reflect the target arrangement characteristics of the target data, filling the gap that the target arrangement characteristics of the target data cannot be determined when the rank is unknown or the rank is known but the dimension value of at least one dimension is unknown. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of data processing provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of a data processing method provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of a memory allocation scenario applied to an image provided in an embodiment of the present application;

[0027] Figure 4A schematic diagram of a compilation scenario applied to an image provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of the hardware structure of a data processing device provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of the structure of an execution device provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the structure of another data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. It should be understood that the terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application.

[0032] In order to better understand the technical solutions described in this application, the key technical terms involved in the embodiments of this application are explained below:

[0033] Since the embodiments of the present application involve the application of neural networks, for ease of understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.

[0034] (1) Neural Network

[0035] A neural network can be composed of neural units. A neural unit can be a s The output of the operation unit with the intercept 1 as input can be expressed as follows:

[0036]

[0037] Where s=1, 2, ...n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neural units.

[0038] There are many types of neural networks, for example, deep neural network (DNN), also known as multi-layer neural network, that is, a neural network with multiple hidden layers; another example is convolutional neural network (CNN), which is a deep neural network with a convolutional structure. This application does not limit the specific type of neural network involved.

[0039] (2) Operator

[0040] An operator refers to a function that implements a specific function. For example, taking the reshape operator as an example, this operator is used to reinterpret the shape of tensor data. For another example, taking the transpose operator as an example, this operator is used to adjust the dimensional order of tensor data. In this application, the commonly used functions used to build deep learning model algorithms are collectively referred to as operators, and any operation performed on any function can be considered an operator. For example, convolution is a mathematical method of integral transformation. For example, function f3 is generated by two functions f1 and f2, then f1, f2 and the convolution result f3 can be regarded as an operator respectively.

[0041] (3) Arrangement characteristics

[0042] Arrangement features can be understood as the arrangement mode used to characterize the target data, such as the target data can be composed of several dimensions of pixels, and the specific value or specific value range of each dimension corresponding to the dimension value, etc. The arrangement features described above may include but are not limited to the rank of the target data, the dimension range, the rank range and / or the shape of the target data, etc.

[0043] (4) Rank

[0044] The rank can be understood as the number of dimensions of the target data. For example, if the target data consists of two-dimensional pixels, the rank of the target data can be 2.

[0045] (5) Artificial Intelligence Processor

[0046] Artificial intelligence processors, also known as dedicated processors, in the embodiments of the present application, artificial intelligence processors refer to processors for specific applications or fields. For example: graphics processing unit (GPU), also known as display core, visual processor, display chip, is a dedicated processor for image computing on personal computers, workstations, game consoles and some mobile devices (such as tablets, smart phones, etc.). Another example: neural network processor (neural processing unit, NPU), is a dedicated processor for matrix multiplication operations in applications in the field of artificial intelligence, using the "data-driven parallel computing" architecture, and is particularly good at processing massive multimedia data such as videos and images.

[0047] (6) Deep Learning Framework

[0048] In order to meet the growing demand for neural networks, deep learning frameworks have emerged. Through deep learning frameworks, researchers only need to focus on the network structure of deep learning algorithms. By writing simple python (a cross-platform computer programming language) scripts to write network structures, they can complete a complex deep learning network task, thereby realizing model reasoning and training on hardware. In other words, deep learning frameworks are used to lower the development threshold in the field of deep learning, provide a basic computing framework for deep learning, and quickly build deep learning applications. The current mainstream deep learning frameworks in the industry mainly include Tensorflow, Torch, Mxnet, Thenao, Caffe, etc. Taking the convolutional neural network framework Caffe as an example, in actual applications, Caffe supports multiple types of deep learning architectures, image classification and image segmentation, and can also support convolutional neural networks (CNN), convolutional neural networks for target detection (region-CNN, RCNN), long short-term memory neural networks (LSTM) and fully connected neural network designs. Deep learning frameworks can support multiple types of basic operators. Specifically, the multiple types of basic operators involved here can include: common neural network operators. For example, common neural network operators include: convolution / deconvolution operators, pooling operators, activation operators, classifier (softmax) operators, and fully connected operators. Among them, activation operators may include but are not limited to ReLU, Sigmoid, Tanh, and other operators that can be implemented by interpolation.

[0049] In the related art, only -1 is used to represent unknown dimension values, and the maximum and minimum values ​​are used to represent the range of the dimension values, so as to complete the shape derivation of the output data in meters. However, in a neural network (NN) model, it is often the case that both the dimension value and the rank value of the data are unknown. In this case, the target arrangement characteristics of the output data that can be derived are likely to be inaccurate, resulting in poor optimization results in subsequent optimization operator compilation and other scenarios.

[0050] In order to solve the above problems, the embodiments of the present application provide a method for more accurately deriving the target arrangement characteristics of the target data when the rank of the target data is in an unknown state, or when the rank is known but the dimension value corresponding to at least one dimension is also in an unknown state. This allows the data processing device to solve the problem of uncontrollable compilation time due to long traversal time during image optimization based on the target arrangement characteristics in subsequent image data compilation and other scenarios, and improves the optimization effect. Figure 1 A flow chart of data processing provided in an embodiment of the present application is shown. Figure 1 It can be seen that n input information is included, n is an integer greater than or equal to 1, and each input information includes the shape (shape), dimension range (dimensions range), rank (range), and rank range (rankrange) that can represent the input information. After the n input information is subjected to rank range derivation, range derivation, shape derivation, and dimensions range derivation through the constraints of the operator, m input information is obtained, m is an integer greater than or equal to 1, and each output information also includes the derived shape, dimensions range, range, and rank range information.

[0051] It should be noted that the data processing described above can be applied to application scenarios such as image, voice or natural language processing. In practical applications, it can also be applied to scenarios such as video, which is not limited here.

[0052] For a better understanding of this application, please refer to Figure 2 , is a schematic diagram of a data processing method provided in an embodiment of the present application, which is as follows:

[0053] 201. Obtain at least one input information, wherein each input information is used to characterize arrangement characteristics of target data, the arrangement characteristics include a first feature, the value of the first feature is a first value or the value of the first feature is a second value, the first value is used to represent the rank of unknown target data, and the second value is used to represent the rank of known target data and the dimension value corresponding to at least one unknown dimension.

[0054] In the embodiment, the target data described may include but is not limited to image, voice and other data.

[0055] Since each input information can represent different arrangement characteristics of the same target data, and different arrangement characteristics reflect different shapes of the target data, for example: in some input information only the rank of the target data is given, and the value of the rank is 2, then after receiving the input information, the data processing device can know that the target data is 2-dimensional data; for example, in some information, the value of the rank of the target data is 4, then the data processing device should know that the target data is 4-dimensional data.

[0056] It should be noted that the arrangement feature may include a first feature, and the first feature can be understood as the rank feature of the target data described above. When the value of the first feature in a certain input information is the first value, it means that the specific value of the rank of the target data cannot be directly obtained from the input information at this time, that is, the data processing device cannot know the dimensionality of the target data. It should be understood that the first value described above is used to represent the rank of unknown target data, and the first value can be -1, or a range, such as [1, 100]. In actual applications, the first value can also be -2, -3, -4, etc., which cannot determine the dimensionality of the target data, and can also be symbols such as #, %, or &, and can also take other ranges, such as [2, 80], etc., which will not be specifically limited here.

[0057] Alternatively, when the value of the first feature in a certain input information is the second value, and the second value can be used to represent the rank of the known target data but the dimension value corresponding to at least one dimension is unknown, it also means that the data processing device can know from the input information how many dimensions the target data is, but the target operator cannot know the dimension value corresponding to each dimension.

[0058] For example, some input information gives one of the shapes of the target data, such as [-1, 80]. At this time, after receiving the input information, the data processing device can also know that the rank of the target data is 2, that is, the target data is 2-dimensional data, and the dimension value corresponding to the second dimension is 80, while the dimension value corresponding to the other first dimension is in an unknown state. For another example, if the value of the first feature is [-1, -1], the data processing device can know that the rank of the target data is 2, but the dimension values ​​corresponding to the first dimension and the second dimension are in an unknown state.

[0059] Based on this, the data processing device can obtain at least one input information from the operator of the previous level, for example: obtain input information 1 from operator 1, obtain input information 2 from operator 2, ... obtain input information n from operator n (n ≥ 1, and is an integer), etc. In this way, after obtaining at least one input information, the data processing device can obtain the constraint conditions corresponding to the target operator according to the user's needs, so as to process the at least one input information based on the constraint conditions of the target operator.

[0060] 202. Obtain constraint conditions corresponding to the target operator, where the constraint conditions are used to characterize the execution logic of the target operator.

[0061] That is to say, it is understood that after obtaining at least one input information for characterizing the arrangement characteristics of the target data, the data processing device can obtain the constraint conditions corresponding to the target operator based on the user's needs.

[0062] For example, if the user wants to obtain the target arrangement features that can better indicate the target data after taking the intersection from at least one of the input information described above, then the data processing device can determine that the target operator is an addition operator, and obtain the constraints corresponding to the addition operator, that is, the data processing device needs to know the execution logic of the target operator, such as broadcasting, contraction, etc. It is worth noting that in the embodiment of the present application, in addition to the add operator described above, the target operator can also be other operators in actual applications, such as: subtraction operator (sub), multiplication operator (mul), exponential operator (exp), etc.

[0063] 203. Based on the constraint condition, the value of the second feature in at least one input information is processed to obtain a first target value, where the first target value is used to reflect the target range of the rank in the target data, wherein the arrangement feature also includes a second feature and a third feature, the value of the second feature is the third value, the third value is used to indicate the range of the rank, the value of the third feature is the fourth value or the value of the third feature is the fifth value, the fourth value is used to indicate the total dimensional range of the target data when the value of the first feature is the first value, and the fifth value is used to indicate the dimensional range corresponding to each dimension when the value of the first feature is the second value.

[0064] In the embodiment, since the arrangement feature may also include a second feature and a third feature, the second feature described may be understood as a rank range feature, and the third feature may be understood as a dimension range feature. It should be understood that the value of the second feature may be a third value, that is, the range of the rank of the target data is represented by the third value. For example, the third value may be a range such as [2, 7], [3, 8], etc., which is not limited here.

[0065] As for the third feature, when the value of the aforementioned first feature is different, the third feature will also have different values. Specifically, when the value of the first feature in a certain input information is the first value, since the first value can be used to represent the rank of unknown target data, it is difficult to obtain the dimensional value corresponding to each dimension from the corresponding input information. Therefore, the value of the third feature can be the fourth value, and the fourth value represents the total dimensional range of the target data. For example, when the first value is -2, the fourth value can be [50, 200], [30, 150], etc., indicating the total dimensional range when the rank of the target data is unknown, which is not limited here.

[0066] Alternatively, when the value of the first feature in a certain input information is the second value, since the second value can be used to indicate the rank of the known target data but the dimension value corresponding to at least one dimension is unknown, the value of the third feature can be the fifth value, and the fifth value indicates the dimensional range corresponding to each dimension. For example, when the second value is [-1, -1], the fifth value can be [[1, 100], [80, 100]]; or when the second value is [-1, 80], the fifth value can be [[1, 100], [80, 80]], etc., and no limitation is given here.

[0067] In this way, the data processing device can process the value of the second feature in at least one input information based on the constraint condition to obtain a first target value, and the first target value can indicate a target range of the rank.

[0068] 204. Process the value of the first feature in at least one input information and the first target value based on the constraint condition to obtain a second target value, where the second target value is used to reflect the target value of the rank.

[0069] 205. Process the value of the third feature in at least one input information and the second target value based on the constraint condition to obtain a third target value, where the third target value is used to reflect the target dimension range of the target data.

[0070] 206. Obtain at least one output information based on the first target value, the second target value, and the third target value.

[0071] In an embodiment, after obtaining the target range of the rank, the data processing device can further process the value of the first feature and the first target value in at least one input information based on the constraint condition to obtain a second target value, and the second target value can represent the target value of the rank. Then the data processing device also determines a third target value based on the value of the third feature in at least one input information and the second target value, and the third target value reflects the target dimension range of the target data. In this way, the data processing device can obtain at least one output information based on the first target value, the second target value, and the third target value, so that each output information can characterize the target arrangement characteristics of the target data.

[0072] For example, assuming that the target data is an image, and the data processing device obtains two input information for characterizing the arrangement characteristics of the image, which are:

[0073] Input information 1: shape_1: [-1, 80], rank_1: 2, rank_range_1: [2, 2], dimensionsrange_1: [[1, 100], [80, 80]];

[0074] Input information 2: shape_2: none, rank_2: -2, rank_range_2: [2, 7], dimensions range_2: [50, 200]. It should be noted that the first value of the first feature can be understood as rank_2: -2 in input information 2, and the second value can be understood as [-1, 80] in input information 1; in addition, the third value of the second feature can be understood as rank_range_1: [2, 2] in input information 1, and rank_range_2: [2, 7] in input information 2; the fourth value of the third feature can be understood as dimensions range_2: [50, 200] in input information 2, and the fifth value of the third feature can be understood as dimensions range_1: [[1, 100], [80, 80]] in input information 1.

[0075] Therefore, when the target operator is the add operator, the data processing device can take the intersection of rank_range_1: [2, 2] and rank_range_2: [2, 7] based on the rank range derivation constraint in the add operator to obtain the first target value as rank range_out: [2, 2]. Then, the data processing device processes the first target value [2, 2] in combination with rank_2: -2 and rank_1: 2 based on the rank derivation constraint in the add operator to obtain the second target value as rank_out: 2. At this time, it can be determined that the target data should be two-dimensional data. Then, the data processing device can also process dimensions range_1: [[1, 100], [80, 80]], dimensions range_2: [50, 200] and the second target value based on the dimension range derivation constraint in the add operator. Specifically, the data processing device takes the intersection of the dimension range [1, 100] of the first dimension in dimensions range_1: [[1, 100], [80, 80]] and dimensions range_2: [50, 200] to obtain [50, 100], and then takes the intersection of the dimension range [80, 80] of the second dimension in dimensions range_1: [[1, 100], [80, 80]] and dimensions range_2: [50, 200] to obtain [80, 80]. In this way, the final third target value is dimensions range_out: [[50, 100], [80, 80]].

[0076] Based on this, the output information of the image can be: rank range_out: [2, 2], rank_out: 2, and dimensions range_out: [[50, 100], [80, 80]]. Therefore, it can be known from the output information that the image is a two-dimensional image, and the dimension range corresponding to each dimension of the image can be reduced to [[50, 100], [80, 80]] based on the constraints of the target operator, which provides an optimization basis for the subsequent optimization scenarios of the image engine and reduces the traversal time and compilation time during the image optimization process.

[0077] It is worth noting that the output information may also include shape_out: [-1, 80]. In addition, the above description only takes two input information and the add operator as examples. In practical applications, one or more input information may be processed based on the constraint conditions of the target operator to obtain at least one output information. The specific understanding may refer to the example of the add operator, which will not be described in detail here.

[0078] It is further explained that at least one output information is obtained as described above, and the number of the output information will depend on the constraint conditions or constraint characteristics of the target operator. For example, for the relu_grad_v2 operator, if two input information are processed, then two output information are obtained, and no specific limitation will be given here.

[0079] Exemplarily, in some embodiments, after processing at least one input information based on the constraint condition to obtain at least one output information, the method further includes:

[0080] A pre-allocated memory range of the corresponding output information is determined based on the first target value and the third target value.

[0081] It is understandable that since the first target value can reflect the target range of rank in the target data, and the third target value can reflect the target dimensional range of the target data, the data processing device can also calculate the memory size required for pre-storing the corresponding output information based on the first target value and the third target value, that is, pre-allocate the memory range. Through the above method, the required memory range can be pre-allocated in advance based on the first target value and the third target value during the image compilation period. Compared with determining the memory range based on the known rank and dimensional range during the image operation period, the memory allocation can be better optimized before operation, so as to reasonably allocate memory and reduce unnecessary memory waste. For example, refer to Figure 3 , is a schematic diagram of a memory allocation scenario applied to an image provided in an embodiment of the present application. Figure 3 It can be seen that after acquiring the input information of the image, the data processing device processes the input information based on the constraint conditions of the target operator described above to obtain output information, and pre-allocates memory according to the output information.

[0082] Exemplarily, in some other embodiments, the data processing method described above may further include: selecting a target template based on the first target value, the target template being used to optimize the target operator.

[0083] In the embodiment, since the first target value can reflect the target range of the rank in the target data, the data processing device can also select a target template that matches the first target value from the database and remove the template that does not match, so as to optimize and compile the target operator based on the target template to reduce the processing branches of the target operator, thereby selecting a suitable execution algorithm and improving the compilation and execution performance of the target operator. For example, see Figure 4 , is a schematic diagram of a compilation scenario applied to an image provided in an embodiment of the present application. Figure 4It can be seen that the data processing device obtains input information, processes the input information based on the constraints of the target operator described above, obtains output information, and further calls the compilation interface provided by the target operator to select a suitable template for compilation based on the first target value, and finally outputs the compilation result.

[0084] 207. Obtain a target arrangement feature of the target data based on any one of the at least one output information.

[0085] In the embodiment, after obtaining at least one output information, the target arrangement feature of the target data can be obtained based on any output information, filling the gap that the target arrangement feature of the target data cannot be determined when the rank is unknown or the rank is known but the dimension value of at least one dimension is unknown. It should be noted that the target arrangement feature also includes the above-mentioned first feature, second feature and third feature, and the value of the first feature in the target arrangement feature can be the above-mentioned second target value, and the value of the second feature in the target arrangement feature is the above-mentioned first target value, and the value of the third feature in the target arrangement feature is the aforementioned third target value.

[0086] Above, a data processing method provided in an embodiment of the present application is introduced. Through the scheme provided in an embodiment of the present application, at least one output information that can be used to characterize the target arrangement characteristics of the target data can be obtained, filling the gap that the target arrangement characteristics of the target data cannot be determined when the rank is unknown or the rank is known but the dimension value of at least one dimension is unknown.

[0087] It is understandable that, in order to realize the above functions, the above data processing equipment includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0088] Describing from the hardware structure, Figures 1 to 4 The data processing device in the embodiment may be implemented by one physical device, or may be implemented by multiple physical devices together, or may be a logical function module within one physical device, and the embodiments of the present application do not specifically limit this.

[0089] For example, Figure 5The hardware structure diagram of the data processing device provided in the embodiment of the present application is shown, which includes: a communication interface 501 and a processor 502 , and may also include a memory 503 .

[0090] The communication interface 501 may use any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0091] Processor 502 includes, but is not limited to, one or more of a central processing unit (CPU), a network processor (NP), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Processor 502 is responsible for communication line 504 and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 503 can be used to store data used by processor 502 when performing operations.

[0092] The memory 503 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor 502 via a communication line 504. The memory 503 may also be integrated with the processor 502. If the memory 503 and the processor 502 are independent devices, the memory 503 and the processor 502 are connected, for example, the memory 503 and the processor 502 can communicate via a communication line 504. The communication interface 501 and the processor 502 may communicate via a communication line 504 , or the communication interface 501 may be directly connected to the processor 502 .

[0093] The communication line 504 may include any number of interconnected buses and bridges, and the communication line 504 links together various circuits including one or more processors 502 represented by the processor 502 and a memory represented by the memory 503. The communication line 504 may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be further described in this application.

[0094] In a specific embodiment, the data processing device may include: a memory for storing computer-readable instructions. It may also include a communication interface coupled to the memory, for obtaining at least one input information, wherein each input information is used to characterize the arrangement characteristics of the target data, the arrangement characteristics include a first feature, the value of the first feature is a first value or the value of the first feature is a second value, the first value is used to represent the rank of unknown target data, and the second value is used to represent the rank of known target data and the dimension value corresponding to at least one unknown dimension; and obtaining the constraint conditions corresponding to the target operator, the constraint conditions are used to characterize the execution logic of the target operator. It also includes: a processor coupled to the communication interface, for executing the computer-readable instructions in the memory to perform the following operations: processing at least one input information based on the constraint conditions to obtain at least one output information, each output information is used to characterize the target arrangement characteristics of the target data, and the value of the target arrangement characteristics includes the target value of the first feature.

[0095] In a specific embodiment, the arrangement feature also includes a second feature and a third feature, the value of the second feature is the third value, the third value is used to indicate the range of the rank, the value of the third feature is the fourth value or the value of the third feature is the fifth value, the fourth value is used to indicate the total dimensional range of the target data when the value of the first feature is the first value, and the fifth value is used to indicate the dimensional range corresponding to each dimension when the value of the first feature is the second value; the processor is specifically used to: process the value of the second feature in at least one input information based on the constraint condition to obtain a first target value, the first target value is used to reflect the target range of the rank in the target data; process the value of the first feature and the first target value in at least one input information based on the constraint condition to obtain a second target value, the second target value is used to reflect the target value of the rank; process the value of the third feature and the second target value in at least one input information based on the constraint condition to obtain a third target value, the third target value is used to reflect the target dimensional range of the target data; obtain at least one output information based on the first target value, the second target value and the third target value.

[0096] In a specific embodiment, the processor is further used to: process at least one input information based on the constraint condition, and after obtaining at least one output information, determine the pre-allocated memory range of the corresponding output information based on the first target value and the third target value.

[0097] In a specific implementation, the processor is specifically configured to: select a target template based on the first target value, where the target template is used to optimize the target operator.

[0098] In a specific embodiment, the processor is specifically used to: obtain a target arrangement feature of the target data based on any one of at least one output information, wherein the target arrangement feature includes a first feature, a second feature and a third feature, the value of the first feature in the target arrangement feature is the second target value, the value of the second feature in the target arrangement feature is the first target value, and the value of the third feature in the target arrangement feature is the third target value.

[0099] In a specific implementation, the first value is a symbol or a numerical value used to indicate that the rank is unknown.

[0100] See also Figure 6 , is a schematic diagram of the structure of an execution device provided in an embodiment of the present application. Figure 6 As shown, the execution device may include a processor 601 , a memory 602 , a communication bus 603 , and a communication interface 604 , and an artificial intelligence processor 605 is connected to the memory 602 and the communication interface 604 via the communication bus 603 .

[0101] The processor 601 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 601 may be a microprocessor or any conventional processor, etc.

[0102] The processor 601 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the data processing method of the present application may be completed by an integrated logic circuit of hardware in the processor 601 or by instructions in the form of software.

[0103] The memory 602 may be a read-only memory (ROM), a random access memory (RAM) or other memory. In the embodiment of the present application, the memory 602 is used to store data and various software programs.

[0104] Optionally, in the embodiment of the present application, the memory 602 may include a physical device for storing information, which is usually to digitize the information and then store it in a medium using electrical, magnetic or optical methods. The memory of this embodiment may also include: a device that stores information using electrical energy, such as RAM, ROM, etc.; a device that stores information using magnetic energy, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information using optical means, such as a CD or a DVD. Of course, there are other types of memory, such as quantum memory, graphene memory, etc.

[0105] The communication interface 604 uses a transceiver such as, but not limited to, a transceiver to implement communication between the execution device and other devices or a communication network. For example, at least one input information may be received through the communication interface 604 .

[0106] Optionally, the execution device may also include at least one artificial intelligence processor 605 .

[0107] The artificial intelligence processor 605 can be mounted on the main CPU (hest CPU) as a coprocessor, and the main CPU assigns tasks to it. In practical applications, the artificial intelligence processor 605 can implement one or more operations. For example, taking the neural network processor (network processing Unit, NPU) NPU as an example, the core part of the NPU is the operation circuit, which is controlled by the controller to extract the matrix data in the memory 602 and perform multiplication and addition operations.

[0108] Optionally, the artificial intelligence processor 605 may include 8 clusters, each cluster including 4 artificial intelligence processor cores.

[0109] Optionally, the artificial intelligence processor 605 may be an artificial intelligence processor with a reconfigurable architecture. Here, a reconfigurable architecture means that if an artificial intelligence processor can utilize reusable hardware resources and flexibly change its own architecture according to different application requirements, so as to provide an architecture that matches each specific application requirement, then this artificial intelligence processor is called a reconfigurable computing system, and its architecture is called a reconfigurable architecture.

[0110] It should be understood that the execution device is only one example provided in the present embodiment, and the execution device may have more or fewer components than those shown, may combine two or more components, or may have different configurations of components.

[0111] The above mainly describes the data processing device provided in the embodiments of the present application from the perspective of entity functions. From the perspective of functional units, the present application can divide the data processing device into functional units according to the above method embodiments. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one functional unit. The above integrated functional unit can be implemented in the form of hardware or in the form of software functional units.

[0112] For example, when each functional unit is divided in an integrated manner, Figure 7 FIG. 1 is a schematic diagram showing the structure of a data processing device provided in an embodiment of the present application. Figure 7 As shown, an embodiment of the data processing device of the present application may include:

[0113] A programming interface module 701 is used to obtain at least one input information, wherein each input information is used to characterize the arrangement characteristics of the target data, the arrangement characteristics include a first feature, the value of the first feature is a first value or the value of the first feature is a second value, the first value is used to represent the rank of the unknown target data, and the second value is used to represent the rank of the known target data and the dimension value corresponding to at least one unknown dimension;

[0114] The programming interface module 701 is used to obtain the constraint conditions corresponding to the target operator, and the constraint conditions are used to characterize the execution logic of the target operator;

[0115] The processing module 702 is used to process at least one input information based on the constraint condition to obtain at least one output information, each output information is used to characterize the target arrangement feature of the target data, and the value of the target arrangement feature includes the target value of the first feature.

[0116] In some embodiments, the arrangement feature further includes a second feature and a third feature, the value of the second feature is the third value, the third value is used to indicate the range of the rank, the value of the third feature is the fourth value or the value of the third feature is the fifth value, the fourth value is used to indicate the total dimension range of the target data when the value of the first feature is the first value, and the fifth value is used to indicate the dimension range corresponding to each dimension when the value of the first feature is the second value; the processing module 702 is specifically used to:

[0117] Processing the value of the second feature in at least one input information according to the constraint condition to obtain a first target value, where the first target value is used to reflect a target range of the rank in the target data;

[0118] Processing a value of a first feature in at least one input information and a first target value according to a constraint condition to obtain a second target value, where the second target value is used to reflect a target value of the rank;

[0119] Processing the value of the third feature in at least one input information and the second target value according to the constraint condition to obtain a third target value, where the third target value is used to range the target dimension range of the target data;

[0120] At least one output information is obtained according to the first target value, the second target value and the third target value.

[0121] In some embodiments, the processing module 702 is further specifically used to: after processing at least one information based on the constraint condition to obtain at least one output information, determine a pre-allocated memory range corresponding to the output information based on the first target value and the third target value.

[0122] In some embodiments, the processing module 702 is used to select a target template according to the first target value, and the target template is used to optimize the target operator.

[0123] In other embodiments, the processing module 702 is also used to obtain a target arrangement feature of the target data based on any one of the at least one output information, wherein the target arrangement feature includes a first feature, a second feature and a third feature, the value of the first feature in the target arrangement feature is the second target value, the value of the second feature in the target arrangement feature is the first target value, and the value of the third feature in the target arrangement feature is the third target value.

[0124] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0125] The computer program product includes one or more computer instructions. When loading and executing the computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instruction can be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium. For example, the computer instruction can be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server, a data center, etc. that includes one or more available media integration. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive Solid State Disk (SSD)), etc.

[0126] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0127] The data processing method and related equipment provided in the embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for data processing, characterized in that, applied to data processing of a neural network, the data processing including image processing, video processing, audio processing or natural language processing, comprising: A data processing device obtains at least one input information, wherein each of the input information is used to characterize the arrangement feature of target data, the arrangement feature including a first feature, the value of the first feature being a first value or the value of the first feature being a second value, the first value being used to indicate that the rank of the target data is unknown, and the second value being used to indicate that the rank of the target data is known and at least one dimension - corresponding dimension value is unknown, and the target data includes images, videos, audios or texts; The data processing device obtains the constraint conditions corresponding to the target operator, and the constraint conditions are used to characterize the execution logic of the target operator; The data processing device processes the at least one input information based on the constraint conditions to obtain at least one output information, and each of the output information is used to characterize the target arrangement feature of the target data, and the value of the target arrangement feature includes the target value of the first feature.

2. The method according to claim 1, characterized in that, The arrangement feature further includes a second feature and a third feature, the value of the second feature being a third value, the third value being used to indicate the range of the rank, the value of the third feature being a fourth value or the value of the third feature being a fifth value, the fourth value being used to indicate the total dimension range of the target data when the value of the first feature is the first value, and the fifth value being used to indicate the dimension range corresponding to each dimension when the value of the first feature is the second value; The data processing device processes the at least one input information based on the constraint conditions to obtain at least one output information, including: The data processing device processes the value of the second feature in the at least one input information based on the constraint conditions to obtain a first target value, and the first target value is used to reflect the target range of the rank in the target data; The data processing device processes the value of the first feature in the at least one input information and the first target value based on the constraint conditions to obtain a second target value, and the second target value is used to reflect the target value of the rank; The data processing device processes the value of the third feature in the at least one input information and the second target value based on the constraint conditions to obtain a third target value, and the third target value is used to reflect the target dimension range of the target data; The data processing device obtains at least one output information based on the first target value, the second target value and the third target value.

3. The method according to claim 2, characterized in that, After the data processing device processes the at least one input information based on the constraint conditions to obtain at least one output information, the method further includes: The data processing device determines the pre - allocated memory range of the corresponding output information based on the first target value and the third target value.

4. The method according to claim 2 or 3, characterized in that, The described processing method further includes: The data processing device selects a target template based on the first target value, and the target template is used to optimize the target operator.

5. The method according to claim 2 or 3, wherein, The described processing method further includes: The data processing device obtains the target arrangement feature of the target data based on any one of the at least one output information, wherein the target arrangement feature includes the first feature, the second feature, and the third feature, the value of the first feature in the target arrangement feature is the second target value, the value of the second feature in the target arrangement feature is the first target value, and the value of the third feature in the target arrangement feature is the third target value.

6. The method according to any one of claims 1-3, wherein, The first value is a symbol or a value used to represent an unknown rank.

7. A data processing device, wherein, Applied to data processing of a neural network, the data processing includes image processing, video processing, audio processing, or natural language processing, and includes: A programming interface module for obtaining at least one input information, wherein each input information is used to characterize the arrangement feature of the target data, the arrangement feature includes a first feature, the value of the first feature is the first value or the value of the first feature is the second value, the first value is used to represent the unknown rank of the target data, the second value is used to represent the known rank of the target data and the unknown dimension value corresponding to at least one dimension, and the target data includes images, videos, audio, or text; The programming interface module is used to obtain the constraint conditions corresponding to the target operator, and the constraint conditions are used to characterize the execution logic of the target operator; A processing module for processing the at least one input information based on the constraint conditions to obtain at least one output information, and each output information is used to characterize the target arrangement feature of the target data, and the value of the target arrangement feature includes the target value of the first feature.

8. The data processing device according to claim 7, wherein, The arrangement feature further includes a second feature and a third feature, the value of the second feature is the third value, the third value is used to represent the range of the rank, the value of the third feature is the fourth value or the value of the third feature is the fifth value, the fourth value is used to represent the total dimension range of the target data when the value of the first feature is the first value, and the fifth value is used to represent the dimension range corresponding to each dimension when the value of the first feature is the second value; The processing module is used to: Process the value of the second feature in the at least one input information according to the constraint conditions to obtain a first target value, and the first target value is used to reflect the target range of the rank in the target data; Process the value of the first feature in the at least one input information and the first target value according to the constraint conditions to obtain a second target value, and the second target value is used to reflect the target value of the rank. Process the value of the third feature in the at least one input information and the second target value according to the constraint condition to obtain a third target value, where the third target value is used to range the target dimension range of the target data; Obtain at least one output information according to the first target value, the second target value, and the third target value.

9. The data processing device according to claim 8, wherein, The processing module is further configured to determine a pre-allocated memory range corresponding to the output information based on the first target value and the third target value after processing the at least one information based on the constraint condition to obtain at least one output information.

10. The data processing device according to claim 8 or 9, wherein, The processing module is configured to select a target template according to the first target value, and the target template is used to optimize the target operator.

11. The data processing device according to claim 8 or 9, wherein, The processing module is further configured to obtain the target arrangement feature of the target data based on any one of the at least one output information, where the target arrangement feature includes the first feature, the second feature, and the third feature, the value of the first feature in the target arrangement feature is the second target value, the value of the second feature in the target arrangement feature is the first target value, and the value of the third feature in the target arrangement feature is the third target value.

12. The data processing device according to claim 8 or 9, wherein, The first value is a symbol or a value used to represent an unknown rank.

13. A data processing device, wherein, Comprising: A memory for storing computer-readable instructions; Further comprising a processor coupled to the memory for executing the computer-readable instructions in the memory to execute the method described in any one of claims 1 to 6.

14. A computer-readable storage medium, wherein, When the instruction runs on a computer device, the computer device is caused to execute the method described in any one of claims 1 to 6.

15. A computer program product, which when run on a computer, enables the computer to execute the method described in any one of claims 1 to 6.

Images