Method and apparatus for data processing

Abstract

Embodiments of the present application disclose a data processing method and device in the field of artificial intelligence. The data processing method comprises: obtaining numbering information of a first node in a neural network graph, the numbering information comprising tensor numbering and axis numbering, a tensor indicated by the tensor numbering being a to-be-split tensor, and an axis indicated by the axis numbering being a splitting axis of the to-be-split tensor; and splitting the to-be-split tensor indicated by the tensor numbering of the first node along the axis indicated by the axis numbering, the neural network graph being used to represent a neural network, and the first node being a calculation unit of the neural network. By marking the numbering information of each node in the neural network graph, obtaining the numbering information of the node, and completing the splitting of the tensor of the node based on the numbering information, embodiments of the present application can complete the splitting of the nodes in any neural network graph, and can enhance the applicability of the splitting scheme of the neural network graph, so that any neural network graph can be directly split.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to a method and apparatus for data processing. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0003] In the field of deep learning, neural network compilers process neural network models as computational graphs. However, due to the large amount of data to be computed, memory limitations often arise, hindering performance improvements. Neural network graph partitioning schemes can effectively address the memory limitations caused by large data volumes. However, the diverse types of operators within neural network graphs result in overly customized partitioning schemes, preventing their application to partitioning other neural network graphs and thus limiting their application and development.

[0004] Therefore, how to enhance the applicability of neural network graph segmentation schemes so that they can directly segment arbitrary neural network graphs is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a data processing method and apparatus that can enhance the applicability of neural network graph segmentation schemes, thereby enabling the direct segmentation of arbitrary neural network graphs.

[0006] In a first aspect, a data processing method is provided, comprising: obtaining the numbering information of a first node in a neural network graph, the numbering information including a tensor number and an axis number, wherein the tensor indicated by the tensor number is a tensor to be segmented, and the axis indicated by the axis number is a segmentation axis of the tensor to be segmented; segmenting the tensor to be segmented indicated by the tensor number of the first node along the segmentation axis indicated by the axis number, wherein the neural network graph is used to represent a neural network, and the first node is used to represent a computational unit of the neural network.

[0007] By labeling the node's identification information in the neural network graph, the identification information corresponding to each node is decoupled from the specific type and operation of each node. Thus, with this decoupling, when segmenting any specific neural network graph, by applying the technical solution of this application embodiment—that is, labeling the identification information of each node in the neural network graph before segmentation—this application embodiment can complete the segmentation of nodes in any neural network graph and improve the applicability of the neural network graph segmentation scheme. Furthermore, network segmentation achieved through identification information can perfectly replace the previous segmentation methods that required manual network analysis. Simultaneously, because the process is fully automated, neural network segmentation achieved through identification information can find the optimal segmentation method through multiple trials, greatly improving the efficiency and performance gains of neural network graph segmentation and promoting the application and development of neural network graph segmentation schemes.

[0008] In one possible implementation, when the tensor to be segmented is the input tensor of the first node, the numbering information also includes an indication value, which is used to indicate whether there is overlap between adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis.

[0009] By carrying an indication value in the numbering information, the data processing device of this application embodiment can flexibly select the numbering information when splitting the nodes of the neural network graph. Since the axes of some operators will have overlapping data after splitting, resulting in a larger data volume and reduced performance benefits, this application embodiment will give priority to selecting numbering information that does not overlap.

[0010] In one possible implementation, the numbering information also includes an indication value, which indicates whether there is overlap between adjacent sub-tensors obtained by splitting the tensor to be split along the splitting axis.

[0011] By carrying an indication value in the numbering information, the embodiments of this application can determine whether the axes indicated by the axis numbers in the numbering information overlap, and will preferentially select the numbering information that does not overlap.

[0012] In one possible implementation, when the tensor to be segmented is an input tensor of the first node, the indicator value is used to indicate that adjacent sub-tensors obtained by segmenting the tensor along the segmentation axis overlap; or, when the tensor to be segmented is an input tensor of the first node, the indicator value is used to indicate that adjacent sub-tensors obtained by segmenting the tensor along the segmentation axis do not overlap; or, when the tensor to be segmented is an output tensor of the first node, the indicator value is used to indicate that adjacent sub-tensors obtained by segmenting the tensor along the segmentation axis do not overlap.

[0013] In one possible implementation, the second node is the predecessor node of the first node, and the second node belongs to the neural network graph. The method further includes: determining M input intervals corresponding to the first input tensor of the first node, where M is the segmentation of the first input tensor; obtaining the segmentation K of the second output tensor of the second node; and determining K output intervals corresponding to the second output tensor of the second node based on the M input intervals and K, where M and K are both positive integers.

[0014] By using M input intervals corresponding to the input tensor of the first node and K segmentation fractions of the output tensor of the second node, and by inversely determining K output intervals corresponding to the output tensor of the second node, and expressing them in interval form, this embodiment of the application can solve the problem of segmentation aggregation caused by different segmentation fractions in the neural network (regardless of parallel computing or core buffer optimization scenarios, segmentation aggregation represents the end of an optimization action, and in these scenarios, the longer the optimization action, the more obvious the performance gain). Therefore, this scheme can improve the effect of neural network segmentation to a certain extent.

[0015] In one possible implementation, based on the M input intervals and K, the K output intervals corresponding to the second output tensor of the second node are determined, including: if M is greater than K, the M input intervals are aggregated into K output intervals; or, if M is equal to K, the M input intervals and K output intervals are in one-to-one correspondence; or, if M is less than K, the M input intervals are divided into K output intervals.

[0016] In one possible implementation, determining the M input intervals corresponding to the first input tensor of the first node includes: determining the M output intervals corresponding to the first output tensor of the first node, where M is the segmentation number corresponding to the first output tensor of the first node; and performing reverse derivation on the M output intervals corresponding to the first output tensor of the first node to determine the M input intervals corresponding to the first input tensor of the first node.

[0017] By determining the M input intervals corresponding to the input tensor of the first node through reverse derivation, the embodiments of this application can determine the multiple output intervals corresponding to the output tensor of the second node based on the M input intervals. This completes the data grouping dependency, avoids the forced aggregation of data, and can improve the performance benefits brought by neural network segmentation.

[0018] In one possible implementation, the second node is the predecessor node of the first and third nodes, and the second and third nodes belong to the neural network graph. The method further includes: determining the M input intervals corresponding to the first input tensor of the first node and the Y input intervals corresponding to the third input tensor of the third node, where M is the segmentation of the first input tensor and Y is the segmentation of the third input tensor; obtaining the segmentation K of the second output tensor of the second node; and determining the K output intervals corresponding to the output tensor of the second node based on the M input intervals, the Y input intervals and K, where M, K and Y are all positive integers.

[0019] Through the above technical solution, when the second node corresponds to multiple subsequent nodes, and different branches yield multiple output intervals corresponding to the output tensors of the second node, the union of the two intervals derived from different branches is taken to finally determine the multiple output intervals corresponding to the output tensors of the second node. In this way, the embodiments of this application can avoid the problem that multiple successor nodes cannot obtain complete data. Since the obtained data is the union of the intervals of each successor node, it can be guaranteed that each successor node can obtain the data block size it needs, avoiding abnormal neural network inference results or training failure due to data trampling.

[0020] In a second aspect, a data processing apparatus is provided, comprising: an acquisition unit for acquiring the numbering information of a first node in a neural network graph, the numbering information including a tensor number and an axis number, wherein the tensor indicated by the tensor number is a tensor to be segmented, and the axis indicated by the axis number is a segmentation axis of the tensor to be segmented; and a segmentation unit for segmenting the tensor to be segmented indicated by the tensor number of the first node along the segmentation axis indicated by the axis number, wherein the neural network graph is used to represent a neural network, and the first node is used to represent a computational unit of the neural network.

[0021] In one possible implementation, when the tensor to be segmented is the input tensor of the first node, the numbering information also includes an indicator value, which is used to indicate whether there is overlap between adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis.

[0022] In one possible implementation, the numbering information also includes indication information, where the indication value is used to indicate whether there is overlap between adjacent sub-tensors obtained by splitting the tensor to be split along the splitting axis.

[0023] In one possible implementation, when the tensor to be segmented is the input tensor of the first node, the indicator value is used to indicate that the adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis overlap; or, when the tensor to be segmented is the input tensor of the first node, the indicator value is used to indicate that the adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis do not overlap; or, when the tensor to be segmented is the output tensor of the first node, the indicator value is used to indicate that the adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis do not overlap.

[0024] In one possible implementation, the second node is the predecessor node of the first node, the second node belongs to the neural network graph, and the device further includes a processing unit, which is used to determine M input intervals corresponding to the first input tensor of the first node, where M is the segmentation of the input tensor; an acquisition unit is also used to acquire the segmentation K of the second output tensor of the second node; the processing unit is also used to determine K output intervals corresponding to the second output tensor of the second node based on the M input intervals and K, where M and K are both positive integers.

[0025] In one possible implementation, the processing unit is used to: if M is greater than K, aggregate the M input intervals into K output intervals; or, if M equals K, correspond one-to-one between the M input intervals and the K output intervals; or, if M is less than K, divide the M input intervals into K output intervals.

[0026] In one possible implementation, the processing unit is further configured to: determine M output intervals corresponding to the first output tensor of the first node, where M is the segmentation number corresponding to the first output tensor of the first node; and perform reverse derivation on the M output intervals corresponding to the first output tensor of the first node to determine the M input intervals corresponding to the first input tensor of the first node.

[0027] In one possible implementation, the second node is the predecessor node of the first and third nodes, and the second and third nodes belong to a neural network graph. The device further includes a processing unit, which is used to determine M input intervals corresponding to the first input tensor of the first node and Y input intervals corresponding to the second input tensor of the third node, where M is the segmentation of the first input tensor and Y is the segmentation of the second input tensor; an acquisition unit is also used to acquire the segmentation K of the second output tensor of the second node; the processing unit is also used to determine K output intervals corresponding to the output tensor of the second node based on the M input intervals, Y input intervals and K, where M, Y and K are all positive integers.

[0028] Thirdly, a computer-readable storage medium is provided for storing a computer program that, when run on a computer or processor, causes the computer or processor to perform the method as described in any one of the first aspects and any possible implementation thereof.

[0029] Fourthly, a computer program product is provided, the computer program product comprising: computer program code, which, when run, implements the method as described in the first aspect and any possible implementation thereof. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of an artificial intelligence main framework provided in an embodiment of this application.

[0031] Figure 2 This is a schematic diagram of an application environment provided in an embodiment of this application.

[0032] Figure 3 This is a schematic diagram of a convolutional neural network structure provided in an embodiment of this application.

[0033] Figure 4 This is a schematic diagram of another convolutional neural network structure provided in the embodiments of this application.

[0034] Figure 5 This is a schematic diagram of the hardware structure of a chip provided in an embodiment of this application.

[0035] Figure 6 This is a schematic diagram of a system architecture provided in an embodiment of this application.

[0036] Figure 7 This is a schematic flowchart of a data processing method provided in an embodiment of this application.

[0037] Figure 8 This is a schematic flowchart of another data processing method provided in the embodiments of this application.

[0038] Figure 9 This is a schematic diagram illustrating the relationship derivation of nodes provided in the embodiments of this application.

[0039] Figure 10 This is a schematic flowchart illustrating another data processing method provided in the embodiments of this application.

[0040] Figure 11 This is a schematic diagram illustrating another node derivation provided in an embodiment of this application.

[0041] Figure 12 This is a schematic structural diagram of a data processing apparatus according to an embodiment of this application.

[0042] Figure 13 This is a schematic diagram of the hardware structure of a data processing device according to an embodiment of this application. Detailed Implementation

[0043] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0044] The terms "first," "second," and "third," etc., in the specification, embodiments, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0045] Figure 1 This is a schematic diagram of an artificial intelligence main framework provided in an embodiment of this application. The main framework describes the overall workflow of an artificial intelligence system and is applicable to general artificial intelligence field needs.

[0046] The above-mentioned artificial intelligence framework will be elaborated in detail from two dimensions: the "intelligent information chain" (horizontal axis) and the "information technology (IT) value chain" (vertical axis). The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom."

[0047] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of artificial intelligence, information (provided and processed by technology) to the industrial ecosystem of systems.

[0048] (1) Infrastructure: Infrastructure provides computing power to support artificial intelligence systems, enables them to communicate with the outside world, and provides support through basic platforms.

[0049] Infrastructure can communicate with the outside world through sensors, and its computing power can be provided by smart chips.

[0050] The intelligent chips here can be hardware acceleration chips such as central processing units (CPUs), neural-network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).

[0051] The basic platform of the infrastructure can include distributed computing frameworks and related platform guarantees and support, such as cloud storage and computing, and interconnected networks.

[0052] For example, for infrastructure, data can be acquired through sensors and external communication, and then this data can be provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0053] (2) Data: The data at the next layer of infrastructure is used to represent data sources in the field of artificial intelligence. This data includes graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0054] (3) Data processing: The data processing described above typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making. Among these, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolic and formalized data. Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0055] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0056] (4) General abilities: After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0057] (5) Smart products and industry applications: Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, urban safety, and intelligent terminals.

[0058] The embodiments of this application can be applied to many fields of artificial intelligence, such as intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, and urban safety.

[0059] Since the embodiments of this application involve a large number of neural network applications, for ease of understanding, the relevant terms and concepts of neural networks that may be involved in the embodiments of this application will be introduced below.

[0060] (1) Neural Network A neural network (NN) can be composed of neural units, which can refer to units represented by x. s The arithmetic unit takes an intercept of 1 as input, and its output can be: (1-1) Where s = 1, 2, ..., n, n is a natural number greater than 1, W s For x s The weights are denoted by b, where b is the bias of the neural unit.

[0061] f is the activation function of the neural network, which introduces nonlinear characteristics to transform the input signal into the output signal. The output signal of this activation function can be used as the input to the next layer.

[0062] A neural network is a network formed by connecting multiple individual neural units, meaning that the output of one neural unit can be the input of another. The input of each neural unit can be connected to the local receptive field of the previous layer to extract features from the local receptive field, which can be a region composed of several neural units.

[0063] (2) Deep Neural Networks A deep neural network (DNN), also known as a multilayer neural network, can be understood as a neural network with multiple hidden layers. Based on the position of the layers, the internal neural network of a DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. The layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer.

[0064] Although DNNs seem complex, the operation of each layer is actually not complicated. Simply put, it involves the following linear relationship expression: ,in, It is the input vector. It is the output vector. It is an offset vector. It is the weight matrix (also called coefficients). It's an activation function. Each layer simply applies the input vector... The output vector is obtained through this simple operation. Due to the large number of layers and coefficients in a DNN... and offset vector The number of parameters is also quite large. These parameters are defined in DNNs as follows: [as coefficients] For example: Suppose in a three-layer DNN, the linear coefficient from the fourth neuron in the second layer to the second neuron in the third layer is defined as... The superscript 3 represents the coefficient. The index corresponds to the third-level index 2 in the output and the second-level index 4 in the input.

[0065] In summary, the Lth The coefficients from the k-th neuron in layer 1 to the j-th neuron in layer L are defined as follows: .

[0066] It should be noted that the input layer does not have... In deep neural networks, more hidden layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can perform more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers in the trained deep neural network (composed of vectors from many layers). The resulting weight matrix.

[0067] (3) Convolutional neural network (CNN) A Convolutional Neural Network (CNN) is a deep neural network with a convolutional structure. It is a deep learning architecture, which refers to learning at multiple levels of abstraction using machine learning algorithms. As a deep learning architecture, a CNN is a feed-forward artificial neural network, where each neuron can respond to an input image.

[0068] A convolutional neural network (CNN) contains a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer is a layer of neurons in a CNN that performs convolution processing on the input signal. In a convolutional layer, a neuron may only be connected to a subset of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are the convolutional kernel. Shared weights can be understood as the way image information is extracted regardless of location. The convolutional kernel can be formalized as a matrix of random size, and during the training process of the CNN, it can learn appropriate weights. Furthermore, the direct benefit of shared weights is reducing the connections between layers in the CNN, while also reducing the risk of overfitting.

[0069] (4) Loss function In training a deep neural network, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight vector of each layer (usually pre-configuring parameters before the initial update). For example, if the prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the deep neural network predicts the target value or a value very close to it. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, and training the deep neural network becomes a process of minimizing this loss.

[0070] (5) Forward propagation algorithm The forward propagation algorithm, also known as the forward calculation algorithm, is an algorithm that performs calculations from front to back. Using the forward propagation algorithm, it starts from the input layer and calculates layer by layer backward until it reaches the output layer to obtain the output result. The forward propagation algorithm obtains the output layer result by performing calculations layer by layer from front to back.

[0071] (6) Backpropagation algorithm Neural networks can employ backpropagation (BP) to correct the parameters of the initial neural network model during training, thereby reducing the reconstruction error loss. Specifically, forward propagation of the input signal to the output generates error loss; this error loss information is then propagated back to update the parameters of the initial neural network model, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining the optimal parameters of the neural network model, such as the weight matrix.

[0072] (7) Tensor A tensor is a multidimensional data storage format, where the dimension of the data is called the order of the tensor. It can be viewed as a generalization of vectors and matrices in multidimensional space; a vector can be considered a one-dimensional tensor, and a matrix a two-dimensional tensor. In deep learning, tensors are used to store data corresponding to higher-order arrays. Tensors include the following basic properties: Shape: Used to count the number of elements in each dimension; Rank: Used to represent the total number of dimensions; An axis is used to indicate a specific dimension.

[0073] Generally, the rank of a tensor indicates the number of axes in the tensor. When an axis is divisible, it can be called a partitioning axis or a divisible axis.

[0074] Figure 2 This is a schematic diagram of a system architecture provided in an embodiment of this application. Figure 2 In this embodiment, the data acquisition device 260 is used to collect training data. Regarding the data processing method of this application embodiment, if the training data is image data, the training data may include training images and corresponding processing results. For example, the classification result corresponding to the training image may be a manually pre-labeled result.

[0075] After collecting the training data, the data acquisition device 260 stores the training data in the database 230, and the training device 220 trains the target model / rule 201 based on the training data maintained in the database 230.

[0076] The following describes how the training device 220 obtains the target model / rule 201 based on the training data. The training device 220 processes the input training data and compares the output result with the training label corresponding to the training input data until the difference between the output result of the training device 220 and the training label is less than a certain threshold, thereby completing the training of the target model / rule 201.

[0077] The target model / rule 201 in this embodiment can specifically be a neural network model or a neural network graph, such as a convolutional neural network. It should be noted that in practical applications, the training data maintained in the database 230 may not all come from the data acquisition device 260; it may also be received from other devices. Furthermore, it should be noted that the training device 220 may not necessarily train the target model / rule 201 entirely based on the training data maintained in the database 230; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0078] The target model / rule 201 trained by training device 220 can be applied to different systems or devices, such as... Figure 2 The execution device 210 shown can be a terminal, such as a mobile phone, tablet computer, laptop computer, augmented reality (AR) / virtual reality (VR) device, in-vehicle terminal, etc., or it can be a server or cloud service. Figure 2 In this embodiment, the execution device 210 is configured with an input / output (I / O) interface 212 for data interaction with external devices. Users can input data to the I / O interface 212 through the client device 240. The input data may include data to be processed input by the client device.

[0079] The preprocessing module 213 and the preprocessing module 214 are used to preprocess the input data (such as data to be processed) received by the I / O interface 212. In this embodiment, the preprocessing module 213 and the preprocessing module 214 may be omitted (or only one of them may be used), and the calculation module 211 may be used directly to process the input data.

[0080] During the preprocessing of input data by the execution device 210, or during the calculation module 211 of the execution device 210 performing calculations and other related processes, the execution device 210 can call data, code, etc. in the data storage system 250 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 250.

[0081] Finally, I / O interface 212 returns the processing result, such as the data processing result obtained above, to client device 240, thereby providing it to the user.

[0082] It is worth noting that the training device 220 can generate corresponding target models / rules 201 based on different training data for different objectives or tasks. The corresponding target models / rules 201 can be used to achieve the above objectives or complete the above tasks, thereby providing the user with the required results.

[0083] exist Figure 2 In the illustrated scenario, the user can manually provide input data, which can be done through the interface provided by I / O interface 212. Alternatively, the client device 240 can automatically send input data to I / O interface 212. If user authorization is required for the client device 240 to automatically send input data, the user can set the corresponding permissions in the client device 240. The user can view the output results of the execution device 210 on the client device 240, which can be presented in various forms such as display, sound, or animation. The client device 240 can also act as a data acquisition terminal, collecting the input data and output results of the input I / O interface 212 as new sample data and storing them in the database 230. Alternatively, data can be collected directly from the I / O interface 212 without going through the client device 240, using the input data and output results of the input I / O interface 212 as new sample data and storing them in the database 230.

[0084] It is worth noting that, Figure 2 This is merely a schematic diagram of a system architecture provided in an embodiment of this application. The positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 2 In this context, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 may also be placed within the execution device 210.

[0085] like Figure 2 As shown, the target model / rule 201 is obtained by training the training device 220. In this embodiment of the application, the target model / rule 201 can be the neural network in this application. Specifically, the neural network used in this embodiment of the application can be CNN, deep convolutional neural network (DCNN), recurrent neural network (RNN), etc.

[0086] Since CNN is a very common type of neural network, the following will combine... Figure 3 This section focuses on a detailed explanation of the structure of CNNs. As mentioned in the basic concept introduction above, a Convolutional Neural Network (CNN) is a deep neural network with a convolutional structure. It is a deep learning architecture, which refers to learning at multiple levels of abstraction through machine learning algorithms. As a deep learning architecture, CNN is a feed-forward artificial neural network, in which each neuron can respond to the input data.

[0087] The data processing method in this application embodiment specifically employs a neural network structure as follows: Figure 3 As shown. In Figure 3 In this example, the Convolutional Neural Network (CNN) 300 may include an input layer 310, convolutional / pooling layers 320 (where pooling layers are optional), and a neural network layer 330. The input layer 310 acquires the data to be processed and passes it to the convolutional / pooling layers 320 and the subsequent neural network layer 330 for processing, thus obtaining the processed data result. The following section... Figure 3 This section provides a detailed introduction to the internal layer structure of CNN 300.

[0088] Convolutional / pooling layers 320: Convolutional layers: like Figure 3 The convolutional / pooling layer 320 shown may include layers as in Examples 321-326. For instance, in one implementation, layer 321 is a convolutional layer, layer 322 is a pooling layer, layer 323 is a convolutional layer, layer 324 is a pooling layer, layer 325 is a convolutional layer, and layer 326 is a pooling layer; in another implementation, layers 321 and 322 are convolutional layers, layer 323 is a pooling layer, layers 324 and 325 are convolutional layers, and layer 326 is a pooling layer. That is, the output of the convolutional layer can be used as the input to a subsequent pooling layer, or as the input to another convolutional layer to continue the convolution operation.

[0089] The following section will use convolutional layer 321 as an example to introduce the internal working principle of a single convolutional layer.

[0090] Convolutional layer 321 can include many convolution operators, also known as kernels. In data processing, a convolution operator acts as a filter that extracts specific information from the input data matrix. A convolution operator can essentially be a weight matrix, which is usually predefined.

[0091] The weight values ​​in these weight matrices need to be obtained through extensive training in practical applications. The weight matrices formed by the weight values ​​obtained through training can be used to extract information from the input data, thereby enabling the convolutional neural network 300 to make correct predictions.

[0092] When a convolutional neural network 300 has multiple convolutional layers, the initial convolutional layers (e.g., 321) tend to extract more general features, which can also be called low-level features. As the depth of the convolutional neural network 300 increases, the features extracted by later convolutional layers (e.g., 326) become more and more complex, such as high-level semantic features. Features with higher semantic levels are more suitable for the problem to be solved.

[0093] Pooling layer: Because it is often necessary to reduce the number of training parameters, pooling layers are often introduced periodically after convolutional layers, such as... Figure 3 In the example of 320, layers 321-326 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. The sole purpose of the pooling layer in data processing is to reduce the spatial size of the data.

[0094] Neural network layer 330: After processing by the convolutional / pooling layers 320, the convolutional neural network 300 is still insufficient to output the required information. As mentioned earlier, the convolutional / pooling layers 320 only extract features and reduce the parameters introduced by the input data. However, to generate the final output information (the required class information or other relevant information), the convolutional neural network 300 needs to utilize neural network layers 330 to generate one or more outputs representing the required number of classes. Therefore, neural network layers 330 can include multiple hidden layers (such as...). Figure 3 As shown in 331, 332 to 33n) and output layer 340, the parameters contained in these multi-layer hidden layers can be pre-trained based on relevant training data for specific task types, such as recognition, classification, etc.

[0095] After the multiple hidden layers in neural network layer 330, the final layer of the entire convolutional neural network 300 is the output layer 340. This output layer 340 has a loss function similar to classification cross-entropy, specifically used to calculate the prediction error. Once the entire convolutional neural network 300 has undergone forward propagation (such as...), the loss function is applied. Figure 3 Propagation from 310 to 340 degrees is forward propagation, and backward propagation (such as...) is completed. Figure 3The propagation from 340 to 310 (backpropagation) will begin to update the weight values ​​and biases of the layers mentioned above, in order to reduce the loss of the convolutional neural network 300 and the error between the output of the convolutional neural network 300 through the output layer and the ideal result.

[0096] The neural network graph segmentation method in this application embodiment specifically employs a neural network structure as follows: Figure 4 As shown. In Figure 4 In this example, the convolutional neural network (CNN) 300 may include an input layer 310, convolutional / pooling layers 320 (where pooling layers are optional), and neural network layers 330. Figure 3 compared to, Figure 4 Multiple convolutional / pooling layers in the convolutional / pooling layer 320 are executed in parallel, and the extracted features are input into the neural network layer 330 for processing.

[0097] It should be noted that, Figure 3 and Figure 4 The convolutional neural networks shown are merely examples of two possible convolutional neural networks for the neural network graph segmentation method of this application embodiment. In specific applications, the convolutional neural networks used in the neural network graph segmentation method of this application embodiment may also exist in the form of other network models.

[0098] Figure 5 The present application provides a hardware structure for a chip, which includes a neural network processor 50. This chip can be configured as follows: Figure 2 The execution device 210 shown is used to perform the calculations of the computing module 211. This chip can also be placed in, for example... Figure 2 The training device 220 shown is used to complete the training work of the training device 220 and output the target model / rule 201. For example... Figure 3 and Figure 4 The algorithms for each layer in the convolutional neural network shown can all be implemented in, for example... Figure 5 This is achieved in the chip shown.

[0099] The Neural Processing Unit (NPU) 50 is mounted as a coprocessor on the main central processing unit (CPU) (host CPU), and tasks are assigned by the host CPU. The core of the NPU is the arithmetic circuit 503, and the controller 504 controls the arithmetic circuit 503 to retrieve data from the memory (weight memory or input memory) and perform calculations.

[0100] In some implementations, the arithmetic circuit 503 internally includes multiple process engines (PEs). In some implementations, the arithmetic circuit 503 is a two-dimensional pulsating array. The arithmetic circuit 503 can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0101] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 502 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 501 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 508.

[0102] The vector computation unit 507 can further process the output of the arithmetic circuit, such as vector multiplication, vector addition, exponentiation, logarithmic operations, size comparisons, etc. For example, the vector computation unit 507 can be used for network computation in non-convolutional / non-FC layers of neural networks, such as pooling, batch normalization, local response normalization, etc.

[0103] In some implementations, the vector computation unit 507 can store the processed output vector into a unified buffer 506. For example, the vector computation unit 507 can apply a nonlinear function to the output of the arithmetic circuit 503, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit 507 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to the arithmetic circuit 503, for example, for use in subsequent layers of a neural network.

[0104] The unified memory 506 is used to store input data and output data.

[0105] The weight data is directly transferred from the external memory to the input memory 501 and / or the unified memory 506 through the direct memory access controller 505 (DMAC), the weight data in the external memory is stored in the weight memory 502, and the data in the unified memory 506 is stored in the external memory.

[0106] The bus interface unit (BIU) 510 is used to enable interaction between the main CPU, DMAC and instruction fetch memory 509 via a bus.

[0107] The instruction fetch buffer 509, connected to the controller 504, is used to store the instructions used by the controller 504. The controller 504 is used to call the instructions cached in the instruction memory 509 to control the operation of the computing accelerator.

[0108] Generally, the unified memory 506, input memory 501, weighted memory 502, and instruction fetch memory 509 are all on-chip memories. External memory is memory outside the NPU, which can be double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM), or other readable and writable memory.

[0109] in, Figure 3 and Figure 4 The operations of each layer in the convolutional neural network shown can be performed by the operation circuit 503 or the vector calculation unit 507.

[0110] The above-mentioned Figure 2 The execution device 210 in the present application is capable of executing each step of the data processing method of the embodiments of the present application. Figure 3 and Figure 4 The CNN model shown and Figure 5 The chip shown can also be used to execute the various steps of the neural network graph segmentation method of the embodiments of this application. The neural network graph segmentation method of the embodiments of this application will be described below with reference to the accompanying drawings.

[0111] like Figure 6 As shown, this application embodiment provides a system architecture 600. The system architecture includes a local device 601, a local device 602, an execution device 610, and a data storage system 650, wherein the local devices 601 and 602 are connected to the execution device 610 through a communication network.

[0112] The execution device 610 can be implemented by one or more servers. Optionally, the execution device 610 can be used in conjunction with other computing devices, such as data storage devices, routers, load balancers, etc. The execution device 610 can be deployed on a single physical site or distributed across multiple physical sites. The execution device 610 can use data in the data storage system 650 or call program code in the data storage system 650 to implement the neural network graph segmentation method of this application embodiment.

[0113] Users can interact with execution device 610 by operating their respective user devices (such as local device 601 and local device 602). Each local device can represent any computing device, such as a personal computer, computer workstation, smartphone, tablet, smart camera, smart car or other type of cellular phone, media consumption device, wearable device, set-top box, game console, etc.

[0114] Each user's local device can interact with the execution device 610 through a communication network of any communication mechanism / standard. The communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0115] In one implementation, local devices 601 and 602 obtain relevant parameters of the target neural network from execution device 610, deploy the target neural network on local devices 601 and 602, and use the target neural network for data classification or recognition, etc.

[0116] In another implementation, the target neural network can be directly deployed on the execution device 610. The execution device 610 obtains the data to be processed from the local devices 601 and 602, and performs classification or other types of data processing on the data to be processed according to the target neural network.

[0117] The aforementioned execution device 610 can also be a cloud device, in which case the execution device 610 can be deployed in the cloud; or, the aforementioned execution device 610 can also be a terminal device, in which case the execution device 610 can be deployed on the user terminal side. This application embodiment does not limit this.

[0118] Currently, when processing neural network graphs, the tensor of each node in the neural network graph is divided, and the first node of the neural network graph reads the data to be processed from the external memory, while the last node saves the final result to the external memory. The multiple nodes in the middle interact with each other through the read and write buffers on the chip, thereby reducing the number of times the external memory is accessed and improving the utilization of storage space.

[0119] However, in practice, the segmentation scheme of a neural network graph is strongly correlated with the types of operators within the graph. A neural network graph typically has one or more segmentation methods that differ from other neural networks. In particular, different operators in a neural network (in this embodiment, these operators can also be referred to as nodes) often have different formats, which makes it difficult to simply express the segmentation scheme. Therefore, a neural network graph often corresponds to only one segmentation scheme. When a new neural network graph emerges, a new segmentation scheme needs to be customized. This results in the segmentation scheme having overly strong customization attributes, making it impossible to apply it to segmenting other neural network graphs, thus restricting the application and development of neural network graph segmentation schemes.

[0120] In view of the above-mentioned technical problems, the embodiments of this application provide a data processing method and apparatus that can enhance the applicability of neural network graph segmentation methods so as to be able to directly segment arbitrary neural network graphs.

[0121] The following will combine Figure 7 The data processing method provided in the embodiments of this application will be further described.

[0122] Figure 7 This is a schematic flowchart illustrating a data processing method provided in an embodiment of this application. Specifically, as follows... Figure 7 As shown.

[0123] S710, Obtain the numbering information of the first node in the neural network graph. The numbering information includes tensor number and axis number. The tensor number indicates the tensor to be segmented, and the axis number indicates the axis to be segmented. S720, along the splitting axis indicated by the axis number, splits the tensor to be split by the tensor number indicated by the tensor number of the first node.

[0124] It should be understood that a neural network graph is used to represent a neural network, and a node is used to represent a computational unit in the neural network graph. The first node is any one of the multiple segmentation nodes in the neural network graph. For example, the first node can be the first node, the last node, or an intermediate node among the multiple segmentation nodes in the neural network graph. Alternatively, the first node can be any segmentation node in the neural network graph, and it can be the first node, an intermediate node, or the last node in the neural network graph. This application does not limit this.

[0125] It should be understood that a neural network graph consists of multiple nodes. A node can be viewed as a formula for a basic computation, and a tensor is the input or output data of that formula. When a neural network graph has one or more input tensors, each node has one or more corresponding output tensors, and the output tensor of a node is the input tensor of the next node, or in other words, the output tensor of a node is used as the input of the next node.

[0126] It should be understood that in this embodiment, if the first node has 5 tensors, each tensor in the first node is assigned a tensor number. For example, tensor_idx=0, tensor_idx=1, tensor_idx=2, tensor_idx=3, tensor_idx=4. Here, tensor_idx=0 represents the first tensor of the first node, tensor_idx=1 represents the second tensor of the first node, tensor_idx=2 represents the third tensor of the first node, tensor_idx=3 represents the fourth tensor of the first node, and tensor_idx=4 represents the fifth tensor of the first node.

[0127] It should be understood that the shape of each tensor in the first node can be the same or different. For example, the shape of the first tensor is [2, 6, 3], the shape of the second tensor is [2, 6], the shape of the third tensor is [2, 6, 3, 10], the shape of the fourth tensor is [2, 6, 3, 20], and the shape of the fifth tensor is [2, 6, 3, 9, 10]. Therefore, in addition to configuring a tensor number for each tensor in the first node, this embodiment of the application configures an axis number for each axis of each tensor in the first node. For example, {tensor_idx=0, axis=2} is used to represent the first tensor of the first node and the third axis of the first tensor.

[0128] It should be understood that all five tensors of the first node can be tensors to be split, or only some of them can be tensors to be split. It should also be understood that the axes of each tensor in the first node can all be split axes, or only some of them can be split axes. This depends on the specific scenario, and this application embodiment does not impose specific limitations.

[0129] It should be understood that, in the embodiments of this application, when labeling the numbering information of each node in the neural network graph, the tensor number in the numbering information corresponds to a tensor, and the axis number corresponds to the axis of the tensor corresponding to the tensor number; however, when segmenting the nodes in the neural network graph, the tensor indicated by the tensor number in the numbering information obtained by the data processing device in the embodiments of this application is the tensor to be segmented, and the axis indicated by the axis number is the segmentation axis of the tensor to be segmented.

[0130] As one possible implementation, this application embodiment only numbers the tensor to be segmented for each node in the neural network graph, and numbers the segmentation axis of the tensor to be segmented. Therefore, the tensor number in the numbering information indicates the tensor to be segmented, and the axis number indicates the segmentation axis of the tensor to be segmented.

[0131] To facilitate the description of the technical solutions of the embodiments of this application, the embodiments of this application use tensors indicated by tensor numbers as tensors to be segmented and axes indicated by axis numbers as the segmentation axes of tensors to be segmented. However, this description method is not intended to limit the scope of the application.

[0132] Specifically, in the specific segmentation process, the data processing device of this application embodiment obtains the number information of the first node to determine which tensor of the first node is the tensor to be segmented, and determines which axis of the tensor to be segmented is the segmentation axis of the tensor to be segmented.

[0133] It should be noted that, in this embodiment of the application, some or all tensors of each node in the neural network graph are configured with a tensor number, and some or all axes of each tensor are configured with an axis number. However, the axis indicated by the tensor number is generally only valid during the segmentation process. The numbering information of the first node obtained by the data processing device in this embodiment of the application during the segmentation process is used to determine which tensor of the first node is to be segmented, and which segmentation axis of the tensor to be segmented is, and to complete the segmentation of the tensor of the first node based on the numbering information.

[0134] It should be understood that, in this embodiment of the application, the node number information of each node in the neural network graph is labeled in the neural network graph. Specifically, the labeling process may be: A neural network graph can be understood as a diagram containing topological relationships obtained by serializing operator instances and their corresponding attributes. Segmentation information (in this embodiment, the numbering information can also be understood as segmentation information) can be understood as an inherent attribute of the operator. By setting a fixed format for the segmentation information, it can be marked as an attribute on the operator instance as a string. This fixed format facilitates parsing during the actual segmentation process. It should be understood that, in the embodiments of this application, the type of each segment node in the multiple segment nodes of the neural network graph can be different, the same, or some segment nodes can be of the same type. When the number of segment nodes in the neural network graph is multiple, the neural network graph also includes at least one numbering information corresponding to each segment node, and there can be at least one identical numbering information among the numbering information corresponding to different segment nodes, or there can be all different numbering information. For example, if the first node corresponds to three numbering information, namely numbering information A, B, and C, and the second node corresponds to three numbering information, namely numbering information A, C, and D, then the first node and the second node both correspond to the same numbering information A.

[0135] Specifically, the number of times the data processing device in this application acquires the numbering information of the first node in the neural network graph can be one or more. For example, when there are two numbering information, such as {tensor_idx=2, axis=2} and {tensor_idx=4, axis=0}, it means that the data processing device in this application needs to segment the tensors indicated by the tensor numbers in the two numbering information respectively. However, it should be understood that regardless of the number of numbering information acquired by the data processing device, the specific method executed by the data processing device is the same, that is, to determine which tensor of the first node is to be segmented according to the tensor number in the numbering information, and to segment the tensor to be segmented based on the axis indicated by the axis number in the numbering information, and obtain at least two sub-tensors.

[0136] It should be understood that the numbering information for the first node can be multiple. For example, if the first node has two tensors, and each tensor has two axes (i.e., the first tensor has a shape of [2, 16], and the second tensor has a shape of [2, 56]), then the numbering information corresponding to the first node is four. Specifically: First number information = {tensor_idx=0, axis=[0]}.

[0137] Second number information = {tensor_idx=0, axis=[1]}.

[0138] The third number information = {tensor_idx=1, axis=[0]}.

[0139] The fourth number information is {tensor_idx=1, axis=[1]}.

[0140] Where tensor_idx=0 represents the first tensor of the first node, and tensor_idx=1 represents the second tensor of the first node. axis=[0] represents the first splitting axis of the corresponding tensor to be split, and axis=[1] represents the second splitting axis of the corresponding tensor to be split.

[0141] Specifically, the first numbering information indicates that the tensor with tensor number 0 is the tensor to be segmented, and the segmentation axis is the axis indicated by axis number 0; the second numbering information indicates that the tensor with tensor number 0 is the tensor to be segmented, and the segmentation axis is the axis indicated by axis number 1; the third numbering information indicates that the tensor with tensor number 1 is the tensor to be segmented, and the segmentation axis is the axis indicated by axis number 0; the fourth numbering information indicates that the tensor with tensor number 1 is the tensor to be segmented, and the segmentation axis is the axis indicated by axis number 1.

[0142] In summary, the number of numbered information entries is related to the number of tensors in the first node. The axis numbers in each numbered information entry can be uniform, but the specific axes indicated by the uniform axis number can be different. For example, if the axis number of the first tensor is 0, it means that the first tensor is segmented along its first axis; this 0 axis can indicate the H-axis of the first tensor. If the axis number of the second tensor is 0, it means that the second tensor is segmented along its first axis; this 0 axis can indicate the W-axis of the second tensor. If the axis number of the third tensor is 0, it means that the third tensor is segmented along its first axis; this 0 axis can indicate the N-axis of the third tensor. Therefore, multiple numbered information entries can correspond to a uniform axis number; that is, the axis numbers included in different numbered information entries are the same. However, for different tensors, the same axis number can indicate different segmentation axes, and the specific meaning of the axis number is related to the shape of the corresponding tensor.

[0143] It should be understood that a numbering information is used to indicate the segmentation of a tensor of the first node. This numbering information includes a tensor number indicating which tensor needs to be segmented, and an axis number indicating the segmentation axis of the tensor to be segmented. The tensor is then segmented along the segmentation axis indicated by the axis number.

[0144] Specifically, the first node corresponds to at least one number, and different numbers are used to represent different segmentation methods of the first node. For example, when the first node is a convolution operator, the same tensor of the first node can correspond to four segmentation methods, and each number represents one segmentation method of the first node; or, a number is used to represent one segmentation strategy of the first node.

[0145] It should be understood that, in the embodiments of this application, the tensor indicated by the tensor number in the numbering information can be either the input tensor of the first node or the output tensor of the first node. For example, when the tensor indicated by the tensor number in the numbering information is the output tensor of the first node, the output tensor is segmented along the axis indicated by the axis number in the numbering information; correspondingly, the input tensor corresponding to the output tensor of the first node will also be segmented in the same way. For instance, if the output tensor is segmented along the segmentation axis indicated by the axis number of the output tensor, then correspondingly, the input tensor corresponding to the output tensor will also be segmented along the corresponding segmentation axis of the input tensor. Again, for example, when the tensor indicated by the tensor number in the numbering information is the input tensor of the first node, the input tensor is segmented along the axis indicated by the axis number in the numbering information; correspondingly, the output tensor corresponding to the input tensor of the first node will also be segmented in the same way. For example, if the input tensor is segmented along the segmentation axis indicated by the axis number of the input tensor, then the corresponding output tensor will also be segmented along the corresponding segmentation axis of the output tensor. Therefore, in this embodiment, the tensor of the first node can be either the input tensor or the output tensor of the first node, but when segmenting the input tensor of the first node, its corresponding output tensor will also be segmented in the same way.

[0146] As one possible implementation, different numbering information corresponding to the first node can be combined together. For example, in this embodiment, the data processing device acquires a group of numbering information for the first node, wherein the group of numbering information includes at least two numbering information, where the tensors indicated by the tensor numbers in the different numbering information are different. For example, the group of numbering information includes first numbering information and second numbering information, wherein the tensor indicated by the tensor number in the first numbering information is different from the tensor indicated by the tensor number in the second numbering information. In this way, this embodiment can complete the simultaneous or ordered segmentation of different tensors.

[0147] By labeling the node's identification information in the neural network graph, and decoupling the identification information from the node's specific type and operation, this embodiment of the application can achieve the segmentation of any neural network graph when partitioning it. Specifically, by labeling the identification information of each node in the neural network graph before partitioning, the application improves the applicability of the partitioning scheme. Furthermore, network partitioning achieved through identification information can perfectly replace previous partitioning methods that required manual network analysis. Simultaneously, because the process is fully automated, neural network partitioning achieved through identification information can find the optimal partitioning method through multiple trials, greatly improving the efficiency and performance gains of neural network graph partitioning and promoting the application and development of neural network graph partitioning schemes.

[0148] As one possible implementation, when the tensor to be segmented is the input tensor of the first node, the numbering information also includes an indicator value, which is used to indicate whether there is overlap between adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis.

[0149] It should be understood that adjacent subtensors obtained by splitting the tensor along the splitting axis may or may not overlap, depending on the type of splitting axis of the tensor. For example, if the first node is a volume operator, the four-dimensional tensor of the first node includes four splitting axes (e.g., H / N / W / C), where splitting axes H and W may overlap, while splitting axes N and C may not overlap.

[0150] Specifically, the indicator value can be a specific numerical value, such as 0, 1, -1, or it can be a letter, such as a, c, etc.

[0151] For example, when the indicator value is 0, the indicator value indicates that there is overlap between adjacent sub-tensors obtained after the tensor to be segmented is segmented along the segmentation axis indicated by the axis number; or, when the indicator value is 1, the indicator value indicates that there is overlap between adjacent sub-tensors obtained after the tensor to be segmented is segmented along the segmentation axis indicated by the axis number; or, when the indicator value is a, the indicator value indicates that there is overlap between adjacent sub-tensors obtained after the tensor to be segmented is segmented along the segmentation axis indicated by the axis number. For another example, when the indicator value is 0, the indicator value indicates that there is no overlap between adjacent sub-tensors obtained after the tensor to be segmented is segmented along the segmentation axis indicated by the axis number; or, when the indicator value is 1, the indicator value indicates that there is no overlap between adjacent sub-tensors obtained after the tensor to be segmented is segmented along the segmentation axis indicated by the axis number; or, when the indicator value is a, the indicator value indicates that there is no overlap between adjacent sub-tensors obtained after the tensor to be segmented is segmented along the segmentation axis indicated by the axis number.

[0152] The following describes an exemplary representation of this numbering information carrying an indication value. Specifically, Number information 1{tensor_idx=0,axis=[0]}.

[0153] Number information 2{tensor_idx=1, axis=[1], overlap=[-1]}.

[0154] Number information 3{tensor_idx=2, axis=[1], overlap=[1]}.

[0155] Number information 4{tensor_idx=3,axis=[1]}.

[0156] Specifically, the tensor indicated by the tensor number in tensor information 1 is the output tensor of the first node; therefore, tensor information 1 does not need to carry an indication value. The tensor indicated by the tensor number in tensor information 2 is the input tensor of the first node; therefore, tensor information 2 needs to carry an indication value. The tensor indicated by the tensor number in tensor information 3 is the output tensor of the first node; therefore, tensor information 3 needs to carry an indication value. The tensor indicated by the tensor number in tensor information 4 is the output tensor of the first node; therefore, tensor information 4 does not need to carry an indication value. Among them, overlap=[1] indicates that there is overlap between adjacent sub-tensors obtained after the tensor to be segmented indicated by the tensor number is segmented along the segmentation axis indicated by the axis number; overlap=[-1] indicates that there is no overlap between adjacent sub-tensors obtained after the tensor to be segmented indicated by the tensor number is segmented along the segmentation axis indicated by the axis number.

[0157] As another example, overlap=[1] can indicate that there is no overlap between adjacent sub-tensors corresponding to the splitting axis; overlap=[-1] indicates that there is overlap between adjacent sub-tensors obtained after the tensor to be split along the splitting axis indicated by the axis number.

[0158] By carrying an indication value in the numbering information, the data processing device of this application embodiment can flexibly select the numbering information when splitting the nodes of the neural network graph. Since the axes of some operators will have overlapping data after splitting, resulting in a larger data volume and reduced performance benefits, this application embodiment will give priority to selecting numbering information that does not overlap.

[0159] As one possible implementation, the numbering information also includes an indicator value, which indicates whether there is overlap between adjacent sub-tensors obtained after the tensor to be segmented is segmented along the segmentation axis.

[0160] Specifically, regardless of whether the tensor indicated by the tensor number in the numbering information is the input tensor or the output tensor of the first node, the numbering information also carries an indication value. By carrying an indication value in the numbering information, the data processing device of this application embodiment can determine whether the axes indicated by the axis numbers in the numbering information overlap, and will preferentially select numbering information that does not overlap.

[0161] For example, an exemplary expression of the number information carrying the indication value is as follows: Number information #1{tensor_idx=0,axis=[0],overlap=[1]}.

[0162] Number information #2{tensor_idx=1,axis=[1],overlap=[-1]}.

[0163] Number information #3{tensor_idx=2,axis=[1],overlap=[1]}.

[0164] Number information #4{tensor_idx=3, axis=[1], overlap=[1]}.

[0165] Specifically, the tensor indicated by the tensor number in information #1 is either the output or input tensor of the first node. The tensor indicated by the tensor number in information #2 is either the input or output tensor of the first node. The tensor indicated by the tensor number in information #3 is either the output or input tensor of the first node. The tensor indicated by the tensor number in information #4 is either the output or input tensor of the first node.

[0166] As one possible implementation, when the tensor to be segmented is the input tensor of the first node, the indicator value is used to indicate that the adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis overlap; or, when the tensor to be segmented is the input tensor of the first node, the indicator value is used to indicate that the adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis do not overlap; or, when the tensor to be segmented is the output tensor of the first node, the indicator value is used to indicate that the adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis do not overlap.

[0167] For example, when the tensor indicated by the tensor number in the number information #1 is the output tensor of the first node, the overlap=[1] is used to indicate that there is no overlap; when the tensor indicated by the tensor number in the number information #2 is the input tensor of the first node, the overlap=[-1] is used to indicate that there is overlap; when the tensor indicated by the tensor number in the number information #3 is the input tensor of the first node, the overlap=[1] is used to indicate that there is no overlap; when the tensor indicated by the tensor number in the number information #4 is the output tensor of the first node, the overlap=[1] is used to indicate that there is no overlap.

[0168] It should be noted that, in the embodiments of this application, the specific number values ​​of tensor number and axis number in the numbering information can be expressed in numerical form, in alphabetical form, or in other ways. The embodiments of this application do not make specific limitations on this.

[0169] By carrying an indication value in the numbering information, the data processing device of this application embodiment can flexibly select the numbering information when segmenting the nodes of the neural network graph, and supports selecting the best-performing segmentation scheme through multiple attempts to achieve maximum performance gains.

[0170] It should be understood that after partitioning the tensor of the first node in the neural network graph, due to the special nature of sliding window operators in neural networks, additional data dependencies will be introduced after the operator partitioning. (For example, after partitioning a forward convolution operator along the H and W directions, the input data will typically depend on more data at both ends of the corresponding directions.) Because the input / output data dependencies of the operators change after partitioning, the data dependencies between the multiple sub-tensors corresponding to the tensor of this first node also change. If the properties of the partitioned tensors are not corrected accordingly, unexpected results may occur (including but not limited to abnormal calculation results, training failure to converge, etc.). Since deep neural networks have many layers, the changes in data dependencies across multiple layers caused by partitioning will have more serious consequences, leading to a severe loss of neural network accuracy, to the point that it cannot support computation.

[0171] In view of this technical problem, this application provides another data processing method. Based on the characteristics of the segmentation (the output data remains consistent with the original data after concatenation), this application example updates the tensor properties of the segmented tensors layer by layer by reverse deducing data dependencies. This method solves the problem that the output results of each node in multiple segmentation nodes are inconsistent with the original input data after concatenation.

[0172] The following will combine Figure 8 Another data processing method provided in the embodiments of this application will be further described.

[0173] S810, determine the M input intervals corresponding to the first input tensor of the first node, where M is the segmentation of the input tensor of the first node; S820, obtain the segmentation K of the second output tensor of the second node; S830 determines the K output intervals corresponding to the second output tensor of the second node based on M input intervals and K.

[0174] Specifically, in this embodiment, an interval is used to represent the data range of a sub-tensor obtained after the tensor of a node is segmented. For example, if the shape of the first input tensor of the first node is [2, 6, 16, 3], this embodiment uses an interval to represent the input tensor of the first node, that is, the expression of the first input tensor is [0 1 0 5 0 15 0 2]. Each pair of numbers represents an interval corresponding to a segmentation axis. For example, [0 1] represents the interval corresponding to the first segmentation axis of the first input tensor, [0 5] represents the interval corresponding to the second segmentation axis of the first input tensor, [0 15] represents the interval corresponding to the third segmentation axis of the first input tensor, and [0 2] represents the interval corresponding to the fourth segmentation axis of the input tensor. When the input tensor is segmented along a certain segmentation axis, only the expression of the interval corresponding to that segmentation axis is changed. For example, if the first input tensor is divided along its third slicing axis, resulting in four sub-tensors (i.e., the first input tensor is divided into 4 segments), then the interval of the first sub-tensor is [0, 4] (shape [2, 6, 5, 3]), the interval of the second sub-tensor is [5, 8] (shape [2, 6, 4, 3]), the interval of the third sub-tensor is [9, 12] (shape [2, 6, 4, 3]), and the interval of the fourth sub-tensor is [13, 15] (shape [2, 6, 3, 3]). In other words, the first input tensor of the first node corresponds to four input intervals, and each input interval corresponds to one sub-tensor of the first input tensor.

[0175] It should also be understood that when there is overlap between adjacent intervals of the four input intervals corresponding to the first input tensor of the first node, that is, the interval of the first sub-tensor is [0, 4] (shape [2, 6, 5, 3]), the interval of the second sub-tensor is [2, 8] (shape [2, 6, 7, 3]), the interval of the third sub-tensor is [7, 12] (shape [2, 6, 6, 3]), and the interval of the fourth sub-tensor is [10, 15] (shape [2, 6, 6, 3]), by representing the data range of each sub-tensor of the first input tensor with intervals, the embodiments of this application numerically reflect whether there is overlap between adjacent sub-tensors (sub-tensors can also be called sub-nodes or slices), which can simplify the offset calculation of sub-tensors in the process of segmenting complex neural network graphs.

[0176] It should be understood that the second node is the predecessor node of the first node, which means that the second node precedes the first node in the data processing order. See [link to documentation] for details. Figure 9 The diagram shows the relationship between the second node, the third node, and the first node. It should be understood that... Figure 9 The solid arrows indicate the order of data processing, while the dashed arrows indicate the order of derivation.

[0177] Specifically, the output tensor of the second node is the input tensor of the first node. After determining the M input intervals corresponding to the first input tensor of the first node, and obtaining the segmentation K of the second output tensor of the second node, the K output intervals corresponding to the second output tensor of the second node can be determined through the M input intervals and K, and each output interval corresponds one-to-one with each sub-tensor of the output tensor of the second node.

[0178] For example, the four input intervals corresponding to the first input tensor of the first node are [0, 4], [5, 8], [9, 12], and [13, 15], respectively. The second output tensor of the second node has a segmentation of 2. Based on the four input intervals and the segmentation of 2, the two output intervals corresponding to the second output tensor of the second node are determined. For example, the two input intervals [0, 4] and [5, 8] of the first node are aggregated into the interval [0, 8], and the other two input intervals [9, 12] and [13, 15] of the first node are aggregated into the interval [9, 15]. Alternatively, the four input intervals corresponding to the first input tensor of the first node are aggregated into one interval, namely [0, 15], and this new interval is re-divided into two intervals, namely [0, 8] and [9, 15], or [0, 6] and [7, 15], etc. The embodiments of this application do not specifically limit this.

[0179] It should be noted that, in the above Figure 8In the illustrated technical solution, the first input tensor of the first node and the second output tensor of the second node actually exist as sub-tensors, rather than as undivided tensors. In other words, after being divided, the first input tensor of the first node corresponds to M sub-tensors, and each sub-tensor corresponds one-to-one with each input interval; after being divided, the second output tensor of the second node corresponds to K sub-tensors, and each sub-tensor corresponds one-to-one with each output interval.

[0180] Specifically, the first input tensor of the first node is a concept that has been split. Additionally, Figure 8 The proposed solution is based on deriving the data dependencies between the sub-tensors of each segmented node after partitioning the neural network graph. Therefore, during the derivation process, the first input tensor of the first node exists as a sub-tensor, rather than as an unpartitioned tensor, and the same applies to the output tensor of the second node.

[0181] By using M input intervals corresponding to the input tensor of the first node and K segmentation fractions of the output tensor of the second node, and by inversely determining K output intervals corresponding to the output tensor of the second node, and expressing them in interval form, this embodiment of the application can solve the problem of segmentation aggregation caused by different segmentation fractions in the neural network (regardless of parallel computing or core buffer optimization scenarios, segmentation aggregation represents the end of an optimization action, and in these scenarios, the longer the optimization action, the more obvious the performance gain). Therefore, this scheme can improve the effect of neural network segmentation to a certain extent.

[0182] As one possible implementation, based on M input intervals and a segmentation K, the K output intervals corresponding to the second output tensor of the second node are determined, including: If M is greater than K, the M input intervals are aggregated into K output intervals; or... If M equals K, then the M input intervals correspond one-to-one with the K output intervals; or... If M is less than K, the M input intervals are divided into K output intervals.

[0183] Specifically, when M is greater than K, for example, M=4 and K=2, the four input intervals of the first node need to be aggregated into two output intervals. This aggregation method includes: 1) aggregating the first and second input intervals into one output interval, and aggregating the third and fourth input intervals into one output interval; 2) aggregating the four input intervals into one interval, and then re-dividing this new interval into two output intervals; 3) aggregating the first and third input intervals into one interval, and aggregating the second and fourth input intervals into one interval. When M=K, the M input intervals correspond one-to-one with the K output intervals, i.e., the first input interval corresponds to the first output interval, the second input interval corresponds to the second output interval, the third input interval corresponds to the third output interval, and the fourth input interval corresponds to the fourth output interval. When M is less than K, for example, M=4 and K=8, the four input intervals of the first node need to be divided into eight output intervals. The partitioning method can include: 1) dividing the first input interval into two output intervals, the second input interval into two output intervals, the third input interval into two output intervals, and the fourth input interval into two output intervals; 2) aggregating the four input intervals into one interval and partitioning the new interval into eight output intervals; 3) dividing the first input interval into three output intervals, the second input interval into two output intervals, the third input interval into two output intervals, and leaving the fourth input interval unpartitioned as a corresponding output interval.

[0184] It should be understood that the partitioning and aggregation methods involved in the above schemes can be uniform partitioning or aggregation, or non-uniform partitioning or aggregation.

[0185] As one possible implementation, the M intervals corresponding to the first input tensor of the first node are determined, including: Determine the M output intervals corresponding to the first output tensor of the first node, where M is the segmentation number corresponding to the first output tensor of the first node; By reverse derivation of the M output intervals corresponding to the first output tensor of the first node, the M input intervals corresponding to the first input tensor of the first node are determined.

[0186] Specifically, after determining the first output tensor of the first node and the corresponding segmentation M, the interval corresponding to the first output tensor is divided into M intervals, and each interval corresponds one-to-one with a sub-tensor of the first output tensor. Then, the M input intervals corresponding to the first input tensor of the first node are determined through reverse derivation. For example, if the shape of the first output tensor of the first node is [2, 6, 16, 3], and the segmentation of the first output tensor of the first node is 2 (the segmentation axis is the third axis of the first output tensor), then the interval expression of the first output tensor of the first node is [0, 15]. Based on the segmentation 2, this interval is divided into two intervals, [0, 7] and [8, 15]. And based on reverse derivation, the two intervals of the first input tensor corresponding to the first input tensor of the first node are obtained, namely [0, 7] and [8, 15]. When there is an overlap between adjacent sub-tensors of the first input tensor of the first node, and the overlap is 1, the two intervals corresponding to the first input tensor of the first node are [0, 8] and [7, 15].

[0187] By determining the M input intervals corresponding to the input tensor of the first node through reverse derivation, the embodiments of this application can determine the multiple output intervals corresponding to the output tensor of the second node based on the M input intervals. This completes the data grouping dependency, avoids the forced aggregation of data, and can improve the performance benefits brought by neural network segmentation.

[0188] To better understand the aforementioned scheme, this application embodiment will further describe the scheme by taking the first node as the tail node of a neural network graph and the overlap between adjacent sub-tensors of the input tensors of each node as an example.

[0189] For example, a neural network graph includes four nodes, from top to bottom: the fourth node, the third node, the second node, and the first node. The first node is the tail node of the neural network graph, the fourth node is the head node, and the second and third nodes are the middle nodes. The first node has a split score of 2, the second node has a split score of 4, the third node has a split score of 4, and the fourth node has a split score of 2.

[0190] The output tensor of the first node has a shape of [2, 6, 16, 3] and a segmentation of 2. The output tensor of the first node needs to be divided into two sub-tensors: a first sub-tensor and a second sub-tensor. The shape of the first sub-tensor is [2, 6, 8, 3] (interval [0, 7]), and the shape of the second sub-tensor is [2, 6, 8, 3] (interval [8, 15]). The segmentation axis of the first node is the third segmentation axis of the tensor to be segmented. After determining that the output tensor of the first node should be divided into two output sub-tensors (or two output intervals) according to the segmentation ratio, the data processing device of this embodiment obtains the two input sub-tensors of the first node, i.e., the output tensor of the second node (or two input intervals), through reverse derivation.

[0191] Since there is overlap between adjacent subtensors obtained after the tensor to be segmented along the segmentation axis of the first node, we assume that the overlap value between adjacent subtensors is 1. The two input subtensors of the first node are [2, 6, 9, 3] (interval [0, 8]) and [2, 6, 9, 3] (interval [7, 15]). When deriving the output tensor of the second node, since the segmentation of the output tensor of the second node is 4 and the segmentation of the output tensor of the first node is 2, the two input subtensors of the first node need to be divided into two subtensors respectively: [2, 6, 9, 3] (interval [0, 8]). [2, 6, 5, 3] (interval [0, 4]) and [2, 6, 4, 3] (interval [5, 8]), [2, 6, 9, 3] (interval [7, 15]) The first node has four output subtensors with intervals of [2, 6, 5, 3] (interval [7, 11]) and [2, 6, 4, 3] (interval [12, 15]). However, due to overlap, the final shapes of the four output subtensors of the second node are [2, 6, 6, 3] (interval [0, 5]), [2, 6, 6, 3] (interval [4, 9]), [2, 6, 7, 3] (interval [6, 12]), and [2, 6, 5, 3] (interval [11, 15]). Since the splitting fraction of the second node is the same as that of the third node, each input subtensor of the second node can be assigned to each output subtensor of the output tensor of the third node without further splitting. Based on the foregoing, the four input subtensors of the third node are [2, 6, 7, 3] (interval [0, 6]), [2, 6, 8, 3] (interval [3, 10]), [2, 6, 9, 3] (interval [5, 13]), and [2, 6, 6, 3] (interval [10, 15]). Since the fourth node has a split of 2 and the third node has a split of 4, the four input subtensors of the third node need to be aggregated into two output subtensors: [2, 6, 7, 3] (interval [0, 6]) + [2, 6, 8, 3] (interval [3, 10]). [2, 6, 11, 3] (interval [0, 10]), [2, 6, 9, 3] (interval [5, 13]) + [2, 6, 6, 3] (interval [10, 15]) [2, 6, 11, 3] (interval [5, 15]). Then, the two input subtensors of the fourth node are [2, 6, 12, 3] (interval [0, 11]) and [2, 6, 12, 3] (interval [4, 15]).

[0192] As an example, if the first node is the tail node of the neural network graph, the data processing device of this application embodiment first obtains the segmentation information of each node in the neural network graph by traversal, determines the output tensor of the first node and the segmentation of the output tensor of the first node, and obtains the output tensor of the predecessor node of the first node according to the above scheme. If the first node is the predecessor node of the tail node of the neural network graph, the input tensor of the predecessor node of the tail node is obtained by reverse deduction through the aforementioned method, provided that the output tensor of the tail node of the neural network graph is determined. If the first node is an intermediate node of the neural network graph, then if the output tensor of the tail node of the neural network graph is determined and the output tensor of the predecessor node of the tail node is obtained by reverse deduction through the aforementioned method, this application embodiment further obtains the M input intervals corresponding to the input tensor of the first node by iterative reverse deduction.

[0193] Figure 10 This is another data processing method provided in the embodiments of this application, the details of which are as follows: Figure 10 As shown.

[0194] S1010, determine the M input intervals corresponding to the first input tensor of the first node and the Y input intervals corresponding to the third input tensor of the third node; S1020, obtain the segmentation K of the second node; S1030, based on M input intervals, Y input intervals and K, determine the K output intervals corresponding to the second output tensor of the second node.

[0195] Specifically, the second node is the predecessor node of the first node, and the second node is also the predecessor node of the third node. That is, the first node, the second node and the third node all belong to the same neural network graph.

[0196] It should be understood that the output tensor of the second node is the input tensor of the first node, and also the input tensor of the third node. For details, please refer to [link to relevant documentation]. Figure 11 The diagram shows the relationship between the second node, the third node, and the first node. It should be understood that... Figure 11 The solid arrows indicate the order of data processing, while the dashed arrows indicate the order of derivation.

[0197] It should be understood that when this embodiment derives the multiple output intervals corresponding to the second output tensor of the second node based on the multiple input intervals corresponding to the input tensor of the second node's subsequent nodes, the derivation is performed via two branches: the two output intervals corresponding to the second output tensor of the second node are derived from the four input intervals corresponding to the first input tensor of the first node, and the two output intervals corresponding to the second output tensor of the second node are derived from the six input intervals corresponding to the third input tensor of the third node. However, the branches of the first and second nodes converge at the second node. Therefore, in order to finally determine the two output intervals corresponding to the second output tensor of the second node, this embodiment needs to perform a union operation on the two corresponding output intervals from the four input intervals of the first node and the two output intervals corresponding to the second output tensor of the second node determined by the six input intervals of the third node, thereby determining the final two output intervals corresponding to the second output tensor of the second node.

[0198] For example, the first input tensor of the first node has a segmentation of 4, the third input tensor of the third node has a segmentation of 6, and the second output tensor of the second node has a segmentation of 2. Based on the four input intervals of the first node (e.g., [0, 6], [3, 10], [5, 13], and [10, 15]), the two output intervals corresponding to the second output tensor of the second node are [0, 10] and [5, 15]. Based on the six input intervals of the third node (e.g., [0, 3], [2, 6], [5, 9], [6, 10], [9, 13], and [12, 15]), the two output intervals corresponding to the second output tensor of the second node are [0, 9] and [6, 15]. Therefore, taking the union of intervals [0, 10] and [0, 9] yields the region. Given the interval [0, 10], the interval [5, 15] is obtained by taking the union of the intervals [5, 15] and [6, 15]. Alternatively, the two output intervals corresponding to the second output tensor of the second node determined by the four input intervals of the first node are [0, 9] and [5, 15], and the two output intervals corresponding to the second output tensor of the second node determined by the six input intervals of the third node are [0, 10] and [6, 15]. Then, the interval [0, 10] is obtained by taking the union of the intervals [0, 9] and [0, 10], and the interval [5, 15] is obtained by taking the union of the intervals [5, 15] and [6, 15].

[0199] It should be understood that the specific process of determining the K output intervals corresponding to the second output tensor of the second node based on the M input intervals corresponding to the first input tensor of the first node can be referred to the above description, and will not be repeated here.

[0200] Through the above technical solution, when the second node corresponds to multiple subsequent nodes, and different branches yield multiple output intervals corresponding to the output tensors of the second node, the union of the two intervals derived from different branches is taken to finally determine the multiple output intervals corresponding to the output tensors of the second node. In this way, the embodiments of this application can avoid the problem that multiple successor nodes cannot obtain complete data. Since the obtained data is the union of the intervals of each successor node, it can be guaranteed that each successor node can obtain the data block size it needs, avoiding abnormal neural network inference results or training failure due to data trampling.

[0201] It should be noted that the aforementioned solution can also be applied to scenarios such as data parallel computing, and the solution is consistent with the aforementioned solution.

[0202] The above text combined Figures 1 to 11 The data processing system and data processing method provided in the embodiments of this application are described below, in conjunction with... Figures 12 to 13 This describes the apparatus embodiments of the present application.

[0203] It should be understood that the description of the data processing method corresponds to the description of the device embodiment; therefore, any parts not described in detail can be found in the description above.

[0204] Figure 12 This is a schematic structural diagram of a data processing apparatus provided in an embodiment of this application. The data processing apparatus 1200 may be located in Figure 1 The execution device 110 or other device shown. The data processing apparatus 1200 includes an acquisition unit 1210 and a segmentation unit 1220.

[0205] The acquisition unit 1210 is used to acquire the number information of the first node in the neural network graph.

[0206] The segmentation unit 1220 is used to segment the tensor of the first node based on the numbering information.

[0207] Optionally, the data processing apparatus further includes a processing unit 1230 for performing... Figure 8 and Figure 10 The steps or methods involved in the scheme shown.

[0208] It should be understood that the above description is merely illustrative. The data processing apparatus is used to execute the methods or steps mentioned in the foregoing method embodiments. Therefore, the data processing apparatus corresponds to the foregoing method embodiments. For details, please refer to the description of the foregoing method embodiments, which will not be repeated here.

[0209] Figure 13 This is a schematic diagram of the hardware structure of the data processing device according to an embodiment of this application. Figure 13 The data processing device 1300 shown includes a memory 1301, a processor 1302, a communication interface 1303, and a bus 1304. The memory 1301, processor 1302, and communication interface 1303 are interconnected via the bus 1304.

[0210] The memory 1301 may be a ROM, a static storage device, or RAM. The memory 1301 may store a program, and when the program stored in the memory 1301 is executed by the processor 1302, the processor 1302 and the communication interface 1303 are used to execute the various steps of the data processing method of the embodiments of this application.

[0211] The processor 1302 may be a general-purpose CPU, microprocessor, ASIC, GPU, or one or more integrated circuits, used to execute relevant programs to achieve the functions required by the units in the data processing apparatus of this application embodiment, or to execute the data processing method of this application method embodiment.

[0212] The processor 1302 can also be an integrated circuit chip with signal processing capabilities; for example, it could be... Figure 4 The chip shown. In the implementation process, each step of the data processing method of this application embodiment can be completed by the integrated logic circuit of the hardware in the processor 4002 or by instructions in the form of software.

[0213] The processor 1302 described above can also be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 1301. The processor 1302 reads the information in memory 1301 and, in conjunction with its hardware, completes the functions required by the units included in the data processing apparatus of the embodiments of this application, or executes the data processing method of the method embodiments of this application.

[0214] The communication interface 1303 uses a transceiver device, such as, but not limited to, a transceiver, to enable communication between the device 1300 and other devices or communication networks. For example, an image to be processed can be acquired through the communication interface 1303.

[0215] Bus 1304 may include a pathway for transmitting information between various components of device 1300 (e.g., memory 1301, processor 1302, communication interface 1303).

[0216] It should be noted that although the above-described device 1300 only shows a memory, processor, and communication interface, those skilled in the art should understand that in specific implementations, device 1300 may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that device 1300 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that device 1300 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 12 All the devices shown.

[0217] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0218] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0219] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0220] This application provides a computer-readable storage medium for storing a computer program that, when run on a computer, causes the computer to perform a data processing method as described in the foregoing method embodiments.

[0221] This application provides a computer program product, which includes computer program code. When the computer program code is run, it implements the data processing method as described in the foregoing method embodiments.

[0222] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0223] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0224] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0225] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0226] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0227] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0228] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0229] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0230] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0231] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method is applied to a neural network processor, which includes a memory access controller, multiple processing units, and on-chip memory. The neural network processor is used to communicate with external memory. The method includes: Obtain the segmentation information of the first node in the neural network graph. The segmentation information includes tensor number and axis number. The tensor number indicates the tensor to be segmented, and the axis number indicates the axis to be segmented. The segmentation information is an attribute marked on the operator instance of the first node in the form of a string. Parse the string in the attribute, and along the splitting axis indicated by the axis number, split the tensor to be split indicated by the tensor number of the first node to obtain the data to be processed. The neural network graph is used to represent the neural network, and the first node is used to represent a computing unit of the neural network. The data to be processed is transferred from the external memory to the on-chip memory through the memory unit access controller; The data to be processed in the on-chip memory is processed in parallel by the multiple processing units to obtain the data processing result; The data processing results are stored in the external memory by accessing the controller through the storage unit. The tensor is used to represent at least one of the following types of data: graphics data, image data, voice data, text data, and Internet of Things data.

2. The method according to claim 1, characterized in that, The numbering information also includes an indication value, which is used to indicate whether there is overlap between adjacent sub-tensors obtained by splitting the tensor to be split along the splitting axis.

3. The method according to claim 2, characterized in that, When the tensor to be segmented is the input tensor of the first node, the indicator value is used to indicate that adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis overlap, or... When the tensor to be segmented is the input tensor of the first node, the indicator value is used to indicate that adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis do not overlap; or, When the tensor to be segmented is the output tensor of the first node, the indicator value is used to indicate that there is no overlap between adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis.

4. The method according to any one of claims 1 to 3, characterized in that, The second node is a predecessor node of the first node, and the second node belongs to the neural network graph. The method further includes: Determine M input intervals corresponding to the first input tensor of the first node, where M is the segmentation of the first input tensor; Obtain the segmentation K of the second output tensor of the second node; Based on the M input intervals and K, determine the K output intervals corresponding to the second output tensor of the second node, where M and K are both positive integers.

5. The method according to claim 4, characterized in that, The step of determining the K output intervals corresponding to the second output tensor of the second node based on the M input intervals and the K includes: If M is greater than K, the M input intervals are aggregated into the K output intervals; or... If M equals K, then the M input intervals correspond one-to-one with the K output intervals; or... If M is less than K, the M input intervals are divided into the K output intervals.

6. The method according to claim 4, characterized in that, Determining the M input intervals corresponding to the first input tensor of the first node includes: Determine M output intervals corresponding to the first output tensor of the first node, where M is the segmentation number corresponding to the first output tensor of the first node; By reverse derivation of the M output intervals corresponding to the first output tensor of the first node, the M input intervals corresponding to the first input tensor of the first node are determined.

7. The method according to any one of claims 1 to 3, characterized in that, The second node is the predecessor node of the first node and the third node, and the second node and the third node belong to the neural network graph. The method further includes: Determine M input intervals corresponding to the first input tensor of the first node and Y input intervals corresponding to the third input tensor of the third node, where M is the segmentation fraction of the first input tensor and Y is the segmentation fraction of the third input tensor; Obtain the segmentation K of the second output tensor of the second node; Based on the M input intervals, the Y input intervals, and K, determine the K output intervals corresponding to the output tensor of the second node, where M, K, and Y are all positive integers.

8. A data processing apparatus, characterized in that, The device is a neural network processor, which includes a memory access controller, multiple processing units, and on-chip memory. The neural network processor is used to communicate with external memory. The device includes: The acquisition unit is used to acquire the segmentation information of the first node in the neural network graph. The segmentation information includes tensor number and axis number. The tensor number indicates the tensor to be segmented, and the axis number indicates the axis of the tensor to be segmented. The segmentation information is an attribute marked on the operator instance of the first node in the form of a string. A segmentation unit is used to parse the string in the attribute and segment the tensor to be segmented indicated by the tensor number of the first node along the segmentation axis indicated by the axis number to obtain the data to be processed. The neural network graph is used to represent the neural network, and the first node is used to represent a computing unit of the neural network. The data to be processed is transferred from the external memory to the on-chip memory through the memory unit access controller; The data to be processed in the on-chip memory is processed in parallel by the multiple processing units to obtain the data processing result; The data processing results are stored in the external memory by accessing the controller through the storage unit. The tensor is used to represent at least one of the following types of data: graphics data, image data, voice data, text data, and Internet of Things data.

9. The apparatus according to claim 8, characterized in that, The numbering information also includes an indication value, which is used to indicate whether there is overlap between adjacent sub-tensors obtained by splitting the tensor to be split along the splitting axis.

10. The apparatus according to claim 9, characterized in that, When the tensor to be segmented is the input tensor of the first node, the indicator value is used to indicate that adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis overlap, or... When the tensor to be segmented is the input tensor of the first node, the indicator value is used to indicate that adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis do not overlap; or, When the tensor to be segmented is the output tensor of the first node, the indicator value is used to indicate that there is no overlap between adjacent sub-tensors obtained by segmenting the tensor to be segmented along the segmentation axis.

11. The apparatus according to any one of claims 8 to 10, characterized in that, The second node is a predecessor node of the first node, and the second node belongs to the neural network graph. The device also includes a processing unit. The processing unit is configured to determine M input intervals corresponding to the first input tensor of the first node, where M is the number of segments of the input tensor. The acquisition unit is further configured to acquire the segmentation K of the second output tensor of the second node; The processing unit is further configured to determine K output intervals corresponding to the second output tensor of the second node based on the M input intervals and K, wherein M and K are both positive integers.

12. The apparatus according to claim 11, characterized in that, The processing unit is used for: If M is greater than K, then the M input intervals are aggregated into the K output intervals; or... If M equals K, then the M input intervals correspond one-to-one with the K output intervals; or... If M is less than K, the M input intervals are divided into the K output intervals.

13. The apparatus according to claim 11, characterized in that, The processing unit is further configured to: Determine M output intervals corresponding to the first output tensor of the first node, where M is the segmentation number corresponding to the first output tensor of the first node; By reverse derivation of the M output intervals corresponding to the first output tensor of the first node, the M input intervals corresponding to the first input tensor of the first node are determined.

14. The apparatus according to any one of claims 8 to 10, characterized in that, The second node is a predecessor node of the first and third nodes, and the second node and the third node belong to the neural network graph. The device also includes a processing unit. The processing unit is configured to determine M input intervals corresponding to the first input tensor of the first node and Y input intervals corresponding to the second input tensor of the third node, wherein M is the segmentation fraction of the first input tensor and Y is the segmentation fraction of the second input tensor; The acquisition unit is further configured to acquire the segmentation K of the second output tensor of the second node; The processing unit is further configured to determine K output intervals corresponding to the output tensor of the second node based on the M input intervals, the Y input intervals and K, wherein M, Y and K are all positive integers.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a computer or processor, causes the computer or processor to perform the method as described in any one of claims 1 to 7.

16. A computer program product, characterized in that, The computer program product includes: computer program code, which, when executed, implements the method as described in any one of claims 1 to 7.

Images