Data processing method, training method, image processing method, and related products

Abstract

The embodiment of the application discloses a data processing method and related products, the method comprises the following steps: a first device calculates a first vector group according to a first matrix and first input data; the first matrix is composed of part of all parameters of a first network layer; the first device sums a first sub-vector group and R second sub-vector groups of the first vector group to obtain a target vector group with n rows; any second sub-vector group of the R second sub-vector groups is a vector group comprising consecutive n rows of vectors in a second vector group; the target vector group with n rows is stored in the first device; and the target vector group with n rows is all input vectors when the first device calculates output data of a second network layer. In the embodiment of the application, the eigenvalues of the network layer are distributedly stored, so that the device provides more memory processing sample data, thereby improving the training efficiency of the model.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a data processing method, training method, image processing method, and related products. Background Technology

[0002] Deep neural network models have achieved significant research results in speech processing, visual processing, and image processing in recent years. When training large-scale models, due to considerations of device memory, a model parallel training method is generally adopted. That is, each device trains a part of the neural network model, and the devices transmit data through specific communication methods. For example, after summing the data on all devices through multiple iterations, the summed data is evenly distributed to each device.

[0003] Current model partitioning methods can only train a small number of samples per iteration, resulting in a significant loss of computing power and low training efficiency. Summary of the Invention

[0004] This application discloses a data processing method, a training method, an image processing method, and related products. The feature values ​​extracted from the training samples after neural network processing are distributed and stored, which reduces the memory usage of the feature value vector group in each parallel processing device, allowing the device to provide more memory to process sample data, thereby improving the training efficiency of the model.

[0005] In a first aspect, this application provides a data processing method, comprising: a first device calculating a first vector group based on a first matrix and first input data; the first matrix being an N-row matrix composed of a portion of all parameters of a first network layer, the first network layer being any layer in a neural network model, N being an integer greater than 0, and the first vector group being an N-row vector group; the first device summing a first sub-vector group and R second sub-vector groups of the first vector group to obtain an n-row target vector group; the first sub-vector group being a vector group comprising n consecutive rows of the first vector group; the R second sub-vector groups being... Any second sub-vector group in the two sub-vector groups is a vector group consisting of n consecutive rows from the aforementioned second vector group. The aforementioned second vector group is an N-row vector group calculated by the second device based on the N-row second matrix and the second input data. The aforementioned second matrix is ​​an N-row matrix composed of a portion of all parameters of the first network layer, where R is an integer greater than 0. The n-row target vector group is stored in the aforementioned first device. The n-row target vector group is all the input vectors when the first device calculates the output data of the second network layer. The aforementioned second network layer and the first network layer are two consecutive layers in the aforementioned neural network model.

[0006] In fact, the parameters of the network layers in the above neural network model initially exist as complete parameter matrices. Since this processing method uses multiple devices to process the network layers in parallel, these parameter matrices are divided into multiple sub-matrices by column and distributed among the devices for parallel processing. Here, we assume that the parameters of the first network layer before division are an N x M matrix, where M is an integer greater than 1. Then, the first matrix is ​​a matrix containing m consecutive columns of this N x M matrix, where m is an integer greater than 0 and less than M. The second matrix is ​​a matrix containing consecutive columns of this N x M matrix, and all columns in the second matrix are in a different position than all columns in the first matrix within the parameter matrix. In other words, the parameter matrix is ​​divided by column. It should be understood that the N x M matrix mentioned here is the undivided matrix. During the data processing process of the above method, it has been divided into multiple sub-matrices and distributed to multiple devices for storage and processing. Therefore, the N x M matrix does not exist during this processing; this concept is introduced here for the reader's convenience. Similarly, the “parameter matrix” mentioned in the following explanation refers to the parameter matrix before the network layer is divided. During the processing, it has been divided into multiple sub-matrices and stored in multiple devices, and no longer exists in its complete form.

[0007] The aforementioned R second sub-vector groups come from R different vector groups, which in turn come from R different devices. The aforementioned first device and the R devices each calculate a vector group. It should be understood that the vector groups calculated by the first device and the R devices are each divided into (R+1) sub-vector groups in an ordered row-by-row manner. In the first device and the aforementioned (R+1) devices, each device retains only one sub-vector group from its own calculated vector group, and then obtains R sub-vector groups corresponding to the positions of the original sub-vector groups from the vector groups calculated by the other R devices. For example, when the value of R is 1 and the values ​​of N and M are both 4, that is, when the first network layer of the neural network model is trained in parallel by the first device and another device (hereinafter referred to as device 1 and device 2 respectively), the parameter matrix of the 4 parallel lines will be divided into sub-matrix 1 and sub-matrix 2 by columns. Device 1 calculates a matrix calculated by 4 (hereinafter referred to as matrix 1) based on its stored input data 1 of the first network layer (i.e., the feature vector group of 2 rows) and sub-matrix 1; Device 2 calculates a matrix calculated by 4 (hereinafter referred to as matrix 2) based on its stored input data 2 of the first network layer (i.e., another feature vector group of 2 rows) and sub-matrix 2; Device 1 obtains the first two rows of matrix 2 and sums them with the first two rows of matrix 1 to obtain 1 / 2 of the complete feature vector group of the first network layer, and stores the resulting vector group in Device 1; Device 2 obtains the last two rows of matrix 1 and sums them with the last two rows of matrix 2 to obtain the other 1 / 2 of the complete feature vector group of the first network layer, and stores the resulting vector group in Device 2.

[0008] In this application example, the parameter matrices of each network layer of the neural network model are vertically divided into multiple devices. By utilizing the matrix operation rules, the communication method between devices is changed, and the feature value vector groups of the network layers are distributed for storage. Each device only stores 1 / (R+1) of the total feature vector groups of each network layer, which reduces the memory occupation of the feature value vector groups in each device, thereby increasing the amount of data processed in a single processing and improving the efficiency of data processing.

[0009] In an optional implementation, the method further includes: the first device calculating a loss value for the final output data of the neural network based on the target vector group of the n rows and the ideal output value of the neural network model; and the first device updating the first matrix based on the loss value.

[0010] In an optional implementation, after calculating the loss value based on the n rows of target vectors and the ideal output value of the neural network model, the method further includes: releasing the memory occupied by the n rows of target vectors in the first device.

[0011] In this implementation, any error function can be used to calculate the aforementioned loss value. It should be understood that when the first network layer is not the last network layer responsible for processing the data in the neural network model, the target vector group with the aforementioned n rows still needs to be processed by subsequent network layers. That is, it is used as input data for the second network layer to calculate the output data of the second network layer. Then, the output data of the second network layer is used as input data for the third network layer to calculate the output data of the third network layer; and so on, until the output data of the last network layer responsible for processing the data in the neural network model is obtained. Then, combined with the output data of the last network layer responsible for processing the data calculated by the other R devices, the final complete output data of the neural network model is obtained. After calculating the aforementioned loss value from the final complete output data and the aforementioned ideal output value, the first device will release the memory occupied by the aforementioned n rows of target vector groups in the first device. The first device then calculates the influence of each parameter value in the aforementioned first matrix on the loss value using the chain rule, and then updates the parameter according to the influence of each parameter on the loss value. The chain rule is used to calculate the partial derivative of the loss value with respect to each parameter. This partial derivative represents the degree of influence of the parameter on the loss value.

[0012] In this application example, the training process of the aforementioned neural network model also includes a backpropagation process; by calculating the loss value between the data obtained from the forward computation and the preset ideal output value, the parameters of the network layer are updated so that the final output value is as close as possible to the preset ideal output value. It should be understood that throughout the entire training process of the aforementioned neural network model, the aforementioned device only needs to store 1 / (R+1) of all feature vector groups of each network layer.

[0013] In an optional implementation, the method further includes: the first device being a device in a first parallel training group, which is a training group in a cluster system for training the neural network model; the first parallel training group also includes R devices; the cluster system also includes at least one second parallel training group, which includes multiple devices; the devices in the first parallel training group and the devices in the second parallel training group are different; both the first and second parallel training groups train the complete neural network model; and the multiple devices in the second parallel training group process the parameter matrix of the network layers of the neural network model in parallel. This parameter matrix is ​​also divided into multiple sub-matrices by columns, and these sub-matrices are stored in the multiple devices respectively. The way the multiple devices process data and the way they communicate with each other are similar to those of the R devices in the first parallel training group. That is, both the first and second parallel training groups process only a portion of the total sample data of the neural network model.

[0014] In this application example, a hybrid parallel approach is used to train the aforementioned neural network model. Specifically, devices within a single parallel training group train the model in parallel, while multiple parallel training groups process sample data in parallel. It should be understood that the aforementioned cluster system includes multiple parallel training groups, each containing the same number of devices. Within each of these parallel training groups, the entire aforementioned neural network model is trained. Each device within a single parallel training group trains a portion of the aforementioned neural network model, and the multiple parallel training groups process a portion of all the input data for the aforementioned neural network model. This solves the problem of high communication costs caused by solely using model parallelism and improves the efficiency of cluster training.

[0015] Secondly, this application provides a method for training a neural network model, the method comprising: processing input sample data to obtain a target vector group; the processing comprising: a first device calculating a first vector group based on a first matrix and first input data; the first matrix being an N-row matrix consisting of a portion of all parameters of a first network layer, the first network layer being any layer in the neural network model, N being an integer greater than 0, and the first vector group being an N-row vector group; summing a first sub-vector group and R second sub-vector groups of the first vector group to obtain the target vector group; the first sub-vector group being a vector group comprising n consecutive rows of the first vector group; and any second sub-vector group of the R second sub-vector groups... The target vector group is a set of n consecutive rows from the second vector group, which is an N-row vector group calculated by the second device based on the second matrix and the second input data. The second matrix is ​​another N-row matrix composed of a portion of all parameters of the first network layer, where R is an integer greater than 0. The target vector group is stored in the first device. The target vector group is all the input vectors when the first device calculates the output data of the second network layer. The second network layer and the first network layer are two consecutive layers in the neural network model. The first device calculates the loss value based on the n-row target vector group and the ideal output value of the neural network model. The first device updates the first matrix based on the loss value.

[0016] Before partitioning, all parameters of the first network layer also exist in the form of a complete parameter matrix. Here, it is also assumed to be an N-row, M-column parameter matrix, where M is an integer greater than 1. Then, the first matrix is ​​a matrix containing m consecutive columns of the N-row, M-column parameter matrix, where m is an integer greater than 0 and less than M. The second matrix is ​​a matrix containing consecutive columns of the N-row, M-column parameter matrix, and all columns in the second matrix are in a different position than all columns in the first matrix in the parameter matrix.

[0017] In this embodiment, during the training of the neural network model, the parameter matrix of each network layer of the neural network model is vertically divided into multiple devices. By utilizing the matrix operation rules, the communication method of data between devices is changed, reducing the memory occupation of the feature value vector group in each device. This increases the number of samples for one iteration of training and improves the training efficiency of the model. In addition, throughout the entire process of training the above neural network model until the backpropagation ends, each device only needs to save 1 / (R+1) of all output feature vector groups of each network layer.

[0018] Thirdly, this application provides an image processing method, which includes: processing input image data using a trained neural network model to obtain a processing result; the processing includes: a first device calculating a first vector group based on a first matrix and first image data; the first matrix is ​​an N-row matrix composed of a portion of all parameters of a first network layer, the first network layer being any layer in the neural network model, N being an integer greater than 0, and the first vector group being an N-row vector group; summing a first sub-vector group and R second sub-vector groups of the first vector group to obtain a target vector group; the first sub-vector group being a vector group comprising n consecutive rows of the first vector group; the R second sub-vector groups being... Any second sub-vector group is a vector group consisting of n consecutive rows from the second vector group. This second vector group is an N-row vector group calculated by the second device based on the N-row second matrix and the second image data. The second matrix is ​​another N-row matrix composed of a portion of all parameters of the first network layer, where R is an integer greater than 0. The target vector group is stored in the first device. The target vector group is all the input vectors when the first device calculates the output data of the second network layer. The second network layer and the first network layer are two consecutive layers in the neural network model. The memory occupied by the first image data in the first device is released. The processing result is output.

[0019] It should be understood that all parameters of the first network layer mentioned above also exist in the form of a complete parameter matrix before partitioning. Here, it is also assumed to be an N-row, M-column parameter matrix, where M is an integer greater than 1. Then, the first matrix mentioned above is a matrix containing m consecutive columns of the N-row, M-column parameter matrix, where m is an integer greater than 0 and m is less than M. The second matrix mentioned above is a matrix containing consecutive columns of the N-row, M-column parameter matrix, and all columns in the second matrix are in different positions from all columns in the first matrix in the parameter matrix.

[0020] It should be understood that the above neural network model is a trained neural network model. When applying the neural network model to process the above image data, after each device uses the output vector group of the previous network layer as the input vector group of this network layer and calculates the target vector group of the network layer, since it no longer needs to backpropagate to update the parameter values, the device will release the memory occupied by the above output vector group in the device.

[0021] In this application example, the parameter matrix of each network layer of the neural network model is vertically divided and distributed to multiple devices. By utilizing the matrix operation rules, the communication method between devices is changed. Each device only stores 1 / (R+1) of the total feature vector group of each network layer, which reduces the memory occupation of the feature vector group in each device, thereby increasing the amount of image data processed in a single processing and improving the efficiency of data processing.

[0022] Fourthly, embodiments of this application provide a data processing apparatus, comprising: a calculation unit, configured to calculate a first vector group based on a first matrix and first input data; the first matrix is ​​a matrix including m consecutive columns of a parameter matrix of a first network layer, the first network layer being any layer in a neural network model, the parameter matrix being an N-row, M-column matrix, where N and M are integers greater than 0, m is an integer greater than 0, and m is less than M, and the first vector group is an N-row vector group; and an aggregation unit, configured to acquire R second sub-vector groups from other devices, and sum the first sub-vector group of the first vector group and the R second sub-vector groups to obtain an n-row target vector group; the first sub-vector group is a matrix including the first... A vector group consisting of n consecutive rows in a vector group; any one of the R second sub-vector groups is a vector group consisting of n consecutive rows in the second vector group, which is an N-row vector group calculated by the second device based on the second matrix with N rows and the second input data; the second matrix is ​​a matrix consisting of consecutive columns in the parameter matrix, and all columns in the second matrix are in different positions from all columns in the first matrix in the parameter matrix; R is an integer greater than 0; a storage unit is used to store the n-row target vector group in the first device; the n-row target vector group is all the input vectors when the first device calculates the output data of the second network layer, and the second network layer and the first network layer are two consecutive layers in the neural network model.

[0023] In one optional implementation, the aggregation unit is specifically used to obtain a sub-vector group from each of the other R devices. This sub-vector group is a sub-vector group of a specified row order from the feature vector group calculated by a single device based on a portion of the input data of the first network layer and a submatrix of the parameter matrix; it is then summed with the sub-vector group of the corresponding row order of its own calculated feature vector group. The row order is determined by the position of the sub-vector group of its own calculated feature vector group retained by the device within the feature vector group. For example, if a device retains the first two rows of its own calculated feature vector group, it also obtains the first two rows of the feature vector groups calculated by those R devices from the other R devices and performs the summation operation.

[0024] In one optional implementation, both the first and second matrices are m-column matrices. That is, the parameter matrix is ​​divided into (R+1) m-column matrices, which are then processed in parallel by the first device and the R devices, including the second device. In this implementation, both the first and second input data are consecutive n-row sub-vector groups of all input data for the first network layer. That is, all input data for the first network layer is also divided into (R+1) sub-vector groups, which are then distributed row-wise to the first device and the R devices, including the second device, for parallel processing.

[0025] In an optional implementation, the apparatus further includes: a reverse unit, configured to calculate a loss value based on the n rows of target vectors and the ideal output value of the neural network model; and update the first matrix based on the loss value; wherein the loss value is calculated by the first device based on the n rows of target vectors and the ideal output value of the neural network model.

[0026] In an optional implementation, the aforementioned storage unit is further configured to release the memory occupied by the target vector group in the first device after the aforementioned reverse unit has calculated the loss value.

[0027] Fifthly, embodiments of this application provide a training apparatus for a neural network model. The model training apparatus includes: a processing unit for processing input sample data to obtain a target vector group; the processing includes: calculating a first vector group based on a first matrix and first input data; wherein the first matrix is ​​a matrix containing m consecutive columns from a parameter matrix of a first network layer, the first network layer is any layer in the neural network model, the parameter matrix is ​​an N-row, M-column matrix, N and M are integers greater than 1, m is an integer greater than 0, and m is less than M, and the first vector group is a vector group with N rows; summing a first sub-vector group and R second sub-vector groups from the first vector group to obtain the target vector group; wherein the first sub-vector group is a vector group containing n consecutive rows from the first vector group; and any one of the R second sub-vector groups is a vector group containing n consecutive rows from the second vector group. The first device calculates an N-row vector group based on a second matrix and the second input data. The second matrix is ​​a matrix containing consecutive columns from the parameter matrix, where all columns in the second matrix are in a different position than those in the first matrix. R is an integer greater than 0. A storage unit stores the target vector group, which is the entire set of input vectors used by the first device to calculate the output data of the second network layer. The second and first network layers are two consecutive layers in the neural network model. A reverse unit calculates a loss value based on the target vector group and the ideal output value of the neural network model. The loss value is calculated by the first device based on the target vector group and the ideal output value of the neural network model. The first matrix is ​​then updated based on the loss value.

[0028] In one optional implementation, both the first and second matrices are m-column matrices. That is, the parameter matrix is ​​divided into (R+1) m-column matrices, which are then processed in parallel by the first device and the R devices, including the second device. In this implementation, both the first and second input data are consecutive n-row sub-vector groups of all input data for the first network layer. That is, all input data for the first network layer is also divided into (R+1) sub-vector groups, which are then distributed row-wise to the first device and the R devices, including the second device, for parallel processing.

[0029] Sixthly, embodiments of this application provide an image processing apparatus, the apparatus comprising: a processing unit, configured to process input image data using a trained neural network model to obtain a processing result; the processing comprising: a first device calculating a first vector group based on a first matrix and first image data; wherein the first matrix is ​​a matrix comprising m consecutive columns in a parameter matrix of a first network layer, the first network layer being any layer in the neural network model, the parameter matrix being an N-row, M-column matrix, N and M being integers greater than 0, m being an integer greater than 0, and m being less than M, and the first vector group being a vector group with N rows; summing a first sub-vector group and R second sub-vector groups of the first vector group to obtain the target vector group; wherein the first sub-vector group is a vector group comprising n consecutive rows in the first vector group; Any one of the R second sub-vector groups is a vector group consisting of n consecutive rows from the second vector group. The second vector group is an N-row vector group calculated by the second device based on the N-row second matrix and the second image data. The second matrix is ​​a matrix consisting of consecutive columns from the parameter matrix, and all columns in the second matrix are in different positions from all columns in the first matrix in the parameter matrix. R is an integer greater than 0. A storage unit is used to store the target vector group in the first device and release the memory occupied by the first image data in the first device. The target vector group is all the input vectors when the first device calculates the output data of the second network layer. The second network layer and the first network layer are two consecutive layers in the neural network model. An output unit is used to output the processing result.

[0030] In an optional implementation, the processing unit is also used to perform correction processing on the image data. In real-world scenarios, images may encounter problems such as incorrect angles affecting recognition, or slight occlusions (such as noise) that make them difficult to identify. Images obtained after correction are easier to classify accurately.

[0031] In one optional implementation, the output unit is specifically used to: transform the processing result and then output it; the output object can be a terminal device such as a mobile phone, tablet, laptop, or desktop computer, or a server. It should be understood that the processing result is actually a feature vector set, and the actual output result should be the classification result of the image data obtained after analyzing and transforming the feature vector set, such as the image being a person, a dog, or a flower; or the image having an 80% probability of being a dog, an 18% probability of being a cat, and a 2% probability of being a hat.

[0032] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the methods described in the first aspect, the second aspect, the third aspect, and any optional implementation thereof.

[0033] Eighthly, embodiments of this application provide an electronic device, the electronic device comprising: a memory for storing a program; and a processor for executing the program stored in the memory, wherein when the program is executed, the processor is configured to execute a method as described in the first aspect, the second aspect, the third aspect, and any optional implementation thereof. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments or background art of this application, the accompanying drawings used in the embodiments or background art of this application will be briefly introduced below.

[0035] Figure 1a A schematic diagram illustrating a data communication method provided in an embodiment of this application;

[0036] Figure 1b A schematic diagram illustrating another data communication method provided in an embodiment of this application;

[0037] Figure 1c A schematic diagram illustrating yet another data communication method provided in an embodiment of this application;

[0038] Figure 2 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0039] Figure 3 A schematic diagram illustrating a data processing method provided in an embodiment of this application;

[0040] Figure 4 An architecture diagram of a neural network model provided in an embodiment of this application;

[0041] Figure 5 A schematic diagram illustrating another data processing method provided in an embodiment of this application;

[0042] Figure 6 A schematic diagram illustrating a model segmentation method provided in an embodiment of this application;

[0043] Figure 7 A flowchart of a model parallel training method provided in this application embodiment;

[0044] Figure 8 This application provides a schematic diagram of the architecture of a model parallel training group.

[0045] Figure 9 This is a schematic diagram of the architecture of a model training cluster system provided in an embodiment of this application;

[0046] Figure 10 An architecture diagram of another model training cluster system provided in this application embodiment;

[0047] Figure 11 This application provides an architecture diagram of another model training cluster system.

[0048] Figure 12 A flowchart illustrating an image processing method provided in an embodiment of this application;

[0049] Figure 13 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0050] Figure 14 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;

[0051] Figure 15 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0052] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below in conjunction with the accompanying drawings.

[0054] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used only to distinguish different objects and not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0055] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0056] In this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0057] Neural networks (NNs) are complex network systems formed by the extensive interconnection of a large number of simple processing units (called neurons). They reflect many fundamental characteristics of human brain function and are highly complex nonlinear dynamic learning systems. Neural networks possess massive parallelism, distributed storage and processing, self-organization, self-adaptation, and self-learning capabilities, making them particularly suitable for handling imprecise and fuzzy information processing problems that require simultaneous consideration of many factors and conditions.

[0058] Data parallelism: Different machines have multiple copies of the same model, each machine is assigned different data, and then the computation results of all machines are merged in some way.

[0059] Model parallelism: Different machines in a distributed system are responsible for different parts of the network model—for example, different network layers of a neural network model are assigned to different machines, or different parameters within the same layer are assigned to different machines.

[0060] Hybrid parallelism: This means that both model parallelism and data parallelism are used. For example, model parallelism can be used on the same machine, while data parallelism can be used between machines.

[0061] Backpropagation: Backpropagation is a step in the training process of a neural network model. It iteratively updates the parameters in the neural network model so that the final output result reaches the ideal value.

[0062] Weighted sum: The sum of each data point multiplied by its weight is called the weighted sum.

[0063] The embodiments of this application are described below with reference to the accompanying drawings.

[0064] exist Figures 1a to 1b In the diagram, Device 1 to Device 4 represent different devices (processes), A1 to A4 represent data on Device 1, B1 to B4 represent data on Device 2, and so on.

[0065] Please see Figure 1a , Figure 1a This is a schematic diagram illustrating the effect of a data communication method provided in an embodiment of this application. Figure 1a As shown, each device stores its own existing data, and then retrieves and stores data that it does not have from all other devices through aggregated communication. From the above... Figure 1a As can be seen from the effect of the data communication method in the above, after the above-mentioned collection communication process is completed, each device saves complete data.

[0066] Please see Figure 1b , Figure 1b This is a schematic diagram illustrating the effect of another data communication method provided in an embodiment of this application. For example... Figure 1b As shown, this data communication method sums the data in all devices, and each device stores a copy of the summed data.

[0067] When training a large-scale neural network model using only a single device, the device may not be able to store the complete parameters of the neural network model, or it may not provide enough memory to complete the calculations on the sample data after storing the complete parameters. In this case, a parallel training approach can be used to train the model, that is, to use multiple devices to train the aforementioned large-scale neural network model in parallel, distributing the parameters of the network layers of the neural network model among the multiple devices for processing. Due to the interrelationship between network layers in the neural network model (for example, the output feature values ​​calculated by the previous network layer can be the input data required for the calculation of the next network layer), after the multiple devices calculate the feature value vector group of a certain network layer based on the partial parameters of a certain network layer stored by themselves and the input data, they need to transmit the data through a specific data communication method, so as to train each network layer in an orderly manner and obtain the final neural network model.

[0068] In existing parallel training methods for models, the method of partitioning the parameter matrix of the network layers and the rules of matrix operations determine the number of devices (such as...) participating in parallel training during the training process. Figure 1a Devices 1-4 or Figure 1b Devices 1-4 in the diagram can only use the following methods: Figure 1a or Figure 1b Data is transmitted using data communication methods. However, by Figure 1a It can be seen that after data transmission between devices 1 and 4, each device needs to obtain and save data that it does not have from the other devices. In other words, after processing a certain network layer, devices 1 to 4 all need to save the complete set of feature vectors for that network layer. Furthermore, from... Figure 1bIt can be seen that after data transmission between devices 1 and 4, each device saves a copy of the sum of all data from all devices. In other words, after processing a certain network layer, the feature vector sets obtained from devices 1 to 4 are summed, and each device needs to save the resulting summed vector set. In summary, existing model parallel algorithms can only reduce the memory footprint of parameters. However, during training, the memory footprint of the feature vector sets for each network layer is not well optimized, resulting in only a small number of samples being trained per iteration. For training cluster systems with high computing power, this leads to a significant loss of computing power. Furthermore, the model's computational communication is relatively small, requiring parameter gradient update communication after training only a few samples, increasing the cluster's additional communication overhead and thus reducing cluster linearity and training performance.

[0069] To address the shortcomings of the data processing methods used in the aforementioned model training process, this application provides a novel data processing method. Compared to conventional methods, this method alters the parameter partitioning and utilizes matrix operation rules to change the data communication method between devices. It distributes and stores the network layer feature vector groups obtained during model training, thus solving the problems of low training efficiency and high communication costs. Before introducing the data processing method flow provided in this application, the data communication method between multiple devices in this method will be explained.

[0070] Please see Figure 1c , Figure 1c This is a schematic diagram illustrating the effect of another data communication method provided in an embodiment of this application. Figure 1c In this diagram, devices 1 through 4 represent different devices (or processes), A1 through A4 represent data on device 1, B1 through B4 represent data on device 2, and so on. The first device mentioned above is equivalent to... Figure 1c Any one of devices 1-4, taking device 1 as the first device mentioned above, then devices 2-4 are the other devices mentioned above, including the second device. For example... Figure 1c As shown, the first device (i.e., device 1) retains data A1 from the first row of its entire data, and obtains B1, C1, and D1 from the first row of the complete data of other devices, performs a summation operation to obtain the first target data (A1+B1+C1+D1), and saves the first target data in the first device; the second device (i.e., device 2) retains data A2 from the second row of its entire data, and obtains B2, C2, and D2 from the second row of the complete data of other devices, performs a summation operation to obtain the second target data (A2+B2+C2+D2), and saves it in device 2, and so on. In other words, the data of all devices are summed and then evenly distributed, with each device only storing 1 / N of the complete data (N is the number of devices participating in the collective communication).

[0071] In this embodiment of the application, multiple devices are still used to train the neural network model in parallel. The data stored in each device is different, but the data processing method is generally the same. The flow of the data processing method is described below from the perspective of a single device.

[0072] Please see Figure 2 , Figure 2 A flowchart illustrating a data processing method provided in this application embodiment is shown, the method comprising:

[0073] 201. The first device calculates the first vector group based on the first matrix and the first input data.

[0074] The first device can be a graphics processing unit (GPU), a central processing unit (CPU), an acceleration device, or a server. The first matrix is ​​a submatrix of the multi-row, multi-column parameter matrix of the first network layer. This submatrix is ​​a matrix formed by selecting consecutive columns from the parameter matrix. The first network layer can be any layer in a neural network model. The input data can be a set of feature value vectors output from the preceding network layer and stored in the first device, or it can be a portion of all vector sets obtained by transforming the sample data initially input into the network model. The first vector set is a vector set with n rows; that is, the first vector set can be considered as a vector set composed of n row vectors.

[0075] For example, the first device calculates the output feature vector of the first network layer based on the feature vector group stored in the first device and the first matrix formed by selecting m consecutive columns from the M-column parameter matrix of the first network layer; where M and m are integers greater than 0, and M is divisible by m. That is, in the embodiment, the parameter matrix of the first network layer is divided into M / m sub-matrices in the column direction, and correspondingly, M / m devices, including the first device, will jointly complete the data processing process.

[0076] 202. The first device sums the first sub-vector group and R second sub-vector groups of the first vector group to obtain a target vector group with n rows.

[0077] The first sub-vector group is a vector group consisting of n consecutive rows from the first vector group. The R second sub-vector groups come from R other devices different from the first device. These other devices can be GPUs, CPUs, acceleration devices, or servers, and the first device and the R other devices can be accelerators controlled by the same server. Each of the R other devices calculates an eigenvalue vector group based on a portion of the eigenvalue vector group obtained by converting the output data of the previous network layer or the sample data initially input into the network model, and a submatrix of the parameter matrix of the first network layer. Then, the first device first obtains a sub-vector group from the eigenvalue vector groups calculated by the other devices. In total, the first device obtains R sub-vector groups from the R devices, and these R sub-vector groups are the R second sub-vector groups. It should be understood that in step 201, the first matrix used by the first device for calculation and the sub-matrices used by other devices for calculation are different. That is, the columns contained in all sub-matrices are in different positions in the parameter matrix. In addition, the input data used by the first device for calculation and the input data used by other devices for calculation are also different. The input data used by each device to calculate the output data of a certain network layer is stored in the device itself. It should be understood that "input" and "output" here refer to the input and output of the network layer, not that the device obtains input data from other devices or outputs data to external devices.

[0078] It should be understood that the data communication method used between the multiple devices in step 202 is as follows: Figure 1c As shown in the diagram. That is, while the first device obtains sub-vector groups from the other R devices, each of the other R devices also obtains a sub-vector group from the first device and (R-1) other devices (excluding itself). These sub-vector groups are then summed with the sub-vector groups of their own calculated feature vector groups to obtain the target vector group to be stored within themselves. The row positions of the sub-vector groups retained by each device should be the same as the row positions of the sub-vector groups they obtained in their source vector groups. For example, if a device retains rows 1-2 of its own calculated feature vector group for summation, then the sub-vector groups it obtains from the other R devices will also be sub-vector groups consisting of rows 1-2 of the feature vector groups calculated by the other R devices. Similarly, if a device retains rows 4-6 of its own calculated feature vector group for summation, then the sub-vector groups it obtains from other devices will also be sub-vector groups consisting of rows 4-6 of the feature vector groups of that network layer calculated by the other devices.

[0079] Step 202 utilizes matrix operation rules and specific data communication methods to distribute the calculated data for storage. Please refer to [link to details]. Figure 3 .

[0080] Figure 3 A schematic diagram of another data communication method provided in this application embodiment. For example... Figure 3 As shown, device 1 stores a portion of the output data 301 of network layer 1 in the neural network model, and device 2 stores another portion of the output data 302 of network layer 1 in the neural network model. After devices 1 and 2 calculate their respective eigenvalue vector groups from their own stored partial output data of network layer 1 and submatrices of the parameter matrix of network layer 2, device 1 performs an aggregation and allocation operation on the eigenvalue vector groups calculated by itself and the other party in row order. That is, it retains the continuous rows of its own calculated eigenvalue vector group, and then obtains the vector group with the corresponding row order from the eigenvalue vector group calculated by device 2 and performs a summation operation to obtain the target vector group 303. The target vector group 303 is then saved as part of the output data of network layer 2. Similarly, device 2 retains the continuous rows of its own calculated eigenvalue vector group, and then obtains the vector group with the corresponding row order from the eigenvalue vector group calculated by device 1 and performs a summation operation to obtain the target vector group 304. The target vector group 304 is then saved as another portion of the output data of network layer 2. At the same time, the target vector group 303 and the target vector group 304 are the input vectors of device 1 and device 2 when processing network layer 3, respectively. In other words, each device stores only half of the output feature vector set for each layer. For example, suppose the parameter matrix of network layer 2 is a 4x4 matrix, which is divided into two 4x2 submatrices and stored in device 1 and device 2 respectively. The input data for network layer 1 is a 4-row vector set, which is divided into two 2-row submatrices (corresponding to...) Figure 3 Data (301 and 302) are stored in Device 1 and Device 2 respectively as input data for Network Layer 2. When calculating the output data of Network Layer 2, according to the rules of matrix operations, Device 1 calculates a 4-row vector group based on its stored 2-row vector group and 4x2 matrix. Similarly, Device 2 also calculates a 4-row vector group. After the above aggregation and allocation operation, Device 1 sums the first two rows of its calculated vector group with the first two rows of Device 2's calculated vector group to obtain a 2-row vector group (corresponding to...). Figure 3 (303) is stored in device 1 as the output vector group of network layer 2; similarly, device 2 sums the last two rows of its calculated vector group with the last two rows of the vector group calculated by device 1 to obtain another two-row vector group (corresponding to Figure 3Vector group 304 is stored in device 2 as the output vector group of network layer 2. Similarly, vector group 303 and vector group 304 will also serve as the complete input vectors of device 1 and device 2 when processing network layer 3, respectively. After performing the calculation and data communication operations described above, the two devices obtain the output vector groups 305 (stored in device 1) and 306 (stored in device 2) of network layer 3, respectively.

[0081] It should be understood Figure 3 The data transmission method shown is the same as Figure 2 It is similar, and is not limited to data communication between two devices, but can also be used for data communication between more than two devices.

[0082] 203. Store the target vector group of the above n rows in the above first device.

[0083] The aforementioned target vector set is part of the complete output vector set of the first network layer. In fact, when the sum of the number of the first device and other devices is N, the target vector set is 1 / N of the complete output vector set of the first network layer. Simultaneously, this target vector set also represents all the input data when the first device calculates the next network layer (hereinafter referred to as the second network layer). That is, the first device performs calculations based on the target vector set and the submatrix formed by consecutive columns in the parameter matrix of the second network layer to obtain the feature value vector set of the second network layer. Then, it performs the same operation of obtaining subvector sets from other devices and summing them to obtain the target vector set of the second network layer to be stored within itself. This process continues until all network layers are trained, resulting in the final output data.

[0084] The data processing method provided in this application has changed the matrix partitioning method and the data communication method between devices. Combined with the matrix operation rules, the output feature vector group is distributed and stored, which increases the amount of data processed in a single processing and improves the processing efficiency.

[0085] Figure 2 The method flow in [the document] can be understood as part of the data processing process during the application of a trained neural network model, or as part of the process during the training of the neural network model. It should be understood that if... Figure 2 The method flow described is a part of the data processing process during the application of a trained neural network model. After step 203 is completed, the target vector group of the first network layer stored in the first device will be used as input data for the second network layer. After the calculation is complete, the first device will release the memory occupied by the target vector group within the first device. If Figure 2The method flow in the text is part of the process of training a neural network model. The neural network model also needs to perform the operation of updating the parameters of the first network layer. The first device will keep the target vector group until the loss value between the output data of the neural model and the rational output value is calculated.

[0086] The following is an architecture diagram of a neural network model provided in an embodiment of this application. Please refer to [link / reference]. Figure 4 .like Figure 4 As shown, the neural network model includes: an input network 403, responsible for acquiring sample data from external input and transcoding the sample data to obtain a set of feature value vectors that can represent the sample data as the initial input data of the neural network model. For example, when the sample data is an image, the color pixel value RGB of each pixel in the image is extracted as the initial input vector set; a processing network 402, responsible for performing forward calculation on the initial input data to obtain the output result. This calculation process is processed in parallel by multiple devices, and this calculation process involves multiple network layers in the neural network model. By dividing the parameters in each network layer into columns and distributing them to the multiple devices for parallel processing, the output vector set of each network layer is distributed and stored in the multiple devices; and an output network 403, responsible for outputting the output result calculated in the processing network 402, and calculating the loss value between the output result and the ideal result according to the loss function. The loss value is then transmitted back to the processing network 402, and the processing network 402 updates the parameters of the neural network according to the loss value.

[0087] The following describes the forward computation. Figure 4 The way to output results.

[0088] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating another data processing method provided in an embodiment of this application. In some embodiments, the neural network model performs forward computation to obtain... Figure 4 The output result method is as follows Figure 5 As shown: For ease of explanation and understanding, it is assumed here that there are two devices responsible for the above processing, i.e. Figure 5 Device 1 and Device 2 are mentioned. Therefore, when Device 1 and Device 2 process data in parallel, the method for partitioning the parameter matrix of the network layers in the aforementioned neural network can be found in [reference needed]. Figure 6 .

[0089] Figure 6 This is a schematic diagram illustrating a model segmentation method provided in an embodiment of this application. Figure 6 As shown, the first and second network layers in the figure are consecutive layers in the neural network model. Matrix 601 is the complete parameter matrix of the first network layer, and it is vertically divided into submatrices 6011 (equivalent to...). Figure 5Submatrix 1) and submatrix 6012 (equivalent to Figure 5 Submatrix 2); Matrix 602 is the complete parameter matrix of this second network layer, and it is vertically divided into submatrix 6021 (equivalent to...). Figure 5 Submatrix 3) and submatrix 6022 (equivalent to Figure 5 Submatrix 4); that is, in the above neural network model, the parameter matrices of consecutive network layers are all divided vertically and evenly distributed. Figure 5 The processing is performed by devices 1 and 2. It should be understood that... Figure 6 The "division" action shown is only for the reader's convenience in understanding how the matrix is ​​divided. Before model training begins, matrices 6011 and 6021 are already stored in device 1, and similarly, matrices 6012 and 6022 are already stored in device 2. That is to say, in the actual training process, there is no "splitting" action on the parameter matrix.

[0090] exist Figure 5 In this process, device 1 calculates a vector group (hereinafter referred to as vector group 1, which can be divided into sub-vector group 1 and sub-vector group 2) based on a portion of the feature vector group of the first network layer it stores; device 2 also calculates another vector group (hereinafter referred to as vector group 2, which can be divided into sub-vector group 3 and sub-vector group 4) based on another portion of the feature vector group of the first network layer it stores. Then, device 1 obtains sub-vector group 3 of vector group 2 from device 2, sums it with sub-vector group 1 of vector group 1, and obtains target vector group 1, which is stored in device 1; similarly, device 2 obtains sub-vector group 2 of vector group 1 from device 1, sums it with sub-vector group 4 of vector group 2, and obtains target vector group 2, which is stored in device 2. The combination of target vector group 1 and target vector group 2 constitutes all the feature vectors of the second network layer.

[0091] Combination Figure 5 Specifically, device 1 stores feature vector group 501, and device 2 stores feature vector group 502. The combination of feature vector groups 501 and 502 constitutes the complete feature value vector group of the first network layer in the aforementioned neural network model. Let's assume this feature value vector group has four rows. Then, feature vector group 501 is a sub-vector group consisting of rows 1 and 2 of this feature value vector group, and feature vector group 502 is a sub-vector group consisting of rows 3 and 4 of this feature value vector group. Device 1 uses a sub-matrix of the parameter matrix of the aforementioned second network layer, i.e. Figure 5Submatrix 1 and eigenvector group 501 are used to calculate a vector group 503. Here, it is assumed that the parameter matrix is ​​a 4x4 matrix. Submatrix 1 is the 4x2 submatrix formed by the first two columns of the parameter matrix, and submatrix 2 is another 4x2 submatrix formed by the last two columns of the parameter matrix. According to the matrix operation rules, vector group 503 is a vector group (or a matrix with 4 rows). Device 1 retains its first and second rows and obtains a subvector group from Device 2. This subvector group is the root vector of Device 2. Based on the submatrix 2 and feature vector group 502, another four-row vector group 504 is calculated using rows 1 and 2. Device 1 sums the subvector group consisting of rows 1 and 2 of vector group 503 with the subvector group obtained from the second device to obtain a new two-row target vector group 505, and saves it. Similarly, Device 2 sums the subvector group consisting of rows 3 and 4 of vector group 504 with the subvector group consisting of rows 3 and 4 of 503 obtained from Device 1 to obtain another new two-row target vector group 506, and saves it. Vector groups 505 and 506 together form the complete output feature vector group of the second network layer, and vector groups 505 and 506 are stored in Device 1 and Device 2, respectively. Similarly, device 1 calculates feature vector group 507 based on its stored target vector group 505 and the submatrix 3 obtained by dividing the third network layer parameter matrix by columns. Device 2 calculates feature vector group 508 based on its stored target vector group 506 and another submatrix 4 obtained by dividing the third network layer parameter matrix by columns. Then, device 1 retains the first two rows of feature vector group 507 and sums them with the first two rows of feature vector group 508 obtained from device 2 to obtain the partial output vector group 509 of the third network layer. Likewise, device 2 retains the last two rows of feature vector group 508 and sums them with the last two rows of feature vector group 507 obtained from device 1 to obtain the partial output vector group 510 of the third network layer. That is to say, in the subsequent data processing of the network layers, both device 1 and device 2 only need to calculate a vector group based on their own stored partial feature value vector groups of the previous network layer and the submatrix obtained by dividing the parameter matrix of the network layer to be processed by columns, and perform the calculations in row-wise order. Figure 3 The aggregation and allocation operation obtains and saves the respective target vector groups until the forward computation process of all network layers of the above network model is completed, resulting in the above output results.

[0092] It should be understood that the entire process of obtaining the above output result is completed in the processing network 402. After that, the output result is sent to the output network 403. The output network calculates the loss value between the output result and the ideal result according to the loss function, and transmits the loss value back to the processing network 402. The processing network 402 updates the parameters of the above neural network according to the loss value.

[0093] Please see Figure 7 , Figure 7 This is a flowchart illustrating a parallel model training method provided in an embodiment of this application. Figure 7 As shown, Figure 4 The neural network model in the model can be obtained through a parallel training method, which may include:

[0094] 701. The first device processes the input sample data to obtain the target vector group of the first network layer.

[0095] The sample data can be images, text, or speech. It should be understood that before the above processing, the sample data has been converted into a set of feature value vectors (hereinafter referred to as the initial vector set) that can be recognized by a machine. This initial vector set can be the input data of the first network layer; it can also be the input data of other network layers preceding the first network layer. The above processing is also a process in which multiple devices work in parallel; for details, please refer to [link to relevant documentation]. Figure 5 The process of multiple devices processing data in parallel, taking the operation mode of a single device as an example, includes the following:

[0096] The first device calculates a first vector group based on the first matrix and the first input data. The first matrix is ​​a matrix that includes consecutive columns of the parameter matrix of the first network layer. The first input data may be a part of the initial vector group or the part of the output vector group of the previous network layer (hereinafter referred to as the third network layer) stored in the first device.

[0097] The target vector group is obtained by summing the first sub-vector group and R second sub-vector groups of the first vector group. The first sub-vector group is a vector group consisting of n consecutive rows from the first vector group. The R second sub-vector groups come from R other devices different from the first device. Each of these R devices stores a portion of the total output vector group of the third network layer and a sub-matrix consisting of consecutive columns of the parameter matrix of the first network layer. Each device calculates a feature vector group based on its stored portion of the total output vector group of the third network layer and the sub-matrix consisting of consecutive columns of the parameter matrix of the first network layer. Each feature vector group is provided to the first device as an n-row second sub-vector group. Therefore, the first device obtains a total of R second sub-vector groups from the R other devices. It should be understood that, among the aforementioned R other devices and the aforementioned first device, the portion of the total output vector set of the aforementioned third network layer stored by each device is different, and the collection of the portion of the output vector set of the aforementioned third network layer stored in all devices constitutes the complete output vector set of the aforementioned third network layer (which is also the complete input vector set of the aforementioned first network layer).

[0098] The target vector group is stored in the first device; the target vector group is all the input vectors when the first device calculates the output data of the second network layer, and the second network layer is the layer after the first network layer in the neural network model.

[0099] In the above processing, the data stored in the various devices are different, but the data processing methods are roughly the same. That is, while the first device obtains sub-vector groups from the other R devices, each of the other R devices also obtains a sub-vector group from the first device and (R-1) other devices (excluding itself). These sub-vector groups are then summed with the sub-vector groups of the feature vector group calculated by the device itself to obtain the target vector group to be stored in itself. The row positions of the sub-vector groups retained by all devices should be the same as the row positions of the sub-vector groups they obtained in their source vector groups. For example, if a device retains rows 1-2 of its own calculated feature value vector group for summation, then the sub-vector groups obtained by that device from the other R devices will also be sub-vector groups consisting of rows 1-2 of the feature value vector groups calculated by the other R devices. Similarly, if a device retains rows 4-6 of its own calculated feature value vector group for summation, then the sub-vector groups obtained by that device from other devices will also be sub-vector groups consisting of rows 4-6 of the feature vector groups of that network layer calculated by the other devices.

[0100] Furthermore, the collection of target vector sets stored in each of the aforementioned R other devices and the aforementioned first device (including the target vector set in the aforementioned first device) constitutes the complete output vector set of the aforementioned first network layer, and also constitutes the complete input vector set of the aforementioned second network layer.

[0101] 702. The first device mentioned above calculates the loss value based on the target vector group and the ideal output value of the neural network model.

[0102] The processing in step 701 is the forward computation process during the training of the aforementioned neural network model. It should be understood that after multiple devices process the sample data using multiple network layers in the aforementioned neural network model in step 701, an output result is obtained after the forward computation. This output result is the result obtained after one iteration of the sample data, processed in an orderly manner by all network layers in the aforementioned neural network model. Simultaneously, the neural network model also has a preset ideal output value and a loss function. This loss function is used to calculate the output error between the ideal output value and the aforementioned output result. After obtaining this output error, the model begins the backpropagation process. Backpropagation involves adjusting the parameters of the network layers in the neural network model to make the output result obtained in the next iteration as close as possible to the aforementioned ideal output value. Taking the process of updating the parameters of the first network layer stored by the first device as an example, after obtaining the output error, the first device uses the output error and the partial output vector group of the first network layer stored therein (i.e., the target vector group of the first network layer stored in the first device) to calculate the influence of each parameter in the partial parameter matrix of the first network layer stored therein on the output error using the chain rule. Taking the parameter w1 of the first network layer stored by the first device as an example, in some embodiments, the influence of w1 on the output error can be calculated by the following formula.

[0103]

[0104] Among them, E total The above output error, To account for the aforementioned level of impact, The first target vector group saved for device 10011 The input to the first network layer is weighted and summed.

[0105] Meanwhile, the other R devices calculate the influence of other sub-matrices on the output error based on their respective stored partial output vector groups of the first network layer and the aforementioned output error.

[0106] 703. The first device updates the parameter matrix of the first network layer based on the loss value.

[0107] The first device updates the first matrix based on the aforementioned loss value and the characteristics of the model (such as the number of model layers or the model learning rate). Again, taking parameter w1 as an example, in some embodiments, the update formula for parameter w1 can be:

[0108]

[0109] Among them, w1 + Here are the updated parameter values, and η is the model's learning rate. This represents the degree of influence of w1 on the output error of the above numbers.

[0110] Simultaneously, the aforementioned R other devices also update the submatrices of the parameter matrix of the first network layer they store using their respective calculated loss values. This process is repeated for subsequent network layers.

[0111] It should be understood that steps 702 and 703 constitute the backpropagation process during the training of the aforementioned neural network model. Similar to the forward computation, the backpropagation process is processed in parallel by multiple devices. During the backpropagation process, the first device and the aforementioned R devices only need to calculate the aforementioned loss value based on the output feature vector set of a certain network layer (actually 1 / (R+1)) that they have saved and the aforementioned output error.

[0112] The following is a schematic diagram of the architecture of a parallel training group for a model provided in an embodiment of this application. Please refer to [link / reference]. Figure 8 ;like Figure 8 As shown, during model training, distributed neural network training software is deployed on the server's CPU. This software generates and distributes computational graphs and sample data that require accelerator computation. Figure 8 If any one of accelerators 1 to 4 is equivalent to the "first device" described in the above method, then the remaining three accelerators are equivalent to R devices, including the second device, in the above method. The accelerators perform calculations according to the computation graph and corresponding sample data and feed back the results to the CPU.

[0113] In some embodiments, the above-mentioned parallel model training group may belong to a model training cluster system (see details). Figure 9 The model training cluster system can include multiple [systems / families / etc.]. Figure 8 Similar to parallel model training groups, multiple parallel model training groups process training sample data in parallel, that is, they train models in a hybrid parallel manner. Figure 8 The network interface card (NIC) in the cluster system is responsible for the parallel data processing between multiple model training groups.

[0114] The following is a schematic diagram of the architecture of a model training cluster system provided in the embodiments of this application. Please refer to [link / reference]. Figure 9 .

[0115] Figure 9 This is a schematic diagram of the architecture of a typical model training cluster system, such as... Figure 9 As shown, the cluster system includes multiple parallel training groups, where each parallel training group (equivalent to...) Figure 8The parallel training group shown includes a CPU, where each central processing unit controls one or more GPUs or other computing acceleration devices for training, and one or more servers are connected via a switch (such as...). Figure 9 The network interconnects the components (902). Taking the parallel training group 901 as an example, the parallel training group 901 includes a central processing unit 9011. The central processing unit 9011 is responsible for controlling three graphics processing units 9012, 9013, and 9014 to train the aforementioned neural network model in parallel. Graphics processing unit 9012 can be regarded as the "first device" mentioned in the previous method, while graphics processing units 9013 and 9014 can be regarded as "R devices" including the second device mentioned in the previous method. They train the aforementioned neural network model in parallel. The parallel training group 901 processes the sample data of the aforementioned neural network model in parallel with other training groups.

[0116] The following describes the architecture diagram of another model training cluster system provided in this application embodiment. The model training cluster system provided in this application embodiment is still based on a hybrid parallel scenario, grouping the accelerators within the cluster. Accelerators within a group perform model parallelism, while accelerators at corresponding positions between groups perform data parallelism. For details, please refer to... Figure 10 .

[0117] Figure 10 This is an architecture diagram of a model training cluster system provided in an embodiment of this application. Figure 10 As shown, the cluster system has multiple parallel training groups, namely parallel training group 1001, parallel training group 1002, parallel training group 1003, and parallel training group 1004. Each parallel training group trains the complete neural network model described above. Each parallel training group contains multiple acceleration devices, which can be GPUs or other computing acceleration devices. Within the same parallel training group, multiple devices train a portion of the complete neural network model in parallel; that is, each of the multiple devices is responsible for processing the parameters of each network layer in the neural network model. It should be understood that the parameters of the neural network model exist in matrix form. During training, the parameter matrix of each network layer is evenly divided along the column direction and stored separately in the multiple devices for training. During training, each device uses... Figure 3 The data communication method shown is used for data transmission through aggregation and allocation, so that during the entire training process, multiple devices in the same parallel training group only need to store a portion of the feature values ​​of each network layer. Taking parallel training group 1001 in the figure as an example, there are four devices used for parallel training in parallel training group 1001: device 10011, device 10012, device 10013, and device 10014. Assume that the parameter matrix of the first network layer in the above neural network model is a 4x4 matrix, denoted as... Since there are 4 devices in the parallel training group, the parameter matrix is ​​divided into columns. and Four 4x1 sub-matrices are processed by the four devices. In this case, the input vector set used by each device to calculate the first network layer should also be a 1-row matrix. It should be understood that the input vector set here can be the complete output feature vector set of the previous network layer, or the feature vector set obtained by transforming the initial sample data; here, we assume it is a 1x4 matrix. According to the rules of matrix operations, after each device calculates this input vector set and its own sub-matrix from the aforementioned sub-matrices, the result is a 4x4 matrix, or in other words, it can be viewed as a vector set containing four row vectors. That is, after training the first network layer, each of the four devices obtains a vector set containing four row vectors. Then, each device retains one row vector from its own calculated vector set containing four row vectors, and obtains the corresponding row vectors from the row vector sets calculated by the other three devices. The four row vectors are then summed to obtain 1 / 4 of the feature vector set of the first network layer, and this is saved. Taking device 10011 in parallel training group 1001 as an example, the aforementioned sub-matrix is ​​stored in device 10011. The input vector group (1x4) is used to calculate a 4x4 matrix, which is a first vector group consisting of four row vectors. Similarly, the other three devices will also obtain three other vector groups consisting of four row vectors each, based on their respective stored sub-matrices and input vector groups. These are referred to as the second, third, and fourth vector groups, respectively. Device 10011 stores the first row of this first vector group and obtains the first rows of the second, third, and fourth vector groups. It then sums these four sub-vector groups to obtain the first target vector group and stores it within itself. This first target vector group is 1 / 4 of the feature vector group of the first network layer. Similarly, devices 10011, 1012, and 10013 store the second, third, and fourth rows of their calculated vector groups, respectively, and obtain the corresponding row order sub-vector groups from the other devices, summing them to obtain their respective second, third, and fourth target vector groups, which are then stored. The complete set of the second, third, and fourth target vector groups, along with the first target vector group, constitutes the complete feature vector group of the first network layer. This complete feature vector group is distributed and stored in the four devices, thus optimizing the memory of the devices. Therefore, more sample data can be processed in one iteration, improving the training efficiency of the model.

[0118] The training process of the aforementioned neural network model also includes a backpropagation process. It should be understood that the training process is the forward computation process of the neural network model. After the multiple devices sequentially complete the computational processing of each network layer, they obtain the final output value of that forward computation. The neural network model has a preset ideal output value. During the backpropagation process, the devices update the parameters of each network layer they have stored based on the output error between the final output value and the ideal output value. After multiple iterations, the final output value is made as close as possible to the ideal output value.

[0119] For example, taking the parameter w1 of the first network layer stored in device 10011 as an example, after the output error is calculated by the preset error function of the neural network model, device 10011 uses the chain rule and its stored feature vector group of the first network layer to determine the degree of influence of parameter w1 on the output error, and updates the value of parameter w1 based on this degree of influence. The error function can be the following function:

[0120]

[0121] Among them, E total The sum of the above output errors is given, where target is the ideal output value and output is the preset value. The degree of influence on parameter w1 can be calculated using the following chain rule:

[0122]

[0123] in The aforementioned degree of impact, The first target vector group saved for device 10011 The input to the first network layer is weighted and summed.

[0124] The update formula for parameter w1 can be:

[0125]

[0126] Among them, w1 + Here are the updated parameter values, and η is the model's learning rate. This represents the degree of influence of w1 on the output error of the above numbers.

[0127] Furthermore, in this embodiment, devices at corresponding positions across multiple parallel training groups process data in parallel. Because each device within a parallel training group retains a sub-vector group of its own calculated vector group with a specified row order during intra-group data communication, and obtains the corresponding row order sub-vector groups of the vector groups calculated by other devices for summation operations, for example... Figure 10Device 10011 (i.e. Figure 10 Taking device D0 as an example, D0 can be considered as the device in the first position in parallel training group 1001. In parallel training groups 1002, 1003, and 1004, the devices corresponding to D0 are D4, D8, and D12, respectively. All four devices retain the first quarter of all rows of the feature value vector group for each network layer. In other words, devices D0, D4, D8, and D12 can be considered as four devices performing data parallelism. Therefore, the devices in corresponding positions across groups process the data input to the cluster system in parallel.

[0128] This embodiment first groups the devices within the cluster. Devices within a group perform model parallelism, while devices at corresponding positions across groups perform data parallelism. This effectively solves the problem of insufficient device memory due to excessive model size, without introducing too much communication overhead, thus improving cluster training efficiency.

[0129] The following is an architecture diagram of another model training cluster system provided in an embodiment of this application. Please refer to [link / reference]. Figure 11 .like Figure 11 As shown, the cluster system includes 32 servers. Each server (e.g., server 1) has 16 GPUs divided into two groups (group 1101 and group 1102). Each group has 8 GPUs forming a model parallel training group, training 64 neural network models in parallel. Data parallelism is achieved between these 64 groups. Model parallelism within a group uses the intra-server network links, while data parallelism between groups uses the inter-server network cards.

[0130] In each parallel training group (e.g., group 1101), eight GPUs process the parameters of the network layers in the aforementioned neural network model in parallel. Here, 11011 represents a network layer (or node) in the neural network model. Each network layer is divided into eight equal parts, indicating that the layer is processed by all eight GPUs. It should be understood that before the cluster system trains the aforementioned neural network model, the parameter matrix of the network layers has already been divided into eight equal parts along the column direction and stored in the eight GPUs. The processing method can be found in [reference needed]. Figure 5The data processing method shown involves each GPU using a portion of the output data from the previous network layer as input data for its current network layer, along with a portion of the parameters of that network layer (actually 1 / 8). This allows them to calculate a vector set. Then, each of the eight GPUs retains 1 / 8 of its calculated vector set (a sub-vector set consisting of 1 / 8 of a specific row order from all rows of that vector set). They then obtain the corresponding sub-vector sets from the vector sets calculated by the other seven GPUs, sum them, and obtain their respective target vector sets to be saved. Each GPU's saved target vector set is 1 / 8 of the total feature vector sets of its current network layer, and also represents all the input data for the GPU storing this target vector set when processing the next network layer. Furthermore, the combined set of target vector sets stored by the eight GPUs constitutes the total feature vector sets of the current network layer. This process continues, completing the forward computation of the model described above.

[0131] After completing the forward computation process described above, the cluster system will also perform a backpropagation process. During this backpropagation, each GPU in the same parallel training group only needs to update a submatrix of the parameter matrix of a network layer it stores based on a partial feature vector set of that network layer. The process can be found in [reference needed]. Figure 10 The backpropagation process of the cluster system will not be elaborated here.

[0132] The following is a flowchart of an image processing method provided in an embodiment of this application. Please refer to [link / reference]. Figure 12 .like Figure 12 As shown, the image processing method includes:

[0133] 1201. Use the trained neural network model to process the input image data and obtain the processing results.

[0134] The training process of this neural network model can include, for example: Figure 7 The training process is shown, and the training scenario for this neural network model can be as follows: Figure 10 or Figure 11 The training cluster system shown indicates that the neural network model is trained in parallel by multiple devices, each of which only stores its own trained portion of the model. Therefore, the image data processing is still performed in parallel by multiple devices.

[0135] The following example illustrates this concept. For ease of understanding, assume the training process of the aforementioned neural network model is performed by two devices (hereinafter referred to as the first device and the second device, which can be a CPU, GPU, or other acceleration device). The parameter matrix of each network layer of the aforementioned neural network model is divided into two parts along the column direction and stored in the first device and the second device, respectively. Then, the aforementioned processing includes:

[0136] The image data to be processed undergoes preprocessing. It should be understood that in real-world scenarios, images may encounter problems such as incorrect angles affecting recognition, or slight occlusion (e.g., noise), making them difficult to identify. Therefore, image data correction is necessary; the corrected image is easier to classify accurately. Furthermore, the preprocessing includes converting the corrected image into an initial feature vector set that characterizes the image's properties. For example, this initial feature vector set can be obtained based on the RGB color values ​​of each pixel in the image. This initial feature vector set serves as the initial input data for the aforementioned neural network model.

[0137] The first device and the second device process the initial feature vector group in parallel, performing data processing in an orderly manner according to the network layer architecture of the neural network model (see details for reference). Figure 5 or Figure 7The data processing involved is as follows: The first device stores the first half of all rows in the initial vector group (which can be regarded as a vector group including multiple row vectors) and the first half of all columns in the parameter matrix of the first network layer to form a first sub-matrix, and calculates a vector group (hereinafter referred to as vector group 1); similarly, the second device stores the last half of all rows in the initial vector group and the last half of all columns in the parameter matrix of the first network layer to form a second sub-matrix, and calculates another vector group (hereinafter referred to as vector group 2). The first device obtains the first half of all rows of vector group 2, sums it with the first half of all rows of vector group 1, and obtains half of the complete feature vector group of the first network layer. This half is stored in the first device, and the memory occupied by the sub-vector groups of the initial vector group stored therein is released in the second device. The second device obtains the last half of all rows of vector group 1, sums it with the last half of all rows of vector group 2, and obtains the other half of the complete feature vector group of the first network layer. This half is stored in the second device, and the memory occupied by the sub-vector groups of the initial vector group stored therein is released in the second device. Similarly, when calculating the feature vector group of the next network layer (hereinafter referred to as the second network layer), the first and second devices calculate a vector group based on the sub-matrices of the parameter matrix of the second network layer they have stored, and a portion of the feature vector group of the first network layer. After the two vector groups undergo the data exchange and summation operations described above, they obtain the portion of the feature vector group of the second network layer that they each want to store, and the memory occupied by the feature vector group of the first network layer in the device is released. Similarly, after the device calculates the feature vector set of a network layer based on the feature vector set of the network layer above it, the device will release the feature vector set of the network layer above it from the memory it occupies until the final processing result is obtained.

[0138] It should be understood that the above neural network model has been trained, and the result obtained after each processing of image data is the ideal output result. Therefore, there is no need to perform backpropagation to adjust the parameters in the model. Thus, after the device calculates the feature vector group of a network layer based on the feature vector group of the network layer above it, the device can release the memory occupied by the feature vector group of the network layer above it, so that the device can process more data in one processing.

[0139] 1202. Output the above processing results.

[0140] The output of this processing result can be from terminal devices such as mobile phones, tablets, laptops, and desktop computers, or it can be from a server. It should be understood that the above processing result is actually a feature vector set, and the actual output result should be the classification result of the image data obtained after analyzing and transforming this feature vector set. For example, the image may be a person, a dog, or a flower; or the image may have an 80% probability of being a dog, an 18% probability of being a cat, and a 2% probability of being a hat.

[0141] It should be understood that conventional neural network models require the device to store the complete feature vector set of each network layer. The neural network model used in the image processing method provided in this application embodiment can distribute and store the feature vector sets of the network layers in multiple devices, so that the devices have more memory to process image data and improve processing efficiency.

[0142] Figure 13 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application, as shown below. Figure 13 As shown, the device includes:

[0143] The calculation unit 1301 is used to calculate a first vector group based on a first matrix and first input data; the first matrix is ​​a matrix with m consecutive columns including the parameter matrix of the first network layer, the first network layer is any layer in the neural network model, and the parameter matrix is ​​an N-row M-column matrix.

[0144] Aggregation unit 1302 is configured to acquire R second sub-vector groups from other devices, and sum the first sub-vector group of the first vector group and the R second sub-vector groups to obtain a target vector group with n rows; the first sub-vector group is a vector group comprising n consecutive rows of the first vector group; any one of the R second sub-vector groups is a vector group comprising n consecutive rows of the second vector group, and the second vector group is an N-row vector group calculated by the second device based on the second matrix with N rows and the second input data; the second matrix is ​​a matrix comprising consecutive columns of the parameter matrix, and all columns in the second matrix are in different positions from all columns in the first matrix in the parameter matrix;

[0145] In one optional implementation, the aggregation unit is specifically used to obtain a sub-vector group from each of the other R devices and sum it with the sub-vector group corresponding to the row order of its own calculated feature vector group. Each sub-vector group is a sub-vector group with a specified row order from the feature vector group calculated by another device based on a portion of the input data of the first network layer and a submatrix of the parameter matrix. The row order is determined by the position of the sub-vector group of its own calculated feature vector group within the feature vector group. For example, if a device retains the first two rows of its own calculated feature vector group, it also obtains the first two rows of the feature vector groups calculated by those R devices from the other R devices and performs the summation operation.

[0146] In one optional implementation, both the first and second matrices are m-column matrices. That is, the parameter matrix is ​​divided into (R+1) m-column matrices, which are then processed in parallel by the first device and the R devices, including the second device. In this implementation, both the first and second input data are consecutive n-row sub-vector groups of all input data for the first network layer. That is, all input data for the first network layer is also divided into (R+1) sub-vector groups, which are then distributed row-wise to the first device and the R devices, including the second device, for parallel processing.

[0147] Storage unit 1303 is used to store the target vector group of the above n rows in the first device; the target vector group of the above n rows is all the input vectors when the first device calculates the output data of the second network layer, and the second network layer and the above first network layer are two consecutive layers in the above neural network model.

[0148] In an alternative implementation, the data processing apparatus is an apparatus used in the neural network model training process, and the apparatus further includes:

[0149] The reverse unit 1304 is used to calculate the loss value based on the target vector group of the n rows and the ideal output value of the neural network model; and update the first matrix based on the loss value.

[0150] In an optional implementation, storage unit 1304 is further configured to: release the memory occupied by the target vector group of the above n rows after the loss value is calculated by the above-mentioned reverse unit.

[0151] Figure 14 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application, as shown below. Figure 14 As shown, the device includes:

[0152] Processing unit 1401 is used to process input sample data to obtain a target vector group; the processing includes: calculating a first vector group based on a first matrix and first input data; the first matrix is ​​a matrix including m consecutive columns of a parameter matrix of a first network layer, the first network layer is any layer in a neural network model, the parameter matrix is ​​an N-row M-column matrix, and the first vector group is an N-row vector group; summing a first sub-vector group and R second sub-vector groups of the first vector group to obtain the target vector group; the first sub-vector group is a vector group including n consecutive rows of the first vector group; any one of the R second sub-vector groups is a vector group including n consecutive rows of the second vector group, and the second vector group is an N-row vector group calculated by a second device based on an N-row second matrix and second input data; the second matrix is ​​a matrix including consecutive columns of the parameter matrix, and all columns in the second matrix are in different positions from all columns in the first matrix in the parameter matrix;

[0153] Storage unit 1402 is used to store the target vector group mentioned above; the target vector group is all the input vectors when the first device calculates the output data of the second network layer, and the second network layer and the first network layer are two consecutive layers in the neural network model.

[0154] The reverse unit 1403 is used to calculate the loss value based on the target vector group of the above n rows and the ideal output value of the above neural network model; and to update the above first matrix based on the loss value.

[0155] In an optional implementation, after the reverse unit calculates the loss value, the storage unit is also used to release the memory occupied by the target vector group of the n rows in the first device.

[0156] In one optional implementation, both the first and second matrices are m-column matrices. That is, the parameter matrix is ​​divided into (R+1) m-column matrices, which are then processed in parallel by the first device and the R devices, including the second device. In this implementation, both the first and second input data are consecutive n-row sub-vector groups of all input data for the first network layer. That is, all input data for the first network layer is also divided into (R+1) sub-vector groups, which are then distributed row-wise to the first device and the R devices, including the second device, for parallel processing.

[0157] Figure 15 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application, as shown below. Figure 15 As shown, the device includes:

[0158] Processing unit 1501 is used to process input image data using a trained neural network model to obtain a processing result. The processing includes: a first device calculating a first vector group based on a first matrix and the first image data; the first matrix is ​​a matrix containing m consecutive columns of a parameter matrix of a first network layer, the first network layer being any layer in the neural network model, the parameter matrix being an N-row, M-column matrix, and the first vector group being an N-row vector group; summing a first sub-vector group and R second sub-vector groups of the first vector group to obtain the target vector group; the first sub-vector group is a vector group containing n consecutive rows of the first vector group; any one of the R second sub-vector groups is a vector group containing n consecutive rows of the second vector group, and the second vector group is an N-row vector group calculated by the second device based on an N-row second matrix and the second image data; the second matrix is ​​a matrix containing consecutive columns of the parameter matrix, and all columns in the second matrix are in different positions from all columns in the first matrix in the parameter matrix.

[0159] In an alternative implementation, processing unit 1501 is further configured to... Figure 15 The image data is corrected. In real-world scenarios, images may encounter problems such as incorrect angles affecting recognition, or slight occlusions (such as noise), making them difficult to identify. Images obtained after correction are easier to classify accurately.

[0160] Storage unit 1502 is used to store the target vector group in the first device and release the memory occupied by the first image data in the first device; the target vector group is all the input vectors when the first device calculates the output data of the second network layer, and the second network layer and the network layer are two consecutive layers in the network model.

[0161] Output unit 1503 is used to output the above processing results; the output object can be a terminal device such as a mobile phone, tablet computer, laptop computer, or desktop computer, or a server. It should be understood that the above processing results are actually a feature vector group, and the actual output result should be the classification result of the above image data obtained after analyzing and transforming the feature vector group, such as the image being a person, a dog, or a flower; or the image having an 80% probability of being a dog, an 18% probability of being a cat, and a 2% probability of being a hat.

[0162] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 16 As shown, the electronic device 160 includes a processor 1601, a memory 1602, and a communication interface 1603; the processor 1601, the memory 1602, and the communication interface 1603 are interconnected via a bus. Figure 16The electronic device mentioned can be the data processing device or the model training device in the foregoing embodiments.

[0163] The memory 1602 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CDROM), and is used to store related instructions and data. The communication interface 1603 is used to receive and send data.

[0164] Processor 1601 may be one or more central processing units (CPUs). If processor 1601 is a single CPU, the CPU includes at least two acceleration devices, which may be GPUs or other acceleration devices. The steps performed by the data processing device and the model training device in the above embodiments can be based on this... Figure 16 The structure of the electronic device is shown. Specifically, the processor 1601 can implement... Figure 13 , Figure 14 and Figure 15 The functions of each unit.

[0165] The processor 1601 in the electronic device 160 is used to read the program code stored in the memory 1602 and execute the data processing method or model training method in the foregoing embodiments.

[0166] In embodiments of this application, another computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, performs the following: calculating a first vector group based on a first matrix and first input data; the first matrix is ​​a matrix comprising m consecutive columns of a parameter matrix of a first network layer, the first network layer being any layer in a neural network model, the parameter matrix being an N-row, M-column matrix, where N and M are integers greater than 0, and m is an integer greater than 0 and less than M; the first vector group is an N-row vector group; obtaining R second sub-vector groups from other devices, and performing the calculation on the first... The first sub-vector group of the vector group is summed with the aforementioned R second sub-vector groups to obtain an n-row target vector group. The aforementioned first sub-vector group is a vector group consisting of n consecutive rows from the aforementioned first vector group. Any one of the aforementioned R second sub-vector groups is a vector group consisting of n consecutive rows from the aforementioned second vector group. The aforementioned second vector group is an N-row vector group calculated by the second device based on the N-row second matrix and the second input data. The aforementioned second matrix is ​​a matrix consisting of consecutive columns from the aforementioned parameter matrix, and all columns in the aforementioned second matrix are in different positions from all columns in the aforementioned first matrix in the aforementioned parameter matrix. The aforementioned R is an integer greater than 0. The aforementioned n-row target vector group is stored in the first device. The aforementioned n-row target vector group is all the input vectors when the aforementioned first device calculates the output data of the second network layer. The aforementioned second network layer and the aforementioned first network layer are two consecutive layers in the aforementioned neural network model.

[0167] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, include: The first device calculates a first vector group based on a first matrix and first input data. The first input data is a part of an initial vector group, which is obtained by converting an image, text, or speech. The first matrix is ​​an N-row matrix composed of a part of all parameters of a first network layer. The first network layer is any layer in a neural network model, where N is an integer greater than 0. The first vector group is an N-row vector group. The first device obtains one sub-vector group from each of the R devices, resulting in R second sub-vector groups; The first device sums the first sub-vector group of the first vector group and the R second sub-vector groups to obtain an n-row target vector group; the first sub-vector group is a vector group that includes n consecutive rows from the first vector group; any one of the R second sub-vector groups is a vector group that includes n consecutive rows from the second vector group, and the second vector group is an N-row vector group calculated by the second device based on an N-row second matrix and second input data; the second matrix is ​​an N-row matrix composed of a portion of all parameters of the first network layer, and R is an integer greater than 0; The target vector group of n rows is stored in the first device; the target vector group of n rows is all the input vectors when the first device calculates the output data of the second network layer, and the second network layer and the first network layer are two consecutive layers in the neural network model.

2. The method according to claim 1, characterized in that, The method further includes: The first device calculates the loss value based on the target vector group of n rows and the ideal output value of the neural network model; The first device updates the first matrix based on the loss value.

3. The method according to claim 2, characterized in that, After calculating the loss value based on the n rows of target vectors and the ideal output value of the neural network model, the method further includes: Release the memory occupied by the n rows of target vectors in the first device.

4. The method according to any one of claims 1 to 3, characterized in that, The first device is a device in a first parallel training group, which is a training group in a cluster system for training the neural network model. The first parallel training group also includes R devices. The cluster system also includes at least one second parallel training group, which includes multiple devices. The devices included in the first parallel training group and the devices included in the second parallel training group are different. Both the first parallel training group and the second parallel training group train the complete neural network model. Both the first parallel training group and the second parallel training group process a portion of all the input data of the neural network model.

5. A method for training a neural network model, characterized in that, The method includes: The input sample data is processed to obtain a target vector group, wherein the sample data is an image, text, or speech; the processing includes: a first device calculating a first vector group based on a first matrix and first input data, wherein the first input data is a part of an initial vector group, and the initial vector group is obtained by converting the sample data; the first matrix is ​​an N-row matrix composed of a part of all parameters of a first network layer, wherein the first network layer is any layer in a neural network model, and N is an integer greater than 0; the first vector group is an N-row vector group; the first device obtains one sub-vector group from each of R devices to obtain R second sub-vector groups; the first device processes the first sub-vector group of the first vector group and the R second sub-vector groups. Summing yields an n-row target vector group; the first sub-vector group is a vector group comprising n consecutive rows from the first vector group; any one of the R second sub-vector groups is a vector group comprising n consecutive rows from the second vector group, the second vector group being an N-row vector group calculated by the second device based on an N-row second matrix and second input data; the second matrix is ​​an N-row matrix composed of a portion of all parameters of the first network layer, where R is an integer greater than 0; the n-row target vector group is stored in the first device; the n-row target vector group is all input vectors when the first device calculates the output data of the second network layer, and the second network layer and the first network layer are two consecutive layers in the neural network model; The first device calculates a loss value based on the target vector group of n rows and the ideal output value of the neural network model; and updates the first matrix based on the loss value.

6. An image processing method, characterized in that, The method includes: The input image data is processed using a trained neural network model to obtain a processing result. The processing includes: a first device calculating a first vector group based on a first matrix and first image data, wherein the first image data is a part of an initial vector group, and the initial vector group is obtained by transforming the image data; the first matrix is ​​an N-row matrix composed of a part of all parameters of a first network layer, wherein the first network layer is any layer in the neural network model, and N is an integer greater than 0; the first vector group is an N-row vector group; the first device obtains one sub-vector group from each of R devices to obtain R second sub-vector groups; the first device sums the first sub-vector group of the first vector group and the R second sub-vector groups to obtain an n-row target vector group; The first sub-vector group is a vector group comprising n consecutive rows from the first vector group; any one of the R second sub-vector groups is a vector group comprising n consecutive rows from the second vector group, the second vector group being an N-row vector group calculated by the second device based on the N-row second matrix and the second image data; the second matrix is ​​an N-row matrix composed of a portion of all parameters of the first network layer, where R is an integer greater than 0; the n-row target vector group is stored in the first device, releasing the memory occupied by the first image data in the first device; the n-row target vector group is all the input vectors when the first device calculates the output data of the second network layer, and the second network layer and the first network layer are two consecutive layers in the neural network model; Output the processing result.

7. A data processing apparatus, characterized in that, The device includes: The computing unit is used to calculate a first vector group based on a first matrix and first input data, wherein the first input data is a part of an initial vector group, which is obtained by converting an image, text, or speech; the first matrix is ​​an N-row matrix composed of a part of all parameters of a first network layer, wherein the first network layer is any layer in a neural network model, and N is an integer greater than 0; the first vector group is an N-row vector group. An aggregation unit is used to obtain one sub-vector group from each of R devices to obtain R second sub-vector groups; The aggregation unit is further configured to sum the first sub-vector group of the first vector group and the R second sub-vector groups to obtain a target vector group with n rows; the first sub-vector group is a vector group comprising n consecutive rows from the first vector group; any one of the R second sub-vector groups is a vector group comprising n consecutive rows from the second vector group, and the second vector group is an N-row vector group calculated by the second device based on an N-row second matrix and second input data; the second matrix is ​​an N-row matrix composed of a portion of all parameters of the first network layer, and R is an integer greater than 0; A storage unit is used to store the n rows of target vectors in a first device; the n rows of target vectors are all the input vectors when the first device calculates the output data of the second network layer, and the second network layer and the first network layer are two consecutive layers in the neural network model.

8. The apparatus according to claim 7, characterized in that, The device further includes: The inverse unit is used to calculate the loss value based on the target vector group of n rows and the ideal output value of the neural network model; and to update the first matrix based on the loss value.

9. The apparatus according to claim 8, characterized in that, The storage unit is also used for: After the inverse unit calculates the loss value, the memory occupied by the n rows of target vectors in the first device is released.

10. A training device for a neural network model, characterized in that, The device includes: A processing unit is configured to process input sample data to obtain a target vector group, wherein the sample data is an image, text, or speech; the processing includes: a first device calculating a first vector group based on a first matrix and first input data, wherein the first input data is a part of an initial vector group, and the initial vector group is obtained by converting the sample data; the first matrix is ​​an N-row matrix composed of a portion of all parameters of a first network layer, wherein the first network layer is any layer in a neural network model, and N is an integer greater than 0; the first vector group is an N-row vector group; the first device obtains one vector from each of R devices. The first device generates R second sub-vector groups from the first vector group; the first device sums the first sub-vector group and the R second sub-vector groups of the first vector group to obtain an n-row target vector group; the first sub-vector group is a vector group that includes n consecutive rows from the first vector group; any one of the R second sub-vector groups is a vector group that includes n consecutive rows from the second vector group, and the second vector group is an N-row vector group calculated by the second device based on an N-row second matrix and second input data; the second matrix is ​​an N-row matrix composed of a portion of all parameters of the first network layer, and R is an integer greater than 0; A storage unit is used to store the n rows of target vector groups in the first device; the target vector groups are all input vectors when the first device calculates the output data of the second network layer, and the second network layer and the first network layer are two consecutive layers in the neural network model; The inverse unit is used to calculate the loss value based on the target vector group of n rows and the ideal output value of the neural network model; and to update the first matrix based on the loss value.

11. An image processing apparatus, characterized in that, The device includes: A processing unit is configured to process input image data using a trained neural network model to obtain a processing result. The processing includes: a first device calculating a first vector group based on a first matrix and first image data, wherein the first image data is a part of an initial vector group, and the initial vector group is obtained by transforming the image data; the first matrix is ​​an N-row matrix composed of a part of all parameters of a first network layer, wherein the first network layer is any layer in the neural network model, and N is an integer greater than 0; the first vector group is an N-row vector group; the first device obtains one sub-vector group from each of R devices to obtain R second sub-vector groups; the first device sums the first sub-vector group of the first vector group and the R second sub-vector groups to obtain an n-row target vector group; the first sub-vector group is a vector group comprising n consecutive rows from the first vector group; any second sub-vector group among the R second sub-vector groups is a vector group comprising n consecutive rows from the second vector group; the second vector group is an N-row vector group calculated by a second device based on an N-row second matrix and the second image data; the second matrix is ​​an N-row matrix composed of a part of all parameters of the first network layer, and R is an integer greater than 0. A storage unit is used to store the n rows of target vectors in the first device and release the memory occupied by the first image data in the first device; the n rows of target vectors are all the input vectors when the first device calculates the output data of the second network layer, and the second network layer and the first network layer are two consecutive layers in the neural network model; The output unit is used to output the processing result.

12. A computer-readable storage medium, characterized in that, The computer storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 6.

13. An electronic device, characterized in that, include: Memory, used to store programs; A processor for executing the program stored in the memory, wherein when the program is executed, the processor is configured to perform the method as described in any one of claims 1 to 6.

Images