Model data processing methods, readable media, and electronic devices

By dividing the data to be processed and the operators of the neural network model into multiple data groups and sub-operator groups, the problem of low computational efficiency of recurrent neural network models is solved, parallel computation of processing units is realized, and overall computational efficiency is improved.

CN116302551BActive Publication Date: 2026-11-10ARM TECH CHINA CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310309617.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-27
Publication Date
2026-11-10
Estimated Expiration
2043-03-27

AI Technical Summary

Technical Problem

Because recurrent neural networks and other neural network models have high operator fusion, the input data at each time step is the output data of the previous time step, which makes it impossible to directly divide the operators of the model, affecting the computational efficiency.

Method used

The data to be processed and the operators of the neural network model are divided into multiple groups of data to be processed and groups of sub-operators. Each sub-operator corresponds to a processing unit, and the calculation of each sub-operator is processed in parallel. Finally, the results are combined and output.

Benefits of technology

This improves the computational efficiency of neural network models, enabling multiple processing units to execute the operation logic of sub-operators in parallel, thereby enhancing overall computational performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116302551B_ABST
    Figure CN116302551B_ABST
Patent Text Reader

Abstract

This application relates to the field of artificial intelligence technology and discloses a model data processing method, a readable medium, and an electronic device. The method includes: determining attribute information of the data to be processed input to a neural network model, wherein the attribute information includes the number of data in a first dimension and the number of data in a second dimension; determining the number of processing units in the electronic device; determining the number of data groups to be processed based on the number of data in the first dimension and the number of processing units; determining the operators of the neural network model, and determining the number of operator groups based on the number of data groups to be processed, wherein each operator group includes at least one sub-operator, and each sub-operator corresponds one-to-one with a processing unit; determining the data groups to be processed input to each operator group; and merging the output data of each operator group to obtain the processing result of the data to be processed. In this method, the processing units corresponding to each sub-operator process the calculations of each sub-operator in parallel, improving computational efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model data processing method, a readable medium, and an electronic device. Background Technology

[0002] With the development of artificial intelligence (AI) technology, neural network models are becoming increasingly complex. During the computation of neural network models, when the amount of data to be processed is large, electronic devices typically divide the data into multiple data blocks and the neural network model into multiple operators. Each operator is executed by a different processing unit (CPU core). Each data block is input into its respective operator, and after obtaining the output data of each operator, the multiple output data are merged into the output data of the neural network model. The parallel execution of the computational logic of each operator by multiple processing units can improve the computational efficiency of the neural network model.

[0003] However, for neural network models such as recursive neural networks (RNNs) used to process data with temporal and / or spatial correlations, such as long short-term memory (LSTM) networks, the high fusion of operators means that the input data at each time step t is the output data of the previous time step t-1. This makes it impossible for the operators of the neural network model to be directly segmented, affecting the computational efficiency of the neural network model. Summary of the Invention

[0004] The purpose of this application is to provide a model data processing method, a readable medium, and an electronic device.

[0005] The first aspect of this application provides a model data processing method applied to an electronic device, wherein the model is capable of processing data that is correlated in time and / or spatial sequences. The method includes: determining attribute information of the data to be processed input to the model, wherein the attribute information includes the number of data in a first dimension M and the number of data in a second dimension L; determining the number of processing units N provided by the electronic device, wherein M, N, and L are all integers greater than 0, N is greater than 1, and at least one of M and L is greater than 1; and dividing the data to be processed into S groups of data to be processed based on the number of data in the first dimension M and the number of processing units N; wherein the number of data groups of data to be processed... The number of data points M and the number of processing units N in the first dimension are determined based on their relationship. The model's operators are determined, and the operators are divided into S operator groups based on the number of data sets to be processed. Each operator group includes at least one sub-operator, and the total number of sub-operators is less than or equal to the number of processing units; each sub-operator corresponds one-to-one with a processing unit. The data sets to be processed are determined as inputs to each operator group, such that each sub-operator in each operator group processes the data at the corresponding time step within its respective data set at each time step. The output data of each sub-operator are merged, and the merged result is used as the processing result for the data to be processed.

[0006] In one possible implementation of the first aspect above, the data to be processed is divided into S groups of data to be processed based on the number of data M in the first dimension and the number of processing units N, including: if the number of data M in the first dimension is greater than the number of processing units N, the number of data groups S is determined to be equal to the number of processing units N, and the data to be processed is divided into N groups of data to be processed in the data block dimension; if the number of data M in the first dimension is less than or equal to the number of processing units N, the number of data groups S is determined to be equal to the number of data M in the first dimension, and the data to be processed is divided into M groups of data to be processed in the data block dimension.

[0007] In one possible implementation of the first aspect above, the operators of the model are determined, and the operators of the model are divided into S operator groups based on the number of data groups to be processed, including: determining the number P of sub-operators in each operator group based on the number of operator groups S and the number of processing units N, where P is an integer greater than 0 and less than or equal to N; and dividing the operators of the model based on the number of sub-operators in each operator group to determine each sub-operator in each operator group.

[0008] In one possible implementation of the first aspect above, the operators of the model are segmented based on the number of sub-operators in each operator group to determine each sub-operator in each operator group, including: quantizing the operators of the model to obtain quantized operators; expanding the quantized operators in the time step dimension to obtain L operators with a time step of 1; and dividing the output dimension of any operator with a time step of 1 into P equal parts to obtain sub-operators.

[0009] In one possible implementation of the first aspect above, determining the data set to be processed for each input operator group includes: determining the i-th data set to be processed for the input i-th operator group, where i is an integer greater than 0 and less than or equal to S; dividing the i-th data set to be processed into L sub-data sets to be processed along the time step dimension; and determining the input data of the P sub-operators in the i-th operator group at the j-th time step as the j-th sub-data set to be processed, where j is an integer greater than 0 and less than or equal to L.

[0010] In one possible implementation of the first aspect above, corresponding to j being greater than 1, the input data of the P sub-operators in the i-th operator group at the j-th time step also includes: the output data of the P sub-operators in the i-th operator group at the (j-1)-th time step; the output data includes hidden state information and unit state information.

[0011] In one possible implementation of the first aspect above, the output data of each sub-operator are merged, and the merged result is used as the processing result of the data to be processed, including: merging the output data of P sub-operators in the S operator groups at the Lth time step to obtain the processing results of the S data groups to be processed; merging the processing results of the S data groups to be processed to obtain the processing result of the data to be processed.

[0012] In one possible implementation of the first aspect above, the method further includes: storing the hidden state information output by the P sub-operators at L time steps into a first set, and outputting each hidden state information in the first set; and / or storing the unit state information output by the P sub-operators at L time steps into a second set, and outputting each unit state information in the second set.

[0013] A second aspect of this application provides a readable medium storing instructions that, when executed on an electronic device, cause the electronic device to perform any of the methods described in the first aspect above.

[0014] A third aspect of this application provides an electronic device, comprising: a memory for storing instructions executable by one or more processors of the electronic device, and a processor, one of the processors of the electronic device, for performing any of the methods described in the first aspect above. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 A schematic diagram of the structure of a neural network unit of an LSTM network is shown according to an embodiment of this application;

[0017] Figure 2 A schematic diagram of an image recognition scenario is shown according to an embodiment of this application;

[0018] Figure 3 A schematic flowchart of a model data processing method is shown according to an embodiment of this application;

[0019] Figure 4 A schematic diagram of a data processing procedure is shown according to an embodiment of this application;

[0020] Figure 5 An embodiment of this application illustrates a schematic diagram of data segmentation to be processed;

[0021] Figure 6 A schematic diagram of an expanded data set to be processed is shown according to an embodiment of this application;

[0022] Figure 7 An embodiment of this application illustrates a schematic diagram of another data processing procedure;

[0023] Figure 8 A schematic diagram illustrating the processing procedure for a group of data to be processed is shown according to an embodiment of this application;

[0024] Figure 9 A schematic diagram of the computation flow of a BasicLSTM operator is shown according to an embodiment of this application;

[0025] Figure 10 A schematic diagram of the structure of an electronic device is shown according to some embodiments of this application. Detailed Implementation

[0026] The illustrative embodiments of this application include, but are not limited to, a model data processing method, a readable medium, and an electronic device.

[0027] To more clearly illustrate the solutions of the embodiments of this application, some contents involved in the embodiments of this application will be explained below.

[0028] (1) LSTM network model

[0029] LSTM (Laser-Sensitive Memory) networks are a special type of RNN (Recurrent Neural Network). In LSTM networks, the state and output of each neural network unit can be passed between hidden layers; that is, the output of each neural network unit is related to the input of the input layer, the output of the previous neural network unit, and the state of the previous neural network unit. Neural network units can filter input data through gate structures, i.e., forgetting unimportant information and remembering important information. These gate structures include input gates, forget gates, and output gates. For example, Figure 1 A schematic diagram of the structure of a neural network unit in an LSTM network is shown according to an embodiment of this application. The neural network unit 10 corresponding to the t-th time step (time t) of the LSTM network model includes: an input gate 11, a forget gate 12, a unit state 13, and an output gate 14.

[0030] Input gate 11 is used to update the cell state. First, the hidden state information h from the previous neural network unit is input... t-1 and the input information x of the current neural network unit t The hidden state information ht-1 of the previous neural network unit and the input information xt of the current neural network unit are passed to the tanh function to determine the candidate vector. Finally, the output value of the sigmoid function is multiplied by the output value of the tanh function, where the output value of the sigmoid function is used to determine which information in the output value of the tanh function is important and needs to be retained. For example, refer to formulas (1) and (2):

[0031] i t =σ(W xi x t +W hi h t-1 +b i (1)

[0032] g t =ψ(W xg x t +W hg h t-1 +b g (2)

[0033] Among them, i t Let W represent the input vector at time step t, σ represent the sigmoid function, and W represent the input vector at time step t. xi The weights of the input information of the input gate, x t W represents the input information at time step t. hi The weights h represent the hidden state information of the input gate. t-1b represents the hidden state information at time step t-1. i G represents the error coefficient of the input gate. t Let W represent the candidate vector, ψ represent the tanh function, and W represent the candidate vector. xg W represents the weights of the candidate input information. hg The weights, b, represent the hidden state information of the candidates. g This represents the error coefficient of the candidate.

[0034] Forget gate 12 is used to determine whether to discard or retain information. It stores the hidden state information h from the previous neural network unit. t-1 and the input information x of the current neural network unit t The result is passed to the sigmoid function. The output value of the sigmoid function is between 0 and 1; the closer to 0, the more likely it should be discarded, and the closer to 1, the more likely it should be retained. For example, refer to formula (3):

[0035] f t =σ(W xf x t +W hf h t-1 +b f (3)

[0036] Among them, f t Let W represent the forgetting vector at time step t, σ represent the sigmoid function, and W represent the forgetting vector at time step t. xf The weights of the input information for the forget gate, x t W represents the input information at time step t. hf h represents the weight of the hidden state information of the forget gate. t-1 b represents the hidden state information at time step t-1. f This represents the error coefficient of the forget gate.

[0037] Unit state 13 is used to multiply the unit state of the previous neural network unit by the forget vector point by point. If the unit state is multiplied by a value close to 0, it means that this information needs to be discarded in the unit state at the current time step. Then, the value of the unit state of the previous neural network unit multiplied by the forget vector is added to the input vector point by point, and the result is updated to the current unit state to obtain the unit state of the current neural network unit. For example, refer to formula (4):

[0038] C t =f t *C t-1 +i t *g t (4)

[0039] Among them, C t f represents the cell state information at time step t.t C represents the forgetting vector at time step t. t-1 This represents the cell state information at time step t-1, i t G represents the input vector at time step t. t This represents the candidate vector for time step t, and * indicates point-by-point multiplication.

[0040] Output gate 14 is used to determine the hidden state information of the current neural network unit. The hidden state information contains the input information of the current neural network unit. First, the hidden state information h of the previous neural network unit is processed. t-1 and the input information x of the current neural network unit t The hidden state information is passed to the tanh function. Then, the output value of the tanh function is multiplied by the output value of the sigmoid function to determine the hidden state information of the current neural network unit. Finally, the hidden state information of the current neural network unit is used as the output of the current neural network unit, and the unit state information and the hidden state information of the current neural network unit are passed to the next neural network unit. For example, refer to formulas (5) and (6):

[0041] o t =σ(W xo x t +W hO h t-1 +b o (5)

[0042] h t =o t *ψ(C t (6)

[0043] Among them, o t Let W represent the output vector at time step t, σ represent the sigmoid function, and W represent the output vector at time step t. xo The weights of the input information of the output gate, x t W represents the input information at time step t. ho h represents the weights of the hidden state information of the output gate. t-1 b represents the hidden state information at time step t-1. o h represents the error coefficient of the forget gate. t Let ψ represent the hidden state information at time step t, and let C represent the tanh function. t This represents the cell state information at time step t.

[0044] (2) Operator

[0045] The computational units that make up a neural network model are called operators (Ops). In the operations of a neural network model, operators correspond to the computational logic within a neural network layer. Operators in a neural network model can be further classified according to their operation type: linear operators and nonlinear operators.

[0046] Nonlinear operators: also known as nonlinear functions or nonlinear mappings, are operators that do not satisfy linear conditions.

[0047] Linear operators: These are linear functions that satisfy the linear condition.

[0048] For example, with Figure 1 Taking the neural network unit 10 of the LSTM network shown as an example, in formula (1), W xi x t +W hi h t-1 +b i It can be an operator, and W xi x t +W hi h t-1 +b i It is a linear operator, W xi x t It is also a linear operator, W hi h t-1 It is also a linear operator. σ(W) xi x t +W hi h t-1 +b i ) is a nonlinear operator, and σ() is a nonlinear operator.

[0049] As mentioned earlier, during the computation of a neural network model, when the amount of data to be processed is large, electronic devices typically divide the data into multiple data blocks and the neural network model into multiple operators. Each operator is executed by a different processing unit. Each data block is input into its respective operator, and after obtaining the output data of each operator, the multiple output data are merged into the output data of the neural network model. The parallel execution of the computational logic of each operator by multiple processing units can improve the computational efficiency of the neural network model.

[0050] For example, Figure 2In the image recognition scenario shown, after the image data to be recognized is input into the neural network model, n processing units (processing unit 1, processing unit 2, ..., processing unit n) simultaneously execute the operation logic of n operators (operator 01, operator 02, ..., operator 0n) on n data blocks (data block 001, data block 002, ..., data block 00n) to obtain the output data of n operators. The output data of n operators are then merged to obtain the image data recognition result.

[0051] However, due to the high fusion of operators in neural networks such as LSTM, that is, the input data of each time step t is the output data of the previous time step t-1, the operators of the neural network model cannot be directly segmented, which affects the computational efficiency of the neural network model.

[0052] Furthermore, the image recognition scenario described above is merely an illustrative example. In other scenarios, such as speech recognition, the data input to the neural network model is speech data to be recognized, which includes multiple frames of speech segments (speech segments in multiple consecutive time intervals); in text generation, the data input to the neural network model is text data to be generated, which includes multiple phrases or sentences that have contextual relationships (spatial sequence relationships). This application embodiment does not limit the specific application scenario of the neural network model.

[0053] To address the aforementioned technical problems, this application proposes a model data processing method, which includes:

[0054] First, determine the attribute information of the data to be processed input to the neural network model. This attribute information includes the number of data points M in the first dimension and L in the second dimension. Determine the number of processing units N in the electronic device, where M, N, and L are all integers of 0, N is greater than 1, and at least one of M or L is greater than 1. Second, based on the relationship between the number of data points M in the first dimension and the number of processing units N, divide the data into S groups. Determine the operators of the neural network model, and based on the number of data groups, divide the operators into S operator groups. Each operator group includes at least one sub-operator, and the total number of sub-operators is less than or equal to the number of processing units. Each sub-operator corresponds one-to-one with a processing unit. This means that each processing unit is used to execute the calculation of only one sub-operator, and the calculation of each sub-operator is performed by only one processing unit.

[0055] Then, the data set to be processed for each input operator group is determined, so that each sub-operator in each operator group processes the data of the corresponding time step in the corresponding data set to be processed at each time step.

[0056] Finally, the output data of each sub-operator are merged, and the merged result is used as the processing result of the data to be processed.

[0057] For example, in some embodiments, the video data to be processed input to the LSTM model includes multiple layers (as an instance of the first dimension, i.e., the first dimension refers to the number of layers) of video data with a preset time length (as an instance of the second dimension, i.e., the second dimension refers to the number of time steps). For example, assuming the video data to be processed includes 3 layers and a time length of 20 seconds, the electronic device provides 6 processing units to process the video data to be processed.

[0058] First, the electronic device determines that the number of data blocks M is 3, the number of time steps L is 20, and the number of processing units N is 6.

[0059] Secondly, the electronic device determines that M is less than N, and sets the number of data groups to be processed to 3 (M), meaning each data group includes one data block (processing data from one layer in the video data). It determines that the LSTM model operator is the BasicLSTM operator, and divides the LSTM model operator into 3 operator groups. Each operator group includes 2 (i.e., N / M = 6 / 3 = 2) sub-operators, and the sub-operator is BasicLSTM / 2. It can be understood that there are a total of 6 sub-operators in the 3 operator groups, and the calculation of these 6 sub-operators is performed by 6 processing units provided by the electronic device.

[0060] Then, the electronic device determines that the data input to each operator group is the data to be processed (the data of each layer of the video data to be processed), and determines that the data input to each sub-operator is the data to be processed corresponding to the operator group. Each sub-operator processes the data of the corresponding time step in the data to be processed in the data to be processed at each time step.

[0061] Finally, the output data of each sub-operator in each operator group are merged to obtain the output data of each operator group. The merged output data of each operator group is then used as the processing result of the data to be processed.

[0062] Or, for example, in other embodiments, suppose the video data to be processed input to the LSTM model includes 10 layers and a duration of 20 seconds, and the electronic device provides 5 processing units to process the video data to be processed.

[0063] First, the electronic device determines that the number of data blocks M is 10, the number of time steps L is 20, and the number of processing units N is 5.

[0064] Secondly, the electronic device determines that M is greater than N, and sets the number of data groups to be processed to 5 (N), meaning each data group includes 2 data blocks (processing data from 2 layers in the video data); it determines that the operator of the LSTM model is the BasicLSTM operator, and divides the LSTM model operator into 5 operator groups, each operator group including 1 sub-operator, which is also a BasicLSTM. It can be understood that there are a total of 5 sub-operators in the 5 operator groups, and the calculation of the 5 sub-operators is performed by 5 processing units provided by the electronic device.

[0065] Then, the electronic device determines that the data input to each operator group is the data group to be processed, and determines that the data input to each sub-operator is the data group to be processed corresponding to the operator group. Each sub-operator processes the data of the corresponding time step in the data group to be processed at each time step.

[0066] Finally, the output data of each sub-operator are merged, and the merged result is used as the processing result of the data to be processed.

[0067] It is understood that the above-described video data processing scenario is merely an illustrative example. In other scenarios, such as speech recognition, the data to be processed is speech data to be recognized. This speech data comprises multiple different frequencies (as an example corresponding to the first dimension, i.e., the first dimension refers to the number of frequencies of speech) with a preset time length (as an example corresponding to the second dimension, i.e., the second dimension refers to the number of time steps). For example, it may include audio data with 10 frequencies and a time length of 30 seconds. It is understood that the number of data blocks in this speech data to be recognized is the same as the number of frequencies, both being 10, and the number of time steps is 30.

[0068] For example, in a text generation scenario, the data input to the neural network model is text data, which includes multiple text blocks (as instances corresponding to the first dimension, i.e., the first dimension refers to the number of text blocks) and sub-blocks (as instances corresponding to the second dimension, i.e., the second dimension refers to the number of sub-blocks) of phrases, sentences, etc., of the paragraph to be generated. For example, the text data includes 3 sentences, and each sentence includes 8 phrases. It can be understood that the number of data blocks in this text data is the same as the number of text blocks, both being 3, and the number of time steps is the same as the number of sub-blocks, both being 8.

[0069] The model data processing method provided in this application divides the data to be processed and the operators of the neural network model into multiple data groups and sub-operators based on the attribute information (number of data blocks) of the data to be processed input to the neural network model and the number of processing units provided by the electronic device. This allows the processing units corresponding to each sub-operator to process the calculation of each sub-operator in parallel, thereby improving the computational efficiency of the neural network model.

[0070] In some embodiments, the electronic device determines the number of data blocks M and the number of time steps L of the data to be processed input to the neural network model, as well as the number of processing units N of the electronic device; based on the number of data blocks M and the number of processing units N, it determines the number of data groups S to be processed, and divides the BasicLSTM operator of the LSTM model into S operator groups, each operator group including the number of sub-operators P, and each sub-operator is a BasicLSTM / P operator; wherein, the number of sub-operators P can be determined by floor(N / S), where floor is a floor operation. Specifically, when the number of data blocks M is greater than or equal to the number of processing units N, the electronic device determines that the number of data groups S to be processed is equal to N, and divides the LSTM model operator into N operator groups; when the number of data blocks M is less than the number of processing units N, the electronic device determines that the number of data groups S to be processed is equal to M, and divides the LSTM model operator into M operator groups.

[0071] In some embodiments, corresponding to each time step, each sub-operator obtains the sub-data to be processed in the current time step of the corresponding data group to be processed, as well as the output data of the sub-operator in the previous time step, and performs calculations on the sub-data to be processed in the current time step until the calculations of all sub-data to be processed in the data group to be processed are completed. The output data of each sub-operator in the Lth time step are then merged to obtain the processing result of the data to be processed. It can be understood that each data group to be processed includes L sub-data to be processed.

[0072] It is understood that the model data processing method provided in this application can be applied to any electronic device, including multi-core processors, capable of running neural network models, including but not limited to mobile phones, wearable devices (such as smartwatches), tablets, desktops, laptops, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, etc. This application does not limit the specific type of electronic device. The models include, but are not limited to, LSTM networks, gated recurrent unit (GRU) networks, and other neural network models capable of processing data that are correlated in time and / or spatial sequences.

[0073] To facilitate understanding of the technical solutions of the embodiments of this application, the following describes the model data processing method provided in the embodiments of this application, using an LSTM network as the neural network model, a data block dimension as the first dimension, a time step dimension as the second dimension, and an electronic device 100 as an example, in conjunction with the accompanying drawings.

[0074] In this application, LSTM networks are explained as an example of models capable of processing data that are correlated in time and / or spatial sequences. It is understood that other models capable of processing data that are correlated in time and / or spatial sequences, such as gated recurrent unit (GRU) networks, recursive neural networks (RNNs), and convolutional neural networks (CNNs), can also be applied to the technical solutions of the embodiments of this application.

[0075] Figure 3 A flowchart illustrating a model data processing method is shown according to an embodiment of this application. It can be understood that... Figure 3 The electronic device 100 is the primary entity executing each step of the process shown. For simplicity, the following description... Figure 3 The execution entity for each step in the illustrated process will not be described again. For example... Figure 3 As shown, the process includes, but is not limited to, the following steps:

[0076] S301: Determine the attribute information of the data to be processed input into the neural network model and the number of processing units provided by the electronic device.

[0077] In some embodiments, the electronic device 100 determines the data to be processed input to the neural network model, the attribute information of the data to be processed, and the number N of processing units provided by the electronic device 100. The attribute information includes the number M of data blocks and the number L of time steps of the data to be processed.

[0078] In other embodiments, the attribute information of the data to be processed may also include the number of hidden neurons.

[0079] It is understood that in some embodiments, the attribute information [Batch, TimeStep, CellSize] of the data to be processed can be represented as [M, N, CellSize]. Here, Batch represents the number of data blocks, TimeStep represents the number of time steps, and CellSize represents the number of hidden neurons.

[0080] S302: Based on the number of data blocks M and the number of processing units N, the data to be processed is divided into S groups of data to be processed.

[0081] In some embodiments, the electronic device 100 determines the number S of data groups to be processed based on the relationship between the number of data blocks M and the number of processing units N, and divides the data to be processed into S data groups to be processed in the data block dimension. It can be understood that the attribute information of each data group to be processed can be represented as [M / S, L, CellSize].

[0082] In some embodiments, corresponding to a data block number M being greater than or equal to the number of processing units N, the electronic device determines that the number of data groups to be processed S is equal to the number of processing units N, and divides the data to be processed into N data groups to be processed in the data block dimension. The attribute information of each data group to be processed can be represented as [M / N, L, CellSize].

[0083] In other embodiments, corresponding to the number of data blocks M being less than the number of processing units N, the number of data groups to be processed S is determined to be equal to the number of data blocks M, and the data to be processed is divided into M data groups to be processed in the data block dimension. The attribute information of each data group to be processed can be represented as [1, L, CellSize].

[0084] S303: Operators that determine the neural network model.

[0085] In some embodiments, the electronic device 100 determines the operators of the neural network model, wherein the operators of the neural network model can represent a merged operator formed by fusing the basic operators of the neural network model.

[0086] For example, the basic operators of an LSTM network may include the above formulas (1) to (6). The basic operators are fused to determine the merged operator BasicLSTM operator. The calculation process of the BasicLSTM operator will be described in detail below, and will not be repeated here.

[0087] S304: Divide the operators of the neural network model into S operator groups based on the number of data groups to be processed.

[0088] In some embodiments, the electronic device 100 divides the operators of the neural network model into S operator groups based on the number S of data groups to be processed. It can be understood that the number of operator groups of the neural network model is equal to the number of data groups to be processed, and there is a one-to-one correspondence between the data groups to be processed and the operator groups, that is, one operator group processes only one data group to be processed, and one data group to be processed is processed by only one operator group.

[0089] In some embodiments, each operator group includes P sub-operators respectively, and the number P of the sub-operators may be determined by floor(N / S), where floor is a floor operation. The total number of the sub-operators (P*S) is less than or equal to the number (N) of processing units, and the sub-operators are in one-to-one correspondence with the processing units. It can be understood that each processing unit is only configured to perform the calculation of one sub-operator, and the calculation of each sub-operator is performed by only one processing unit.

[0090] In some embodiments, corresponding to that the number S of to-be-processed data groups is equal to the number N of processing units, the electronic device 100 determines that the number of operator groups is N. Each of the N operator groups includes 1 sub-operator (that is, P=floor(N / S)=floor(N / N)=1), and each sub-operator is a BasicLSTM operator. It can be understood that the total number of the sub-operators is N (P=1, S=N, P*S=1*N=N), and the total number of the sub-operators is equal to the number of processing units provided by the electronic device 100. The N sub-operators are in one-to-one correspondence with the N processing units, and the calculation of each sub-operator is performed by the corresponding processing unit, that is, the N processing units perform the calculation of the N sub-operators in parallel (synchronously).

[0091] It can be understood that the operation logic of the BasicLSTM operator is represented by formula (1) to formula (6), which will not be repeated here.

[0092] In other embodiments, corresponding to that the number S of to-be-processed data groups is equal to M, the electronic device 100 determines that the number of operator groups is M. Each of the M operator groups includes P sub-operators, and each sub-operator is a BasicLSTM / P operator. The total number of the sub-operators is P=floor(N / M). Since N is greater than M, floor(N / M) is less than N / M, that is, P is less than N / M. It can be understood that the total number of the sub-operators is less than the number N of processing units provided by the electronic device 100 (P=floor(N / M), S=M, P*S=floor(N / M)*M<N / M*M=N). The P sub-operators are in one-to-one correspondence with the P processing units, and the calculation of each sub-operator is performed by the corresponding processing unit, that is, the P processing units perform the calculation of the P sub-operators in parallel.

[0093] In some embodiments, the BasicLSTM / P operator may be determined in the following manner: first, quantizing the BasicLSTM operator; then, expanding the BasicLSTM operator in the time step dimension to obtain L BasicLSTM operators each with a time step of 1; finally, equally dividing the output dimension of any BasicLSTM operator with a time step of 1 into P parts, to obtain the BasicLSTM / P operator.

[0094] The operational logic of the BasicLSTM / P operator can be found in formulas (7) to (12).

[0095]

[0096] Among them, i t Let W represent the input vector at time step t, σ represent the sigmoid function, and W represent the input vector at time step t. xi The weights of the input information of the input gate, x t W represents the input information at time step t. hi The weights h represent the hidden state information of the input gate. t-1 b represents the hidden state information at time step t-1. i G represents the error coefficient of the input gate. t Let W represent the candidate vector, ψ represent the tanh function, and W represent the candidate vector. xg W represents the weights of the candidate input information. hg The weights, b, represent the hidden state information of the candidates. g f represents the error coefficient of the candidate. t W represents the forgetting vector at time step t. xf W represents the weight of the input information for the forget gate. hf The weight of the hidden state information of the forget gate, b f C represents the error coefficient of the forget gate. t C represents the cell state information at time step t. t-1 This represents the cell state information at time step t-1, where * indicates point-by-point multiplication, and o t W represents the output vector at time step t. xo W represents the weight of the input information of the output gate. ho The weight b represents the hidden state information of the output gate. o h represents the error coefficient of the forget gate. t This represents the hidden state information at time step t, and P represents the number of sub-operators in each operator group.

[0097] S305: Determine the data groups to be processed for each input operator group.

[0098] In some embodiments, the electronic device 100 determines the data sets to be processed that are input to each operator group; that is, one data set to be processed is input to one operator group. It can be understood that there is a one-to-one correspondence between the operator group and the data set to be processed, that is, the input data of the i-th operator group is the i-th data set to be processed, where i is an integer greater than 0 and less than or equal to S.

[0099] In some embodiments, the electronic device 100 divides each group of data to be processed into L sub-data groups along the time step dimension; the input data of each sub-operator at each time step is the sub-data to be processed at the corresponding time step. That is, the input data of P sub-operators in the i-th operator group at the j-th time step is the j-th sub-data to be processed, where i is an integer greater than 0 and less than or equal to S, and j is an integer greater than 0 and less than or equal to L. If j is greater than 1 (not the first time step), the input data of each sub-operator also includes the output data of each sub-operator at the previous time step (the (j-1)-th time step). The output data of each sub-operator at each time step includes the hidden state information and the unit state information of the corresponding time step.

[0100] It can be understood that the input data of the sub-operators in each operator group is the input data of the corresponding operator group. In each operator group, each sub-operator processes the data of the corresponding time step in the corresponding data group to be processed at each time step.

[0101] S306: Merge the output data of each sub-operator and use the merged result as the processing result of the data to be processed.

[0102] In some embodiments, the electronic device 100 first merges the output data of P sub-operators in each operator group to obtain the processing results of S data groups to be processed; then it merges the processing results of the S data groups to be processed to obtain the processing result of the data to be processed.

[0103] In some embodiments, corresponding to N operator groups, each operator group includes one sub-operator, and the electronic device 100 merges (concats) the output data of the N sub-operators to obtain the processing result of the data to be processed.

[0104] In some embodiments, corresponding to M operator groups, each operator group includes P sub-operators. The electronic device 100 first merges the output data of the P sub-operators in each operator group to obtain the processing results of M data groups to be processed; then it merges the processing results of the M data groups to be processed to obtain the processing result of the data to be processed.

[0105] In some embodiments, the calculation of each sub-operator includes calculations over L time steps. The calculation at the j-th time step includes: each sub-operator acquiring the data at the j-th time step of the corresponding data group to be processed (the j-th sub-data to be processed), and the output data of the sub-operator at the (j-1)-th time step, and performing calculations on the data at the j-th time step. Here, j is an integer greater than 0 and less than L. It can be understood that when j is 1 (the first time step), each sub-operator acquires the data at the j-th time step of the corresponding data group to be processed and performs calculations on the data at the j-th time step.

[0106] It can be understood that the processing result of the S groups of data to be processed is the combined result of the output data of the P sub-operators in the S groups of operators at the Lth time step. The output data includes hidden state information and cell state information. The processing result of the data to be processed is the combined result of the processing results of the S groups of data to be processed.

[0107] In some embodiments, if the output mode of the operator (operator before segmentation) of the neural network model further includes continuously outputting the hidden state information of each time step, then the hidden state information h of the first L time steps of each sub-operator is... i Store each hidden state in the corresponding set (H_ALL) and output the hidden state information in H_ALL.

[0108] In some embodiments, if the output mode of the operator of the neural network model further includes continuously outputting the unit state information of each time step, then the unit state information C of each sub-operator at L time steps is... i Store the data into the corresponding set (C_ALL) and output the status information of each cell in C_ALL.

[0109] Therefore, the model data processing method provided in this application, based on the number of data blocks and time steps of the data to be processed input to the neural network model, and the number of processing units provided by the electronic device, divides the data to be processed and the operators of the neural network model into multiple data groups to be processed and sub-operators. Each sub-operator processes the data of the corresponding time step of the data group to be processed at each time step, so that the processing units corresponding to each sub-operator process the calculation of each sub-operator in parallel, thereby improving the computational efficiency of the neural network model. The following is combined with... Figures 4 to 8 The technical solutions of the embodiments of this application will be further described.

[0110] Electronic device 100 determines the number of data blocks M, the number of time steps L, and the number of processing units N.

[0111] In some embodiments, if the electronic device determines that M is greater than N, refer to Figure 4 The diagram shows the processing procedure.

[0112] First, the electronic device 100 determines that the number of data groups to be processed is N, and groups the data to be processed into N data groups to be processed (the first data group to be processed to the Nth data group).

[0113] For data splitting to be processed, please refer to... Figure 5 As shown. Assuming the data to be processed is image information, the electronic device 100 expands the data to be processed in the data block dimension, obtaining N groups of data to be processed, such as... Figure 5The first data group to be processed is shown in the diagram, followed by the Nth data group. Each data group contains L rows of pixels (data from L time steps), as shown below. Figure 5 The first row of pixels, the second row of pixels, the third row of pixels, ..., the Lth row of pixels are shown in the diagram.

[0114] Electronic device 100 expands the data groups to be processed, obtaining N data groups and L time steps of data.

[0115] For expanding the data set to be processed, please refer to... Figure 6 Assuming the data to be processed is image information, electronic device 100 expands the data in the time step dimension to obtain data for L time steps (L rows of pixels), such as... Figure 6 The first row of pixels, the second row of pixels, ..., the Lth row of pixels are shown in the diagram.

[0116] Secondly, the electronic device 100 determines that the operator of the LSTM model is a BasicLSTM operator and divides the operator of the LSTM model into N operator groups. Each operator group includes one sub-operator, which is a BasicLSTM operator. There are a total of N sub-operators in the N operator groups. The calculation of the N sub-operators is executed by N processing units provided by the electronic device. That is, processing unit 1 executes the calculation of BasicLSTM operator 1, and so on, processing unit N executes the calculation of BasicLSTM operator N.

[0117] Then, the electronic device 100 determines that the data input to the BasicLSTM operator 1 is the first data to be processed. The BasicLSTM operator 1 processes the data of the corresponding time step in the data to be processed group at each time step, and so on. The data input to the BasicLSTM operator N is the Nth data to be processed, and the BasicLSTM operator N processes the data of the corresponding time step in the data to be processed group at each time step.

[0118] Finally, the electronic device 100 merges the output data of each sub-operator and uses the merged result as the processing result of the data to be processed.

[0119] In some embodiments, if the electronic device determines that M is less than or equal to N, refer to Figure 7 The diagram shows the processing procedure.

[0120] Electronic device 100 determines the number of data groups to be processed to be M, and groups the data to be processed to obtain M data groups to be processed (the first data group to be processed to the Mth data group); the processing results of the M data groups to be processed are merged to obtain the processing result of the data to be processed.

[0121] The following is combined Figure 8This section describes the processing procedure for the i-th data group out of M data groups to be processed. Here, i is an integer greater than 0 and less than or equal to M.

[0122] First, the electronic device 100 expands the data groups to be processed, obtaining data from L time steps for each of the N data groups. The expansion of the data groups can be referenced from... Figure 6 Assuming the data to be processed is image information, the data to be processed is expanded along the time step dimension to obtain data for L time steps, and the data for each time step includes the pixels of each row.

[0123] Secondly, the electronic device 100 determines that the operator of the LSTM model is the BasicLSTM operator, and divides the operator of the LSTM model into M operator groups, each operator group including P sub-operators, which are BasicLSTM / P operators. The calculation of the P sub-operators is executed by P processing units provided by the electronic device, that is, processing unit 1 executes the calculation of BasicLSTM sub-operator 1, and so on, processing unit P executes the calculation of BasicLSTM sub-operator P.

[0124] Then, the electronic device 100 determines that the data input to the BasicLSTM sub-operator 1 is the first data to be processed. The BasicLSTM sub-operator 1 processes the data of the corresponding time step in the data to be processed group at each time step. Similarly, the data input to the BasicLSTM sub-operator P is the Pth data to be processed. The BasicLSTM sub-operator P processes the data of the corresponding time step in the data to be processed group at each time step.

[0125] Finally, the electronic device 100 merges the output data of each sub-operator and uses the merged result as the processing result of the i-th data group to be processed.

[0126] It is understood that in other embodiments, the above... Figure 3 The steps shown can be combined, deleted, or replaced with other steps that are more conducive to achieving the purpose of this application. For example, steps S303 and S304 can be combined into one step. This application does not impose any restrictions on this.

[0127] The following is combined with Figure 9 A detailed introduction to the computation process of BasicLSTM operators. Figure 9 A schematic diagram of the computation flow of a BasicLSTM operator is shown according to an embodiment of this application. It can be understood that... Figure 9 The electronic device 100 is the primary entity executing each step of the process shown. For simplicity, the following description... Figure 9 The execution entity for each step in the illustrated process will not be described again. For example... Figure 9As shown, the process includes, but is not limited to, the following steps:

[0128] S901: Obtain input data.

[0129] In some embodiments, the electronic device 100 acquires input data of the BasicLSTM operator and determines the number of time steps j of the input data.

[0130] S902: Determine if the number of time steps is greater than 0.

[0131] In some embodiments, if the determination result is yes, that is, the number of time steps is greater than 0, it means that the input data of the BasicLSTM operator has not been calculated and needs to continue to be calculated. Then, step S403 is executed to determine the input sub-data of the current time step.

[0132] In other embodiments, if the determination result is negative, that is, the number of time steps is less than or equal to 0, it means that the input data of the BasicLSTM operator has been calculated and the output data of the BasicLSTM operator needs to be determined. Then, step S406 is executed to determine the output data.

[0133] S903: Determine the input data for the current time step.

[0134] In some embodiments, the first time step corresponds to the current time step, and the electronic device 100 acquires the data of the current time step from the input data as the input data of the current time step.

[0135] In other embodiments, when the current time step is not the first time step, the electronic device 100 acquires the data of the current time step and the output data of the previous time step from the input data as the input data of the current time step.

[0136] S904: Executes the BasicLSTM operator's operation logic on the input data of the current time step.

[0137] In some embodiments, the electronic device 100 performs the operation logic of the BasicLSTM operator on the input data of the current time step. The BasicLSTM operator can be referred to in formulas (1) to (6) above, and will not be described in detail here.

[0138] S905: Determine the output data for the current time step.

[0139] In some embodiments, the electronic device 100 determines the output data of the BasicLSTM operator at the current time step, decrements the number of time steps by 1, and continues to execute step S402: determining whether the number of time steps is greater than 0.

[0140] S906: Determine the output data.

[0141] In some embodiments, the electronic device 100 determines that the input data (data set to be processed) of each operator group has been calculated, and uses the last output data of the BasicLSTM operator as the output data of the BasicLSTM operator. Corresponding to each time step, each sub-operator obtains the data of the current time step of the corresponding data set to be processed and the output data of the sub-operator in the previous time step, calculates the data of the current time step, until the calculation of the data of all time steps of the data set to be processed is completed, and merges the output data of each sub-operator to obtain the processing result of the data to be processed.

[0142] In summary, the model data processing method provided in this application divides the data to be processed and the operators of the neural network model into multiple data groups and sub-operators based on the attribute information (number of data blocks and number of time steps) of the data to be processed input to the neural network model and the number of processing units provided by the electronic device. This allows the processing units corresponding to each sub-operator to process the calculation of each sub-operator in parallel, thereby improving the computational efficiency of the neural network model.

[0143] Furthermore, Figure 10 A schematic diagram of the structure of an electronic device 100 is shown according to some embodiments of this application. For example... Figure 10 As shown, the electronic device 100 includes one or more processors 101, system memory 102, non-volatile memory (NVM) 103, communication interface 104, input / output (I / O) devices 105, and system control logic 106 for coupling the processor 101, system memory 102, NVM 103, communication interface 104, and input / output (I / O) devices 105. Wherein:

[0144] The processor 101 can be used to control an electronic device to execute the model data processing method of this application, wherein the processor 101 may include multiple processing units.

[0145] System memory 102 is volatile memory, such as random-access memory (RAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc. System memory is used for temporary storage of data and / or instructions.

[0146] The non-volatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), compact disc (CD), digital versatile disc (DVD), solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 103 may also be a removable storage medium, such as a secure digital (SD) memory card, etc.

[0147] Specifically, system memory 102 and non-volatile memory 103 may each include a temporary copy and a permanent copy of instruction 107. Instruction 107 may include, when executed by processor 101, causing electronic device 100 to implement the model data processing method provided in the embodiments of this application.

[0148] The communication interface 104 may include a transceiver for providing a wired or wireless communication interface for the electronic device 100, thereby enabling communication with any other suitable device via one or more networks. In some embodiments, the communication interface 104 may be integrated into other components of the electronic device 100, for example, the communication interface 104 may be integrated into the processor 101. In some embodiments, the electronic device 100 may communicate with other devices through the communication interface 104; for example, the electronic device 100 may obtain a model data processing method to be executed from other electronic devices through the communication interface 104.

[0149] Input / output (I / O) device 105 may include input devices such as keyboard, mouse, etc., and output devices such as monitor, etc. Users can interact with electronic device 100 through input / output (I / O) device 105.

[0150] System control logic 106 may include any suitable interface controller to provide any suitable interface to other modules of electronic device 100. For example, in some embodiments, system control logic 106 may include one or more memory controllers to provide an interface to system memory 102 and non-volatile memory 103.

[0151] In some embodiments, at least one of the processors 101 may be packaged together with the logic of one or more controllers for system control logic 106 to form a system in package (SiP). In other embodiments, at least one of the processors 101 may also be integrated on the same chip with the logic of one or more controllers for system control logic 106 to form a system on chip (SoC).

[0152] Understandable. Figure 10 The structure of the electronic device 100 shown is merely an example. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine certain components, or divide certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0153] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0154] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0155] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0156] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0157] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0158] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0159] It should be noted that, in the examples and description of this patent, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0160] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A model data processing method applied to an electronic device, wherein the model is capable of processing data correlated in time and / or spatial sequences, characterized in that, The method includes: Determine the attribute information of the data to be processed input into the model, wherein the attribute information includes the number of data in the first dimension M and the number of data in the second dimension L, where the number of data in the first dimension M is the number of data blocks of the data to be processed and the number of data in the second dimension L is the number of time steps; Determine the number N of processing units provided by the electronic device, wherein the processing unit is the processor core of the electronic device, M, N, and L are all integers greater than 0, N is greater than 1, and at least one of M and L is greater than 1; Based on the amount of data M in the first dimension and the number of processing units N, the data to be processed is divided into S groups of data to be processed; wherein, the number of data groups to be processed is determined based on the relationship between the amount of data M in the first dimension and the number of processing units N. The operators of the model are determined, and the operators of the model are divided into S operator groups based on the number of data groups to be processed; wherein each operator group includes P sub-operators, and the total number of all sub-operators P*S is less than or equal to the number of processing units, the sub-operators correspond one-to-one with the processing units, the i-th operator group corresponds to the i-th data group to be processed, i is an integer greater than 0 and less than or equal to S, and P is an integer greater than 0 and less than or equal to N; Determine the data sets to be processed for each of the input operator groups, wherein the input data of the P sub-operators in the i-th operator group at the j-th time step is the j-th data set to be processed, where j is an integer greater than 0 and less than or equal to L, such that... In the i-th operator group, each sub-operator processes the j-th sub-data to be processed at the j-th time step; The output data of each of the sub-operators are merged, and the merged result is used as the processing result of the data to be processed; wherein... The data to be processed is divided into S groups based on the number of data M in the first dimension and the number of processing units N, including: If the number of data M corresponding to the first dimension is greater than the number of processing units N, the number of data groups to be processed S is determined to be equal to the number of processing units N, and the data to be processed is divided into N data groups to be processed in the data block dimension. If the number of data M corresponding to the first dimension is less than or equal to the number of processing units N, then the number S of the data groups to be processed is determined to be equal to the number of data M in the first dimension. The data to be processed is then divided into M data groups in the data block dimension. The step of determining the operators of the model and dividing the operators of the model into S operator groups based on the number of data groups to be processed includes: Based on the number S of the operator groups and the number N of the processing units, determine the number P of sub-operators in each operator group; Based on the number of sub-operators in each operator group, the operators of the model are segmented to determine each sub-operator in each operator group, wherein... The step of segmenting the model's operators based on the number of sub-operators in each operator group, and determining each sub-operator in each operator group, includes: The operators of the model are quantized to obtain quantized operators; Expanding the quantization operator along the time step dimension yields L operators with a time step of 1; The output dimension of any operator with a time step of 1 is divided into P equal parts to obtain the sub-operator.

2. The method according to claim 1, characterized in that, The process of determining the data set to be processed for each of the input operator groups includes: Determine the i-th data group to be processed from the i-th input operator group; The i-th data group to be processed is divided into L sub-data groups to be processed along the time step dimension; The input data of P sub-operators in the i-th operator group at the j-th time step is determined as the j-th sub-data to be processed.

3. The method according to claim 2, characterized in that, For a value greater than 1, the input data of the P sub-operators in the i-th operator group at the j-th time step further includes: The output data of P sub-operators in the i-th operator group at the (j-1)th time step; The output data includes hidden state information and unit state information.

4. The method according to claim 3, characterized in that, The step of merging the output data of each of the sub-operators and using the merged result as the processing result of the data to be processed includes: The output data of P sub-operators in the S operator groups are merged at the Lth time step to obtain the processing results of S data groups to be processed. The processing results of the S groups of data to be processed are merged to obtain the processing result of the data to be processed.

5. The method according to claim 3, characterized in that, The method further includes: The hidden state information output by the P sub-operators at L time steps is stored in a first set, and each hidden state information in the first set is output; and / or The unit state information output by the P sub-operators at L time steps is stored in the second set, and each unit state information in the second set is output.

6. A readable medium, characterized in that, The readable medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 5.

7. An electronic device, characterized in that, include: Memory, used to store instructions executed by one or more processors of an electronic device, and A processor is one of the processors in an electronic device, used to perform the method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Neural network model splitting method and device, storage medium and computer system

    CN111562977A

  • Neural network computational graph processing method and processing device

    CN114881214A