A data processing method, a data processing device and a storage medium

Abstract

The embodiment of the application discloses a data processing method and device and a storage medium in the field of artificial intelligence, which are used for providing a neural network model with coding and decoding modules arranged in a parallel mode, so that the calculation can be accelerated in parallel, the time performance is optimized, and the huge calculation amount of the full connection layer is reduced. The foregoing method comprises the following steps: obtaining a to-be-input data set; inputting the to-be-input data set into a neural network model to obtain target extraction data, wherein the neural network model comprises at least two coding and decoding modules, and each coding and decoding module in the at least two coding and decoding modules is arranged in a parallel mode.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a data processing method, a data processing device, and a storage medium. Background Technology

[0002] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0003] The Seq2Seq model is a structural model composed of an encoder and a decoder, while the Transformer structure is a Seq2Seq model built on an attention mechanism. It can be applied to many tasks in the field of AI, such as natural language processing (NLP) and automatic speech recognition (ASR), and has achieved great success.

[0004] However, existing Transformer structures used in NLP, ASR, and many other tasks employ a stacked approach, meaning multiple encoder-decoder modules, consisting of multi-head attention mechanisms and feedforward layers, are stacked in series. While this allows for deep extraction of abstract information, the series-stacking of multiple encoder-decoder modules does not adequately optimize the Transformer structure's time performance. Furthermore, due to the series-stacking method, computation can only proceed to the next encoder-decoder module after the previous one has finished, resulting in fully connected layers consuming a significant amount of computation. Additionally, some existing solutions share the attention weights of several adjacent layers in the encoder and decoder within the same encoder-decoder module to save memory and achieve speedup. However, the overall Transformer structure still uses a series-stacking approach, repeatedly stacking multiple encoder-decoder modules consisting of multi-head attention mechanisms and feedforward layers.

[0005] Therefore, how to optimize the time performance of the Transformer structure as a whole and reduce the huge computational cost of the fully connected layers has become an urgent problem to be solved. Summary of the Invention

[0006] This application provides a data processing method, a data processing device, and a storage medium. By providing a neural network model that arranges encoding and decoding modules in parallel, it can not only accelerate computation in parallel but also optimize time performance and reduce the huge amount of computation in fully connected layers.

[0007] In a first aspect, embodiments of this application provide a data processing method, which may include: acquiring a dataset to be input; inputting the dataset to be input into a neural network model to obtain target extraction data, wherein the neural network model includes at least two encoding / decoding modules, and each of the at least two encoding / decoding modules is arranged in parallel. By providing a neural network model with encoding / decoding modules arranged in parallel, not only can parallel computation be accelerated, but time performance is also optimized.

[0008] In some embodiments, after obtaining the input dataset, the method further includes: processing the input dataset based on a target seed function to obtain a first input dataset, the first input dataset including at least one first input data, each of the at least one first input data corresponding to a different resolution; correspondingly, inputting the input dataset into a neural network model to obtain target extraction data, including: inputting each first input data in the first input dataset into each of the at least two encoding / decoding modules to obtain at least one output data; and obtaining target extraction data based on the at least one output data. Through the above method, by introducing a target seed function, the neural network model can extract target data more effectively.

[0009] In some embodiments, processing the input dataset to obtain a first input dataset based on a target seed function includes: sampling the input dataset to obtain a first input dataset based on a preset sampling rate corresponding to the target seed function. This method ensures that the first input dataset has a certain degree of representativeness overall, facilitating the extraction of target data.

[0010] In some embodiments, processing the input dataset based on a target seed function to obtain a first input dataset includes: grouping the input dataset to obtain M groups of input data; and grouping the dataset by a preset sampling rate corresponding to the target seed function to obtain M sub-sampling rates, where M is an integer and M≥2; sampling each group of input data based on the M sub-sampling rates to obtain M groups of first input data; and obtaining the first input dataset based on the M groups of first input data. This method improves the representativeness of the overall input dataset by enhancing the representativeness of each group of finally sampled first input data, thereby improving the accuracy of the final extracted target data.

[0011] In some embodiments, the method further includes: constraining a first output format to a normal distribution according to preset distribution parameters, wherein the first output format is the output format corresponding to the dimension of the data output by the self-attention mechanism layer in the encoding / decoding module. By constraining the output format of the self-attention mechanism layer in each encoding / decoding module to a normal distribution in the above manner, the expressive power of the neural network model is enriched.

[0012] In some embodiments, the method further includes: dividing the dimensions of the data input to the feedforward layer in the encoding / decoding module to obtain N groups of first dimensions, where N is an integer and N≥2; sequentially performing dimensionality increase and decrease on each of the N groups of first dimensions to obtain N groups of second dimensions; and concatenating the N groups of second dimensions to obtain a third dimension, wherein the third dimension is the same as the dimension of the data input to the feedforward layer. Through this method, the reduction-based feedforward layer can significantly reduce the number of parameters while preserving data to the maximum extent.

[0013] Secondly, this application provides a data processing device, which may include: an acquisition unit for acquiring a dataset to be input; and a processing unit for inputting the dataset to be input acquired by the acquisition unit into a neural network model to obtain target extraction data, wherein the neural network model includes at least two encoding and decoding modules, and each of the at least two encoding and decoding modules is arranged in parallel.

[0014] In some embodiments, the processing unit is further configured to, after the acquisition unit acquires the input dataset, process the input dataset based on a target seed function to obtain a first input dataset, the first input dataset including at least one first input data, each of the at least one first input data corresponding to a different resolution; input each of the at least two encoding / decoding modules into each encoding / decoding module to obtain at least one output data; and obtain target extraction data based on the at least one output data.

[0015] In some embodiments, the processing unit is configured to sample the input dataset according to a preset sampling rate corresponding to the target seed function to obtain a first input dataset.

[0016] In some embodiments, the processing unit is configured to:

[0017] The dataset to be input is grouped to obtain M groups of data to be input, and the preset sampling rate corresponding to the target seed function is grouped to obtain M sub-sampling rates, where M is an integer and M≥2;

[0018] For each of the M sets of input data, each set of input data is sampled based on the M sub-sampling rates to obtain the M sets of first input data;

[0019] The first input dataset is obtained based on the M sets of first input data.

[0020] In some embodiments, the processing unit is configured to:

[0021] The first output format is constrained to be a normal distribution according to the preset distribution parameters, wherein the first output format is the output format corresponding to the dimension of the data output by the self-attention mechanism layer in the encoding and decoding module.

[0022] In some embodiments, the processing unit is configured to:

[0023] The dimensions of the data input to the feedforward layer in the encoding and decoding module are divided to obtain N groups of first dimensions, where N is an integer and N≥2;

[0024] Each of the N groups of first dimensions is sequentially upgraded and reduced in dimensionality to obtain N groups of second dimensions;

[0025] The N sets of second dimensions are concatenated to obtain a third dimension, which is the same as the dimension of the data input to the feedforward layer.

[0026] Thirdly, embodiments of this application provide a computer-readable storage medium including a computer program that, when run on a processor, causes the processor to perform a method as described in the first aspect or any possible implementation thereof.

[0027] Fourthly, embodiments of this application provide a computer program product containing a computer program, which, when run on a processor, causes the processor to perform a method as described in the first aspect or any possible implementation thereof.

[0028] Fifthly, embodiments of this application provide a chip system including a processor for supporting a data processing device in implementing the functions involved in the first aspect or any possible implementation thereof. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the data processing device. This chip system may be composed of chips or may include chips and other discrete devices.

[0029] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0030] In this embodiment of the application, by providing a neural network model that arranges the encoding and decoding modules in a parallel manner, not only can parallel computation be accelerated, but time performance is also optimized, reducing the huge amount of computation in the fully connected layer. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application.

[0032] Figure 1 This is a schematic diagram of the system architecture provided in the embodiments of this application;

[0033] Figure 2 This is a schematic diagram of object detection using the convolutional neural network model provided in the embodiments of this application;

[0034] Figure 3 This is a schematic diagram of a chip hardware structure provided in an embodiment of this application;

[0035] Figure 4 This is a schematic diagram of a stacked Transformer structure in an existing scheme.

[0036] Figure 5 This is a weight-sharing Transformer structure in existing solutions;

[0037] Figure 6A schematic diagram of a Transformer structure provided in an embodiment of this application;

[0038] Figure 7 This is a schematic diagram of an embodiment of a data processing method provided in this application.

[0039] Figure 8 This is a schematic diagram of a normally distributed self-attention mechanism layer provided in an embodiment of this application;

[0040] Figure 9a A schematic diagram of the feedforward layer provided in the existing scheme;

[0041] Figure 9b This is a schematic diagram of a reduction feedforward layer provided in an embodiment of this application;

[0042] Figure 10 This is a schematic diagram of the hardware structure of the communication device provided in the embodiments of this application;

[0043] Figure 11 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0044] This application provides a data processing method, a data processing device, and a storage medium. By providing a neural network model that arranges encoding and decoding modules in parallel, it can not only accelerate computation in parallel but also optimize time performance and reduce the huge amount of computation in fully connected layers.

[0045] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0046] The embodiments of this application are mainly used in scenarios such as language processing and text processing, such as NLP, ASR, etc., and are not limited in specific terms in this application.

[0047] The method provided in this application is described below from the perspectives of model training and model application:

[0048] The neural network training method provided in this application involves natural language processing, automatic language recognition, etc., and can be applied to data training, machine learning, deep learning, etc. It performs symbolic and formal intelligent information modeling, extraction, preprocessing, and training on training data (such as the input dataset in this application) to finally obtain a trained neural network, which is convenient for extracting target data.

[0049] The data processing method provided in this application embodiment can utilize the trained neural network described above to input the dataset to be input (e.g., voice or text to be input) into the trained neural network and obtain output data (e.g., target extraction data in this application). It should be noted that the neural network training method and the data processing method provided in this application embodiment are inventions based on the same concept, and can also be understood as two parts of a system, or two stages of an overall process: such as the model training stage and the model application stage.

[0050] Since the embodiments of this application involve a large number of neural network applications, for ease of understanding, the relevant terms and concepts such as neural networks involved in the embodiments of this application will be introduced below.

[0051] (1) Neural Network

[0052] Neural networks can be composed of neural units, which can refer to units represented by x. s The arithmetic unit that takes an intercept of 1 as input can output the following:

[0053]

[0054] Where s = 1, 2, ..., n, n is a natural number greater than 1, W S For x s The weights are denoted by b, where b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input to the next convolutional layer. The activation function can be the sigmoid function. A neural network is a network formed by connecting many of the above-mentioned individual neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0055] (2) Deep Neural Networks

[0056] A deep neural network (DNN) can be understood as a neural network with many hidden layers. The term "many" doesn't have a specific metric; the multi-layered neural networks and deep neural networks are essentially the same thing. DNNs can be categorized into three layers based on their position: input layers, hidden layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. Layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer. Although DNNs seem complex, the operation of each layer is actually quite simple, represented by the following linear relationship: y′=α(Wx′+b), where x′ is the input vector, y′ is the output vector, b is the offset vector, W is the weight matrix (also called coefficients), and α() is the activation function. Each layer simply operates on the input vector x′ to obtain the output vector y′. Because DNNs have many layers, the number of coefficients W and offset vectors b is also large. So, how are the specific parameters defined in a DNN? First, let's look at the definition of the coefficient W. Taking a three-layer DNN as an example, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as follows: The superscript 3 represents the layer number where coefficient W resides, while the subscript corresponds to the output third layer index 2 and the input second layer index 4. In summary, the coefficients from the k-th neuron in layer L-1 to the j-th neuron in layer L are defined as... Note that the input layer does not have a W parameter. In deep neural networks, more hidden layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can accomplish more complex learning tasks.

[0057] (3) Convolutional Neural Network

[0058] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer is a layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron may only be connected to some of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of a series of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are called the convolutional kernel. Shared weights can be understood as the way image information is extracted regardless of location. The convolutional kernel can be initialized as a matrix of random size, and during the training process of the CNN, the kernel can learn appropriate weights. Furthermore, the direct benefit of shared weights is that it reduces the connections between layers in the CNN, while also reducing the risk of overfitting.

[0059] (4) Recurrent Neural Networks (RNNs), also known as recurrent neural networks, are used to process sequential data. In traditional neural network models, the layers are fully connected from the input layer to the intermediate layers and then to the output layer, but the nodes within each layer are unconnected. While this type of ordinary neural network has solved many difficult problems, it is still powerless against many others. For example, to predict the next word in a sentence, you generally need to use the preceding words because the words in a sentence are not independent. RNNs are called recurrent neural networks because the current output of a sequence is related to the previous outputs. Specifically, the network memorizes previous information and applies it to the calculation of the current output. That is, the nodes within the intermediate layers are no longer unconnected but connected, and the input of the intermediate layers includes not only the output of the input layer but also the output of the intermediate layer at the previous time step. Theoretically, RNNs can process sequential data of any length. Training RNNs is the same as training traditional CNNs or DNNs.

[0060] (5) Residual Neural Network (ResNet)

[0061] Residual neural networks were proposed to address the degradation problem that occurs when a neural network has too many hidden layers. Degradation refers to the phenomenon where, as the number of hidden layers increases, the network's accuracy saturates and then rapidly declines. This degradation is not caused by overfitting, but rather by insufficient gradient updates during backpropagation, where gradients become less correlated at lower levels, leading to reduced accuracy in the final model's label predictions. When a neural network degrades, shallower networks can achieve better training results than deeper networks. If features from lower layers are passed to higher layers, the results should at least be comparable to those of shallower networks. This can be achieved using an identity mapping called a residual connection (shortcut). Optimizing this residual mapping is easier than optimizing the original mapping.

[0062] (6) Classifier

[0063] Many neural network architectures ultimately include a classifier to classify objects in an image. A classifier typically consists of fully connected layers and a softmax function, capable of outputting probabilities for different classes based on the input.

[0064] The above provides a brief introduction to some basic concepts of neural networks. However, it is understood that in the following embodiments of this application, the neural network model mentioned can be understood as an optimized model based on the aforementioned convolutional neural network. The following will combine... Figure 1 The system architecture of the embodiments of this application will be described in detail.

[0065] Figure 1 This is a schematic diagram of the system architecture according to an embodiment of this application. For example... Figure 1 As shown, the system architecture 100 includes an execution device 110, a training device 120, a database 130, a client device 140, a data storage system 150, and a data acquisition device 160.

[0066] Additionally, the execution device 110 includes a calculation module 111, an I / O interface 112, a preprocessing module 113, and a preprocessing module 114. The calculation module 111 may include a target model / rule 101, while the preprocessing modules 113 and 114 are optional.

[0067] The data acquisition device 160 is used to collect training data. For the data processing method of this embodiment, the training data may include a dataset of voice or text to be input. After collecting the training data, the data acquisition device 160 stores this training data in the database 130, and the training device 120 trains the target model / rule 101 based on the training data maintained in the database 130.

[0068] The following describes how the training device 120 obtains the target model / rule 101 based on the training data. The training device 120 processes the input dataset and compares the output target extraction data with the pre-labeled output results until the difference between the output target extraction data of the training device 120 and the pre-labeled output results is less than a certain threshold, thereby completing the training of the target model / rule 101.

[0069] The aforementioned target model / rule 101 can be used to implement the data processing method of this application embodiment. That is, by inputting the dataset to be input (after relevant preprocessing) into the target model / rule 101, the target extraction data can be obtained. Specifically, the target model / rule 101 in this application embodiment can be a neural network. It should be noted that in practical applications, the training data maintained in the database 130 may not all come from the data acquisition device 160; it may also be received from other devices. Furthermore, it should be noted that the training device 120 may not necessarily train the target model / rule 101 entirely based on the training data maintained in the database 130; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as a limitation of this application embodiment.

[0070] The target model / rule 101 trained using training device 120 can be applied to different systems or devices, such as... Figure 1 The execution device 110 shown can be a terminal, such as a mobile phone terminal, tablet computer, laptop computer, augmented reality (AR) / virtual reality (VR) device, vehicle terminal, etc., or it can be a server or cloud service. Figure 1 In this embodiment, the execution device 110 is configured with an input / output (I / O) interface 112 for data interaction with external devices. Users can input data into the I / O interface 112 through a client device 140. This input data may include a dataset to be input from the client device. Specifically, the client device 140 may be a terminal device.

[0071] Preprocessing module 113 and preprocessing module 114 are used to preprocess the input data (such as the dataset to be input) received by I / O interface 112. In this embodiment, preprocessing module 113 and preprocessing module 114 may be omitted (or only one of them may be used), and the calculation module 111 may be used directly to process the input data.

[0072] During the preprocessing of input data by the execution device 110, or during the calculation module 111 of the execution device 110 performing calculations and other related processes, the execution device 110 can call data, code, etc. in the data storage system 150 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 150.

[0073] Finally, I / O interface 112 presents the processing results, such as the target extracted data obtained above, to client device 140, thereby providing it to the user.

[0074] It is worth noting that the training device 120 can generate corresponding target models / rules 101 based on different training data for different objectives or tasks. The corresponding target models / rules 101 can be used to achieve the above objectives or complete the above tasks, thereby providing the user with the required results.

[0075] exist Figure 1 In the scenario shown, the user can manually provide input data, which can be done through the interface provided by I / O interface 112. Alternatively, the client device 140 can automatically send input data to I / O interface 112. If user authorization is required for the client device 140 to automatically send input data, the user can set the corresponding permissions in the client device 140. The user can view the output results of the execution device 110 on the client device 140, which can be presented in various forms such as display, sound, or animation. The client device 140 can also act as a data acquisition terminal, collecting the input data and output results of the input I / O interface 112 as new sample data and storing them in the database 130. Alternatively, data can be collected directly from the I / O interface 112 without going through the client device 140, using the input data and output results of the input I / O interface 112 as new sample data and storing them in the database 130.

[0076] It is worth noting that, Figure 1 This is merely a schematic diagram of a system architecture provided in an embodiment of this application. The positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 1In this context, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed within the execution device 110.

[0077] like Figure 1 As shown, the target model / rule 101 is trained by the training device 120. In this embodiment of the application, it can be the neural network model of this application. Specifically, the neural network model provided in this embodiment of the application can be an optimized model based on CNN and deep convolutional neural networks (DCNN), etc.

[0078] Since CNN is a very common type of neural network, the following will combine... Figure 2 This section focuses on a detailed explanation of the structure of CNNs. As mentioned in the basic concept introduction above, a Convolutional Neural Network (CNN) is a deep neural network with a convolutional structure. It is a deep learning architecture, which refers to learning at multiple levels of abstraction through machine learning algorithms. As a deep learning architecture, CNN is a feed-forward artificial neural network, where each neuron responds to the input image.

[0079] like Figure 2 As shown, the Convolutional Neural Network (CNN) 200 may include an input layer 210, convolutional / pooling layers 220 (where pooling layers are optional), and neural network layers 230. The details of these layers are described below.

[0080] Convolutional / pooling layers 220:

[0081] Convolutional layers:

[0082] like Figure 2 The convolutional / pooling layer 220 shown may include layers as in Examples 221-226. For instance, in one implementation, layer 221 is a convolutional layer, layer 222 is a pooling layer, layer 223 is a convolutional layer, layer 224 is a pooling layer, layer 225 is a convolutional layer, and layer 226 is a pooling layer; in another implementation, layers 221 and 222 are convolutional layers, layer 223 is a pooling layer, layers 224 and 225 are convolutional layers, and layer 226 is a pooling layer. That is, the output of the convolutional layer can be used as the input to a subsequent pooling layer, or as the input to another convolutional layer to continue the convolution operation.

[0083] The following section will use convolutional layer 221 as an example to introduce the internal working principle of a convolutional layer.

[0084] Convolutional layer 221 can include multiple convolution operators, also known as kernels. In image processing, a convolution operator acts as a filter to extract specific information from the input image matrix. Essentially, a convolution operator can be a weight matrix, which is usually predefined. During the convolution operation, the weight matrix typically processes the input image pixel by pixel (or two pixels by two pixels, depending on the stride) along the horizontal direction, thus extracting specific features from the image. The size of the weight matrix should be related to the image size. It's important to note that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image; during convolution, the weight matrix extends to the entire depth of the input image. Therefore, convolution with a single weight matrix produces a single-depth convolutional output. However, in most cases, a single weight matrix is ​​not used; instead, multiple weight matrices of the same size (rows × columns) are applied—multiple identical matrices. The outputs of each weight matrix are stacked to form the depth dimension of the convolutional image; this dimension can be understood as being determined by the "multiple" mentioned above. Different weight matrices can be used to extract different features from an image. For example, one weight matrix can be used to extract edge information, another to extract specific colors, and yet another to blur unwanted noise. These multiple weight matrices have the same size (rows × columns), and the resulting convolutional feature maps are also of the same size. These extracted convolutional feature maps are then merged to form the output of the convolution operation.

[0085] The weight values ​​in these weight matrices need to be obtained through extensive training in practical applications. The weight matrices formed by the weight values ​​obtained through training can be used to extract information from the input image, thereby enabling the convolutional neural network 200 to make correct predictions.

[0086] When a convolutional neural network 200 has multiple convolutional layers, the initial convolutional layers (e.g., 221) tend to extract more general features, which can also be called low-level features. As the depth of the convolutional neural network 200 increases, the features extracted by later convolutional layers (e.g., 226) become more and more complex, such as high-level semantic features. Features with higher semantic levels are more suitable for the problem to be solved.

[0087] Pooling layer:

[0088] Because it is often necessary to reduce the number of training parameters, pooling layers are often introduced periodically after convolutional layers, such as... Figure 2Layers 221-226 in example 220 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. In image processing, the sole purpose of pooling layers is to reduce the spatial size of the image. Pooling layers can include average pooling and / or max pooling operators to sample the input image to obtain a smaller image size. The average pooling operator calculates the average value of pixel values ​​within a specific range as the result of average pooling. The max pooling operator takes the pixel with the largest value within a specific range as the result of max pooling. Furthermore, just as the size of the weight matrix in a convolutional layer should be related to the image size, the operators in a pooling layer should also be related to the image size. The size of the output image after processing by the pooling layer can be smaller than the size of the input image of the pooling layer. Each pixel in the output image of the pooling layer represents the average or maximum value of the corresponding sub-region of the input image of the pooling layer.

[0089] Neural network layer 230:

[0090] After processing by the convolutional / pooling layers 220, the convolutional neural network 200 is still insufficient to output the required information. As mentioned earlier, the convolutional / pooling layers 220 only extract features and reduce the parameters introduced by the input image. However, to generate the final output information (the required class information or other relevant information), the convolutional neural network 200 needs to utilize neural network layers 230 to generate one or more outputs representing the required number of classes. Therefore, neural network layers 230 can include multiple hidden layers (such as...). Figure 2 As shown in layers 231, 232 to 23n) and output layer 240, the parameters contained in these multi-layer hidden layers can be pre-trained based on relevant training data for specific task types, such as image recognition, image classification, image super-resolution reconstruction, etc.

[0091] After the multiple hidden layers in neural network layer 230, the final layer of the entire convolutional neural network 200 is the output layer 240. This output layer 240 has a loss function similar to classification cross-entropy, specifically used to calculate the prediction error. Once the entire convolutional neural network 200 has undergone forward propagation (such as...), the loss function is applied. Figure 2 Propagation from 210 to 240 degrees is considered forward propagation, while backward propagation (e.g.) is completed. Figure 2 The propagation from 240 to 210 (backpropagation) will begin to update the weight values ​​and biases of the layers mentioned above, in order to reduce the loss of the convolutional neural network 200 and the error between the output of the convolutional neural network 200 through the output layer and the ideal result.

[0092] It should be noted that, as Figure 2The convolutional neural network 200 shown is only an example of a convolutional neural network. In specific applications, convolutional neural networks can also exist in the form of other network models.

[0093] It should be understood that this can be adopted. Figure 2 The convolutional neural network (CNN) 200 shown performs the object detection method of the embodiments of this application, such as... Figure 2 As shown, the target extraction data can be obtained after the input dataset is processed by the input layer 210, the convolutional / pooling layer 220 and the neural network layer 230.

[0094] Figure 3 This application provides a chip hardware structure, which includes a neural network processor. The chip can be configured as follows: Figure 1 The execution device 110 shown is used to perform the calculations of the calculation module 111. This chip can also be located in, for example... Figure 1 The training device 120 shown is used to complete the training work of the training device 120 and output the target model / rule 101. For example... Figure 2 The algorithms for each layer in the convolutional neural network shown can all be implemented in, for example... Figure 3 This is achieved in the chip shown.

[0095] The Neural Processing Unit (NPU) is mounted as a coprocessor on the main central processing unit (CPU) (host CPU), and tasks are assigned by the host CPU. The core of the NPU is the arithmetic circuit 303, and the controller 304 controls the arithmetic circuit 303 to retrieve data from the memory (weight memory or input memory) and perform calculations.

[0096] In some implementations, the arithmetic circuit 303 internally includes multiple process engines (PEs). In some implementations, the arithmetic circuit 303 is a two-dimensional pulsating array. The arithmetic circuit 303 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 303 is a general-purpose matrix processor.

[0097] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit 303 retrieves the corresponding data for matrix B from the weight memory 302 and caches it in each PE (Engineer Component) of the arithmetic circuit 303. The arithmetic circuit 303 retrieves the data for matrix A from the input memory 301 and performs matrix operations with matrix B. The partial or final result of the obtained matrix is ​​stored in the accumulator 308.

[0098] The vector computation unit 307 can further process the output of the arithmetic circuit 303, such as vector multiplication, vector addition, exponentiation, logarithmic operations, size comparisons, etc. For example, the vector computation unit 307 can be used for network computation in non-convolutional / non-FC layers of neural networks, such as pooling, batch normalization, local response normalization, etc.

[0099] In some implementations, the vector computation unit 307 can store the processed output vector into a unified buffer 306. For example, the vector computation unit 307 can apply a nonlinear function to the output of the arithmetic circuit 303, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit 307 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to the arithmetic circuit 303, for example, for use in subsequent layers of a neural network.

[0100] The unified memory 306 is used to store input data and output data.

[0101] The weight data is directly transferred from the external memory to the input memory 301 and / or the unified memory 306 through the direct memory access controller 305 (DMAC), the weight data in the external memory is stored in the weight memory 302, and the data in the unified memory 306 is stored in the external memory.

[0102] The bus interface unit (BIU) 310 is used to enable interaction between the main CPU, DMAC and instruction fetch memory 309 via a bus.

[0103] The instruction fetch buffer 309, which is connected to the controller 304, is used to store the instructions used by the controller 304.

[0104] The controller 304 is used to call the instructions cached in the instruction memory 309 to control the operation of the computing accelerator.

[0105] Generally, the unified memory 306, input memory 301, weight memory 302, and instruction fetch memory 309 are all on-chip memories, while the external memory is memory outside the NPU. This external memory can be double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM), or other readable and writable memory.

[0106] in, Figure 2 The operations of each layer in the convolutional neural network shown can be performed by the computation circuit or the vector computation module 307.

[0107] The above-mentioned Figure 1 The execution device 110 in the present application is capable of executing the various steps of the data processing method according to the embodiments of the present application. Figure 2 The CNN model shown and Figure 3 The chip shown can also be used to perform various steps of the data processing method of the embodiments of this application. It is understood that the above... Figure 2 The input layer 210, convolutional / pooling layer 220, and neural network layer 230 described herein can be understood as an encoding / decoding module, while Figure 2 This description only covers one encoding / decoding module. In practical applications, multiple encoding / decoding modules can be arranged in parallel to facilitate parallel computing. See the following sections for details. Figure 6 Understand the content described.

[0108] In related technologies, the Transformer structure used in many tasks such as NLP and ASR adopts a stacked approach, that is, multiple encoding and decoding modules composed of multi-head attention mechanism layers, feedforward layers, etc. are stacked in series. Figure 4 A schematic diagram of a stacked Transformer structure in existing schemes is shown. Figure 4 It can be seen that the encoding and decoding modules in the Transformer architecture are mainly composed of multi-head attention (MHA) layers and feed-forward (FF) layers; and from... Figure 4 It can also be learned that multiple modules need to be connected in series, such as Figure 4 Nx is shown in the figure. Additionally... Figure 4It also demonstrates the introduction of positional encoding at the input to record historical information, while residual connection layers are used in this Transformer structure to optimize gradient propagation in multi-layer networks. However, although the above can be utilized... Figure 4 The Transformer structure shown extracts abstract information at a deep level, but by stacking multiple encoding and decoding modules in a series manner, the time performance of the Transformer structure cannot be fully optimized. Moreover, due to the series manner, the next encoding and decoding module can only proceed to the next one after the previous one has completed its computation, which makes the fully connected layer account for a large amount of computation.

[0109] To save memory space, Figure 5 This illustrates a weight-sharing Transformer structure in an existing scheme. From Figure 5 It can be seen that by sharing the attention weights of several adjacent layers of the encoder and decoder in the encoding and decoding module, such as sharing the weights between the attention mechanism layer of layer m and the attention mechanism layer of layer m+i. However, from Figure 5 As shown in the Transformer structure, the overall Transformer structure still stacks multiple encoding and decoding modules in a series manner, which makes it impossible to fully optimize the time performance, and the fully connected layer still occupies a large amount of computation.

[0110] Therefore, in order to solve the above Figure 4 as well as Figure 5 The problems caused by the Transformer structure shown are addressed in this application embodiment by providing a data processing method that can be applied to NLP, ASR, and other application scenarios. Please refer to... Figure 6 This is a schematic diagram of a Transformer structure provided in an embodiment of this application. Figure 6As shown, at least two encoder-decoder modules are arranged in parallel. Each encoder-decoder module may include a convolutional layer (CONV), a normally distributed self-attention layer, a residual layer, and a reduced feedforward layer (FF). Thus, after acquiring the input data, the input data can optionally be processed through a target seed function to obtain first input datasets of different resolutions. Furthermore, each first input data point in this first input dataset can be input into a separate encoder-decoder module. Then, the first input data is processed in parallel by each encoder-decoder module, resulting in the final output data being processed through a fully connected layer to obtain the target extraction data. It is understandable that the output format of the normally distributed self-attention mechanism layer in each encoding and decoding module conforms to a normal distribution, which enables the extracted target data to be more fully expressed. The reduction feedforward layer, on the other hand, divides the dimensions of the input data into different groups, and then performs dimensionality increase and decrease on each group in turn, so that the dimension of the data output by the reduction feedforward layer is consistent with the dimension of the input data. However, the number of parameters is reduced to 1 / N of the number of parameters before processing, where N is the number of groups and N≥2.

[0111] To better understand the data processing methods proposed in the embodiments of this application, please refer to... Figure 7 This is a schematic diagram illustrating an embodiment of a data processing method provided in this application. Figure 7 As shown, the method may include:

[0112] 701. Obtain the dataset to be input.

[0113] In this embodiment, the dataset to be input includes at least one piece of data to be input, which may include, but is not limited to, text to be input or voice to be input, and is not specifically limited in this embodiment.

[0114] 702. The input dataset is processed based on the target seed function to obtain a first input dataset, the first input dataset including at least one first input data, and each of the at least one first input data has a different resolution.

[0115] In this embodiment, since the target seed function is a function that can sample the input dataset to obtain inputs at different resolutions, it can guarantee that... Figure 6The Transformer structure shown allows for sufficient data extraction. Therefore, by processing the input dataset using the target seed function, the resolution of each first input data point in the resulting first input dataset is different. Each first input data point with a different resolution reflects the input data randomly sampled from the input dataset and input to each of the aforementioned encoding / decoding modules. This ensures that the input data input to each encoding / decoding module is random and representative, reflecting the correlation between these input data points, thus facilitating sufficient extraction.

[0116] Specifically, the form of the above-mentioned target seed function can be understood as follows:

[0117] f seed =f factorized (f select (input))

[0118] Here, the input mentioned above can be understood as the dataset to be input, f seed () can be understood as the target seed function; while the selection function f select () is mainly used to process the input to obtain a second input dataset containing at least one second input data of different resolutions; while f factorized The main function is to decompose the second input dataset. For example, the second input dataset is first split into at least one set of second input data, and then the at least one set of second input data is merged so that the final decomposed value is used as the aforementioned first input dataset. Moreover, the data volume of the first input dataset is compressed compared to directly inputting the aforementioned second input dataset into the encoding and decoding module.

[0119] Optionally, in some embodiments, the input dataset is processed based on the target seed function to obtain a first input dataset, which can be understood as follows:

[0120] The first method involves sampling the input dataset based on the preset sampling rate corresponding to the target seed function to obtain the first input dataset.

[0121] In this embodiment, the preset sampling rate can be determined as needed, and no limitation is specified here. Furthermore, after obtaining the input dataset, sampling can be performed based on the preset sampling rate corresponding to the target seed function, ensuring that the final sampled first input dataset has a certain degree of representativeness overall, thus facilitating the full extraction of the target data.

[0122] The second method involves grouping the dataset to be input into M groups of data, and grouping the preset sampling rate corresponding to the target seed function into M sub-sampling rates, where M is an integer and M≥2; for each group of data to be input into M groups, sampling is performed on each group of data to be input into M groups based on the M sub-sampling rates to obtain M first input data; and the first input dataset is obtained based on the M first input data.

[0123] In this embodiment, assuming a preset sampling rate of σ', this σ' is further subdivided into M sub-sampling rates, such as: σ1', σ2', ..., σ M Similarly, the obtained input dataset X also needs to be divided into M groups of input data, such as: X1, X2, ..., X... M In this way, the M sets of input data are sampled according to the M sub-sampling rates, and then the first input data obtained from each set are integrated into the first input dataset.

[0124] Furthermore, it should be noted that the above σ'=∑X i σ' i / X, 0 < i ≤ M, M ≥ 2. #

[0125] Understandably, the above method can improve the representativeness of the overall input dataset by increasing the number of samples obtained from each final sample, thereby improving the accuracy of the final extracted target data. Furthermore, compared to the first method, which directly samples from the entire input dataset based on a preset sampling rate, the second sampling method uses grouping, ensuring that each sampled first input data set is highly representative when the input dataset exhibits significant stratification.

[0126] 703. Input each first input data in the first input dataset into each of the at least two encoding and decoding modules to obtain at least one output data.

[0127] In this embodiment, after obtaining the first input dataset, the process can be carried out as described above. Figure 6 The Transformer structure shown inputs each first input data point from the first input dataset into parallel encoder-decoder modules. After processing by each encoder-decoder module, each module outputs its corresponding output data.

[0128] 704. Obtain target extraction data based on the at least one output data.

[0129] In this embodiment, the output data of each encoding and decoding module is used as the input of the fully connected layer, so that the target extracted data can be obtained after processing by the fully connected layer.

[0130] It is understood that each of the encoding / decoding modules and fully connected layers described above can be included in the deep convolutional neural network modules described above.

[0131] Alternatively, in other embodiments, in order to enable Figure 6 The Transformer structure shown has a more comprehensive expressive power. The above data processing method may also include: constraining the first output format to a normal distribution according to a preset distribution parameter, wherein the first output format is the output format corresponding to the dimension of the data output by the self-attention mechanism layer in the encoding and decoding module.

[0132] In this embodiment, since the normal distribution can fully express the distribution state of the data, the first output format of the self-attention mechanism layer can be limited according to the preset distribution function. Specifically, the output format corresponding to the dimension of the data output by the self-attention mechanism layer in the encoding and decoding module is constrained to a normal distribution. Figure 8 This diagram illustrates a normally distributed self-attention mechanism layer provided in an embodiment of this application. Figure 8 It can be seen that, assuming the data output by the self-attention mechanism layer has a dimension of 512, if it needs to go through 8 multi-head attention layers, then the first output format of the self-attention layer can be constrained to a normal distribution in the form of {32,32,48,48,80,80,96,96}.

[0133] It is understandable that the above description Figure 8 This is merely an illustrative description. In practical applications, the first output format of the self-attention mechanism layer can also be constrained in other forms of normal distribution, which is not limited in the specific embodiments of this application.

[0134] Optionally, in other embodiments, the above data processing method may further include: dividing the dimensions of the data input to the feedforward layer in the encoding / decoding module to obtain N groups of first dimensions, where N is an integer and N≥2; sequentially increasing and scaling each of the N groups of first dimensions to obtain N groups of second dimensions; and concatenating the N groups of second dimensions to obtain a third dimension, wherein the third dimension is the same as the dimension of the data input to the feedforward layer.

[0135] Understandably, the data input to the feedforward layer is first divided into N groups based on the first dimension. Then, for each group of first dimensions, the dimension is first increased and then decreased. Finally, the second dimension is concatenated. This not only ensures that the final third dimension matches the dimension of the data input to the feedforward layer but also reduces the number of parameters. For example, assuming the data input to the feedforward layer has a dimension of 'a' and the intermediate extended dimension is 'b', if it is divided into N groups, the number of parameters would be...

[0136] For example, please refer to Figure 9a This is a schematic diagram of the feedforward layer provided in the existing scheme. As can be seen from 9a, assuming the input data to the feedforward layer has a dimension of 512, it is directly enlarged to 2048 dimensions, and then scaled back to 512 dimensions. However, using... Figure 9a The method has a parameter count of 2×512×2048.

[0137] Please see Figure 9b This is a schematic diagram of a reduced feedforward layer provided in an embodiment of this application. From Figure 9b As can be seen, assuming the input data to the feedforward layer has a dimension of 512, this 512 dimension is divided into two groups of 256 dimensions. Then, for each group of 256 dimensions, the dimension is first increased to 1024, and then reduced back to 256. The two groups of 256 dimensions are then concatenated to obtain a 512-dimensional system. Figure 9b The parameter count is 2 × (256 / 2 × 1024 / 2 + 1024 / 2 + 256 / 2) = 256 × 1024. Clearly, through... Figure 9b The reduced feedforward layer significantly reduces the number of parameters.

[0138] It is understood that in practical applications, the dimensions of the feedforward layer input data can be divided according to other proportions, and the first dimension and the second dimension described above can be other dimensions, which are not limited in this embodiment.

[0139] In this embodiment, by providing a neural network model that arranges encoding and decoding modules in parallel, not only can parallel computation be accelerated, but the introduction of a target seed function enables the neural network model to extract target data more effectively. Furthermore, by constraining the output format of the self-attention mechanism layer in each encoding and decoding module to a normal distribution, the expressive power of the neural network model is enriched. The reduction-style feedforward layer can greatly reduce the number of parameters while preserving data to the maximum extent.

[0140] The foregoing primarily describes the solutions provided in the embodiments of this application from a methodological perspective. It is understood that the aforementioned network management and control system, in order to achieve the above functions, includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the functions described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0141] From the perspective of physical devices, the aforementioned data processing device can be implemented by a single physical device, or by multiple physical devices working together, or it can be a logical functional unit within a single physical device. This application does not specifically limit this aspect.

[0142] For example, the aforementioned data processing equipment can be provided by Figure 10 It is implemented using communication devices. Figure 10 This is a schematic diagram of the hardware structure of a communication device provided in an embodiment of this application. The communication device includes at least one processor 1001, a memory 1002, and a transceiver 1003.

[0143] The processor 1001 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more ASICs used to control the execution of the program in this application.

[0144] Transceiver 1003, using any transceiver-like device, is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Transceiver 1003 can be connected to processor 1001.

[0145] The memory 1002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 1002 may exist independently or may be connected to the processor 1001. The memory 1002 may also be integrated with the processor 1001.

[0146] The memory 1002 stores computer execution instructions for implementing the scheme of this application, and its execution is controlled by the processor 1001. The processor 1001 executes the computer execution instructions stored in the memory 1002, thereby implementing the data processing method provided in the above-described method embodiments of this application.

[0147] In one possible implementation, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0148] In a specific implementation, as one example, the processor 1001 may include one or more CPUs, for example... Figure 10 CPU0 and CPU1 in the CPU.

[0149] From the perspective of functional units, this application can divide the data processing device into functional units according to the above method embodiments. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one functional unit. The integrated functional unit can be implemented in hardware or in software.

[0150] For example, when dividing the functional units using an integrated approach. Figure 11 A schematic diagram of the structure of a data processing device provided in an embodiment of this application is shown. Figure 11 As shown, one embodiment of the data processing device 110 of this application may include:

[0151] Acquisition unit 1101 is used to acquire the dataset to be input;

[0152] The processing unit 1102 is used to input the dataset to be input obtained by the acquisition unit 1101 into the neural network model to obtain the target extraction data. The neural network model includes at least two encoding and decoding modules, and each of the at least two encoding and decoding modules is arranged in parallel.

[0153] In some embodiments, the processing unit 1102 is further configured to, after the acquisition unit 1101 acquires the input dataset, process the input dataset based on the target seed function to obtain a first input dataset, the first input dataset including at least one first input data, each of the at least one first input data corresponding to a different resolution; input each of the at least two encoding and decoding modules into each encoding and decoding module to obtain at least one output data; and obtain target extraction data based on the at least one output data.

[0154] In some embodiments, the processing unit 1102 is configured to sample the input dataset according to a preset sampling rate corresponding to the target seed function to obtain a first input dataset.

[0155] In some embodiments, the processing unit 1102 is configured to:

[0156] The dataset to be input is grouped to obtain M groups of data to be input, and the preset sampling rate corresponding to the target seed function is grouped to obtain M sub-sampling rates, where M is an integer and M≥2;

[0157] For each of the M sets of input data, each set of input data is sampled based on the M sub-sampling rates to obtain the M sets of first input data;

[0158] The first input dataset is obtained based on the M sets of first input data.

[0159] In some embodiments, the processing unit 1102 is configured to:

[0160] The first output format is constrained to be a normal distribution according to the preset distribution parameters, wherein the first output format is the output format corresponding to the dimension of the data output by the self-attention mechanism layer in the encoding and decoding module.

[0161] In some embodiments, the processing unit 1102 is configured to:

[0162] The dimensions of the data input to the feedforward layer in the encoding and decoding module are divided to obtain N groups of first dimensions, where N is an integer and N≥2;

[0163] Each of the N groups of first dimensions is sequentially upgraded and reduced in dimensionality to obtain N groups of second dimensions;

[0164] The N sets of second dimensions are concatenated to obtain a third dimension, which is the same as the dimension of the data input to the feedforward layer.

[0165] The data processing device 110 provided in this embodiment is used to perform... Figure 7 , Figure 8 as well as Figure 9b The methods in the corresponding method embodiments are as follows, therefore the embodiments of this application can be referred to. Figure 7 , Figure 8 as well as Figure 9b The relevant parts of the corresponding method embodiments should be understood.

[0166] In this embodiment, the data processing device 110 is presented in an integrated manner, divided into various functional units. Here, "functional unit" can refer to an application-specific integrated circuit (ASIC), a processor and memory executing one or more software or firmware programs, integrated logic circuits, and / or other devices that can provide the aforementioned functions. In a simple embodiment, those skilled in the art will understand that the data processing device 110 can employ... Figure 10 As shown in the figure.

[0167] for example, Figure 10 The processor 1001 can invoke computer execution instructions stored in the memory 1002 to cause the data processing device 110 to perform operations. Figure 7 , Figure 8 as well as Figure 9b The method executed by the data processing device in the corresponding method embodiment.

[0168] Specifically, Figure 11 The function / implementation process of the processing unit 1102 can be achieved through... Figure 10 The processor 1001 in the memory calls the computer execution instructions stored in the memory 1002 to achieve this. Figure 11 The function / implementation process of the acquisition unit 1101 can be achieved through... Figure 10 This is achieved through the transceiver device 1003.

[0169] In this application Figure 10The various components in the device are communicatively connected, that is, the processing unit (or processor), storage unit (or memory), and transceiver unit (transceiver) communicate with each other through internal connection paths to transmit control and / or data signals. The above-described method embodiments of this application can be applied to a processor, or the steps of the above-described method embodiments can be implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above-described method embodiments can be completed by the integrated logic circuit in the processor or by instructions in the form of software. The processor mentioned above may be a central processing unit (CPU), a network processor (NP) or a combination of CPU and NP, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules within the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method. Although only one processor is shown in the figure, the device may include multiple processors or the processor may include multiple processing units. Specifically, the processor can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor.

[0170] Memory is used to store computer instructions executed by the processor. Memory can be a storage circuit or a memory. Memory can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. Non-volatile memory can be read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, or flash memory. Volatile memory can be random access memory, used as an external cache. Memory can be independent of the processor or it can be a storage unit within the processor; this is not limited here. Although only one memory is shown in the figure, the device may include multiple memories or the memory may include multiple storage units.

[0171] A transceiver is used to enable content interaction between the processor and other units or network elements. Specifically, the transceiver can be a communication interface of the device, a transceiver circuit or communication unit, or a transceiver transceiver. The transceiver can also be the processor's communication interface or transceiver circuit. Optionally, the transceiver can be a transceiver chip. The transceiver can also include a transmitting unit and / or a receiving unit. In one possible implementation, the transceiver can include at least one communication interface. In another possible implementation, the transceiver can also be a unit implemented in software. In various embodiments of this application, the processor can interact with other units or network elements through the transceiver. For example, the processor can acquire or receive content from other network elements through the transceiver. If the processor and the transceiver are physically separate components, the processor can interact with other units of the device without going through the transceiver.

[0172] In one possible implementation, the processor, memory, and transceiver can be interconnected via a bus. This bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. The bus can be categorized as an address bus, data bus, control bus, etc.

[0173] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0174] In the various embodiments of this application, numerous examples have been provided for ease of understanding. However, these examples are merely illustrations and do not imply the best way to implement this application.

[0175] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, they can be implemented in whole or in part in the form of a computer program product.

[0176] The computer program product includes one or more computer instructions. When the computer execution instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0177] The technical solutions provided in this application have been described in detail above. Specific examples have been used in this application to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing method, characterized in that, include: Obtain the input dataset; The dataset to be input is fed into a neural network model to obtain the target extraction data. The neural network model includes at least two encoding and decoding modules, each of which is arranged in parallel. Each of the at least two encoding and decoding modules performs parallel computation. The first output format is constrained to be a normal distribution according to the preset distribution parameters, wherein the first output format is the output format corresponding to the dimension of the data output by the self-attention mechanism layer in the encoding and decoding module.

2. The method according to claim 1, characterized in that, After obtaining the input dataset, the method further includes: The input dataset is processed based on the target seed function to obtain a first input dataset. The first input dataset includes at least one first input data, and each of the at least one first input data corresponds to a different resolution. Correspondingly, the dataset to be input is fed into a deep convolutional neural network model to obtain target extraction data, including: Each first input data in the first input dataset is input into each of the at least two encoding and decoding modules to obtain at least one output data. Target extraction data is obtained based on at least one of the output data.

3. The method according to claim 2, characterized in that, The input dataset is processed based on the target seed function to obtain the first input dataset, which includes: The input dataset is sampled based on the preset sampling rate corresponding to the target seed function to obtain the first input dataset.

4. The method according to claim 2, characterized in that, The input dataset is processed based on the target seed function to obtain the first input dataset, which includes: The dataset to be input is grouped to obtain M groups of data to be input, and the preset sampling rate corresponding to the target seed function is grouped to obtain M sub-sampling rates, where M is an integer and M≥2; For each of the M sets of input data, each set of input data is sampled based on the M sub-sampling rates to obtain the M sets of first input data; The first input dataset is obtained based on the M sets of first input data.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: The dimensions of the data input to the feedforward layer in the encoding and decoding module are divided to obtain N groups of first dimensions, where N is an integer and N≥2; Each of the N groups of first dimensions is sequentially upgraded and reduced in dimensionality to obtain N groups of second dimensions; The N sets of second dimensions are concatenated to obtain a third dimension, which is the same as the dimension of the data input to the feedforward layer.

6. A data processing device, characterized in that, include: The acquisition unit is used to acquire the dataset to be input. A processing unit is configured to input the dataset to be input obtained by the acquisition unit into a neural network model to obtain target extraction data. The neural network model includes at least two encoding and decoding modules, each of which is arranged in parallel. Each of the at least two encoding and decoding modules performs parallel computation. The processing unit is used for: The first output format is constrained to be a normal distribution according to the preset distribution parameters, wherein the first output format is the output format corresponding to the dimension of the data output by the self-attention mechanism layer in the encoding and decoding module.

7. The data processing device according to claim 6, characterized in that, The processing unit is further configured to process the input dataset based on the target seed function after the acquisition unit acquires the input dataset to obtain a first input dataset, wherein the first input dataset includes at least one first input data, and each of the at least one first input data corresponds to a different resolution; The processing unit is used for: Each first input data in the first input dataset is input into each of the at least two encoding and decoding modules to obtain at least one output data. Target extraction data is obtained based on at least one of the output data.

8. The data processing device according to claim 7, characterized in that, The processing unit is used to sample the input dataset according to the preset sampling rate corresponding to the target seed function to obtain the first input dataset.

9. The data processing device according to claim 7, characterized in that, The processing unit is used for: The dataset to be input is grouped to obtain M groups of data to be input, and the preset sampling rate corresponding to the target seed function is grouped to obtain M sub-sampling rates, where M is an integer and M≥2; For each of the M sets of input data, each set of input data is sampled based on the M sub-sampling rates to obtain the M sets of first input data; The first input dataset is obtained based on the M sets of first input data.

10. The data processing apparatus according to any one of claims 6-9, characterized in that, The processing unit is used for: The dimensions of the data input to the feedforward layer in the encoding and decoding module are divided to obtain N groups of first dimensions, where N is an integer and N≥2; Each of the N groups of first dimensions is sequentially upgraded and reduced in dimensionality to obtain N groups of second dimensions; The N sets of second dimensions are concatenated to obtain a third dimension, which is the same as the dimension of the data input to the feedforward layer.

11. A data processing device, characterized in that, include: Processor, memory; The processor and the memory communicate with each other; The memory is used to store computer programs; The processor is configured to execute the computer program in the memory, performing the method as described in any one of claims 1-5.

12. A computer-readable storage medium comprising a computer program that, when executed on a processor, causes the processor to perform the method as described in any one of claims 1-5.

Images