Data processing method, data processing apparatus, and computer program product

Abstract

To provide a data processing method, a data processing apparatus and a program for improving the quality of output data when input data has a large data length.SOLUTION: A data processing method includes the steps of: generating, by a first neural network model, a first number of first hidden state vectors larger than 1, based on input data, each vector corresponding to one numerical value position on one predetermined distribution curve; mapping each of the first number of first hidden state vectors corresponding to one numerical value position on the one predetermined distribution curve, on the one predetermined distribution curve; and performing sampling on each of the first number of predetermined distribution curves to obtain a first numerical value group including a first number of numerical values each corresponding to one numerical value position on the one predetermined distribution curve; and generating, by a second neural network model, output data based on the first numerical value group. Each of the first number of first hidden state vectors represents a part of information included in the input data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the field of data processing, and more particularly to a data processing method, a data processing device, and a computer program product, which may be utilized for data augmentation. [Background technology]

[0002] Currently, deep learning models have achieved results in many fields, but because training deep learning models relies on large amounts of labeled data, problems of poor performance of deep learning models often occur due to a lack of labeled data or an imbalance in the distribution of data.Data augmentation is a technology that automatically expands training data, and can improve the performance of deep learning models by reducing the cost of acquiring labeled data. Summary of the Invention [Problem to be solved by the invention]

[0003] The object of the present disclosure is to provide a data processing method, a data processing device, and a computer program product that can generate different output data corresponding to input data based on the input data, while ensuring diversity in the output data and improving the quality of the output data when the input data is long. [Means for solving the problem]

[0004] According to one aspect of the present disclosure, there is provided a data processing method including: a first neural network model generating a first number of first hidden state vectors greater than 1 based on input data; mapping the first number of first hidden state vectors to the first predetermined numerical distribution curves, each of the first number of first hidden state vectors corresponding to a numerical position on the predetermined distribution curve; sampling each of the first number of predetermined distribution curves to obtain a first group of numbers, the first group of numbers including the first number of numerical values, each of the numerical values corresponding to a numerical position on the predetermined distribution curve; and a second neural network model generating output data based on the first group of numbers, wherein each of the first number of first hidden state vectors represents a portion of information contained in the input data, and the information represented by each of the first hidden state vectors does not completely overlap.

[0005] According to one embodiment of the present disclosure, the first neural network model includes an encoding unit that encodes the input data and generates a second hidden state vector, and the first number of information extraction units that extract a portion of information included in the second hidden state vector and generate each of the first hidden state vectors.

[0006] According to one embodiment of the present disclosure, the information extractor generates the first hidden state vector using an attention mechanism.

[0007] According to one embodiment of the present disclosure, the first number of first hidden state vectors are respectively mapped to the first number of predetermined distribution curves by an invertible generative model.

[0008] According to one embodiment of the present disclosure, the reversible generative model is a flow-based model or a diffusion model.

[0009] According to one embodiment of the present disclosure, the first number of first hidden state vectors are respectively mapped to the first number of predetermined distribution curves by a Variational AutoEncoder (VAE).

[0010] According to one embodiment of the present disclosure, the input data is text data.

[0011] According to an embodiment of the present disclosure, the first group of values is different from a group of values corresponding to a numerical position on a predetermined distribution curve of the first number of the input data.

[0012] According to another aspect of the present disclosure, there is provided a data processing device including a processor and a memory having one or more computer programs stored therein, wherein when the one or more computer programs are executed by the processor, the processor performs the above-described data processing method.

[0013] According to another aspect of the present disclosure, there is provided a computer program product including a computer program which, when executed by a processor, causes the processor to perform the data processing method described above.

[0014] According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the data processing method described above. [Effects of the Invention]

[0015] According to the data processing method, data processing device, computer program product, and non-transitory computer-readable storage medium provided by the present disclosure, in the data processing process, different information contained in the input data is represented based on multiple first hidden state vectors, so that more information contained in the input data can be retained and the quality of output data can be improved when the input data length is long.In addition, multiple first hidden state vectors can be mapped to different predetermined distribution curves and sampled according to the predetermined distribution curves, so that diversity of data results can be achieved. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic diagram of an exemplary system for implementing a data processing method according to an embodiment of the present disclosure. [Figure 2] FIG. 2 shows a flowchart of a data processing method according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a flowchart illustrating generation of a first hidden state vector by a first neural network model according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a flowchart illustrating a second neural network model generating output data based on a first group of values according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is a flowchart illustrating a process for training a model used in a data processing method according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a schematic diagram of a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although several embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure may be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather, these embodiments are provided to enable a more thorough and complete understanding of the present disclosure. It should be understood that the accompanying drawings and embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] As used herein, the term "comprises" and variations thereof are open-ended, i.e., "including, but not limited to." The term "based on" means "based at least in part on." The term "in one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one other embodiment," and "some embodiments" means "at least some embodiments." Definitions of other terms are provided in the description below.

[0019] It should be understood that the concepts of "first", "second", etc. referred to in this disclosure are merely intended to distinguish different devices, modules or units, and are not intended to limit the order of functions performed by these devices, modules or units or their dependencies on each other.

[0020] It should be noted that the modifications "a" and "a plurality" referred to in this disclosure are exemplary rather than limiting, and it should be understood by those skilled in the art that they should be understood as "one or more" unless the context clearly dictates otherwise.

[0021] Existing data augmentation methods have issues, such as poor quality of the generated augmented data when the diversity of the generated augmented data is low or when the length of the input data is long. For example, when using existing data augmentation methods, only one augmented data may be generated from one input data, resulting in poor data diversity. Furthermore, when the input data is text and the length of the text is long, the text obtained using existing data augmentation methods may contain a sequence of characters whose meaning is difficult to understand, which reduces the quality of the augmented data and makes it unusable as effective data.

[0022] The present disclosure has been made in view of the above-mentioned problems. According to a data processing method, a data processing device, a computer program product, and a non-transitory computer-readable storage medium according to embodiments of the present disclosure, it is possible to realize diversity of the augmented data and improve the quality of the augmented data.

[0023] The idea of the present disclosure is that multiple hidden state vectors are set to extract different information contained in the input data, mapped to a predetermined distribution curve, sampled against the predetermined distribution curve, and the sampling results are decoded to obtain output data.

[0024] FIG. 1 is a schematic diagram of an exemplary system for implementing a data processing method according to an embodiment of the present disclosure.

[0025] As shown in Fig. 1, a system for implementing a data processing method according to an embodiment of the present disclosure may include a first neural network model 110 and a second neural network model 120. The first neural network model 110 processes input information to generate a plurality of first hidden state vectors. For ease of explanation, the following description will be given assuming that the number of the plurality of first hidden state vectors is two. Accordingly, Fig. 1 illustrates a system for implementing a data processing method according to an embodiment of the present disclosure, including two first hidden state vectors h 112 , h 113As an example, the first neural network model 110 may include an encoding unit 111 and two information extracting units 112 and 113 to perform the above operations. 112 , h 113 may be mapped to two predetermined distribution curves 130, 131, respectively, and each of the first hidden state vectors may correspond to one numerical position on the predetermined distribution curve. For example, the second neural network model 120 includes hidden state vector mapping units 122, 123, and the first hidden state vector h 112 and h 113may be mapped to a numerical position on the predetermined distribution curves 130, 131 (represented by a small hollow circle on the predetermined distribution curves 130, 131 in FIG. 1 ). As a result, a combination of a numerical position on the predetermined distribution curve 130 and a numerical position on the predetermined distribution curve 131 represents a group of two first hidden state vectors, and the group of two first hidden state vectors may correspond to one input data (e.g., one text). Further, by sampling the two predetermined distribution curves 130, 131, a first group of numbers (represented by a filled dot on the predetermined distribution curves 130, 131 in FIG. 1 ) may be obtained. The first group of numbers may include two numerical values, each of which may correspond to one numerical position on one predetermined distribution curve. The second neural network model 120 may process based on the first group of numbers to generate output data. The second neural network model 120 may include, for example, a decoding unit 121 and the two hidden state vector mapping units 122 and 123 described above, and may generate output data based on the sampled group of values. The hidden state vector mapping units 122 and 123 may be, for example, a reversible generative model that can map the first hidden state vector to a numerical position on a predetermined distribution curve and generate the first hidden state vector from the numerical position on the predetermined distribution curve. As a result, the hidden state vector mapping units 122 and 123 can convert the numerical values included in the sampled first group of values into the first hidden state vector. Thereafter, the decoding unit 121 may decode the first hidden state vector obtained by the conversion as output data. The decoding unit 121 may be any appropriate neural network model, for example, a decoder. In the embodiment of the present disclosure, the numerical values included in the first group of values may correspond to any numerical position on the predetermined distribution curves 130 and 131. In one embodiment, for the correlation of the output data with the input data, for example, a predetermined offset may be set as the relationship between the numerical position to which the first group of values corresponds and the numerical position to which the input data is mapped on a predetermined distribution curve.

[0026] It should be understood by those skilled in the art that FIG. 1 is merely an example of a system configuration for implementing embodiments of the present disclosure, and that other system configurations may be used to implement embodiments of the present disclosure. For example, the hidden state vector mapping units 122 and 123 are not necessarily reversible generative models, but may be capable of mapping a first hidden state vector to a predetermined distribution curve and mapping a sample on the predetermined distribution curve to the first hidden state vector. The hidden state vector mapping units 122 and 123 may not be part of the second neural network model 120. The second neural network model 120 may not require the hidden state vector mapping units 122 and 123 to obtain the first hidden state vector in advance, but may instead directly generate output data from a predetermined distribution curve. The first neural network model 110 does not necessarily include an encoding unit and multiple information extraction units, but may be capable of processing input information and generating multiple first hidden state vectors.

[0027] The neural network model according to the present disclosure may employ various known models and may be trained in a conventional manner.

[0028] According to an embodiment of the present disclosure, multiple first hidden state vectors are generated based on input data and mapped to multiple predetermined distribution curves. The multiple distribution curves represent different information contained in the input data, thereby retaining more information contained in the input data and improving the quality of output data when the input data length is long. Furthermore, by sampling different predetermined distribution curves as needed, diversity in data processing results can be achieved. Furthermore, because various types of data can be generated quickly and diversified with high quality, data can be generated as needed without having to store a large amount of data in advance, and the amount of storage space required by the computer can be reduced.

[0029] 2 is a flowchart of a data processing method 200 according to an embodiment of the present disclosure. The data processing method 200 includes steps S210 to S240. As with FIG. 1, in the following description, for simplicity, the number of hidden state vectors is assumed to be two.

[0030] As shown in FIG. 2, in step S210, the data processing method according to the present disclosure includes: generating a first number (two) of first hidden state vectors h by a first neural network model 110 based on input data; 112 , h 113 Next, in step S220, the two first hidden state vectors are mapped to two predetermined distribution curves 130, 131, respectively. Here, each of the first number of first hidden state vectors corresponds to one numerical position on the predetermined distribution curve. Next, in step S230, sampling is performed on the two predetermined distribution curves 130, 131 to obtain a first group of values. Here, the first group of values includes two numerical values, each corresponding to one numerical position on one of the predetermined distribution curves. Next, in step S240, the second neural network model 120 generates output data based on the first group of values. The above-mentioned two hidden state vectors h 112 , h 113Each of the first hidden state vectors represents a portion of the information contained in the input data, and the information represented by each first hidden state vector does not completely overlap. As a result, two first hidden state vectors are mapped to two corresponding predetermined distribution curves, respectively, and different information contained in the input data is represented based on the two first hidden state vectors, thereby retaining more information contained in the input data. This allows more information to be used when generating output data, improving the quality of the generated data, which is particularly advantageous when the input data is long. For example, if the input data is long text data, the information in the input data may be, for example, nouns, verbs, or subject-predicate relationships in the text. Since multiple first hidden state vectors are used and mapped to multiple predetermined distribution curves, much of this information is retained, and the generated output text may have a character order that allows for better semantic understanding. Furthermore, sampling can be performed as needed for different predetermined distribution curves, thereby achieving diversity in data processing results.

[0031] In one embodiment, the input data may be text data. More specifically, the input data may be a sentence. Here, a sentence is merely an example, and the input data may be a word, which is shorter than a sentence, or a paragraph, which is longer than a sentence. In this case, the input data may be preprocessed before executing method 200. When executing the data processing method by a computer, since the computer cannot process a sentence in natural language as is, the sentence must be converted into a vector format. For example, each word in the sentence may be converted into a vector with a specific number of dimensions through an embedding operation. Alternatively, the sentence may be segmented into words, and then each of the segmented words may be converted into a vector with a specific number of dimensions. When a sentence is processed using the data processing method of the present disclosure, the process of converting the sentence in natural language format into a vector format may be considered preprocessing. Since the embedding operation may be performed using an existing method, a detailed description thereof will be omitted here. In the following, to more accurately describe the data processing method of the present disclosure, no distinction is made between whether the input sentence is in natural language format or vector format. Those skilled in the art may, if necessary, convert the sentences into vector format in any possible way and then process them using the data processing method according to the present disclosure.

[0032] If the input data is text data, the data processing method 200 takes, for example, a sentence as input data, and in step S210, processes the input data using the first neural network model 110 to generate a first number of first hidden state vectors h 112 , h 113 Step S210 in FIG. 2 will be described in detail below.

[0033] As described above, in the present disclosure, the number of first hidden state vectors (i.e., the first number) is described as 2 for ease of explanation, but is not limited to 2 and may be any integer equal to or greater than 1. The first number may be set in the training stage. In one embodiment, the first number may be set or adjusted as a hyper-parameter according to the data length in the training dataset.

[0034] As described above, in order to improve the quality of the output data, the data processing method of the present disclosure attempts to extract information from the input data based on multiple first hidden state vectors to generate output data, so it should ensure that the information of the input data extracted by each first hidden state vector does not completely overlap.

[0035] In one embodiment, the input data is divided into different parts, and the first neural network model 110 includes an encoding unit 111, which encodes different parts of the input data and obtains different encoded vectors as different first hidden state vectors, thereby realizing that each of the first hidden state vectors extracts different information from the input information.

[0036] In another embodiment, the first neural network model 110 may further include a first number (e.g., two) of information extraction units 112 and 113 in addition to the encoding unit 111. The information extraction units of the first neural network model may be configured such that the information extraction units 112 and 113 each extract a different part of the encoded vector output from the encoding unit 111, or each extract a different part of the encoded vector output from the encoding unit 111 with a different weight, thereby generating different first hidden state vectors. For example, the information extraction units 112 and 113 may be associated with different weight vectors and perform operations using the different weight vectors and the encoded vector of the input data to generate different first hidden state vectors, so that the information of the input data extracted based on each first hidden state vector does not completely overlap.

[0037] For example, the information extraction units 112 and 113 may set different tags, and a self-attention mechanism between the tag vectors may be used to ensure that the tag vectors are different from each other. Then, a cross-attention mechanism may be used to calculate the tag vector and the input data encoding vector, thereby generating first hidden state vectors so that the information of the input data extracted by each first hidden state vector does not completely overlap. Note that the above-described embodiment is merely an example, and those skilled in the art may employ a variety of methods to ensure that the information of the input data extracted by each first hidden state vector does not completely overlap.

[0038] 3 shows an example flowchart of generating a first hidden state vector by a first neural network model according to an embodiment of the present disclosure. As shown in FIG. 3, the above-mentioned step S210 may include substeps S211 to S213. In step S211, a tag vector weighted by a self-attention mechanism, generated when training the first neural network model, may be read.

[0039] Below we describe the process of generating a self-attention weighted tag vector when training the first neural network model.

[0040] Specifically, different tags are preset in each information extraction unit during the training stage. The tags may be, for example, numbers, alphabets, or other formats. For example, the information extraction unit 112 may be preset with tag "1" during training, and the information extraction unit 113 may be preset with tag "2." The tag "1" of the information extraction unit 112 and the tag "2" of the information extraction unit 113 may be processed and converted into different vectors. For example, the tags of the information extraction units 112 and 113 may be processed in the same way as the embedding operation used in the preprocessing of the input data, and the tag vector l of the information extraction unit may be converted into different vectors. 112 , l 113 where the vector of tags l 112 , l 113 The number of dimensions of can be set flexibly. For example, if "1" and "2" are used as tags, the tag vector l 112 , l 113 The number of dimensions may be set to, for example, 1*256.

[0041] Next, the vector l of the two tags 112 and 113 The self-attention mechanism is used to calculate the weighted tag vector sa 112 and sa 113 Specifically, the tag vector weighted by the self-attention mechanism can be obtained by, for example, the following equation (1).

number

[0042] The above tag vector l 112 and 113 are the first and second rows of matrix I, respectively. Since the self-attention mechanism is used here, the matrices Q, K, and V in the above formula (1) are the matrix I and matrix W, respectively. Q , W K , W V It is obtained by calculating the inner product of and can be written as follows: Q=IWq ; K=IW k ; V=IW v .

[0043] where W q , W k , W v is a parameter matrix obtained by pre-training. The number of dimensions of each parameter matrix correlates with the number of dimensions of the tag vector obtained by embedding the tag in the information extraction unit. More specifically, the number of rows of each parameter matrix is equal to the number of columns of the tag vector. For example, the tag vector l 112 If the number of dimensions of is set to 1*256, W q , W k , W v The number of dimensions of may be set to 256*x, where x is any suitable integer that can be set. T " represents the transpose matrix of K, "Softmax()" represents the softmax function, and "d k " represents the size of the number of dimensions (number of columns) of K.

[0044] In this case, Attention(Q,K,V) calculated by the above formula (1) is a matrix with 2*x dimensions. The first row of the matrix is the tag vector sa weighted by the self-attention mechanism. 112 The second row is the weighted tag vector sa 113 becomes.

[0045] The first number and the tags of each information extraction unit are determined during the training phase and remain unchanged when generating output data using the data processing method of the present disclosure. Furthermore, the parameter matrices used to calculate the weights used by the self-attention mechanism are determined at the end of the training phase and remain unchanged when generating output data using the data processing method of the present disclosure. Therefore, the tag vectors weighted by the self-attention mechanism may be stored at the end of the training phase and read out and used for subsequent processing when generating output data using the data processing method.

[0046] Returning to FIG. 3, in step S211, the tag vector sa 112 and sa 113 After being read, in step S212, the encoding unit 111 encodes the input data and generates a second hidden state vector h 111 Generate.

[0047] In one embodiment, the encoding unit 111 includes a pre-trained neural network that encodes the input data and generates a second hidden state vector h 111 The second hidden state vector h 111 The number of dimensions may be correlated with the data length of the input data. For example, if the input data is a sentence consisting of seven characters, the number of dimensions of the second hidden state vector may be 7*256, where the number of columns of the second hidden state vector is determined when performing the embedding operation on the input data and is equal to the number of columns of the vector obtained by performing the embedding operation on the tag of the information extraction units 112 and 113 described above.

[0048] The encoding unit 111 may include any neural network capable of encoding input data, such as, but not limited to, a recurrent neural network (RNN), a bidirectional RNN (BiRNN), a long short-term memory (LSTM) network, a bidirectional LSTM (BiLSTM) network, etc. In the present disclosure, the type and configuration of the neural network of the encoding unit are not particularly limited.

[0049] Here, the second hidden state vector h 111 Step S212, where the weighted tag vector sa is generated by the self-attention mechanism, is 112 and sa 113 may be executed before step S211, which is the step from which the step S211 was read. Also, step S211 and step S212 may be executed in parallel.

[0050] Then, in step S213, the information extraction units 112 and 113 extract the tag vector sa weighted by the self-attention mechanism read in the above-mentioned step S211. 112 and sa 113 and the second hidden state vector h generated in step S212 above. 111 Based on the cross-attention mechanism, we obtain a tag vector sa with weights by the self-attention mechanism. 112 and sa 113 and the second hidden state vector h 111 The weights of the cross-attention mechanism are calculated for each of the first hidden state vector h 112 and h 113 Generate.

[0051] Specifically, the tag vector sa with weights added by the self-attention mechanism 112 and sa 113 are the first and second rows of the matrix J, respectively. The weighted tag vector sa 112 and sa 113 (the matrix J consisting of the second hidden state vector h 111 In this case, Q in the above formula (1) is the second hidden state vector h 111 and vector W Q ' and V are obtained by performing an inner product operation on the matrices J and W, respectively. k ', W v ' and ', and can be written as follows: Q=h 111 W q '; K=JW k '; V=JW v '.

[0052] where W q ', W k ', W v' is a parameter matrix obtained by pre-training. The number of dimensions of each parameter matrix correlates with the number of dimensions of the tag vector obtained by embedding the tag of the information extraction unit and the number of dimensions of the vector obtained by embedding the input data, and more specifically, may be equal to the number of columns of the tag vector and the second hidden state vector. For example, if the number of dimensions of the second hidden state vector is 7*256, W q , W k , W v The number of dimensions of may be set to 256*x, where x is any suitable integer that can be set.

[0053] In this case, the Attention(Q,K,V) calculated using the above formula (1) is a matrix with a dimension of 2*x. The first row of the matrix is the first hidden state vector h 112 and the second row is the first hidden state vector h 113 becomes.

[0054] Note that the above-mentioned formula (1) for calculating self-attention and cross-attention is only an example, and the tag vector sa 112 ,sa 113 and / or the first hidden state vector h 112 , h 113 may be calculated by other appropriate formulas. Furthermore, the above-mentioned numbers of dimensions of each matrix and vector are merely examples, and each matrix and vector may have other appropriate numbers of dimensions.

[0055] In one embodiment, a first number of first hidden state vectors h 112、 h 113 Each of the first hidden state vectors represents a portion of the information contained in the input data, and the information represented by each of the first hidden state vectors does not completely overlap.

[0056] Specifically, the neural network has the characteristic of being inactive during the training stage, and does not repeatedly learn features that have already been acquired. Therefore, in the training stage, different tags are assigned to each information extraction unit, and in step S211, a tag vector weighted by the self-attention mechanism is read out. In step S213, the tag vector weighted by the self-attention mechanism and the second hidden state vector generated by the encoding unit are calculated based on a cross-attention mechanism to generate the first hidden state vector h 112 is the first hidden state vector h 113 In this configuration, the first hidden state vector h generated by the first neural network model 110 does not learn features that have already been learned, and vice versa. 112 and h 113 Unlike the first number, each has its own unique characteristics and can be guaranteed to represent different information contained in the input data. When the first number is appropriately set according to the size of the data in the dataset, it can improve the expressiveness of the input data and improve the quality of the output data when the input data length is long, compared to other data processing methods.

[0057] Returning to Figure 2, in step S210, the first neural network model 110 generates a first number of first hidden state vectors h 112 and h 113 After generating the first hidden state vector h, in step S220, 112 , h 113 is mapped to the first number of predetermined distribution curves, and the first number of first hidden state vectors h 112 , h 113 Each of the hidden state vectors corresponds to a numerical value position on a predetermined distribution curve. The second neural network model 120 may include a first number (e.g., two) of hidden state vector mapping units (122, 123 in FIG. 1). Step S220 will be described below.

[0058] In one embodiment, the predetermined distribution curve may be a normal distribution curve, but is not limited to this and may be any other known distribution curve.

[0059] In one embodiment, a first number of first hidden state vectors h are generated based on a reversible generative model. 112 , h 113 may be mapped to a first number of predetermined distribution curves. In other words, each of the hidden state vector mapping units 122 and 123 may be a reversible generative model. In this embodiment, the reversible generative model and the variational autoencoder described below are examples of hidden state vector mapping units. As described above, in one embodiment, the second neural network model 120 may include a first number (two) of such hidden state vector mapping units (122 and 123 in FIG. 1).

[0060] In one embodiment, the reversible generative model may be a flow-based model or a diffusion model.

[0061] In another embodiment, the first number of first hidden state vectors h 112 , h 113 may be mapped onto a first number of predetermined distribution curves based on a Variational AutoEncoder (VAE). In other words, each of the hidden state vector mapping units 122 and 123 may be a VAE.

[0062] Here, the flow-based model, the diffusion model, and the VAE all use the first hidden state vector h 112 , h 113 to a predetermined distribution curve. In this embodiment, other models that can achieve the above-mentioned objective may be used, and multiple variations of the flow-based model, the diffusion model, and the VAE may be used.

[0063] In the following, an example will be described in which the hidden state vector mapping units 122 and 123 are flow-based models and the predetermined distribution curve is a normal distribution curve.

[0064] In the example of FIG. 1, in the second neural network model 120, the first hidden state vector h 112 , h 113 may be mapped to normal distribution curves 130 and 131. Here, the flow-based models 122 and 123 are pre-trained. In this case, the flow-based models can receive one first hidden state vector as input and output one numerical position on the normal distribution curve, with each first hidden state vector corresponding to one numerical position. When the first neural network model 110 receives input data, the numerical position on each normal distribution curve of a first hidden state vector generated based on the input data is different from the numerical position on the same normal distribution curve of a first hidden state vector generated based on another input data.

[0065] For example, the first two hidden state vectors generated based on the input data a are denoted by h 112,a and h 113,a and the flow-based model 122 is h 112,a The numerical position where the normal distribution curve 130 is mapped is g 130,a The flow-based model 123 is 113,a The numerical position where the normal distribution curve 131 is mapped is g 131,a Also, let h be the first hidden state vector generated based on input data b that is different from input data a. 112,b and h 113,b and the flow-based model 122 is h 112,b The numerical position where the normal distribution curve 130 is mapped is g 130,b The flow-based model 123 is 113,b The numerical position where the normal distribution curve 131 is mapped is g 131,b In this case, g 130,a and g 130,bUnlike and / or g 131,a and g 131,b is different.

[0066] Returning to FIG. 2, in step S220, a first number of first hidden state vectors h 112 , h 113 After each of the values is mapped to a first number of predetermined distribution curves, in step S230, sampling is performed on each of the first number of predetermined distribution curves to obtain a first group of values. The first group of values includes the first number of values, each of which corresponds to a numerical position on one of the predetermined distribution curves. In the following, step S230 will be described assuming that the predetermined distribution curve is a normal distribution curve.

[0067] The first group of values may be different from the corresponding group of values in the normal distribution curve of the first number of input data, i.e., the first group of values may be sampled at a different numerical position on each normal distribution curve from the numerical positions of the corresponding first hidden state vectors of the current input data mapped to each normal distribution curve.

[0068] Specifically, for example, if the current input data is a, and the flow-based model 122 is h 112,a The numerical position where the normal distribution curve 130 is mapped is g 130,a The flow-based model 123 is 113,a The numerical position where the normal distribution curve 131 is mapped is g 131,a When sampling, the numerical position g on the normal distribution curve 130 130,a Numerical position g that is different from 130,a ' and on the normal distribution curve 131, the numerical position g 131,a Numerical position g that is different from 131,a In this case, the numerical group corresponding to the numerical position on the normal distribution curves 130 and 131 of the input data a is [g 130,a ,g 131,a ] and the first group of numbers sampled is written as [g 130,a ',g 131,a'] may be written as

[0069] In one embodiment, sampling may be performed at numeric positions that are offset based on a random number.

[0070] For example, by generating non-zero random numbers, we can create a group of numbers [g 130,a ,g 131,a ] may be added to, subtracted from, or multiplied by the random number to form the elements of the first numerical group. 130,a ,g 131,a ] and the first random number generated is [r1, r2],

number

[0071] Additionally, sampling may be performed on value positions that are close to each member of the group of values.

[0072] In this case, for example, the range of the value of each element of the random numbers [r1, r2] may be limited.

number

[0073] Furthermore, instead of using random numbers, the magnitude of the offset amount for the numerical position of the numerical group when sampling may be specified.

[0074] The sampling method to be adopted may be flexibly selected depending on the purpose of application of the data processing method, the size of the data set, and the like.

[0075] Returning to FIG. 2, after the first group of values is obtained in step S230, output data is generated based on the first group of values by the second neural network model 120 in step S240. Step S240 will be described below with reference to FIG. 4. FIG. 4 is a flowchart showing how the second neural network model generates output data based on the first group of values. As shown in FIG. 4, step S240 may specifically include substeps S241 to S242.

[0076] First, in step S241, a first group of sampled values representing positions on a normal distribution curve is converted into a sampled and restored first hidden state vector h 112 , h 113 Here, "sampling restoration" refers to the process in which the hidden state vector mapping unit generates a first hidden state vector from the numerical position on the normal distribution curve. In the above example, the first numerical group [g 130,a ',g 131,a After obtaining '], the flow-based model 122 calculates g 130,a The first hidden state vector h is sampled and restored based on 112 The flow-based model 122 generates g 131,a The first hidden state vector h is sampled and restored based on 113 may be generated.

[0077] Next, in step S242, the output data is the resampled first hidden state vector h 112 , h 113 Here, the second neural network model 120 may include a decoding unit 121.

[0078] The decoding unit 121 receives a pre-trained, resampled first hidden state vector h 112 , h 113The neural network used in the decoding unit 121 may include, but is not limited to, any neural network capable of decoding the input signal, such as a recurrent neural network (RNN), a bidirectional RNN (BiRNN), a long short-term memory (LSTM) network, a bidirectional LSTM (BiLSTM) network, etc. In the present disclosure, the type and configuration of the neural network used in the decoding unit 121 are not particularly limited.

[0079] In the above-described step S230, a numerical value position on the predetermined distribution curve different from the numerical value position corresponding to the input data is sampled, so that the output data obtained by the decoding unit 121 can be made different from the input data of the data processing method, and data expansion can be achieved. Furthermore, since any position on the predetermined distribution curve can be sampled and the output data differs depending on the sampling position, the data processing method of the present disclosure can achieve diversity in output data.

[0080] The data processing method according to the present disclosure has been described above with reference to the drawings. However, the encoding unit 111, the information extraction units 112 and 113 included in the first neural network model 110, and the hidden state vector mapping units 122 and 123 and the decoding unit 121 included in the second neural network model 120 used in the above-described method are pre-trained. Below, a process 500 for training the models used in the data processing method according to the present disclosure will be described with reference to FIG. 5. Detailed descriptions of steps that are the same as those already described above will be omitted below.

[0081] 5, first, in step S510, a first number is set, and tags are set for the first number of information extraction units 112, 113, and the tags of the information extraction units are different from each other. Here, the first number correlates with the length of the input data, and for example, when the input data is text data, the longer the length of the text in the training dataset, the larger the first number may be set.

[0082] Next, in step S520, an embedding operation is performed on the tags of the information extraction units 112 and 113, and a vector of the tags is generated.

[0083] Next, in step S530, each information extraction unit performs calculations based on the self-attention mechanism for the tag vector obtained in step S520, and generates a tag vector weighted by the self-attention mechanism. The calculation process here is the same as described above. However, in the process of calculating the weights by the self-attention mechanism, W q , W k , W v is the parameter matrix that is the training target, and the values of the parameter matrix may be initially set randomly and continuously updated during the training phase. The dimensionality of each parameter matrix correlates with the dimensionality of the tag vector.

[0084] Then, in step S540, the encoding unit 111 included in the first neural network model 110 encodes the training input data to generate a second hidden state vector of the training stage. Here, the parameters of the encoding unit 111 are preset initial parameters, which may be continuously updated during the training stage.

[0085] The step of generating a tag vector weighted by the self-attention mechanism and the step of generating a second hidden state vector may be performed simultaneously or in the order of precedence and succeeding. That is, step S540 may be performed before steps S520 and S530, or may be performed simultaneously with steps S520 and S530. Thereafter, in step S550, each information extractor performs a calculation on the tag vector weighted by the self-attention mechanism and the second hidden state vector based on the cross-attention mechanism to generate the first hidden state vector h of the training stage. 112 , h 113 However, in the process of calculating the weights by the self-attention mechanism, W q ', W k ', Wv The value of ' may be set randomly initially and continuously updated during the training phase. The dimensionality of each parameter matrix correlates with the dimensionality of the tag vector.

[0086] Then, in step S560, the second neural network model 120 (specifically, the hidden state vector mapping units 122 and 123) maps the first hidden state vector h 112 , h 113 , respectively, to a first number of predetermined distribution curves, and a first number of first hidden state vectors h 112 , h 113 Each of the values corresponds to a numerical position on a given distribution curve.

[0087] Next, in step S570, the hidden state vector mapping units 122 and 123 are sampled for the first number of corresponding numerical positions, i.e., without offsetting the numerical positions, to generate the sampled and restored first hidden state vector h 112 , h 113 The parameters of the hidden state vector mapping units 122 and 123 are preset initial parameters, which may be continuously updated during the training stage.

[0088] Here, the first loss function loss1 is calculated. Ideally, the numerical position on the normal distribution curve is mapped to the first hidden state vector h by the hidden state vector mapping units 122 and 123. 112 , h 113and then re-sampling the numerical position, the resulting output vector (the restored first hidden state vector) should be the same as the original first hidden state vector. In practice, when the hidden state vector mapping units 122 and 123 convert the first hidden state vector into a numerical position on a normal distribution curve, and when the first hidden state vector is generated based on the sampled numerical position on the normal distribution curve, there may be a loss of information. The first loss function loss1 indicates the difference between the first hidden state vector obtained by the hidden state vector mapping units 122 and 123 through the restoration of sampling and the original first hidden state vector. The first loss function loss1 may be an existing loss function depending on the type of neural network model adopted in the hidden state vector mapping units.

[0089] Next, in step S580, the first hidden state vector of the training stage (i.e., the first hidden state vector that has not been processed by the hidden state vector mapping units 122 and 123) is input directly to the decoding unit 121, and the decoding unit 121 decodes the first number of first hidden state vectors of the training stage to generate training stage output data. Here, the parameters of the decoding unit 121 are preset initial parameters that may be continuously updated during the training stage. Here, a second loss function loss2 is calculated. The output data of the training stage is compared with the input data, and the second loss function loss2 indicates the difference between the output data of the training stage and the input data. As the second loss function loss2, an existing loss function may be adopted depending on the type of neural network model adopted in the encoding unit and the decoding unit.

[0090] As described above, when training a model used in the data processing method, the overall loss function may be composed of two parts. The training goal is to minimize the two loss functions, i.e., the first loss function loss1 and the second loss function loss2. Then, in step S590, the parameters of each of the above parts are updated based on backpropagation (BP).

[0091] In one embodiment, the first loss function loss1 and the second loss function loss2 may be optimized separately.

[0092] In another embodiment, these two loss functions may be combined. For example, the sum of the first loss function loss1 and the second loss function loss2 may be used as the overall optimization goal of the model. Alternatively, the weighted sum of the first loss function loss1 and the second loss function loss2 may be used as the overall optimization goal of the model. In this case, the weights of the first loss function loss1 and the second loss function loss2 may be adjusted based on the convergence speed of each model during training.

[0093] The above steps S530 to S590 may be repeatedly performed until the loss function falls to a preset training target threshold or a preset number of iterations have been performed.

[0094] As described above, the data processing method according to one embodiment of the present disclosure has been described in detail.

[0095] According to another aspect of the present disclosure, there is further provided a data processing apparatus: Figure 6 shows a schematic diagram of a data processing apparatus 600 according to an embodiment of the present disclosure.

[0096] 6, a data processing device 600 according to an embodiment of the present disclosure includes a processor 610 and a memory 620. The memory 620 stores one or more computer programs.

[0097] The processor 610 is a program control device such as a microprocessor, and operates according to a program installed in, for example, memory 620. The memory 620 is a storage element such as a ROM or RAM. The program executed by the processor 610 is stored in the memory 620. The data processing device 600 shown in FIG. 6 may be used to implement the data processing method disclosed herein.

[0098] According to an embodiment of the present disclosure, a data processing device can represent different information contained in input data based on a first number of first hidden state vectors. When an appropriate first number is set according to the data length of the data set, the expressiveness for the input data can be improved, and the quality of output data can be improved when the input data length is long. Since numerical positions on a predetermined distribution curve that are different from the numerical positions corresponding to the input data are sampled, the output data obtained by the decoding unit can be different from the input data of the data processing method, thereby enabling data expansion. Furthermore, since the sampled positions may be different, the output data may also be different, thereby achieving diversity in the output data.

[0099] According to an embodiment of the present disclosure, there is further provided a computer program product including a computer program, which, when executed by a processor, performs the data processing method disclosed herein.

[0100] According to an embodiment of the present disclosure, there is further provided a non-transitory computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the data processing method disclosed herein.

[0101] In the above-described embodiment, the word vector form of each word in a sentence is used as input for subsequent processing. However, it is also possible to first convert each word in a sentence into phonemes and then input the word vector form of the phonemes for each word. A phoneme is the smallest unit of speech divided based on the natural properties of speech, and a sentence can be converted into a sequence of phonemes in several ways. For example, when converting a sentence into a sequence of phonemes using the Python language, a Chinese sentence can be converted into a sequence of pinyin using the "pypinyin" library, an English sentence can be converted into a sequence of phonetic transcription using the "nltk" library, and a Japanese sentence can be converted into a sequence of romanized characters using the "romkan" library. The above-described methods for converting a sentence into phonemes are merely exemplary, and those skilled in the art can convert a sentence into phoneme format using any possible method; the present disclosure is not particularly limited thereto. In this case, the augmented sentence generated by the data processing method can be made different from the input sentence at the phoneme level, thereby obtaining an augmented result that is closer to the input sentence. This method of converting each word of a sentence into phonemes and processing them can be used, for example, to improve the performance of an automatic speech recognition (ASR) system. Specifically, when a conventional speech recognition system receives a speech signal as input and outputs text as a speech recognition result, the text may be incorrectly recognized, necessitating an ASR post-processing module to correct the recognized erroneous text. The post-processing module is obtained through training and requires text pairs such as <correct text, incorrect text> as training data. However, constructing a dataset of <correct text, incorrect text> text pairs requires a large amount of computational power and effort. Therefore, by automatically generating the corresponding incorrect text based on the correct text using the data processing method disclosed herein, the <correct text, incorrect text> text pairs can be constructed, reducing the cost of constructing a training dataset for the post-processing module and thereby improving the post-processing module's ability to correct erroneous text.The augmented sentences generated by the data processing method and the input sentences differ at the phoneme level, which makes the augmented sentences more suitable for speech recognition systems.

[0102] In the above-described embodiments, text data has been used as an example, but it should be understood that the data processing method according to the present disclosure is not limited to text data and can also be applied to other data such as image data. When the input data is image data, applying the data processing method according to the present disclosure makes it possible to generate an output image that differs from the input image.

[0103] In the above-described embodiment, steps S210 to S220 are performed to generate augmented data corresponding to the input data. In some other cases, the data processing method according to the present disclosure may omit steps S210 to S220 described above. That is, the data processing method according to the present disclosure may employ only a first number of pre-trained hidden state vector mapping units and a decoding unit. The method may sample each predetermined distribution curve, input a number of numerical groups including a first number of numerical positions to each hidden state vector mapping unit to obtain a first number of sampled and restored first hidden state vectors, and then decode these sampled and restored first hidden state vectors using a decoding unit to generate output data. In this case, the output data generated by the data processing method according to the present disclosure may be applied to cases where a certain type of data needs to be augmented because there is no corresponding input data, for example, when the data set is small or the distribution of different types of data in the data set is unbalanced.

[0104] The entire hardware computing device described in this disclosure, or parts thereof, may be implemented by various suitable hardware means, including, but not limited to, FPGA, ASIC, SoC, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The devices, apparatus, methods, and systems disclosed herein are not limited to any particular hardware architecture or configuration. Components in the disclosed devices, apparatus, and systems may be separate, integrated, combined in different ways, and / or replaced or supplemented by other components. It should be understood that the teachings of this disclosure may be embodied in various forms of hardware, software, firmware, special purpose processors, or combinations thereof.

[0105] Block diagrams of devices, apparatus, methods, and systems according to the present disclosure are merely exemplary and are not intended to require or imply that they must be connected, arranged, or configured in the manner illustrated in the block diagrams. Those skilled in the art will recognize that these circuits, apparatus, devices, apparatus, and systems may be connected, arranged, or configured in any manner that achieves a desired purpose.

[0106] In the above description, the present invention has been described based on the embodiments. These embodiments have been described merely as examples, and it should be understood by those skilled in the art that various modifications are possible to the combination of the components and processes of the embodiments, and such modifications are also within the scope of the present invention.

Claims

1. a first neural network model generating a first number of first hidden state vectors greater than one based on the input data; mapping the first number of first hidden state vectors to the first numerical predetermined distribution curves, each of the first number of first hidden state vectors corresponding to a numerical position on a predetermined distribution curve; sampling each of the first number of predetermined distribution curves to obtain a first group of values, the first group of values including the first number of numerical values, each of the numerical values corresponding to a numerical position on one of the predetermined distribution curves; a second neural network model generating output data based on the first group of values; each of the first number of first hidden state vectors represents a portion of information contained in the input data; A data processing method in which the information represented by each of the first hidden state vectors is completely non-overlapping.

2. The first neural network model is an encoding unit that encodes the input data to generate a second hidden state vector; The data processing method according to claim 1 , further comprising: the first number of information extractors extracting a portion of information contained in the second hidden state vector to generate each of the first hidden state vectors.

3. The data processing method according to claim 2 , wherein the information extractor generates the first hidden state vector by utilizing an attention mechanism.

4. The data processing method of claim 1 , wherein the first number of first hidden state vectors are respectively mapped to the first number of predetermined distribution curves by a reversible generative model.

5. The data processing method according to claim 4 , wherein the reversible generative model is a flow-based model or a diffusion model.

6. 2. The data processing method of claim 1, wherein the first number of first hidden state vectors are respectively mapped to the first number of predetermined distribution curves by a variational autoencoder.

7. The data processing method according to claim 1 , wherein the input data is text data.

8. The data processing method according to claim 1 , wherein the first group of values is different from a group of values corresponding to a numerical position on a predetermined distribution curve of the first number of the input data.

9. a processor; a memory having one or more computer programs stored therein; A data processing apparatus, wherein the data processing method according to any one of claims 1 to 8 is performed by the processor when the one or more computer programs are executed by the processor.

10. A computer program product comprising a computer program, A computer program product which, when executed by a processor, performs the data processing method according to any one of claims 1 to 8.

Images