A method and device for predicting target data

Abstract

The present invention provides a method and device for predicting target data, and relates to the field of neural network technology. The method includes: obtaining multiple first target data that meet long-term dependency conditions; inputting multiple first target data into a neural network model to output second target data; the second target data has a serial correlation with multiple first target data; the neural network model is trained by the following steps: dividing the sample data set into multiple batches of input items, inputting the input items into the initial network model in the order of the batches, calculating the output items of the initial network model according to the dilated convolution algorithm, and training the initial network model according to the input items, output items and loss functions to obtain the neural network model. The method and device for predicting target data provided by the present invention use the trained neural network model to quickly predict the second target data with serial correlation for multiple first target data with ultra-long dependencies.

Description

Technical Field

[0001] The present invention relates to the field of neural network technology, and in particular to a method and device for predicting target data. Background Art

[0002] Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics, but there are important differences. Natural language processing is not a general study of natural language, but the development of computer systems that can effectively realize natural language communication, especially the software systems therein. Therefore, it is part of computer science.

[0003] Natural language processing can be applied to many technical fields. For example, in the field of speech recognition, natural language processing is performed on the previous text to predict the next text. In the field of quantitative trading, natural language processing is performed on historical prices to predict future prices. In the field of intelligent recommendation, natural language processing is performed on users' past browsing records to predict the content that users are interested in.

[0004] However, the current common natural language processing process has the problem of being unable to simultaneously take into account fast prediction and capture of very long dependencies. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide a method and device for predicting target data that overcomes the above problems or at least partially solves the above problems.

[0006] According to a first aspect of the present invention, a method for predicting target data is provided, the method comprising: obtaining a plurality of first target data satisfying a preset long-term dependency condition; inputting the plurality of first target data into a trained neural network model, and outputting second target data; wherein the second target data has a serial correlation with the plurality of first target data; the neural network model is trained by the following steps: dividing a sample data set into a plurality of batches of input items, and inputting the input items into an initial network model in the order of the batches, calculating the output items of the initial network model according to an expanded convolution algorithm, and training the initial network model according to the input items, the output items and a loss function to obtain the neural network model.

[0007] Optionally, dividing the sample data set into multiple batches of input items includes: converting each sample data in the sample data set into a sample vector; dividing all the sample vectors into multiple batches of input items according to a preset length; wherein the number of the sample data divided by the length is equal to the number of the batches.

[0008] Optionally, after dividing the sample data set into a plurality of batches of input items, the method further comprises: shifting each of the input items backward by one bit of the sample vector, and obtaining labeled data corresponding to each of the input items by dividing according to the length.

[0009] Optionally, inputting the input items into the initial network model in batch order includes: splicing the current input item with the output item of the previous network layer in the initial network model; and inputting the spliced ​​sample vector into the current network layer in the initial network model.

[0010] Optionally, the network layer of the neural network model includes multiple dilated convolutional layers, and the storage space of each dilated convolutional layer is a matrix space obtained by multiplying the length by the dimension of the sample vector.

[0011] According to the second aspect of the present invention, a device for predicting target data is provided, which includes: an acquisition module for acquiring multiple first target data that meet preset long-term dependency conditions; an input module for inputting the multiple first target data into a trained neural network model and outputting second target data; wherein the second target data has a serial correlation with the multiple first target data; the device also includes: a training module for dividing a sample data set into multiple batches of input items, and inputting the input items into an initial network model in the order of batches, calculating the output items of the initial network model according to an expanded convolution algorithm, and training the initial network model according to the input items, the output items and the loss function to obtain the neural network model.

[0012] Optionally, the training module includes: a sample conversion module, used to convert each sample data in the sample data set into a sample vector; a sample division module, used to divide all the sample vectors into multiple batches of input items according to a preset length; wherein the number of the sample data divided by the length is equal to the number of the batches.

[0013] Optionally, the training module also includes: a sample labeling module, which is used to shift each of the input items backward by one bit of the sample vector after the sample division module divides all the sample vectors into multiple batches of input items, and obtain the labeling data corresponding to each of the input items according to the length division.

[0014] Optionally, the training module includes: a sample splicing module, used to splice the current input item with the output item of the previous network layer in the initial network model; and input the spliced ​​sample vector into the current network layer in the initial network model.

[0015] Optionally, the network layer of the neural network model includes multiple dilated convolutional layers, and the storage space of each dilated convolutional layer is a matrix space obtained by multiplying the length by the dimension of the sample vector.

[0016] The embodiments of the present invention may have the following beneficial effects:

[0017] The embodiment of the present invention provides a method and device for predicting target data, which obtains multiple first target data that meet preset long-term dependency conditions, inputs the multiple first target data into a trained neural network model, and outputs second target data. The multiple first target data and the second target data have serial correlation. The multiple first target data that meet the long-term dependency conditions can be understood as multiple first target data with super-long dependency relationships. In other words, the neural network model can be used to predict second target data with serial correlation for multiple first target data with super-long dependency relationships.

[0018] Moreover, the training steps of the above neural network model may include: dividing the sample data set into multiple batches of input items, and inputting the input items into the initial network model in the order of the batches, calculating the output items of the initial network model according to the dilated convolution algorithm, and training the initial network model according to the input items, output items and loss function to obtain the neural network model. That is, in the training process of the above neural network model, the output items are calculated according to the input items using the dilated convolution algorithm. The dilated convolution algorithm has the advantage of fast calculation speed, and therefore, the above neural network model also has the advantage of fast prediction results. That is, the above neural network model can quickly predict the second target data for the first target data.

[0019] In summary, the embodiments of the present invention can use a trained neural network model to quickly predict second target data with serial correlation for multiple first target data with ultra-long dependencies.

[0020] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0022] Figure 1 A schematic flow chart of a method for predicting target data according to the first embodiment of the present invention is shown;

[0023] Figure 2 A schematic diagram showing a flow chart of a training method for a neural network model according to a second embodiment of the present invention;

[0024] Figure 3 A schematic diagram of the network structure of an initial network model according to an embodiment of the present invention is shown;

[0025] Figure 4 A schematic diagram of a network structure of a neural network model according to an embodiment of the present invention is shown;

[0026] Figure 5 A schematic diagram showing the structure of a target data prediction device according to a third embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0028] Embodiment 1

[0029] Reference Figure 1 , Figure 1 A schematic flow chart of a target data prediction method according to the first embodiment of the present invention is shown.

[0030] The target data prediction method provided by the embodiment of the present invention may specifically include the following steps.

[0031] Step S101, obtaining a plurality of first target data satisfying a preset long-term dependency condition.

[0032] In an embodiment of the present invention, the first target data may be specific data in an actual application scenario. For example, in application scenarios such as banking, securities, express delivery, and flights, the first target data may be numbers. Moreover, multiple first target data that meet the preset long-term dependency conditions may be multiple numbers with an extremely long dependency relationship, such as numbers in a bank card number, numbers in a securities transaction order number, numbers in an express order number, numbers in a flight number, etc. Generally speaking, multiple first target data that meet the long-term dependency conditions may be represented by a data sequence x(t). Where t is a natural number, and x(t) represents the tth first target data.

[0033] Step S102, inputting a plurality of the first target data into the trained neural network model, and outputting second target data.

[0034] In an embodiment of the present invention, the trained neural network model can be obtained by training the following steps: dividing the sample data set into a plurality of batches of input items, and inputting the input items into the initial network model in the order of the batches, calculating the output items of the initial network model according to the dilated convolution algorithm, and training the initial network model according to the input items, the output items and the loss function to obtain the neural network model. The above-mentioned neural network model is used to predict the second target data for the plurality of first target data, and the second target data has a serial correlation with the plurality of first target data. For example, if the plurality of first target data are the first few words in a sentence, the second target data can be the last word or the last few words in the sentence. For another example, if the plurality of first target data are the first few numbers of a bank card number, the second target data can be the last number or the last few numbers of the bank card number.

[0035] A method for predicting target object target data provided by an embodiment of the present invention obtains multiple first target object target data that meet a preset long-term dependency condition, inputs the multiple first target object target data into a trained neural network model, and outputs second target object target data. Among them, the multiple first target object target data and the second target object target data have serial correlation. The multiple first target object target data that meet the long-term dependency condition can be understood as multiple first target object target data with super-long dependency. In other words, the above-mentioned neural network model can be used to predict the second target object target data with serial correlation for the multiple first target object target data with super-long dependency.

[0036] Moreover, the training steps of the above neural network model may include: dividing the sample data set into multiple batches of input items, and inputting the input items into the initial network model in the order of the batches, calculating the output items of the initial network model according to the dilated convolution algorithm, and training the initial network model according to the input items, output items and loss function to obtain the neural network model. That is, in the training process of the above neural network model, the output items are calculated according to the input items using the dilated convolution algorithm. The dilated convolution algorithm has the advantage of fast calculation speed, and therefore, the above neural network model also has the advantage of fast prediction results. That is, the above neural network model can quickly predict the second target object target data for the first target object target data.

[0037] In summary, the embodiments of the present invention can use the trained neural network model to quickly predict second target object target data with serial correlation for multiple first target object target data with ultra-long dependencies.

[0038] Embodiment 2

[0039] Reference Figure 2 , Figure 2 A schematic flow chart of a training method for a neural network model according to a second embodiment of the present invention is shown.

[0040] The training method of the neural network model provided by the embodiment of the present invention may specifically include the following steps.

[0041] Step S201, obtaining a sample data set, and performing a division process on the sample data set.

[0042] In the embodiment of the present invention, the sample data set may include multiple paragraphs of text, multiple bank card numbers, multiple order numbers, etc. The embodiment of the present invention is introduced by taking the natural language processing scenario as an example. The sample data sets of other application scenarios may be different, but the execution process of the division processing of the sample data sets may refer to each other. Therefore, the sample data set x in the embodiment of the present invention may be the following paragraph of text:

[0043] "The poem says: Before the chaos was divided, the world was in turmoil, and it was so vast that no one could see it. Since Pangu broke through the chaos, the clear and the turbid have been distinguished. In ancient times, all living things looked up to the Supreme Benevolence, and all things were created for good. If you want to know the merits of the creation, you must read Journey to the West. I heard that the number of heaven and earth is 129,600 years for one Yuan. One Yuan is divided into 12 sessions, which are the twelve branches of Zi, Chou, Yin, Mao, Chen, Si, Wu, Wei, Shen, You, Xu, and Hai. Each session should be 1800 years. Let's talk about one day: Zi gets Yang energy, Chou crows like a cock, and Yin does not let in light."

[0044] Among them, each character or punctuation mark can be understood as a sample data in the sample data set x. After obtaining the sample data set x, each sample data in the sample data set x is converted into a sample vector x(i), and the sample vector x(i) represents the i-th sample data. i can represent the serial number of the sample data in the sample data set x. The dimension of each sample data is d, and this dimension can be understood as the representation dimension of the sample data in the computer.

[0045] Then, according to the preset length L, all the sample vectors x(i) in the sample data set x are divided into multiple batches of input items. Specifically, the number N of sample data or the number N of sample vectors can be divided by the preset length L to obtain the number of batches N / L. The preset length L is the number of sample vectors included in one batch of input items.

[0046] Therefore, the above sample data set x can be divided into the following several batches of input items:

[0047] First batch: [Poetry says: Before chaos divided, heaven and earth were in turmoil, vast and boundless, no one could see. Since Pangu broke through the chaos, the world has been cleared and opened up.]

[0048] Second batch: [To distinguish the turbid. Throughout history, all living beings have looked up to the greatest kindness, and all things created are good. To know the profound mystery of creation, one must read Journey to the West]

[0049] Third batch: [To Release Sufferings. It is said that the number of heaven and earth is 129,600 years for one yuan. Divide one yuan into twelve]

[0050] Fourth batch: [periods, which are the twelve Earthly Branches of Zi, Chou, Yin, Mao, Chen, Si, Wu, Wei, Shen, You, Xu, and Hai.]

[0051] Fifth batch: [Each period lasts for 10,800 years. Just taking one day as an example: At子时, yang qi emerges, and at丑, the rooster crows; at寅, there is no light]

[0052] Taking the input item of the first batch as an example for illustration, the introduction of the input items of other batches can refer to the relevant description of the input item of the first batch. In the input item of the first batch, L is the number of "Poetry", "says", ……, "clear" (including characters and punctuation marks), that is, L = 32. The input item of the first batch can be represented as x(1), x(2), x(3), ……, x(32) using the sample vector x(i). It should be noted that for easier understanding of the embodiments of the present invention, i in the sample vector can be directly replaced with the sample data itself, that is, x(1) is replaced with x(Poetry), x(2) is replaced with x(says), x(3) is replaced with x(:), ……, x(32) is replaced with x(clear). Therefore, the input item of the first batch can also be represented as (Poetry), x(says), x(:), ……, x(clear).

[0053] Step S202, obtaining the annotation data of the input item.

[0054] In an embodiment of the present invention, after obtaining the above batches of input items, each input item may be shifted backward by one sample vector and divided according to a preset length to obtain the labeled data corresponding to the input items of each batch.

[0055] Continuing with the previous example, we still take the first batch of input items as an example for explanation. In the sample data set x, the first batch of input items (poem), x(said), x(:), ..., x(clear) are shifted backward by one sample vector and divided according to the preset length L to obtain the labeled data corresponding to the first batch of input items. That is, the labeled data corresponding to the first batch of input items (poem), x(said), x(:), ..., x(clear) are 32 sample vectors starting from x(said) to x(turbid).

[0056] Finally, the first batch of input items are (poem), x(said), x(:), ..., x(clear). The labeling data corresponding to the first batch of input items are x(said), x(:), x(mixed), ..., x(clear). Similarly, the labeling data corresponding to the input items of other batches can be obtained, and the embodiments of the present invention are not repeated here.

[0057] The labeled data of each batch of input items can be used as input items of each batch to input into the neural network model to obtain calibration data of the output items. That is to say, each batch of input items is input into the neural network model, and the obtained output items are compared with the corresponding labeled data, and then the network parameters of each network layer of the neural network model are adjusted according to the comparison results until the neural network model after the network parameters are adjusted meets the convergence conditions, that is, after each batch of input items is input into the neural network model after the network parameters are adjusted, the obtained output items are the same, close or meet the requirements of similarity with the corresponding labeled data.

[0058] Step S203, inputting the input items of each batch into the initial network model, and calculating the output items according to the dilated convolution algorithm.

[0059] In an embodiment of the present invention, the initial network model may include multiple network layers. The network layer may be divided into an input layer, an output layer, and an intermediate layer. A corresponding storage space may be allocated in advance for each intermediate layer. The storage space of the intermediate layer is consistent with the storage space of the input item, that is, the storage space of the intermediate layer is a matrix of L*d. If the initial network model has M intermediate layers (in practical applications, M may be 6), then M*L*d storage space needs to be allocated.

[0060] Moreover, the embodiment of the present invention can also set a corresponding memory layer for each middle layer. The storage space of the memory layer can be greater than or equal to the storage space of the middle layer. The input items of the middle layer can be copied to the memory layer, that is, the memory layer can be used as a backup of the middle layer. Initially, the memory layer is empty.

[0061] In an exemplary embodiment of the present invention, the initial network model may include two intermediate layers in addition to the input layer and the output layer. Figure 3 As shown, Figure 3 A schematic diagram of the network structure of the initial network model is shown. First, the first batch of input items can be input into the input layer, and the first batch of input items can be input from the input layer into the first intermediate layer, and the output items of the first intermediate layer can be copied to the memory layer corresponding to the first intermediate layer. Then, the first batch of input items are spliced ​​with the output items of the first intermediate layer stored in the memory layer, and the spliced ​​sample vector is input into the second intermediate layer as the input item of the second intermediate layer. In other words, in addition to the first batch of input items, the current input items can be spliced ​​with the output items of the previous network layer in the initial network model, and the spliced ​​sample vector is input into the current network layer as the input item of the current network layer.

[0062] In an exemplary embodiment of the present invention, each intermediate layer in the neural network model may be an expanded convolution layer, that is, the storage space of each expanded convolution layer is a matrix of L*d.

[0063] Step S204, training the initial network model according to the loss function.

[0064] In an embodiment of the present invention, a loss function corresponding to the application scenario can be selected to train the initial network model, and finally a trained neural network model is obtained.

[0065] Based on the above description of the training method of a neural network model, the following introduces the network structure of a neural network model. Figure 4 As shown, Figure 4 A schematic diagram of the network structure of a neural network model is shown. Figure 4 In the example, the middle layer of the neural network model can include two dilated convolutional layers. Moreover, for each dilated convolutional layer, a corresponding memory layer is also set. The memory layer is used to back up the input items of the previous batch input to the dilated convolutional layer and the output items obtained after calculation. For example, the input item of the current batch (current input) is x(i), and the input item of the previous batch (historical input) is x'(i). Concatenate x(i) and x'(i) into As the input of the current dilated convolutional layer. When calculating the output of each dilated convolutional layer, according to calculate in, represents the value of the i-th position of the j-th layer. And the calculation is is stored in the corresponding memory layer. The value stored in the memory layer is represented as After two dilated convolutional layers of the neural network model, the output items corresponding to the input items of the current batch are obtained (current output). The memory layer corresponding to the last dilated convolutional layer stores the output items of the previous batch (historical output).

[0066] An embodiment of the present invention trains a neural network model that can both calculate output items with low latency and capture input items with ultra-long dependencies.

[0067] The embodiment of the present invention concatenates the current input with the historical input as the input item of the current network layer, thereby realizing the function of capturing the input items with super-long dependencies.

[0068] The embodiments of the present invention can use loss functions in different application scenarios to train the initial network model, thereby obtaining neural network models for different application scenarios, thereby improving the scope of application of the neural network model.

[0069] Embodiment 3

[0070] Reference Figure 5 , Figure 5 A schematic diagram of the structure of a target data prediction device according to the third embodiment of the present invention is shown. The device may specifically include the following modules:

[0071] An acquisition module 51 is used to acquire a plurality of first target data satisfying a preset long-term dependency condition;

[0072] An input module 52, used for inputting a plurality of the first target data into the trained neural network model, and outputting second target data;

[0073] Wherein, the second target data has a serial correlation with a plurality of the first target data;

[0074] The device also includes: a training module 53, which is used to divide the sample data set into multiple batches of input items, and input the input items into the initial network model in the order of the batches, calculate the output items of the initial network model according to the dilated convolution algorithm, and train the initial network model according to the input items, the output items and the loss function to obtain the neural network model.

[0075] In an exemplary embodiment of the present invention, the training module 53 includes:

[0076] A sample conversion module, used for converting each sample data in the sample data set into a sample vector;

[0077] A sample division module, used for dividing all the sample vectors into multiple batches of input items according to a preset length;

[0078] The number of sample data divided by the length equals the number of batches.

[0079] In an exemplary embodiment of the present invention, the training module 53 further includes:

[0080] The sample labeling module is used to shift each input item backward by one bit of the sample vector after the sample division module divides all the sample vectors into multiple batches of input items, and obtain the labeling data corresponding to each input item according to the length division.

[0081] In an exemplary embodiment of the present invention, the training module 53 includes:

[0082] The sample splicing module is used to splice the current input item with the output item of the previous network layer in the initial network model; and input the spliced ​​sample vector into the current network layer in the initial network model.

[0083] In an exemplary embodiment of the present invention, the network layer of the neural network model includes multiple dilated convolutional layers, and the storage space of each dilated convolutional layer is a matrix space of the length multiplied by the dimension of the sample vector.

[0084] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0085] It is easy for a person skilled in the art to think that any combination of the above embodiments is feasible, so any combination of the above embodiments is an implementation scheme of the present invention. However, due to space limitations, this specification will not describe them in detail here.

[0086] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0087] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0088] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0089] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0090] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A method for predicting target data, characterized in that: The method comprises: Acquire multiple first target data satisfying a preset long-term dependency condition, wherein the multiple first target data satisfying the preset long-term dependency condition include multiple numbers having an ultra-long dependency relationship; the multiple first target data satisfying the preset long-term dependency condition are represented by an x(t) data sequence, wherein t is a natural number, and x(t) represents the tth first target data; Inputting a plurality of the first target data into the trained neural network model, and outputting second target data; Wherein, the second target data has a serial correlation with the plurality of the first target data, and if the plurality of the first target data are the first several numbers of a bank card number, the second target data is the last number or the last several numbers of the bank card number; The neural network model is trained by the following steps: Dividing the sample data set into a plurality of batches of input items, and inputting the input items into the initial network model in the order of the batches, calculating the output items of the initial network model according to the dilated convolution algorithm, and training the initial network model according to the input items, the output items and the loss function to obtain the neural network model; Wherein, each sample data in the sample data set is converted into a sample vector, the initial network model includes multiple network layers, the network layers include an input layer, an output layer and an intermediate layer, a corresponding storage space is pre-allocated to each intermediate layer, the storage space of the intermediate layer is consistent with the storage space of the input item, each intermediate layer is an expanded convolution layer, and the storage space of each expanded convolution layer is a matrix of L﹡d, wherein L is a preset length, and d is the dimension of each sample data. If the initial network model has M intermediate layers, M﹡L﹡d storage space needs to be allocated; except for the first batch of input items, the current input item is spliced ​​with the output item of the previous network layer in the initial network model, and the spliced ​​sample vector is input to the current network layer as the input item of the current network layer.

2. The method according to claim 1, characterized in that The input items for dividing the sample data set into multiple batches include: Dividing all the sample vectors into a plurality of batches of the input items according to the preset length; The number of sample data divided by the length equals the number of batches.

3. The method according to claim 1, characterized in that After dividing the sample data set into a plurality of batches of input items, the method further comprises: Each of the input items is shifted backward by one bit of the sample vector, and the labeled data corresponding to each of the input items is obtained by dividing the data according to the length.

4. The method according to any one of claims 2 to 3, characterized in that The network layer of the neural network model includes a plurality of the dilated convolutional layers, and the storage space of each of the dilated convolutional layers is a matrix space obtained by multiplying the length by the dimension of the sample vector.

5. A target data prediction device, characterized in that: The device comprises: An acquisition module is used to acquire a plurality of first target data satisfying a preset long-term dependency condition, wherein the plurality of first target data satisfying the preset long-term dependency condition comprises a plurality of numbers having an ultra-long dependency relationship; the plurality of first target data satisfying the preset long-term dependency condition is represented by an x(t) data sequence, wherein t is a natural number, and x(t) represents the tth first target data; An input module, used for inputting a plurality of the first target data into the trained neural network model, and outputting second target data; Wherein, the second target data has a serial correlation with the plurality of the first target data, and if the plurality of the first target data are the first several numbers of a bank card number, the second target data is the last number or the last several numbers of the bank card number; The device further includes: a training module, which is used to divide the sample data set into a plurality of batches of input items, and input the input items into the initial network model in the order of the batches, calculate the output items of the initial network model according to the dilated convolution algorithm, and train the initial network model according to the input items, the output items and the loss function to obtain the neural network model; Wherein, each sample data in the sample data set is converted into a sample vector, the initial network model includes multiple network layers, the network layer includes an input layer, an output layer and an intermediate layer, and a corresponding storage space is pre-allocated to each intermediate layer, and the storage space of the intermediate layer is consistent with the storage space of the input item. Each intermediate layer is an expanded convolution layer, and the storage space of each expanded convolution layer is a matrix of L﹡d, wherein L is a preset length, and d is the dimension of each sample data. If the initial network model has M layers of intermediate layers, M﹡L﹡d storage space needs to be allocated; except for the first batch of input items, the current input item is spliced ​​with the output item of the previous network layer in the initial network model, and the spliced ​​sample vector is input into the current network layer as the input item of the current network layer.

6. The device according to claim 5, characterized in that The training module comprises: A sample division module, used for dividing all the sample vectors into a plurality of batches of the input items according to the preset length; The number of sample data divided by the length equals the number of batches.

7. The device according to claim 6, characterized in that The training module further includes: The sample labeling module is used to shift each input item backward by one bit of the sample vector after the sample division module divides all the sample vectors into multiple batches of input items, and obtain the labeling data corresponding to each input item according to the length division.

8. The device according to any one of claims 6 to 7, characterized in that The network layer of the neural network model includes a plurality of the dilated convolutional layers, and the storage space of each of the dilated convolutional layers is a matrix space obtained by multiplying the length by the dimension of the sample vector.

Images