Data processing method, document data processing method, computing device, and storage medium

Abstract

Embodiments of the present disclosure provide a data processing method, a document data processing method, a computing device, and a storage medium. The document data processing method comprises: determining document data to be processed, and inputting the document data to be processed into a data processing model; in the data processing model, segmenting the document data to be processed, to obtain multiple document data blocks arranged in sequence; sequentially taking any document data block among the multiple document data blocks as a target document data block, and obtaining a second prediction result of the target document data block on the basis of a first prediction result of a preceding document data block and the target document data block, the preceding document data block being a document data block among the multiple document data blocks in front of the target document data block; on the basis of the first prediction result and the second prediction result, obtaining a target prediction result of the document data to be processed.

Description

Data processing methods, document data processing methods, computing devices and storage media

[0001] This disclosure claims priority to Chinese Patent Application No. 202410705685.3, filed with the China Patent Office on May 31, 2024, entitled “Data Processing Method, Document Data Processing Method, Computing Device and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of artificial intelligence technology, and in particular to a document data processing method. One or more embodiments of this disclosure also relate to a data processing method, a data processing model training method, another document data processing method, a document data processing apparatus, a data processing apparatus, a data processing model training apparatus, another document data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0003] With the continuous development of computer technology and artificial intelligence, neural network models can be used to perform the required document data processing operations during document data processing, thereby obtaining the corresponding data processing results.

[0004] In the process of using neural network models to process document data, existing technologies often fail to accurately process large document data, resulting in low-accuracy data processing results. Therefore, how to enable neural network models to obtain accurate data processing results based on large document data has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of this disclosure provide a document data processing method. One or more embodiments of this disclosure simultaneously relate to a data processing method, a data processing model training method, another document data processing method, a document data processing apparatus, a data processing apparatus, a data processing model training apparatus, another document data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical shortcomings of existing neural network models that cannot obtain accurate data processing results based on large document data.

[0006] According to a first aspect of the present disclosure, a document data processing method is provided, comprising:

[0007] Determine the document data to be processed and input the document data to be processed into the data processing model;

[0008] In the data processing model, the document data to be processed is segmented to obtain multiple document data blocks arranged in sequence;

[0009] The document data block is selected as the target document data block in sequence, and a second prediction result of the target document data block is obtained based on the first prediction result of the preceding document data block and the target document data block. The preceding document data block is the document data block that is located before the target document data block in the plurality of document data blocks.

[0010] Based on the first prediction result and the second prediction result, the target prediction result of the document data to be processed is obtained.

[0011] According to a second aspect of the present disclosure, a document data processing apparatus is provided, comprising:

[0012] The data determination module is configured to determine the document data to be processed and input the document data to be processed into the data processing model.

[0013] The data segmentation module is configured to segment the document data to be processed in the data processing model to obtain multiple document data blocks arranged in sequence.

[0014] The data prediction module is configured to sequentially take any one of the plurality of document data blocks as the target document data block, and obtain a second prediction result of the target document data block based on the first prediction result of the preceding document data block and the target document data block, wherein the preceding document data block is the document data block located before the target document data block among the plurality of document data blocks;

[0015] The result determination module is configured to obtain the target prediction result of the document data to be processed based on the first prediction result and the second prediction result.

[0016] According to a third aspect of the present disclosure, a data processing method is provided, comprising:

[0017] Identify the data to be processed and input the data to be processed into the data processing model;

[0018] In the data processing model, the data to be processed is segmented to obtain multiple data blocks arranged in sequence;

[0019] The data block is selected as the target data block in sequence, and a second prediction result of the target data block is obtained based on the first prediction result of the preceding data block and the target data block. The preceding data block is the data block that is located before the target data block in the plurality of data blocks.

[0020] Based on the first prediction result and the second prediction result, the target prediction result of the data to be processed is obtained.

[0021] According to a fourth aspect of the present disclosure, a data processing apparatus is provided, comprising:

[0022] The data determination module is configured to determine the data to be processed and input the data to be processed into the data processing model;

[0023] The data segmentation module is configured to segment the data to be processed in the data processing model to obtain multiple data blocks arranged in sequence;

[0024] The data prediction module is configured to sequentially take any one of the plurality of data blocks as the target data block, and obtain a second prediction result of the target data block based on the first prediction result of the preceding data block and the target data block, wherein the preceding data block is the data block that is located before the target data block among the plurality of data blocks;

[0025] The result determination module is configured to obtain the target prediction result of the data to be processed based on the first prediction result and the second prediction result.

[0026] According to a fifth aspect of the present disclosure, a data processing model training method is provided, comprising:

[0027] Determine the data processing model to be trained, and the sample data associated with the target task;

[0028] The sample data is input into the data processing model to be trained. In the data processing model, the sample data is segmented to obtain multiple data blocks arranged in sequence.

[0029] The data block is selected as the target data block in sequence, and a second prediction result of the target data block is obtained based on the first prediction result of the preceding data block and the target data block. The preceding data block is the data block that is located before the target data block in the plurality of data blocks.

[0030] Based on the first prediction result and the second prediction result, the sample prediction result of the sample data is obtained;

[0031] Based on the sample prediction results, the model parameters of the data processing model are adjusted to obtain a trained data processing model, wherein the data processing model is used to perform the target task.

[0032] According to a sixth aspect of the present disclosure, a data processing model training apparatus is provided, comprising:

[0033] The data determination module is configured to determine the data processing model to be trained, as well as the sample data associated with the target task;

[0034] The data segmentation module is configured to input the sample data into the data processing model to be trained, and in the data processing model, segment the sample data to obtain multiple data blocks arranged in sequence.

[0035] The data prediction module is configured to sequentially take any one of the plurality of data blocks as the target data block, and obtain a second prediction result of the target data block based on the first prediction result of the preceding data block and the target data block, wherein the preceding data block is the data block that is located before the target data block among the plurality of data blocks;

[0036] The result determination module is configured to obtain the sample prediction result of the sample data based on the first prediction result and the second prediction result;

[0037] The model training module is configured to adjust the model parameters of the data processing model based on the sample prediction results to obtain a trained data processing model, wherein the data processing model is used to perform the target task.

[0038] According to a seventh aspect of the embodiments of this disclosure, another document data processing method is provided, including:

[0039] Identify the document data to be processed;

[0040] The document data to be processed is input into a data processing model to obtain a target prediction result for the document data to be processed. The data processing model obtains the target prediction result based on the document data to be processed in the following way: In the data processing model, the document data to be processed is segmented to obtain multiple document data blocks arranged in sequence. Any one of the multiple document data blocks is taken as the target document data block in sequence. A second prediction result for the target document data block is obtained based on the first prediction result of the preceding document data block and the target document data block. The preceding document data block is the document data block that is located before the target document data block among the multiple document data blocks. The target prediction result for the document data to be processed is obtained based on the first prediction result and the second prediction result.

[0041] According to an eighth aspect of the present disclosure, another document data processing apparatus is provided, comprising:

[0042] The data determination module is configured to determine the data of the document to be processed.

[0043] The result determination module is configured to input the document data to be processed into a data processing model to obtain a target prediction result for the document data to be processed. The data processing model obtains the target prediction result based on the document data to be processed by: segmenting the document data to be processed into multiple sequentially arranged document data blocks; sequentially using any one of the multiple document data blocks as the target document data block; and obtaining a second prediction result for the target document data block based on a first prediction result of the preceding document data block and the target document data block. The preceding document data block is the document data block located before the target document data block among the multiple document data blocks. The target prediction result for the document data to be processed is obtained based on the first prediction result and the second prediction result.

[0044] According to a ninth aspect of the present disclosure, a computing device is provided, comprising:

[0045] Memory and processor;

[0046] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, they implement the steps of the above-described document data processing method, data processing method, data processing model training method, or another document data processing method.

[0047] According to a tenth aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program / instructions, which, when executed by a processor, implement the steps of the above-described document data processing method, data processing method, data processing model training method, or another document data processing method.

[0048] According to the eleventh aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described document data processing method, data processing method, data processing model training method, or another document data processing method.

[0049] This disclosure provides a document data processing method in one or more embodiments. Considering the problem that large document data can lead to neural network models being unable to accurately process it, the method utilizes a data processing model to segment the document data into multiple sequentially arranged document data blocks during processing. This facilitates targeted prediction of each document data block, resulting in accurate predictions. Furthermore, considering the correlation between the multiple document data blocks segmented from the document data, to ensure accurate target prediction results, any one of the multiple data blocks is sequentially used as the target data block. A second prediction result for the target data block is obtained based on the first prediction result of the preceding data block and the target data block. Finally, an accurate target prediction result is obtained based on the first and second prediction results. This avoids the problem of low accuracy in data processing results caused by the neural network model's inability to accurately process large document data. Therefore, this method achieves accurate data processing results using a data processing model for large document data. Attached Figure Description

[0050] Figure 1 is a schematic diagram illustrating the application of a document data processing method provided in an embodiment of this disclosure;

[0051] Figure 2 is a flowchart of a document data processing method provided in an embodiment of this disclosure;

[0052] Figure 3 is a flowchart of a document data processing method provided in an embodiment of this disclosure;

[0053] Figure 4 is a schematic diagram of an improved attention matrix in a document data processing method provided in an embodiment of this disclosure;

[0054] Figure 5 is a schematic diagram of a comparative learning training method for compressing placeholder differences in a document data processing method provided in an embodiment of this disclosure;

[0055] Figure 6 is a flowchart of a data processing method provided in an embodiment of this disclosure;

[0056] Figure 7 is a flowchart of a data processing model training method provided in an embodiment of this disclosure;

[0057] Figure 8 is a flowchart of another document data processing method provided in an embodiment of this disclosure;

[0058] Figure 9 is a schematic diagram of the structure of a document data processing apparatus provided in an embodiment of this disclosure;

[0059] Figure 10 is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure;

[0060] Figure 11 is a schematic diagram of the structure of a data processing model training device provided in an embodiment of the present disclosure;

[0061] Figure 12 is a schematic diagram of another document data processing apparatus provided in an embodiment of the present disclosure;

[0062] Figure 13 is a structural block diagram of a computing device provided in an embodiment of this disclosure. Detailed Implementation

[0063] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0064] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0065] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0066] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0067] In one or more embodiments of this disclosure, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0068] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0069] First, the terms and concepts involved in one or more embodiments of this disclosure will be explained.

[0070] Language model: A statistical model used to understand and generate natural language. Its goal is to analyze a piece of text and predict the probability distribution of the next word or character. Language models can be used for a variety of text generation tasks.

[0071] Long-context: Long-context is a common problem in language models, referring to the difficulty models face in capturing lengthy contextual dependencies when processing long text sequences. This leads to a situation where, when a language model receives excessively long text sequences during inference or prediction, the prediction performance of subsequent text sequences is often affected, or even very poor. Examples include generated content that is irrelevant or inconsistent.

[0072] Large language models refer to neural network models with greater scale and capabilities. By increasing the number of parameters, layers, and attention mechanisms, large models can better capture the semantics, structure, and relationships of text. Large models have achieved remarkable results in natural language processing tasks. While large models perform impressively well in predicting texts of a certain length, they are still criticized in scenarios with extremely long contexts.

[0073] In the field of natural language processing, large models refer to neural network models with greater scale and capabilities. By increasing the number of parameters, layers, and attention mechanisms, large models can better capture the semantics, structure, and relationships of text. Large models have achieved significant results in natural language processing tasks such as machine translation, text generation, and question answering systems.

[0074] Placeholder: A placeholder is a symbol that occupies a fixed position before content is added to that position. Placeholders are widely used in the editing of various documents on computers.

[0075] However, large models also face challenges and problems when processing long texts, namely the long-context problem. When a model receives excessively long text sequences during the inference stage, its prediction performance for subsequent text sequences drops significantly. For example, when the model cannot correctly understand the contextual information of long texts, the predictions for subsequent text sequences may not match the actual context. Due to the complex dependencies in long texts, the model may miss information or overfit, thus affecting the coherence and accuracy of subsequent text sequences.

[0076] The main reason for the poor performance of large models in long-context scenarios is that excessively long text sequences contain a large amount of information and semantics, which the model struggles to fully understand and utilize. Furthermore, due to the large dependencies and semantic distances in long texts, the model struggles to accurately capture the correct semantics and structure when predicting subsequent text sequences. In addition, long text sequences may contain more noise and interference, making prediction even more difficult. Specifically, large models often face the following dilemma when processing long documents: either the provided context is insufficient, resulting in poor generated responses; or sufficient context is provided, but the text length is too long to generate a proper response. To address the problem of large models performing poorly in long-context scenarios...

[0077] To address the aforementioned issues, this disclosure provides three solutions. The first is a multi-stage training scheme: This scheme requires ① collecting a large amount of natural language text data with contexts of various lengths; ② grouping the data according to text length; ③ training the model sequentially from the shorter to the longer groups of data. However, this scheme has the following drawbacks: the key to its effectiveness lies in the carefully designed training strategies for each stage, including at least: ① how to choose the timing for switching data for training; ② carefully designed parameters for each stage of training; ③ the trained model can only adapt to contexts of the corresponding length and will not be effective with longer contexts; ④ training effects may overlap, meaning that after adapting to the context length of later training, the model may perform worse in relatively shorter contexts.

[0078] The second approach introduces an additional special placeholder, [GIST], which forces the large model to use as a special marker for compressed instructions during training. This prediction scheme is designed to concatenate the instruction and [GIST] and input them into the large model, using the hidden vector marked with [GIST] to replace the entire instruction, thus achieving compression. However, this approach has the following drawbacks: ① It is only used to compress "task instructions," while this approach is suitable for general scenarios that extend the model's (large or small) ability to handle long contexts; ② The above schemes require additional construction of triplet data (original text, instruction, answer); ③ Regarding the use of the special placeholder, the above schemes can only use [GIST] to replace the instruction text, and [GIST] is used only once per task.

[0079] The third approach proposes a method of compression using special placeholders. This approach ① introduces additional special placeholders [ANCHOR], inserting them at certain positions in each input text according to specific rules; ② forces the large model to use [ANCHOR] placeholders to compress text contained between previous [ANCHOR] instances during training; ③ also randomly inserts some [ANCHOR] instances during prediction, and discards preceding text each time [ANCHOR] is used for inference, retaining only [ANCHOR] and its following text for subsequent inference. The drawback of this approach is that, in order to "extremely maximize compression," the [ANCHOR] method does not allow special placeholders to interact with each other, thus failing to guarantee the continuity of compressed information.

[0080] Based on this, this disclosure provides a document data processing method. This disclosure also relates to a data processing method, a data processing model training method, another document data processing method, a document data processing apparatus, a data processing apparatus, a data processing model training apparatus, another document data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0081] Referring to Figure 1, Figure 1 illustrates an application diagram of a document data processing method according to an embodiment of the present disclosure, wherein the document data processing method includes:

[0082] Determine the document data to be processed and input the document data to be processed into the data processing model;

[0083] In the data processing model, the document data to be processed is segmented to obtain multiple document data blocks arranged in sequence;

[0084] The document data block is selected as the target document data block in sequence, and a second prediction result of the target document data block is obtained based on the first prediction result of the preceding document data block and the target document data block. The preceding document data block is the document data block that is located before the target document data block in the plurality of document data blocks.

[0085] Based on the first prediction result and the second prediction result, the target prediction result of the document data to be processed is obtained.

[0086] Specifically, as shown in Figure 1, the user sends the paper document to the server 104 through the terminal 102. The server 104 can use the LLM large model to process the paper document and obtain the corresponding analysis text. The specific execution method is as follows: After receiving the document data to be processed, the server 104 can input the document data to be processed into the data processing model. In the data processing model, the document data to be processed is segmented to obtain multiple document data blocks arranged in sequence. Any one of the multiple document data blocks is taken as the target document data block. Based on the first analysis text of the preceding document data block and the target document data block, the second analysis text of the target document data block is obtained. The preceding document data block is the document data block that is located before the target document data block among the multiple document data blocks. Based on the first analysis text and the second analysis text, the analysis text corresponding to the paper document to be processed is obtained.

[0087] After obtaining the analyzed text, the server 104 will send the analyzed text to the terminal 102, which will then display it to the user.

[0088] Based on the above, this document data processing method addresses the issue that large document data can hinder the accurate processing of neural network models. Therefore, during the processing of the document data, the data processing model segments the document data into multiple sequentially arranged data blocks. This facilitates targeted prediction of each data block, resulting in accurate predictions. Furthermore, considering the correlation between the multiple data blocks segmented from the document data, to ensure accurate target prediction, any one of the data blocks is sequentially used as the target data block. A second prediction result for the target data block is obtained based on the first prediction result of the preceding data blocks and the target data block. Finally, the accurate target prediction result is obtained based on both the first and second prediction results. This avoids the problem of low accuracy in data processing results caused by the neural network model's inability to accurately process large document data. Thus, this method achieves accurate data processing results even with large document data using a data processing model.

[0089] Referring to Figure 2, Figure 2 shows a flowchart of a document data processing method according to an embodiment of the present disclosure, which specifically includes the following steps.

[0090] Step 202: Determine the document data to be processed and input the document data to be processed into the data processing model.

[0091] The document data to be processed can be understood as document data that needs to be processed using a data processing model. For example, the document data to be processed can be a thesis document, an e-book document, a product user manual document, etc. There is no specific limitation on the document data to be processed here. In one or more embodiments provided in this disclosure, the document data to be processed can be a long text sequence.

[0092] The data processing model can be understood as a model capable of analyzing and processing document data. The data processing model can be a language model, a large model, etc. In one or more embodiments provided in this disclosure, natural language processing can be performed on the document data to be processed by utilizing the data processing model.

[0093] In one or more embodiments provided in this disclosure, determining the document data to be processed includes:

[0094] Receive unprocessed document data sent by the client, wherein the unprocessed document data is sent by the client based on the user's document upload operation for the unprocessed document data;

[0095] It should be noted that the document data processing method provided in one or more embodiments of this disclosure can be applied to a server, which can be understood as a server, server cluster, virtual machine, container, or other device; the server is connected to the client; the user can instruct the server to execute the steps of the document data processing method provided in one or more embodiments of this disclosure through human-computer interaction with the client.

[0096] Specifically, users send document data to be processed to the client by performing a document upload operation on the client; the client then sends the document data to the server based on the user's operation.

[0097] The server-side application of the document data processing method, after receiving the document data to be processed, will use the data processing model to perform document data processing operations on it, thereby meeting the user's processing needs for the document data to be processed.

[0098] Step 204: In the data processing model, the document data to be processed is segmented to obtain multiple document data blocks arranged in sequence.

[0099] The segmentation of the document data to be processed can be understood as dividing the document data into multiple document data blocks according to a preset data size. For example, when the document data to be processed is a thesis document, the thesis document is segmented according to a preset data size (e.g., 100 characters or 200 characters) to obtain multiple thesis document data blocks; each thesis document data block may contain character data corresponding to the preset data size (e.g., containing 100 characters or 200 characters).

[0100] In this context, sequential arrangement can be understood as arranging according to a natural order. For example, taking multiple document data blocks as five document data blocks, they can be arranged in a natural order such as first document data block, second document data block, third document data block, fourth document data block, and fifth document data block.

[0101] A document data block can be understood as a data block obtained by segmenting the document data to be processed. Each document data block contains a portion of the data content of the document data to be processed. For example, if the document data to be processed is a long text sequence, the long text sequence can be segmented into multiple text sequences. As another example, if the document data to be processed is a thesis document, the thesis document can be segmented into multiple document data blocks.

[0102] In one or more embodiments provided in this disclosure, the step of segmenting the document data to be processed in the data processing model to obtain a plurality of sequentially arranged document data blocks includes:

[0103] In the data processing model, the data to be processed is divided into multiple data blocks to be segmented using the data prediction size of the data processing model.

[0104] Determine any two adjacent data blocks to be split among the plurality of data blocks to be split, and insert a compression placeholder between the two adjacent data blocks to be split;

[0105] The document data to be processed, which includes the compression placeholder and the plurality of data blocks to be segmented, is segmented to obtain the plurality of document data blocks arranged in sequence, wherein the compression placeholder exists between any two adjacent document data blocks.

[0106] The data prediction size can be understood as the size that the data processing model can handle during the data prediction process. The data prediction size can be understood as the window size of the data processing model. It should be noted that the data prediction size can be set according to the actual application scenario, and this disclosure does not impose specific restrictions on it.

[0107] The data block to be segmented can be understood as the data block that needs to be segmented; that is, in the process of segmentation, the document data to be processed is first divided into multiple data blocks to be segmented according to the predicted data size (or preset data size), and then the multiple data blocks to be segmented are segmented to obtain multiple document data blocks arranged in order.

[0108] A compression placeholder can be understood as a placeholder used to compress document data to be processed. In one or more embodiments provided in this disclosure, a compression placeholder can be used to guide the data processing model to compress the text information in its window during the model training process or model application process. The text information (the document data block in this example) can be compressed into the compression placeholder.

[0109] Specifically, the document data processing method provided in one or more embodiments of this disclosure can divide the document data in the document data to be processed into multiple data blocks to be segmented by using the data prediction size of the data processing model in the data processing model, so as to facilitate subsequent rapid segmentation and obtain multiple data blocks.

[0110] Then, any two adjacent data blocks to be segmented can be determined from the data blocks to be segmented. A compression placeholder is inserted between any two adjacent data blocks to be segmented. By segmenting the document data to be processed, which includes the compression placeholder and multiple data blocks to be segmented, the multiple document data blocks are obtained in sequence. It should be noted that the compression placeholder exists between any two adjacent document data blocks in the multiple document data blocks obtained after segmentation. This facilitates the subsequent compression of the vectors corresponding to the document data blocks into the compression placeholders. This allows the vectors using the compression placeholders to be used to replace the accumulation of vector information in the text (i.e., document data blocks), thereby improving the prediction accuracy of the model.

[0111] Taking the application of the document data processing method provided in this disclosure in the scenario of processing ultra-long context input as an example, the document data processing method is described. The document data to be processed is a paper document, i.e., ultra-long context input; the data processing model is model M; based on this, after receiving the paper document, the paper document is input into model M; using model M, according to the window size of model M, the paper document is divided into multiple data blocks corresponding to the window, and a [COMP] separator (i.e., compression placeholder) is inserted between two data blocks. Then, the text (i.e., the paper document) is divided into multiple segments (i.e., multiple document data blocks) according to the window, which facilitates subsequent serial reasoning.

[0112] It should be noted that the step of segmenting the document data to be processed in the data processing model can be implemented through the data processing module in the data processing model. The data processing module can be understood as the module in the data processing model that segments the document data to be processed. This data processing module can be understood as one or more network layers in the data processing model; or, the data processing module can be understood as a sub-model in the data processing model. For example, using the data processing module in model M, the paper document is divided into multiple data blocks corresponding to the window size of model M, and a [COMP] separator is inserted between two data blocks. Then, the text is segmented into multiple segments according to the window.

[0113] Step 206: Sequentially select any one of the plurality of document data blocks as the target document data block, and obtain the second prediction result of the target document data block based on the first prediction result of the preceding document data block and the target document data block, wherein the preceding document data block is the document data block located before the target document data block among the plurality of document data blocks.

[0114] In this context, the preceding document data block can be understood as the document data block that precedes any other document data block among a series of sequentially arranged document data blocks. For example, consider the five document data blocks: the first document data block, the second document data block, the third document data block, the fourth document data block, and the fifth document data block. When the second document data block is the target document data block, the preceding document data block of the target document data block can be the first document data block.

[0115] It should be noted that, in one or more embodiments provided in this disclosure, when it is determined that the target document data block exists in the preceding document data block, a second prediction result of the target document data block is obtained based on the first prediction result of the preceding document data block and the target document data block.

[0116] If the target document data block does not have a corresponding preceding document data block, a second prediction result of the target document data block can be obtained based on the target document data block.

[0117] Specifically, in the data processing model, when it is determined that the target document data block does not have a corresponding preceding document data block, data prediction is performed based on the target document data block to obtain a second prediction result for the target document data block. For example, taking multiple document data blocks as a first document data block, a second document data block, a third document data block, a fourth document data block, and a fifth document data block as an example; when the first document data block is the target document data block, and this target document data block does not have a corresponding preceding document data block, therefore, model M is used to perform data prediction on the first document data block to obtain the prediction result corresponding to the first document data block (i.e., the second prediction result).

[0118] The prediction result can be understood as the inferred text obtained after reasoning about the document data block. The inferred text can be a token or a vector. Correspondingly, the first prediction result is the prediction result of the preceding document data block; the second prediction result is the prediction result of the target document data block. In one or more embodiments provided in this disclosure, the prediction result can be understood as a compressed placeholder with compressed inferred text, which refers to the inferred text obtained after reasoning about the document data block.

[0119] In one or more embodiments provided in this disclosure, the step of sequentially taking any one of the plurality of document data blocks as the target document data block, and obtaining a second prediction result of the target document data block based on the first prediction result of the preceding document data block and the target document data block includes:

[0120] Determine the target document data block among the plurality of document data blocks, wherein the target document data block is any one of the plurality of document data blocks;

[0121] If it is determined that the target document data block in the plurality of document data blocks has a corresponding preceding document data block, the first prediction result of the preceding document data block is obtained;

[0122] Based on the first prediction result and the target document data block, a second prediction result for the target document data block is obtained.

[0123] It should be noted that the step of sequentially taking any one of the plurality of document data blocks as the target document data block, and obtaining the second prediction result of the target document data block based on the first prediction result of the preceding document data block and the target document data block, can be implemented using a data prediction module in a data processing model. This data prediction module is used to perform prediction processing on the document data blocks to obtain the corresponding prediction results. This data prediction module can be understood as one or more network layers in the data processing model, or it can be understood as a sub-model in the data processing model. For example, the data prediction module can be an inference network layer for language inference in the data processing model.

[0124] Following the previous example, the inference network layer in model M used for language inference is used to infer the text segment of the first window (i.e., the first document data block) to obtain the corresponding inferred text (i.e., the prediction result). Then, for the text segment of the second window, the inferred text corresponding to the text segment of the first window and the text segment of the second window are used to infer the corresponding inferred text. The above process is repeated until the encoding is completed and the inferred text of all text segments is obtained.

[0125] In the above embodiments, by sequentially taking any one of the multiple document data blocks as the target document data block and determining the preceding document data block corresponding to the target document data block, and then performing prediction processing based on the first prediction result of the preceding document data block and the target document data block, a second prediction result of the target document data block is obtained. This fully considers the correlation between the multiple document data blocks segmented from the document data to be processed. Therefore, by performing prediction processing based on the first prediction result of the preceding document data block and the target document data block, accurate target prediction results can be obtained subsequently.

[0126] In one or more embodiments provided in this disclosure, the step of predicting based on the first prediction result and the target document data block to obtain a second prediction result for the target document data block includes:

[0127] Using the data prediction module in the data processing model, prediction is performed based on the first prediction result and the target document data block to obtain the predicted text data corresponding to the target document data block.

[0128] Using the data compression module in the data processing model, a target compression placeholder corresponding to the target document data block is determined, and the predicted text data is compressed into the target compression placeholder using a preset attention matrix to obtain the second prediction result of the target document data block. The target compression placeholder is a compression placeholder between the target document data block and the next document data block corresponding to the target document data block.

[0129] The predicted text data can be understood as the inference text corresponding to the target document data block.

[0130] The data compression module can be understood as a module in the data processing model used to compress the predicted text data to a compression placeholder. The data compression module can be one or more network layers in the data processing model, or it can be a sub-model in the data processing model; for example, the data compression module can be a window compression attention mechanism module.

[0131] The target compression placeholder corresponding to the target document data block can be understood as a compression placeholder located after the target document data block; for example, after inserting a compression placeholder between the second document data block and the third document data block, when the second document data block is the target document data block, the compression placeholder can be the target compression placeholder corresponding to the target document data block.

[0132] The preset attention matrix can be understood as a pre-designed attention matrix used to compress the predicted text data into the target compression placeholder. It should be noted that the preset attention matrix can be a matrix that uses a mask to segment each document data block. The specific preset attention matrix can be seen in Figure 4. Based on the attention matrix A in Figure 4, it can be seen that the preset attention matrix can use a window mask to separate the text in each window from other windows, and only retain the compression placeholder [COMP] of each window to interact with the text (i.e., the inference text) of other windows.

[0133] In one or more embodiments provided in this disclosure, the prediction result may be a compressed placeholder containing the predicted text data.

[0134] Following the previous example, we use the reasoning network layer in model M for language reasoning to reason about the text segment in the first window and obtain the corresponding reasoned text (i.e., the token).

[0135] Using the window compression attention mechanism module in model M, when inference is completed in the text segment of the first window, the obtained token is compressed into the placeholder [COMP]1 based on the attention weight matrix to obtain the vector representation of [COMP]1.

[0136] The placeholder [COMP]1 and the text of the second window are used as the actual input data for the second round of iterative inference. They are input into the inference network layer of model M to infer the text of the second window.

[0137] Similarly, after reasoning through the text of the second window, the obtained token is compressed into the placeholder [COMP]2 to obtain the vector representation of [COMP]2;

[0138] The vector representations of [COMP]2 and [COMP]1 are concatenated to obtain a concatenated vector. The text of the next window and the concatenated vector are then re-inputted into model M for inference of the next window.

[0139] Repeat the above process until the last [COMP] placeholder is encoded.

[0140] In one or more embodiments provided in this disclosure, the step of using the data prediction module in the data processing model to predict based on the first prediction result and the target document data block to obtain the predicted text data corresponding to the target document data block includes:

[0141] Using the position determination module in the data processing model, the first position code corresponding to the preceding document data block and the first prediction result is determined, and the second position code corresponding to the target document data block is determined based on the first position code;

[0142] The first prediction result, the target document data block, and the second position code are input into the data prediction module in the data processing model. The data prediction module then performs a prediction based on the first prediction result, the target document data block, and the second position code to obtain the predicted text data corresponding to the target document data block.

[0143] The location determination module can be understood as a module in the data processing model used to determine the location encoding. The location determination module can be one or more network layers in the data processing model, or it can be a sub-model in the data processing model; for example, the location determination module can be a location encoding continuation module.

[0144] This location encoding can be understood as an encoding used to characterize the location of document data blocks and prediction results. For example, this location encoding can be the actual location encoding. Correspondingly, the first position code can be the position code corresponding to the preceding document data block and the first prediction result; the second position code can be the position code corresponding to the target document data block.

[0145] Following the previous example, the reasoning network layer in Model M used for language reasoning is used to reason about the text segment of the first window to obtain the corresponding reasoning text (i.e., the token). Furthermore, the window compression attention mechanism module in Model M is used to compress the obtained token into the placeholder [COMP]1 when the reasoning of the text segment of the first window is completed, thus obtaining the vector representation of [COMP]1.

[0146] The positional encoding continuation module in model M is used to perform positional encoding on the placeholder [COMP]1 to obtain the positional encoding corresponding to the placeholder [COMP]1 (e.g., the actual positional encoding during the training phase mentioned above). ).

[0147] The placeholder [COMP]1 and the position code are used as the actual input data for the second round of iterative inference and are input into the inference network layer of model M to perform inference on the second window.

[0148] Similarly, after reasoning through the text of the second window, the obtained token is compressed into the placeholder [COMP]2 to obtain the vector representation of [COMP]2;

[0149] The vector representations of [COMP]2 and [COMP]1 are concatenated to obtain a concatenated vector. The text of the next window, the concatenated vector, and the actual positions of the two placeholders ([COMP]2 and [COMP]1) are encoded and re-input into model M for inference of the next window.

[0150] Repeat the above process until the last [COMP] placeholder is encoded.

[0151] In one or more embodiments provided in this disclosure, after determining the target document data block among the plurality of document data blocks, the method further includes:

[0152] If the target document data block does not have a corresponding preceding document data block, a prediction is made based on the target document data block to obtain a second prediction result for the target document data block.

[0153] Following the previous example, the reasoning network layer in Model M used for language reasoning is used to reason about the text segment of the first window to obtain the corresponding reasoning text (i.e., the token). Furthermore, using the window compression attention mechanism module in Model M, when the reasoning of the text segment of the first window is completed, the obtained token is compressed into the placeholder [COMP]1 to obtain the vector representation of [COMP]1 (i.e., the second prediction result).

[0154] Step 208: Based on the first prediction result and the second prediction result, obtain the target prediction result of the document data to be processed.

[0155] In one or more embodiments provided in this disclosure, obtaining the target prediction result of the document data to be processed based on the first prediction result and the second prediction result includes:

[0156] The first prediction result and the second prediction result are converted into text to obtain the predicted text data corresponding to the document data to be processed.

[0157] Continuing with the previous example, concatenate all the [COMP] placeholders with the text of the last window, and perform true model reasoning, identical to traditional reasoning methods, to obtain the final model generation result text.

[0158] In one or more embodiments provided in this disclosure, after obtaining the target prediction result of the document data to be processed based on the first prediction result and the second prediction result, the method further includes:

[0159] The target prediction result is sent to the client so that the client can display the target prediction result to the user.

[0160] Specifically, after using the data processing model to perform document data processing operations on the document data to be processed and obtaining the target prediction result, the target prediction result is sent to the client, and then the target prediction result is displayed to the user through the client, thereby meeting the user's processing needs for the document data to be processed.

[0161] This disclosure provides a document data processing method in one or more embodiments. Considering the problem that large document data can lead to neural network models being unable to accurately process it, the method utilizes a data processing model to segment the document data into multiple sequentially arranged document data blocks during processing. This facilitates targeted prediction of each document data block, resulting in accurate predictions. Furthermore, considering the correlation between the multiple document data blocks segmented from the document data, to ensure accurate target prediction results, any one of the multiple data blocks is sequentially used as the target data block. A second prediction result for the target data block is obtained based on the first prediction result of the preceding data block and the target data block. Finally, an accurate target prediction result is obtained based on the first and second prediction results. This avoids the problem of low accuracy in data processing results caused by the neural network model's inability to accurately process large document data. Therefore, this method achieves accurate data processing results using a data processing model for large document data.

[0162] The following description, in conjunction with Figure 3, uses the application of the document data processing method provided in this disclosure in processing scenarios with extremely long contextual inputs as an example to further illustrate the document data processing method. Figure 3 shows a flowchart of the processing procedure of a document data processing method provided in an embodiment of this disclosure.

[0163] As shown in Figure 3, the document data processing method provided in this disclosure includes a training phase and an inference phase. The training phase includes a data preprocessing module and a model training module for training the model M (e.g., an LLM model). The inference phase includes a model inference module for performing inference operations on the trained model M. The specific explanations of the training and inference phases are as follows:

[0164] The training phase can be implemented through a data preprocessing module and a model training module. The specific execution methods of the data preprocessing module and the model training module are as follows:

[0165] The data preprocessing module is executed as follows:

[0166] Given a neural network-based language model M, the text length that M can process is pre-defined as k, meaning that most data is less than K in length.

[0167] It should be noted that models trained using the standard method perform well in predicting texts of length K or less, but perform poorly when the text length is greater than K.

[0168] To address the above issues, this solution defines the existing training data available for the M-model as follows: Here, X is used to refer to any piece of data.

[0169] As shown in Figure 3, for any training data X with a text length exceeding k (k < K) in, the model is divided into multiple data blocks that match the window size according to its own window size k. That is, every k lengths, a special placeholder [COMP] is added at the corresponding position in the text; in this way, the ultra-long context input (such as a paper document) is divided into multiple data blocks with better prediction effects.

[0170] Among them, [COMP] is used to guide the model M to compress the information of the text within its window into [COMP] during the training process.

[0171] It should be noted that: all [COMP] are the same placeholder, and its subscript distinction is only used to distinguish different usage positions. During the training and prediction processes, its specific hidden layer vector will change with the previous context, but they are all used to require the model to "compress the previous text", and its vector can be used to replace the cumulative vector information of all the previous text.

[0172] Among them, the execution method of this model training module is:

[0173] The model training module of this solution further includes 3 sub-modules: the window compression attention mechanism module, the compression placeholder differentiation module, and the position encoding continuation module. Among them, the window compression attention mechanism module is the core part of this solution, and the other two sub-modules are enhancement modules for this core module; the following will introduce these three sub-modules one by one.

[0174] For the window compression attention mechanism module, its execution method is:

[0175] For each piece of training data X, only by masking the attention weight matrix (Attention Matrix) in the model, the model is forced to compress the text information within each window into [COMP].

[0176] As shown in the attention matrix A in Figure 4, in this solution, based on the generative attention weight matrix, the text within each window is separated from other windows by a window mask, and only the placeholder [COMP] in each window can interact with the text of other windows. Among them, C n is the abbreviation of [COMP] n

[0177] By using such an attention mask, without modifying any other parameters or training methods, the model can be forced to learn the training objective (argmax...) shown in Figure 3, that is, to force the model to compress the text information in the corresponding window into the [COMP] placeholder of each window.

[0178] It should be noted that attention matrices B and C in Figure 4 are examples of masking the attention matrices when applying this scheme to models of other existing frameworks. For example, in the bidirectional encoding framework (e.g., applied to the BERT model) and encoder-decoder framework (e.g., applied to the T5 model) in Figure 4, only the masking method of the attention weight matrix needs to be adjusted accordingly, and the same inference method can be performed.

[0179] The execution method for the compressed placeholder differentiation module is as follows:

[0180] Based on the shortcomings of the [ANCHOR] scheme mentioned above, if only the [COMP] placeholder is used to continuously compress the preceding text, on the one hand, it is easy for subsequent [COMP] placeholders to contain information that is identical to their preceding [COMP] placeholders; on the other hand, it is easy for all [COMP] information to reach the information bottleneck, for example, the cosine similarity of vector representations is extremely high.

[0181] Based on this, this solution employs a contrastive learning training task. By introducing this task, the model is guided to prioritize compressing text information between the current and previous [COMPs] when compressing text information into [COMPs], while simultaneously reducing the amount of identical information between the current [COMP] and its predecessors. This ensures both the continuity and differentiation of information between [COMPs].

[0182] The following example will explain how the compression placeholder differentiation module is executed.

[0183] Taking the input text sequence X = (x1, x2, x3, x4, x5, x6, x7) from Figure 3 as an example, the specific contrastive learning training task described above is as follows:

[0184] 1. Insert [COMP]1 (C1) and [COMP]2 (C2) into the text sequence to form a new text sequence.

[0185] 2. Transform the new text sequence Convert the word embedding vector matrix into a word embedding vector matrix as the model input. The embedding vector matrix is... in and The initialization is the same vector, and the subscripts 1 and 2 are only used to distinguish the [COMP] placeholders in different positions.

[0186] 3. For the sequence Assign position sequence A position-encoded vector matrix can be obtained by vector mapping from the position sequence.

[0187] 4. and Represents a text sequence The input model is encoded to obtain a vector representation matrix. It should be noted that the contrastive learning task will be applied to vector representations of matrices. superior.

[0188] 5. For new text sequences As shown in Figure 5, the positive examples of C1 are the text X1 and X2 in the preceding window; C1 cannot see the following text, and therefore cannot see the subsequent compressed placeholder C2. Thus, C1 has no negative examples. Based on the positive and negative examples of C1, its vector representation needs to be determined. They are brought closer to h1 and h2 respectively.

[0189] 6. Similarly, as shown in Figure 5, the positive examples of C2 are the text x3, x4, and x5 in the preceding window; the negative examples of C2 are all the previously compressed placeholders, i.e., C1; based on the positive and negative examples of C2, their vector representations are needed. Closer to h3, h4, and h5 respectively, and with Pull it away.

[0190] Based on the above, we can see that the optimization objective (i.e., loss function) of the comparison training task can be found in the following formula.

[0191] in, For any compressed placeholder C * The vector representation of , C * The range of subscripts for positive example words. C * The vector representation of the contrast negative example.

[0192] It should be noted that the example above only shows the case where the input text contains two compressed placeholders. If there are more compressed placeholders, the second and subsequent compressed placeholders will be executed in the same way as C2, which will not be elaborated on here.

[0193] The execution method for the position encoding continuation module is as follows:

[0194] It should be noted that the explanation section regarding the compression placeholder differentiation module has already described how this scheme affects the new text sequence after inserting the compression placeholder. The method for assigning positions is as follows: assign position 3 to C1, and continue the positions of X3 to X5 accordingly; assign position 7 to C2, and continue the positions of X6 to X7 accordingly; and so on in this natural continuation. Position sequence Then obtain the corresponding position code.

[0195] It should be noted that the method proposed in this technical solution is contrasted with the "compressed placeholder position reset method," which resets the position each time a placeholder is encountered; the resulting position sequence is as follows: That is, C1 is reset to position 1, and the positions of x3 to x5 are extended accordingly; C2 continues the position of the previous compressed placeholder, continuing to position 2, and the positions of x6 to x7 are extended accordingly.

[0196] To address the shortcomings of the "compressed placeholder position resetting method", this solution proposes a method that is more conducive to the model's perception of the original text order, thereby enabling the model to perceive the current text length. During the training phase, the model can learn "how to generate long text", and during the inference phase, it can avoid mistakenly assuming that the current text length is short, which would lead to the tendency to generate longer and redundant text.

[0197] Based on the above description of the model training phase, it can be seen that the proposed technique is to adapt the data to the model. Therefore, this approach does not require grouping the data by length, nor does it require carefully designing different training strategies for multiple stages, and it will not produce training effect coverage. In addition, by introducing the [COMP] placeholder to compress the text within a certain window, the model can theoretically handle contextual inputs of text length close to Kk+k(1-k) / 2.

[0198] To address the shortcomings of the [GIST] scheme, this approach overcomes the limitation of compressing only commands. Instead, it extends the compression scope to the text of all windows and achieves exponential compression of extremely long context inputs using multi-window compression, with k as the base. Furthermore, this method requires only simple modification of the training data and does not require the additional construction of task-specific triplet data.

[0199] The execution process of this inference phase can be implemented through the model inference module, and the specific execution method of the model inference module is as follows:

[0200] The model inference module of this solution further includes two sub-modules: a window compression attention mechanism module and a positional encoding continuation module. The compressed placeholder differentiation module is only used during the model training phase and does not participate in the execution of the model inference phase. Therefore, the execution method of the model inference phase is as follows:

[0201] 1. When receiving input with extremely long context (such as a thesis document), the input text will also be input into the data processing module.

[0202] 2. Using the data processing module, divide the data into blocks corresponding to the window based on the model's window size, insert a [COMP] separator between two data blocks, and then divide the text into multiple segments according to the window size to facilitate subsequent inference in a serial manner.

[0203] 3. Using the reasoning network layer in model M for language reasoning, reason about the text segment in the first window to obtain the corresponding reasoned text (i.e., the token).

[0204] 4. Using the window compression attention mechanism module in model M, when the inference is completed in the text segment of the first window, the obtained token is compressed into the placeholder [COMP]1 to obtain the vector representation of [COMP]1.

[0205] 5. Use the positional encoding continuation module in model M to perform positional encoding on the placeholder [COMP]1 to obtain the positional encoding corresponding to the placeholder [COMP]1 (e.g., the actual positional encoding during the training phase mentioned above). ).

[0206] 6. Use the placeholder [COMP]1 and the position code as the actual input data for the second round of iterative inference, and input them into the inference network layer of model M to perform inference on the second window.

[0207] 7. After completing the reasoning for the text in the second window, compress the obtained token into the placeholder [COMP]2 to obtain the vector representation of [COMP]2;

[0208] 8. Concatenate the vector representations of [COMP]2 and [COMP]1, add the actual position encodings of the two placeholders ([COMP]2 and [COMP]1), and re-input them into model M for the next window of inference.

[0209] 9. Repeat the above process until the last [COMP] placeholder is encoded.

[0210] 10. Concatenate all [COMP] placeholders with the text of the last window, perform true model reasoning identical to traditional reasoning methods, and obtain the final model generation result text. Traditional reasoning methods can include: performing reasoning analysis on the paper to obtain a brief overview of the paper; or, based on user needs, performing reasoning analysis on the paper to obtain the outline of the paper, etc.; no specific restrictions are placed on traditional reasoning methods here.

[0211] Based on the above, this disclosure provides a general framework for generative model training and inference that handles ultra-long contextual inputs. This framework breaks through the constraints of multi-stage training methods and proposes a text compression training and inference scheme that adapts data to the model, making full use of the model's capabilities to compress input text exponentially, thereby overcoming the limitation of ultra-long contextual inputs.

[0212] Compared with existing technologies, the proposed solution is to adapt the data to the model; therefore, it does not require data grouping or designing different training strategies in multiple stages. Theoretically, it can adapt to the exponentially long context of the original window and will not produce the effect of training effect coverage.

[0213] It should be noted that, regarding the training method of special placeholders, considering that the [ANCHOR] scheme does not allow special placeholders to interact with each other in order to "extremely maximize the compression effect", this technical solution believes that interaction between special placeholders is necessary to ensure the continuity of compressed information.

[0214] This approach introduces an additional contrastive learning training subtask to make the information compressed by special placeholders more differentiated and avoid high information duplication.

[0215] Furthermore, this scheme employs a continuation process for positional encoding, meaning that during training and inference, each character / special placeholder in the model's input text is assigned a positional encoding in its natural order, rather than resetting the positional encoding. This avoids the model's misperception of sequence length, thus preventing the model from tending to generate excessively long text.

[0216] Referring to Figure 6, Figure 6 shows a flowchart of a data processing method provided according to an embodiment of the present disclosure, which specifically includes the following steps.

[0217] Step 602: Determine the data to be processed and input the data to be processed into the data processing model.

[0218] Step 604: In the data processing model, the data to be processed is segmented to obtain multiple data blocks arranged in sequence.

[0219] Step 606: Sequentially select any one of the plurality of data blocks as the target data block, and obtain the second prediction result of the target data block based on the first prediction result of the preceding data block and the target data block, wherein the preceding data block is the data block located before the target data block among the plurality of data blocks.

[0220] Step 608: Based on the first prediction result and the second prediction result, obtain the target prediction result of the data to be processed.

[0221] In one or more embodiments provided in this disclosure, the step of segmenting the data to be processed in the data processing model to obtain a plurality of sequentially arranged data blocks includes:

[0222] In the data processing model, the data to be processed is divided into multiple data blocks to be segmented using the data prediction size of the data processing model;

[0223] Determine any two adjacent data blocks to be split among the plurality of data blocks to be split, and insert a compression placeholder between the two adjacent data blocks to be split;

[0224] The unprocessed data, which includes the compression placeholder and the plurality of data blocks to be segmented, is segmented to obtain the plurality of data blocks arranged in sequence, wherein the compression placeholder exists between any two adjacent data blocks.

[0225] In one or more embodiments provided in this disclosure, the step of sequentially taking any one of the plurality of data blocks as the target data block and obtaining a second prediction result of the target data block based on the first prediction result of the preceding data block and the target data block includes:

[0226] Determine the target data block among the plurality of data blocks, wherein the target data block is any one of the plurality of data blocks;

[0227] If it is determined that the target data block among the plurality of data blocks has a corresponding preceding data block, the first prediction result of the preceding data block is obtained;

[0228] Based on the first prediction result and the target data block, a second prediction result for the target data block is obtained.

[0229] In one or more embodiments provided in this disclosure, the step of predicting based on the first prediction result and the target data block to obtain a second prediction result for the target data block includes:

[0230] Using the data prediction module in the data processing model, prediction is made based on the first prediction result and the target data block to obtain the predicted data corresponding to the target data block;

[0231] Using the data compression module in the data processing model, a target compression placeholder corresponding to the target data block is determined, and the predicted data is compressed into the target compression placeholder using a preset attention matrix to obtain the second prediction result of the target data block. The target compression placeholder is a compression placeholder between the target data block and the next data block corresponding to the target data block.

[0232] In one or more embodiments provided in this disclosure, the step of using the data prediction module in the data processing model to make a prediction based on the first prediction result and the target data block to obtain the predicted data corresponding to the target data block includes:

[0233] Using the position determination module in the data processing model, the first position code corresponding to the preceding data block and the first prediction result is determined, and the second position code corresponding to the target data block is determined based on the first position code.

[0234] The first prediction result, the target data block, and the second position code are input into the data prediction module in the data processing model. The data prediction module then performs a prediction based on the first prediction result, the target data block, and the second position code to obtain the predicted data corresponding to the target data block.

[0235] In one or more embodiments provided in this disclosure, after determining the target data block among the plurality of data blocks, the method further includes:

[0236] If the target data block does not have a corresponding preceding data block, a prediction is made based on the target data block to obtain a second prediction result for the target data block.

[0237] In one or more embodiments provided in this disclosure, determining the data to be processed includes:

[0238] Receive pending data sent by the client, wherein the pending data is sent by the client based on the user's data upload operation for the pending data;

[0239] After obtaining the target prediction result of the data to be processed based on the first prediction result and the second prediction result, the method further includes:

[0240] The target prediction result is sent to the client so that the client can display the target prediction result to the user.

[0241] The data processing method provided in one or more embodiments of this disclosure addresses the problem that large amounts of data to be processed can cause neural network models to be unable to accurately process the data. Therefore, during the processing of the data to be processed, the data processing model is used to segment the data to be processed, obtaining multiple data blocks arranged in sequence. This facilitates targeted prediction of each data block and obtains accurate prediction results. Furthermore, considering the correlation between the multiple data blocks segmented from the data to be processed, in order to ensure accurate target prediction results, any one of the multiple data blocks is sequentially taken as the target data block. A second prediction result for the target data block is obtained based on the first prediction result of the preceding data block and the target data block. Finally, an accurate target prediction result is obtained based on the first prediction result and the second prediction result. This avoids the problem that the neural network model cannot accurately process the large amounts of data to be processed, resulting in low accuracy in the output data processing results. It achieves the goal of obtaining accurate data processing results using a data processing model for large amounts of data to be processed.

[0242] The above is an illustrative scheme of a data processing method according to this embodiment. It should be noted that the technical solution of this data processing method belongs to the same concept as the technical solution of the document data processing method described above. For details not described in detail in the technical solution of the data processing method, please refer to the description of the technical solution of the document data processing method described above.

[0243] Referring to Figure 7, Figure 7 shows a flowchart of a data processing model training method according to an embodiment of the present disclosure, which specifically includes the following steps.

[0244] Step 702: Determine the data processing model to be trained, and the sample data associated with the target task;

[0245] Step 704: Input the sample data into the data processing model to be trained. In the data processing model, the sample data is segmented to obtain multiple data blocks arranged in sequence.

[0246] Step 706: Sequentially select any one of the plurality of data blocks as the target data block, and obtain the second prediction result of the target data block based on the first prediction result of the preceding data block and the target data block, wherein the preceding data block is the data block located before the target data block among the plurality of data blocks;

[0247] Step 708: Based on the first prediction result and the second prediction result, obtain the sample prediction result of the sample data;

[0248] Step 710: Adjust the model parameters of the data processing model based on the sample prediction results to obtain a trained data processing model, wherein the data processing model is used to perform the target task.

[0249] In one or more embodiments provided in this disclosure, obtaining a second prediction result for the target data block based on a first prediction result of the preceding data block and the target data block includes:

[0250] Based on the first prediction result of the preceding data block and the target data block, a prediction is made to obtain the prediction result to be adjusted for the target data block;

[0251] Using the comparison adjustment unit in the data processing model, the first prediction result is used as negative example data, the target data block is used as positive example data, and the prediction result to be adjusted is adjusted based on the negative example data and the positive example data to obtain the second prediction result of the target data block.

[0252] The comparison adjustment unit can be understood as a module in the data processing model used to adjust the prediction results. The comparison adjustment unit can be one or more network layers in the data processing model, or it can be a sub-model in the data processing model. For example, the comparison adjustment unit can be a compression placeholder differentiation module.

[0253] Specifically, the data processing model training method provided in one or more embodiments of this disclosure can make predictions based on the first prediction results of the preceding data block and the target data block during the process of obtaining the second prediction result of the target data block, thereby obtaining the prediction result to be adjusted for the target data block.

[0254] Then, using the comparison adjustment unit in the data processing model, the first prediction result is used as negative example data, the target data block is used as positive example data, and the prediction result to be adjusted is adjusted to have a high similarity to the positive example data and a low similarity to the negative example data; the adjusted prediction result is used as the second prediction result of the target data block.

[0255] For example, taking the input text sequence X = (x1, x2, x3, x4, x5, x6, x7) from Figure 3 as an example, the specific contrastive learning training task described above is as follows:

[0256] 1. Insert [COMP]1 (C1) and [COMP]2 (C2) into the text sequence to form a new text sequence.

[0257] 2. Transform the new text sequence Convert the word embedding vector matrix into a word embedding vector matrix as the model input. The embedding vector matrix is... in and The initialization is the same vector, and the subscripts 1 and 2 are only used to distinguish the [COMP] placeholders in different positions.

[0258] 3. For the sequence Assign position sequence A position-encoded vector matrix can be obtained by vector mapping from the position sequence.

[0259] 4. and Represents a text sequence The input model is encoded to obtain a vector representation matrix. It should be noted that the contrastive learning task will be applied to vector representations of matrices. superior.

[0260] 5. For new text sequences As shown in Figure 5, the positive examples of C1 are the text X1 and X2 in the preceding window; C1 cannot see the following text, and therefore cannot see the subsequent compressed placeholder C2. Thus, C1 has no negative examples. Based on the positive and negative examples of C1, its vector representation needs to be determined. They are brought closer to h1 and h2 respectively.

[0261] Specifically, as shown in Figure 5, the positive examples of C2 are the text x3, x4, and x5 in the preceding window; the negative examples of C2 are all the previous compressed placeholders, i.e., C1. Based on the positive and negative examples of C2, it is necessary to use the compressed placeholder differentiation module to represent them as vectors. Closer to h3, h4, and h5 respectively, and... Pull it away.

[0262] The data processing model training method provided in one or more embodiments of this disclosure addresses the problem that large amounts of data to be processed can prevent neural network models from accurately processing such data. Therefore, to ensure that the data processing model can handle large amounts of data and obtain accurate results, it trains the model using sample data. During model training, the sample data is segmented within the data processing model to obtain multiple sequentially arranged data blocks. This facilitates targeted prediction of each data block, leading to accurate prediction results. Furthermore, considering the correlation between the multiple data blocks segmented from the sample data, to ensure accurate sample prediction results, the multiple data blocks are sequentially... Any data block in the data block is used as the target data block. Based on the first prediction result of the preceding data block and the target data block, a second prediction result of the target data block is obtained. Then, based on the first and second prediction results, an accurate sample prediction result is obtained. Finally, the model parameters of the data processing model are adjusted using the sample prediction results to obtain a data processing model that can process large amounts of data and obtain accurate data processing results. This avoids the problem that the neural network model cannot accurately process large amounts of data, resulting in low accuracy in the output data processing results. This achieves the goal of obtaining accurate data processing results using a data processing model for large amounts of data.

[0263] The above is an illustrative scheme of a data processing model training method according to this embodiment. It should be noted that the technical solution of this data processing model training method belongs to the same concept as the document data processing method and the data processing method described above. For details not described in detail in the technical solution of the data processing model training method, please refer to the descriptions of the document data processing method and the data processing method described above.

[0264] Referring to Figure 8, Figure 8 shows a flowchart of another document data processing method provided according to an embodiment of the present disclosure, which specifically includes the following steps.

[0265] Step 802: Determine the document data to be processed;

[0266] Step 804: Input the document data to be processed into the data processing model to obtain the target prediction result of the document data to be processed. The data processing model obtains the target prediction result based on the document data to be processed in the following way: In the data processing model, the document data to be processed is segmented to obtain multiple document data blocks arranged in sequence. Any one of the multiple document data blocks is taken as the target document data block in sequence. Based on the first prediction result of the preceding document data block and the target document data block, a second prediction result of the target document data block is obtained. The preceding document data block is the document data block located before the target document data block among the multiple document data blocks. Based on the first prediction result and the second prediction result, the target prediction result of the document data to be processed is obtained.

[0267] This disclosure provides another document data processing method according to one or more embodiments. Considering the problem that large document data can lead to neural network models being unable to accurately process it, this method utilizes a data processing model to segment the document data into multiple sequentially arranged document data blocks. This facilitates targeted prediction of each document data block, resulting in accurate predictions. Furthermore, considering the correlation between the multiple document data blocks segmented from the document data, to ensure accurate target prediction results, any one of the multiple data blocks is sequentially used as the target data block. A second prediction result for the target data block is obtained based on the first prediction result of the preceding data block and the target data block. Finally, an accurate target prediction result is obtained based on the first and second prediction results. This avoids the problem of low accuracy in data processing results caused by the neural network model's inability to accurately process large document data. Therefore, this method achieves accurate data processing results using a data processing model for large document data.

[0268] The above is an illustrative scheme of another document data processing method in this embodiment. It should be noted that the technical solution of this other document data processing method belongs to the same concept as the above-described document data processing method and data processing method. For details not described in detail in the technical solution of the other document data processing method, please refer to the descriptions of the above-described document data processing method and data processing method.

[0269] Corresponding to the above method embodiments, this disclosure also provides a document data processing apparatus embodiment. Figure 9 shows a schematic diagram of the structure of a document data processing apparatus provided in one embodiment of this disclosure. As shown in Figure 9, the apparatus includes:

[0270] The data determination module 902 is configured to determine the document data to be processed and input the document data to be processed into the data processing model;

[0271] The data segmentation module 904 is configured to segment the document data to be processed in the data processing model to obtain multiple document data blocks arranged in sequence.

[0272] The data prediction module 906 is configured to sequentially take any one of the plurality of document data blocks as the target document data block, and obtain a second prediction result of the target document data block based on the first prediction result of the preceding document data block and the target document data block, wherein the preceding document data block is the document data block located before the target document data block among the plurality of document data blocks;

[0273] The result determination module 908 is configured to obtain the target prediction result of the document data to be processed based on the first prediction result and the second prediction result.

[0274] Optionally, the data segmentation module 904 is further configured to:

[0275] In the data processing model, the data to be processed is divided into multiple data blocks to be segmented using the data prediction size of the data processing model.

[0276] Determine any two adjacent data blocks to be split among the plurality of data blocks to be split, and insert a compression placeholder between the two adjacent data blocks to be split;

[0277] The unprocessed data, including the compression placeholder and the plurality of data blocks to be segmented, is segmented to obtain the plurality of document data blocks arranged in sequence, wherein the compression placeholder exists between any two adjacent document data blocks.

[0278] Optionally, the data prediction module 906 is further configured to:

[0279] Determine the target document data block among the plurality of document data blocks, wherein the target document data block is any one of the plurality of document data blocks;

[0280] If it is determined that the target document data block in the plurality of document data blocks has a corresponding preceding document data block, the first prediction result of the preceding document data block is obtained;

[0281] Based on the first prediction result and the target document data block, a second prediction result for the target document data block is obtained.

[0282] Optionally, the data prediction module 906 is further configured to:

[0283] Using the data prediction module in the data processing model, prediction is performed based on the first prediction result and the target document data block to obtain the predicted text data corresponding to the target document data block.

[0284] Using the data compression module in the data processing model, a target compression placeholder corresponding to the target document data block is determined, and the predicted text data is compressed into the target compression placeholder using a preset attention matrix to obtain the second prediction result of the target document data block. The target compression placeholder is a compression placeholder between the target document data block and the next document data block corresponding to the target document data block.

[0285] Optionally, the data prediction module 906 is further configured to:

[0286] Using the position determination module in the data processing model, the first position code corresponding to the preceding document data block and the first prediction result is determined, and the second position code corresponding to the target document data block is determined based on the first position code;

[0287] The first prediction result, the target document data block, and the second position code are input into the data prediction module in the data processing model. The data prediction module then performs a prediction based on the first prediction result, the target document data block, and the second position code to obtain the predicted text data corresponding to the target document data block.

[0288] Optionally, the data prediction module 906 is further configured to:

[0289] If the target document data block does not have a corresponding preceding document data block, a prediction is made based on the target document data block to obtain a second prediction result for the target document data block.

[0290] Optionally, the result determination module 908 is further configured to:

[0291] The first prediction result and the second prediction result are converted into text to obtain the predicted text data corresponding to the document data to be processed.

[0292] Optionally, the data determination module 902 is further configured to:

[0293] Receive pending data sent by the client, wherein the pending data is sent by the client based on the user's data upload operation for the pending data;

[0294] The document data processing device further includes a result sending module, configured as follows:

[0295] The target prediction result is sent to the client so that the client can display the target prediction result to the user.

[0296] The document data processing apparatus provided in one or more embodiments of this disclosure addresses the problem that large document data can lead to neural network models being unable to accurately process it. Therefore, during the processing of the document data, the data processing model is used to segment the document data into multiple sequentially arranged document data blocks. This facilitates targeted prediction of each document data block, resulting in accurate prediction results. Furthermore, considering the correlation between the multiple document data blocks segmented from the document data, to ensure accurate target prediction results, any one of the multiple data blocks is sequentially used as the target data block. A second prediction result for the target data block is obtained based on the first prediction result of the preceding data block and the target data block. Finally, an accurate target prediction result is obtained based on the first and second prediction results. This avoids the problem that large document data prevents neural network models from accurately processing the document data, resulting in low-accuracy data processing results. Thus, it achieves accurate data processing results using a data processing model for large document data.

[0297] The above is an illustrative scheme of a document data processing apparatus according to this embodiment. It should be noted that the technical solution of this document data processing apparatus and the technical solution of the document data processing method described above belong to the same concept. For details not described in detail in the technical solution of the document data processing apparatus, please refer to the description of the technical solution of the document data processing method described above.

[0298] Corresponding to the above method embodiments, this disclosure also provides a data processing apparatus embodiment. FIG10 shows a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this disclosure. As shown in FIG10, the apparatus includes:

[0299] The data determination module 1002 is configured to determine the data to be processed and input the data to be processed into the data processing model;

[0300] The data segmentation module 1004 is configured to segment the data to be processed in the data processing model to obtain multiple data blocks arranged in sequence.

[0301] The data prediction module 1006 is configured to sequentially take any one of the plurality of data blocks as the target data block, and obtain a second prediction result of the target data block based on the first prediction result of the preceding data block and the target data block, wherein the preceding data block is the data block located before the target data block among the plurality of data blocks;

[0302] The result determination module 1008 is configured to obtain the target prediction result of the data to be processed based on the first prediction result and the second prediction result.

[0303] Optionally, the data segmentation module 1004 is further configured to:

[0304] In the data processing model, the data to be processed is divided into multiple data blocks to be segmented using the data prediction size of the data processing model;

[0305] Determine any two adjacent data blocks to be split among the plurality of data blocks to be split, and insert a compression placeholder between the two adjacent data blocks to be split;

[0306] The unprocessed data, which includes the compression placeholder and the plurality of data blocks to be segmented, is segmented to obtain the plurality of data blocks arranged in sequence, wherein the compression placeholder exists between any two adjacent data blocks.

[0307] Optionally, the data prediction module 1006 is further configured to:

[0308] Determine the target data block among the plurality of data blocks, wherein the target data block is any one of the plurality of data blocks;

[0309] If it is determined that the target data block among the plurality of data blocks has a corresponding preceding data block, the first prediction result of the preceding data block is obtained;

[0310] Based on the first prediction result and the target data block, a second prediction result for the target data block is obtained.

[0311] Optionally, the data prediction module 1006 is further configured to:

[0312] Using the data prediction module in the data processing model, prediction is made based on the first prediction result and the target data block to obtain the predicted data corresponding to the target data block;

[0313] Using the data compression module in the data processing model, a target compression placeholder corresponding to the target data block is determined, and the predicted data is compressed into the target compression placeholder using a preset attention matrix to obtain the second prediction result of the target data block. The target compression placeholder is a compression placeholder between the target data block and the next data block corresponding to the target data block.

[0314] Optionally, the data prediction module 1006 is further configured to:

[0315] Using the position determination module in the data processing model, the first position code corresponding to the preceding data block and the first prediction result is determined, and the second position code corresponding to the target data block is determined based on the first position code.

[0316] The first prediction result, the target data block, and the second position code are input into the data prediction module in the data processing model. The data prediction module then performs a prediction based on the first prediction result, the target data block, and the second position code to obtain the predicted data corresponding to the target data block.

[0317] Optionally, the data prediction module 1006 is further configured to:

[0318] If the target data block does not have a corresponding preceding data block, a prediction is made based on the target data block to obtain a second prediction result for the target data block.

[0319] Optionally, the data determination module 1002 is further configured to:

[0320] Receive unprocessed document data sent by the client, wherein the unprocessed document data is sent by the client based on the user's document upload operation for the unprocessed document data;

[0321] The data processing device further includes a result sending module, configured to:

[0322] The target prediction result is sent to the client so that the client can display the target prediction result to the user.

[0323] The data processing apparatus provided in one or more embodiments of this disclosure addresses the problem that large amounts of data to be processed can cause neural network models to be unable to accurately process the data. Therefore, during the processing of the data to be processed, the data processing model is used to segment the data to be processed, obtaining multiple data blocks arranged in sequence. This facilitates targeted prediction of each data block and obtains accurate prediction results. Furthermore, considering the correlation between the multiple data blocks segmented from the data to be processed, in order to ensure accurate target prediction results, any one of the multiple data blocks is sequentially taken as the target data block. A second prediction result for the target data block is obtained based on the first prediction result of the preceding data block and the target data block. Finally, an accurate target prediction result is obtained based on the first prediction result and the second prediction result. This avoids the problem that the neural network model cannot accurately process the large amounts of data to be processed, resulting in low accuracy in the output data processing results. It achieves the goal of obtaining accurate data processing results using a data processing model for large amounts of data to be processed.

[0324] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0325] Corresponding to the above method embodiments, this disclosure also provides an embodiment of a data processing model training device. Figure 11 shows a schematic diagram of the structure of a data processing model training device provided in one embodiment of this disclosure. As shown in Figure 11, the device includes:

[0326] The data determination module 1102 is configured to determine the data processing model to be trained and the sample data associated with the target task;

[0327] The data segmentation module 1104 is configured to input the sample data into the data processing model to be trained, and to segment the sample data in the data processing model to obtain multiple data blocks arranged in sequence.

[0328] The data prediction module 1106 is configured to sequentially take any one of the plurality of data blocks as the target data block, and obtain a second prediction result of the target data block based on the first prediction result of the preceding data block and the target data block, wherein the preceding data block is the data block located before the target data block among the plurality of data blocks;

[0329] The result determination module 1108 is configured to obtain the sample prediction result of the sample data based on the first prediction result and the second prediction result;

[0330] The model training module 1110 is configured to adjust the model parameters of the data processing model based on the sample prediction results to obtain a trained data processing model, wherein the data processing model is used to perform the target task.

[0331] Optionally, the data prediction module 1106 is further configured to:

[0332] Based on the first prediction result of the preceding data block and the target data block, a prediction is made to obtain the prediction result to be adjusted for the target data block;

[0333] Using the comparison adjustment unit in the data processing model, the first prediction result is used as negative example data, the target data block is used as positive example data, and the prediction result to be adjusted is adjusted based on the negative example data and the positive example data to obtain the second prediction result of the target data block.

[0334] The data processing model training apparatus provided in one or more embodiments of this disclosure addresses the problem that large amounts of data to be processed can prevent neural network models from accurately processing such data. Therefore, to ensure the data processing model can handle large amounts of data and obtain accurate results, it trains the model using sample data. During model training, the sample data is segmented within the data processing model to obtain multiple sequentially arranged data blocks. This facilitates targeted prediction of each data block, leading to accurate prediction results. Furthermore, considering the correlation between the multiple data blocks segmented from the sample data, to ensure accurate sample prediction results, the multiple data blocks are sequentially... Any data block in the data block is used as the target data block. Based on the first prediction result of the preceding data block and the target data block, a second prediction result of the target data block is obtained. Then, based on the first and second prediction results, an accurate sample prediction result is obtained. Finally, the model parameters of the data processing model are adjusted using the sample prediction results to obtain a data processing model that can process large amounts of data and obtain accurate data processing results. This avoids the problem that the neural network model cannot accurately process large amounts of data, resulting in low accuracy in the output data processing results. This achieves the goal of obtaining accurate data processing results using a data processing model for large amounts of data.

[0335] The above is an illustrative scheme of a data processing model training device according to this embodiment. It should be noted that the technical solution of this data processing model training device and the technical solution of the data processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the data processing model training device, please refer to the description of the technical solution of the data processing model training method described above.

[0336] Corresponding to the above method embodiments, this disclosure also provides another document data processing apparatus embodiment. FIG12 shows a schematic diagram of the structure of another document data processing apparatus provided in an embodiment of this disclosure. As shown in FIG12, the apparatus includes:

[0337] Data determination module 1202 is configured to determine the document data to be processed;

[0338] The result determination module 1204 is configured to input the document data to be processed into a data processing model to obtain a target prediction result for the document data to be processed. The data processing model obtains the target prediction result based on the document data to be processed by: segmenting the document data to be processed into multiple sequentially arranged document data blocks; sequentially using any one of the multiple document data blocks as a target document data block; and obtaining a second prediction result for the target document data block based on a first prediction result of the preceding document data block and the target document data block. The preceding document data block is the document data block located before the target document data block among the multiple document data blocks. The target prediction result for the document data to be processed is obtained based on the first prediction result and the second prediction result.

[0339] This disclosure provides another document data processing apparatus in one or more embodiments. Considering the problem that large document data can lead to neural network models being unable to accurately process it, this apparatus, during the processing of the document data, utilizes a data processing model to segment the document data into multiple sequentially arranged document data blocks. This facilitates targeted prediction of each document data block, resulting in accurate prediction results. Furthermore, considering the correlation between the multiple document data blocks segmented from the document data, to ensure accurate target prediction results, any one of the multiple data blocks is sequentially used as the target data block. A second prediction result for the target data block is obtained based on the first prediction result of the preceding data block and the target data block. Finally, an accurate target prediction result is obtained based on the first and second prediction results. This avoids the problem of low accuracy in data processing results caused by the neural network model's inability to accurately process large document data. Therefore, this apparatus achieves accurate data processing results using a data processing model for large document data.

[0340] The above is an illustrative scheme of another document data processing apparatus according to this embodiment. It should be noted that the technical solution of this other document data processing apparatus belongs to the same concept as the technical solution of the other document data processing method described above. For details not described in detail in the technical solution of the other document data processing apparatus, please refer to the description of the technical solution of the other document data processing method described above.

[0341] Figure 13 shows a structural block diagram of a computing device 1300 according to an embodiment of the present disclosure. The components of the computing device 1300 include, but are not limited to, a memory 1310 and a processor 1320. The processor 1320 is connected to the memory 1310 via a bus 1330, and a database 1350 is used to store data.

[0342] The computing device 1300 also includes an access device 1340, which enables the computing device 1300 to communicate via one or more networks 1360. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1340 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0343] In one embodiment of this disclosure, the aforementioned components of the computing device 1300, as well as other components not shown in FIG. 13, may also be connected to each other, for example, via a bus. It should be understood that the computing device block diagram shown in FIG. 13 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0344] The computing device 1300 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1300 can also be a mobile or stationary server.

[0345] The processor 1320 is configured to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described document data processing method, data processing method, data processing model training method, or another document data processing method.

[0346] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to embodiments of a document data processing method, a data processing method, a data processing model training method, or another document data processing method, so the description is relatively simple. Relevant parts can be referred to in the description of embodiments of a document data processing method, a data processing method, a data processing model training method, or another document data processing method.

[0347] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described document data processing method, data processing method, data processing model training method, or another document data processing method.

[0348] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are relatively simple in description because they are substantially similar to an embodiment of a document data processing method, a data processing method, a data processing model training method, or another document data processing method. Relevant details can be found in the descriptions of the embodiments of a document data processing method, a data processing method, a data processing model training method, or another document data processing method.

[0349] An embodiment of this disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described document data processing method, data processing method, data processing model training method, or another document data processing method.

[0350] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the above-described document data processing method, data processing method, data processing model training method, or another document data processing method. For details not described in detail in the technical solution of the computer program product, please refer to the description of the above-described document data processing method, data processing method, data processing model training method, or another document data processing method.

[0351] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0352] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0353] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this disclosure are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0354] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0355] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit this disclosure to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. This disclosure selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: determining to-be-processed data and inputting the to-be-processed data into a data processing model; splitting the to-be-processed data in the data processing model to obtain a plurality of data blocks arranged in sequence; sequentially taking any data block in the plurality of data blocks as a target data block, and obtaining a second prediction result of the target data block according to a first prediction result of a preceding data block and the target data block, wherein the preceding data block is a data block located before the target data block in the plurality of data blocks; obtaining a target prediction result of the to-be-processed data according to the first prediction result and the second prediction result.

2. The data processing method of claim 1, wherein the splitting the to-be-processed data in the data processing model to obtain a plurality of data blocks arranged in sequence comprises: dividing the to-be-processed data into a plurality of to-be-split data blocks by using a data prediction size of the data processing model in the data processing model; determining any two adjacent to-be-split data blocks in the plurality of to-be-split data blocks, and inserting a compression placeholder between the any two adjacent to-be-split data blocks; splitting the to-be-processed data containing the compression placeholder and the plurality of to-be-split data blocks to obtain the plurality of data blocks arranged in sequence, wherein the compression placeholder exists between any two adjacent data blocks in the plurality of data blocks.

3. The data processing method of claim 1 or 2, wherein the sequentially taking any data block in the plurality of data blocks as a target data block, and obtaining a second prediction result of the target data block according to a first prediction result of a preceding data block and the target data block comprises: determining the target data block in the plurality of data blocks, wherein the target data block is any data block in the plurality of data blocks; in a case where the target data block in the plurality of data blocks exists corresponding to the preceding data block, obtaining the first prediction result of the preceding data block; performing prediction according to the first prediction result and the target data block to obtain a second prediction result of the target data block.

4. The data processing method of claim 3, wherein the performing prediction according to the first prediction result and the target data block to obtain a second prediction result of the target data block comprises: performing prediction according to the first prediction result and the target data block by using a data prediction module in the data processing model to obtain prediction data corresponding to the target data block; determining a target compression placeholder corresponding to the target data block by using a data compression module in the data processing model, and compressing the prediction data into the target compression placeholder by using a preset attention matrix to obtain the second prediction result of the target data block, wherein the target compression placeholder is a compression placeholder between the target data block and a next data block corresponding to the target data block.

5. The data processing method of claim 4, wherein the predicting, by the data predicting module in the data processing model, according to the first prediction result and the target data block, to obtain the prediction data corresponding to the target data block comprises: determining, by a position determining module in the data processing model, a first position code corresponding to the preceding data block and the first prediction result, and determining a second position code corresponding to the target data block based on the first position code; and inputting the first prediction result, the target data block, and the second position code into the data predicting module in the data processing model, and predicting, by the data predicting module, according to the first prediction result, the target data block, and the second position code, to obtain the prediction data corresponding to the target data block.

6. The data processing method of any one of claims 3-5, wherein after the determining the target data block in the plurality of data blocks, the method further comprises: predicting, according to the target data block, to obtain a second prediction result of the target data block, if the target data block does not have a corresponding preceding data block.

7. The data processing method of any one of claims 1-6, wherein the determining the to-be-processed data comprises: receiving to-be-processed data sent by a client, wherein the to-be-processed data is sent by the client based on a data uploading operation of a user on the to-be-processed data; and after the obtaining the target prediction result of the to-be-processed data according to the first prediction result and the second prediction result, the method further comprises: sending the target prediction result to the client, so that the client displays the target prediction result to the user.

8. A document data processing method, comprising: determining to-be-processed document data, and inputting the to-be-processed document data into a data processing model; splitting, in the data processing model, the to-be-processed document data to obtain a plurality of document data blocks arranged in sequence; sequentially taking any one of the plurality of document data blocks as a target document data block, and obtaining a second prediction result of the target document data block according to a first prediction result of a preceding document data block and the target document data block, wherein the preceding document data block is a document data block located before the target document data block in the plurality of document data blocks; and obtaining a target prediction result of the to-be-processed document data according to the first prediction result and the second prediction result.

9. The document data processing method of claim 8, wherein the splitting, in the data processing model, the to-be-processed document data to obtain a plurality of document data blocks arranged in sequence comprises: dividing, in the data processing model, the to-be-processed document data into a plurality of to-be-split data blocks by using a data prediction size of the data processing model; and determining any two adjacent to-be-split data blocks in the plurality of to-be-split data blocks, and inserting a compression placeholder between the any two adjacent to-be-split data blocks. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ splitting the to-be-processed document data including the compression placeholder and the plurality of to-be-split data blocks to obtain the plurality of document data blocks arranged in sequence, wherein the compression placeholder exists between any two adjacent document data blocks in the plurality of document data blocks.

10. The document data processing method of claim 8 or 9, wherein the sequentially taking any document data block in the plurality of document data blocks as a target document data block and obtaining a second prediction result of the target document data block according to the first prediction result of the preceding document data block and the target document data block comprises: determining the target document data block in the plurality of document data blocks, wherein the target document data block is any document data block in the plurality of document data blocks; in a case where the target document data block in the plurality of document data blocks has the corresponding preceding document data block, obtaining the first prediction result of the preceding document data block; and predicting according to the first prediction result and the target document data block to obtain a second prediction result of the target document data block.

11. The document data processing method of claim 10, wherein the predicting according to the first prediction result and the target document data block to obtain a second prediction result of the target document data block comprises: using a data prediction module in the data processing model to predict according to the first prediction result and the target document data block to obtain predicted text data corresponding to the target document data block; using a data compression module in the data processing model to determine a target compression placeholder corresponding to the target document data block, and using a preset attention matrix to compress the predicted text data into the target compression placeholder to obtain a second prediction result of the target document data block, wherein the target compression placeholder is a compression placeholder between the target document data block and a next document data block corresponding to the target document data block.

12. The document data processing method of claim 11, wherein the using the data prediction module in the data processing model to predict according to the first prediction result and the target document data block to obtain predicted text data corresponding to the target document data block comprises: using a position determination module in the data processing model to determine a first position code corresponding to the preceding document data block and the first prediction result, and determining a second position code corresponding to the target document data block based on the first position code; and inputting the first prediction result, the target document data block, and the second position code into the data prediction module in the data processing model, and using the data prediction module to predict according to the first prediction result, the target document data block, and the second position code to obtain predicted text data corresponding to the target document data block.

13. The document data processing method of any one of claims 10-12, wherein after the determining the target document data block in the plurality of document data blocks, the method further comprises: ​ ​ ​ ​ ​ ​ ​ If the target document data block does not correspond to the preceding document data block, a second prediction result of the target document data block is obtained by prediction based on the target document data block.

14. The document data processing method of any one of claims 8-13, wherein the obtaining of the target prediction result of the to-be-processed document data based on the first prediction result and the second prediction result comprises: performing text conversion on the first prediction result and the second prediction result to obtain predicted text data corresponding to the to-be-processed document data.

15. The document data processing method of any one of claims 8-14, wherein the determining of the to-be-processed document data comprises: receiving to-be-processed document data sent by a client, wherein the to-be-processed document data is sent by the client based on a document upload operation of a user on the to-be-processed document data. After the obtaining of the target prediction result of the to-be-processed document data based on the first prediction result and the second prediction result, the method further comprises: sending the target prediction result to the client to enable the client to display the target prediction result to the user.

16. A data processing model training method, comprising: determining a data processing model to be trained and sample data associated with a target task; inputting the sample data into the data processing model to be trained, splitting the sample data in the data processing model to be trained to obtain a plurality of data blocks arranged in sequence; sequentially taking any data block in the plurality of data blocks as a target data block, and obtaining a second prediction result of the target data block based on a first prediction result of a preceding data block and the target data block, wherein the preceding data block is a data block located before the target data block in the plurality of data blocks; obtaining a sample prediction result of the sample data based on the first prediction result and the second prediction result; performing model parameter adjustment on the data processing model based on the sample prediction result to obtain a trained data processing model, wherein the data processing model is used to perform the target task.

17. The data processing model training method of claim 16, wherein the obtaining of the second prediction result of the target data block based on the first prediction result of the preceding data block and the target data block comprises: obtaining a to-be-adjusted prediction result of the target data block by prediction based on the first prediction result of the preceding data block and the target data block; using a comparison adjustment unit in the data processing model to take the first prediction result as negative example data, take the target data block as positive example data, and adjust the to-be-adjusted prediction result based on the negative example data and the positive example data to obtain the second prediction result of the target data block.

18. A document data processing method, comprising: determining to-be-processed document data; inputting the to-be-processed document data into a data processing model to obtain a target prediction result of the to-be-processed document data, wherein the data processing model obtains the target prediction result according to the to-be-processed document data in the following manner: in the data processing model, the to-be-processed document data is split to obtain a plurality of sequentially arranged document data blocks, any document data block in the plurality of document data blocks is sequentially taken as a target document data block, a first prediction result of a preceding document data block and the target document data block are used to obtain a second prediction result of the target document data block, the preceding document data block is a document data block in the plurality of document data blocks located before the target document data block, and the target prediction result of the to-be-processed document data is obtained according to the first prediction result and the second prediction result.

19. A computing device comprising: a memory and a processor; the memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the data processing method of any one of claims 1 to 7, the document data processing method of any one of claims 8 to 15, the data processing method of any one of claims 16 to 17, or the document data processing method of claim 18.

20. A computer-readable storage medium storing computer programs / instructions, which, when executed by a processor, implement the steps of the data processing method of any one of claims 1 to 7, the document data processing method of any one of claims 8 to 15, the data processing method of any one of claims 16 to 17, or the document data processing method of claim 18.

21. A computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the data processing method of any one of claims 1 to 7, the document data processing method of any one of claims 8 to 15, the data processing method of any one of claims 16 to 17, or the document data processing method of claim 18.

Images