Data processing method and data processing apparatus

Abstract

The application provides a data processing method and a data processing device, relates to the field of artificial intelligence, improves text review efficiency, and avoids the problem of non-uniform review standards caused by manual review. The method comprises the following steps: obtaining text content of a document to be reviewed; inputting the text content of the document to be reviewed into a text review model to obtain revised content of the document, wherein the revised content comprises format revision, argument revision, evidence revision and argumentation revision; wherein the text review model is obtained by training based on text content of multiple historical versions of the document in a review process, revised content of the text content of the multiple historical versions, and reference materials of the revised content.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a data processing method and a data processing device. BACKGROUND

[0002] In order to improve the efficiency of text review, automatic review of text has been realized, and the automatic review method of text mainly focuses on the review of typos, grammar and other document formats in the document. However, the review of the content of the document such as arguments, evidence and reasoning is still in the form of manual review.

[0003] However, the above-mentioned manual review of text has the problem of long review time, and due to the differences in cognitive thinking and knowledge level of different people, it may also cause the problem of non-uniform review standard of text, resulting in no guiding significance of revision feedback to future text and causing burden to text management work. SUMMARY

[0004] The present application provides a data processing method and a data processing device, which improves the efficiency of text review and avoids the problem of non-uniform review standard caused by manual review.

[0005] In a first aspect, a data processing method is provided, comprising: obtaining the text content of a document to be reviewed; inputting the text content of the document to be reviewed into a text review model to obtain the revision content of the document, the revision content including format revision, argument revision, evidence revision and reasoning revision; wherein the text review model is obtained by training based on the text content of multiple historical versions of the document in the review process, the revision content of the multiple historical versions of the text content and the reference materials of the revision content.

[0006] In the present application, by inputting the text content of the document to be reviewed into the text review model, the revision content including the format, arguments, evidence and reasoning of the document is obtained. Compared with the existing method, the review of typos, grammar and other formats in the text content of the document is in an automatic manner, and the review of arguments, evidence and reasoning in the text content of the document is in a manual manner. The present application improves the efficiency of text review. In addition, since the text review of the present application is realized by the above-mentioned text review model, the problem of non-uniform review standard caused by manual review is avoided.

[0007] In a certain implementation form of the first aspect, before the text content of the document to be reviewed is obtained, the method further comprises: obtaining to-be-processed data, the to-be-processed data comprising text content of a plurality of historical versions of the plurality of documents in a review process, revision content of the plurality of historical versions of the text content, and reference materials of the revision content; inputting the to-be-processed data into a pre-trained model to obtain an output matrix of a self-attention layer, the pre-trained model adopting an architecture of a bidirectional encoder; taking the output matrix of the self-attention layer as a matrix of original text content feature vectors of the plurality of historical versions, training the pre-trained model to obtain the text review model.

[0008] In a certain implementation form of the first aspect, inputting the to-be-processed data into a pre-trained model to obtain an output matrix of a self-attention layer comprises: based on the to-be-processed data, performing text embedding and feature extraction to obtain a matrix of feature vectors of the text content, a matrix of feature vectors of the revision content, and a matrix of feature vectors of the reference materials; based on the matrix of feature vectors of the text content, the matrix of feature vectors of the revision content, and the matrix of feature vectors of the reference materials, obtaining a matrix of feature vectors of a concatenation of weighted text content and reference materials; based on the matrix of feature vectors of the revision content, the matrix of feature vectors of the concatenation of weighted text content and reference materials, and the self-attention layer of the pre-trained model, obtaining a first target matrix of feature vectors, a second target matrix of feature vectors, and a third target matrix of feature vectors; re-inputting the first target matrix of feature vectors, the second target matrix of feature vectors, and the third target matrix of feature vectors into the self-attention layer to obtain the output matrix of the attention layer.

[0009] In the present application, by inputting the text content of a plurality of historical versions of the plurality of documents in a review process, revision content of the plurality of historical versions of the text content, and reference materials of the revision content into a pre-trained model, and performing feature extraction by using an improved attention unit in the model, the pre-trained model is trained so that the pre-trained model can focus on revisions of arguments, evidence, and demonstrations of document content in addition to typos, syntax, and other document format aspects in the document, and a text review model is obtained. The efficiency of text review is improved, and the problem of non-uniform standards caused by manual review is avoided.

[0010] In a certain implementation form of the first aspect, the feature vector matrix of the text content, the feature vector matrix of the revised content and the feature vector matrix of the reference material are concatenated to obtain a feature vector matrix of the concatenated weighted text content and reference material, including: point-multiplying the feature vector matrix of the text content and the normalized feature vector matrix of the revised content to obtain a weighted feature vector matrix of the text content; and concatenating the weighted feature vector matrix of the text content and the feature vector matrix of the reference material to obtain the feature vector matrix of the concatenated weighted text content and reference material.

[0011] In a certain implementation form of the first aspect, before the feature vector matrix of the text content, the feature vector matrix of the revised content and the feature vector matrix of the reference material are obtained by inputting the to-be-processed data into the pre-trained model for text embedding and feature extraction, the method further includes: constructing a text library, the text library including at least one relevant text, the similarity of the at least one relevant text to the historical text being greater than or equal to a first preset threshold; obtaining a pre-trained word vector based on the at least one relevant text and the to-be-processed data; and obtaining the feature vector matrix of the text content, the feature vector matrix of the revised content and the feature vector matrix of the reference material based on the pre-trained word vector.

[0012] In a certain implementation form of the second aspect, the obtaining module is configured to: obtain to-be-processed data, the to-be-processed data including the text content of the multiple historical versions of the documents in the review process, the revised content of the multiple historical versions of the documents and the reference material of the revised content; and the processing module is configured to: input the to-be-processed data into a pre-trained model to obtain an output matrix of a self-attention layer, the pre-trained model adopting a bidirectional encoder architecture; take the output matrix of the self-attention layer as a feature vector matrix of the original text content of the multiple historical versions, train the pre-trained model to obtain the text review model.

[0013] In a certain implementation form of the second aspect, the obtaining module is configured to: obtain to-be-processed data, the to-be-processed data including the text content of the multiple historical versions of the documents in the review process, the revised content of the multiple historical versions of the documents and the reference material of the revised content; and the processing module is configured to: input the to-be-processed data into a pre-trained model to obtain an output matrix of a self-attention layer, the pre-trained model adopting a bidirectional encoder architecture; take the output matrix of the self-attention layer as a feature vector matrix of the original text content of the multiple historical versions, train the pre-trained model to obtain the text review model.

[0014] In some implementations of the second aspect, the processing module is configured to: perform text embedding and feature extraction based on the to-be-processed data to obtain a feature vector matrix of the text content, a feature vector matrix of the revised content, and a feature vector matrix of the reference material, the pre-trained model adopting a bidirectional encoder architecture; obtain a feature vector matrix of the weighted text content and reference material spliced based on the feature vector matrix of the text content, the feature vector matrix of the revised content, and the feature vector matrix of the reference material; obtain a first target feature vector matrix, a second target feature vector matrix, and a third target feature vector matrix based on the feature vector matrix of the revised content, the feature vector matrix of the weighted text content and reference material spliced, and a self-attention layer of the pre-trained model; and re-input the first target feature vector matrix, the second target feature vector matrix, and the third target feature vector matrix to the self-attention layer to obtain an output matrix of the attention layer.

[0015] In some implementations of the second aspect, the processing module is configured to: multiply the feature vector matrix of the text content and the normalized feature vector matrix of the revised content to obtain a weighted feature vector matrix of the text content; and splice the weighted feature vector matrix of the text content and the feature vector matrix of the reference material to obtain a feature vector matrix of the weighted text content and reference material spliced.

[0016] In some implementations of the second aspect, the obtaining module is configured to: construct a text library, the text library including at least one related text, the similarity of the at least one related text to the historical text being greater than or equal to a first preset threshold; obtain a pre-trained word vector based on the at least one related text and the to-be-processed data; and obtain the feature vector matrix of the text content, the feature vector matrix of the revised content, and the feature vector matrix of the reference material based on the pre-trained word vector.

[0017] In a third aspect, a processor is provided, including: an input circuit, an output circuit, and a processing circuit. The processing circuit is configured to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method in any possible implementation manner of the first aspect.

[0018] In the implementation process, the processor can be a chip, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a flip-flop, and various logic circuits. The input signal received by the input circuit can be received and input by, for example but not limited to, a receiver, the output signal output by the output circuit can be output to and transmitted by, for example but not limited to, a transmitter, and the input circuit and the output circuit can be the same circuit which is used as the input circuit and the output circuit at different times. The embodiments of the present application do not limit the specific implementation of the processor and various circuits.

[0019] In a fourth aspect, a processing apparatus is provided, including a processor and a memory. The processor is configured to read instructions stored in the memory, and can receive signals through a receiver and transmit signals through a transmitter to perform the method in any possible implementation manner of the first aspect.

[0020] Optionally, the processor is one or more, and the memory is one or more.

[0021] Optionally, the memory can be integrated with the processor, or the memory and the processor are separately arranged.

[0022] In the implementation process, the memory can be a non-transitory memory, for example, a read only memory (ROM), which can be integrated on the same chip as the processor, or arranged on different chips respectively. The embodiments of the present application do not limit the type of memory and the arrangement of the memory and the processor.

[0023] The processing apparatus in the fourth aspect above can be a chip, and the processor can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which is implemented by reading software codes stored in the memory. The memory can be integrated in the processor or exist independently.

[0024] In a fifth aspect, a computer program product is provided, including a computer program (also referred to as code or instructions), which, when executed, causes a computer to perform the method in any possible implementation manner of the first aspect.

[0025] In a sixth aspect, a computer readable storage medium is provided, which stores a computer program (also referred to as code or instructions), which, when executed on a computer, causes the computer to perform the method in any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1 is a schematic flow chart of a data processing method provided by an embodiment of the present application;

[0028] Figure 2 is a schematic diagram of a pre-training model provided by an embodiment of the present application;

[0029] Figure 3 is a schematic flow chart of a converter model training process provided by an embodiment of the present application;

[0030] Figure 4 is a schematic flow chart of an output matrix acquisition process of a self-attention layer provided by an embodiment of the present application;

[0031] Figure 5 is a schematic flow chart of another data processing method provided by an embodiment of the present application;

[0032] Figure 6 is a schematic block diagram of a data processing apparatus provided by an embodiment of the present application;

[0033] Figure 7 is a schematic block diagram of still another data processing apparatus provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments made by those skilled in the art according to the inspiration of the present embodiments are within the scope of protection of the present application.

[0035] It should be noted that the data processing method and data processing apparatus of the present disclosure can be used in the field of artificial intelligence, and can also be used in any field other than the field of artificial intelligence. The data processing method and data processing apparatus of the present disclosure are not limited to the application field.

[0036] In order to improve the efficiency of text review, automatic review of text has been realized, and the automatic review method of text mainly focuses on the review of typos, syntax and other document formats in the document, while the review of the content of the document such as arguments, evidence and argumentation still adopts the manual review mode.

[0037] However, the above-mentioned manual review mode of text has the problem of long review time, and due to the differences in cognitive thinking and knowledge level of different people, it may also cause the problem of non-uniform standard of text review, resulting in no guiding significance of revision feedback to future text and causing burden to text management work.

[0038] Therefore, the present application provides a data processing method and a data processing device, which obtains the text content of multiple historical versions of a plurality of documents in a review process, the revision content of the multiple historical versions of the text content and the reference materials of the revision content, inputs them into a pre-trained model, and uses the attention unit in the model to extract features, trains the parameters of the pre-trained model, and obtains a text review model. The output of the text review model includes revision content in terms of document format, arguments, evidence and argumentation. The efficiency of text review is improved, and the problem of non-uniform standard caused by manual review is avoided.

[0039] Before introducing the data processing method and data processing device provided by the embodiments of the present application, the following points are explained.

[0040] First, in the embodiments shown below, each term and English abbreviation, such as a listening event, is an example given for convenience of description, and should not constitute any limitation on the present application. The present application does not exclude the possibility of defining other terms capable of achieving the same or similar functions in existing or future protocols.

[0041] Second, in the embodiments shown below, the first, second and various numbers are only for differentiation for convenience of description, and do not limit the scope of the embodiments of the present application.

[0042] Third, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent the following cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including single item or any combination of multiple items. For example, at least one of a, b and c can represent: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c, where a, b and c can be single or multiple.

[0043] In order to make the purpose, technical solutions of the present application more clear and intuitive, the data processing method and data processing device provided by the present application will be described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0044] Figure 1 is a schematic flow chart of the data processing method 100 provided by the embodiments of the present application. The method can be executed by a data processing device. As shown in Figure 1 the method 100 can include the following steps:

[0045] S101, the data processing device acquires to-be-processed data, the to-be-processed data including text content of a plurality of historical versions of a plurality of documents in a review process, revision content of the plurality of historical versions of the text content, and reference materials of the revision content.

[0046] Exemplarily, the to-be-processed data includes text content of a historical version of a document in a review process, revision content of the historical version of the text content, and reference materials of the revision content.

[0047] Table 1 shows the correspondence between the historical version of the text content, the revision content of the historical version of the text content, and the reference materials of the revision content.

[0048] Table 1

[0049] Historical version of the text content "The technical solution of the present application can replace the existing technical solution, and can realize the automation of text review. At the same time, it can guide the writing of future text materials in the future" Revision content of the historical version "The technical solution of the present application can replace the existing technical solution, and can realize the automation of text review. At the same time, due to the existence of unified standards, it can guide the writing of future text materials in the future" Reference materials of revision content [Material 1, Material 2, …, Material n]

[0050] As shown in Table 1, the revision content of the historical version is to add the text "because there is a unified standard" after the "simultaneously" in the text content of the historical version, to revise "custom" in the text content of the historical version to "management", and to revise "know" in the text content of the historical version to "guidance".

[0051] Optionally, the un-revised part in the text content of the historical version can be replaced by a special character in the revision content.

[0052] Exemplarily, the revision content can be represented as "---------------------------------because there is a unified standard----management-------------guidance--". In the revision content, the un-revised part in the text content of the historical version is replaced by a special character "-".

[0053] It should be understood that the above to-be-processed data adopts historical data, so the revision content of the above historical version can be obtained by the existing review method. Among them, the text "because there is a unified standard" is added after "at the same time" in the text content of the above historical version, and the added text "because there is a unified standard" and the following text "can play a guiding role in the management of future text materials" have a logical relationship, so it can be considered as a revision in the aspect of arguments, evidence or reasoning, so the revision content is obtained by the existing manual review method. The revision of "custom" in the text content of the historical version to "management" and the revision of "know" in the text content of the historical version to "guidance" can be considered as a revision in the aspect of format, which can be obtained by the existing automatic review method. In other words, the revision content of the historical version in the above to-be-processed data includes not only the revision content in the aspect of document format such as typos and syntax, but also the revision content in the aspect of arguments, evidence, reasoning and the like.

[0054] S102, the data processing device inputs the above to-be-processed data into a pre-trained model to perform text embedding and feature extraction, and obtains a feature vector matrix of the text content, a feature vector matrix of the revision content and a feature vector matrix of the reference material, the pre-trained model adopting a bidirectional encoder architecture.

[0055] In a possible implementation, the data processing device can obtain the feature vector matrix of the text content, the feature vector matrix of the revision content and the feature vector matrix of the reference material based on a pre-trained word vector.

[0056] It should be understood that the above pre-trained word vector is used to convert words into vectors, that is, the words in the text content of the multiple historical versions of the above multiple documents in the review process, the revision content of the multiple historical versions and the reference material of the revision content can be converted into a vector matrix.

[0057] Optionally, the above pre-trained word vector can be a word vector processed by a text library.

[0058] It should be understood that the above text library includes at least one relevant text, and the similarity between the at least one relevant text and the text content of the above document is greater than or equal to a first preset threshold. Therefore, the above word vector processed by the text library can be better adapted to the text in the above to-be-processed data, and the extraction efficiency of the above features can be improved.

[0059] S103, the data processing device obtains a feature vector matrix of the spliced weighted text content and reference material based on the feature vector matrix of the text content, the feature vector matrix of the revision content and the feature vector matrix of the reference material.

[0060] In a possible implementation, the data processing device can perform dot multiplication on the feature vector matrix of the text content and the feature vector matrix of the revised content after normalization to obtain a feature vector matrix of the weighted text content, and can concatenate the feature vector matrix of the weighted text content and the feature vector matrix of the reference material to obtain a feature vector matrix of the weighted text content and the reference material.

[0061] In S104, the data processing device obtains a first target feature vector matrix, a second target feature vector matrix and a third target feature vector matrix based on the feature vector matrix of the revised content, the feature vector matrix of the weighted text content and the reference material, and the self-attention layer of the pre-trained model.

[0062] In a possible implementation, the data processing device can obtain the first target feature vector matrix based on the feature vector matrix of the revised content, and obtain the second target feature vector matrix and the third target feature vector matrix based on the feature vector matrix of the weighted text content and the reference material.

[0063] In S105, the data processing device re-inputs the first target feature vector matrix, the second target feature vector matrix and the third target feature vector matrix into the self-attention layer to obtain an output matrix of the self-attention layer.

[0064] For example, the output matrix of the self-attention layer can be calculated by the following formula: wherein, is the output matrix of the self-attention layer, is a flexible maximum value transmission function, is the first target feature vector matrix, is the transpose of the second target feature vector matrix, is the third target feature vector matrix, is the dimension of the second target feature vector matrix, is the first weight matrix.

[0065] In S106, the data processing device trains the pre-trained model by taking the output matrix of the self-attention layer as the feature vector matrix of the original content of the plurality of historical versions to obtain a text review model.

[0066] ​In the embodiment of the present application, the data processing device inputs the text content of multiple historical versions of a document in a review process, the revision content of the text content of the multiple historical versions, and the reference materials of the revision content into a pre-trained model, and uses an improved attention unit in the model to perform feature extraction, trains the pre-trained model, so that the pre-trained model can focus on the revision of the document content in terms of arguments, evidence, and reasoning, etc. in addition to the document format such as typos and syntax, obtains a text review model, improves the efficiency of text review, and avoids the problem of inconsistent review standards in terms of arguments, evidence, and reasoning, etc. caused by manual review.

[0067] For S102 described above, Figure 2 The pre-trained model 200 provided by the present application is shown, as Figure 2 The pre-trained model described above includes an encoder and a decoder, and the output of the encoder is connected to the input of the decoder, wherein the encoder is used for feature extraction of input data to provide effective text information for the decoder. The encoder includes at least one encoding layer, and the decoder includes at least one decoding layer, and the output of the encoding layer is connected to the input of the decoding layer.

[0068] In addition, each of the at least one encoding layer includes at least one self-attention layer, and each of the at least one decoding layer includes at least one self-attention layer.

[0069] It should be understood that in the case where the encoding layer and the decoding layer include one self-attention layer, the self-attention layer can learn to focus on a feature in the input data. The encoding layer and the decoding layer can also include a multi-head self-attention layer, which can include multiple self-attention layers. The multiple self-attention layers in the multi-head self-attention layer can focus on and learn multiple features in the input data, respectively, so as to balance the bias that can be generated by one self-attention layer and improve the effect of the model.

[0070] It should be understood that the pre-trained model can be a bidirectional encoder architecture, such as a transformer model, which is not limited by the present application.

[0071] The following will take the transformer model as an example of the pre-trained model described above, and combine Figure 3 The training process of the pre-trained model provided by the present application will be further described.

[0072] Figure 3 The training process 300 of the pre-trained model provided by the present application is shown. As Figure 3As shown, the data processing device can input the above-mentioned to-be-processed data into the above-mentioned transformer model, perform text embedding and feature extraction, and respectively convert the text content of the multiple historical versions of the multiple documents in the review process, the revision content of the multiple historical versions of the text content, and the reference materials of the revision content into a feature vector matrix of the text content, a feature vector matrix of the revision content, and a feature vector matrix of the reference materials.

[0073] Exemplarily, the feature vector matrix of the text content can be represented as , the feature vector matrix of the revision content can be represented as , and the feature vector matrix of the reference materials can be represented as The feature vector matrix of the reference materials can be obtained by summing and averaging.

[0074] The data processing device obtains an output matrix through an encoder based on the feature vector matrix of the text content, the feature vector matrix of the revision content, and the feature vector matrix of the reference materials, and can input the output matrix into a decoder to train parameters in the transformer model and output the revision content of the document, so as to complete training of the transformer model and obtain a text review model.

[0075] It should be understood that the output matrix is obtained through the self-attention layer in the encoder, so it can also be an output matrix of the self-attention layer.

[0076] Figure 4 The acquisition process 400 of the output matrix of the self-attention layer is shown, as shown in Figure 4 , the input data of the present application includes the feature vector matrix of the text content , the feature vector matrix of the revision content , and the feature vector matrix of the reference materials Therefore, the encoding layer in the above-mentioned transformer model can include a multi-head self-attention layer, and the multi-head self-attention layer can include at least three self-attention layers (i.e., a first self-attention layer, a second self-attention layer, and a third self-attention layer). Among them, the three self-attention layers can respectively focus on learning the features of the text content of the multiple historical versions of the multiple documents in the review process, the features of the revision content of the multiple historical versions of the text content, and the features of the reference materials of the revision content in the above-mentioned pre-processed data.

[0077] As shown in Figure 4 , the data processing device can obtain a first target feature vector matrix and an output matrix of the first self-attention layer through the first self-attention layer.

[0078] For example, a data processing device can use a parameter matrix Feature vector matrix of the revised content Multiplying them together yields the first target feature vector matrix. and the other two feature vector matrices of the first self-attention layer and Data processing equipment can be processed using formulas. The result matrix of the first self-attention layer is obtained, where, This is the result matrix of the first self-attention layer mentioned above. For flexible maximum transfer function, The first target feature vector matrix, This is the transpose of the feature vector matrix of the first self-attention layer. This is the feature vector matrix of the first self-attention layer. The above eigenvector matrix The dimensions. Then the data processing device can process the above result matrix. With the second weight matrix Multiplying them together yields the output matrix of the first attention layer. .

[0079] like Figure 4 As shown, the data processing device can process text based on the feature vector matrix of the text content. The feature vector matrix obtained is the concatenation of weighted text content and cited materials, the second target feature vector matrix, and the third target feature vector matrix.

[0080] For example, the data processing device standardizes the output matrix of the first self-attention layer. And multiply the feature vector matrix of the text content by the dot product. The weighted text vector matrix is ​​obtained. Data processing equipment can process weighted text vector matrices. The eigenvector matrix of the above-mentioned cited materials By concatenating the weighted text content and cited materials, we obtain the feature vector matrix. Data processing equipment can use a feature vector matrix concatenated from weighted text content and cited materials. The second target feature vector matrix is ​​obtained through the second self-attention layer. and the third target feature vector matrix .

[0081] like Figure 4 As shown, the data processing device can, based on the first target feature vector matrix obtained above, The second target feature vector matrix and a third target feature vector matrix obtaining, by a third self-attention layer, an output matrix of the self-attention layer (also referred to as an output matrix of the third self-attention or an output matrix of the multi-head self-attention layer).

[0082] Exemplarily, the data processing device can obtain the output matrix of the self-attention layer by a formula and further improve the decoder to obtain the revised content of the document. Wherein, is the output matrix of the third self-attention layer, is a flexible maximum transmission function, is the first target feature vector matrix, is the transpose of the second target feature vector matrix, is the third target feature vector matrix, is the dimension of the second target feature vector matrix , and is the first weight matrix.

[0083] Optionally, after obtaining the text review model, the data processing device can compare the revised text output by the text review model with the artificial revised text (such as the revised content of the text content of the plurality of historical versions) to determine whether the similarity is greater than a preset threshold. If the similarity is greater than the preset threshold, it can be indicated that the performance of the text review model is better, and the data processing device does not need to perform any operation. If the similarity is less than or equal to the preset threshold, it can be considered that the performance of the text review model is poor, and the data processing device can obtain new preprocessed data, incrementally train the text review model, and ensure the generalization ability of the model until the similarity between the revised text output by the text review model and the artificial revised text is greater than the preset threshold, thereby avoiding the problem that the revision feedback has no guiding significance for text modification due to the low review accuracy, and further reducing the burden of text management.

[0084] Optionally, after obtaining the text review model, the data processing device can also update the text library, thereby obtaining the feature vector matrix of the new text content, the feature vector matrix of the revised content, and the feature vector matrix of the reference material, and retraining the text review model, thereby avoiding the problem that the text library is out of date, and the model cannot accurately represent the semantic features of the text.

[0085] The model training process of the present application is described above in combination with Figure 1 to Figure 4 The model use process of the present application is described below in combination with Figure 5 .

[0086] As an optional embodiment, after obtaining the text review model, the document text content can be input into the text review model to obtain the revised content of the document. Since the revised content includes the revised content of the arguments, the evidence and the demonstration other than the format of the document, the problems of the text content of the document in the format, the arguments, the evidence and the demonstration can be determined based on the revised content of the document output by the text review model, thereby guiding the future writing of the text content of the document and reducing the burden of text management.

[0087] Figure 5 Another data processing method 500 provided by an embodiment of the present application is shown, as shown in the figure, the method 500 includes the following steps: Figure 5

[0088] S501, the data processing device acquires the text content of the document to be reviewed.

[0089] S502, the data processing device inputs the text content of the above-mentioned document to be reviewed into the text review model to obtain the revised content of the document, which includes the format revision, the argument revision, the evidence revision and the demonstration revision; wherein the text review model is obtained based on the text content of multiple historical versions of the document in the review process, the revised content of the text content of the multiple historical versions and the reference materials of the revised content.

[0090] It should be understood that the above-mentioned format revision can include the revision of the wrong word and the syntax in the text content. The above-mentioned argument revision, evidence revision and demonstration revision can include the revision of the logical relationship in the text content, such as the cause-effect relationship, the parallel relationship, etc.

[0091] In the present application, the data processing device can obtain the revised content of the document by inputting the text content of the document to be reviewed into the above-mentioned text review model, which includes the format revision, the argument revision, the evidence revision and the demonstration revision. Compared with the existing method, the review of the arguments, the evidence and the demonstration in the text content of the document is in an artificial way, while the review of the wrong word, the syntax and the format in the text content of the document is in an automatic way, the present application improves the text review efficiency. In addition, since the text review of the present application is realized through the above-mentioned text review model, the problem of non-uniform review standard caused by artificial review is avoided.

[0092] ​Optionally, the data processing device can also perform manual review on the text content of the document after using the text review model to obtain manual revision content, compare the revision content of the document output by the text review model with the manual revision content, determine the similarity between the two, and further determine the performance of the text review model. As above, if the similarity is greater than a preset threshold, it can be indicated that the model performance is better, and the data processing device does not need to perform any operation. If the similarity is less than or equal to the preset threshold, it can be indicated that the model performance is poor, and the data processing device can use the manual revision content to perform incremental training on the model.

[0093] Optionally, the data processing device can also update the text library after using the text review model, and further retrain the text review model to ensure that the model can accurately represent the semantic features of the text.

[0094] It should be understood that the size of the serial number of the above processes does not mean the order of execution, and the execution order of the processes should be determined according to their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0095] In order to realize the functions in the method provided by the above embodiments of the present application, the data processing device can include hardware structures and / or software modules to realize the above functions in the form of hardware structures, software modules, or hardware structures plus software modules. Whether a certain function in the above functions is executed in the form of hardware structure, software module, or hardware structure plus software module depends on the specific application and design constraints of the technical solution.

[0096] The data processing method provided by the embodiments of the present application is described in detail above. Figure 1 to Figure 5 The data processing device provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings. Figure 6 and Figure 7 The data processing device provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0097] Figure 6 The data processing device 600 provided by the embodiments of the present application is shown, which includes an acquisition module 601 and a processing module 602.

[0098] The acquisition module 601 is configured to acquire the text content of the document to be reviewed. The processing module 602 is configured to input the text content of the document to be reviewed into a text review model to obtain revision content of the document, which includes format revision, argument revision, evidence revision, and argumentation revision. The text review model is trained based on the text content of multiple historical versions of the document in the review process, the revision content of the multiple historical versions of the text content, and the reference materials of the revision content.

[0099] Optionally, the acquisition module 601 is configured to acquire to-be-processed data, the to-be-processed data including text content of a plurality of historical versions of a plurality of documents in a review process, revision content of the plurality of historical versions of the text content, and reference materials of the revision content; and the processing module 602 is configured to perform text embedding and feature extraction based on the to-be-processed data to obtain an output matrix of a self-attention layer, the pre-training model adopting a bidirectional encoder architecture; take the output matrix of the self-attention layer as a plurality of historical version original text content feature vector matrix, train the pre-training model to obtain the text review model.

[0100] Optionally, the processing module 602 is configured to perform text embedding and feature extraction based on the to-be-processed data to obtain a feature vector matrix of the text content, a feature vector matrix of the revision content, and a feature vector matrix of the reference materials; obtain a feature vector matrix of weighted text content and reference materials splicing based on the feature vector matrix of the text content, the feature vector matrix of the revision content, and the feature vector matrix of the reference materials; obtain a first target feature vector matrix, a second target feature vector matrix, and a third target feature vector matrix based on the feature vector matrix of the revision content, the feature vector matrix of the weighted text content and reference materials splicing, and the self-attention layer of the pre-training model; and re-input the first target feature vector matrix, the second target feature vector matrix, and the third target feature vector matrix to the self-attention layer to obtain an output matrix of the attention layer.

[0101] Optionally, the processing module 602 is configured to perform text embedding and feature extraction based on the to-be-processed data to obtain a feature vector matrix of the text content, a feature vector matrix of the revision content, and a feature vector matrix of the reference materials; obtain a feature vector matrix of weighted text content and reference materials splicing based on the feature vector matrix of the text content, the feature vector matrix of the revision content, and the feature vector matrix of the reference materials; obtain a first target feature vector matrix, a second target feature vector matrix, and a third target feature vector matrix based on the feature vector matrix of the revision content, the feature vector matrix of the weighted text content and reference materials splicing, and the self-attention layer of the pre-training model; and re-input the first target feature vector matrix, the second target feature vector matrix, and the third target feature vector matrix to the self-attention layer to obtain an output matrix of the attention layer.

[0102] Optionally, the processing module 602 is configured to construct a text library, the text library including at least one relevant text, the at least one relevant text having a similarity to the historical text greater than or equal to a first preset threshold; obtain a pre-training word vector based on the at least one relevant text and the to-be-processed data; and obtain the feature vector matrix of the text content, the feature vector matrix of the revision content, and the feature vector matrix of the reference materials based on the pre-training word vector.

[0103] It should be understood that the apparatus 600 herein is embodied in the form of functional modules. The term "module" herein can refer to an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated or group) and memory that execute one or more software or firmware programs, a combinational logic circuit, and / or other suitable components that provide the described functionality. In one optional example, those skilled in the art can understand that the apparatus 600 can be embodied in the data processing device in the above-mentioned embodiments, or the functions of the data processing device in the above-mentioned embodiments can be integrated in the apparatus 600, and the apparatus 600 can be used to execute the respective processes and / or steps corresponding to the data processing device in the above-mentioned method embodiments. To avoid repetition, details are not described herein.

[0104] The apparatus 600 described above has the functions of performing the respective steps of the data processing device in the above-mentioned method; the above-mentioned functions can be realized by hardware, or realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

[0105] In the embodiments of the present application, Figure 6 The apparatus 600 in the above-mentioned embodiments can also be a chip or a chip system, for example, a system on chip (SoC).

[0106] Figure 7 Another data processing apparatus 700 provided by the embodiments of the present application is shown. The apparatus 700 includes a processor 701, a memory 702, a communication interface 703 and a bus 704. The memory 702 is configured to store instructions, and the processor 701 is configured to execute the instructions stored in the memory 702. The processor 701, the memory 702 and the communication interface 703 are communicatively connected with each other through the bus 704.

[0107] The processor 701 is configured to: obtain text content of a document to be reviewed; and input the text content of the document to be reviewed into a text review model to obtain revised content of the document, the revised content including format revision, argument revision, evidence revision and reasoning revision; wherein the text review model is trained based on text content of multiple historical versions of the document in a review process, revised content of the multiple historical versions of the text content and reference materials of the revised content.

[0108] Optionally, the processor 701 is configured to: obtain to-be-processed data, the to-be-processed data including text content of a plurality of historical versions of a plurality of documents in a review process, revision content of the plurality of historical versions of the text content, and reference materials of the revision content; input the to-be-processed data into a pre-trained model to obtain an output matrix of a self-attention layer, the pre-trained model adopting an architecture of a bidirectional encoder; and take the output matrix of the self-attention layer as a feature vector matrix of original text content of the plurality of historical versions, train the pre-trained model, and obtain the text review model.

[0109] Optionally, the processor 701 is configured to: based on the to-be-processed data, perform text embedding and feature extraction to obtain a feature vector matrix of the text content, a feature vector matrix of the revision content, and a feature vector matrix of the reference materials; based on the feature vector matrix of the text content, the feature vector matrix of the revision content, and the feature vector matrix of the reference materials, obtain a feature vector matrix of weighted text content and reference materials spliced together; based on the feature vector matrix of the revision content, the feature vector matrix of the weighted text content and reference materials spliced together, and the self-attention layer of the pre-trained model, obtain a first target feature vector matrix, a second target feature vector matrix, and a third target feature vector matrix; and re-input the first target feature vector matrix, the second target feature vector matrix, and the third target feature vector matrix into the self-attention layer to obtain an output matrix of the attention layer.

[0110] Optionally, the processor 701 is configured to: multiply the feature vector matrix of the text content and the feature vector matrix of the revision content after normalization to obtain a feature vector matrix of weighted text content; and splice the feature vector matrix of the weighted text content and the feature vector matrix of the reference materials to obtain a feature vector matrix of weighted text content and reference materials spliced together.

[0111] Optionally, the processor 701 is configured to: construct a text library, the text library including at least one relevant text, the at least one relevant text having a similarity to the historical text greater than or equal to a first preset threshold; based on the at least one relevant text and the to-be-processed data, obtain a pre-trained word vector; and based on the pre-trained word vector, obtain the feature vector matrix of the text content, the feature vector matrix of the revision content, and the feature vector matrix of the reference materials.

[0112] It should be understood that the apparatus 700 can be specifically a data processing device in the above-described embodiments, or the functions of the data processing device in the above-described embodiments can be integrated in the apparatus 700, and the apparatus 700 can be used to perform each step and / or process corresponding to the data processing device in the above-described method embodiments. Alternatively, the memory 702 can include a read-only memory and a random access memory, and provide instructions and data for the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store device type information. The processor 701 can be used to execute the instructions stored in the memory, and when the processor executes the instructions, the processor can perform each step and / or process corresponding to the storage device in the above-described method embodiments.

[0113] It should be understood that in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0114] In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution completion, or executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor executes the instructions in the memory, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0115] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, apparatus and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0117] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0118] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0119] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into one unit.

[0120] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0121] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data processing method, characterized by, The method comprises the following steps: obtaining to-be-processed data, wherein the to-be-processed data comprises text content of multiple historical versions of a plurality of documents in a review process, revision content of the text content of the multiple historical versions, and reference materials of the revision content; inputting the to-be-processed data into a pre-trained model to obtain an output matrix of a self-attention layer, wherein the pre-trained model adopts a bidirectional encoder architecture; inputting the output matrix of the self-attention layer into the pre-trained model as a feature vector matrix of original text content of the multiple historical versions, training the pre-trained model, and obtaining a text review model; obtaining text content of a to-be-reviewed document; inputting the text content of the to-be-reviewed document into the text review model to obtain revision content of the document, wherein the revision content comprises format revision, argument revision, evidence revision, and argumentation revision; wherein the text review model is obtained by training based on the text content of the multiple historical versions of the plurality of documents in the review process, the revision content of the text content of the multiple historical versions, and the reference materials of the revision content; inputting the to-be-processed data into a pre-trained model to obtain an output matrix of a self-attention layer, comprising: based on the to-be-processed data, performing text embedding and feature extraction to obtain a feature vector matrix of the text content, a feature vector matrix of the revision content, and a feature vector matrix of the reference materials; based on the feature vector matrix of the text content, the feature vector matrix of the revision content, and the feature vector matrix of the reference materials, obtaining a feature vector matrix of weighted text content and reference materials spliced together; based on the feature vector matrix of the revision content, the feature vector matrix of the weighted text content and reference materials spliced together, and the self-attention layer of the pre-trained model, obtaining a first target feature vector matrix, a second target feature vector matrix, and a third target feature vector matrix; re-inputting the first target feature vector matrix, the second target feature vector matrix, and the third target feature vector matrix into the self-attention layer to obtain an output matrix of the self-attention layer.

2. The method of claim 1, wherein, The method comprises the following steps: based on the feature vector matrix of the text content, the feature vector matrix of the revision content, and the feature vector matrix of the reference materials, obtaining a feature vector matrix of weighted text content and reference materials spliced together, comprising: point-multiplying the feature vector matrix of the text content and the normalized feature vector matrix of the revision content to obtain a feature vector matrix of weighted text content; 3. The method according to claim 1 or 2, characterized in that, splicing the feature vector matrix of the weighted text content and the feature vector matrix of the reference materials to obtain a feature vector matrix of weighted text content and reference materials spliced together. The method comprises the following steps: constructing a text library, wherein the text library comprises at least one relevant text, and a similarity between the at least one relevant text and the text of the historical version is greater than or equal to a first preset threshold; based on the at least one relevant text and the to-be-processed data, obtaining a pre-trained word vector; Based on the pre-trained word vector, a feature vector matrix of the text content, a feature vector matrix of the revised content and a feature vector matrix of the reference material are obtained.

4. A data processing apparatus, characterized by, The method comprises the steps of: obtaining the text content of the document to be reviewed; inputting the text content of the document to be reviewed into a text review model to obtain the revised content of the document, wherein the revised content comprises format revision, argument revision, evidence revision and argumentation revision; wherein the text review model is obtained by training based on the text content of multiple historical versions of the document in the review process, the revised content of the multiple historical versions of the text content and the reference material of the revised content; the obtaining module is configured to obtain the to-be-processed data, wherein the to-be-processed data comprises the text content of multiple historical versions of the document in the review process, the revised content of the multiple historical versions of the text content and the reference material of the revised content; the processing module is configured to: input the to-be-processed data into a pre-trained model to obtain an output matrix of a self-attention layer, wherein the pre-trained model adopts the architecture of a bidirectional encoder; use the output matrix of the self-attention layer as the feature vector matrix of the original text content of the multiple historical versions, train the pre-trained model and obtain the text review model; the processing module is configured to: based on the to-be-processed data, perform text embedding and feature extraction to obtain the feature vector matrix of the text content, the feature vector matrix of the revised content and the feature vector matrix of the reference material; based on the feature vector matrix of the text content, the feature vector matrix of the revised content and the feature vector matrix of the reference material, obtain a feature vector matrix of the spliced weighted text content and reference material; based on the feature vector matrix of the revised content, the feature vector matrix of the spliced weighted text content and reference material and the self-attention layer of the pre-trained model, obtain a first target feature vector matrix, a second target feature vector matrix and a third target feature vector matrix; re-input the first target feature vector matrix, the second target feature vector matrix and the third target feature vector matrix into the self-attention layer to obtain the output matrix of the self-attention layer.

5. A data processing apparatus, characterized by, The method comprises the steps of: a processor coupled to a memory, wherein the memory is configured to store a computer program, and when the processor invokes the computer program, the apparatus executes the data processing method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, A computer program is stored, and the computer program comprises instructions for implementing the data processing method according to any one of claims 1 to 3.

7. A computer program product, characterised in that, A computer program is stored, and the computer program comprises instructions for implementing the data processing method according to any one of claims 1 to 3.

Images