Text annotation method, model training method, device and electronic device

Abstract

This disclosure proposes a text annotation method, model training method, device, and electronic device, relating to the fields of artificial intelligence technology, particularly natural language processing, deep learning, and large-scale model technology. The method comprises: obtaining a target text; performing problem detection on the target text to obtain a problem detection result for the target text; obtaining a prompt text based on the target text and the problem detection result; and inputting the prompt text into a large-scale model, whereby the large-scale model annotates the target text based on the problem detection result to obtain an annotated text.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of natural language processing, deep learning, and large model technology, and in particular to a text annotation method, a model training method, an apparatus, an electronic device, a storage medium, and a computer program product. Background Art

[0002] With the continuous development of artificial intelligence technology, large models, with their advantages such as good generalization, have been widely used in fields such as information extraction, text credibility assessment, and machine translation. For example, in the field of text annotation, large models can be used to generate annotated text. However, text annotation methods in related technologies suffer from low efficiency and accuracy. Summary of the Invention

[0003] The present disclosure proposes a text annotation method, a model training method, an apparatus, an electronic device, a storage medium, and a computer program product.

[0004] According to a first aspect of the present disclosure, a text annotation method is proposed, comprising: acquiring a target text; performing problem detection on the target text to obtain a problem detection result of the target text; obtaining a prompt text based on the target text and the problem detection result; inputting the prompt text into a large model, and annotating the target text based on the problem detection result through the large model to obtain annotated text.

[0005] According to the second aspect of the present disclosure, a model training method is proposed, including: obtaining a sample text and a sample annotated text of the sample text; performing problem detection on the sample text to obtain a sample problem detection result of the sample text; obtaining a sample prompt text based on the sample text and the sample problem detection result; inputting the sample prompt text into a large model, and annotating the sample text based on the sample problem detection result by the large model to obtain a predicted annotated text; and training the large model based on the predicted annotated text and the sample annotated text.

[0006] According to a third aspect of the present disclosure, a text annotation device is proposed, including: a first acquisition module for acquiring a target text; a detection module for performing problem detection on the target text to obtain a problem detection result of the target text; a second acquisition module for obtaining a prompt text based on the target text and the problem detection result; and an annotation module for inputting the prompt text into a large model, and annotating the target text based on the problem detection result through the large model to obtain an annotated text.

[0007] According to a fourth aspect of the present disclosure, a model training device is proposed, comprising: a first acquisition module for acquiring a sample text and a sample annotated text of the sample text; a detection module for performing problem detection on the sample text to obtain a sample problem detection result of the sample text; a second acquisition module for obtaining a sample prompt text based on the sample text and the sample problem detection result; a labeling module for inputting the sample prompt text into a large model, and labeling the sample text based on the sample problem detection result by the large model to obtain a predicted annotated text; and a training module for training the large model based on the predicted annotated text and the sample annotated text.

[0008] According to a fifth aspect of the present disclosure, an electronic device is proposed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the text annotation method proposed in the first aspect and the model training method proposed in the second aspect.

[0009] According to the sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to enable the computer to execute the text annotation method proposed in the first aspect and the model training method proposed in the second aspect.

[0010] According to the seventh aspect of the present disclosure, a computer program product is proposed, including a computer program, which, when executed by a processor, implements the text annotation method proposed in the first aspect and the model training method proposed in the second aspect.

[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0013] Figure 1 A flowchart of a text annotation method according to an embodiment of the present disclosure is shown;

[0014] Figure 2 A flowchart of a text annotation method according to another embodiment of the present disclosure is shown;

[0015] Figure 3 A flowchart of a text annotation method according to another embodiment of the present disclosure is shown;

[0016] Figure 4 A schematic diagram of a text annotation method according to an embodiment of the present disclosure;

[0017] Figure 5 Schematic diagram of a flow chart of a model training method according to an embodiment of the present disclosure;

[0018] Figure 6 This is a structural diagram of a text annotation device according to an embodiment of the present disclosure;

[0019] Figure 7 This is a schematic structural diagram of a model training device according to an embodiment of the present disclosure;

[0020] Figure 8 A schematic block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0021] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] Artificial Intelligence (AI) is a discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. Currently, AI technology has been widely used due to its high degree of automation, high precision, and low cost.

[0023] Natural Language Processing (NLU) is a science that studies computer systems, especially software systems, that can effectively implement natural language communication. It is an important direction in the fields of computer science and artificial intelligence.

[0024] DL (Deep Learning) is a new research direction in the field of ML (Machine Learning). It is a science that learns the inherent laws and representation levels of sample data, enabling machines to have analytical and learning capabilities like humans and recognize data such as text, images, and sounds. It is widely used in speech and image recognition.

[0025] Large models refer to machine learning models with large parameter sizes and complexity. They require extensive computing resources and storage space for training and storage, and often require distributed computing and specialized hardware acceleration technologies. Large models possess stronger generalization and expressive capabilities. These include LLMs (Large Language Models). Large language models are deep learning models trained using large amounts of text data and can generate natural language text or understand the meaning of text. Large language models can handle a variety of natural language tasks, such as text classification, question-answering, and conversation, and are an important path to artificial intelligence.

[0026] Figure 1 FIG. 1 is a flow chart of a text annotation method according to an embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0027] S101, obtaining target text.

[0028] It should be noted that the execution entity of the text annotation method of the embodiments of the present disclosure may be a hardware device with data information processing capabilities and / or the necessary software to drive the operation of the hardware device. Optionally, the execution entity may include a workstation, server, computer, user terminal, and other intelligent device. User terminals include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, and in-vehicle terminals.

[0029] It should be noted that the target text refers to the text to be annotated, and there are no excessive restrictions on the target text. For example, it can include texts in any language such as Chinese and English.

[0030] In one embodiment, obtaining the target text includes obtaining the original text and preprocessing the original text to obtain the target text. It should be noted that the preprocessing can be implemented using any text data preprocessing method in the relevant art, such as data cleaning, gap filling, format conversion, etc., which are not limited here.

[0031] In one embodiment, obtaining the target text includes receiving the target text sent by the client. For example, taking the execution entity of the text annotation method as a server, the client can obtain the target text based on the operation information of the user operating the client (e.g., text entered by the user, uploaded files, clicked icons, user voice messages, etc.), and send the target text to the server. Correspondingly, the server can receive the target text sent by the client.

[0032] In one embodiment, obtaining the target text includes receiving a text annotation request sent by a client, and obtaining the target text based on the text annotation request.

[0033] For example, if the text annotation request carries the identifier of the target text, the text identified by the identifier of the target text can be used as the target text. It should be noted that there are no excessive restrictions on the identifier, for example, it can include a name, storage path, etc.

[0034] For example, the text annotation request carries the target text, and the target text can be extracted from the text annotation request.

[0035] In one embodiment, obtaining the target text includes crawling the data source to obtain the target text. It should be noted that the data source is not limited to many aspects, for example, it may include a web page, an APP (Application), etc.

[0036] S102, performing problem detection on the target text to obtain a problem detection result of the target text.

[0037] It should be noted that the problem detection for the target text can be implemented by using any text problem detection method in the relevant technology, and no further limitations are made here.

[0038] It should be noted that the problem detection results can be independent of the target text or carried by the target text, and there are no excessive restrictions here. There are no excessive restrictions on the problem detection results. For example, it can include whether there is a problem text in the target text, the problem text, the position of the problem text in the target text, the error cause of the problem text, the annotation rules, the modification method, etc. Among them, the problem text refers to the text fragment with problems (errors) in the target text. The number of problem texts can be at least one, and different problem texts can correspond to different error causes, annotation rules and modification methods.

[0039] It should be noted that there are not too many restrictions on the position of the question text in the target text. For example, it can include paragraph numbers, line numbers, column numbers, etc.

[0040] It should be noted that there are no excessive restrictions on the causes of errors in question texts. For example, they may include format errors in the question text, spelling errors in the question text (such as typos), semantic errors in the question text (such as missing, redundant, unclear), the question text meeting the set conditions in the annotation rules, risk errors in the question text (such as sensitive words), and knowledge errors in the question text. For example, if the target text is "The gas with the highest content in the atmosphere is oxygen", then "oxygen" in the target text is the question text, and "oxygen" contains a knowledge error and should be "nitrogen".

[0041] It should be noted that annotation rules refer to the rules used for text annotation. There are no specific restrictions on annotation rules. For example, they can include rules for format, spelling, semantics, risk, knowledge, etc. For example, they can include the date format of "xxxx year xx month xx day", the removal of sensitive words from the target text, and the correction of knowledge errors in the target text.

[0042] It should be noted that there are not too many restrictions on the modification method. For example, it may include replacing the question text with the reference text corresponding to the question text, changing xx in the question text to xx, deleting xx in the question text, adding xx to a certain position in the question text (such as before or after a text fragment in the question text, or between two text fragments in the question text), etc.

[0043] In one embodiment, the problem detection result is in a table format. For example, the rows in the table correspond one-to-one to the question text, and a row of data in the table is a problem detection result corresponding to a question text. For example, it may include the question text, the position of the question text in the target text, the cause of the error in the question text, the marking rules, the modification method, etc.

[0044] In one embodiment, problem detection is performed on the target text to obtain a problem detection result of the target text, including inputting the target text into a problem detection plug-in, and having the problem detection plug-in output the problem detection result. It should be noted that there are no excessive limitations on the problem detection plug-in.

[0045] In one embodiment, performing problem detection on a target text to obtain a problem detection result for the target text includes obtaining M problem detection strategies, performing problem detection on the target text according to the jth problem detection strategy, obtaining the jth detection result, and obtaining a problem detection result based on the M detection results. Here, M is a positive integer, and j is a positive integer not greater than M. Thus, multiple problem detection strategies can be used to perform problem detection on the target text, respectively, obtaining respective detection results, and then comprehensively considering the multiple detection results to obtain a problem detection result.

[0046] In some examples, a problem detection result is obtained based on the M detection results, including performing deduplication processing on the M detection results to obtain the problem detection result.

[0047] S103, obtaining a prompt text based on the target text and the question detection result.

[0048] It should be noted that prompt text refers to the text input into the large model, and the prompt text is used to obtain the annotated text. For example, the prompt text can be composed of natural language and / or non-natural language (such as vectors).

[0049] In one embodiment, obtaining the prompt text based on the target text and the question detection result includes concatenating the target text and the question detection result to obtain the prompt text. For example, the target text and the question detection result may be concatenated in a set order to obtain the prompt text.

[0050] In one embodiment, a prompt text is obtained based on the target text and the problem detection result, including obtaining the marking position of the problem detection result on the target text, and marking the target text based on the problem detection result and the marking position corresponding to the problem detection result to obtain the prompt text.

[0051] It should be noted that there are not too many restrictions on the marking positions corresponding to the problem detection results. For example, it may include the first page, last page, first paragraph, last paragraph, and the marking area of ​​the target text (such as the right area and lower area of ​​the target text).

[0052] In some examples, obtaining the annotated position of the question detection result on the target text includes determining the annotated position of the question detection result on the target text based on the position of the question text corresponding to the question detection result in the target text. For example, the surrounding area (such as the right area, the lower area, etc.) of the position of the question text corresponding to the question detection result in the target text can be used as the annotated position of the question detection result on the target text.

[0053] S104: Input the prompt text into the big model, and use the big model to annotate the target text based on the question detection result to obtain the annotated text.

[0054] It should be noted that the large model can be implemented using any large model in the relevant art, without further limitation. For example, it can be a Transformer model, which is a neural network model based on the self-attention mechanism. For example, the large model can include a large language model.

[0055] It should be noted that the annotated text refers to the text after the target text is annotated. There are no specific restrictions on the annotated text. For example, it can include text in any language, such as Chinese or English. Annotating the target text in the embodiments of the present disclosure includes steps such as modifying the target text and adding annotated content. For details on the annotated content, please refer to the following embodiments and will not be repeated here.

[0056] For example, if the target text is "Today is 2023-8-5", the problem detection result is "The target text contains the problem text "2023-8-5", the error reason of the problem text "2023-8-5" is the format error, the annotation rule of the problem text "2023-8-5" is that the date format is "xxxx year xx month xx day", and the modification method of the problem text "2023-8-5" is to replace "2023-8-5" with "2023 August 5"", then the annotation text may include "Today is August 5, 2023".

[0057] For example, if the target text is "The gas with the highest content in the atmosphere is oxygen", and the problem detection result is "The target text contains the problem text "oxygen", the error reason for the problem text "oxygen" is the existence of knowledge errors, the annotation rule for the problem text "oxygen" is to modify the knowledge errors in the question text, and the modification method for the problem text "oxygen" is to replace "oxygen" with "nitrogen", then the annotation text may include "The gas with the highest content in the atmosphere is nitrogen".

[0058] In one embodiment, after obtaining the annotated text, a training sample is obtained based on the annotated text. For example, the annotated text is used as the training sample, or the label of the annotated text is obtained and the annotated text and the label of the annotated text are associated to obtain the training sample. It should be noted that there are no excessive restrictions on the model used to train the training sample. For example, it can include a large language model (such as a text classification model, a text segmentation model, etc.), a speech model (such as a speech synthesis model), an image model (such as a text recognition model), etc.

[0059] The text annotation method proposed in the present disclosure obtains the target text, performs problem detection on the target text, obtains the problem detection result of the target text, obtains the prompt text based on the target text and the problem detection result, inputs the prompt text into the large model, and annotates the target text based on the problem detection result through the large model to obtain the annotated text. In this way, the target text can be detected for problems and the problem detection result can be obtained. Taking the target text and the problem detection result into consideration, the prompt text can be obtained and input into the large model. That is, the large model in this solution can use the known problem detection result to annotate the target text. Compared with the related art that uses the large model for problem detection and text annotation at the same time, the text annotation efficiency and accuracy are low due to the limited input length of the large model and the insufficient capacity of the large model. In this solution, the large model does not perform problem detection, but only needs to perform text annotation. The input length and capacity requirements of the large model are relatively low, thereby improving the text annotation efficiency and accuracy.

[0060] In the above embodiment, the steps after obtaining the annotated text in step S104 can be combined with Figure 2 Further understanding, Figure 2 FIG. 1 is a flow chart of a text annotation method according to another embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0061] S201, obtaining target text.

[0062] S202, performing problem detection on the target text to obtain a problem detection result of the target text.

[0063] S203, obtaining a prompt text based on the target text and the question detection result.

[0064] S204: Input the prompt text into the large model, and use the large model to annotate the target text based on the question detection result to obtain the annotated text.

[0065] The relevant contents of steps S201-S204 can be found in the above embodiment and will not be repeated here.

[0066] S205: Update the marked text to the target text, and return to execute problem detection on the target text and subsequent steps until the set end condition is met.

[0067] S206: The most recently obtained annotation text is used as the final annotation text.

[0068] It should be noted that there are no specific limitations on the termination conditions. For example, the method may further include setting the termination condition as the problem detection result indicating that the target text does not contain problem text and / or the number of text annotations reaching a first set threshold. Thus, the process can be terminated when the target text does not contain problem text, that is, when the target text is correct and / or when the number of text annotations reaches a high threshold.

[0069] It should be noted that there are no excessive restrictions on the first set threshold, for example, it may include 4, 10 times, etc.

[0070] For example, the end conditions are set to include the problem detection result indicating that the target text does not contain problem text, and the number of text annotations reaches 4.

[0071] An original text may be obtained, preprocessed to obtain a target text A, and steps S202 to S204 may be executed to obtain an annotated text 1 after the target text A is annotated.

[0072] Update the annotated text 1 to the target text B, and perform problem detection on the target text B to obtain the problem detection result of the target text B. If the problem detection result of the target text B indicates that there is a problem text in the target text B, continue to execute steps S203-S204 to obtain the annotated text 2 after the target text B is annotated.

[0073] The marked text 2 is updated to the target text C, and the problem detection is performed on the target text C to obtain the problem detection result of the target text C.

[0074] If the problem detection result of the target text C indicates that there is no problem text in the target text C, the annotated text 2 is used as the final annotated text.

[0075] Alternatively, if the problem detection result of the target text C indicates that the target text C contains problem text, continue to perform steps S203 - S204 to obtain the annotated text 3 after the target text C is annotated.

[0076] Update the annotated text 3 to the target text D, and perform problem detection on the target text D to obtain the problem detection result of the target text D. If the problem detection result of the target text D indicates that there is a problem text in the target text D, continue to execute steps S203-S204 to obtain the annotated text 4 after the target text D is annotated. At this time, the number of text annotations reaches 4, and the annotated text 4 is used as the final annotated text.

[0077] The text annotation method proposed in this disclosure updates the annotated text to the target text after obtaining it, and then returns to perform problem detection on the target text and subsequent steps until a set end condition is met, with the most recently obtained annotated text being used as the final annotated text. Thus, after obtaining the annotated text, the annotated text can be updated to the target text, and the text annotation steps can be repeated, achieving multiple rounds of text annotation and improving the accuracy of text annotation.

[0078] In the above embodiment, in step S104, the target text is annotated based on the problem detection result by the large model to obtain the annotated text, which can be combined with Figure 3 Further understanding, Figure 3 FIG. 1 is a flow chart of a text annotation method according to another embodiment of the present disclosure. Figure 3 As shown, the method includes:

[0079] S301, obtaining target text.

[0080] S302: Perform problem detection on the target text to obtain a problem detection result of the target text.

[0081] S303: Based on the target text and the question detection result, a prompt text is obtained and input into the large model.

[0082] The relevant contents of steps S301-S303 can be found in the above embodiment and will not be repeated here.

[0083] S304: Modify the target text based on the problem detection result through the large model to obtain a modified text.

[0084] It should be noted that the modified text refers to the text after the target text is modified.

[0085] In one implementation, based on the problem detection result, the large model modifies the target text to obtain the modified text, including determining the problem text in the target text based on the problem detection result by the large model, and modifying the problem text in the target text by the large model to obtain the modified text.

[0086] In one implementation, based on the problem detection result, the large model modifies the target text to obtain the modified text, including determining the problem text in the target text based on the problem detection result by the large model, obtaining the reference text corresponding to the problem text by the large model, and replacing the problem text in the target text with the reference text corresponding to the problem text by the large model to obtain the modified text, so as to obtain the modified text.

[0087] It should be noted that there is a corresponding relationship between the problem text and the reference text. For example, the problem text and the reference text correspond one by one.

[0088] In some examples, determining the problem text in the target text based on the problem detection result by the large model includes extracting the problem text in the target text from the problem detection result by the large model.

[0089] In some examples, determining the problem text in the target text based on the problem detection result by the large model includes extracting the position of the problem text in the target text from the problem detection result by the large model, and determining the problem text in the target text based on the position of the problem text in the target text by the large model.

[0090] In some examples, obtaining the reference text corresponding to the problem text by the large model includes obtaining the annotation rule corresponding to the problem text by the large model, and obtaining the reference text corresponding to the problem text based on the annotation rule corresponding to the problem text. For example, if the problem text is "2023-8-5" and the annotation rule for the problem text "2023-8-5" is that the date format is "xxxx year xx month xx day", then the reference text corresponding to the problem text "2023-8-5" is "August 5, 2023".

[0091] In some examples, obtaining a reference text corresponding to the question text through the large model involves extracting at least one of the error cause, annotation rules, and modification method of the question text from the question detection results through the large model, using this as a modification reference for the question text. The large model then obtains the reference text corresponding to the question text based on the modification reference of the question text. Thus, the modification reference of the question text can be obtained by taking into account at least one of the error cause, annotation rules, and modification method of the question text. The large model can utilize the known modification reference to obtain the reference text, which reduces the requirements on the large model's capabilities and improves the efficiency and accuracy of text annotation.

[0092] For example, if the target text is "Today is 2023-8-5", the problem detection result is "The target text has the problem text "2023-8-5", the error reason of the problem text "2023-8-5" is the format error, the marking rule of the problem text "2023-8-5" is that the date format is "xxxx year xx month xx day", and the modification method of the problem text "2023-8-5" is to replace "2023-8-5" with "August 5, 2023"."

[0093] The big model can be used to extract at least one of the error cause, marking rule, and modification method of the problem text "2023-8-5" from the above problem detection results as a modification reference for the problem text "2023-8-5". Based on the modification reference of the problem text "2023-8-5" through the big model, the reference text corresponding to the problem text "2023-8-5" is "August 5, 2023".

[0094] S305, obtaining the annotated text based on the modified text through the large model.

[0095] In one embodiment, obtaining the annotated text based on the modified text by the large model includes using the modified text as the annotated text by the large model.

[0096] In one embodiment, the annotated text is obtained based on the modified text using the macro model, including obtaining the annotated content of the target text using the macro model, and obtaining the annotated text based on the modified text and the annotated content using the macro model. Thus, the annotated text can be obtained by obtaining the annotated content of the target text using the macro model and comprehensively considering the modified text and the annotated content, thereby improving the comprehensiveness of the annotated text.

[0097] It should be noted that the annotation content can be independent of the target text, or it can be carried by the target text and / or the modified text, and there are no excessive restrictions here. There are no excessive restrictions on the annotation content, for example, it can include the question text, the position of the question text in the target text, the cause of the error in the question text, the annotation rules, the modification method, the modification result, etc. Among them, different question texts can correspond to different modification results, and there are no excessive restrictions on the modification results, for example, it can include the reference text corresponding to the question text, whether the question text has been modified, and whether the question text has been modified successfully.

[0098] In some examples, the large model is used to obtain annotations for the target text, including obtaining annotations based on question detection results and modifications to the question text. For example, the large model can be used to concatenate question detection results and modifications to the question text to obtain annotations.

[0099] In some examples, the annotated text is obtained by using a large model based on the modified text and the annotated content, including the following possible implementations:

[0100] Method 1: Generate a first text independent of the target text based on the annotated content through the large model, and obtain the annotated text based on the modified text and the first text through the large model.

[0101] In some examples, the first text is in a table format. For example, the rows in the table correspond one-to-one to the question text, and a row of data in the table is the annotation content corresponding to the question text. For example, it may include the question text, the position of the question text in the target text, the cause of the error in the question text, the annotation rules, the modification method, the modification result, etc.

[0102] In some examples, generating a first text independent of the target text based on the annotated content using the large model may include concatenating multiple annotated contents using the large model to generate the first text. For example, the large model may concatenate multiple annotated contents in a predetermined order to generate the first text.

[0103] In some examples, obtaining the annotated text based on the modified text and the first text using the large model includes concatenating the modified text and the first text using the large model to obtain the annotated text. For example, the modified text and the first text can be concatenated using the large model in a set order to obtain the annotated text.

[0104] Thus, the first text can be generated based on the annotated content through the large model, and the annotated text can be obtained by comprehensively considering the modified text and the first text which are independent of each other.

[0105] Method 2: Obtain the annotation position of the annotation content on the modified text through the large model, and annotate the modified text based on the annotation content and the annotation position corresponding to the annotation content through the large model to obtain the annotated text.

[0106] It should be noted that there are not too many restrictions on the annotation position corresponding to the annotation content. For example, it can include the first page, last page, first paragraph, last paragraph, and the annotation area of ​​the modified text (such as the right area and lower area of ​​the modified text).

[0107] In some examples, obtaining the annotation position of the annotation content on the modified text through the large model includes determining the annotation position of the annotation content on the modified text based on the position of the question text corresponding to the annotation content in the target text through the large model. For example, there is a correspondence between the text position in the target text and the text position in the modified text. Based on the correspondence and the position of the question text corresponding to the annotation content in the target text, the large model can determine the surrounding area (such as the right area, the lower area, etc.) of the position on the modified text corresponding to the above position as the annotation position of the annotation content on the modified text.

[0108] In some examples, obtaining the annotation position of the annotation content on the modified text using the large model includes obtaining the question text corresponding to the annotation content using the large model, and determining the annotation position of the annotation content on the modified text based on the position of the reference text corresponding to the question text corresponding to the annotation content on the modified text using the large model. For example, the surrounding area (such as the right area, the lower area, etc.) of the position of the reference text corresponding to the question text corresponding to the annotation content in the modified text can be used as the annotation position of the annotation content on the modified text.

[0109] Therefore, the annotation position of the annotation content on the modified text can be obtained through the large model, and the modified text can be annotated by comprehensively considering the annotation content and the annotation position corresponding to the annotation content to obtain the annotated text.

[0110] The text annotation method proposed in this disclosure uses a large model to modify the target text based on the problem detection results to obtain a modified text, and then uses the large model to obtain the annotated text based on the modified text. Therefore, in this solution, the large model can use the known problem detection results to modify the target text to obtain the modified text, and then obtain the annotated text.

[0111] In the above embodiment, in step S102, problem detection is performed on the target text to obtain a problem detection result of the target text, which includes the following possible implementation methods:

[0112] Method 1: Convert the annotation rules to obtain the target code, execute the target code, and use the target code to detect problems in the structured text in the target text to obtain problem detection results.

[0113] It should be noted that the target code is used to detect problems in structured text within the target text. There are no specific restrictions on the target code; for example, it can include regular expressions. Structured text can include text such as dates and times, such as "2023-8-5" and "09:35:28."

[0114] In one embodiment, converting the marking rules to obtain the target code includes converting the marking rules according to a set conversion method to obtain the target code. It should be noted that the set conversion method is not limited to many details.

[0115] Thus, the annotation rules can be converted to obtain target code, which is executed to detect problems in the structured text in the target text and obtain problem detection results suitable for problem detection in the structured text in the target text.

[0116] Method 2: Segment the target text to obtain multiple second texts, obtain the first text features of the second texts, and if the similarity between the first text features and the second text features in the feature library is greater than a second set threshold, determine that the second text is a problem text, and obtain a problem detection result based on the second text.

[0117] It should be noted that the second text is a text fragment in the target text. The granularity of the second text can be characters, words, etc. There are no excessive restrictions on the first text features and the second text features. For example, it can be a text vector (such as a word vector). The feature library includes multiple second text features. For example, the second text features include text features of sensitive words.

[0118] It should be noted that the target text can be segmented using any text segmentation method in the related art, and the first text feature can be obtained using any text feature acquisition method in the related art, without further limitation. For example, the second text can be input into a text encoder, and the text encoder outputs the first text feature of the second text.

[0119] It should be noted that there are no excessive restrictions on the second set threshold. For example, taking the similarity value range as 0 to 100%, the second set threshold may be 80%.

[0120] In one embodiment, based on the second text, a problem detection result is obtained, including at least one of the following: the second text is a problem text, the error cause of the second text is a risk error, the annotation rule of the second text (such as deleting sensitive words from the target text), and the modification method of the second text (such as deleting the second text from the target text, replacing the second text with non-sensitive words, etc.) as the problem detection result.

[0121] Therefore, the target text can be segmented to obtain multiple second texts, and the first text features of the second texts can be obtained. If the similarity between the first text feature and the second text feature in the feature library is large, it indicates that the second text is similar to the text corresponding to the second text feature (such as sensitive words). The second text can be determined to be a problem text, and based on the second text, the problem detection result can be obtained, which is suitable for risk problem detection in the target text, especially for sensitive word problem detection in the target text.

[0122] Method 3: Extract a third text under the target knowledge field from the target text, extract the knowledge data under the target knowledge field from the knowledge graph, and perform problem detection on the third text under the target knowledge field based on the knowledge data under the target knowledge field to obtain the problem detection result.

[0123] It should be noted that the third text is a text fragment under the target knowledge field in the target text. The granularity of the third text can be characters, words, sentences, etc. The knowledge graph includes knowledge data under multiple candidate knowledge fields, and does not impose too many restrictions on the knowledge fields. For example, it can include mathematics, astronomy, earth science, literature, news, etc.

[0124] In one embodiment, based on the knowledge data under the target knowledge field, problem detection is performed on the third text under the target knowledge field to obtain a problem detection result, including identifying whether the knowledge data under the target knowledge field and the third text under the target knowledge field match. If the knowledge data under the target knowledge field and the third text under the target knowledge field do not match, the third text is determined to be a problem text, and a problem detection result is obtained based on the third text.

[0125] In some examples, based on the third text, a problem detection result is obtained, including at least one of the following: the third text is a problem text, the cause of the error in the third text is a knowledge error, the annotation rule of the third text (such as modifying the knowledge error in the problem text), and the modification method of the third text (such as replacing the third text with xx, etc.) as the problem detection result.

[0126] For example, if the target text is "The gas with the highest content in the atmosphere is oxygen", the third text "oxygen" in the field of earth science can be extracted from the above target text, and the knowledge data "The gas with the highest content in the atmosphere is nitrogen" in the field of earth science can be extracted from the knowledge graph. It can be identified that the knowledge data "The gas with the highest content in the atmosphere is nitrogen" in the field of earth science does not match the third text "oxygen" in the field of earth science, and "oxygen" is determined to be the problem text. Based on "oxygen", the problem detection result is obtained.

[0127] For example, "The target text contains the problem text "oxygen", the error reason of the problem text "oxygen" is the existence of knowledge errors, the annotation rule of the problem text "oxygen" is to modify the knowledge errors in the problem text, and the modification method of the problem text "oxygen" is to replace "oxygen" with "nitrogen" as the problem detection result.

[0128] Therefore, the third text under the target knowledge field can be extracted from the target text, and the knowledge data under the target knowledge field can be extracted from the knowledge graph. Based on the knowledge data under the target knowledge field, problem detection is performed on the third text under the target knowledge field to obtain problem detection results, which are suitable for knowledge problem detection in the target text.

[0129] Method 4: Obtain N question detection models, where N is a positive integer, input the target text into the i-th question detection model, and the i-th question detection model outputs the i-th detection result, where i is a positive integer not greater than N. Based on the N detection results, obtain the question detection result.

[0130] It should be noted that there are no specific restrictions on question detection models. For example, they can be deep learning models, language models, small models, etc. Different detection results may correspond to the same or different question categories. For example, there is a one-to-one correspondence between detection results and question categories, that is, the i-th question detection model is used to detect the i-th question category for the target text.

[0131] In one embodiment, the problem detection processes of the N problem detection models are performed in parallel.

[0132] In one embodiment, the method also includes obtaining a sample text and N sample detection results of the sample text, inputting the sample text into the i-th question detection model, outputting the i-th predicted detection result by the i-th question detection model, and training the i-th question detection model based on the i-th predicted detection result and the i-th sample detection result to achieve training of N question detection models.

[0133] It should be noted that the i-th problem detection model is trained based on the i-th prediction detection result and the i-th sample detection result. Any model training method in the relevant technology can be used to achieve this, which will not be repeated here. For example, based on the i-th prediction detection result and the i-th sample detection result, the loss function of the i-th problem detection model is obtained, and based on the loss function of the i-th problem detection model, the i-th problem detection model is trained. It should be noted that there are not too many restrictions on the loss function. For example, it can include CE (Cross Entropy), MSE (Mean-Square Error), KL (Kullback-Leibler) divergence, contrast loss function, etc.

[0134] In one embodiment, obtaining a problem detection result based on the N detection results includes concatenating the N detection results to obtain the problem detection result. For example, the N detection results may be concatenated in a set order to obtain the problem detection result.

[0135] In one embodiment, obtaining a problem detection result based on the N detection results includes performing deduplication processing on the N detection results to obtain the problem detection result.

[0136] Therefore, the target text can be input into multiple problem detection models respectively to obtain the corresponding detection results, so as to obtain the problem detection results, which is particularly suitable for detecting more difficult problems in the target text.

[0137] Based on any of the above embodiments, Figure 4 As shown, the text annotation system includes four modules: a preprocessing module, a central control module, a problem detection module, and a large language model. The problem detection module includes a rule detection tool, a vector database, a knowledge graph, and a problem detection model. It should be noted that the rule detection tool is built based on the target code in the above embodiment and is used to execute the target code. The vector database is a type of feature library in the above embodiment and will not be described in detail here.

[0138] The preprocessing module is used to preprocess the original text to obtain the target text and send the target text to the central control module.

[0139] The problem detection module is used to detect problems in the target text, obtain the problem detection results of the target text, and send the problem detection results to the central control module.

[0140] The central control module is used to obtain the original text and send the original text to the preprocessing module, and send the target text sent by the preprocessing module to the problem detection module, and obtain the prompt text based on the target text and the problem detection results, and send the prompt text to the large language model.

[0141] The large language model is used to annotate the target text based on the question detection results, obtain the annotated text, and send the annotated text to the central control module.

[0142] The central control module is also used to update the annotation text to the target text, and return to execute the target text to be sent to the problem detection module and its subsequent steps until the set end condition is reached, and the most recently obtained annotation text is used as the final annotation text.

[0143] Figure 5 FIG. 1 is a flow chart of a model training method according to an embodiment of the present disclosure. Figure 5 As shown, the method includes:

[0144] S501: Obtain sample text and sample annotated text of the sample text.

[0145] S502: Perform problem detection on the sample text to obtain a sample problem detection result of the sample text.

[0146] S503: Obtain a sample prompt text based on the sample text and the sample question detection result.

[0147] S504: Input the sample prompt text into the large model, and use the large model to annotate the sample text based on the sample question detection result to obtain the predicted annotated text.

[0148] The relevant contents of steps S501-S504 can be found in the above embodiment and will not be repeated here.

[0149] It should be noted that the execution entity of the model training method in the embodiments of the present disclosure may be a hardware device with data information processing capabilities and / or the necessary software to drive the operation of the hardware device. Optionally, the execution entity may include a workstation, server, computer, user terminal, and other intelligent devices. Among them, user terminals include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, and in-vehicle terminals.

[0150] It should be noted that, for the relevant content of the sample text, please refer to the relevant content of the target text in the above embodiment; for the relevant content of the sample problem detection results, please refer to the relevant content of the problem detection results of the target text in the above embodiment; for the relevant content of the sample prompt text, please refer to the relevant content of the prompt text in the above embodiment; for the relevant content of the predicted annotated text and the sample annotated text, please refer to the relevant content of the annotated text in the above embodiment, and no further details will be given here.

[0151] It should be noted that the sample prompt text is input into the big model, and the big model annotates the sample text based on the sample question detection result to obtain the predicted annotated text. Please refer to the above embodiment of inputting the prompt text into the big model, and annotating the target text based on the question detection result by the big model to obtain the relevant content of the annotated text. It will not be repeated here.

[0152] In one embodiment, after obtaining the predicted annotation text, it also includes updating the predicted annotation text to a sample text, and returning to perform problem detection on the sample text and its subsequent steps until the set end condition is reached, and the most recently obtained predicted annotation text is used as the final predicted annotation text.

[0153] In one embodiment, the sample text is annotated based on the sample question detection results by the large model to obtain predicted annotated text, including modifying the sample text based on the sample question detection results by the large model to obtain predicted modified text, and obtaining predicted annotated text based on the predicted modified text by the large model.

[0154] S505: Training the large model based on the predicted annotation text and the sample annotation text.

[0155] It should be noted that the training of the large model based on the predicted conversation text and the sample conversation text can be achieved by using any model training method in the relevant technology, which will not be repeated here. For example, the training of the large model based on the predicted annotated text and the sample annotated text includes obtaining the loss function of the large model based on the predicted annotated text and the sample annotated text, and training the large model based on the loss function of the large model. It should be noted that there are no excessive restrictions on the loss function. For example, it can include CE (Cross Entropy), MSE (Mean-Square Error), KL (Kullback-Leibler) divergence, contrastive loss function, etc.

[0156] In one embodiment, before training the large model based on the predicted annotated text and sample annotated text, the large model is also pre-trained based on sample text from multiple knowledge domains. Thus, the large model can be pre-trained based on sample text from multiple knowledge domains. During the pre-training process, the large model can learn sample text from multiple knowledge domains, enabling the large model to generate general text data.

[0157] The model training method proposed in the present disclosure obtains sample text and sample annotated text of the sample text, performs problem detection on the sample text, obtains sample problem detection results of the sample text, obtains sample prompt text based on the sample text and the sample problem detection results, inputs the sample prompt text into the big model, annotates the sample text based on the sample problem detection results by the big model, obtains predicted annotated text, and trains the big model based on the predicted annotated text and the sample annotated text, so that the big model can learn the relationship between the sample prompt text and the sample annotated text during the training process, so that the trained big model can generate annotated text based on the prompt text for text annotation.

[0158] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0159] According to an embodiment of the present disclosure, the present disclosure further provides a text annotation device for implementing the above-mentioned text annotation method.

[0160] Figure 6 is a block diagram of a text annotation apparatus according to an embodiment of the present disclosure.

[0161] like Figure 6 As shown, the text annotation device 600 includes: a first acquisition module 601 , a detection module 602 , a second acquisition module 603 and an annotation module 604 .

[0162] A first acquisition module 601 is used to acquire a target text;

[0163] A detection module 602 is configured to perform problem detection on the target text and obtain a problem detection result of the target text;

[0164] A second acquisition module 603 is configured to obtain a prompt text based on the target text and the question detection result;

[0165] The annotation module 604 is used to input the prompt text into the large model, and annotate the target text based on the problem detection result through the large model to obtain annotated text.

[0166] In one embodiment of the present disclosure, after obtaining the annotated text, the annotation module 604 is further used to: update the annotated text to the target text, and return to execute the problem detection on the target text and its subsequent steps until the set end condition is reached; and use the most recently obtained annotated text as the final annotated text.

[0167] In one embodiment of the present disclosure, the annotation module 604 is further configured to: use the problem detection result to indicate that there is no problem text in the target text, and / or use the number of text annotations reaching a first set threshold as the set end condition.

[0168] In one embodiment of the present disclosure, the annotation module 604 is further used to: modify the target text based on the problem detection result through the large model to obtain a modified text; and obtain the annotated text based on the modified text through the large model.

[0169] In one embodiment of the present disclosure, the annotation module 604 is further used to: determine the question text in the target text based on the question detection result through the big model; obtain the reference text corresponding to the question text through the big model; replace the question text in the target text with the reference text corresponding to the question text through the big model to obtain the modified text.

[0170] In one embodiment of the present disclosure, the annotation module 604 is further used to: extract at least one of the error cause, annotation rule, and modification method of the problem text from the problem detection result through the large model as a modification reference for the problem text; and obtain a reference text corresponding to the problem text based on the modification reference of the problem text through the large model.

[0171] In one embodiment of the present disclosure, the annotation module 604 is further configured to: obtain the annotation content of the target text through the large model; and obtain the annotated text through the large model based on the modified text and the annotation content.

[0172] In one embodiment of the present disclosure, the annotation module 604 is further used to: generate a first text independent of the target text based on the annotation content through the large model; and obtain the annotated text based on the modified text and the first text through the large model.

[0173] In one embodiment of the present disclosure, the annotation module 604 is further used to: obtain the annotation position of the annotation content on the modified text through the large model; and annotate the modified text based on the annotation content and the annotation position corresponding to the annotation content through the large model to obtain the annotated text.

[0174] In one embodiment of the present disclosure, the detection module 602 is further used to: convert the annotation rules to obtain target code; execute the target code, and perform problem detection on the structured text in the target text through the target code to obtain the problem detection result.

[0175] In one embodiment of the present disclosure, the detection module 602 is further used to: segment the target text to obtain multiple second texts; obtain a first text feature of the second text; if the similarity between the first text feature and the second text feature in the feature library is greater than a second set threshold, determine that the second text is a problem text; and obtain the problem detection result based on the second text.

[0176] In one embodiment of the present disclosure, the detection module 602 is further used to: extract a third text under the target knowledge field from the target text; extract knowledge data under the target knowledge field from the knowledge graph; and perform problem detection on the third text under the target knowledge field based on the knowledge data under the target knowledge field to obtain the problem detection result.

[0177] In one embodiment of the present disclosure, the detection module 602 is further used to: obtain N problem detection models, where N is a positive integer; input the target text into the i-th problem detection model, and have the i-th problem detection model output the i-th detection result, where i is a positive integer not greater than N; and obtain the problem detection result based on the N detection results.

[0178] The text annotation device proposed in the present disclosure obtains the target text, performs problem detection on the target text, obtains the problem detection result of the target text, obtains the prompt text based on the target text and the problem detection result, inputs the prompt text into the large model, and annotates the target text based on the problem detection result through the large model to obtain the annotated text. In this way, the target text can be detected for problems and the problem detection result can be obtained. Taking the target text and the problem detection result into consideration, the prompt text can be obtained and input into the large model. That is, the large model in this solution can use the known problem detection result to annotate the target text. Compared with the related art that uses the large model for problem detection and text annotation at the same time, the text annotation efficiency and accuracy are low due to the limited input length of the large model and the insufficient capacity of the large model. In this solution, the large model does not perform problem detection, but only needs to perform text annotation. The input length and capacity requirements of the large model are relatively low, thereby improving the text annotation efficiency and accuracy.

[0179] According to an embodiment of the present disclosure, the present disclosure also provides a model training device for implementing the above-mentioned model training method.

[0180] Figure 7 4 is a block diagram of a model training device according to an embodiment of the present disclosure.

[0181] like Figure 7As shown, the model training device 700 includes: a first acquisition module 701, a detection module 702, a second acquisition module 703, a labeling module 704 and a training module 705.

[0182] A first acquisition module 701 is used to acquire a sample text and a sample annotated text of the sample text;

[0183] A detection module 702 is configured to perform problem detection on the sample text to obtain a sample problem detection result of the sample text;

[0184] A second acquisition module 703 is configured to obtain a sample prompt text based on the sample text and the sample question detection result;

[0185] The annotation module 704 is used to input the sample prompt text into the large model, and annotate the sample text based on the sample question detection result by the large model to obtain a predicted annotation text;

[0186] The training module 705 is used to train the large model based on the predicted annotation text and the sample annotation text.

[0187] The model training device proposed in the present disclosure obtains sample text and sample annotated text of the sample text, performs problem detection on the sample text, obtains sample problem detection results of the sample text, obtains sample prompt text based on the sample text and the sample problem detection results, inputs the sample prompt text into the big model, annotates the sample text based on the sample problem detection results by the big model, obtains predicted annotated text, and trains the big model based on the predicted annotated text and the sample annotated text, so that the big model can learn the relationship between the sample prompt text and the sample annotated text during the training process, so that the trained big model can generate annotated text based on the prompt text for text annotation.

[0188] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0189] Figure 8 A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0190] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0191] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 806, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0192] The computing unit 801 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the text annotation method and the model training method. For example, in some embodiments, the text annotation method and the model training method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the text annotation method and the model training method described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the text annotation method and the model training method in any other appropriate manner (for example, by means of firmware).

[0193] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0195] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0196] To initiate interaction with a user account, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user account; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user account can provide input to the computer. Other types of devices can also be used to initiate interaction with the user account; for example, feedback provided to the user account can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user account can be received in any form (including acoustic input, voice input, or tactile input).

[0197] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user account computer with a graphical user account interface or a web browser through which a user account can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0198] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0199] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the steps of the text annotation method and the model training method described in the above embodiments of the present disclosure are implemented.

[0200] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0201] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A text annotation method, comprising: Get the target text; Performing problem detection on the target text to obtain a problem detection result of the target text, wherein the problem detection result includes the problem text, the position of the problem text in the target text, the cause of the error of the problem text, the marking rule and the modification method; The target text and the question detection result are spliced ​​in a set order to obtain a prompt text; Inputting the prompt text into the macro model; Determining the question text in the target text based on the question detection result in the prompt text by the large model; Extracting the error cause, marking rules and modification method of the problem text from the problem detection results in the prompt text through the large model as a reference for modifying the problem text; Obtaining a reference text corresponding to the question text by modifying the large model based on the question text; Replacing the question text in the target text with a reference text corresponding to the question text using the large model to obtain a modified text; Obtaining the annotation content of the target text through the large model, the annotation content including the question text, the position of the question text in the target text, the cause of the error in the question text, the annotation rules, the modification method and the modification result; Obtaining annotated text based on the modified text and the annotated content through the large model; The step of obtaining the annotated text based on the modified text and the annotated content through the large model includes: generating a first text independent of the target text based on the annotated content by using the large model; Obtaining the annotated text based on the modified text and the first text through the large model; or, Obtaining the marking position of the marking content on the modified text through the large model; Annotating the modified text by the large model based on the annotated content and the annotated position corresponding to the annotated content to obtain the annotated text; After obtaining the annotated text, the method further includes: The annotated text is used as a training sample to train other models.

2. The method according to claim 1, wherein After obtaining the annotated text, the method further includes: Updating the marked text to the target text, and returning to execute the problem detection on the target text and subsequent steps until the set end condition is met; The most recently obtained annotation text is used as the final annotation text.

3. The method according to claim 2, wherein: The method further comprises: The problem detection result indicates that there is no problem text in the target text, and / or the number of text annotations reaches a first set threshold, as the set end condition.

4. The method according to any one of claims 1 to 3, wherein The performing problem detection on the target text to obtain a problem detection result of the target text includes: Convert the annotation rules to obtain the target code; The target code is executed, and problem detection is performed on the structured text in the target text through the target code to obtain the problem detection result.

5. The method according to any one of claims 1 to 3, wherein The performing problem detection on the target text to obtain a problem detection result of the target text includes: Segmenting the target text to obtain a plurality of second texts; Obtaining a first text feature of the second text; If the similarity between the first text feature and the second text feature in the feature library is greater than a second set threshold, determining that the second text is a question text; Based on the second text, the problem detection result is obtained.

6. The method according to any one of claims 1 to 3, wherein The performing problem detection on the target text to obtain a problem detection result of the target text includes: extracting a third text under a target knowledge domain from the target text; Extracting knowledge data under the target knowledge domain from the knowledge graph; Based on the knowledge data in the target knowledge domain, problem detection is performed on the third text in the target knowledge domain to obtain the problem detection result.

7. The method according to any one of claims 1 to 3, wherein The performing problem detection on the target text to obtain a problem detection result of the target text includes: Obtain N problem detection models, where N is a positive integer; Inputting the target text into the i-th question detection model, and having the i-th question detection model output an i-th detection result, where i is a positive integer not greater than N; Based on the N detection results, the problem detection result is obtained.

8. A model training method comprising: Obtaining a sample text and a sample annotated text of the sample text, wherein the sample annotated text is obtained using the text annotation method according to any one of claims 1 to 7; Performing problem detection on the sample text to obtain a sample problem detection result of the sample text; Obtaining a sample prompt text based on the sample text and the sample question detection result; Inputting the sample prompt text into the large model, and using the large model to annotate the sample text based on the sample question detection result to obtain a predicted annotated text; The large model is trained based on the predicted annotated text and the sample annotated text.

9. A text annotation device, comprising: A first acquisition module is used to acquire the target text; A detection module is used to detect problems in the target text and obtain a problem detection result of the target text, wherein the problem detection result includes the problem text, the position of the problem text in the target text, the cause of the error in the problem text, the marking rules and the modification method; A second acquisition module is used to splice the target text and the question detection result in a set order to obtain a prompt text; A marking module, used for inputting the prompt text into the macro model; Determining the question text in the target text based on the question detection result in the prompt text by the large model; Extracting the error cause, marking rules, and modification method of the problem text from the problem detection result in the prompt text through the large model as a modification reference for the problem text; obtaining a reference text corresponding to the problem text based on the modification reference of the problem text through the large model; Replacing the question text in the target text with a reference text corresponding to the question text using the macro model to obtain a modified text; obtaining annotated content of the target text using the macro model, the annotated content including the question text, the position of the question text in the target text, the cause of the error in the question text, the annotation rule, the modification method, and the modification result; obtaining an annotated text based on the modified text and the annotated content using the macro model; Wherein, the marking module is further used for: generating a first text independent of the target text based on the annotated content by using the large model; Obtaining the annotated text based on the modified text and the first text through the large model; or, Obtaining the marking position of the marking content on the modified text through the large model; Annotating the modified text by the large model based on the annotated content and the annotated position corresponding to the annotated content to obtain the annotated text; After obtaining the annotated text, the device further executes: The annotated text is used as a training sample to train other models.

10. The device according to claim 9, wherein After obtaining the annotated text, the annotating module is further used to: Updating the marked text to the target text, and returning to execute the problem detection on the target text and subsequent steps until the set end condition is met; The most recently obtained annotation text is used as the final annotation text.

11. The device according to claim 10, wherein The marking module is further used to: The problem detection result indicates that there is no problem text in the target text, and / or the number of text annotations reaches a first set threshold, as the set end condition.

12. The device according to any one of claims 9 to 11, wherein: The detection module is further used to: Convert the annotation rules to obtain the target code; The target code is executed, and problem detection is performed on the structured text in the target text through the target code to obtain the problem detection result.

13. The device according to any one of claims 9 to 11, wherein: The detection module is further used for: Segmenting the target text to obtain a plurality of second texts; Obtaining a first text feature of the second text; If the similarity between the first text feature and the second text feature in the feature library is greater than a second set threshold, determining that the second text is a question text; Based on the second text, the problem detection result is obtained.

14. The device according to any one of claims 9 to 11, wherein: The detection module is further used to: extracting a third text under a target knowledge domain from the target text; Extracting knowledge data under the target knowledge domain from the knowledge graph; Based on the knowledge data in the target knowledge domain, problem detection is performed on the third text in the target knowledge domain to obtain the problem detection result.

15. The device according to any one of claims 9 to 11, wherein: The detection module is further used for: Obtain N problem detection models, where N is a positive integer; Inputting the target text into the i-th question detection model, and having the i-th question detection model output an i-th detection result, where i is a positive integer not greater than N; Based on the N detection results, the problem detection result is obtained.

16. A model training device comprising: A first acquisition module, configured to acquire a sample text and a sample annotated text of the sample text, wherein the sample annotated text is acquired using the text annotation method according to any one of claims 1 to 7; A detection module, configured to perform problem detection on the sample text and obtain a sample problem detection result of the sample text; A second acquisition module is used to obtain a sample prompt text based on the sample text and the sample question detection result; A labeling module, configured to input the sample prompt text into a large model, and label the sample text based on the sample question detection result by the large model to obtain a predicted labeling text; A training module is used to train the large model based on the predicted annotation text and the sample annotation text.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images