Text information labeling method and device, storage medium and electronic device

Abstract

The invention discloses a text information labeling method and device, a storage medium and an electronic device, and the method comprises the steps: obtaining to-be-labeled text information and a target labeling instruction, the target labeling instruction indicating to label the to-be-labeled text information by adopting N labeling modes in a group of labeling modes; m labeling models corresponding to the target labeling instruction are selected from a group of labeling models according to the target labeling instruction, each labeling model in the M labeling models adopts N labeling modes, one group of labeling models is a model trained by using a group of training data, and the M labeling models comprise a chat generation type pre-training model ChatGPT; the M labeling models are used for labeling the to-be-labeled text information in the N labeling modes to obtain a first group of labeling results, and each labeling model in the M labeling models adopts at least one labeling mode in the N labeling modes to label the to-be-labeled text information.

Description

Technical Field

[0001] The present application relates to the field of smart home / intelligent family technology, and more specifically, to a text information annotation method and device, a storage medium, and an electronic device. Background Art

[0002] Natural language annotation refers to the classification, analysis, interpretation and other operations of words, sentences, paragraphs and other units in the text to facilitate the subsequent natural language understanding and generation. Usually, this task requires a lot of manpower to complete, and has high requirements on the professional quality of personnel. The personnel who handle this task need to have a high level of linguistic knowledge. In addition, due to the subjectivity of personnel, the final result of the annotation may be quite different from the actual result. Therefore, after the annotation is completed, it is still necessary to invest a lot of manpower and time costs to complete the verification and proofreading.

[0003] It can be seen that the method of annotating text information in the related art has the problem of low annotation efficiency in the process of annotating text. Summary of the invention

[0004] The embodiments of the present application provide a text information annotation method and device, a storage medium, and an electronic device to at least solve the problem of low annotation efficiency in the text information annotation method in the related art during the text annotation process.

[0005] According to one aspect of an embodiment of the present application, a method for annotating text information is provided, which is applied to a smart device, and includes: obtaining text information to be annotated, and obtaining a target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, and N is a positive integer greater than or equal to 1; selecting M annotation models corresponding to the target annotation instruction from a group of annotation models according to the target annotation instruction, wherein M is a positive integer greater than or equal to 1, and the annotation methods used by each of the M annotation models are the N annotation methods in total, and the group of annotation models is a model trained using a group of training data, and the M annotation models include a chat generative pre-trained model ChatGPT; using the M annotation models to respectively annotate the text information to be annotated using the N annotation methods to obtain a first group of annotation results corresponding to the text information to be annotated, wherein each of the M annotation models is used to annotate the text information to be annotated using at least one annotation method in the N annotation methods.

[0006] According to another aspect of an embodiment of the present application, a text information annotation device is also provided, which is applied to an intelligent device, including: a first acquisition unit, used to acquire text information to be annotated, and acquire a target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that the text information to be annotated is annotated using N annotation methods in a group of annotation methods, and N is a positive integer greater than or equal to 1; a selection unit, used to select M annotation models corresponding to the target annotation instruction from a group of annotation models according to the target annotation instruction, wherein M is a positive integer greater than or equal to 1, and the annotation methods adopted by each of the M annotation models are the N annotation methods in total, and the group of annotation models is a model trained using a group of training data, and the M annotation models include a chat generative pre-training model ChatGPT; a annotation unit, used to use the M annotation models to annotate the text information to be annotated using the N annotation methods, respectively, to obtain a first group of annotation results corresponding to the text information to be annotated, wherein each of the M annotation models is used to annotate the text information to be annotated using at least one annotation method in the N annotation methods.

[0007] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned text information annotation method when running.

[0008] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the text information annotation method through the computer program.

[0009] In an embodiment of the present application, a method of automatically annotating the text information to be annotated by using a group of annotation models is adopted to obtain the text information to be annotated, and obtain a target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, and N is a positive integer greater than or equal to 1; according to the target annotation instruction, M annotation models corresponding to the target annotation instruction are selected from a group of annotation models, wherein M is a positive integer greater than or equal to 1, and the annotation methods used by each of the M annotation models are N annotation methods in total, and a group of annotation models is a model trained using a group of training data, and the M annotation models include a chat generation formula Pre-trained model ChatGPT; use M annotation models to adopt N annotation methods to respectively annotate the text information to be annotated, and obtain a first group of annotation results corresponding to the text information to be annotated, wherein each of the M annotation models is used to adopt at least one of the N annotation methods to annotate the text information to be annotated. Since the annotation process is performed by a group of annotation models, no manual annotation is required. At the same time, a group of annotation models can be used to implement annotation of a variety of different annotation methods for the text information to be annotated, so as to obtain a variety of annotation results in a shorter time, which can improve the annotation efficiency, and thus solve the problem of low annotation efficiency in the text annotation process of the annotation method of text information in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0012] Figure 1 is a schematic diagram of a hardware environment of a text information annotation method according to an embodiment of the present application;

[0013] Figure 2 is a flowchart of an optional text information annotation method according to an embodiment of the present application;

[0014] Figure 3 is a flowchart of another optional text information annotation method according to an embodiment of the present application;

[0015] Figure 4is a schematic diagram of an optional method for marking text information according to an embodiment of the present application;

[0016] Figure 5 is a schematic diagram of another optional text information annotation method according to an embodiment of the present application;

[0017] Figure 6 is a flowchart of another optional text information annotation method according to an embodiment of the present application;

[0018] Figure 7 is a structural block diagram of an optional text information annotation device according to an embodiment of the present application;

[0019] Figure 8 It is a structural block diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] According to one aspect of an embodiment of the present application, a text information annotation method is provided, which can be applied to smart devices. The text information annotation method is widely used in smart home (Smart Home), smart home, smart home device ecology, smart residential (Intelligence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above text information annotation method can be applied to Figure 1In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0023] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI, Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing equipment, a smart dishwasher, a smart projection equipment, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart speaker, a smart fresh air equipment, a smart kitchen and bathroom equipment, a smart bathroom equipment, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification equipment, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock and other smart devices.

[0024] The text information annotation method of the embodiment of the present application can be executed by the server 104, or can be executed by the server 104 and the terminal device 102 together. Taking the text information annotation method of the embodiment executed by the server 104 as an example, Figure 2 is a flow chart of an optional text information annotation method according to an embodiment of the present application, such as Figure 2 As shown, the process of the method may include the following steps:

[0025] Step S202, obtaining the text information to be annotated, and obtaining the target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, and N is a positive integer greater than or equal to 1.

[0026] The annotation method of text information in this embodiment can be applied to the scene of annotating natural language. The annotation processing of natural language refers to the classification, analysis, interpretation and other operations of words, sentences, paragraphs and other units in the text, so as to facilitate the subsequent natural language understanding and generation. The annotation of natural language includes the functions of syntactic analysis, part-of-speech tagging, keyword extraction, named entity recognition, knowledge annotation, sentiment analysis and so on.

[0027] Normally, the above-mentioned annotation work requires a lot of manpower to complete, and these people also need to have professional linguistic knowledge. In addition, due to the subjectivity of personnel, there is a possibility that the final result of the annotation is quite different from the actual result. After the annotation is completed, it is still necessary to invest a lot of manpower and time costs to complete the verification and proofreading. In the context of the current high-speed iteration of artificial intelligence, there are higher requirements for data quality and data volume. Simply carrying out manual data annotation is not only costly, but also shows that the data cannot keep up with the speed of algorithm iteration.

[0028] In the prior art, although there are some open source or commercial natural language annotation tools, such as BRAT (a web-based text annotation tool), YEDDA (a desktop text annotation tool based on the Tkinter (cross-platform graphical user interface toolkit) library), Chinese Annotator (a web text annotation tool based on the Flask (a framework or platform for displaying and interacting with animation and multimedia content) framework and Vue.js (a progressive JavaScript (a lightweight, interpreted or just-in-time compiled programming language with function priority) framework) technology), etc. These tools can meet the needs of partial entity annotation and text classification. Among them, BRAT is mainly used for the annotation of entities and relationships. The technical principle of BRAT is to use JavaScript and SVG (Scalable Vector Graphics) technology to realize the visual interface on the browser side, and use Python (a cross-platform computer programming language) and Flask framework to realize data processing and storage on the server side. BRAT supports input and output in multiple formats, such as XML (EXtensible Markup Language), JSON (JavaScript Object Notation), CoNLL (a linguistic data format), etc., and provides some automatic annotation functions, such as predefined dictionaries, regular expressions, machine learning models, etc. YEDDA is mainly used for the annotation of entities and attributes. The technical principle of YEDDA is to use the Tkinter library to provide a simple and easy-to-use graphical user interface, and use Python to implement data processing and storage. YEDDA supports a variety of shortcut key operations, such as undo, redo, jump, etc., and provides some intelligent annotation functions, such as auto-completion, auto-correction, auto-recommendation, etc. Chinese Annotator is mainly used for the annotation of entities and relationships in Chinese texts. The technical principle of Chinese Annotator is to use the Flask framework to implement data processing and storage on the server side, and use Vue.js technology to implement a visual interface on the browser side. Chinese Annotator supports input and output in multiple formats, such as CSV (Comma-Separated Values), JSON (JavaScript Object Notation), etc., and provides some auxiliary functions, such as word segmentation, part-of-speech tagging, named entity recognition, etc.

[0029] However, the annotation functions of the above-mentioned natural language annotation tools are single and have limitations, and the annotation results are difficult to unify and manage. For example, the annotation interface of BRAT is not friendly and intuitive enough, and the operation is complicated and cumbersome. Users need to be familiar with various shortcut keys and commands, and can only annotate entities and relationships. Users also need to manually export and import data files. YEDDA lacks the support of intelligent algorithms in the annotation process, resulting in a lot of manual repetitive work and low efficiency. It can only annotate entities and attributes, and users also need to manually save and load data files. Although Chinese Annotator has realized the localization of the interface, it lacks intelligent algorithm support. It currently only supports the annotation of entities and relationships, and users also need to manually download and upload data files.

[0030] In order to at least partially solve the above problems, in this embodiment, an automatic annotation platform configured with multiple annotation models can be constructed to support multiple annotation methods. It can output multiple different annotation results for the same text and is suitable for a variety of different scenarios. While realizing the diversification of annotation functions, it can improve the efficiency of text annotation.

[0031] In the automatic annotation platform, text information to be annotated and target annotation instructions corresponding to the text information to be annotated can be obtained. Here, the target annotation instruction can be used to indicate that N annotation methods in a set of annotation methods are used to annotate the text information to be annotated. N is a positive integer greater than or equal to 1.

[0032] The above-mentioned set of annotation methods may include but are not limited to part-of-speech tagging, named entity tagging, syntax tagging, keyword tagging, knowledge tagging, and mood tagging. Among them, part-of-speech tagging is the process of determining the grammatical category of each word in a given sentence, determining its part of speech and tagging it. In layman's terms, after the sentence is segmented, the properties of the words in the sentence are marked after the segmented words, such as noun (n), verb (v), etc. Named entity tagging is to find the text span that constitutes a proper noun and mark the type of entity. The four most common entity tags are: PER (Person), LOC (Location), ORG (Organization), or GPE (Geo-Political Entity). However, the term named entity is often extended to include things that are not entities themselves, including dates, times and other types of time expressions, and even numerical expressions such as prices. Syntactic tagging is to analyze and tag the syntax in a given sentence. Keyword tagging is to extract words or phrases that reflect the core content in a given sentence or text and tag them. Knowledge annotation can be used to annotate knowledge points of words in a given text. For example, if "library" is detected in a given text, "library" can be annotated with relevant knowledge points such as "library is an institution that collects, organizes, and stores books and materials for people to read and refer to..." Different business scenarios can have different annotation focuses. Tone annotation is mainly used to analyze the current emotional scenes of characters. The N annotation methods can be one or more or even all of the above-mentioned annotation methods.

[0033] Step S204, according to the target labeling instruction, select M labeling models corresponding to the target labeling instruction from a group of labeling models, where M is a positive integer greater than or equal to 1, and each of the M labeling models adopts a total of N labeling methods, a group of labeling models is a model trained using a group of training data, and the M labeling models include a chat generative pre-trained model ChatGPT.

[0034] According to the target labeling instruction, M labeling models corresponding to the target labeling instruction can be selected from a set of labeling models. Here, a set of labeling models can be a set of labeling models in the aforementioned automatic labeling platform. M can be a positive integer greater than or equal to 1. The labeling method adopted by each of the M labeling models is N labeling methods in total, and M and N can be different values, that is, each of the M models can respectively adopt one or more of the above-mentioned set of labeling methods, and the M labeling models can include the chat generative pre-trained model ChatGPT (ChatGenerative Pre-trained Transformer). ChatGPT can perform a set of labeling methods including but not limited to part-of-speech tagging, named entity tagging, syntactic tagging, keyword tagging, knowledge tagging, and tone tagging.

[0035] The above-mentioned set of labeled models can be models trained using a set of training data. Figure 3 As shown in the figure, in the database of the automatic annotation platform, computing resources (for model training and reasoning) and storage resources (for server deployment and data storage, etc.) can be prepared. The training and deployment of multiple models can be completed through computing resources and storage resources. During deployment, performance optimization can be completed according to engineering needs. A set of training data can be the computing resources stored in the database of the automatic annotation platform. The most core intelligent and automated part of the automatic annotation platform is the multi-model reasoning part that includes a set of annotation models. In order to build this part, a large amount of model training data needs to be prepared. The training data can include supervised samples and unsupervised samples. The process of using supervised samples is to first train the model through known training samples (such as known inputs and corresponding outputs) to obtain an optimal model, and then apply this model to new data and map it to output results. After this process, the model has predictive ability. The process of using unsupervised samples is to directly use data for modeling and analysis, that is, the model needs to learn and explore on its own.

[0036] Step S206, use M annotation models to adopt N annotation methods to respectively annotate the text information to be annotated, and obtain a first group of annotation results corresponding to the text information to be annotated, wherein each of the M annotation models is used to annotate the text information to be annotated using at least one of the N annotation methods.

[0037] After determining the M annotation models, the M annotation models can be used to respectively annotate the text information to be annotated using N annotation methods to obtain a first set of annotation results corresponding to the text information to be annotated. Each of the M annotation models can be used to annotate the text information to be annotated using at least one of the N annotation methods.

[0038] The process of annotating the text information to be annotated by each of the M annotation models may be performed simultaneously or sequentially, which is not limited in this embodiment.

[0039] Through the above steps S202 to S206, the text information to be annotated is obtained, and the target annotation instruction corresponding to the text information to be annotated is obtained, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, and N is a positive integer greater than or equal to 1; according to the target annotation instruction, M annotation models corresponding to the target annotation instruction are selected from a group of annotation models, wherein M is a positive integer greater than or equal to 1, and each of the M annotation models adopts a total of N annotation methods, and a group of annotation models is a model trained using a group of training data, and the M annotation models include a chat generation pre-training model ChatGPT; the M annotation models are used to use N annotation methods to annotate the text information to be annotated, and a first group of annotation results corresponding to the text information to be annotated are obtained, wherein each of the M annotation models is used to annotate the text information to be annotated using at least one annotation method in the N annotation methods, which solves the problem of low annotation efficiency in the text information annotation method in the related art during the text annotation process, thereby improving the annotation efficiency.

[0040] In an exemplary embodiment, obtaining text information to be annotated includes:

[0041] S11, obtaining text information to be annotated input on the annotation interactive interface; or,

[0042] S12, acquiring the collected voice signal, and converting the voice signal into text information to be annotated through a language recognition model of the automatic annotation platform.

[0043] The above-mentioned acquisition of the text information to be annotated may refer to acquisition of the text information to be annotated input on the annotation interactive interface, or may refer to acquisition of the collected voice signal and conversion of the voice signal into the text information to be annotated through the language recognition model of the automatic annotation platform. The annotation interactive interface may be the interactive interface of the aforementioned automatic annotation platform.

[0044] The above-mentioned voice signal may be collected by a terminal device associated with the automatic annotation platform. The terminal device associated with the automatic annotation platform may be a mobile terminal or a PC (Personal Computer) terminal.

[0045] When using the automatic annotation platform for annotation, there are two input methods. The first is to directly input text in the interactive interface and perform automatic annotation through the multiple models in the automatic annotation platform. The second method is through voice input. The automatic annotation platform can pass the collected voice into the language recognition model (such as a neural network model) to complete voice recognition, convert it into text, and then input it into the multiple models for automatic annotation.

[0046] Through this embodiment, the information to be annotated is input into the automatic annotation platform by direct text input or voice input, which can improve the convenience of using the automatic annotation platform and further improve the user experience.

[0047] In an exemplary embodiment, obtaining a target annotation instruction corresponding to the text information to be annotated includes:

[0048] S21, obtaining a target annotation instruction input on the annotation interaction interface; or

[0049] S22, obtaining scene description information input on the annotation interaction interface, wherein the scene description information is used to describe the scene to which the annotation result of the text information to be annotated is applied; generating a target annotation instruction matching the scene description information, wherein the annotation result obtained by annotating the text information to be annotated using N annotation methods indicated by the target annotation instruction is the annotation result allowed to be used in the scene.

[0050] The target annotation instruction corresponding to the text information to be annotated may be directly input by the user on the annotation interaction interface, that is, the user may input the target annotation instruction simultaneously when inputting the text information to be annotated.

[0051] Optionally, the method of inputting the target annotation instruction can be the same as the method of inputting the text information to be annotated, that is, in the case where obtaining the text information to be annotated is to obtain the text information to be annotated input on the annotation interaction interface, the method of obtaining the target annotation instruction can also be to obtain the target annotation instruction input on the annotation interaction interface. In the case where obtaining the text information to be annotated is to obtain the collected voice signal and convert the voice signal into the text information to be annotated through the language recognition model of the automatic annotation platform, the method of obtaining the target annotation instruction can also be to obtain the collected voice signal and convert the voice signal into the target annotation instruction through the language recognition model.

[0052] The target annotation instruction corresponding to the text information to be annotated may also be generated based on the scene description information input by the user. That is, the scene description information input on the annotation interaction interface is obtained, and a target annotation instruction matching the scene description information is generated. Here, the scene description information may be used to describe the scene to which the annotation result of the text information to be annotated is applied. The annotation result obtained by annotating the text information to be annotated using the N annotation methods indicated by the target annotation instruction may be the annotation result allowed to be used in the scene.

[0053] The input of the scene description information may also be the same as the aforementioned method of inputting the text information to be annotated, which will not be elaborated in this embodiment.

[0054] The above-mentioned scenario description information may include, but is not limited to, description information of text matching scenarios, human-computer dialogue scenarios, knowledge graph construction scenarios, intent parsing scenarios, and command control scenarios. The text matching scenario may be a match between two texts (such as the match between words and document resources in information retrieval, the match between two document resources in text data deduplication, etc.), and the text matching may be achieved through annotation results such as keywords. In the human-computer dialogue scenario, the intelligent device may determine the user's current emotional state (happy, confused, etc.) by annotating the tone in the sentence input by the user. The knowledge graph construction scenario may be a knowledge base with a graph structure, including application scenarios such as semantic search and intelligent question and answer for knowledge graphs, and the knowledge graph may be constructed through annotation results such as knowledge. The intent parsing scenario mainly extracts the syntactic features of a given sentence through syntactic analysis, and determines the intent in the sentence based on the syntactic features. The command control scenario may be a scenario in which a user controls an intelligent device through voice or other means, in which the user's instructions may be determined based on the annotation results such as named entities or parts of speech, and related operations may be performed. It should be noted that among the above-mentioned multiple scenarios, there may be scenarios with overlapping annotation requirements, for example, command control scenarios, intent analysis scenarios and human-computer dialogue scenarios may have partial overlaps.

[0055] Optionally, in the case where the scene description information is description information of a text matching scene, the generated target annotation instruction may indicate annotation methods including keyword annotation methods to annotate the text information to be annotated. In the case where the scene description information is description information of a human-computer dialogue scene, the generated target annotation instruction may indicate annotation methods including tone annotation methods to annotate the text information to be annotated. In the case where the scene description information is description information of a knowledge graph construction scene, the generated target annotation instruction may indicate annotation methods including knowledge annotation methods to annotate the text information to be annotated. In the case where the scene description information is description information of an intent parsing scene, the generated target annotation instruction may indicate annotation methods including syntactic annotation methods to annotate the text information to be annotated. In the case where the scene description information is description information of an instruction control scene, the generated target annotation instruction may indicate annotation methods including named entity and / or part-of-speech tagging methods to annotate the text information to be annotated.

[0056] Through this embodiment, the automatic annotation platform determines the annotation instructions for the corresponding text through direct text input or voice input, which can improve the intelligence of the automatic annotation platform.

[0057] In an exemplary embodiment, M annotation models are used to respectively annotate the text information to be annotated using N annotation methods to obtain a first group of annotation results corresponding to the text information to be annotated, including:

[0058] S31, when M is greater than or equal to 2, annotate the text information to be annotated using the i-th annotation model among the M annotation models through the following steps, where i is a positive integer greater than or equal to 1 and less than or equal to M:

[0059] S32, obtaining the jth annotation result, wherein the jth annotation result is an annotation result obtained by annotating the text information to be annotated using the jth annotation model among the M annotation models, i and j are different, and j is a positive integer greater than or equal to 1 and less than or equal to M;

[0060] S33, inputting the jth annotation result and the text information to be annotated into the ith annotation model to obtain the ith annotation result, wherein the ith annotation result is the annotation result obtained by the ith annotation model annotating the text information to be annotated according to the jth annotation result.

[0061] In the process of using M annotation models to adopt N annotation methods to respectively annotate the text information to be annotated, the output result of one annotation model after annotating the text information to be annotated can be input into another annotation model together with the text information to be annotated to improve the accuracy of the annotation result of the other annotation model.

[0062] In this embodiment, when M is greater than or equal to 2, the following steps can be used to annotate the text information to be annotated using the i-th annotation model among the M annotation models, and obtain the j-th annotation result. The j-th annotation result can be output as one of the annotation results in the first group of annotation results corresponding to the text information to be annotated. The j-th annotation result can also be used as input data of the i-th annotation model, that is, the j-th annotation result and the text information to be annotated are input into the i-th annotation model to obtain the i-th annotation result. Here, i is a positive integer greater than or equal to 1 and less than or equal to M. The j-th annotation result is the annotation result obtained by annotating the text information to be annotated using the j-th annotation model among the M annotation models, i is different from j, and j is a positive integer greater than or equal to 1 and less than or equal to M. The i-th annotation result is the annotation result obtained by annotating the text information to be annotated using the i-th annotation model according to the j-th annotation result.

[0063] For example, when keyword annotation and knowledge annotation are performed by two annotation models respectively, annotation model one can first annotate the text information with keywords, and then based on the keyword annotation results, annotation model two can annotate the text information. When annotating the text information, annotation model two can focus on the annotation results of the keywords in the text information.

[0064] Through this embodiment, based on the annotation result of one annotation model on the text information to be annotated, the text information to be annotated is annotated in another annotation model using other annotation methods, which can improve the accuracy of the annotation result of the other annotation model.

[0065] In an exemplary embodiment, M annotation models are used to adopt N annotation methods to respectively annotate the text information to be annotated, and a first group of annotation results corresponding to the text information to be annotated is obtained, including at least one of the following:

[0066] S41, when the target annotation instruction is used to indicate that the text information to be annotated is annotated in a first annotation manner, obtaining a first annotation result output by a first annotation model matching the first annotation manner in a group of annotation models, wherein the first annotation manner is an annotation manner that uniquely corresponds to the first annotation model;

[0067] S42, when the target annotation instruction is used to indicate that the text information to be annotated is annotated using multiple second annotation methods matching the same annotation model, obtain the second annotation result corresponding to each second method in the multiple second annotation methods output by a second annotation model matching the multiple second annotation methods in a group of annotation models.

[0068] When using M annotation models to adopt N annotation methods to respectively annotate the text information to be annotated, the M annotation models can respectively adopt the annotation method corresponding to each annotation model among the N annotation methods for annotation, and each annotation model can adopt one or more annotation methods for annotation.

[0069] In this embodiment, when the target annotation instruction is used to indicate that the text information to be annotated is annotated in a first annotation mode, a first annotation result output by a first annotation model matching the first annotation mode in a group of annotation models can be obtained. Here, the first annotation mode is an annotation mode that uniquely corresponds to the first annotation model.

[0070] When the target annotation instruction is used to instruct the use of multiple second annotation methods matching the same annotation model to annotate the text information to be annotated, a second annotation result corresponding to each second method in the multiple second annotation methods output by a second annotation model matching the multiple second annotation methods in a set of annotation models can be obtained.

[0071] Through this embodiment, according to different annotation requirements, different annotation models are selected to annotate the text information to be annotated, which can improve the efficiency of obtaining the annotation results.

[0072] In an exemplary embodiment, when the target annotation instruction is used to indicate that the text information to be annotated is annotated in a first annotation manner, obtaining a first annotation result output by a first annotation model matching the first annotation manner in a group of annotation models includes at least one of the following:

[0073] S51, when the target annotation instruction is used to indicate that the text information to be annotated is annotated in a keyword annotation manner, obtaining a keyword annotation result output by a statistical model in a set of annotation models;

[0074] S52, when the target annotation instruction is used to indicate that the text information to be annotated is annotated in a knowledge annotation manner, obtaining a knowledge annotation result output by a pre-trained generative model in a set of annotation models;

[0075] Among them, the multiple second annotation methods include at least one of the following: part-of-speech tagging, named entity tagging, syntax tagging, and mood tagging, and the second annotation model is a pre-trained language model.

[0076] In this embodiment, a set of annotation models can be Figure 3As shown, they include statistical models, pre-trained generative models and pre-trained language models. Statistical models can be used for annotation of keywords, etc., and can be TF-IDF (termfrequency–inverse document frequency) models. Pre-trained generative models can be used for annotation of knowledge, etc., and can be GPT (Generative Pre-Trained Transformer, a generative pre-trained Transformer (a neural network deep learning model) model) models, such as GPT2, GPT3, etc. Pre-trained language models can be used for annotation of parts of speech, named entity recognition, syntax, mood, etc., and can be neural network models, such as BERT (Bidirectional Encoder Representations from Transformers, a bidirectional semantic encoding representation model built on transformer) models.

[0077] The TF-IDF model is a commonly used weighting technique for information retrieval and text mining to evaluate the importance of a word to a document set or a document in a corpus. The implicit assumption behind TF is that the query keyword should be more important than other words, and the importance of a document, that is, the relevance, is proportional to the number of times the word appears in the document. The relevance can be expressed as Figure 4 As shown, both keywords and documents are expressed as vectors, and then the similarity between the keyword vector and the document vector is determined by the operation between the vectors to determine the relevance. The highest similarity means the highest relevance. TF-IDF is actually: TF*IDF (Term Frequency*Inverse Document Frequency). The high frequency of a term in a specific file and the low file frequency of the term in the entire file collection can generate a high-weight TF-IDF.

[0078] The BERT model can use a large amount of unlabeled text for self-supervised learning, learn the deep semantic representation of the text, and adapt to different downstream natural language processing tasks through fine-tuning. Figure 5 As shown in the figure, the BERT model is composed of multiple transformer encoders stacked together.

[0079] In this embodiment, when the target annotation instruction is used to indicate that the text information to be annotated is annotated by a keyword annotation method, the keyword annotation results output by a statistical model in a set of annotation models can be obtained. When the target annotation instruction is used to indicate that the text information to be annotated is annotated by a knowledge annotation method, the knowledge annotation results output by a pre-trained generative model in a set of annotation models can be obtained.

[0080] In the case where the target annotation instruction is used to instruct the text information to be annotated to be annotated in a set of annotation methods other than the keyword annotation method and the knowledge annotation method, the annotation results output by the pre-trained language model in a set of annotation models can be obtained. Here, the annotation results output by the pre-trained language model may include at least one of the following: part-of-speech tagging results, named entity tagging results, syntactic tagging results, and mood tagging results. Correspondingly, the aforementioned multiple second annotation methods may include at least one of the following: part-of-speech tagging, named entity tagging, syntactic tagging, and mood tagging. The second annotation model may be a pre-trained language model.

[0081] Optionally, the annotation model that matches the above-mentioned first annotation method and multiple second annotation methods may also be the aforementioned ChatGPT.

[0082] Through this embodiment, according to different annotation requirements, different annotation models are selected to annotate the text information to be annotated, which can improve the efficiency of obtaining the annotation results.

[0083] In an exemplary embodiment, after the text information to be annotated is annotated using M annotation models and N annotation methods respectively to obtain a first group of annotation results corresponding to the text information to be annotated, the method further includes:

[0084] S61, obtaining a second group of annotation results obtained by correcting the first group of annotation results;

[0085] S62, saving the second set of annotation results and the text information to be annotated to a designated database, and using the second set of annotation results and the text information to be annotated to train at least part of a set of annotation models, wherein the designated database is a database for storing a set of training data.

[0086] The first set of annotation results corresponding to the acquired text information to be annotated can be verified, and the first set of annotation results can be corrected according to the verification result. The verification method can be manual verification or model verification, which is not limited in this embodiment.

[0087] In this embodiment, after obtaining a first group of annotation results corresponding to the text information to be annotated, a second group of annotation results obtained by correcting the first group of annotation results may be obtained.

[0088] In order to improve the annotation accuracy of the annotation model, the second group of annotation results and the text information to be annotated can be saved in a designated database, and the second group of annotation results and the text information to be annotated can be used to train at least part of the annotation models in a group of annotation models. Here, the second group of annotation results input into the designated database can retain the correct results and corrected error results of the first group of annotation results, and the designated database is a database for storing a set of training data. The training of the annotation model can be optimization training or re-training. Optimization training can refer to the optimization of the annotation model after a group of annotation models are trained during the use of the aforementioned automatic annotation platform. Re-training can refer to the re-training of the annotation model during the training of a group of annotation models.

[0089] Through this embodiment, the corrected annotation results and corresponding text information are stored in a designated database to train the annotation model, which can improve the annotation accuracy of the annotation model.

[0090] In an exemplary embodiment, after obtaining a second set of annotation results obtained by correcting the first set of annotation results, the method further includes:

[0091] S71, displaying a second group of annotation results and the text information to be annotated on the annotation display interface, wherein the second group of annotation results is a group of annotation results obtained by adjusting at least part of the annotation results in the first group of annotation results in response to the input adjustment instruction.

[0092] After obtaining the second set of annotation results obtained by correcting the first set of annotation results, in this embodiment, the second set of annotation results and the text information to be annotated can also be displayed on the annotation display interface. Here, the annotation display interface can be the interface for displaying annotation results in the aforementioned automatic annotation platform.

[0093] The second set of annotation results may be a set of annotation results obtained by adjusting at least part of the annotation results in the first set of annotation results in response to the input adjustment instruction, that is, the second set of annotation results displayed on the display interface may be a set of annotation results after adjustment and determined to be correctly annotated. The adjustment instruction here may be input by the object performing the verification, that is, when the verification is manual verification, the adjustment instruction may be input by the verification personnel, and when the verification is model verification, the adjustment instruction may be input by the verification model.

[0094] Through this embodiment, the adjusted automatic annotation results are displayed on the annotation display interface, which can make the interface interaction more intuitive and the annotation results easier to unify and manage.

[0095] The following is an explanation of the annotation method of text information in the embodiment of the present application in conjunction with an optional example. In this optional example, a group of annotation models includes a statistical model, a pre-trained generative model, and a pre-trained language model.

[0096] This optional example provides an automatic annotation platform based on natural language processing of multi-algorithm models, which can continuously improve the level of intelligence with a small number of manual parameters, forming a positive cycle to make the automatic annotation effect better and better. At the same time, it can also complete multiple natural language annotation tasks efficiently and accurately.

[0097] like Figure 6 As shown, the process of the text information annotation method in this optional example may include the following steps:

[0098] Step S602: input text to be annotated in the automatic annotation platform.

[0099] Including direct input and voice input in the interactive interface.

[0100] Step S604: The text to be annotated is processed by the multiple models in the automatic annotation platform.

[0101] Step S606, outputting the tagging results including at least part-of-speech tagging, named entity recognition, syntax tagging, keyword tagging, knowledge tagging, and tone tagging.

[0102] Step S608: verify the automatic labeling result.

[0103] Step S610, adding the verified data to the model training, and continuously optimizing the model.

[0104] Step S612, the result of automatic annotation is displayed interactively on the interface.

[0105] Through this optional example, multiple natural language annotation tasks can be completed. At the same time, due to the participation of multiple algorithm models, the platform has a high level of intelligence, and during the operation of the entire platform, the platform's intelligence level can be continuously improved. At the same time, automatic annotation through the platform can reduce manpower and time costs and greatly improve output efficiency. With higher data output efficiency and data annotation accuracy, it can also directly improve the iteration speed of artificial intelligence.

[0106] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0107] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware server, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0108] According to another aspect of an embodiment of the present application, a text information annotation device for implementing the above-mentioned text information annotation method is also provided, and the text information annotation device can be applied to a smart device. Figure 7 is a structural block diagram of an optional text information annotation device according to an embodiment of the present application, such as Figure 7 As shown, the device may include:

[0109] The first acquisition unit 702 is used to acquire the text information to be annotated, and acquire a target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, where N is a positive integer greater than or equal to 1;

[0110] A selection unit 704 is connected to the first acquisition unit 702 and is used to select M annotation models corresponding to the target annotation instruction from a group of annotation models according to the target annotation instruction, wherein M is a positive integer greater than or equal to 1, and each of the M annotation models uses a total of N annotation methods, a group of annotation models is a model trained using a group of training data, and the M annotation models include a chat generative pre-trained model ChatGPT;

[0111] The labeling unit 706 is connected to the selection unit 704, and is used to use M labeling models to adopt N labeling methods to respectively label the text information to be labeled, and obtain a first group of labeling results corresponding to the text information to be labeled, wherein each of the M labeling models is used to label the text information to be labeled using at least one of the N labeling methods.

[0112] It should be noted that the first acquisition unit 702 in this embodiment can be used to execute the above step S202, the selection unit 704 in this embodiment can be used to execute the above step S204, and the marking unit 706 in this embodiment can be used to execute the above step S206.

[0113] Through the above module, the text information to be annotated and the target annotation instruction are obtained, and the target annotation instruction indicates that N annotation methods in a group of annotation methods are used to annotate the text information to be annotated respectively; according to the target annotation instruction, M annotation models corresponding to the target annotation instruction are selected from a group of annotation models, wherein M is a positive integer greater than or equal to 1, and each of the M annotation models adopts a total of N annotation methods, and a group of annotation models is a model trained using a group of training data; M annotation models are used to adopt N annotation methods to annotate the text information to be annotated respectively, and a first group of annotation results are obtained, and each of the M annotation models adopts at least one annotation method in the N annotation methods to annotate the text information to be annotated, which solves the problem of low annotation efficiency in the annotation method of text information in the related art in the process of annotating text, thereby improving the annotation efficiency.

[0114] In an exemplary embodiment, the first acquiring unit includes:

[0115] A first acquisition module is used to acquire text information to be annotated that is input on the annotation interaction interface; or,

[0116] The second acquisition module is used to acquire the collected voice signal and convert the voice signal into text information to be annotated through the language recognition model of the automatic annotation platform.

[0117] In an exemplary embodiment, the first acquiring unit includes:

[0118] A third acquisition module is used to acquire a target annotation instruction input on the annotation interaction interface; or

[0119] The fourth acquisition module is used to obtain scene description information input on the annotation interaction interface, wherein the scene description information is used to describe the scene to which the annotation results of the text information to be annotated are applied; generate a target annotation instruction that matches the scene description information, wherein the annotation results obtained by annotating the text information to be annotated using the N annotation methods indicated by the target annotation instruction are the annotation results allowed to be used in the scene.

[0120] In an exemplary embodiment, the labeling unit includes:

[0121] The annotation module is used for annotating the text information to be annotated by using the i-th annotation model among the M annotation models through the following steps when M is greater than or equal to 2, wherein i is a positive integer greater than or equal to 1 and less than or equal to M:

[0122] Obtain the jth annotation result, wherein the jth annotation result is an annotation result obtained by annotating the text information to be annotated using the jth annotation model among the M annotation models, i and j are different, and j is a positive integer greater than or equal to 1 and less than or equal to M;

[0123] The jth annotation result and the text information to be annotated are input into the ith annotation model to obtain the ith annotation result, wherein the ith annotation result is the annotation result obtained by the ith annotation model annotating the text information to be annotated according to the jth annotation result.

[0124] In an exemplary embodiment, the annotation unit includes at least one of the following:

[0125] a fifth acquisition module, configured to acquire, when the target annotation instruction is used to indicate that the text information to be annotated is annotated in a first annotation manner, a first annotation result output by a first annotation model matching the first annotation manner in a group of annotation models, wherein the first annotation manner is an annotation manner uniquely corresponding to the first annotation model;

[0126] The sixth acquisition module is used to obtain the second annotation result corresponding to each second method in a plurality of second annotation methods output by a second annotation model matching the plurality of second annotation methods in a set of annotation models when the target annotation instruction is used to indicate that the text information to be annotated is annotated using a plurality of second annotation methods matching the same annotation model.

[0127] In an exemplary embodiment, the fifth acquisition module includes at least one of the following:

[0128] A first acquisition submodule is used to acquire a keyword annotation result output by a statistical model in a set of annotation models when the target annotation instruction is used to indicate that the text information to be annotated is annotated in a keyword annotation manner;

[0129] A second acquisition submodule is used to obtain a knowledge annotation result output by a pre-trained generative model in a set of annotation models when the target annotation instruction is used to indicate that the text information to be annotated is annotated in a knowledge annotation manner;

[0130] Among them, the multiple second annotation methods include at least one of the following: part-of-speech tagging, named entity tagging, syntax tagging, and mood tagging, and the second annotation model is a pre-trained language model.

[0131] In an exemplary embodiment, the above device further comprises:

[0132] A second acquisition unit is used to use M annotation models to adopt N annotation methods to respectively annotate the text information to be annotated, and after obtaining a first group of annotation results corresponding to the text information to be annotated, obtain a second group of annotation results obtained by correcting the first group of annotation results;

[0133] An execution unit is used to save the second set of annotation results and the text information to be annotated into a designated database, and use the second set of annotation results and the text information to be annotated to train at least part of a set of annotation models, wherein the designated database is a database used to store a set of training data.

[0134] In an exemplary embodiment, the above device further comprises:

[0135] A display unit is used to display the second group of annotation results and the text information to be annotated on the annotation display interface after obtaining a second group of annotation results obtained by correcting the first group of annotation results, wherein the second group of annotation results is a group of annotation results obtained by adjusting at least part of the annotation results in the first group of annotation results in response to an input adjustment instruction.

[0136] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the above modules as part of the device can be run in Figure 1 In the hardware environment shown, it can be implemented by software or by hardware, wherein the hardware environment includes a network environment.

[0137] According to another aspect of the embodiment of the present application, a storage medium is also provided, and the storage medium can be located on a smart device. Optionally, in this embodiment, the storage medium can be used to execute the program code of any of the above-mentioned text information annotation methods in the embodiment of the present application.

[0138] Optionally, in this embodiment, the storage medium may be located on at least one network device among a plurality of network devices in the network shown in the above embodiment.

[0139] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps:

[0140] S1, obtaining text information to be annotated, and obtaining a target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, where N is a positive integer greater than or equal to 1;

[0141] S2, according to the target labeling instruction, select M labeling models corresponding to the target labeling instruction from a group of labeling models, where M is a positive integer greater than or equal to 1, and each labeling model in the M labeling models adopts a total of N labeling methods, a group of labeling models is a model trained using a group of training data, and the M labeling models include a chat generative pre-training model ChatGPT;

[0142] S3, using M annotation models to adopt N annotation methods to respectively annotate the text information to be annotated, and obtaining a first group of annotation results corresponding to the text information to be annotated, wherein each of the M annotation models is used to annotate the text information to be annotated using at least one of the N annotation methods.

[0143] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, which will not be described in detail in this embodiment.

[0144] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

[0145] According to another aspect of an embodiment of the present application, an electronic device for implementing the above-mentioned text information annotation method is also provided. The electronic device may be a smart device, a server, a terminal, or a combination thereof.

[0146] Figure 8 is a structural block diagram of an optional electronic device according to an embodiment of the present application, such as Figure 8 As shown, it includes a processor 802, a communication interface 804, a memory 806 and a communication bus 808, wherein the processor 802, the communication interface 804 and the memory 806 communicate with each other through the communication bus 808, wherein,

[0147] Memory 806, used for storing computer programs;

[0148] The processor 802 is used to implement the following steps when executing the computer program stored in the memory 806:

[0149] S1, obtaining text information to be annotated, and obtaining a target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, where N is a positive integer greater than or equal to 1;

[0150] S2, according to the target labeling instruction, select M labeling models corresponding to the target labeling instruction from a group of labeling models, where M is a positive integer greater than or equal to 1, and each labeling model in the M labeling models adopts a total of N labeling methods, a group of labeling models is a model trained using a group of training data, and the M labeling models include a chat generative pre-training model ChatGPT;

[0151] S3, using M annotation models to adopt N annotation methods to respectively annotate the text information to be annotated, and obtaining a first group of annotation results corresponding to the text information to be annotated, wherein each of the M annotation models is used to annotate the text information to be annotated using at least one of the N annotation methods.

[0152] Optionally, the communication bus may be a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The communication interface is used for communication between the electronic device and other devices.

[0153] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0154] As an example, the memory 806 may include but is not limited to the first acquisition unit 702, the selection unit 704 and the annotation unit 706 in the text information annotation device. In addition, it may also include but is not limited to other module units in the text information annotation device, which will not be repeated in this example.

[0155] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0156] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0157] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only. The device for implementing the above-mentioned text information annotation method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 8 The structure of the electronic device is not limited. Figure 8 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 8 Different configurations shown.

[0158] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0159] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0160] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0161] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0162] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0163] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution provided in this embodiment.

[0164] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist physically separately, or at least two units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0165] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for marking text information, characterized in that: include: Acquire text information to be annotated, and acquire a target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, where N is a positive integer greater than or equal to 1; According to the target labeling instruction, M labeling models corresponding to the target labeling instruction are selected from a group of labeling models, wherein M is a positive integer greater than or equal to 1, and the labeling method adopted by each labeling model in the M labeling models is the N labeling methods in total, and the group of labeling models is a model trained using a group of training data, and the M labeling models include a chat generative pre-training model ChatGPT; The text information to be annotated is annotated respectively by using the M annotation models and the N annotation methods to obtain a first group of annotation results corresponding to the text information to be annotated, wherein each of the M annotation models is used to annotate the text information to be annotated by using at least one of the N annotation methods.

2. The method according to claim 1, characterized in that The obtaining of the text information to be annotated includes: Acquire the text information to be annotated input on the annotation interactive interface; or, The collected speech signal is acquired, and the speech signal is converted into the text information to be annotated through a language recognition model.

3. The method according to claim 1, characterized in that The obtaining of the target marking instruction corresponding to the text information to be marked includes: Acquire the target annotation instruction input on the annotation interaction interface; or Acquire scene description information input on the annotation interaction interface, wherein the scene description information is used to describe the scenario to which the annotation result of the text information to be annotated is applied; generate the target annotation instruction matching the scene description information, wherein the annotation result obtained by annotating the text information to be annotated using the N annotation methods indicated by the target annotation instruction is the annotation result allowed to be used in the scenario.

4. The method according to claim 1, characterized in that The using the M annotation models to respectively annotate the text information to be annotated in the N annotation methods to obtain a first group of annotation results corresponding to the text information to be annotated includes: When M is greater than or equal to 2, the text information to be annotated is annotated using the i-th annotation model among the M annotation models through the following steps, where i is a positive integer greater than or equal to 1 and less than or equal to M: Obtaining a j-th annotation result, wherein the j-th annotation result is an annotation result obtained by annotating the text information to be annotated using the j-th annotation model among the M annotation models, i and j are different, and j is a positive integer greater than or equal to 1 and less than or equal to M; The j-th annotation result and the text information to be annotated are input into the i-th annotation model to obtain the i-th annotation result, wherein the i-th annotation result is the annotation result obtained by the i-th annotation model annotating the text information to be annotated according to the j-th annotation result.

5. The method according to claim 1, characterized in that The using the M annotation models to respectively annotate the text information to be annotated in the N annotation methods to obtain a first group of annotation results corresponding to the text information to be annotated, including at least one of the following: When the target annotation instruction is used to indicate that the text information to be annotated is annotated in a first annotation manner, obtaining a first annotation result output by a first annotation model in the group of annotation models that matches the first annotation manner, wherein the first annotation manner is a annotation manner that uniquely corresponds to the first annotation model; When the target annotation instruction is used to instruct the use of multiple second annotation methods matching the same annotation model to annotate the text information to be annotated, obtain a second annotation result corresponding to each second method in the multiple second annotation methods output by a second annotation model in the group of annotation models that matches the multiple second annotation methods.

6. The method according to claim 5, characterized in that When the target annotation instruction is used to indicate that the text information to be annotated is annotated in a first annotation manner, obtaining a first annotation result output by a first annotation model in the group of annotation models that matches the first annotation manner includes at least one of the following: When the target annotation instruction is used to indicate that the text information to be annotated is annotated in a keyword annotation manner, obtaining a keyword annotation result output by a statistical model in the set of annotation models; When the target annotation instruction is used to instruct to annotate the text information to be annotated by using a knowledge annotation method, obtaining a knowledge annotation result output by a pre-trained generative model in the set of annotation models; Among them, the multiple second annotation methods include at least one of the following: part-of-speech tagging, named entity tagging, syntax tagging, and mood tagging, and the second annotation model is a pre-trained language model.

7. The method according to claim 1, characterized in that After the M annotation models are used to respectively annotate the text information to be annotated using the N annotation methods to obtain a first group of annotation results corresponding to the text information to be annotated, the method further includes: Obtaining a second set of annotation results obtained by correcting the first set of annotation results; The second group of annotation results and the text information to be annotated are saved in a designated database, and the second group of annotation results and the text information to be annotated are used to train at least part of the annotation models in the group of annotation models, wherein the designated database is a database used to store the group of training data.

8. The method according to claim 7, characterized in that After obtaining the second group of annotation results obtained by correcting the first group of annotation results, the method further includes: The second group of annotation results and the text information to be annotated are displayed on an annotation display interface, wherein the second group of annotation results is a group of annotation results obtained by adjusting at least part of the annotation results in the first group of annotation results in response to the input adjustment instruction.

9. A text information annotation device, characterized in that: include: A first acquisition unit is used to acquire text information to be annotated, and acquire a target annotation instruction corresponding to the text information to be annotated, wherein the target annotation instruction is used to indicate that N annotation methods in a group of annotation methods are used to respectively annotate the text information to be annotated, and N is a positive integer greater than or equal to 1; A selection unit is used to select M annotation models corresponding to the target annotation instruction from a group of annotation models according to the target annotation instruction, wherein M is a positive integer greater than or equal to 1, and the annotation method adopted by each of the M annotation models is the N annotation methods in total, and the group of annotation models is a model trained using a group of training data, and the M annotation models include a chat generative pre-trained model ChatGPT; A labeling unit is used to use the M labeling models to adopt the N labeling methods to respectively label the text information to be labeled, so as to obtain a first group of labeling results corresponding to the text information to be labeled, wherein each of the M labeling models is used to label the text information to be labeled using at least one labeling method among the N labeling methods.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 8 when executed.

11. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 8 through the computer program.

Images