Data processing method based on large language model and related product

Abstract

The invention discloses a data processing method based on a large language model and a related product. The data processing method comprises the steps of collecting original text data; extracting a semantic tag of the original text data; generating a plurality of associated texts related to the semantic tag; and combining the semantic tag and the corresponding multiple associated texts to construct a semantic annotation data set. According to the method and the device, the semantic tag in the text data can be extracted, and the multiple associated texts related to the extracted semantic tag can be generated. Thereafter, a plurality of associated costs are combined with semantic tags, and a semantic annotation data set with rich and diversified contents can be constructed, so that the constructed semantic annotation data set can provide sufficient context information. Therefore, when data processing is carried out, the large language model can utilize rich context semantic information to solve problems caused by ambiguity of words, and the semantic offset problem caused by ambiguity of words in data enhancement is effectively solved.

Description

Technical Field

[0001] The present application generally relates to the technical field of artificial intelligence. More specifically, the present application relates to a data processing method based on a large language model and related products. Background Art

[0002] Large language models mainly refer to language models with very large parameter scales, usually reaching tens of billions or even hundreds of billions of parameters. The goal of large language models is to improve language learning and understanding capabilities by increasing model capacity, so as to enhance the effects of downstream natural language processing tasks. After obtaining a large amount of text data for pre-training, large language models can learn strong language representation capabilities and extract general language knowledge. This language knowledge can be applied to different natural language processing tasks, such as text classification, question-answering systems, dialogue systems, etc., to significantly improve the performance of these tasks.

[0003] In order to further enhance the language processing capabilities of large language models, the method of expanding the training data set is often used for optimization. The core idea is to generate new training samples based on the original training data through data enhancement technology, so that the model can learn on a wider range of data. However, the grammatical and semantic complexity of natural language makes it difficult to apply data enhancement technology in natural language processing tasks.

[0004] In view of this, there is an urgent need to provide a data processing solution based on a large language model to improve the accuracy of language understanding and processing by the language model. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a data processing solution based on a large language model in multiple aspects.

[0006] In a first aspect, the present application provides a data processing method based on a large language model, which includes: collecting original text data; extracting semantic tags from the original text data; generating multiple associated texts related to the semantic tags; and combining the semantic tags and the corresponding multiple associated texts to construct a semantically annotated dataset.

[0007] In one embodiment, generating multiple associated texts related to the semantic tag includes: creating a prompt box template based on the extracted semantic tag, wherein the prompt box includes a placeholder symbol at a position corresponding to the semantic tag; inputting the prompt box template and the original text data into the large language model to generate a first associated text; and iteratively replacing keywords in the prompt box to generate multiple associated texts.

[0008] In one embodiment, extracting the semantic tags of the original text data includes: preprocessing the original text data; inputting the preprocessed data into a large language model to obtain annotation information of the original text data; parsing the output annotation information; and extracting the semantic tags based on the parsed annotation information.

[0009] In one embodiment, the parsing the outputted annotation information includes: parsing the format of the annotation information to convert the annotation message into a lightweight data exchange format.

[0010] In one embodiment, extracting the semantic tags based on the parsed annotation information includes: obtaining the annotation grammatical structure of the annotation information; obtaining the annotation text segment and the annotation type according to the annotation grammatical structure; and directly extracting or generating corresponding semantic tags according to predetermined rules based on the semantic meaning of the annotation type and the keywords of the annotation text segment.

[0011] In one embodiment, the data processing method further includes: adjusting the network structure of the large language model according to the constructed semantic annotation dataset and the corresponding semantic tags.

[0012] In one embodiment, the large language model includes a Generative Pre-trained Transforme model and / or a Text Davinci model.

[0013] In one embodiment, the constructed semantically annotated dataset is used to train an open source open domain named entity recognition model.

[0014] In a second aspect, the present application provides an electronic device comprising: a processor; and a memory storing program instructions of a data processing method based on a large language model, wherein when the program instructions are executed by the processor, the device implements the method described in any one of the first aspects.

[0015] In a third aspect, the present application provides a computer-readable storage medium having stored thereon computer-readable instructions for a data processing method based on a large language model, wherein when the computer-readable instructions are executed by one or more processors, the method described in any one of the first aspects is implemented.

[0016] Through the data processing scheme based on the large language model provided by the above aspects and embodiments, the present application can extract semantic tags from text data and generate multiple associated texts related to the extracted semantic tags. Thereafter, by combining multiple association costs with semantic tags, a rich and diverse semantic annotation dataset can be constructed, and the semantic annotation dataset constructed in this way can provide sufficient contextual information. Therefore, when performing data processing, the large language model can use rich contextual semantic information to resolve the problems caused by the ambiguity of words, and effectively solve the semantic deviation problem caused by the ambiguity of words in data enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0018] Figure 1 An exemplary flow chart showing a data processing method based on a large language model according to some embodiments of the present disclosure;

[0019] Figure 2 An exemplary flow chart showing a method for generating multiple associated texts according to some embodiments of the present disclosure is shown;

[0020] Figure 3 An exemplary flow chart showing a method for extracting semantic tags according to some embodiments of the present disclosure is shown;

[0021] Figure 4 An exemplary flow chart showing a method for extracting semantic tags based on annotation information according to some embodiments of the present disclosure is shown;

[0022] Figure 5 An exemplary structural block diagram of an electronic device according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0024] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0025] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, the singular forms of "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0026] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0027] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0028] With the development of natural language processing technology, artificial intelligence plays an increasingly important role in complex tasks. However, efficient artificial intelligence models require a large amount of labeled data for training. To solve the data demand problem, data augmentation technology came into being. Its core idea is to generate new data based on the original data so that the model can learn on a wider distribution. Although it is widely used in the field of computer vision, due to the complexity of language syntax and semantics, data augmentation is more difficult in the field of natural language processing, especially in general, pan-domain, and unrestricted category named entity recognition models.

[0029] In view of this, the embodiment of the present application provides a data processing method based on a large language model, which extracts semantic tags from original text data and regenerates multiple associated texts related to the semantic tags to fully express the multiple meanings of the semantic tags. In addition, the generated new text and semantic tags are combined to construct a rich semantic annotation dataset, and then the enhanced semantic annotation dataset is used to train the large language model, which can effectively solve the ambiguity problem in data enhancement.

[0030] Figure 1An exemplary flow chart of a data processing method 100 based on a large language model according to some embodiments of the present disclosure is shown.

[0031] like Figure 1 As shown, the data processing method 100 of the disclosed embodiment may include step S101, at which raw text data is collected. In some embodiments, the raw text data can be obtained by direct manual input, web crawler collection, database query, etc. It should be noted that for large-scale training, the input is mainly automatically collected through programming scripts. Specifically, the characteristics of the training data are analyzed to determine the source of the data, such as online articles, user comments, etc., and then, according to the source of the data, a targeted crawler program is designed to automatically capture relevant content in a targeted manner.

[0032] Next, in step S102, the semantic tags of the original text data are extracted. The semantic tags are important semantic information parsed and extracted from the original text data, which can be concept tags, such as "name", "place name"; or other semantic role tags, such as "event subject", "event location", etc. In an implementation scenario, when processing data, the extracted semantic tags can be used to guide the generation of new semantic expressions to achieve more intelligent and original data enhancement results.

[0033] Furthermore, at step S103, multiple associated texts related to the semantic tags are generated. In an implementation scenario, the semantic tags can provide key information such as entity categories, relationship indications, and attribute prompts for generating associated texts. In addition, different tag combinations can generate a variety of different associated expressions. By constructing rich and diverse multiple associated texts, the scale of the existing knowledge graph can be expanded to cover more entities, so as to enrich the relationship expressions between entities, thereby enhancing the understanding ability of the large language model for grammar and pragmatics.

[0034] Then, at step S104, the semantic tags and the corresponding multiple associated texts are combined to construct a semantically annotated data set. In an implementation scenario, by combining the generated new text and the corresponding semantic tags, a high-quality semantically annotated data set can be constructed to help the model resolve the ambiguity of polysemous words and achieve more accurate and effective data enhancement. Among them, the semantically annotated data set includes original text, semantic tags, semantically annotated text, and knowledge graph. Specifically, semantically annotated text refers to text that has been marked with semantic components in the original text by semantic annotation on the basis of the original text. The knowledge graph refers to a knowledge network constructed based on semantic tags and multiple associated texts, which can reflect the entity nodes extracted from the original text, the relationship edges between entities, and the knowledge paths connecting entities.

[0035] If the original text is Zhang San goes to Beijing for a trip, the semantically annotated text is [Zhang San] name [go] event [Beijing] location [travel] event. Taking this annotated text as an example, different semantic tag combinations can generate the following associated expressions:

[0036] 1. [Zhang San] name [first time] time [go] event [Beijing] place [travel] event.

[0037] 2. [Zhang San] name [excited] status [go] event [Beijing] location [travel] event.

[0038] 3. [Zhang San] name [long planned] time [go] event [Beijing] place [travel] event.

[0039] 4. [Zhang San] name [and friend] name [went] to event [Beijing] location [travel] event.

[0040] Rich and diverse related texts can be used as augmented data to expand the semantic knowledge scope of the large language model, improve the semantic inference ability of the large language model, enable the large language model to more accurately generate new samples that are consistent with the original meaning, and effectively solve the semantic deviation problem caused by polysemy in data augmentation.

[0041] Furthermore, in some embodiments, the network structure of the large language model can be adjusted according to the constructed semantic annotation dataset and the corresponding semantic labels. Specifically, the adjustment can be performed by transfer learning, fine-tuning, and multi-task learning. In the above manner, the semantic annotation dataset can be effectively used to adjust the structure and parameters of the large language model, enhance the semantic understanding ability of the large language model, and improve the quality of semantic annotation.

[0042] In one implementation scenario, the constructed semantically annotated dataset is used to train an open-source open-domain named entity recognition model. Open-domain named entity recognition usually requires users to set the extracted named entity names themselves, which leads to ambiguity and insufficient semantic representation. The ambiguity problem means that the named entity names set by the user may express different meanings in different contexts, but the model cannot distinguish them, resulting in semantic errors. Insufficient semantics means that the named entity annotation does not express all the semantic information of the named entity, resulting in an incomplete understanding of the semantics by the model. The existence of these two problems reduces the effectiveness of the model in learning semantics, resulting in the inability to effectively improve the performance of open-domain named entity recognition.

[0043] The semantically annotated dataset containing rich associated text constructed by the present disclosure can enhance the model's learning of semantics from rich contexts, improve the model's ability to distinguish word meanings, and supplement the semantic information of named entities with multi-angle semantic descriptions to enhance the model's understanding of semantics. This helps to reduce semantic uncertainty in the open domain named entity recognition process. Furthermore, the quality, scale, and coverage of the semantically annotated dataset can be iteratively improved, and training can be repeated to obtain a better open domain entity recognition model.

[0044] It should be noted here that the large language model may include a Generative Pre-trained Transform (GPT) model and / or a Text Davinci model, in particular the GPT-3.5-Turbo model and the Text-Davinci-003 model. Among them, the GPT-3.5-Turbo model is a GPT model with 175 billion parameters, which has higher efficiency and faster processing speed. This makes it more practical in actual application scenarios. For example, in tasks such as natural language generation, text summarization, and machine translation, GPT-3.5-Turbo has demonstrated outstanding capabilities. The Text-Davinci-003 model is a large model that has been specifically optimized for natural language understanding and generation, containing 175 billion parameters. It simulates language structure through deep learning technology and can automatically generate human language to enrich the semantic expression of associated text. Applying these two types of representative large language models to the generation of associated text guided by prompt boxes can give full play to the rich semantic knowledge contained in the model, reasonably expand the text semantics, enhance the quality of generated text, and provide high-quality semantic data for subsequent open domain named entity recognition.

[0045] Figure 2 An exemplary flow chart of a method 200 for generating a plurality of associated texts according to some embodiments of the present disclosure is shown.

[0046] At step S201, a prompt box template is created according to the extracted semantic tags, wherein the prompt box includes a placeholder symbol corresponding to the semantic tag. In an implementation scenario, the prompt box template refers to a given template framework, and then the complete text output is completed according to the framework. Then, at step S202, the prompt box template and the original text data are input into the large language model to generate the first associated text.

[0047] If the original text is Zhang San going to Beijing for a trip, the prompt box text is [Zhang San] name [go] event [Beijing] location [travel] event, and the prompt box template is [] name [] event [] location [] event. The original text and the prompt box template are input into the large language model to generate the associated text as Zhang San name very excited to go to event Beijing location travel event or Zhang San happily went to Beijing travel event. It is understandable that the generated associated text can be selected with or without semantic tags. If the generated associated text is to continue to be used to provide semantic information, all semantic tags can be retained. In addition, some semantic tags can also be retained. Whether to retain the semantic tags can be determined based on the next use purpose of the associated text, which is not limited here.

[0048] At step S203, the keywords in the prompt box are iteratively replaced to generate multiple associated texts. In an implementation scenario, iteratively replacing keywords means keeping the structure of the prompt box template unchanged and only replacing keywords. For example, [Zhang San] name [go to] event [Beijing] location [travel] event, replace one keyword and input it into the large language model to generate [Li Si] name [fly to] event [Shanghai] location [business trip] event. Repeat this process until all keyword combinations are iteratively replaced. By replacing the names, events, places and events in the prompt box template, multiple associated texts are generated.

[0049] Iteratively replacing keywords in the prompt box template can generate multiple related texts that are similar but have semantic changes. This approach of obtaining enhanced data by combining keyword changes can improve the semantic learning ability of the model from multiple aspects and better enhance the semantic modeling ability of the model. Furthermore, for each original text, a large language model is used to generate multiple related expressions based on the extracted semantic tags, and then the semantic tags and related texts of all original texts are aggregated to form multiple semantic annotation data sets. Thereafter, the network structure of the large language model is adjusted through transfer learning using multiple semantic annotation data sets, so that the adjusted large language model has stronger adaptability to semantic annotation tasks, thereby achieving better results in open domain named entity recognition.

[0050] In a preferred embodiment, error cases of the above-mentioned associated texts can also be collected and extracted, and the collected error cases can be qualitatively and quantitatively analyzed to analyze the causes of the errors. Furthermore, semantic and grammatical constraints can be added according to the causes of the errors to reduce the generation of error cases. Through this analysis, correction and improvement of error cases, the quality of associated text generation can be better improved, the error rate can be reduced, and the semantic learning ability of the language model can be enhanced.

[0051] Figure 3 An exemplary flow chart of a method 300 for extracting semantic tags according to some embodiments of the present disclosure is shown.

[0052] At step S301, the original text data is preprocessed. In an implementation scenario, the specific method of preprocessing may include cleaning special symbols, part-of-speech tagging and sentence analysis. Specifically, it may include removing non-semantic information symbols in the original text data, such as web page codes, special formats, etc. Then attach the corresponding part of speech, such as verbs, nouns, etc. to each word in the original text data. Then analyze the grammatical structure and components of each sentence in the original text data, and clarify the grammatical relationships such as subject, predicate, and object. By preprocessing the original text, a good foundation can be laid for subsequent semantic analysis and annotation, and the accuracy of semantic tag extraction can be guaranteed.

[0053] Next, at step S302, the preprocessed data is input into the large language model to obtain the annotation information of the original text data. In an implementation scenario, the annotation information may include a semantic framework. A semantic framework is a structured semantic representation used to describe practices, actions, and the relationship between participants (people or things) related thereto. Semantic tags can be further determined based on the semantic framework. If the text data is "Zhang San goes to Beijing for a trip", the corresponding semantic framework is that someone is doing something in a certain place. Through semantic framework analysis, we can clarify the semantic structure of the text describing the event and extract key information for subsequent semantic analysis.

[0054] Furthermore, at step S302, the output annotation information needs to be parsed. In one embodiment, the purpose of the parsing is to convert the annotation results into a structured representation that can be better understood and processed by the computer for direct processing by the computer program. Specifically, the parsed annotation information can be represented in a computer-processable form such as a tree structure, a graph structure, a tuple, etc. In another embodiment, the annotation information can be format parsed to convert the annotation message into a lightweight data exchange format, more specifically, it can be converted into JSON (JavaScript Object Notation). While improving the convenience of computer processing, data coupling is reduced to facilitate the storage, transmission and sharing of annotation information.

[0055] Next, in step S303, semantic tags are extracted based on the parsed annotation information. In one implementation scenario, a formatting framework for semantic tags is first defined, and then the phrases representing the semantic tag portion are directly identified and extracted from the annotation information according to the defined formatting framework.

[0056] Figure 4 An exemplary flow chart of a method 400 for extracting semantic tags based on annotation information according to some embodiments of the present disclosure is shown.

[0057] At step S401, the annotated grammatical structure of the annotation information is obtained. In an implementation scenario, annotating the grammatical structure refers to annotating the syntactic structure of the text to mark the grammatical relationship between words and components in the sentence. It mainly includes part-of-speech tagging (verbs, nouns) and syntactic component tagging (subject, predicate, object), so as to parse the grammatical relationship of the sentence through the grammatical structure of the sentence, which is conducive to further semantic framework tagging. Syntactic structure tagging provides support at the syntactic level, marks the grammatical relationship between sentence components, and is the basis of natural language understanding. The semantic framework further abstracts deeper semantic information based on the syntactic structure, and the two complement each other to achieve the purpose of computers better understanding natural language.

[0058] At step S402, the annotated text segment and the annotation type are obtained according to the annotated grammatical structure. In an implementation scenario, annotating a text segment refers to extracting words or phrases that can represent specific semantic information from the original text and annotating them when annotating the syntactic structure. The annotation type refers to the semantic category used when annotating the text segment. If "Zhang San goes to Beijing for tourism" is annotated with parts of speech, it is Zhang San (noun) goes to (verb) Beijing (noun) travel (noun). If "Zhang San goes to Beijing for tourism" is annotated with syntactic components, it is Zhang San (subject) goes to (predicate) Beijing (place adverbial) travel (purpose adverbial). Among them, the annotated text segment is "Zhang San", which, as a subject, can be annotated as a person's name (annotation type); "Beijing" as a place adverbial can be annotated as a place name (annotation type); "Go to... for tourism" as a predicate and purpose adverbial can be annotated as an event (annotation type).

[0059] At step S403, the corresponding semantic tags are directly extracted or generated according to the semantic meaning of the annotation type and the keywords of the annotated text fragment through predetermined rules. In an implementation scenario, the semantic meaning of the annotation type refers to selecting the annotation type name that can reflect this layer of semantics according to the text semantics. For example: the text fragment "Zhang San", the annotation type "name"; the text fragment "travel to...", the annotation type "event". The annotated text fragment usually selects keywords or keyword groups in the text to annotate in order to fully extract semantic information. The keywords contain the core semantic information of the text fragment. For example, "Zhang San" is a keyword representing a name, and "Beijing" is a keyword representing a place. Preset rules such as name-generate "person" tag; place name-generate "place" tag; event-generate "event" tag. The final semantic annotation result is [person] Zhang San [place] Beijing [event] travel.

[0060] By applying preset rules to convert annotation information into structured semantic tags that contain semantic types and contents, the model can automatically parse the annotation information, generate unified semantic tags, achieve structured annotation of text, and perform semantic extraction.

[0061] When the method of the present application is used to process and enhance data, the following exemplary specific instructions can be used to guide the data enhancement process:

[0062] 1.There's a type of knowledge representation"\{\textbf{Operator}\}".Please generate 10examples about in JSON format with the following keys:\{\textbf{The concept names in operator one by one}\}

[0063] 2.Please describe the semantic meaning ofˋˋ\{Knowledge\}” in 10variousstyles.

[0064] Corresponding to the aforementioned application function implementation method embodiment, the present disclosure embodiment also provides a method as follows: Figure 5 Electronic equipment shown. Figure 5 An exemplary structural block diagram of an electronic device 500 according to some embodiments of the present disclosure is shown.

[0065] Figure 5 The electronic device 500 shown includes: a processor 510; and a memory 520, wherein the memory 520 stores executable program instructions. When the program instructions are executed by the processor 510, the electronic device implements any of the methods described above.

[0066] exist Figure 5 In the electronic device 500, only the components related to this embodiment are shown. Therefore, it is obvious to those skilled in the art that the electronic device 500 may also include Figure 5. The processor 510 can control the operation of the electronic device 500. For example, the processor 510 controls the operation of the electronic device 500 by executing a program stored in the memory 520 on the electronic device 500. The processor 510 can be implemented by a central processing unit (CPU), an application processor (AP), an artificial intelligence processor chip (IPU), etc. provided in the electronic device 500. However, the present disclosure is not limited to this. In this embodiment, the processor 510 can be implemented in any appropriate manner. For example, the processor 510 can take the form of a computer-readable medium, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc., such as a microprocessor or a processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor.

[0067] The memory 520 may be used to store hardware of various data and instructions processed in the electronic device 500. For example, the memory 520 may store processed data and data to be processed in the electronic device 500. The memory 520 may store data sets that have been processed or to be processed by the processor 510. In addition, the memory 520 may store applications, drivers, etc. to be driven by the electronic device 500. For example: the memory 520 may store various programs related to model building and constraint setting to be executed by the processor 510. The memory 520 may be a DRAM, but the present disclosure is not limited thereto. The memory 520 may include at least one of a volatile memory or a non-volatile memory. The non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a phase change RAM (PRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), a ferroelectric RAM (FRAM), etc. The volatile memory may include a dynamic RAM (DRAM), a static RAM (SRAM), a synchronous DRAM (SDRAM), a PRAM, an MRAM, an RRAM, a ferroelectric RAM (FeRAM), etc. In an embodiment, the memory 520 may include at least one of a hard disk drive (HDD), a solid state drive (SSD), a high-density flash memory (CF), a secure digital (SD) card, a micro secure digital (Micro-SD) card, a mini secure digital (Mini-SD) card, an extreme digital (XD) card, caches, or a memory stick.

[0068] In summary, the specific functions implemented by the memory 520 and the processor 510 of the electronic device 500 provided in the embodiments of this specification can be explained in comparison with the aforementioned embodiments in this specification, and can achieve the technical effects of the aforementioned embodiments, and will not be repeated here.

[0069] Furthermore, the present disclosure may also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which computer program instructions (or computer program, or computer instruction code) are stored. When the computer program instructions (or computer program, or computer instruction code) are executed by a processor of an electronic device (or electronic device, server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present disclosure.

[0070] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A data processing method based on a large language model, characterized in that: include: Collecting original text data; Extracting semantic tags from the original text data; Generate multiple associated texts related to the semantic tag; as well as The semantic labels and corresponding multiple associated texts are combined to construct a semantic annotation dataset.

2. The data processing method according to claim 1, characterized in that: The generating of a plurality of associated texts related to the semantic tag comprises: Creating a prompt box template according to the extracted semantic tag, wherein the prompt box includes a placeholder symbol at a position corresponding to the semantic tag; Inputting the prompt box template and the original text data into the large language model to generate a first associated text; and The keywords in the prompt box are iteratively replaced to generate a plurality of associated texts.

3. The data processing method according to claim 1, characterized in that: The extracting of semantic tags from the original text data comprises: Preprocessing the original text data; Inputting the preprocessed data into a large language model to obtain annotation information of the original text data; parsing the outputted annotation information; and The semantic tag is extracted based on the parsed annotation information.

4. The data processing method according to claim 3, characterized in that: The parsing of the outputted annotation information includes: The annotation information is format parsed to convert the annotation message into a lightweight data exchange format.

5. The data processing method according to claim 3, characterized in that: The extracting the semantic tag based on the parsed annotation information includes: Acquire the annotation grammatical structure of the annotation information; Acquire a marked text segment and a marked type according to the marked grammatical structure; and According to the semantic meaning of the annotation type and the keywords of the annotated text segment, corresponding semantic tags are directly extracted or generated through predetermined rules.

6. The data processing method according to claim 1, characterized in that: Also includes: Adjust the network structure of the large language model according to the constructed semantic annotation dataset and the corresponding semantic labels.

7. The data processing method according to claim 1, characterized in that: The large language model includes a Generative Pre-trained Transforme model and / or a Text Davinci model.

8. The data processing method according to claim 1, characterized in that: The constructed semantic annotation dataset is used to train an open source open domain named entity recognition model.

9. An electronic device, characterized in that: include: processor; A memory storing program instructions of a data processing method based on a large language model, wherein when the program instructions are executed by the processor, the device implements the data processing method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions of a data processing method based on a large language model are stored thereon, and when the computer-readable instructions are executed by one or more processors, the method described in any one of claims 1-8 is implemented.

Images