Method and device for automatically generating rich text tag and medium

Abstract

The invention discloses a method and device for automatically generating a rich text tag and a medium. The method comprises the following steps: acquiring a multi-source rich text data set from multiple preset ways; the rich text data set is input into a pre-trained language model, a text label set of the rich text data set is output, and the language model is obtained by integrating a pre-training model, a machine learning model and a statistical model; and optimizing and filtering the text label set to obtain a final text label of the rich text data set.

Description

Technical Field

[0001] The present invention relates to the field of text processing technology, and more particularly, to a method, device and medium for automatically generating rich text tags. Background Art

[0002] In actual work, we often encounter situations where we need to extract keywords from large amounts of text content. Manual processing often takes a lot of time, and some keywords may be missed. Therefore, how to quickly and accurately extract keywords from large amounts of text has become a technical problem that needs to be solved urgently. Summary of the Invention

[0003] In view of the deficiencies in the prior art, the present invention provides a method, device and medium for automatically generating rich text tags.

[0004] According to one aspect of the present invention, a method for automatically generating rich text tags is provided, comprising:

[0005] Acquire rich text datasets from multiple sources through various preset channels;

[0006] Input a rich text dataset into a pre-trained language model and output a text label set for the rich text dataset, where the language model is an integration of a pre-trained model, a machine learning model, and a statistical model;

[0007] The text label set is optimized and filtered to obtain the final text labels of the rich text dataset.

[0008] Optionally, the method further includes: performing a cleaning operation on the rich text data set to obtain a cleaned rich text data set;

[0009] Optionally, the language model training process is:

[0010] Leverage natural language processing technology to build pre-trained model architectures;

[0011] Pre-train the pre-trained model architecture using a preset specific keyword extraction task to obtain a pre-trained model;

[0012] The pre-trained model is integrated with the machine learning model and the statistical model to obtain a language model.

[0013] Optionally, the pre-trained model architecture is pre-trained using a preset specific keyword extraction task to obtain a pre-trained model, including:

[0014] The keyword extraction task and the named entity recognition task are combined to pre-train the pre-training model architecture to obtain a pre-training model, in which the attention mechanism is introduced into the pre-training model.

[0015] Optionally, the text label set is optimized and filtered to obtain the final text labels of the rich text dataset, including:

[0016] The text label set is filtered and sorted based on the label weight and frequency of occurrence to determine the final text label of the rich text dataset.

[0017] According to another aspect of the present invention, there is provided an apparatus for automatically generating rich text tags, comprising:

[0018] The acquisition module is used to obtain rich text datasets from multiple sources from multiple preset channels;

[0019] The output module is used to input the rich text dataset into a pre-trained language model and output the text label set of the rich text dataset, where the language model is obtained by integrating the pre-trained model, the machine learning model, and the statistical model;

[0020] The optimization module is used to optimize and filter the text label set to obtain the final text labels of the rich text dataset.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method according to any one of the above aspects of the present invention.

[0022] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; and the processor for reading the executable instructions from the memory and executing the instructions to implement the method described in any one of the above aspects of the present invention.

[0023] Therefore, the present invention utilizes a pre-trained model to integrate with a machine learning model and a statistical model to obtain a language model to extract keywords from rich text and analyze the rich text to obtain text labels for the rich text data, which can quickly generate prepared labels and save a lot of extraction time. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0025] Figure 1 1 is a flow chart of a method for automatically generating rich text tags provided by an exemplary embodiment of the present invention;

[0026] Figure 2 1 is a schematic structural diagram of an apparatus for automatically generating rich text tags provided by an exemplary embodiment of the present invention;

[0027] Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0028] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0029] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0030] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present invention are only used to distinguish different steps, devices or modules, and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0031] It should also be understood that, in the embodiments of the present invention, “a plurality of” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0032] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0033] In addition, the term "and / or" in this invention merely describes an association relationship between related objects, indicating that three possible relationships exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Furthermore, the character " / " in this invention generally indicates that the related objects are in an "or" relationship.

[0034] It should also be understood that the description of the various embodiments of the present invention focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced with each other. For the sake of brevity, they will not be described one by one.

[0035] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0036] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0037] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0038] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0039] Embodiments of the present invention can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate in conjunction with numerous other general-purpose or specialized computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with terminal devices, computer systems, servers, and other electronic devices include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above.

[0040] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system-executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media, including storage devices.

[0041] Exemplary Methods

[0042] Figure 1 This is a flow chart of a method for automatically generating rich text tags provided by an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the method 100 for automatically generating rich text tags includes the following steps:

[0043] Step 101, obtaining a multi-source rich text dataset from a plurality of preset pathways;

[0044] Step 102: Input the rich text dataset into a pre-trained language model and output a text label set of the rich text dataset, wherein the language model is obtained by integrating the pre-trained model, the machine learning model, and the statistical model;

[0045] Step 103 : Optimize and filter the text label set to obtain the final text labels of the rich text dataset.

[0046] Specifically, for large amounts of text content, manually extracting keywords is usually time-consuming, so AI is used to automatically crawl rich text and generate relevant tags. The specific implementation steps are as follows:

[0047] 1. Data preparation: Collect and organize datasets containing rich text. This data can come from various sources such as web pages, documents, and news. Ensure the quality and diversity of the dataset.

[0048] 2. Text preprocessing: Preprocess rich text, including removing HTML tags, cleaning special characters and spaces, etc., to obtain clean text data.

[0049] 3. Build a language model: Use natural language processing (NLP) technology to build a suitable language model. You can use pre-trained NLP models such as BERT and GPT, or train a custom model based on specific tasks:

[0050] (Details are as follows: Fine-tuning pre-trained model: Use a pre-trained model (such as BERT or GPT) as the initial model and perform fine-tuning on a specific keyword extraction task. Fine-tuning refers to further training the pre-trained model on a specific task to adapt it to the requirements of the specific task. Through fine-tuning, the model can better understand the input text and generate more accurate keywords.

[0051] Constructing the input sequence: Take the text containing the keywords to be extracted as the input sequence and process it according to the input requirements of the pre-trained model. For the BERT model, the input sequence typically needs to include special markers (such as CLS and SEP) and be segmented. For the GPT model, the input sequence can be continuous text. Ensure that the input sequence is encoded and segmented according to the model's requirements.

[0052] Multi-task learning: Pre-trained models are often trained on multiple tasks besides keyword extraction. Consider combining keyword extraction with other related tasks for joint training. For example, you can combine it with the named entity recognition (NER) task, allowing the model to simultaneously learn to extract keywords and recognize entities.

[0053] Introducing an attention mechanism: Pre-trained models typically incorporate an attention mechanism to focus on key parts of the input sequence. This attention mechanism can be further tuned and optimized to make it more suitable for keyword extraction tasks. For example, the attention weight associated with keywords can be increased to emphasize their importance.

[0054] Integrated model: You can integrate pre-trained models with other models to achieve better keyword extraction performance. For example, you can combine the output of a pre-trained model with a traditional statistical or machine learning model to improve the accuracy of keyword extraction through model fusion.

[0055] 4. Label generation: Using a trained language model, pre-processed rich text is fed into the model. The model will understand and analyze the text and generate relevant labels based on the context.

[0056] 5. Tag optimization and filtering: Optimize and filter the generated tags to ensure that they are relevant and meaningful to the text content. Tags can be filtered and sorted based on their weight, frequency of occurrence, or other metrics.

[0057] 6. Model training and optimization: Based on demand and feedback, the language model is iteratively trained and optimized to improve the accuracy and quality of generated labels.

[0058] 7. Verification and Evaluation: Use a portion of the labeled data to verify and evaluate the generated labels. By comparing them with manually annotated labels, evaluate the performance and accuracy of the model and make necessary adjustments and improvements.

[0059] 8. Deployment and Application: Deploy the trained model to real-world applications, such as web pages and document processing tools. Ensure the stability and performance of the model so that it can automatically crawl rich text and generate relevant tags in real-world scenarios.

[0060] Therefore, the present invention utilizes a pre-trained model to integrate with a machine learning model and a statistical model to obtain a language model to extract keywords from rich text and analyze the rich text to obtain text labels for the rich text data, which can quickly generate prepared labels and save a lot of extraction time.

[0061] Exemplary devices

[0062] Figure 2 FIG is a schematic diagram of a structure of an apparatus for automatically generating rich text tags provided by an exemplary embodiment of the present invention. Figure 2 As shown, the apparatus 200 includes:

[0063] An acquisition module 210 is configured to acquire a multi-source rich text dataset from a plurality of preset pathways;

[0064] Output module 220, configured to input a rich text dataset into a pre-trained language model and output a text tag set of the rich text dataset, wherein the language model is obtained by integrating a pre-trained model, a machine learning model, and a statistical model;

[0065] The optimization module 230 is used to optimize and filter the text label set to obtain the final text labels of the rich text data set.

[0066] Optionally, the apparatus 200 further includes: a cleaning module, configured to perform a cleaning operation on the rich text data set to obtain a cleaned rich text data set;

[0067] Optionally, the language model training process is:

[0068] Leverage natural language processing technology to build pre-trained model architectures;

[0069] Pre-train the pre-trained model architecture using a preset specific keyword extraction task to obtain a pre-trained model;

[0070] The pre-trained model is integrated with the machine learning model and the statistical model to obtain a language model.

[0071] Optionally, the pre-trained model architecture is pre-trained using a preset specific keyword extraction task to obtain a pre-trained model, including:

[0072] The keyword extraction task and the named entity recognition task are combined to pre-train the pre-training model architecture to obtain a pre-training model, in which the attention mechanism is introduced into the pre-training model.

[0073] Optionally, the optimization module 230 includes:

[0074] The determination submodule is used to filter and sort the text label set based on the label weight and frequency of occurrence, and determine the final text label of the rich text dataset.

[0075] Exemplary electronic devices

[0076] Figure 3 This is the structure of an electronic device provided by an exemplary embodiment of the present invention. Figure 3 As shown, the electronic device 30 includes one or more processors 31 and a memory 32 .

[0077] The processor 31 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0078] The memory 32 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 31 may execute the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above and / or other desired functions. In one example, the electronic device may further include: an input device 33 and an output device 34, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0079] In addition, the input device 33 may also include, for example, a keyboard, a mouse, and the like.

[0080] The output device 34 can output various information to the outside. The output device 34 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto.

[0081] Of course, to simplify, Figure 3 Only some of the components related to the present invention in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0082] Exemplary computer program products and computer-readable storage media

[0083] In addition to the above-mentioned methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the method according to various embodiments of the present invention described in the above "Exemplary Method" section of this specification.

[0084] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0085] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method for information mining of historical change records according to various embodiments of the present invention described in the above "Exemplary Method" section of this specification.

[0086] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, system or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0087] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0088] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. For system embodiments, since they largely correspond to method embodiments, their description is relatively simple. For relevant parts, references to the description of the method embodiments are sufficient.

[0089] The block diagrams of the devices, systems, equipment, and systems involved in the present invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, systems, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "including," "comprising," "having," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0090] The method and system of the present invention may be implemented in many ways. For example, the method and system of the present invention may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above sequence of steps for the method is for illustration only, and the steps of the method of the present invention are not limited to the sequence specifically described above, unless otherwise specified. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers recording media that store programs for executing the method according to the present invention.

[0091] It should also be noted that, in the system, device and method of the present invention, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. The above description of the disclosed aspects is provided to enable any technician in this field to make or use the present invention. Various modifications to these aspects will be very obvious to those skilled in the art, and the general principles defined here can be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown here, but according to the widest scope consistent with the principles disclosed here and novel features.

[0092] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present invention to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for automatically generating rich text tags, characterized in that: include: Obtain rich text datasets from multiple sources from a variety of preset channels; Inputting the rich text data set into a pre-trained language model, and outputting a text tag set of the rich text data set, wherein the language model is obtained by integrating a pre-trained model, a machine learning model, and a statistical model; The text label set is optimized and filtered to obtain final text labels of the rich text data set.

2. The method according to claim 1, characterized in that Also includes: A cleaning operation is performed on the rich text data set to obtain a cleaned rich text data set.

3. The method according to claim 1, characterized in that The training process of the language model is: Use natural language processing technology to build a pre-trained model architecture; Pre-training the pre-trained model architecture by using a preset specific keyword extraction task to obtain a pre-trained model; The pre-trained model, the machine learning model and the statistical model are integrated to obtain the language model.

4. The method according to claim 3, characterized in that The pre-trained model architecture is pre-trained using a preset specific keyword extraction task to obtain a pre-trained model, including: The keyword extraction task is combined with the named entity recognition task to pre-train the pre-trained model architecture to obtain the pre-trained model, wherein an attention mechanism is introduced into the pre-trained model.

5. The method according to claim 1, characterized in that: The text label set is optimized and filtered to obtain the final text label of the rich text data set, including: The text label set is screened and sorted based on the weight and occurrence frequency of the label to determine the final text label of the rich text data set.

6. A device for automatically generating rich text tags, characterized in that: include: The acquisition module is used to obtain rich text data sets from multiple sources from multiple preset channels; An output module, used to input the rich text data set into a pre-trained language model, and output a text tag set of the rich text data set, wherein the language model is obtained by integrating a pre-trained model, a machine learning model, and a statistical model; The optimization module is used to optimize and filter the text label set to obtain the final text label of the rich text data set.

7. The device according to claim 6, characterized in that Also includes: The cleaning module is used to perform a cleaning operation on the rich text data set to obtain a cleaned rich text data set.

8. The device according to claim 6, characterized in that The training process of the language model is: Use natural language processing technology to build a pre-trained model architecture; Pre-training the pre-trained model architecture by using a preset specific keyword extraction task to obtain a pre-trained model; The pre-trained model, the machine learning model and the statistical model are integrated to obtain the language model.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 5.

10. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1 to 5.

Images