Keyword extraction method and device for vertical field text, equipment and medium

By training and sequence annotation of the initial pre-trained language model, the domain keyword extraction model is obtained, which solves the problem of low accuracy in vertical field text extraction and realizes accurate keyword extraction of vertical field text.

CN118966207AInactive Publication Date: 2024-11-15ATHENAEYES CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411434814.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-15
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, in the keyword extraction of vertical domain texts, the extraction accuracy is low due to factors such as high similarity between unstructured characteristics and domain feature texts.

Method used

By obtaining sample data, the initial pre-trained language model is trained, and the basic pre-trained language model is obtained, and sequence annotation is performed to obtain the domain keyword extraction model. This model is used to extract and sort the to-process text and select the preset number keywords as target keyword collection.

Benefits of technology

The accuracy of vertical field text keyword extraction is improved, and accurate keyword extraction of specific field text is achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966207A_ABST
    Figure CN118966207A_ABST
Patent Text Reader

Abstract

The invention discloses a keyword extraction method and device for a vertical field text, equipment and a medium. The method comprises the following steps: acquiring sample data; training the initial pre-training language model based on the sample data to obtain a basic pre-training language model; performing sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information, and training the basic pre-training language model by adopting the sequence labeling data set to obtain a domain keyword extraction model; obtaining a to-be-processed field text, and inputting the to-be-processed field text into the field keyword extraction model for candidate keyword extraction to obtain a field candidate keyword set; sorting the keywords in the field candidate keyword set to obtain a sorting result; and based on the sorting result, sequentially selecting a preset number of keywords from front to back to obtain a target keyword set. By adopting the method and the device, the accuracy of vertical field keyword extraction can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of text data processing, and in particular to a keyword extraction method, device, computer equipment and medium for vertical field text. Background Art

[0002] As humans enter the era of mobile Internet and cloud computing, artificial intelligence represented by robots will play a vital role in the manufacturing industry. People who are freed from heavy production will pay more attention to their own needs. Entertainment, medical care, environmental protection, education, sports, etc. will usher in a golden period of development. Vertical fields refer to the provision of specific services to limited groups, including entertainment, medical care, environmental protection, education, sports and other industries. Vertical field text is the text involved in the vertical field. Compared with non-vertical fields, vertical field text focuses more on exclusive fields in terms of text representation, feature extraction and classification methods. Based on this, vertical field text keyword extraction, vertical search engine, information classification and other applications can provide professional, refined and in-depth information and related services for specific users or specific needs.

[0003] The vertical field text has the attributes of a specific field, which makes the text expression closely related to the application scenario. On the one hand, the unstructured nature of the text itself brings certain difficulties to the text keyword extraction; on the other hand, the text that can represent the characteristics of the field in a specific scenario has a high similarity, and the application service requires more accurate keyword extraction and text classification methods, so the requirements for text keyword extraction are higher than those for general generalized fields.

[0004] Among the current implementations, some are to extract keywords by means of label weights (e.g., keyword extraction based on the PageRank algorithm in graph theory), some are to extract keywords by term frequency-inverse document frequency (TF-IDF), some are to extract keywords by Latent Dirichlet Allocation (LDA) topic model, and some are to extract keywords by Rapid Automatic keyword extraction (RAKE) algorithm. In the process of implementing the present invention, the inventors realized that the existing methods have at least the following technical problems: Although the existing methods can achieve the expected results after training under normal circumstances, the accuracy of keyword extraction is low due to the unstructured characteristics of the vertical field text itself and the high similarity between the texts representing the field characteristics. Summary of the invention

[0005] The embodiments of the present invention provide a method, an apparatus, a computer device and a storage medium for extracting keywords from vertical field texts, so as to improve the accuracy of keyword extraction from vertical field texts.

[0006] In order to solve the above technical problems, the embodiment of the present application provides a keyword extraction method for vertical field text, including: Acquire sample data, wherein the sample data is text data in a target vertical field; Training the initial pre-trained language model based on the sample data to obtain a basic pre-trained language model; Performing sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information, and using the sequence labeling data set to train the basic pre-trained language model to obtain a domain keyword extraction model; Acquire a domain text to be processed, and input the domain text to be processed into the domain keyword extraction model to extract candidate keywords, so as to obtain a domain candidate keyword set; The keywords in the field candidate keyword set are sorted to obtain a sorting result; and a preset number of keywords are selected from the front to the back in sequence based on the sorting result to obtain a target keyword set.

[0007] Optionally, the acquiring sample data includes: Collect text datasets in the target vertical field as initial data; Performing standardization processing on the initial data to obtain standardized data; According to a preset division method, the standardized data is divided into text pre-training data X1 and fine-tuning training data X2, and the text pre-training data X1 and the fine-tuning training data X2 are used as the sample data.

[0008] Optionally, the training the initial pre-trained language model based on the sample data to obtain a basic pre-trained language model includes: Get the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in the above table are used as first candidate keywords, and all the first candidate keywords constitute a first candidate word set; Using an unsupervised keyword extraction method to expand the first candidate word set to obtain a second candidate word set; Each text in the text pre-training data X1 is used as an original text, and positive samples and negative samples are generated based on the original text and the keywords in the second candidate word set, and the original text, positive samples and negative samples are used as pre-training corpus; The initial pre-trained language model is trained using the pre-trained corpus to obtain a basic pre-trained language model, wherein the initial pre-trained language model is a masked language model.

[0009] Optionally, the generating positive samples and negative samples based on the original text and the keywords in the second candidate word set, and using the original text, the positive samples and the negative samples as pre-training corpus includes: For each original text, obtaining a keyword corresponding to the original text in the second candidate word set as a target keyword; A special mask is used to mask and replace the target keyword, and the masked and replaced text is used as a positive sample corresponding to the original text; A special mask is used to replace words other than the target keyword in the original text, and the masked and replaced text is used as a negative sample corresponding to the original text.

[0010] Optionally, performing sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information includes: Get the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in the above are used as third candidate keywords, and all the third candidate keywords constitute a third candidate word set; A prompt text for the keyword extraction task is constructed for each candidate keyword in each third candidate word set, and a large-scale pre-trained language model is used to judge and screen each candidate keyword to obtain a high-quality candidate keyword set as the sequence annotation dataset.

[0011] In order to solve the above technical problems, the embodiment of the present application further provides a keyword extraction device for vertical field text, including: An acquisition module, used to acquire sample data, wherein the sample data is text data in a target vertical field; A training module, used to train the initial pre-trained language model based on the sample data to obtain a basic pre-trained language model; A labeling module, used to perform sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information, and use the sequence labeling data set to train the basic pre-trained language model to obtain a domain keyword extraction model; An extraction module is used to obtain a domain text to be processed, and input the domain text to be processed into the domain keyword extraction model to extract candidate keywords, thereby obtaining a domain candidate keyword set; The screening module is used to sort the keywords in the candidate keyword set of the field to obtain a sorting result; and select a preset number of keywords from the front to the back based on the sorting result to obtain a target keyword set.

[0012] Optionally, the acquisition module includes: The collection unit is used to collect text datasets in the target vertical field as initial data; A standardization unit, used for performing standardization processing on the initial data to obtain standardized data; The division unit is used to divide the standardized data into text pre-training data X1 and fine-tuning training data X2 according to a preset division method, and use the text pre-training data X1 and the fine-tuning training data X2 as the sample data.

[0013] Optionally, the training module includes: The first matching unit is used to obtain the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in the above table are used as first candidate keywords, and all the first candidate keywords constitute a first candidate word set; An expansion unit, configured to expand the first candidate word set by using an unsupervised keyword extraction method to obtain a second candidate word set; A pre-training corpus determination unit, used to take each text in the text pre-training data X1 as an original text, and generate positive samples and negative samples based on the original text and the keywords in the second candidate word set, and take the original text, positive samples and negative samples as pre-training corpus; A training unit is used to train the initial pre-trained language model using the pre-training corpus to obtain a basic pre-trained language model, wherein the initial pre-trained language model is a masked language model.

[0014] Optionally, the pre-training corpus determining unit includes: A screening subunit, configured to obtain, for each original text, a keyword corresponding to the original text in the second candidate word set as a target keyword; The masking subunit is used to mask and replace the target keyword with a special mask, and use the masked and replaced text as a positive sample corresponding to the original text; The replacement subunit is used to replace the words other than the target keyword in the original text with a special mask, and use the masked and replaced text as a negative sample corresponding to the original text.

[0015] Optionally, the marking module includes: The second matching unit is used to obtain the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in the above are used as third candidate keywords, and all the third candidate keywords constitute a third candidate word set; The screening unit is used to construct a prompt text for the keyword extraction task for each candidate keyword in each third candidate word set, and use a large-scale pre-trained language model to judge and screen each candidate keyword to obtain a high-quality candidate keyword set as the sequence annotation data set.

[0016] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned keyword extraction method for vertical field text when executing the computer program.

[0017] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the keyword extraction method for the above vertical field text.

[0018] The keyword extraction method, device, computer equipment and storage medium of the vertical field text provided by the embodiment of the present invention are as follows: obtaining sample data, wherein the sample data is text data of the target vertical field; training the initial pre-trained language model based on the sample data to obtain the basic pre-trained language model; performing sequence annotation on the sample data to obtain a sequence annotation data set with field keyword sequence label information, and using the sequence annotation data set to train the basic pre-trained language model to obtain a field keyword extraction model; obtaining the field text to be processed, and inputting the field text to be processed into the field keyword extraction model to extract candidate keywords to obtain a field candidate keyword set; sorting the keywords in the field candidate keyword set to obtain a sorting result; and selecting a preset number of keywords from the front to the back based on the sorting result to obtain a target keyword set. Accurate extraction of vertical field keywords is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0020] Figure 1 is an exemplary system architecture diagram to which the present application may be applied; Figure 2 It is a flowchart of an embodiment of a method for extracting keywords from vertical field texts of the present application; Figure 3 It is a structural schematic diagram of an embodiment of a keyword extraction device for vertical field text according to the present application; Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0022] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 ,like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0025] Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages, etc.

[0026] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0027] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0028] It should be noted that the keyword extraction method for vertical field text provided in the embodiment of the present application is executed by a server, and accordingly, the keyword extraction device for vertical field text is arranged in the server.

[0029] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. The terminal devices 101, 102, 103 in the embodiment of the present application may specifically correspond to application systems in actual production.

[0030] See also Figure 2 , Figure 2 A keyword extraction method for vertical field text provided by an embodiment of the present invention is shown. The method is applied in Figure 1 The server in is used as an example to illustrate the details as follows: S201: Acquire sample data, where the sample data is text data in a target vertical field.

[0031] In a specific optional implementation, in step S201, obtaining sample data includes: Collect text datasets in the target vertical field as initial data; Standardize the initial data to obtain standardized data; According to a preset division method, the standardized data is divided into text pre-training data X1 and fine-tuning training data X2, and the text pre-training data X1 and the fine-tuning training data X2 are used as sample data.

[0032] Specifically, after extensive research, we collected public Chinese text data in specific fields (such as medical and health fields), analyzed the format specifications of each text data, formulated a unified data format, and subsequently formed text pre-training data with a unified format. , fine-tune the training data . Use Chinese wiki and public domain dictionaries to build domain vocabulary for specific fields .

[0033] S202: Train the initial pre-trained language model based on the sample data to obtain a basic pre-trained language model.

[0034] In a specific optional implementation, in step S202, the initial pre-trained language model is trained based on the sample data to obtain a basic pre-trained language model including: Get the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in are taken as the first candidate keywords, and all the first candidate keywords constitute the first candidate word set; An unsupervised keyword extraction method is used to expand the first candidate word set to obtain a second candidate word set; Each text in the text pre-training data X1 is taken as an original text, and positive samples and negative samples are generated based on the original text and the keywords in the second candidate word set, and the original text, positive samples and negative samples are taken as pre-training corpus; The initial pre-trained language model is trained using the pre-trained corpus to obtain a basic pre-trained language model, wherein the initial pre-trained language model is a masked language model.

[0035] In a specific optional implementation, generating positive samples and negative samples based on the original text and the keywords in the second candidate word set, and using the original text, the positive samples and the negative samples as pre-training corpus includes: For each original text, obtain the keyword corresponding to the original text in the second candidate word set as the target keyword; A special mask is used to cover and replace the target keyword, and the replaced text is used as the positive sample corresponding to the original text; A special mask is used to replace words other than the target keywords in the original text, and the masked and replaced text is used as the negative sample corresponding to the original text.

[0036] Specifically, BERT (Bidirectional Encoder Representation from Transformers) and other pre-trained language models based on Transformers can effectively model and learn the semantics and structure of domain text through self-supervised learning methods, thereby obtaining dynamic and high-quality text semantic representation vectors for downstream tasks. By adding explicit modeling of domain keywords in the self-supervised language model, high-quality domain text semantic representation vectors can be obtained, thereby improving the effect of domain text keyword extraction.

[0037] In this embodiment, the data obtained in S201 is used and defined domain lexicon , obtained by text matching Each text in the domain vocabulary The pseudo candidate keyword set in At the same time, the candidate keyword set is expanded using existing unsupervised keyword extraction methods (such as TextRank, SIF, TF-IDF, etc.) ; Then the pre-training data Processing is performed to obtain positive and negative samples for modeling domain keywords: the definition is as follows: Any one of them is used for pre-training text, and the first candidate word (pseudo candidate keyword) is collected The text in which the keywords are replaced by a special mask ([MASK]) is a positive sample, and the text in which the special mask is replaced by non-keywords is a negative sample, as shown below: The KD (Keyword Distance) pre-training task is defined, and the original text is used as the distance origin. The ternary loss function is used to describe the distance between the positive and negative samples and the original samples, reducing the distance between the positive samples and the original text, and increasing the distance between the negative samples and the original text.

[0038] In a specific positive example, the original text is "Many people do not want to suffer from epilepsy, but the treatment of epilepsy patients is still related to many factors." After special mask replacement, the text is "Many people do not want to suffer from [MASK][MASK][MASK], but the treatment of [MASK][MASK] patients is still related to many factors." Among them, the pseudo keywords are epilepsy and epilepsy.

[0039] In a specific negative example, the original text is "Graduated from the Department of Traditional Chinese Medicine of Nanjing University of Traditional Chinese Medicine", and the text after special mask replacement is "[MASK][MASK] in the Department of Traditional Chinese Medicine of Nanjing University of Traditional Chinese Medicine", in which there are no pseudo keywords.

[0040] Optionally, the pre-training task in this embodiment is MLM (Masked Language Model) + KD (Keyword Distance).

[0041] S203: Perform sequence labeling on the sample data to obtain a sequence labeling dataset with domain keyword sequence label information, and use the sequence labeling dataset to train a basic pre-trained language model to obtain a domain keyword extraction model.

[0042] In a specific optional implementation, step S203, performing sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information includes: Get the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in are used as the third candidate keywords, and all the third candidate keywords constitute the third candidate word set; For each candidate keyword in each third candidate word set, a prompt text for the keyword extraction task is constructed, and a large-scale pre-trained language model is used to judge and screen each candidate keyword to obtain a high-quality candidate keyword set as a sequence annotation dataset.

[0043] In a specific example, first align the fine-tuning dataset collected by S1 Automatically annotate keywords in Chinese text to obtain a sequence annotation dataset with domain keyword BIO tag information The data label format is shown in Table 1 below: Table 1

[0044] The specific automatic annotation process is as follows: 1. Domain vocabulary obtained during use Obtain fine-tuning dataset using text matching The set of pseudo candidate keywords for each text in .

[0045] 2. Use the prompt text template to create a pseudo candidate keyword set for each text Each candidate keyword in the task constructs a prompt for the keyword extraction task, and uses LLM (large-scale pre-trained language models) to judge and screen each candidate keyword to obtain a high-quality candidate keyword set. , build a sequence annotation dataset Used for subsequent sequence annotation fine-tuning tasks. Examples of prompt text are shown in Table 2 below: Table 2

[0046] 3. Use automatically labeled sequence annotation datasets Perform supervised fine-tuning on the pre-trained language model M obtained in S2 to obtain a fine-tuned domain keyword extraction model .

[0047] S204: Obtain the domain text to be processed, and input the domain text to be processed into the domain keyword extraction model to extract candidate keywords, so as to obtain a set of domain candidate keywords.

[0048] S205: Sort the keywords in the field candidate keyword set to obtain a sorting result; and select a preset number of keywords from the front to the back based on the sorting result to obtain a target keyword set.

[0049] Specifically, in order to highlight the importance of each keyword in the domain text T, it is necessary to obtain the domain keyword set in S4 Sort the keywords and select the top-k keywords to output as the final keyword set .

[0050] The specific steps are as follows: Keyword collection for the field Each keyword in , get the corresponding special mask overlay text , use the pre-trained language model in S2 to obtain text The corresponding text semantic vector The text semantic vector corresponding to the text T .

[0051] Calculate text semantic vector With text semantic vector Semantic similarity between (For example: cosine similarity between vectors, vector edit distance, etc.).

[0052] Calculate text semantic vector Corresponding distance penalty coefficient , defined as follows: .

[0053] In the above formula Indicates keywords The index value of the first occurrence in the text T, length represents the keyword The length of the string, is a hyperparameter used to adjust the influence of position information. The larger the value of , the smaller the influence of location information in calculating the distance penalty coefficient, and vice versa.

[0054] Count each keyword Importance score , defined as follows: .

[0055] in accordance with Keyword set for candidate fields Sort in descending order and select the top-k keywords as the final field keyword set .

[0056] In this embodiment, sample data is obtained, wherein the sample data is text data of the target vertical field; the initial pre-trained language model is trained based on the sample data to obtain a basic pre-trained language model; the sample data is sequence labeled to obtain a sequence labeling data set with field keyword sequence label information, and the sequence labeling data set is used to train the basic pre-trained language model to obtain a field keyword extraction model; the field text to be processed is obtained, and the field text to be processed is input into the field keyword extraction model to extract candidate keywords to obtain a field candidate keyword set; the keywords in the field candidate keyword set are sorted to obtain a sorting result; and a preset number of keywords are selected from the front to the back based on the sorting result to obtain a target keyword set. Accurate extraction of vertical field keywords is achieved.

[0057] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0058] Figure 3 The principle block diagram of the keyword extraction device of the vertical field text corresponding to the keyword extraction method of the vertical field text in the above embodiment is shown. Figure 3 As shown, the keyword extraction device for vertical field text includes a data separation module 31, a missing simulation module 32, a local repair module 33, a secondary repair module 34 and a result summary module 35. The detailed description of each functional module is as follows: An acquisition module 31 is used to acquire sample data, wherein the sample data is text data in a target vertical field; A training module 32, used to train the initial pre-trained language model based on sample data to obtain a basic pre-trained language model; The labeling module 33 is used to perform sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information, and use the sequence labeling data set to train the basic pre-trained language model to obtain a domain keyword extraction model; An extraction module 34 is used to obtain the domain text to be processed, and input the domain text to be processed into a domain keyword extraction model to extract candidate keywords, thereby obtaining a set of domain candidate keywords; The screening module 35 is used to sort the keywords in the field candidate keyword set to obtain a sorting result; and select a preset number of keywords from the front to the back based on the sorting result to obtain a target keyword set.

[0059] Optionally, the acquisition module 31 includes: The collection unit is used to collect text datasets in the target vertical field as initial data; A standardization unit is used to perform standardization processing on the initial data to obtain standardized data; The division unit is used to divide the standardized data into text pre-training data X1 and fine-tuning training data X2 according to a preset division method, and use the text pre-training data X1 and the fine-tuning training data X2 as sample data.

[0060] Optionally, the training module 32 includes: The first matching unit is used to obtain the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in are taken as the first candidate keywords, and all the first candidate keywords constitute the first candidate word set; An expansion unit, used to expand the first candidate word set by using an unsupervised keyword extraction method to obtain a second candidate word set; A pre-training corpus determination unit is used to take each text in the text pre-training data X1 as an original text, and generate positive samples and negative samples based on the original text and the keywords in the second candidate word set, and take the original text, positive samples and negative samples as pre-training corpus; The training unit is used to train the initial pre-trained language model using the pre-training corpus to obtain a basic pre-trained language model, wherein the initial pre-trained language model is a masked language model.

[0061] Optionally, the pre-training corpus determining unit includes: A screening subunit is used to obtain, for each original text, a keyword corresponding to the original text in the second candidate word set as a target keyword; The occlusion subunit is used to occlude and replace the target keyword with a special mask, and use the occluded and replaced text as the positive sample corresponding to the original text; The replacement subunit is used to replace words other than the target keywords in the original text with a special mask, and use the masked and replaced text as the negative sample corresponding to the original text.

[0062] Optionally, the marking module 33 includes: The second matching unit is used to obtain the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in are used as the third candidate keywords, and all the third candidate keywords constitute the third candidate word set; The screening unit is used to construct a prompt text for the keyword extraction task for each candidate keyword in each third candidate word set, and use a large-scale pre-trained language model to judge and screen each candidate keyword to obtain a high-quality candidate keyword set as a sequence annotation data set.

[0063] For the specific limitations on the keyword extraction device for vertical field text, please refer to the limitations on the keyword extraction method for vertical field text above, which will not be repeated here. Each module in the above-mentioned keyword extraction device for vertical field text can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0064] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0065] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components connected to the memory 41, the processor 42, and the network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0066] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0067] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as the program code of the keyword extraction method of the vertical field text, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0068] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the program code stored in the memory 41 or process data, such as running the program code of the keyword extraction method of the vertical field text.

[0069] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0070] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to perform the steps of the keyword extraction method for vertical field text as described above.

[0071] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0072] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A keyword extraction method for vertical field text, characterized in that: include: Acquire sample data, wherein the sample data is text data in a target vertical field; Training the initial pre-trained language model based on the sample data to obtain a basic pre-trained language model; Performing sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information, and using the sequence labeling data set to train the basic pre-trained language model to obtain a domain keyword extraction model; Acquire a domain text to be processed, and input the domain text to be processed into the domain keyword extraction model to extract candidate keywords, so as to obtain a domain candidate keyword set; The keywords in the field candidate keyword set are sorted to obtain a sorting result; and a preset number of keywords are selected from the front to the back in sequence based on the sorting result to obtain a target keyword set.

2. The keyword extraction method for vertical field text according to claim 1, characterized in that: The obtaining of sample data comprises: Collect text datasets in the target vertical field as initial data; Performing standardization processing on the initial data to obtain standardized data; According to a preset division method, the standardized data is divided into text pre-training data X1 and fine-tuning training data X2, and the text pre-training data X1 and the fine-tuning training data X2 are used as the sample data.

3. The keyword extraction method for vertical field text according to claim 2, characterized in that: The training of the initial pre-trained language model based on the sample data to obtain a basic pre-trained language model comprises: Get the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in the above table are used as first candidate keywords, and all the first candidate keywords constitute a first candidate word set; Using an unsupervised keyword extraction method to expand the first candidate word set to obtain a second candidate word set; Each text in the text pre-training data X1 is used as an original text, and positive samples and negative samples are generated based on the original text and the keywords in the second candidate word set, and the original text, positive samples and negative samples are used as pre-training corpus; The initial pre-trained language model is trained using the pre-trained corpus to obtain a basic pre-trained language model, wherein the initial pre-trained language model is a masked language model.

4. The keyword extraction method for vertical field text according to claim 3 is characterized in that: The generating positive samples and negative samples based on the original text and the keywords in the second candidate word set, and using the original text, the positive samples and the negative samples as pre-training corpus comprises: For each original text, obtaining a keyword corresponding to the original text in the second candidate word set as a target keyword; A special mask is used to mask and replace the target keyword, and the masked and replaced text is used as a positive sample corresponding to the original text; A special mask is used to replace words other than the target keyword in the original text, and the masked and replaced text is used as a negative sample corresponding to the original text.

5. The keyword extraction method for vertical field text according to claim 1, characterized in that: The performing sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information includes: Get the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in the above are used as third candidate keywords, and all the third candidate keywords constitute a third candidate word set; A prompt text for the keyword extraction task is constructed for each candidate keyword in each third candidate word set, and a large-scale pre-trained language model is used to judge and screen each candidate keyword to obtain a high-quality candidate keyword set as the sequence annotation dataset.

6. A keyword extraction device for vertical field text, characterized in that: include: An acquisition module, used to acquire sample data, wherein the sample data is text data in a target vertical field; A training module, used to train the initial pre-trained language model based on the sample data to obtain a basic pre-trained language model; A labeling module is used to perform sequence labeling on the sample data to obtain a sequence labeling data set with domain keyword sequence label information, and use the sequence labeling data set to train the basic pre-trained language model to obtain a domain keyword extraction model; An extraction module is used to obtain a domain text to be processed, and input the domain text to be processed into the domain keyword extraction model to extract candidate keywords, thereby obtaining a domain candidate keyword set; The screening module is used to sort the keywords in the candidate keyword set of the field to obtain a sorting result; and select a preset number of keywords from the front to the back based on the sorting result to obtain a target keyword set.

7. The keyword extraction device for vertical field text according to claim 6, characterized in that: The acquisition module comprises: The collection unit is used to collect text datasets in the target vertical field as initial data; A standardization unit, used for performing standardization processing on the initial data to obtain standardized data; The division unit is used to divide the standardized data into text pre-training data X1 and fine-tuning training data X2 according to a preset division method, and use the text pre-training data X1 and the fine-tuning training data X2 as the sample data.

8. The keyword extraction device for vertical field text according to claim 6, characterized in that: The training module includes: The first matching unit is used to obtain the preset domain vocabulary , by text matching, we can obtain the words that appear in the domain vocabulary in each text in the text pre-training data X1 The keywords in the above table are used as first candidate keywords, and all the first candidate keywords constitute a first candidate word set; An expansion unit, configured to expand the first candidate word set by using an unsupervised keyword extraction method to obtain a second candidate word set; A pre-training corpus determination unit, configured to take each text in the text pre-training data X1 as an original text, generate positive samples and negative samples based on the original text and keywords in the second candidate word set, and take the original text, positive samples and negative samples as pre-training corpus; A training unit is used to train the initial pre-trained language model using the pre-training corpus to obtain a basic pre-trained language model, wherein the initial pre-trained language model is a masked language model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the keyword extraction method for vertical field text as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for extracting keywords from vertical field texts as described in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Natural language processing method and device and computer equipment

    CN114528919A

  • News text and comment correlation analysis method based on comparative learning

    CN115146629A

  • Document keyword determination method and device, equipment and medium

    CN118035388A

  • Intention recognition sample generation method and device, model training method and device and intention recognition method and device

    CN118551765A

  • Generator set typical equipment operation and maintenance knowledge intelligent extraction method based on contrast example learning framework

    CN118551833A