Industrial text keyword extraction method and device based on large model, equipment and medium

Abstract

The invention discloses an industry text keyword extraction method and device based on a large model, equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a knowledge text from a preset data source, preprocessing the knowledge text, and constructing a corpus based on the obtained preprocessed knowledge text; training a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model; receiving a to-be-extracted text input by the user side, and processing the to-be-extracted text to generate a keyword generation cue word corresponding to the to-be-extracted text; and inputting the keyword generation cue word into a target keyword extraction model, so as to generate a plurality of target keywords corresponding to the to-be-extracted text based on the keyword generation cue word through the target keyword extraction model. Therefore, the keywords of the industry professional knowledge text can be efficiently and accurately extracted through the large model, so that the requirements of a specific industry in the aspects of text processing and information extraction are met.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for extracting industry text keywords based on a large model. Background Art

[0002] With the advent of the big data era, the rapid growth of text data has led to an increasing demand for keyword extraction technology in fields such as information retrieval, text summarization, and content analysis. Existing keyword extraction methods have problems such as low extraction efficiency and low accuracy when processing large-scale text data, and are not highly adaptable to specific industries. When processing knowledge texts in niche areas, they are unable to accurately extract relevant text keywords. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a method, device, equipment and storage medium for extracting keywords from industry text based on a large model, which can efficiently and accurately extract keywords from industry professional knowledge texts through a large model to meet the needs of specific industries in text processing and information extraction. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a method for extracting keywords from industry texts based on a large model, comprising:

[0005] Acquire knowledge text from a preset data source, and preprocess the knowledge text to build a corpus based on the obtained preprocessed knowledge text;

[0006] Performing model training on a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model;

[0007] Receiving a text to be extracted input by a user terminal, and processing the text to be extracted to generate a keyword generation prompt word corresponding to the text to be extracted;

[0008] The keyword generation prompt word is input into the target keyword extraction model, so as to generate a plurality of target keywords corresponding to the text to be extracted based on the keyword generation prompt word through the target keyword extraction model.

[0009] Optionally, the acquiring of knowledge text from a preset data source and preprocessing of the knowledge text to construct a corpus based on the obtained preprocessed knowledge text includes:

[0010] Crawling knowledge texts in a number of preset data sources in real time through a web crawler to obtain a number of knowledge texts, and performing relevance analysis on the number of knowledge texts according to preset relevant keywords, so as to remove irrelevant texts from the number of knowledge texts based on the obtained analysis results, and obtain a number of pre-processed knowledge texts;

[0011] Professional terms are extracted from the plurality of preprocessed knowledge texts to construct a corpus based on the professional terms.

[0012] Optionally, before the method of training a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model, the method further includes:

[0013] If there are new knowledge texts in the plurality of preset data sources, the new knowledge texts are crawled by a web crawler, and the corpus is updated by the new knowledge texts.

[0014] Optionally, the performing model training on a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model includes:

[0015] Constructing a training set, a validation set, and a test set according to the preprocessed knowledge text in the corpus;

[0016] Performing model training on a preset keyword extraction model according to the training set, and adjusting model parameters of the preset keyword extraction model through the verification set to obtain a trained keyword extraction model;

[0017] The model performance of the trained keyword extraction model is evaluated by using the test set, and if the model performance of the trained keyword extraction model meets the preset model performance requirement, the trained keyword extraction model is used as the target keyword extraction model;

[0018] If the model performance of the trained keyword extraction model does not meet the preset model performance requirements, jump to the step of performing model training on the preset keyword extraction model according to the training set, and adjusting the model parameters of the preset keyword extraction model through the verification set to obtain the trained keyword extraction model for the next round of model training.

[0019] Optionally, the receiving of the text to be extracted input by the user terminal and processing the text to be extracted to generate a keyword generation prompt word corresponding to the text to be extracted includes:

[0020] Receiving a text to be extracted input by a user terminal, and extracting a corresponding number of first preset keywords from the text to be extracted by using a first preset extraction algorithm and based on a preset keyword extraction number; the first preset extraction algorithm is a TextRank algorithm;

[0021] Extracting a corresponding number of second preset keywords from the text to be extracted by a second preset extraction algorithm and based on the preset keyword extraction number; the second preset extraction algorithm is a TF-IDF algorithm;

[0022] A keyword generation prompt word corresponding to the text to be extracted is generated based on the first preset keyword and the second preset keyword.

[0023] Optionally, the generating a keyword generation prompt word corresponding to the to-be-extracted text based on the first preset keyword and the second preset keyword includes:

[0024] Comparing the first preset keyword and the second preset keyword, and removing duplicates from the first preset keyword and the second preset keyword according to the comparison result, so as to obtain a plurality of removed duplicates keywords;

[0025] The plurality of keywords after deduplication are used as keyword generation prompt words corresponding to the text to be extracted.

[0026] Optionally, the step of generating a plurality of target keywords corresponding to the text to be extracted based on the keyword generation prompt word by the target keyword extraction model includes:

[0027] Generate a preset number of keywords to be filtered corresponding to the text to be extracted by using the target keyword extraction model;

[0028] The keywords to be filtered are filtered according to a preset keyword blacklist to obtain a number of target keywords.

[0029] In a second aspect, the present application discloses a device for extracting keywords from industry texts based on a large model, comprising:

[0030] A corpus construction module is used to obtain knowledge text from a preset data source and preprocess the knowledge text to construct a corpus based on the obtained preprocessed knowledge text;

[0031] A model training module, used for performing model training on a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model;

[0032] A prompt word generation module, used for receiving the text to be extracted input by the user terminal, and processing the text to be extracted to generate a keyword generation prompt word corresponding to the text to be extracted;

[0033] The keyword generation module is used to input the keyword generation prompt word into the target keyword extraction model, so as to generate a plurality of target keywords corresponding to the text to be extracted based on the keyword generation prompt word through the target keyword extraction model.

[0034] In a third aspect, the present application discloses an electronic device, comprising:

[0035] Memory, used to store computer programs;

[0036] A processor is used to execute the computer program to implement the aforementioned large model-based industry text keyword extraction method.

[0037] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned large model-based industry text keyword extraction method.

[0038] In the present application, knowledge text can be obtained from a preset data source, and the knowledge text can be preprocessed to construct a corpus based on the obtained preprocessed knowledge text; a preset keyword extraction model can be trained according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model; the text to be extracted input by the user end is received, and the text to be extracted is processed to generate a keyword generation prompt word corresponding to the text to be extracted; the keyword generation prompt word is input into the target keyword extraction model to generate a number of target keywords corresponding to the text to be extracted based on the keyword generation prompt word through the target keyword extraction model. It can be seen that through the method of the present application, the knowledge text obtained from the preset data source can be preprocessed, and a corpus can be constructed based on the obtained preprocessed knowledge text, and a preset keyword extraction model can be trained based on the data in the corpus to obtain a target keyword extraction model, when the text to be extracted input by the user end is received, a keyword generation prompt word corresponding to the text to be extracted can be generated, and a number of target keywords corresponding to the text to be extracted can be generated based on the keyword generation prompt word through the target keyword extraction model. In this way, by using large pre-trained models and performing industry-specific fine-tuning on them, it is possible to more accurately identify and extract keywords in texts, thereby providing higher-quality text analysis results. In addition, keyword extraction can be made more consistent with the terminology and context of specific industries, thereby improving the model's adaptability and understanding of industry knowledge. Furthermore, by using a combination of algorithms to extract keywords, it is ensured that the extracted keywords are highly relevant to the text topic while covering the main concepts and entities in the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0040] Figure 1 A flowchart of a method for extracting keywords from industry texts based on a large model disclosed in this application;

[0041] Figure 2 A timing diagram of a method for extracting keywords from industry texts based on a large model disclosed in this application;

[0042] Figure 3 A flow chart of another method for extracting keywords from industry texts based on a large model disclosed in this application;

[0043] Figure 4A keyword recognition flow chart disclosed in this application;

[0044] Figure 5 This is a schematic diagram of the structure of an industry text keyword extraction device based on a large model disclosed in this application;

[0045] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Existing keyword extraction methods have problems such as low extraction efficiency and low accuracy when processing large-scale text data, and are not highly adaptable to specific industries. When processing knowledge texts in niche areas, they are unable to accurately extract relevant text keywords.

[0048] In order to overcome the above-mentioned technical problems, the present application discloses a method, device, equipment and storage medium for extracting industry text keywords based on a large model, which can efficiently and accurately extract keywords from industry professional knowledge texts through a fine-tuned large model to meet the needs of specific industries in text processing and information extraction.

[0049] See also Figure 1 As shown, the embodiment of the present invention discloses a method for extracting keywords from industry texts based on a large model, comprising:

[0050] Step S11: Acquire knowledge text from a preset data source, and preprocess the knowledge text to construct a corpus based on the obtained preprocessed knowledge text.

[0051] In this embodiment, it is necessary to obtain knowledge text from a preset data source, and then preprocess the obtained knowledge text to construct a corpus based on the preprocessed knowledge text. Specifically, Figure 2As shown, the knowledge texts in several preset data sources can be crawled in real time by the web crawler in the corpus construction module to obtain several knowledge texts, for example, industry-related documents are collected from the Internet through technologies such as web crawlers. Including academic papers, technical reports, industry analysis, news reports, etc. Then, the correlation analysis of several knowledge texts is performed according to the preset relevant keywords, so as to remove irrelevant texts from several knowledge texts based on the obtained analysis results, and obtain several pre-processed knowledge texts. For example, if it is necessary to obtain knowledge texts of the medical industry and the financial industry, the keywords of the medical industry and the financial industry can be set, and the knowledge texts of the non-medical industry can be compared to ensure the text quality of the corpus. Then, the keywords in the document need to be annotated to obtain the pre-processed knowledge texts, and provide accurate training data for the model training module. Further, it is necessary to extract professional terms from several pre-processed knowledge texts to build a corpus based on professional terms, that is, industry terms and common words can be extracted from the pre-processed knowledge texts to build an industry-specific corpus.

[0052] It should be noted that in the case of real-time update of knowledge texts in the data source, in order to ensure the timeliness and coverage of the corpus, the knowledge texts in the data source can be crawled in real time through a web crawler. If there are new knowledge texts in several preset data sources, the new knowledge texts are crawled through a web crawler, and the corpus is updated with the new knowledge texts. In this way, the real-time nature of the constructed corpus can be effectively guaranteed.

[0053] Step S12: training a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model.

[0054] In this embodiment, it is necessary to train the preset keyword extraction model through the preprocessed knowledge text in the corpus to obtain the target keyword extraction model. Specifically, Figure 2As shown, the preset keyword extraction model needs to be trained by the model training module. The training set, validation set, and test set can be constructed based on the preprocessed knowledge text in the corpus. The preset keyword extraction model is fine-tuned through the labeled training set and validation set to adapt it to the keyword extraction task in a specific industry. Specifically, the preset keyword extraction model can be trained based on the training set, and the model parameters of the preset keyword extraction model can be adjusted through the validation set to obtain the trained keyword extraction model. Then, the model performance of the trained keyword extraction model is evaluated through the test set. According to the model evaluation results, the model is iteratively trained, and the model parameters with the best test effect are saved. If the model performance of the trained keyword extraction model meets the preset model performance requirements, such as the model performance reaches the preset model performance threshold, the trained keyword extraction model is used as the target keyword extraction model for keyword extraction. In another case, if the model performance of the trained keyword extraction model does not meet the preset model performance requirements, it is necessary to jump to the step of training the preset keyword extraction model based on the training set and adjusting the model parameters of the preset keyword extraction model through the validation set to obtain the trained keyword extraction model for the next round of model training. Finally, the fine-tuned industry large language model is deployed to provide keyword extraction services for industry professional knowledge text.

[0055] Step S13: Receive the text to be extracted input by the user terminal, and process the text to be extracted to generate a keyword generation prompt corresponding to the text to be extracted.

[0056] In this embodiment, it is necessary to receive the text to be extracted input by the user terminal and, as Figure 2 shown, perform a series of preprocessing operations on the unstandardized input text through the data preprocessing module to convert the original text to be extracted into a format suitable for model processing, including operations such as text cleaning, word segmentation, and stop word removal. Text cleaning is to remove irrelevant characters in the text, such as link tags, special symbols, numbers, and punctuation marks, etc.; stop word removal is to exclude common stop words, such as "de", "zai", etc., to reduce noise. Word segmentation is to split the text into words or phrases. After preprocessing, the preprocessed text to be extracted is obtained. After obtaining the preprocessed text to be extracted, the corresponding number of first preset keywords can be extracted from the preprocessed text to be extracted according to the preset keyword extraction quantity by the first preset extraction algorithm, and the corresponding number of second preset keywords can be extracted from the preprocessed text to be extracted according to the preset keyword extraction quantity by the second preset extraction algorithm. Then, a corresponding keyword generation prompt is generated based on the first preset keyword and the second preset keyword.

[0057] Step S14: input the keyword generation prompt word into the target keyword extraction model, so as to generate a plurality of target keywords corresponding to the text to be extracted based on the keyword generation prompt word through the target keyword extraction model.

[0058] In this embodiment, the keyword generation prompt word can be input into the target keyword extraction model, so that the target keyword extraction model in the keyword recognition module can generate several target keywords corresponding to the text to be extracted according to the keyword generation prompt word. Specifically, the target keyword extraction model after fine-tuning by the model training module can be used to let the target keyword extraction model automatically generate new keywords with reference to the keyword generation prompt word. It should be noted that the following instruction can be input into the target keyword extraction model to assist in generating new keywords: "{Extract ten relatively long hot keywords from the following text, separated by spaces, prohibit the extraction of names and dates as keywords, only entities, nouns, etc. can be extracted, and predicate short sentences cannot be extracted. The keywords can cover all the core contents of the following text as much as possible: {Input text}, you can refer to the following content {Keyword generation prompt word}}", because the vocabulary generated by the keyword extraction algorithm is generally short, so the ability of the large model is needed to generate vocabulary phrases. By telling the large model the number of generated keywords, the large model will automatically output keyword phrases. Further, due to security issues, if you want to avoid the large model outputting some words, you can set a blacklist for filtering. Therefore, the keywords to be filtered generated by the model can be filtered according to the preset keyword blacklist to obtain several target keywords. In this way, model training can be performed using industry-specific data sets, making keyword extraction more in line with the terminology and context of a specific industry, improving the model's adaptability and understanding of industry knowledge, and by using a large model for keyword extraction, it is possible to more accurately identify and extract keywords in the text, thereby providing higher quality text analysis results.

[0059] It can be seen that, through the method of the present application, the knowledge text obtained from the preset data source can be preprocessed, and a corpus can be constructed through the obtained preprocessed knowledge text, and the preset keyword extraction model can be trained through the data in the corpus to obtain a target keyword extraction model. When the text to be extracted input by the user end is received, a keyword generation prompt word corresponding to the text to be extracted can be generated, and a target keyword extraction model can generate a number of target keywords corresponding to the text to be extracted based on the keyword generation prompt word. In this way, by using a large pre-trained model and performing industry-specific fine-tuning on it, keywords in the text can be more accurately identified and extracted, thereby providing higher quality text analysis results, and keyword extraction can be made more in line with the terminology and context of a specific industry, improving the model's adaptability and understanding of industry knowledge. Furthermore, by using a combination algorithm to extract keywords, it is ensured that the extracted keywords are highly relevant to the text theme, while covering the main concepts and entities in the text.

[0060] Based on the above embodiments, it can be known that through the method of the present application, it is necessary to generate keyword generation prompt words corresponding to the text to be extracted, so that the keyword extraction model generates several target keywords corresponding to the text to be extracted based on the keyword generation prompt words. To this end, this embodiment describes in detail how to generate keyword generation prompt words and the role of keyword generation prompt words. Figure 3 As shown, the embodiment of the present invention discloses a method for extracting keywords from industry texts based on a large model, comprising:

[0061] Step S21: receiving a text to be extracted input by a user terminal, and extracting a corresponding number of first preset keywords from the text to be extracted by using a first preset extraction algorithm and based on a preset keyword extraction number.

[0062] In this embodiment, Figure 4 As shown, it is necessary to receive the text to be extracted input by the user end, and extract a corresponding number of first preset keywords from the text to be extracted through a first preset extraction algorithm and based on a preset keyword extraction quantity, wherein the first preset extraction algorithm is the TextRank algorithm. Specifically, since the TextRank algorithm is a graph-based sorting algorithm for keyword extraction and document summarization, by using the TextRank algorithm, keywords can be extracted based on the semantics between words within the document, and keywords and keyword groups of the text can be extracted from a given text. Therefore, a preset keyword extraction quantity can be set, for example, 10, and then 10 first preset keywords are extracted from the text to be extracted input by the user end through the TextRank algorithm.

[0063] Step S22: extracting a corresponding number of second preset keywords from the text to be extracted using a second preset extraction algorithm and based on the preset keyword extraction number.

[0064] In this embodiment, a corresponding number of second preset keywords can be extracted from the text to be extracted by a second preset extraction algorithm and based on the preset keyword extraction quantity, wherein the second preset extraction algorithm is a TF-IDF algorithm. Specifically, TF-IDF (term frequency–inverse document frequency) is a weighted technology used for information retrieval and text mining. The principle is that the importance of a word increases in direct proportion to the number of times it appears in a document, but at the same time it decreases in inverse proportion to the frequency of its appearance in the corpus. Therefore, by combining the semantic extraction algorithm with the word frequency extraction algorithm, the accuracy of keyword extraction can be improved, thereby extracting 10 second preset keywords from the text to be extracted input by the user end by the TF-IDF algorithm.

[0065] Step S23: extracting a corresponding number of second preset keywords from the text to be extracted using a second preset extraction algorithm and based on the preset keyword extraction number.

[0066] In this embodiment, the obtained 10 first preset keywords and 10 second preset keywords can be compared, and then the repeated keywords in the 10 first preset keywords and the 10 second preset keywords are determined according to the obtained comparison results, and the repeated keywords are removed from the 10 first preset keywords and the 10 second preset keywords to obtain a number of deduplicated keywords, and finally the number of deduplicated keywords are used as keyword generation prompt words corresponding to the text to be extracted. In this way, a combination algorithm can be used to extract keywords, ensuring that the extracted keywords are highly relevant to the text theme and cover the main concepts and entities in the text.

[0067] It can be seen that in this embodiment, the text to be extracted input by the user end is first received, and a corresponding number of first preset keywords are extracted from the text to be extracted by a first preset extraction algorithm based on a preset keyword extraction number, and then a corresponding number of second preset keywords are extracted from the text to be extracted by a second preset extraction algorithm based on the preset keyword extraction number, and finally a corresponding number of second preset keywords are extracted from the text to be extracted by a second preset extraction algorithm based on the preset keyword extraction number. In this way, a combination algorithm can be used to extract keywords to ensure that the extracted keywords are highly relevant to the text theme and cover the main concepts and entities in the text.

[0068] See Figure 5As shown, the embodiment of the present invention discloses a device for extracting keywords from industry texts based on a large model, comprising:

[0069] The corpus construction module 11 is used to obtain knowledge text from a preset data source and preprocess the knowledge text to construct a corpus based on the obtained preprocessed knowledge text;

[0070] A model training module 12, used for performing model training on a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model;

[0071] The prompt word generation module 13 is used to receive the text to be extracted input by the user terminal, and process the text to be extracted to generate a keyword generation prompt word corresponding to the text to be extracted;

[0072] The keyword generation module 14 is used to input the keyword generation prompt word into the target keyword extraction model, so as to generate a plurality of target keywords corresponding to the text to be extracted based on the keyword generation prompt word through the target keyword extraction model.

[0073] In the present application, knowledge text can be obtained from a preset data source, and the knowledge text can be preprocessed to construct a corpus based on the obtained preprocessed knowledge text; a preset keyword extraction model can be trained according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model; the text to be extracted input by the user end is received, and the text to be extracted is processed to generate a keyword generation prompt word corresponding to the text to be extracted; the keyword generation prompt word is input into the target keyword extraction model to generate a number of target keywords corresponding to the text to be extracted based on the keyword generation prompt word through the target keyword extraction model. It can be seen that through the method of the present application, the knowledge text obtained from the preset data source can be preprocessed, and a corpus can be constructed based on the obtained preprocessed knowledge text, and a preset keyword extraction model can be trained based on the data in the corpus to obtain a target keyword extraction model, when the text to be extracted input by the user end is received, a keyword generation prompt word corresponding to the text to be extracted can be generated, and a number of target keywords corresponding to the text to be extracted can be generated based on the keyword generation prompt word through the target keyword extraction model. In this way, by using large pre-trained models and performing industry-specific fine-tuning on them, it is possible to more accurately identify and extract keywords in texts, thereby providing higher-quality text analysis results. In addition, keyword extraction can be made more consistent with the terminology and context of specific industries, thereby improving the model's adaptability and understanding of industry knowledge. Furthermore, by using a combination of algorithms to extract keywords, it is ensured that the extracted keywords are highly relevant to the text topic while covering the main concepts and entities in the text.

[0074] In some embodiments, the corpus construction module 11 may specifically include:

[0075] A data acquisition unit is used to crawl knowledge texts from a plurality of preset data sources in real time through a web crawler to obtain a plurality of knowledge texts, and to perform relevance analysis on the plurality of knowledge texts according to preset relevant keywords, so as to remove irrelevant texts from the plurality of knowledge texts based on the obtained analysis results, and obtain a plurality of pre-processed knowledge texts;

[0076] The corpus construction unit is used to extract professional terms from the plurality of preprocessed knowledge texts to construct a corpus based on the professional terms.

[0077] In some embodiments, the industry text keyword extraction device based on the large model may further include:

[0078] The corpus updating unit is used to crawl the newly added knowledge text by a web crawler if there is newly added knowledge text in the plurality of preset data sources, and update the corpus by the newly added knowledge text.

[0079] In some embodiments, the model training module 12 may specifically include:

[0080] A data set construction unit, used to construct a training set, a validation set and a test set according to the preprocessed knowledge text in the corpus;

[0081] A parameter adjustment unit, configured to perform model training on a preset keyword extraction model according to the training set, and adjust model parameters of the preset keyword extraction model through the verification set to obtain a trained keyword extraction model;

[0082] A performance evaluation unit, used to evaluate the model performance of the trained keyword extraction model through the test set, and if the model performance of the trained keyword extraction model meets the preset model performance requirement, the trained keyword extraction model is used as the target keyword extraction model;

[0083] A step jump unit is used to jump to the step of performing model training on the preset keyword extraction model according to the training set and adjusting the model parameters of the preset keyword extraction model through the verification set to obtain the trained keyword extraction model if the model performance of the trained keyword extraction model does not meet the preset model performance requirements, so as to carry out the next round of model training.

[0084] In some embodiments, the prompt word generating module 13 may specifically include:

[0085] A first preset keyword extraction unit is used to receive a text to be extracted input by a user terminal, and extract a corresponding number of first preset keywords from the text to be extracted based on a preset keyword extraction number through a first preset extraction algorithm; the first preset extraction algorithm is a TextRank algorithm;

[0086] A second keyword extraction unit is used to extract a corresponding number of second preset keywords from the text to be extracted based on the preset keyword extraction number by using a second preset extraction algorithm; the second preset extraction algorithm is a TF-IDF algorithm;

[0087] The prompt word generation submodule is used to generate a keyword generation prompt word corresponding to the text to be extracted based on the first preset keyword and the second preset keyword.

[0088] In some embodiments, the prompt word generation submodule may specifically include:

[0089] A keyword deduplication unit, configured to perform a comparison process on the first preset keyword and the second preset keyword, and perform a deduplication process on the first preset keyword and the second preset keyword according to the comparison result, so as to obtain a plurality of deduplication keywords;

[0090] The prompt word generating unit is used to generate prompt words by taking the plurality of keywords after duplication removal as keywords corresponding to the text to be extracted.

[0091] In some embodiments, the keyword generation module 14 may specifically include:

[0092] A keyword generating unit, configured to generate a preset number of keywords to be filtered corresponding to the text to be extracted through the target keyword extraction model;

[0093] The keyword filtering unit is used to filter the keywords to be filtered according to a preset keyword blacklist to obtain a plurality of target keywords.

[0094] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0095] Figure 6 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the large model-based industry text keyword extraction method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0096] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0097] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0098] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the industry text keyword extraction method based on the large model performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0099] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed method for extracting keywords from industry texts based on a large model is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0100] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0101] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0102] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0103] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0104] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for extracting keywords from industry texts based on a large model, characterized in that: include: Acquire knowledge text from a preset data source, and preprocess the knowledge text to build a corpus based on the obtained preprocessed knowledge text; Performing model training on a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model; Receiving a text to be extracted input by a user terminal, and processing the text to be extracted to generate a keyword generation prompt word corresponding to the text to be extracted; The keyword generation prompt word is input into the target keyword extraction model, so as to generate a plurality of target keywords corresponding to the text to be extracted based on the keyword generation prompt word through the target keyword extraction model.

2. The method for extracting keywords from industry texts based on a large model according to claim 1 is characterized in that: The step of acquiring knowledge text from a preset data source and preprocessing the knowledge text to construct a corpus based on the preprocessed knowledge text includes: Crawling knowledge texts in a number of preset data sources in real time through a web crawler to obtain a number of knowledge texts, and performing relevance analysis on the number of knowledge texts according to preset relevant keywords, so as to remove irrelevant texts from the number of knowledge texts based on the obtained analysis results, and obtain a number of pre-processed knowledge texts; Professional terms are extracted from the plurality of preprocessed knowledge texts to construct a corpus based on the professional terms.

3. The method for extracting keywords from industry texts based on a large model according to claim 2 is characterized in that: Before the method of training a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model, the method further includes: If there are new knowledge texts in the plurality of preset data sources, the new knowledge texts are crawled by a web crawler, and the corpus is updated by the new knowledge texts.

4. The method for extracting keywords from industry texts based on a large model according to claim 1 is characterized in that: The performing model training on a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model includes: Constructing a training set, a validation set, and a test set according to the preprocessed knowledge text in the corpus; Performing model training on a preset keyword extraction model according to the training set, and adjusting model parameters of the preset keyword extraction model through the verification set to obtain a trained keyword extraction model; The model performance of the trained keyword extraction model is evaluated by using the test set, and if the model performance of the trained keyword extraction model meets the preset model performance requirement, the trained keyword extraction model is used as the target keyword extraction model; If the model performance of the trained keyword extraction model does not meet the preset model performance requirements, jump to the step of performing model training on the preset keyword extraction model according to the training set, and adjusting the model parameters of the preset keyword extraction model through the verification set to obtain the trained keyword extraction model for the next round of model training.

5. The method for extracting keywords from industry texts based on a large model according to claim 1 is characterized in that: The receiving of the text to be extracted input by the user terminal and processing the text to be extracted to generate a keyword generation prompt word corresponding to the text to be extracted includes: Receiving a text to be extracted input by a user terminal, and extracting a corresponding number of first preset keywords from the text to be extracted by using a first preset extraction algorithm and based on a preset keyword extraction number; the first preset extraction algorithm is a TextRank algorithm; Extracting a corresponding number of second preset keywords from the text to be extracted by a second preset extraction algorithm and based on the preset keyword extraction number; the second preset extraction algorithm is a TF-IDF algorithm; A keyword generation prompt word corresponding to the text to be extracted is generated based on the first preset keyword and the second preset keyword.

6. The method for extracting keywords from industry texts based on a large model according to claim 5 is characterized in that: The step of generating a keyword generation prompt word corresponding to the text to be extracted based on the first preset keyword and the second preset keyword includes: Comparing the first preset keyword and the second preset keyword, and removing duplicates from the first preset keyword and the second preset keyword according to the comparison result, so as to obtain a plurality of removed duplicates keywords; The plurality of keywords after deduplication are used as keyword generation prompt words corresponding to the text to be extracted.

7. The method for extracting keywords from industry texts based on a large model according to any one of claims 1 to 6, characterized in that: The step of generating a plurality of target keywords corresponding to the text to be extracted based on the keyword generation prompt word by the target keyword extraction model includes: Generate a preset number of keywords to be filtered corresponding to the text to be extracted by using the target keyword extraction model; The keywords to be filtered are filtered according to a preset keyword blacklist to obtain a number of target keywords.

8. A device for extracting keywords from industry texts based on a large model, characterized in that: include: A corpus construction module is used to obtain knowledge text from a preset data source and preprocess the knowledge text to construct a corpus based on the obtained preprocessed knowledge text; A model training module, used for performing model training on a preset keyword extraction model according to the preprocessed knowledge text in the corpus to obtain a target keyword extraction model; A prompt word generation module, used for receiving the text to be extracted input by the user terminal, and processing the text to be extracted to generate a keyword generation prompt word corresponding to the text to be extracted; The keyword generation module is used to input the keyword generation prompt word into the target keyword extraction model, so as to generate a plurality of target keywords corresponding to the text to be extracted based on the keyword generation prompt word through the target keyword extraction model.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the industry text keyword extraction method based on a large model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for extracting industry text keywords based on a large model as described in any one of claims 1 to 7 is implemented.

Images