Text feature data processing method and system based on BERT model and TF-IDF weighting

Abstract

The invention belongs to the technical field of intelligent data processing, and particularly relates to a BERT model and TF-IDF weighting-based text feature data processing method and system, and the method comprises the steps: firstly obtaining to-be-analyzed data of a target domain text, and carrying out the preprocessing of word segmentation, stop word removal, text format unification and the like through a jieba library; then, an effect dictionary is constructed, including construction of an initial effect word dictionary, dictionary expansion and manual checking and labeling; in the feature extraction stage, a TF-IDF value is calculated to evaluate the importance of words in the document. And an innovative scoring link: matching the effect words through a regular expression, performing secondary judgment by using a large language model, performing secondary weighting on the common effect words, and finally scoring the innovativeness of the target domain text based on a TF-IDF value. According to the method, the deep semantic understanding of the BERT model and the text statistical characteristics of the TF-IDF are combined, and the accuracy and efficiency of innovative evaluation of the text in the target field are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent data processing, and specifically relates to a text feature data processing method and system based on a BERT model and TF-IDF weighting. Background Art

[0002] In the current technological innovation environment, the number and quality of a company's patents are important indicators of its innovation capabilities. However, traditional manual evaluation methods are not only time-consuming and labor-intensive, but the evaluation results are easily affected by subjective factors, making it difficult to ensure accuracy and consistency. With the development of natural language processing technology, especially the emergence of pre-trained language models such as BERT and text feature extraction methods such as TF-IDF, new technical means have been provided for patent innovation evaluation.

[0003] Although there are some patent analysis tools based on natural language processing in the prior art, these tools often only focus on simple statistical features or keyword matching of patent texts, lack in-depth semantic understanding and comprehensive evaluation models, resulting in inaccurate and incomplete evaluation results. Therefore, it is necessary to develop an innovativeness evaluation system and method that combines the BERT model and TF-IDF weighted scoring to achieve a more accurate and efficient evaluation of the innovativeness of patent abstracts. Summary of the invention

[0004] The purpose of the present invention is to provide a text feature data processing method and system based on the BERT model and TF-IDF weighting to solve the problems raised in the background technology.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0006] In the first aspect, the present invention proposes a text feature data processing method based on the BERT model and TF-IDF weighting, the method comprising:

[0007] Obtain the text data to be analyzed of the target domain text;

[0008] Preprocessing the text data to be analyzed by segmenting the text data, removing stop words, and unifying the text format to generate text data to be processed;

[0009] After the text data to be processed is input into a pre-built effect dictionary for effect word matching and evaluation, a large language model is used to perform secondary judgment and secondary weighting on the matched effect words to generate a text feature data processing result of the target domain text;

[0010] The pre-constructed effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by word vector model, and an effect dictionary of newly marked words.

[0011] Furthermore, the preprocessing of segmenting the text data to be analyzed, removing stop words and unifying the text format includes:

[0012] Use the jieba library to segment the text data to be analyzed;

[0013] Prepare a stop word list, including common conjunctions, prepositions, etc., use jieba to segment words and filter stop words;

[0014] Cleans punctuation and invalid characters, retaining characters that are useful for parsing.

[0015] Furthermore, the pre-constructed effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by the word vector model, and an effect dictionary of newly annotated words, including:

[0016] Construct and optimize a dictionary of key words that describe the effects within the target domain text, including the construction of an initial dictionary of effect words, dictionary expansion, and manual inspection and annotation after the fusion dictionary;

[0017] The dictionary expansion is specifically achieved through co-occurrence analysis, training word vector models, and using the BERT model to perform context analysis of words;

[0018] The co-occurrence analysis includes collecting corpus, calculating co-occurrence frequency and screening co-occurring words;

[0019] The training word vector model includes using the transformers library to load the pre-trained BERT model to convert each word into a vector representation and calculate the semantic similarity. The context analysis includes constructing a sentence containing the initial effect word.

[0020] The BERT model is used to encode sentences containing initial effect words, obtain contextual representations of the words, and then analyze the contextual representations, identify changes in the meanings of the words in different contexts, and add new annotation information to the initial effect words.

[0021] Furthermore, the co-occurrence analysis specifically includes the following steps:

[0022] Collect the corpus and perform word segmentation on the text in the corpus;

[0023] For each initial effect word, count the number of times it co-occurs with other words within a set number of word windows, create a co-occurrence matrix, and record the co-occurrence frequency of each word pair;

[0024] Set a threshold and filter out words with co-occurrence frequency higher than the threshold as potential effect word candidates.

[0025] Furthermore, the training word vector model specifically includes the following steps:

[0026] Use the transformers library to load the pre-trained BERT model and convert each word into a vector representation;

[0027] For each initial effect word, calculate its cosine similarity with all other words in the dictionary, sort the similarity results, and select the n words with the highest similarity as synonyms.

[0028] Furthermore, the step of inputting the text data to be processed into a pre-built effect dictionary for effect word matching and evaluation includes:

[0029] Calculate the TF-IDF value of each word in the text data to be processed;

[0030] The effect words are matched and scored according to the positions and TF-IDF values ​​of the words in the effect dictionary in the text data to be processed.

[0031] Furthermore, the use of the large language model to perform secondary judgment and secondary weighting on the matched effect words to generate the text feature data processing results of the target domain text includes:

[0032] According to the words in the effect dictionary, a regular expression matching pattern is constructed to locate the effect words in the text data to be processed;

[0033] Input the sentence containing the effect word into the large language model for secondary judgment to obtain the inference effect word;

[0034] Compare the inference effect words output by the large language model with the effect words matched by the regular expression, and perform secondary weighting on the effect words that exist in both the inference results and the matching results;

[0035] Based on the TF-IDF value and combined with the effect word score after secondary empowerment, the innovation of text in a single target field is scored.

[0036] Furthermore, the secondary weighted score is calculated as follows:

[0037] Set the weight coefficient α, for each common effect word t commonk , the effect word score is calculated as the original TF-IDF value multiplied by the weighting coefficient α.

[0038] Furthermore, the evaluation of the innovation of the text in a single target field is specifically calculated according to the following formula:

[0039] The text summary D contains t effect words {t1, t2, …, t m}, the innovation score of the target field text is:

[0040]

[0041] Among them, f commoni is a common effect word commoni The number of times it appears in the data to be analyzed D; f non-commonj Yes and No Common Effect Words non-commonj The number of occurrences; m is the total number of effect words matched by the regular expression, k is the number of common effect words, and mk is the number of non-common effect words.

[0042] In a second aspect, the present invention proposes a text feature data processing system based on the BERT model and TF-IDF weighting, characterized in that it is applied to execute any of the above data processing methods, and the system includes:

[0043] A data acquisition module is used to acquire the text data to be analyzed of the target domain text;

[0044] A data preprocessing module, used for preprocessing the text data to be analyzed by segmenting, removing stop words and unifying the text format to generate text data to be processed;

[0045] A matching analysis module is used to input the text data to be processed into a pre-built effect dictionary for effect word matching and evaluation, and then use a large language model to perform secondary judgment and secondary weighting on the matched effect words to generate a text feature data processing result of the target domain text;

[0046] The pre-constructed effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by word vector model, and an effect dictionary of newly marked words.

[0047] The beneficial effects of the present invention are:

[0048] 1. By combining the BERT model and TF-IDF weighted scoring, the present invention can deeply mine the key effect words in the text and accurately evaluate their importance. The application of the BERT model enables a deeper understanding of the context of words, effectively avoiding misjudgments caused by simple matching or statistics in traditional methods. By constructing an enhanced effect dictionary containing initial effect words, co-occurrence analysis words, word vector model synonyms, and newly annotated words, this method covers a wider range of effect words and improves the comprehensiveness of the evaluation.

[0049] 2. This method provides an objective and scientific basis for evaluating the innovation of patents through the quantified TF-IDF value and the innovation score after secondary weighting. This helps enterprises, scientific research institutions and individuals make more informed decisions during the patent application, examination and maintenance process. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a text feature data processing method based on the BERT model and TF-IDF weighting provided in an embodiment of the present application;

[0051] Figure 2 A structural diagram of a text feature data processing system based on the BERT model and TF-IDF weighting provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The present application is further described in detail below in conjunction with the accompanying drawings. It is necessary to point out here that the following specific implementation methods are only used to further illustrate the present application and cannot be understood as limiting the scope of protection of the present application. Technical personnel in this field can make some non-essential improvements and adjustments to the present application based on the above application content.

[0053] Example 1

[0054] like Figure 1 As shown, this embodiment proposes a text feature data processing method based on the BERT model and TF-IDF weighting, comprising the following steps:

[0055] S1. Obtain the text data to be analyzed in the target field; how to determine the target enterprise, access the patent database (such as USPTO, EPO, CNIPA, etc.), and use API to extract patent abstracts.

[0056] S2. Preprocess the text data to be analyzed by segmenting, removing stop words and unifying the text format to generate the text data to be processed

[0057] S3. After the text data to be processed is input into a pre-constructed effect dictionary for effect word matching and evaluation, a large language model is used to perform secondary judgment and secondary weighting on the matched effect words to generate the text feature data processing result of the target field text;

[0058] The pre-constructed effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by word vector model, and an effect dictionary of newly marked words.

[0059] Further preferably, the preprocessing of segmenting the text data to be analyzed, removing stop words and unifying the text format includes:

[0060] Use the jieba library to segment the text data to be analyzed;

[0061] Prepare a stop word list, including common conjunctions, prepositions, etc., use jieba to segment words and filter stop words;

[0062] Cleans punctuation and invalid characters, retaining characters that are useful for parsing.

[0063] 6. Further preferably, the pre-constructed effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by word vector model and effect dictionary of newly marked words, including:

[0064] Construct and optimize a dictionary of key words that describe the effects within the target domain text, including the construction of an initial dictionary of effect words, dictionary expansion, and manual inspection and annotation after the fusion dictionary;

[0065] The dictionary expansion is specifically achieved through co-occurrence analysis, training word vector models, and using the BERT model to perform context analysis of words;

[0066] The co-occurrence analysis includes collecting corpus, calculating co-occurrence frequency and screening co-occurring words;

[0067] The training word vector model includes using the transformers library to load the pre-trained BERT model to convert each word into a vector representation and calculate the semantic similarity. The context analysis includes constructing a sentence containing the initial effect word.

[0068] The BERT model is used to encode sentences containing initial effect words, obtain contextual representations of the words, and then analyze the contextual representations, identify changes in the meanings of the words in different contexts, and add new annotation information to the initial effect words.

[0069] Further preferably, the co-occurrence analysis specifically comprises the following steps:

[0070] Collect the corpus and perform word segmentation on the text in the corpus;

[0071] For each initial effect word, count the number of times it co-occurs with other words within a set number of word windows, create a co-occurrence matrix, and record the co-occurrence frequency of each word pair;

[0072] Set a threshold and filter out words with co-occurrence frequency higher than the threshold as potential effect word candidates.

[0073] Further preferably, the training word vector model specifically includes the following steps:

[0074] Use the transformers library to load the pre-trained BERT model and convert each word into a vector representation;

[0075] For each initial effect word, calculate its cosine similarity with all other words in the dictionary, sort the similarity results, and select the n words with the highest similarity as synonyms.

[0076] Further preferably, the step of inputting the text data to be processed into a pre-built effect dictionary for effect word matching and evaluation includes:

[0077] Calculate the TF-IDF value of each word in the text data to be processed;

[0078] The effect words are matched and scored according to the positions and TF-IDF values ​​of the words in the effect dictionary in the text data to be processed.

[0079] Further preferably, the use of the large language model to perform secondary judgment and secondary weighting on the matched effect words to generate the text feature data processing result of the target domain text includes:

[0080] According to the words in the effect dictionary, a regular expression matching pattern is constructed to locate the effect words in the text data to be processed;

[0081] Input the sentence containing the effect word into the large language model for secondary judgment to obtain the inference effect word;

[0082] Compare the inference effect words output by the large language model with the effect words matched by the regular expression, and perform secondary weighting on the effect words that exist in both the inference results and the matching results;

[0083] Based on the TF-IDF value and combined with the effect word score after secondary weighting, the innovation of a single public text is scored.

[0084] Further preferably, the secondary weighted score is calculated as follows:

[0085] Set the weight coefficient α, for each common effect word t commonk , the effect word score is calculated as the original TF-IDF value multiplied by the weighting coefficient α.

[0086] Further preferably, the evaluation of the innovation of the text in a single target field is calculated specifically according to the following formula:

[0087] The text summary D contains t effect words {t1, t2, …, t m}, the innovation score of the target field text is:

[0088]

[0089] Among them, f commoni is a common effect word commoni The number of times it appears in the abstract text D; f non-commonj Yes and No Common Effect Words non-commonj The number of occurrences; m is the total number of effect words matched by the regular expression, k is the number of common effect words, and mk is the number of non-common effect words.

[0090] It should be noted that the text feature data processing method based on the BERT model and TF-IDF weighting proposed in this embodiment can also be applied to the content feature extraction of patent texts, thereby improving the evaluation of innovation.

[0091] As an example, the innovativeness evaluation according to the above embodiment specifically includes the following steps:

[0092] 1. Data Acquisition

[0093] 1. Obtain all patent summaries of enterprises in the target field:

[0094] Identify target companies, access patent databases (such as USPTO, EPO, CNIPA, etc.), and use API to extract patent abstracts.

[0095] 2. Data Preprocessing

[0096] Use the jieba library to segment patent abstract text:

[0097] ①Word segmentation optimization:

[0098] Collect industry-related professional effect vocabulary and use jieba.load_userdict() to load a custom dictionary.

[0099] ②Remove stop words:

[0100] Prepare a stop word list, including common conjunctions, prepositions, etc. At the same time, optimize the dictionary based on the original stop word dictionary to ensure the effectiveness of the word segmentation results, use jieba word segmentation, and filter stop words.

[0101] ③ Unified text format:

[0102] Cleans punctuation and invalid characters, retaining characters that are useful for parsing.

[0103] Build effects dictionary

[0104] 3. Construction of the initial effect word dictionary:

[0105] Identify key words that describe effects in the field, such as "high efficiency", "energy saving", "improvement", etc., summarize these words and form an initial effect word dictionary.

[0106] 2. Dictionary expansion:

[0107] ① Co-occurrence analysis:

[0108] Use statistical methods to calculate word co-occurrence frequencies and identify words that co-occur with known effect words.

[0109] a. Collect corpus: Based on the target field, collect and organize text corpus, and perform word segmentation on the text in the corpus to identify words.

[0110] b. Calculate the co-occurrence frequency: For each initial effect word, count the number of times it co-occurs with other words within the 5-word window, create a co-occurrence matrix, and record the co-occurrence frequency of each word pair.

[0111] d. Filter co-occurring words: Set a threshold and filter out words with a co-occurrence frequency higher than the threshold as potential effect word candidates.

[0112] ②Training word vector model:

[0113] a. Training word vector model:

[0114] Use the transformers library to load the pre-trained BERT model and convert each word into a vector representation through the BERT model.

[0115] b. Calculate semantic similarity:

[0116] For each initial effect word, calculate its cosine similarity with all other words in the dictionary and sort the similarity results.

[0117] The cosine similarity is calculated using the following formula:

[0118]

[0119] Select the n words with the highest similarity as synonyms.

[0120] ③ Use the BERT model to perform contextual analysis of words, and then combine the initial dictionary and newly annotated words to generate an enhanced effect dictionary.

[0121] a. Context analysis:

[0122] Construct sentences containing initial effect words, ensure that the words appear in different contexts, and use the BERT model to encode these sentences to obtain the contextual representation of the words.

[0123] b. New word marking:

[0124] Analyze the contextual representation, identify the changes in the meaning of words in different contexts, and add new annotation information to the initial effect words based on the analysis results.

[0125] 3. Fusion Dictionary:

[0126] Merge the initial dictionary, the words obtained from the co-occurrence analysis, the synonyms obtained from the word vector model, and the newly annotated words.

[0127] 4. Manual inspection and annotation:

[0128] The merged dictionary is deduplicated and verified to ensure its quality, and the validity and relevance of words are confirmed based on domain knowledge.

[0129] 4. Feature Extraction

[0130] 1. Calculate TF-IDF

[0131] ① Calculate the TF (word frequency) of the word:

[0132]

[0133] ② Calculate the IDF (inverse document frequency) of the word:

[0134]

[0135] ③Calculate TF-IDF:

[0136] TF-IDF(t,d)=TF(t,d)×IDF(t)

[0137] 2. Effect word scoring:

[0138] The words in the document are sorted according to the TF-IDF value. The words with higher scores are considered to be more discriminative or important. The score of the effect word is TF-IDF(t i ,D).

[0139] 5. Innovation Scoring

[0140] 1. Effect word matching

[0141] According to the words in the effect dictionary, a regular expression matching pattern is constructed to locate the effect words in the abstract text, and all the effect words and sentences containing effect words in patent abstract D are obtained. According to the TF-IDF value calculated previously, a score is assigned to each matched effect word.

[0142] 2. Input the sentence containing the effect word into the large language model, and use the large language model to make a secondary judgment on the effect word to obtain the inferred effect words {w1,w2,w3,…,w n}, compare the inference effect words output by the large language model with the effect words matched by the regular expression, and compare the effect words {t common1 ,t common2 ,…,t commonk}Perform secondary weighting.

[0143] Based on the accuracy of the large language model, the performance of the large language model in natural language processing tasks is usually better than that of simple regular expression matching. At the same time, the common effect words appear in both judgments, indicating that these words are more important in the patent abstract and contribute more to the patent's innovation. Therefore, the effect words confirmed by the large language model have higher credibility and should be given higher weight.

[0144] Set the weighting coefficient α = 1.5, for each common effect word t commonk , whose score is calculated as:

[0145] TF-IDF(t commoni ,D)×α

[0146] 3. Based on the TF-IDF value, score the innovation of a single patent after secondary weighting:

[0147] Patent abstract D, contains t effect words {t1, t2, …, t m}, the innovation score of patent claim D is:

[0148]

[0149] Among them, f commoni is a common effect word commoni The number of times it appears in the abstract text D; f non-commonj Yes and No Common Effect Words non-commonj The number of occurrences; m is the total number of effect words matched by the regular expression, k is the number of common effect words, and mk is the number of non-common effect words.

[0150] According to the above embodiment of the present invention, when implementing the solution, all patent abstracts of the target enterprise can be obtained from the patent database, which is the basic data for evaluation. Subsequently, the patent abstracts are segmented using the jieba library, and stop words are removed and the text format is unified to ensure the quality of the data and the effectiveness of the analysis.

[0151] Next, this application constructs an effect dictionary, which not only contains the initial effect words, but also expands and optimizes them through methods such as co-occurrence analysis, word vector model, and context analysis of the BERT model, so as to more comprehensively cover the effect words in the target field.

[0152] During the feature extraction stage, this application calculated the TF-IDF value of each word to evaluate its importance and distinctiveness in the patent abstract.

[0153] Finally, this application uses the constructed effect dictionary and TF-IDF value to match and score the effect words in the patent abstract. At the same time, the introduction of a large language model to make a secondary judgment on the effect words further improves the accuracy of the evaluation. Through comprehensive calculation, the innovation score of the patent is obtained, which provides strong support for the evaluation of patent technology.

[0154] Based on the same inventive concept, Figure 2 As shown, this embodiment also proposes a text feature data processing system based on the BERT model and TF-IDF weighting, which is applied to execute the above data processing method, and the system includes:

[0155] A data acquisition module is used to obtain summary texts of public texts in the target field;

[0156] A data preprocessing module, used for preprocessing the summary text by segmenting the text, removing stop words and unifying the text format;

[0157] A matching analysis module is used to input the text data to be processed into a pre-built effect dictionary for effect word matching and evaluation, and then use a large language model to perform secondary judgment and secondary weighting on the matched effect words to generate a text feature data processing result of the target domain text;

[0158] The pre-constructed effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by word vector model, and an effect dictionary of newly marked words.

[0159] It should be noted here that each module in the above-mentioned data processing system corresponds to steps S1 to S3 in implementing the above-mentioned data processing method, and the instances and application scenarios implemented by multiple modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0160] This embodiment further provides an electronic device, including:

[0161] a processor; a memory for storing instructions executable by the processor;

[0162] Among them, the processor is configured to execute instructions to implement the text feature data processing method based on the BERT model and TF-IDF weighting as mentioned above.

[0163] This embodiment also proposes a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the text feature data processing method based on the BERT model and TF-IDF weighting as described above.

[0164] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0165] In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0166] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0167] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A text feature data processing method based on the BERT model and TF-IDF weighting, characterized in that: The method comprises: Obtain the text data to be analyzed of the target domain text; Preprocessing the text data to be analyzed by segmenting the text data, removing stop words, and unifying the text format to generate text data to be processed; After the text data to be processed is input into a pre-built effect dictionary for effect word matching and evaluation, a large language model is used to perform secondary judgment and secondary weighting on the matched effect words to generate a text feature data processing result of the target domain text; The pre-constructed effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by word vector model, and an effect dictionary of newly marked words.

2. The text feature data processing method based on the BERT model and TF-IDF weighting according to claim 1, characterized in that: The preprocessing of segmenting the text data to be analyzed, removing stop words and unifying the text format includes: Use the jieba library to segment the text data to be analyzed; Prepare a stop word list, including common conjunctions, prepositions, etc., use jieba to segment words and filter stop words; Cleans punctuation and invalid characters, retaining characters that are useful for parsing.

3. The text feature data processing method based on the BERT model and TF-IDF weighting according to claim 1, characterized in that: The pre-built effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by the word vector model, and an effect dictionary of newly annotated words, including: Construct and optimize a dictionary of key words that describe the effects within the target domain text, including the construction of an initial dictionary of effect words, dictionary expansion, and manual inspection and annotation after the fusion dictionary; The dictionary expansion is specifically achieved through co-occurrence analysis, training word vector models, and using the BERT model to perform context analysis of words; The co-occurrence analysis includes collecting corpus, calculating co-occurrence frequency and screening co-occurring words; The training word vector model includes using the transformers library to load the pre-trained BERT model to convert each word into a vector representation and calculate the semantic similarity. The context analysis includes constructing a sentence containing the initial effect word. The BERT model is used to encode sentences containing initial effect words, obtain contextual representations of the words, and then analyze the contextual representations, identify changes in the meanings of the words in different contexts, and add new annotation information to the initial effect words.

4. The text feature data processing method based on the BERT model and TF-IDF weighting according to claim 3, characterized in that: The co-occurrence analysis specifically includes the following steps: Collect the corpus and perform word segmentation on the text in the corpus; For each initial effect word, count the number of times it co-occurs with other words within a set number of word windows, create a co-occurrence matrix, and record the co-occurrence frequency of each word pair; Set a threshold and filter out words with co-occurrence frequency higher than the threshold as potential effect word candidates.

5. The text feature data processing method based on the BERT model and TF-IDF weighting according to claim 3, characterized in that: The training word vector model specifically includes the following steps: Use the transformers library to load the pre-trained BERT model and convert each word into a vector representation; For each initial effect word, calculate its cosine similarity with all other words in the dictionary, sort the similarity results, and select the n words with the highest similarity as synonyms.

6. The text feature data processing method based on the BERT model and TF-IDF weighting according to claim 1, characterized in that: The step of inputting the text data to be processed into a pre-built effect dictionary for effect word matching and evaluation includes: Calculate the TF-IDF value of each word in the text data to be processed; The effect words are matched and scored according to the positions and TF-IDF values ​​of the words in the effect dictionary in the text data to be processed.

7. The text feature data processing method based on the BERT model and TF-IDF weighting according to claim 6, characterized in that: The method of using the large language model to perform secondary judgment and secondary weighting on the matched effect words to generate the text feature data processing result of the target domain text includes: According to the words in the effect dictionary, a regular expression matching pattern is constructed to locate the effect words in the text data to be processed; Input the sentence containing the effect word into the large language model for secondary judgment to obtain the inference effect word; Compare the inference effect words output by the large language model with the effect words matched by the regular expression, and perform secondary weighting on the effect words that exist in both the inference results and the matching results; Based on the TF-IDF value and combined with the effect word score after secondary empowerment, the innovation of text in a single target field is scored.

8. The text feature data processing method based on the BERT model and TF-IDF weighting according to claim 7, characterized in that: The secondary weighted score calculation method is: Set the weight coefficient α, for each common effect word t commonk , the effect word score is calculated as the original TF-IDF value multiplied by the weighting coefficient α.

9. The text feature data processing method based on the BERT model and TF-IDF weighting according to claim 8, characterized in that: The evaluation of the innovation of the text in a single target field is specifically calculated according to the following formula: The text summary D contains t effect words {t1, t2, …, t m }, the innovation score of the target field text is: Among them, f commoni is a common effect word commoni The number of times it appears in the data to be analyzed D; f non-commonj Yes and No Common Effect Words non-commonj The number of occurrences; m is the total number of effect words matched by the regular expression, k is the number of common effect words, and mk is the number of non-common effect words.

10. A text feature data processing system based on the BERT model and TF-IDF weighting, characterized in that: Applied to executing the data processing method according to any one of claims 1 to 9, the system comprises: A data acquisition module is used to acquire the text data to be analyzed of the target domain text; A data preprocessing module, used for preprocessing the text data to be analyzed by segmenting, removing stop words and unifying the text format to generate text data to be processed; A matching analysis module is used to input the text data to be processed into a pre-built effect dictionary for effect word matching and evaluation, and then use a large language model to perform secondary judgment and secondary weighting on the matched effect words to generate a text feature data processing result of the target domain text; The pre-constructed effect dictionary includes initial effect words, words obtained by co-occurrence analysis, synonyms obtained by word vector model, and an effect dictionary of newly marked words.

Images