A method and system for constructing enterprise digital terminology library based on semi-supervised learning

Through a semi-supervised learning method, the text data of enterprise annual report is processed using ERNIE and GlobalPointer models, which solves the problems of poor scalability and low efficiency in the construction of enterprise digital term library, and achieves efficient and accurate term recognition and dictionary expansion.

CN118194867BActive Publication Date: 2024-11-29WUHAN UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202311666252.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-05
Publication Date
2024-11-29
Estimated Expiration
2043-12-05

AI Technical Summary

Technical Problem

The prior art has problems such as poor scalability, low efficiency, low recognition accuracy and reliance on large-scale text annotation when building enterprise digital term databases.

Method used

Using a semi-supervised learning method, the text data of the enterprise annual report is characterized and featured by ERNIE and GlobalPointer models, the labeled data set is expanded through sorting algorithms and pseudo-labels, the term recognition model is gradually optimized, and the enterprise digital term library is constructed.

Benefits of technology

It improves the efficiency of enterprise digital term recognition, enhances the scalability of term dictionaries, reduces the dependence on large-scale text annotations, and adapts to complex and changeable language environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118194867B_ABST
    Figure CN118194867B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for constructing a digital terminology library for an enterprise based on semi-supervised learning, including: S1: collecting annual report text data of an enterprise; S2: using ERNIE to characterize and vectorize the annual report text; S3: using the GlobalPointer model to extract features and identify terms for the annual report text, and using the final output probability as the basis for identifying term boundaries and their types; S4: evaluating the performance of the model, and if the exit condition is met, obtaining the final term identification model, otherwise proceeding to S5; S5: using the model to perform term identification on an unlabeled data set, and sorting all samples by output probability; S6: taking the top N samples with the highest ranking and the recognition results corresponding to the top N samples to expand the labeled data set, correcting the model, and returning to S2 until the final exit condition in S4 is met to obtain the final digital term extraction model. The present invention combines deep learning and semi-supervised learning technologies with domain problems, improves the efficiency of digital term identification, and enhances the scalability of digital term dictionaries.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for constructing a digital terminology library for an enterprise based on semi-supervised learning. Background Art

[0002] With the innovative development of digital information technology, the digital technology innovation capabilities in key areas continue to improve, and implementing digital transformation has become an important strategic priority for many companies.

[0003] In order to test the implementation performance of the digital transformation of enterprises, the first task is to effectively measure the "digital transformation". The annual report of listed companies discloses the main business information, operating conditions and management's discussion and analysis of future development of the enterprise, which is of great reference value for grasping the business strategy, decision-making planning and development path of the enterprise. The frequency of relevant terms in the annual report just reflects the strategic attention and resource investment of the enterprise in digital technology innovation and application practice. Therefore, it is feasible and scientific to use the word frequency information of digital-related terms in the annual report text of listed companies as a measure of the degree of digital transformation of enterprises. However, when measuring the degree of digital transformation of enterprises based on text analysis, how to build a systematic and complete digital-related terminology database is a basic and key issue.

[0004] Among the existing solutions, there are solutions that use a method for constructing an enterprise digital terminology library based on manual screening. However, this method is subject to the subjectivity and knowledge level of the summarizer, the extraction task is arduous, time-consuming, and inefficient. The final constructed enterprise digital terminology library is limited in scale and has poor scalability. There are solutions that provide an enterprise digital terminology library construction method based on similar word expansion. First, the seed words of the digital terms are selected, and the similar words of each seed word are extracted using the Word2vec model. However, the algorithm of this technology is relatively simple, and the number of identified terms is still limited, and the scalability of the algorithm is not fundamentally solved. There is also a solution that provides a general terminology recognition method. This method uses multiple grammatical rules to determine candidate terms, and performs the final term extraction based on statistical methods, but this technology Only some linguistic features in the text sequence are used while the deeper semantic features are ignored. The recognition accuracy is low and it cannot adapt to the complex and changeable language environment and terminology extraction in professional fields. There is also a solution that provides an intelligent construction method for a construction professional terminology library based on hybrid deep semantic mining, which combines engineering construction information management, text processing technology and machine learning methods. It overcomes the shortcomings of previous manual extraction of professional terminology, deepens the semantic analysis of construction texts, improves the efficiency and accuracy of construction professional terminology recognition, reduces errors in professional terminology extraction, and makes the construction of construction professional terminology library more intelligent. However, this technology still relies on large-scale text annotation, which requires a lot of time and financial costs, and is not suitable for the construction of digital terminology libraries. Summary of the invention

[0005] The present invention provides a method and system for constructing a digital terminology library for an enterprise based on semi-supervised learning, so as to solve the defects existing in the process of constructing a digital terminology library for an enterprise in the prior art.

[0006] In a first aspect, the present invention provides a method for constructing an enterprise digital terminology library based on semi-supervised learning, comprising:

[0007] Step S1, collecting the annual report text data of the enterprise;

[0008] Step S2, characterizing the annual report text data based on the ERNIE model to obtain an annual report text vector sequence;

[0009] Step S3, using the GlobalPointer model to perform feature extraction on the annual report text vector sequence, and determining the final output probability of the initial term recognition model, wherein the final output probability is used to identify term boundaries and term types;

[0010] Step S4, evaluating the initial term recognition model according to a preset evaluation index, and if the preset training end condition is met, outputting the final term recognition model, otherwise proceeding to step S5;

[0011] Step S5, performing term recognition on the unlabeled data set based on the initial term recognition model, and sorting all samples by output probability according to the sorting algorithm to obtain sorted annual report text sample data;

[0012] Step S6, using the top N samples with the highest ranking after the sorting and the recognition results corresponding to the top N samples to expand the labeled data set, obtaining a final term recognition model from the labeled data set, and outputting the enterprise digital term library based on the final term recognition model.

[0013] According to a method for constructing an enterprise digital terminology library based on semi-supervised learning provided by the present invention, step S1 comprises:

[0014] Collect annual report text data of listed companies within a preset range, and extract management discussion and analysis data from the annual report text data;

[0015] All terms in the management discussion and analysis data are manually annotated sentence by sentence in a sequence annotation manner according to multiple digital dimensions to obtain training model annotation data corresponding to the annual report text data.

[0016] According to a method for constructing a digital terminology library for an enterprise based on semi-supervised learning provided by the present invention, step S2 comprises:

[0017] Segment the annual report text data into single sentences to obtain a set of segmented sentences;

[0018] The word-segmented single sentences are input into the ERNIE model, and the encoded annual report text vector sequence is output.

[0019] According to a method for constructing an enterprise digital terminology library based on semi-supervised learning provided by the present invention, step S3 comprises:

[0020] Perform two linear transformations on the encoded annual report text vector sequence to obtain a first vector representation starting with any term and a second vector representation ending with any term, respectively, wherein the first vector representation is composed of a first vector representation matrix of any term, a vector representation corresponding to any word in the annual report text, and a first vector offset of any term, and the second vector representation is composed of a last vector representation matrix of any term, a vector representation corresponding to any word in the annual report text, and a last vector offset of any term;

[0021] Calculating the probability of a sequence segment from the beginning of any term to the end of any term being any term type entity based on the first vector representation and the second vector representation;

[0022] Determine the transformation matrix of the relative position of the injected digitized term of the initial term recognition model by rotating the position code, and substitute the transformation matrix into the probability of any term type entity;

[0023] Determine a head and tail length threshold of any term, and perform mask processing on a portion of the term exceeding the head and tail length threshold;

[0024] The final output probability is used to determine the term boundary and term type corresponding to any term type entity probability.

[0025] According to a method for constructing an enterprise digital terminology library based on semi-supervised learning provided by the present invention, step S4 comprises:

[0026] Collecting the term recognition results output by the initial term recognition model, and obtaining the number of correctly recognized terms, the number of incorrectly recognized terms, and the number of unrecognized terms in the term recognition results;

[0027] The precision is obtained from the number of correctly recognized terms and the number of incorrectly recognized terms, the recall is obtained from the number of correctly recognized terms and the number of unrecognized terms, and the F1score index is obtained from the precision and the recall;

[0028] After multiple rounds of training, if it is determined that the F1 score indicator has not been improved, the final term recognition model is output.

[0029] According to a method for constructing an enterprise digital terminology library based on semi-supervised learning provided by the present invention, step S5 comprises:

[0030] Based on the initial term recognition model, a final probability matrix is ​​obtained that each annual report text sample data contains any type of term, and a final term output result is obtained according to step S3;

[0031] Assigning a pseudo label to the final term output result to obtain the unlabeled data set;

[0032] The average probability of any term type entity corresponding to the continuous substring fragment of the term in each sample of the unlabeled data set is calculated, and all samples are sorted according to the average value to obtain the sorted annual report text sample data.

[0033] According to a method for constructing an enterprise digital terminology library based on semi-supervised learning provided by the present invention, step S6 comprises:

[0034] Expand the labeled data set using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples, train and correct the initial term recognition model using the expanded annual report text data set, return to step S2, until the preset training end condition in step S4 is met, and output the final term recognition model;

[0035] The enterprise digital terminology library is constructed using the enterprise terminology output by the final terminology recognition model.

[0036] In a second aspect, the present invention further provides a system for constructing an enterprise digital terminology library based on semi-supervised learning, comprising:

[0037] The collection module is used to collect the annual report text data of the enterprise;

[0038] A preprocessing module, used to characterize the annual report text data based on the ERNIE model to obtain an annual report text vector sequence;

[0039] An extraction module, used to extract features from the annual report text vector sequence using a GlobalPointer model, and determine a final output probability of an initial term recognition model, wherein the final output probability is used to identify term boundaries and term types;

[0040] An evaluation module is used to evaluate the initial term recognition model according to a preset evaluation index, and if the preset training end condition is met, the final term recognition model is output, otherwise the processing flow of the annotation module is entered;

[0041] A labeling module, used to perform term recognition on the unlabeled data set based on the initial term recognition model, and to sort all samples by output probability according to a sorting algorithm to obtain sorted annual report text sample data;

[0042] The expansion module is used to expand the labeled data set using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples, obtain a final term recognition model from the labeled data set, and output the enterprise digital term library based on the final term recognition model.

[0043] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for constructing a digital enterprise terminology library based on semi-supervised learning as described in any one of the above-mentioned methods.

[0044] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for constructing a digital enterprise terminology library based on semi-supervised learning.

[0045] The method and system for constructing an enterprise digital terminology library based on semi-supervised learning provided by the present invention can automatically identify professional terms in the text of the enterprise digitalization field by combining deep learning and semi-supervised learning technologies with domain problems, solving the current situation that the terminology recognition method in this field mainly relies on manual recognition and simple similar word expansion, improving the efficiency of terminology recognition and enhancing the scalability of the terminology dictionary. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is a flowchart of a method for constructing a digital terminology library for an enterprise based on semi-supervised learning provided by the present invention;

[0048] Figure 2 It is a framework diagram of the digital terminology recognition method provided by the present invention;

[0049] Figure 3 It is a structural diagram of a digital term recognition model provided by the present invention;

[0050] Figure 4 It is a structural block diagram of the enterprise digital terminology library construction system provided by the present invention;

[0051] Figure 5 It is a structural diagram of the model training module provided by the present invention;

[0052] Figure 6 It is a structural diagram of the model prediction module provided by the present invention;

[0053] Figure 7 It is a structural schematic diagram of the enterprise digital terminology library construction system based on semi-supervised learning provided by the present invention;

[0054] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] In the existing professional terminology processing technology, it is difficult to effectively identify the professional field of enterprise digital terminology. The scalability of digital terminology extraction methods is generally poor. No matter which method is used, it is difficult to fundamentally solve the problems of terminology expansion and new terminology identification. In the general terminology recognition method, it only relies on simple linguistic features and adopts simple statistical methods. It is difficult to adapt to the complex and changeable language environment, and the terminology recognition performance is poor. In the terminology recognition in professional fields, it still relies on large-scale text annotation, which requires a lot of time and financial costs, and the recognition performance of long and difficult cross-domain terms still needs to be improved. Unlike other fields, the construction of the enterprise digital terminology library involved in the present invention is very challenging. First, listed companies belong to a variety of industries and fields, and cross-industry and cross-field term recognition has extremely high performance requirements for models and algorithms; second, corporate digital terms are extremely sparsely distributed in corporate annual report texts, and new terms continue to emerge with the development of the times, which brings great challenges to the accurate recognition of corporate digital terms; third, there are a large number of compound long terms in corporate digital terms, which are usually composed of multiple words or characters with independent meanings, and it is difficult to accurately identify the boundaries of terms; in addition, most previous related studies have used small-scale manual extraction methods to construct term sets, lacking mature large-scale training expectations. Therefore, identifying and extracting relevant terms from annual report texts is a challenging task.

[0057] Therefore, the present invention proposes a method for constructing an enterprise digital terminology library based on deep learning and semi-supervised learning, aiming to effectively identify digitalization-related terms from corporate annual reports with only a small amount of labeled data and to construct a more comprehensive enterprise digital terminology library.

[0058] Figure 1 is a flow chart of a method for constructing a digital enterprise terminology library based on semi-supervised learning provided by an embodiment of the present invention. Figure 1 As shown, including:

[0059] Step S1, collecting the annual report text data of the enterprise;

[0060] Step S2, characterizing the annual report text data based on the ERNIE model to obtain an annual report text vector sequence;

[0061] Step S3, using the GlobalPointer model to perform feature extraction on the annual report text vector sequence, and determining the final output probability of the initial term recognition model, wherein the final output probability is used to identify term boundaries and term types;

[0062] Step S4, evaluating the initial term recognition model according to a preset evaluation index, and if the preset training end condition is met, outputting the final term recognition model, otherwise proceeding to step S5;

[0063] Step S5, performing term recognition on the unlabeled data set based on the initial term recognition model, and sorting all samples by output probability according to the sorting algorithm to obtain sorted annual report text sample data;

[0064] Step S6, using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples to expand the labeled data set, obtain a final term recognition model from the labeled data set, and output the enterprise digital term library based on the final term recognition model.

[0065] Specifically, Figure 2 As shown, the processing steps of the embodiment of the present invention include:

[0066] S1: Collect the annual report text data of the enterprise to label the data of the training model; S2: Use the Enhanced language Representation with Informative Entities (ERNIE) technology to represent the annual report text and vectorize it; S3: Use the GlobalPointer model of the named entity recognition task to extract features from the annual report text, and use the final output probability as the basis for identifying the term boundaries and their types; S4: Evaluate the performance of the model. If the exit condition is met, the final term recognition model is obtained, otherwise proceed to S5; S5: Use the model to identify terms in the unlabeled data set, and sort all samples according to the output probability according to the sorting algorithm; S6: Use the top N recognition results with high confidence to expand the labeled data set, correct the model, and return to S2 until the final exit condition in S4 is met to obtain the final digital term extraction model.

[0067] The above process includes a learning process and a sample selection process. The two processes are complementary and cyclical. The existing small amount of samples can be expanded to a large amount of data sets to achieve high-precision training of the model.

[0068] It is understandable that the deep learning and semi-supervised learning technologies used in the present invention to extract digital enterprise terminology are compared with the previous methods of mainly using manual screening of similar words to expand the enterprise digital terminology dictionary. On the one hand, this type of approach is largely subject to the subjectivity and knowledge level of the summarizer, and the extraction task is arduous, time-consuming, and inefficient, resulting in the limited size of the final digital terminology dictionary. On the other hand, digital terms are extremely sparsely distributed in the text of corporate annual reports, and new terms continue to emerge with the development of the times, which poses a great challenge to the further expansion of the digital terminology dictionary. The present invention uses deep learning technology for the first time to identify relevant terms from the perspective of automatic term extraction, solving the problem of low scalability of previous methods, thereby improving the efficiency of enterprise digital terminology recognition.

[0069] Based on the above embodiment, step S1 includes:

[0070] Collect annual report text data of listed companies within a preset range, and extract management discussion and analysis data from the annual report text data;

[0071] All terms in the management discussion and analysis data are manually annotated sentence by sentence in a sequence annotation manner according to multiple digital dimensions to obtain training model annotation data corresponding to the annual report text data.

[0072] Specifically, when collecting annual report text data of enterprises for labeling data of training models, the embodiments of the present invention mainly collect annual report text data of Chinese listed companies in recent years and extract Management Discussion and Analysis (MD&A) data therein.

[0073] All terms are manually annotated sentence by sentence in the form of sequence annotation according to the five dimensions of digital technology, digital model, digital industry, digital system and digital infrastructure.

[0074] Based on the above embodiment, step S2 includes:

[0075] Segment the annual report text data into single sentences to obtain a set of segmented sentences;

[0076] The word-segmented single sentences are input into the ERNIE model, and the encoded annual report text vector sequence is output.

[0077] Specifically, refer to Figure 3The digital term recognition model structure diagram shown in the figure takes "intelligent environmental protection with AI...intelligent" as an example, and uses ERNIE to characterize the original annual report text. First, the annual report text is divided into words according to single sentences, and the set of words after the single sentence is obtained is sentence = {cls, w1, w2, w3, ..., w n-1 ,w n};in cls represents the token that marks the beginning of a sentence, w t (t=1,2,3,…,n) represents the tth word in the sentence, and n represents the length of the sentence. Input the annual report text sentence of the single sentence into the ERNIE model to obtain the encoded vector sequence The expression is H=[h1 h2 h3 … h n ].in, Represents the i-th word w in the annual report text i The corresponding vector representation.

[0078] Based on the above embodiment, step S3 includes:

[0079] Perform two linear transformations on the encoded annual report text vector sequence to obtain a first vector representation starting with any term and a second vector representation ending with any term, respectively, wherein the first vector representation is composed of a first vector representation matrix of any term, a vector representation corresponding to any word in the annual report text, and a first vector offset of any term, and the second vector representation is composed of a last vector representation matrix of any term, a vector representation corresponding to any word in the annual report text, and a last vector offset of any term;

[0080] Calculating the probability of a sequence segment from the beginning of any term to the end of any term being any term type entity based on the first vector representation and the second vector representation;

[0081] Determine the transformation matrix of the relative position of the injected digitized term of the initial term recognition model by rotating the position code, and substitute the transformation matrix into the probability of any term type entity;

[0082] Determine a head and tail length threshold of any term, and perform mask processing on a portion of the term exceeding the head and tail length threshold;

[0083] The final output probability is used to determine the term boundary and term type corresponding to any term type entity probability.

[0084] Specifically, Figure 3 As shown in the figure, the encoded annual report text vector sequence H is subjected to two linear transformations to obtain the vector representation q starting with term α. i,α and the vector table k at the endi,α :

[0085] q i,α =W q,α h i +b q,α

[0086] k i,α =W k,α h i +b k,α

[0087] in, W q,α and W k,α represents the weight matrix used to calculate the first and last vector representations of terms, b q,α and b k,α For bias.

[0088] Then, by q i,α and k i,α Calculate the probability that the sequence segment from i to j is an entity of type α Finally, by rotating the position code, use The transformation matrix R i The relative position information of digital terms is explicitly injected into the digital term recognition model to enhance the model's sensitivity to term length and span, and to obtain the probability that the final sequence segment from i to j is an α-type entity. Here, i represents the beginning position of the entity, and j represents the end position of the entity. sentence[i:j] represents a continuous substring consisting of the i-th to j-th elements of the annual report text, s α (i,j) represents the probability that the sentence[i:j] sequence segment is an entity of type α.

[0089] In particular, in order to improve the calculation speed and recognition performance of the model, the present invention sets ji≤30, where 30 is an example of the head-tail length threshold, that is, when calculating s α (i, j) is masked for the segments that exceed the length limit. On the one hand, this masking strategy reduces the number of negative samples (non-digital terms) from n(n+1) / 2-k to 30n-435-k (where n is the sample length and k is the number of digital terms in the sample). For longer samples, the number of negative samples is greatly reduced. On the other hand, it also effectively reduces the computational cost and improves the computational speed of the model. Finally, a probability threshold of 0.5 is used to determine whether the sentence[i:j] sequence segment belongs to an α-type entity. If s α (i,j)≥0.5, then the fragment is an α-type entity; if s α (i,j)≤0.5, the fragment is a non-entity or non-α type entity.

[0090] The present invention uses an improved GlobalPointer model to extract features from annual report texts, and has better recognition capabilities for long, difficult, and compound terms involved in digital terms. Unlike other fields, there are a large number of compound long terms in digital terms. Such terms are usually composed of multiple words or characters with independent meanings, and it is difficult to accurately identify term boundaries. The traditional Conditional Random Field (CRF) model is difficult to effectively identify such long terms, while GlobalPointer treats the beginning and end of the term as a whole, embeds its position information, and uses the idea of ​​global normalization to extract the deep features of the text. The extraction of digital terms based on GlobalPointer can achieve better performance than the traditional CRF model.

[0091] Based on the above embodiment, step S4 includes:

[0092] Collecting the term recognition results output by the initial term recognition model, and obtaining the number of correctly recognized terms, the number of incorrectly recognized terms, and the number of unrecognized terms in the term recognition results;

[0093] The precision is obtained from the number of correctly recognized terms and the number of incorrectly recognized terms, the recall is obtained from the number of correctly recognized terms and the number of unrecognized terms, and the F1score index is obtained from the precision and the recall;

[0094] After multiple rounds of training, if it is determined that the F1 score indicator has not been improved, the final term recognition model is output.

[0095] Specifically, in the embodiment of the present invention, the performance of the model is evaluated. If the model training exit condition is met, the model training is terminated to obtain the final term recognition model. Otherwise, the model sample expansion stage is entered.

[0096] In the embodiment of the present invention, the criterion for model evaluation is whether the boundary of the identified term is correct and whether the type of the term is correctly labeled. Therefore, the embodiment of the present invention uses three indicators, namely, precision, recall and F1score, to measure the model performance:

[0097]

[0098]

[0099]

[0100] Among them, TP represents the number of correctly recognized terms, that is, the number of terms with correct boundaries and correct type annotations, FP represents the number of incorrectly recognized terms, and FN represents the number of unrecognized terms.

[0101] It should be noted that the exit condition set in the embodiment of the present invention is: after multiple consecutive rounds of training, the F1-score of the model on the test set has not improved.

[0102] Based on the above embodiment, step S5 includes:

[0103] Based on the initial term recognition model, a final probability matrix is ​​obtained that each annual report text sample data contains any type of term, and a final term output result is obtained according to step S3;

[0104] Assigning a pseudo label to the final term output result to obtain the unlabeled data set;

[0105] The average probability of any term type entity corresponding to the continuous substring fragment of the term in each sample of the unlabeled data set is calculated, and all samples are sorted according to the average value to obtain the sorted annual report text sample data.

[0106] Specifically, the present invention uses the model to predict the final probability matrix of each sample containing a certain type of term, and obtains the final result according to step S3, assigns a pseudo label, as shown in step S3, and calculates the probability s of each sample sentence[i:j] segment α The average of (i,j)≥0.5, sort all samples by size.

[0107] Based on the above embodiment, step S6 includes:

[0108] Expand the labeled data set using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples, train and correct the initial term recognition model using the expanded annual report text data set, return to step S2, until the preset training end condition in step S4 is met, and output the final term recognition model;

[0109] The enterprise digital terminology library is constructed using the enterprise terminology output by the final terminology recognition model.

[0110] Specifically, an embodiment of the present invention selects the top 1000 samples with the highest confidence in the prediction results, expands the labeled data set, and based on the expanded data set, corrects the model and returns to step S2 until the final exit condition in step S4 is met, thereby obtaining the final enterprise digital terminology extraction model.

[0111] The present invention solves the current problem of insufficient training corpus by using semi-supervised learning technology combined with a pre-trained language model. Most previous related studies have used small-scale manual extraction to construct a term set, resulting in a lack of mature large-scale training corpus in this field, and traditional machine learning methods often find it difficult to learn enough information from limited samples. In order to solve the above problems, on the one hand, the present invention adopts the ERNIE pre-trained language model to represent the text, and fine-tunes the linguistic knowledge learned by the ERNIE model on large-scale text during the pre-training process, and learns rich semantic representations in the case of small samples, thereby alleviating the above problems; on the other hand, the present invention adopts semi-supervised learning technology, using a large amount of unlabeled data to improve the effect of supervised learning. This strategy helps to improve the generalization performance of the model, and can achieve better performance with the same small number of labeled samples.

[0112] Figure 4 is a structural block diagram of a system for constructing a digital terminology library for an enterprise provided by an embodiment of the present invention. Figure 4 As shown, it includes: a model training module and a model prediction module;

[0113] The model training module is used to train the enterprise digital terminology recognition model, including collecting the annual report text data of Chinese listed companies in recent years and extracting the management discussion and analysis data, data preprocessing, model training, training set correction and terminology storage management, and finally obtaining a model that can be used for enterprise digital terminology recognition;

[0114] The model prediction module is used to identify digitalization-related terms and their types from a large amount of input annual report texts and build a system's enterprise digitalization terminology library.

[0115] The enterprise digital terminology library construction system provided by the present invention collects the annual report text data of Chinese listed companies in recent years and extracts the management discussion and analysis data therein as samples for model training, and then trains and corrects the model through the technical framework mentioned in Example 1 to obtain an enterprise digital terminology recognition model. The trained model can be used to recognize enterprise digital terms, and the text data to be recognized is input into the recognition model for recognition, and the digital-related terms and their types in the text are accurately recognized, supporting the development of quantitative research related to digitalization.

[0116] Based on the above embodiment, the model training module is as follows: Figure 5 As shown, it includes: data acquisition module, data preprocessing module, model training module and term base management module;

[0117] The data acquisition module collects the annual report text data of Chinese listed companies in recent years and extracts the management discussion and analysis data as samples for model training;

[0118] Data preprocessing module, which performs data preprocessing operations before feature extraction, data annotation and text normalization, etc.;

[0119] Model training module: trains and modifies the model according to the technical framework to obtain the final digital term recognition model;

[0120] The terminology library management module stores all the terms identified during the above model prediction and training process to obtain a digital terminology library.

[0121] Based on the above embodiment, the model prediction module is as follows: Figure 6 As shown, it includes: a data input module, a data preprocessing module, a terminology recognition module, a user interface module, an analysis and visualization module, and an update and maintenance module;

[0122] A data input module obtains the text that needs to be digitized for term recognition;

[0123] The data preprocessing module performs the same data preprocessing as in the training module on the text to be recognized;

[0124] A term recognition module, which uses the digital term recognition model finally obtained in the training module to recognize all digital terms and their types in the input text;

[0125] The user interface module provides a user interface to facilitate users to use the digital terminology library construction system, allowing users to submit text, search for terms, or browse recognized terms.

[0126] Analysis and visualization modules, which allow viewing basic statistics and other useful information about terms in the library;

[0127] The update and maintenance module determines whether the term identified by the current user appears in the existing digital terminology database. If it is a new term, it will be submitted to the background for manual review and then stored in the database to maintain the accuracy and effectiveness of the term.

[0128] The following is a description of the enterprise digital terminology library construction system based on semi-supervised learning provided by the present invention. The enterprise digital terminology library construction system based on semi-supervised learning described below and the enterprise digital terminology library construction method based on semi-supervised learning described above can refer to each other.

[0129] Figure 7 is a schematic diagram of the structure of a system for constructing an enterprise digital terminology library based on semi-supervised learning provided by an embodiment of the present invention. Figure 7 As shown, it includes: a collection module 71, a preprocessing module 72, an extraction module 73, an evaluation module 74, a labeling module 75 and an expansion module 76, wherein:

[0130] The collection module 71 is used to collect the annual report text data of the enterprise; the preprocessing module 72 is used to characterize the annual report text data based on the ERNIE model to obtain the annual report text vector sequence; the extraction module 73 is used to extract features from the annual report text vector sequence using the GlobalPointer model to determine the final output probability of the initial term recognition model, and the final output probability is used to identify term boundaries and term types; the evaluation module 74 is used to evaluate the initial term recognition model according to preset measurement indicators, and if the preset training end conditions are met, the final term recognition model is output, otherwise the processing flow of the annotation module is entered; the annotation module 75 is used to perform term recognition on the unlabeled data set based on the initial term recognition model, and sort all samples according to the output probability according to the sorting algorithm to obtain the sorted annual report text sample data; the expansion module 76 is used to expand the labeled data set using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples, and obtain the final term recognition model from the labeled data set, and output the enterprise digital term library based on the final term recognition model.

[0131] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8As shown, the electronic device may include: a processor (processor) 810, a communication interface (Communications Interface) 820, a memory (memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logic instructions in the memory 830 to execute a method for constructing a digital terminology library for an enterprise based on semi-supervised learning, the method comprising: step S1, collecting annual report text data of the enterprise; step S2, characterizing the annual report text data based on the ERNIE model to obtain an annual report text vector sequence; step S3, using the GlobalPointer model to extract features of the annual report text vector sequence to determine the final output probability of the initial term recognition model, the final output probability is used to identify term boundaries and term types; step S4, evaluating the initial term recognition model according to a preset measurement indicator, if the preset training end condition is met, then outputting the final term recognition model, otherwise entering step S5; step S5, performing term recognition on the unlabeled data set based on the initial term recognition model, sorting all samples according to the output probability according to the sorting algorithm, and obtaining sorted annual report text sample data; step S6, using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples to expand the labeled data set, obtaining the final term recognition model from the labeled data set, and outputting the digital terminology library for the enterprise based on the final term recognition model.

[0132] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0133] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for constructing a digital enterprise terminology library based on semi-supervised learning provided by the above methods, the method comprising: step S1, collecting the annual report text data of the enterprise; step S2, characterizing the annual report text data based on the ERNIE model to obtain an annual report text vector sequence; step S3, using the GlobalPointer model to extract features from the annual report text vector sequence, and determining the final output probability of the initial term recognition model, the final output probability is used to identify term boundaries and Term type; Step S4, evaluate the initial term recognition model according to the preset measurement index, if the preset training end condition is met, output the final term recognition model, otherwise enter step S5; Step S5, perform term recognition on the unlabeled data set based on the initial term recognition model, sort all samples according to the output probability according to the sorting algorithm, and obtain the sorted annual report text sample data; Step S6, use the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples to expand the labeled data set, obtain the final term recognition model from the labeled data set, and output the enterprise digital term library based on the final term recognition model.

[0134] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0135] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing an enterprise digital terminology library based on semi-supervised learning, characterized in that: include: Step S1, collecting the annual report text data of the enterprise; Step S2, characterizing the annual report text data based on the information entity enhanced language representation ERNIE model to obtain an annual report text vector sequence; Step S3, using the GlobalPointer model of the named entity recognition task to extract features from the annual report text vector sequence, and determining the final output probability of the initial term recognition model, wherein the final output probability is used to identify term boundaries and term types; Step S4, evaluating the initial term recognition model according to a preset evaluation index, and if the preset training end condition is met, outputting the final term recognition model, otherwise proceeding to step S5; Step S5, performing term recognition on the unlabeled data set based on the initial term recognition model, and sorting all samples by output probability according to the sorting algorithm to obtain sorted annual report text sample data; Step S6, using the top N samples with the highest ranking after the sorting and the recognition results corresponding to the top N samples to expand the labeled data set, obtaining a final term recognition model from the labeled data set, and outputting the enterprise digital term library based on the final term recognition model; Step S1 includes: Collect annual report text data of listed companies within a preset range, and extract management discussion and analysis data from the annual report text data; Manually annotate all terms in the management discussion and analysis data sentence by sentence in a sequence annotation manner according to multiple digital dimensions to obtain training model annotation data corresponding to the annual report text data; Step S3 includes: Perform two linear transformations on the encoded annual report text vector sequence to obtain a first vector representation starting with any term and a second vector representation ending with any term, respectively, wherein the first vector representation is composed of a first vector representation matrix of any term, a vector representation corresponding to any word in the annual report text, and a first vector offset of any term, and the second vector representation is composed of a last vector representation matrix of any term, a vector representation corresponding to any word in the annual report text, and a last vector offset of any term; Calculating the probability of a sequence segment from the beginning of any term to the end of any term being any term type entity based on the first vector representation and the second vector representation; Determine the transformation matrix of the relative position of the injected digitized term of the initial term recognition model by rotating the position code, and substitute the transformation matrix into the probability of any term type entity; Determine a head and tail length threshold of any term, and perform mask processing on a portion of any term exceeding the head and tail length threshold; Determine the term boundary and term type corresponding to any term type entity probability using the final output probability; Step S6 includes: Expand the labeled data set using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples, train and correct the initial term recognition model using the expanded annual report text data set, return to step S2, until the preset training end condition in step S4 is met, and output the final term recognition model; The enterprise digital terminology library is constructed using the enterprise terminology output by the final terminology recognition model.

2. The method for constructing an enterprise digital terminology library based on semi-supervised learning according to claim 1, characterized in that: Step S2 includes: Segment the annual report text data into single sentences to obtain a set of segmented sentences; The word-segmented single sentences are input into the ERNIE model, and the encoded annual report text vector sequence is output.

3. The method for constructing an enterprise digital terminology library based on semi-supervised learning according to claim 1, characterized in that: Step S4 includes: Collecting the term recognition results output by the initial term recognition model, and obtaining the number of correctly recognized terms, the number of incorrectly recognized terms, and the number of unrecognized terms in the term recognition results; The precision is obtained from the number of correctly recognized terms and the number of incorrectly recognized terms, the recall is obtained from the number of correctly recognized terms and the number of unrecognized terms, and the F1score index is obtained from the precision and the recall; After multiple rounds of training, if it is determined that the F1 score indicator has not been improved, the final term recognition model is output.

4. The method for constructing an enterprise digital terminology library based on semi-supervised learning according to claim 1, characterized in that: Step S5 includes: Based on the initial term recognition model, a final probability matrix is ​​obtained that each annual report text sample data contains any type of term, and a final term output result is obtained according to step S3; Assigning a pseudo label to the final term output result to obtain the unlabeled data set; The average probability of any term type entity corresponding to the continuous substring fragment of the term in each sample of the unlabeled data set is calculated, and all samples are sorted according to the average value to obtain the sorted annual report text sample data.

5. A system for building a digital terminology library for an enterprise based on semi-supervised learning, characterized in that: include: The collection module is used to collect the annual report text data of the enterprise; A preprocessing module, used to characterize the annual report text data based on the ERNIE model to obtain an annual report text vector sequence; An extraction module, used to extract features from the annual report text vector sequence using a GlobalPointer model, and determine a final output probability of an initial term recognition model, wherein the final output probability is used to identify term boundaries and term types; An evaluation module is used to evaluate the initial term recognition model according to a preset evaluation index, and if the preset training end condition is met, the final term recognition model is output, otherwise the processing flow of the annotation module is entered; A labeling module, used to perform term recognition on the unlabeled data set based on the initial term recognition model, and to sort all samples by output probability according to a sorting algorithm to obtain sorted annual report text sample data; An expansion module, used to expand the labeled data set by using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples, obtain a final term recognition model from the labeled data set, and output the enterprise digital term library based on the final term recognition model; The collection module is specifically used for: Collect annual report text data of listed companies within a preset range, and extract management discussion and analysis data from the annual report text data; Manually annotate all terms in the management discussion and analysis data sentence by sentence in a sequence annotation manner according to multiple digital dimensions to obtain training model annotation data corresponding to the annual report text data; The extraction module is specifically used for: Perform two linear transformations on the encoded annual report text vector sequence to obtain a first vector representation starting with any term and a second vector representation ending with any term, respectively, wherein the first vector representation is composed of a first vector representation matrix of any term, a vector representation corresponding to any word in the annual report text, and a first vector offset of any term, and the second vector representation is composed of a last vector representation matrix of any term, a vector representation corresponding to any word in the annual report text, and a last vector offset of any term; Calculating the probability of a sequence segment from the beginning of any term to the end of any term being any term type entity based on the first vector representation and the second vector representation; Determine the transformation matrix of the relative position of the injected digitized term of the initial term recognition model by rotating the position code, and substitute the transformation matrix into the probability of any term type entity; Determine a head and tail length threshold of any term, and perform mask processing on a portion of any term exceeding the head and tail length threshold; Determine the term boundary and term type corresponding to any term type entity probability using the final output probability; The expansion module is specifically used for: Expand the labeled data set using the top N samples with the highest ranking after sorting and the recognition results corresponding to the top N samples, train and correct the initial term recognition model using the expanded annual report text data set, return to step S2, until the preset training end condition in step S4 is met, and output the final term recognition model; The enterprise digital terminology library is constructed using the enterprise terminology output by the final terminology recognition model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for constructing an enterprise digital terminology library based on semi-supervised learning as described in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for constructing an enterprise digital terminology library based on semi-supervised learning as described in any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • Event extraction method and device

    CN116663545A