A label recognition method and device based on industry knowledge
By using a pre-trained model and the deep neural network TextCNN to automatically match custom labels with standard labels, the problem of non-standard labels in vertical industries has been solved, achieving efficient and accurate label matching and information standardization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING PERCENT INFORMATION TECH CO LTD
- Filing Date
- 2023-01-29
- Publication Date
- 2026-06-05
AI Technical Summary
In vertical industries, provincial and ministerial-level systems have standardized labels, while city, district, and county-level systems have non-standard labels, and there are even issues with pinyin rules, initial letter problems, and empty labels. Existing technologies rely on manual observation of sample data to summarize rule logic, which leads to inaccurate label matching and is labor-intensive.
We employ an industry-knowledge-based label recognition method. Through a pre-trained model and the deep neural network TextCNN, we construct a standard label hierarchy, perform data preprocessing, vectorization, feature extraction, and classification, and use the Softmax layer for final classification, automatically matching custom labels with standard labels.
It improves the accuracy of tag matching, reduces human intervention, lowers the reliance on expert experience, and achieves tag standardization and facilitates information exchange.
Smart Images
Figure CN116089610B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computers, specifically a label recognition method and apparatus based on industry knowledge. Background Technology
[0002] With the rise and rapid development of the Internet and artificial intelligence, text data has exploded. When setting tags for documents, companies in the industry customize the tags according to their own data. However, in the broader industry, with the implementation of artificial intelligence, standard tags for each industry have already been set. There is an urgent need for an efficient technology to match the custom tags of each company with the standard tags stipulated in the industry, so as to help companies or organizations quickly obtain standard and useful information.
[0003] Tag recognition technology refers to the use of natural language processing algorithms to automatically identify the standard tags to which data units belong and match them with industry-standard tags. This allows people to understand the true standard meaning of tag data through the tags, greatly facilitating our understanding of the semantic information in text data.
[0004] The technical problem this proposal aims to solve is:
[0005] Regarding the data in the database, in vertical industries, the provincial and ministerial level systems are relatively standardized and the labels are relatively accurate; in the city, district, and county level systems, the labels are relatively non-standard, and the labels may have problems such as pinyin rules and initial letters; in single actual business systems, there are even empty labels. This technology solves the problem of standard label recognition.
[0006] Currently, to determine the standard entity name corresponding to a given entity name, it's necessary to manually observe the characteristics of sample data consisting of non-standard entity names and standard entity names, summarize some rule logic, and then determine the standard tag corresponding to the custom tag based on these rule logics. However, the manually observed sample data is relatively limited. Therefore, the rule logic formulated based on manual observation cannot cover all situations, resulting in some non-standard custom tags failing to match the corresponding standard tags. Moreover, summarizing rule logic through manual observation of sample data requires a high level of experience and consumes significant human resources.
[0007] Although machine learning methods have not yet been applied, they cannot capture the deep semantic information hidden in text. Summary of the Invention
[0008] The purpose of this invention is to address the problem of standard label recognition in databases. In vertical industries, provincial and ministerial-level systems are relatively standardized and their labels are relatively accurate; in city, district, and county-level systems, the labels are relatively non-standard and may have issues such as incorrect spelling rules or initial letters; in single-business systems, there are even empty labels. This invention aims to solve the problem of standard label recognition.
[0009] A label recognition method based on industry knowledge includes the following steps:
[0010] Step 1: Sample the raw data to obtain data from the business system, and construct a hierarchical relationship for the standard labels according to the standard to obtain one or more sets of data from the business system and the standard labels corresponding to the data in the standard library;
[0011] Step 2: Data preprocessing, adding corresponding standard labels to the collected data tables, fields, and sample data under the fields;
[0012] Step 3: Vectorize the data processed in Step 2. Obtain the vectorized representation of the given data after training the pre-trained model.
[0013] Step 4: Extract feature vectors from the vectors in Step 3. Using a deep neural network, select feature vectors suitable for data classification by setting filters of different sizes and pooling operations. The fully connected layer acts as a classifier by adding hidden layers after the fusion layer and the final Softmax layer to perform the final classification of the text.
[0014] Step 5: Archive the standard category labels and data obtained in Step 4.
[0015] In the above technical solution, step 1 specifically includes the following steps:
[0016] Step 1.1: Obtain data from the business system;
[0017] Step 1.2: Obtain the standard label from the industry standard;
[0018] Step 1.3: Construct a standard label hierarchy based on the specifications in the industry standards.
[0019] In the above technical solution, step 2 specifically includes the following steps:
[0020] Step 2.1: The data unit in the business system includes a data table, fields, and data stored under the corresponding fields.
[0021] Any one or more of the sample data;
[0022] Step 2.2: Attach standard label types to the sample data in the business system.
[0023] In the above technical solution, step 3 specifically includes the following steps:
[0024] Step 3.1: For the data processed in Step 2, use the pre-trained model to convert it into a vector representation, and then concatenate all the vectors together to form a vector matrix, which is used as the input to the subsequent deep neural network model;
[0025] In the above technical solution, step 4 specifically includes the following steps:
[0026] Step 4.1: Use the deep neural network TextCNN to extract feature vectors from the vector matrix in Step 3. The steps can be divided into convolutional layers, pooling layers, fusion layers, and fully connected layers.
[0027] Step 4.2: The convolutional layer is responsible for extracting the intrinsic features of the text. TextCNN uses multiple convolutional kernels of different sizes to obtain different feature representations. The convolutional kernels filter and extract features from words within the window size range in the sentence in the form of a sliding window. The obtained features are then concatenated to obtain the overall feature vector.
[0028] Step 4.3: Max pooling is used in the pooling layer, that is, the maximum value is selected from the feature vectors generated by each sliding window in step 4.2, and then these maximum values are concatenated to form the feature vector representation;
[0029] Step 4.4: The fusion layer concatenates the features obtained from the pooling layer in step 4.3 and merges them into a vector that is more representative of the text vector.
[0030] Step 4.5: The fully connected layer acts as a classifier by adding a hidden layer after the fusion layer and the final Softmax layer to classify the vectors obtained in Step 4.4.
[0031] In the above technical solution, step 5 specifically includes the following steps:
[0032] Step 5.1: Take the standard label with the highest probability after the function processing in Step 4 as the final label.
[0033] Step 5.2: The trained model can be used to classify and identify the original data units to be classified, and then archive them after labeling;
[0034] Step 5.3: If a field belongs to a second-level tag under a certain level tag, it also belongs to that level tag.
[0035] The present invention also provides a label recognition device based on industry knowledge, comprising the following modules:
[0036] Standard Tag Module: Samples the raw data, obtains data from the business system, and constructs a hierarchical relationship of standard tags according to the standard to obtain one or more sets of data from the business system and the standard tags corresponding to the data in the standard library;
[0037] Data preprocessing module: Data preprocessing, which adds corresponding standard labels to the collected data tables, fields, and sample data under the fields;
[0038] Vectorization module: Vectorizes the data processed by the data preprocessing module. It obtains a vectorized representation of the given data after training the pre-trained model, resulting in a vector matrix.
[0039] Classification module: Feature vectors are extracted from the vectorization module using a deep neural network. By setting filters of different sizes and pooling operations, feature vectors suitable for data classification are selected. The fully connected layer acts as a classifier by adding hidden layers after the fusion layer and the final Softmax layer to perform the final classification of the text.
[0040] Archive module: Archives the standard category labels and data obtained from the classification module.
[0041] The standard label module in the above device specifically includes the following steps:
[0042] Step 1.1: Obtain data from the business system;
[0043] Step 1.2: Obtain the standard label from the industry standard;
[0044] Step 1.3: Construct a standard label hierarchy based on the specifications in the industry standards.
[0045] In the above-mentioned device, the data preprocessing module specifically includes the following steps:
[0046] Step 2.1: The data unit in the business system includes any one or more of the following: data table, field, and sample data stored under the corresponding field;
[0047] Step 2.2: Attach standard label types to the sample data in the business system.
[0048] In the above-mentioned device, the vectorization module specifically includes the following steps:
[0049] Step 3.1: For the data processed in Step 2, use the pre-trained model to convert it into a vector representation, and then concatenate all the vectors together to form a vector matrix, which is used as the input to the subsequent deep neural network model;
[0050] The classification module specifically includes the following steps:
[0051] Step 4.1: Use the deep neural network TextCNN to extract feature vectors from the vector matrix in Step 3. The steps can be divided into convolutional layers, pooling layers, fusion layers, and fully connected layers.
[0052] Step 4.2: The convolutional layer is responsible for extracting the intrinsic features of the text. TextCNN uses multiple convolutional kernels of different sizes to obtain different feature representations. The convolutional kernels filter and extract features from words within the window size range in the sentence in the form of a sliding window. The obtained features are then concatenated to obtain the overall feature vector.
[0053] Step 4.3: Max pooling is used in the pooling layer, that is, the maximum value is selected from the feature vectors generated by each sliding window in step 4.2, and then these maximum values are concatenated to form the feature vector representation;
[0054] Step 4.4: The fusion layer concatenates the feature vectors obtained from the pooling layer in step 4.3 and merges them into a vector that is more representative of the text vector.
[0055] Step 4.5: The fully connected layer acts as a classifier by adding a hidden layer after the fusion layer and the final Softmax layer to classify the vectors obtained in Step 4.4;
[0056] The archiving module specifically includes the following steps:
[0057] Step 5.1: Take the standard label with the highest probability after the function processing in Step 4 as the final label.
[0058] Step 5.2: The trained model can be used to classify and identify the original data units to be classified, and then archive them after labeling;
[0059] Step 5.3: If a field belongs to a second-level tag under a certain level tag, it also belongs to that level tag.
[0060] Because this invention employs the above-mentioned technical means, it possesses the following beneficial effects:
[0061] 1. Combine pre-trained models for large-scale corpus pre-training to obtain complete data features, and apply the learned representation features to downstream tasks to significantly improve their performance.
[0062] 2. Deep neural network models not only have lower requirements for feature extraction, do not require expert participation, and have less human intervention, but also extract features more comprehensively.
[0063] 3. Tag matching corresponds to the mapping relationship between custom tags and standard tags, which can make the information of various enterprises more standardized and facilitate communication within the industry.
[0064] 4. Current technology integrates pre-trained models and deep neural network models. This allows for large-scale corpus pre-training using pre-trained models, leveraging the advantages of both comprehensive features and deep neural network feature extraction. These hidden features reflect the information carried by each level of the label. Furthermore, the two-stage standard label recognition, combining the global learning capability of pre-trained models with the local feature extraction capability of deep neural networks, is not merely a matter of combining the strengths of both. Attached Figure Description
[0065] Figure 1 A flowchart for model classification.
[0066] Figure 2 This is a diagram of the TextCNN model. Detailed Implementation
[0067] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.
[0068] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.
[0069] A label recognition method based on industry knowledge includes the following steps:
[0070] Step 1: Obtain data. Sample the raw data, including structured or unstructured data, document data, and Internet data. Construct a hierarchical relationship for standard tags based on standards to obtain one or more sets of data from business systems and the standard tags corresponding to the data in the standard library.
[0071] Step 2: Data preprocessing. Add corresponding standard labels to the collected fields and tables, including Chinese characters, numbers, English characters, sample data, etc.
[0072] Step 3: Data vectorization. The given data is trained using the pre-trained BERT model to obtain a vectorized representation.
[0073] Step 4: Feature vector extraction. Using the deep neural network TextCNN, feature vectors suitable for data classification are selected by setting filters of different sizes and pooling operations. The fully connected layer acts as a classifier by adding hidden layers after the fusion layer and the final Softmax layer to perform the final classification of the text.
[0074] Step 5: Tag and archive the data.
[0075] Step 1 above specifically includes the following steps:
[0076] Step 1.1: Obtain data from the business system, including structured or unstructured data, document data, and internet data;
[0077] Step 1.2: Obtain the standard label from the industry standard;
[0078] Step 1.3: Construct a standard tag hierarchy based on the specifications in the industry standards. For example, in government data tags, the first-level tags include data, enterprise, and person. Under the enterprise tag, there are attribute type and feature type. Under the attribute type tag, there are enterprise nature, industry classification, whether it is a group enterprise, etc.
[0079] Step 2 above specifically includes the following steps:
[0080] Step 2.1: The data units in the business system include any one or more of the following: data tables, fields, sample data, etc.
[0081] Step 2.2: Attach standard label types to the data in the business system, such as "37 years old", corresponding to the first-level label "person", the second-level label "feature", and the third-level label "youth: 18-40 years old".
[0082] Step 3 above specifically includes the following steps:
[0083] Step 3.1: For the given data, use the pre-trained model BERT to transform it into a vector representation, and then concatenate all the vectors together to form a vector matrix as the input to the subsequent deep neural network model.
[0084] Step 4 above specifically includes the following steps:
[0085] Step 4.1: Use the deep neural network TextCNN. The steps can be divided into convolutional layers, pooling layers, fusion layers, and fully connected layers.
[0086] Step 4.2: The convolutional layer is responsible for extracting the intrinsic features of the text. TextCNN uses multiple convolutional kernels of different sizes to obtain representations of different features. The convolutional kernels filter and extract features from words within a window of the sentence in the form of a sliding window, and then concatenate the obtained features to obtain the overall feature vector. Here, the TextCNN model contains three filters with convolutional kernel sizes of 2, 3, and 4. These filters transform a 3×3×1 node matrix into a unit node matrix. Assuming... This represents the weight of the i-th node in the output unit node matrix, used to filter the input node (x, y). Let represent the bias term parameter of the i-th output node, then the value of the i-th node in the identity matrix is... for;
[0087] (1)
[0088] In formula (1), For the values of node (x, y) in the filter, This is the activation function. All The resulting unit vector is the feature map extracted by the convolutional layer, denoted as A, and serves as the input to the pooling layer.
[0089] Step 4.3: Max pooling is used in the pooling layer, which involves selecting the maximum value from the feature vectors generated by each sliding window and then concatenating these maximum values to form a vector representation. Pooling effectively reduces the number of parameters in the network, speeds up the computation process, and also reduces the risk of overfitting.
[0090] Step 4.4: The fusion layer concatenates the features obtained from the pooling layer and merges them into a vector that is more representative of the text vector.
[0091] Step 4.5: The fully connected layer acts as a classifier by adding a hidden layer and a final Softmax layer after the fusion layer to classify the vectors. The Softmax function is used to obtain the probability of each category. This function maps the scores of each category label learned by the model to a range of 0 to 1, and the sum of the scores of all category labels is 1. During prediction, the label with the highest score after processing by the Softmax function is selected. The calculation formula is as follows:
[0092] (2)
[0093] Where i is the output value of the i-th node, and n is the number of output nodes, i.e., the number of categories. The Softmax function can be used to convert the output values of multi-class classification into a probability distribution ranging from [0, 1] with a sum of 1.
[0094] Step 5 above specifically includes the following steps:
[0095] Step 5.1: Select the standard label with the highest probability after processing by the Softmax function as the final label;
[0096] Step 5.2: The trained model can be used to classify and identify the original data units to be classified, and then archive them after labeling;
[0097] Step 5.3: If a field belongs to a second-level tag under a certain level tag, it also belongs to that level tag.
[0098] As another embodiment of the present invention, the present invention also provides a label recognition device based on industry knowledge, comprising the following modules:
[0099] Standard Tag Module: Samples the raw data, obtains data from the business system, and constructs a hierarchical relationship of standard tags according to the standard to obtain one or more sets of data from the business system and the standard tags corresponding to the data in the standard library;
[0100] Data preprocessing module: Data preprocessing, which adds corresponding standard labels to the collected data tables, fields, and sample data under the fields;
[0101] Vectorization module: Vectorizes the data processed by the data preprocessing module. It obtains a vectorized representation of the given data after training the pre-trained model, resulting in a vector matrix.
[0102] Classification module: Feature vectors are extracted from the vectorization module using a deep neural network. By setting filters of different sizes and pooling operations, feature vectors suitable for data classification are selected. The fully connected layer acts as a classifier by adding hidden layers after the fusion layer and the final Softmax layer to perform the final classification of the text.
[0103] Archive module: Archives the standard category labels and data obtained from the classification module.
[0104] The standard label module in the above device specifically includes the following steps:
[0105] Step 1.1: Obtain data from the business system;
[0106] Step 1.2: Obtain the standard label from the industry standard;
[0107] Step 1.3: Construct a standard label hierarchy based on the specifications in the industry standards.
[0108] In the above-mentioned device, the data preprocessing module specifically includes the following steps:
[0109] Step 2.1: The data unit in the business system includes any one or more of the following: data table, field, and sample data stored under the corresponding field;
[0110] Step 2.2: Attach standard label types to the sample data in the business system.
[0111] In the above-mentioned device, the vectorization module specifically includes the following steps:
[0112] Step 3.1: For the data processed in Step 2, use the pre-trained model to convert it into a vector representation, and then concatenate all the vectors together to form a vector matrix, which is used as the input to the subsequent deep neural network model;
[0113] The classification module specifically includes the following steps:
[0114] Step 4.1: Use the deep neural network TextCNN to extract feature vectors from the vector matrix in Step 3. The steps can be divided into convolutional layers, pooling layers, fusion layers, and fully connected layers.
[0115] Step 4.2: The convolutional layer is responsible for extracting the intrinsic features of the text. TextCNN uses multiple convolutional kernels of different sizes to obtain different feature representations. The convolutional kernels filter and extract features from words within the window size range in the sentence in the form of a sliding window. The obtained features are then concatenated to obtain the overall feature vector.
[0116] Step 4.3: Max pooling is used in the pooling layer, that is, the maximum value is selected from the feature vectors generated by each sliding window in step 4.2, and then these maximum values are concatenated to form the feature vector representation;
[0117] Step 4.4: The fusion layer concatenates the feature vectors obtained from the pooling layer in step 4.3 and merges them into a vector that is more representative of the text vector.
[0118] Step 4.5: The fully connected layer acts as a classifier by adding a hidden layer after the fusion layer and the final Softmax layer to classify the vectors obtained in Step 4.4;
[0119] The archiving module specifically includes the following steps:
[0120] Step 5.1: Take the standard label with the highest probability after the function processing in Step 4 as the final label.
[0121] Step 5.2: The trained model can be used to classify and identify the original data units to be classified, and then archive them after labeling;
[0122] Step 5.3: If a field belongs to a second-level tag under a certain level tag, it also belongs to that level tag.
Claims
1. A label recognition method based on industry knowledge, characterized in that, Includes the following steps: Step 1: Sample the raw data to obtain data from the business system, and construct a hierarchical relationship for the standard labels according to the standard to obtain one or more sets of data from the business system and the standard labels corresponding to the data in the standard library; Step 1 specifically includes the following steps: Step 1.1: Obtain data from the business system; Step 1.2: Obtain the standard label from the industry standard; Step 1.3: Construct a standard label hierarchy based on the specifications in industry standards; Step 2: Data preprocessing, adding corresponding standard labels to the collected data tables, fields, and sample data under the fields; Step 3: Vectorize the data processed in Step 2. After training the given data with a pre-trained model, obtain a vectorized representation and get a vector matrix. Step 4: Extract feature vectors from the vectors in Step 3. Using a deep neural network, select feature vectors suitable for data classification by setting filters of different sizes and pooling operations. The fully connected layer acts as a classifier by adding hidden layers after the fusion layer and the final Softmax layer to perform the final classification of the text. Step 5: Archive the standard category labels and data obtained in Step 4; Step 5 specifically includes the following steps: Step 5.1: Take the standard label with the highest probability after function processing in Step 4 as the final label; Step 5.2: The trained model can be used to classify and identify the original data units to be classified, and then archive them after labeling; Step 5.3: If a field belongs to a second-level tag under a certain level tag, it also belongs to that level tag.
2. The label recognition method based on industry knowledge according to claim 1, characterized in that, Step 2 specifically includes the following steps: Step 2.1: The data unit in the business system includes any one or more of the following: data table, field, and sample data stored under the corresponding field; Step 2.2: Attach standard label types to the sample data in the business system.
3. The label recognition method based on industry knowledge according to claim 1, characterized in that, Step 3 specifically includes the following steps: The data processed in step 2 is transformed into a vector representation using a pre-trained model, and then all vectors are concatenated together to form a vector matrix, which serves as the input to the subsequent deep neural network model.
4. The label recognition method based on industry knowledge according to claim 1, characterized in that, Step 4 specifically includes the following steps: Step 4.1: Use the deep neural network TextCNN to extract feature vectors from the vector matrix in Step 3. The steps can be divided into convolutional layers, pooling layers, fusion layers, and fully connected layers. Step 4.2: The convolutional layer is responsible for extracting the intrinsic features of the text. TextCNN uses multiple convolutional kernels of different sizes to obtain different feature representations. The convolutional kernels filter and extract features from words within the window size range in the sentence in the form of a sliding window. The obtained features are then concatenated to obtain the overall feature vector. Step 4.3: Max pooling is used in the pooling layer, that is, the maximum value is selected from the feature vectors generated by each sliding window in step 4.2, and then these maximum values are concatenated to form the feature vector representation; Step 4.4: The fusion layer concatenates the feature vectors obtained from the pooling layer in step 4.3 and merges them into a vector that is more representative of the text vector. Step 4.5: The fully connected layer acts as a classifier by adding a hidden layer after the fusion layer and the final Softmax layer to classify the vectors obtained in Step 4.
4.
5. A label recognition device based on industry knowledge, characterized in that, Includes the following modules: Standard Tag Module: Samples the raw data, obtains data from the business system, and constructs a hierarchical relationship of standard tags according to the standard to obtain one or more sets of data from the business system and the standard tags corresponding to the data in the standard library; The implementation of the standard label module includes the following steps: Step 1.1: Obtain data from the business system; Step 1.2: Obtain the standard label from the industry standard; Step 1.3: Construct a standard label hierarchy based on the specifications in industry standards; Data preprocessing module: Data preprocessing, which adds corresponding standard labels to the collected data tables, fields, and sample data under the fields; Vectorization module: Vectorizes the data processed by the data preprocessing module. It obtains a vectorized representation of the given data after training the pre-trained model, resulting in a vector matrix. Classification module: Feature vectors are extracted from the vectorization module using a deep neural network. By setting filters of different sizes and pooling operations, feature vectors suitable for data classification are selected. The fully connected layer acts as a classifier by adding hidden layers after the fusion layer and the final Softmax layer to perform the final classification of the text. Archiving module: Archives the standard category labels and data obtained from the classification module; The archiving module specifically includes the following steps: Step 5.1: Select the standard label with the highest probability after function processing as the final label; Step 5.2: The trained model can be used to classify and identify the original data units to be classified, and then archive them after labeling; Step 5.3: If a field belongs to a second-level tag under a certain level tag, it also belongs to that level tag.
6. The label recognition device based on industry knowledge according to claim 5, characterized in that, The data preprocessing module specifically includes the following steps: Step 2.1: The data unit in the business system includes any one or more of the following: data table, field, and sample data stored under the corresponding field; Step 2.2: Attach standard label types to the sample data in the business system.
7. The label recognition device based on industry knowledge according to claim 6, characterized in that, The vectorization module specifically includes the following steps: The data processed by the data preprocessing module is transformed into a vector representation using a pre-trained model, and then all vectors are concatenated together to form a vector matrix, which serves as the input to the subsequent deep neural network model.