Methods, systems, and storage media for extracting entity samples

Abstract

This invention relates to artificial intelligence and provides a data augmentation method for entity extraction samples. It uses clustering to assess the validity of existing large-scale manual annotations, removing samples with significantly different confidence levels from cluster centers to ensure high-quality data for model training. Simultaneously, the removed samples are compared with existing samples for similarity assessment, and highly similar samples are recalled, generating a limited set of labeled tags. This completes hard-match filtering, ensuring the orderly quality control of the labeled data. This method significantly reduces manpower and is applicable to NER entity recognition scenarios with large datasets.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and to a method, system, electronic device, and computer-readable storage medium for extracting entity samples. Background Technology

[0002] In entity sample recognition, sample extraction is typically performed using the labeled tags of entity samples. However, existing methods for sample extraction using labeled tags have the following drawbacks in actual production: 1) The labeled data is voluminous and has a low validity rate, leading to a large amount of repetitive work in sample extraction; 2) The inability to generate a complete set of labeled tags makes quality control of the labeled data impossible, and filtering through hard matching is not possible, resulting in poor robustness and generalization of the entity recognition model; 3) Adding a manual review step for quality control not only increases labor costs but is also unsuitable for NER (Named-entity recognition) entity recognition tasks with large datasets.

[0003] Therefore, there is an urgent need for a low-cost and efficient method for extracting entity samples. Summary of the Invention

[0004] This invention provides a method, system, electronic device, and computer-readable storage medium for extracting entity samples, which solves the problem of low efficiency in large-scale entity sampling in the prior art.

[0005] To achieve the above objective, a method for extracting entity samples is provided, the method comprising:

[0006] The label data of entity samples is classified, and the full set of label data of entity samples is obtained according to the classified label categories;

[0007] The acquired full set of tag data is preprocessed by word segmentation and encoding according to tag category to obtain the tag matrix for each tag category;

[0008] Principal component analysis is performed on the label matrix for each label category to obtain the main features corresponding to each label category;

[0009] The tag data for each tag category is filtered using the main features corresponding to each tag category, and the tag data that meets the preset criteria is used as high-confidence labeling.

[0010] A high-confidence label pool is generated from the high-confidence label sets of each label category, and samples are extracted using the high-confidence label pool.

[0011] Furthermore, preferably, a method for filtering label data for each label category using the main features corresponding to each label category, and selecting label data that meets preset standards as high-confidence labels, includes:

[0012] Treat a key feature of the label category as a cluster class;

[0013] Obtain the distance between the label data and the corresponding main feature in each cluster and form a distance set; then, use the distance set to determine the distance anomaly threshold;

[0014] Labels whose distance to the corresponding main feature is less than the distance anomaly threshold are selected as high-confidence labels.

[0015] Furthermore, preferably, labeled data whose distance from the corresponding main feature is greater than the distance anomaly threshold is selected as low-confidence labels.

[0016] Furthermore, preferably, the label data in the low-confidence label set for each label category is filtered, and the label data with a similarity greater than a preset similarity threshold is included in the high-confidence label pool; the label data in the filtered low-confidence label set is used to generate an abnormal label pool.

[0017] Further, preferably, the method for obtaining the distance between the label data in each cluster and the corresponding principal feature includes: obtaining the distance between the label data in each cluster and the corresponding principal feature by calculating the projection distance of the vector of each entity label on the principal component.

[0018] Furthermore, preferably, the method for selecting tag data with a similarity greater than a preset similarity threshold and including them in the high-confidence tag pool includes:

[0019] Semantic similarity calculation based on cosine value and character similarity calculation are performed on the label data in the low confidence label set for each label category; wherein, the cosine vector value is obtained by semantic similarity calculation based on cosine value, and the character similarity value is obtained by character similarity calculation.

[0020] We assign a weight of 50% to semantic similarity and 50% to character similarity, and use the cosine vector value and character similarity value as weights to obtain the overall similarity.

[0021] Filter tag data whose overall similarity value is greater than the similarity threshold;

[0022] Then, the label data will be categorized into the high-confidence label pool.

[0023] Furthermore, preferably, the method of performing word segmentation and encoding preprocessing on the acquired full tag data according to tag categories to obtain the tag matrix for each tag category includes:

[0024] The acquired full set of tag data is segmented into words according to tag categories;

[0025] Obtain the characters contained in the full data tags after word segmentation, and construct a text set from the characters;

[0026] The text set is one-hot encoded in the dimensions of words, sentences, paragraphs, and articles, and padding is performed to construct corresponding matrices at the four levels of words, sentences, paragraphs, and articles, thereby obtaining word matrix, sentence matrix, paragraph matrix, and article matrix.

[0027] To address the above problems, the present invention also provides a system for extracting entity samples, the system comprising:

[0028] The full data label acquisition unit is used to classify the label data of entity samples and acquire the full label data of entity samples according to the classified label categories.

[0029] The tag matrix acquisition unit is used to perform word segmentation and encoding preprocessing on the acquired full tag data according to the tag category to obtain the tag matrix for each tag category;

[0030] The main feature acquisition unit is used to perform principal component analysis on the label matrix of each label category to obtain the main features corresponding to each label category;

[0031] The high-confidence label pool acquisition unit is used to filter the label data of each label category according to the main features corresponding to each label category, and to select the label data that meets the preset standards as high-confidence labels; to generate a high-confidence label pool from the high-confidence label sets of each label category, and to use the high-confidence label pool for sample extraction.

[0032] To address the aforementioned problems, the present invention also provides an electronic device comprising: a memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the steps in the entity sample extraction method described above.

[0033] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the entity sample extraction method described above.

[0034] This invention discloses a method, system, electronic device, and storage medium for extracting entity samples. The method involves classifying the label data of entity samples to obtain full-volume label data according to label categories; preprocessing the obtained full-volume label data by word segmentation and encoding according to the label categories to obtain label matrices for each label category; performing principal component analysis on the label matrices for each label category to obtain the main features corresponding to each label category; filtering the label data for each label category using the main features corresponding to each label category, selecting label data that meets preset criteria as high-confidence labels; generating a high-confidence label pool from the high-confidence label sets for each label category; and using the high-confidence label pool for sample extraction. The beneficial effects are as follows:

[0035] This study employs clustering to evaluate the effectiveness of existing large-scale manual annotations, removing samples with significantly different confidence levels from cluster centers to ensure high-quality data for model training. Simultaneously, the removed samples are compared with existing samples to determine whether they require manual review or can be modified based on high-confidence samples. This process recalls highly similar samples, generating a limited set of labeled tags and completing hard-matching filtering, ensuring the orderly quality control of the labeled data. This approach significantly reduces manual labor input while improving the robustness and generalization of the entity recognition model, demonstrating its effectiveness in NER entity recognition scenarios with large datasets. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating a method for extracting entity samples according to an embodiment of the present invention.

[0037] Figure 2 A schematic diagram of the modules of an entity sample extraction system provided in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the internal structure of an electronic device that implements a method for extracting entity samples according to an embodiment of the present invention.

[0039] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0040] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0041] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0042] Foundational technologies in artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0043] The entity sample extraction method of the present invention is applied to the quality screening process of manual annotation of a large number of entities.

[0044] Reference Figure 1 The diagram shown is a flowchart illustrating a method for extracting entity samples according to an embodiment of the present invention. This method can be executed by a system, which can be implemented in software and / or hardware.

[0045] In this embodiment, the method for extracting entity samples includes steps S110 to S150:

[0046] S110. Classify the label data of entity samples and obtain the full amount of label data of entity samples according to the classified label categories.

[0047] It should be noted that the entity annotation acquisition method in this embodiment can be executed by a hardware device with data processing capabilities and / or the necessary software to drive the hardware device. Optionally, the executing entity may include a workstation, server, computer, user terminal, and other intelligent devices. User terminals include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, and vehicle terminals. The tag categories are set according to the actual application scenario. In a smart healthcare application scenario, the entity sample can be natural language text displayed on the server or an electronic medical record; the natural language text is medical text, which can be an electronic healthcare record, an electronic personal health record, including medical records, electrocardiograms, medical images, and other electronic records with archival value; while the tag categories can be symptoms, medical history, frequency of onset, etc.

[0048] S120. Perform word segmentation and encoding preprocessing on the acquired full tag data according to tag categories to obtain the tag matrix for each tag category.

[0049] It should be noted that the word segmentation and encoding preprocessing of the full set of labeled data is performed on the labeled content of each category of labels. For word segmentation, the jieba word segmentation system can be used, but is not limited to; the specific implementation method of encoding preprocessing is not specifically limited, and OneHot encoding or a pre-trained model can be used. The implementation method of the pre-trained model is not specifically limited here; the pre-trained model can be, but is not limited to, word2vec, ELMo, OpenAI GPT, BERT, etc. Specifically, firstly, the text data of the acquired full set of labeled data is segmented into Chinese words, and then the text is cleaned to remove redundant information, including removing stop words and special symbols. In the specific implementation process, it may also include statistical analysis of word frequency, followed by further cleaning based on the frequency. Specifically, to improve overall processing efficiency, words with extremely low frequency are removed, because rare words cannot represent the commonalities of the text.

[0050] In a specific embodiment, a method for preprocessing the acquired full-volume tag data according to tag categories through word segmentation and encoding to obtain tag matrices for each tag category includes: S121, segmenting the acquired full-volume tag data according to tag categories; S122, obtaining the characters contained in the tags of the segmented full-volume data and forming a text set from the characters; S123, performing one-hot encoding on the text set in the dimensions of words, sentences, paragraphs, and articles, and performing padding operations to construct corresponding matrices at the four levels of words, sentences, paragraphs, and articles, obtaining word matrices, sentence matrices, paragraph matrices, and article matrices. That is, the encoding preprocessing can be, but is not limited to, one-hot encoding. Specifically, when the encoding preprocessing is one-hot encoding, one-hot encoding is performed to obtain a one-hot matrix, where the number of rows and columns in the one-hot matrix are the number of unique representations, and the matrix value is initialized to 0; the value in each row corresponding to the position of the corresponding level identifier sequence is set to 1. Padding operations are performed to form the one-hot matrix; the padding operation involves filling a series of variable-length tensors with padding_value to make them of equal length.

[0051] S130. Perform principal component analysis on the label matrix of each label category to obtain the main features corresponding to each label category.

[0052] Principal Component Analysis (PCA) is used to calculate the first principal component of the cluster from the label matrix. In other words, the first set of vectors identified by PCA is used as the principal components of the cluster, representing the vectors that best describe the most important features of the cluster. From this set of features, a new set of features is calculated, ordered from highest to lowest importance. These new features are linear combinations of the original features and are uncorrelated. The mapping values ​​of the original features onto the new features are the new dimensionality-reduced samples. In other words, the goal of PCA is to transform the original features with a set of orthogonal vectors to obtain new features, which are linear combinations of the original features. By transforming existing features into one or two new features through PCA, the dimensionality of the feature space is reduced, making subsequent classifier design computationally easier. Furthermore, it eliminates the correlation between original features, reduces data redundancy, and is more conducive to classification.

[0053] In practice, principal component analysis (PCA) is used to directly extract the first principal component from the label matrix, which serves as the most prominent feature of the cluster. Alternatively, the first and second principal components can be extracted from the label matrix as the most prominent features of each cluster.

[0054] S140. The label data for each label category is filtered using the main features corresponding to each label category, with the selected label data meeting the preset criteria used as high-confidence labels. Furthermore, samples in each cluster that are far from the principal component are removed.

[0055] Further, preferably, the method of filtering label data for each label category using the main features corresponding to each label category, and filtering label data that meets the preset criteria as high-confidence labels, includes: S141, taking one main feature of the label category as a cluster; S142, obtaining the distance between the label data in each cluster and the corresponding main feature and forming a distance set; and using the distance set to determine a distance anomaly threshold; S143, filtering label data whose distance to the corresponding main feature is less than the distance anomaly threshold as high-confidence labels.

[0056] The specific steps for determining the distance between entity labels and their corresponding principal features in each cluster are as follows: The distance between the entity label vector and its corresponding principal feature is calculated by projecting the vector onto the principal components. Obtaining the distance between principal features and entity labels is a commonly used technique in principal component analysis.

[0057] The distance anomaly threshold is obtained by calculating the average and standard deviation of the distances between each labeled entity in each cluster and the principal features. Specifically, for each labeled entity in each cluster, the distance to the principal components is calculated, resulting in a set of distances. The average and standard deviation of these distances can then be calculated. The average and standard deviation of the distance set for each cluster are then calculated. According to the statistical definition of an outlier, a value exceeding three times the standard deviation of the mean is considered an outlier.

[0058] S150. Generate a high-confidence label pool from the high-confidence label sets of each label category, and use the high-confidence label pool to extract samples.

[0059] In one specific embodiment, labeled data whose distance to the corresponding primary feature is greater than a distance anomaly threshold is selected as low-confidence labels. In summary, entity labels whose distance to the corresponding primary feature is less than the distance anomaly threshold are considered high-confidence labels, while those whose distance to the corresponding primary feature is greater than the distance anomaly threshold are considered low-confidence labels. The high-confidence label set is used to generate a high-confidence label pool, while the low-confidence label set is used to generate an anomaly label pool. Clustering is used to assess the effectiveness of existing large-scale manual labeling, removing samples whose confidence levels differ significantly from the cluster centers to ensure high-quality data for model training.

[0060] Because word embedding-based clustering methods focus heavily on semantic components, but symptoms are often short in description, even a difference of 1-2 words can cause significant semantic discrepancies, even if the labeling is correct in terms of specific part of speech and content. To address this characteristic, a recall module was added after the initial outlier identification module. This recall module compares the similarity of rejected samples with existing samples to determine whether the sample requires manual review or can be modified by referring to high-confidence samples. This significantly reduces manpower and is suitable for NER tasks with large datasets. Specifically, the recall module filters the label data in the low-confidence label set for each label category, selecting label data with a similarity greater than a preset similarity threshold and adding them to the high-confidence label pool; then, the label data in the filtered low-confidence label set is used to generate an outlier label pool.

[0061] In other words, for the selected outliers, the semantic and character similarity between them and the labels that were not removed (considered as correctly labeled) is calculated, and the top 3 similar labels are selected. If any label has a similarity greater than the threshold of 0.95, it can be considered that the label can be re-added to the training set.

[0062] The similarity filtering process can be implemented using convolutional neural networks (CNNs). Multi-dimensional convolutional layers are used to obtain the similarity between the rejected samples and the existing samples. Then, max pooling layers and fully connected layers are connected to obtain the probability distribution of the similarity in the CNN. Finally, the softmax function is used for probability normalization. In other words, similarity filtering is achieved through a neural network-based classification model. Low-confidence labels are input into the similarity classification model, and convolutional feature information is extracted from these labels. This convolutional feature information is then input into the classification layer for prediction to obtain the corresponding predicted classification label. The loss function value between the predicted classification label and the corresponding data classification label is calculated. Backpropagation training is performed based on the loss function value, and the gradient of the network parameters is calculated. The network parameters are updated through training until the neural network meets the termination condition, and the trained similarity classification model is output.

[0063] In the specific implementation process, the method for selecting label data with a similarity greater than a preset similarity threshold and classifying it into the high-confidence label pool includes: performing semantic similarity calculation and character similarity calculation based on cosine values ​​on the label data in the low-confidence label set for each label category; wherein, the cosine vector value is obtained by calculating the semantic similarity based on cosine values, and the character similarity value is obtained by calculating the character similarity; a weight of 50% is set for semantic similarity and character similarity respectively, and the overall similarity is obtained by weighting the cosine vector value and the character similarity value; the label data with an overall similarity value greater than the similarity threshold is selected; then, the label data is classified into the high-confidence label pool.

[0064] Using the methods described above, along with post-recall through similarity comparison, the number of anomalous labels decreased from over 20,000 to over 10,000 in the application scenario described above. These anomalous labels can be corrected manually later. If there are many high-quality labels and very few anomalous labels, they can be discarded directly.

[0065] This data augmentation method for batch entity extraction samples classifies the labeled data of entity samples, obtaining full-volume labeled data for each entity sample according to label category. The obtained full-volume labeled data is then preprocessed by word segmentation and encoding according to label category to obtain a label matrix for each label category. Principal component analysis is performed on the label matrices for each label category to obtain the main features corresponding to each label category. The label data for each label category is then filtered using the main features, selecting label data that meets preset criteria as high-confidence labels. A high-confidence label pool is generated from the high-confidence label pool, and samples are extracted using this pool. Clustering is used to assess the effectiveness of existing large-scale manual annotations, removing samples with significantly different confidence levels from the cluster centers to ensure high-quality data for model training. Simultaneously, the removed samples are compared with existing samples to determine whether they require manual review or can be modified by referring to high-confidence samples, significantly reducing manpower input and demonstrating effective technical performance for NER tasks on large datasets.

[0066] like Figure 2 As shown, this invention provides an entity sample extraction system 200, which can be installed in an electronic device. Depending on the functions implemented, the entity sample extraction system 200 may include a full data label acquisition unit 210, a label matrix acquisition unit 220, a main feature acquisition unit 230, and a high-confidence label pool acquisition unit 240. The unit described in this invention can also be called a module, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, stored in the memory of the electronic device.

[0067] In this embodiment, the functions of each module / unit are as follows:

[0068] The full data label acquisition unit 210 is used to classify the label data of entity samples and acquire the full label data of entity samples according to the classified label categories.

[0069] The tag matrix acquisition unit 220 is used to perform word segmentation and encoding preprocessing on the acquired full tag data according to the tag category to obtain the tag matrix of each tag category;

[0070] The main feature acquisition unit 230 is used to perform principal component analysis on the label matrix of each label category to obtain the main features corresponding to each label category;

[0071] The high-confidence label pool acquisition unit 240 is used to filter the label data of each label category according to the main features corresponding to each label category, so as to select the label data that meets the preset standard as high-confidence labels; generate a high-confidence label pool from the high-confidence label set of each label category, and use the high-confidence label pool to extract samples.

[0072] Furthermore, it also includes a recall unit 220, which is used to filter the label data in the low confidence label set of each label category, filter the label data with similarity greater than a preset similarity threshold and classify them into the high confidence label pool; and generate an abnormal label pool from the label data in the filtered low confidence label set.

[0073] The entity sample extraction system 200 of this invention classifies the label data of entity samples and obtains the full set of label data of entity samples according to the label categories. The obtained full set of label data is preprocessed by word segmentation and encoding according to the label categories to obtain the label matrix of each label category. Principal component analysis is performed on the label matrix of each label category to obtain the main features corresponding to each label category. The label data of each label category is filtered using the main features corresponding to each label category, and label data that meets the preset criteria is selected as high-confidence labels. A high-confidence label pool is generated from the high-confidence label pool of each label category, and samples are extracted using the high-confidence label pool. The system uses clustering to judge the effectiveness of existing large-scale manual labeling, and samples with significantly different confidence levels from the cluster centers are removed to ensure high-quality data for model training. Simultaneously, the removed samples are compared with existing samples to determine whether the sample needs manual review or can be modified by referring to high-confidence samples, greatly reducing manpower input and demonstrating technical effectiveness suitable for NER tasks with large datasets.

[0074] like Figure 3 As shown, the present invention provides an electronic device 3 for a method of extracting entity samples.

[0075] The electronic device 3 may include a processor 30, a memory 31, and a bus. It may also include a computer program stored in the memory 31 and executable on the processor 30, such as a physical sample extraction program 32. The memory 31 may include both internal storage units for the artificial intelligence physical sample extraction system and external storage devices. The memory 31 can be used not only to store application software and various types of data installed on the artificial intelligence-assisted device, such as the code of the artificial intelligence-assisted program, but also to temporarily store data that has been output or will be output.

[0076] The memory 31 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 31 can be an internal storage unit of the electronic device 3, such as a portable hard drive. In other embodiments, the memory 31 can be an external storage device of the electronic device 3, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, the memory 31 can include both internal and external storage units of the electronic device 3. The memory 31 can be used not only to store application software and various types of data installed on the electronic device 3, such as the code of a physical sample extraction program, but also to temporarily store data that has been output or will be output.

[0077] In some embodiments, the processor 30 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 30 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 31 (e.g., physical sample extraction programs) and calls data stored in the memory 31 to perform various functions of the electronic device 3 and process data.

[0078] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 31 and at least one processor 30, etc.

[0079] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device 3, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0080] For example, although not shown, the electronic device 3 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 30 through a power management system, thereby enabling functions such as charging management, discharging management, and power consumption management through the power management system. The power supply may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 3 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0081] Furthermore, the electronic device 3 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 3 and other electronic devices.

[0082] Optionally, the electronic device 3 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 3 and to display a visual user interface.

[0083] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0084] The entity sample extraction program 32 stored in the memory 31 of the electronic device 3 is a combination of multiple instructions. When run in the processor 30, it can: classify the label data of entity samples; obtain the full set of label data of entity samples according to the classified label categories; perform word segmentation and encoding preprocessing on the obtained full set of label data according to the label categories to obtain the label matrix of each label category; perform principal component analysis on the label matrix of each label category to obtain the main features corresponding to each label category; use the main features corresponding to each label category to filter the label data of each label category, and select the label data that meets the preset criteria as high confidence labels; generate a high confidence label pool from the high confidence label set of each label category; and use the high confidence label pool to extract samples.

[0085] Furthermore, preferably, a method for filtering label data for each label category using the main features corresponding to each label category, and selecting label data that meets preset standards as high-confidence labels, includes:

[0086] Treat a key feature of the label category as a cluster class;

[0087] Obtain the distance between the label data and the corresponding main feature in each cluster and form a distance set; then, use the distance set to determine the distance anomaly threshold;

[0088] Labels whose distance to the corresponding main feature is less than the distance anomaly threshold are selected as high-confidence labels.

[0089] Furthermore, preferably, labeled data whose distance from the corresponding main feature is greater than the distance anomaly threshold is selected as low-confidence labels.

[0090] Furthermore, preferably, the label data in the low-confidence label set for each label category is filtered, and the label data with a similarity greater than a preset similarity threshold is included in the high-confidence label pool; the label data in the filtered low-confidence label set is used to generate an abnormal label pool.

[0091] Further, preferably, the method for obtaining the distance between the label data in each cluster and the corresponding principal feature includes: obtaining the distance between the label data in each cluster and the corresponding principal feature by calculating the projection distance of the vector of each entity label on the principal component.

[0092] Furthermore, preferably, the method for selecting tag data with a similarity greater than a preset similarity threshold and including them in the high-confidence tag pool includes:

[0093] Semantic similarity calculation based on cosine value and character similarity calculation are performed on the label data in the low confidence label set for each label category; wherein, the cosine vector value is obtained by semantic similarity calculation based on cosine value, and the character similarity value is obtained by character similarity calculation.

[0094] We assign a weight of 50% to semantic similarity and 50% to character similarity, and use the cosine vector value and character similarity value as weights to obtain the overall similarity.

[0095] Filter tag data whose overall similarity value is greater than the similarity threshold;

[0096] Then, the label data will be categorized into the high-confidence label pool.

[0097] Furthermore, preferably, the method of performing word segmentation and encoding preprocessing on the acquired full tag data according to tag categories to obtain the tag matrix for each tag category includes:

[0098] The acquired full set of tag data is segmented into words according to tag categories;

[0099] Obtain the characters contained in the full data tags after word segmentation, and construct a text set from the characters;

[0100] The text set is one-hot encoded in the dimensions of words, sentences, paragraphs, and articles, and padding is performed to construct corresponding matrices at the four levels of words, sentences, paragraphs, and articles, thereby obtaining word matrix, sentence matrix, paragraph matrix, and article matrix.

[0101] This invention also provides a computer-readable storage medium, which may be non-volatile or volatile. The storage medium stores a computer program, which, when executed by a processor, performs the following: classifying the label data of entity samples; obtaining the full set of label data of entity samples according to the classified label categories; performing word segmentation and encoding preprocessing on the obtained full set of label data according to the label categories to obtain the label matrix of each label category; performing principal component analysis on the label matrix of each label category to obtain the main features corresponding to each label category; filtering the label data of each label category using the main features corresponding to each label category to select label data that meets the preset criteria as high-confidence labels; generating a high-confidence label pool from the high-confidence label set of each label category; and using the high-confidence label pool for sample extraction.

[0102] Furthermore, preferably, a method for filtering label data for each label category using the main features corresponding to each label category, and selecting label data that meets preset standards as high-confidence labels, includes:

[0103] Treat a key feature of the label category as a cluster class;

[0104] Obtain the distance between the label data and the corresponding main feature in each cluster and form a distance set; then, use the distance set to determine the distance anomaly threshold;

[0105] Labels whose distance to the corresponding main feature is less than the distance anomaly threshold are selected as high-confidence labels.

[0106] Furthermore, preferably, labeled data whose distance from the corresponding main feature is greater than the distance anomaly threshold is selected as low-confidence labels.

[0107] Furthermore, preferably, the label data in the low-confidence label set for each label category is filtered, and the label data with a similarity greater than a preset similarity threshold is included in the high-confidence label pool; the label data in the filtered low-confidence label set is used to generate an abnormal label pool.

[0108] Further, preferably, the method for obtaining the distance between the label data in each cluster and the corresponding principal feature includes: obtaining the distance between the label data in each cluster and the corresponding principal feature by calculating the projection distance of the vector of each entity label on the principal component.

[0109] Furthermore, preferably, the method for selecting tag data with a similarity greater than a preset similarity threshold and including them in the high-confidence tag pool includes:

[0110] Semantic similarity calculation based on cosine value and character similarity calculation are performed on the label data in the low confidence label set for each label category; wherein, the cosine vector value is obtained by semantic similarity calculation based on cosine value, and the character similarity value is obtained by character similarity calculation.

[0111] We assign a weight of 50% to semantic similarity and 50% to character similarity, and use the cosine vector value and character similarity value as weights to obtain the overall similarity.

[0112] Filter tag data whose overall similarity value is greater than the similarity threshold;

[0113] Then, the label data will be categorized into the high-confidence label pool.

[0114] Furthermore, preferably, the method of performing word segmentation and encoding preprocessing on the acquired full tag data according to tag categories to obtain the tag matrix for each tag category includes:

[0115] The acquired full set of tag data is segmented into words according to tag categories;

[0116] Obtain the characters contained in the full data tags after word segmentation, and construct a text set from the characters;

[0117] The text set is one-hot encoded in the dimensions of words, sentences, paragraphs, and articles, and padding is performed to construct corresponding matrices at the four levels of words, sentences, paragraphs, and articles, thereby obtaining word matrix, sentence matrix, paragraph matrix, and article matrix.

[0118] Specifically, the specific implementation method of the computer program when executed by the processor can be referred to the description of the relevant steps in the entity sample extraction method of the embodiment, and will not be repeated here.

[0119] In the several embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0120] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0121] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0122] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0123] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0124] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0125] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim may also be implemented by a single unit or system through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for extracting entity samples, characterized in that, The methods include: The label data of entity samples is classified, and the full set of label data of entity samples is obtained according to the classified label categories; The acquired full set of tag data is preprocessed by word segmentation and encoding according to tag category to obtain the tag matrix for each tag category; Principal component analysis is performed on the label matrix for each label category to obtain the main features corresponding to each label category; The tag data for each tag category is filtered using the main features corresponding to each tag category, and the tag data that meets the preset criteria is used as high confidence label; A high-confidence label pool is generated from the high-confidence label sets of each label category, and samples are extracted using the high-confidence label pool.

2. The method for extracting entity samples as described in claim 1, characterized in that, A method for filtering label data for each label category based on its main features, and selecting label data that meets preset criteria as high-confidence labels, includes: Treat a key feature of the label category as a cluster class; Obtain the distance between the label data and the corresponding main feature in each cluster and form a distance set; then use the distance set to determine the distance anomaly threshold; Labels whose distance to the corresponding main feature is less than the distance anomaly threshold are selected as high-confidence labels.

3. The method for extracting entity samples as described in claim 2, characterized in that, Labels whose distance from the corresponding main feature is greater than the distance anomaly threshold are selected as low-confidence labels.

4. The method for extracting entity samples as described in claim 3, characterized in that, The label data in the low-confidence label set of each label category is filtered, and the label data with similarity greater than a preset similarity threshold is included in the high-confidence label pool; the label data in the filtered low-confidence label set is used to generate an abnormal label pool.

5. The method for extracting entity samples as described in claim 2, characterized in that, A method for obtaining the distance between the label data in each cluster and the corresponding principal feature includes: obtaining the distance between the label data in each cluster and the corresponding principal feature by calculating the projection distance of the vector of each entity label onto the principal component.

6. The method for extracting entity samples as described in claim 4, characterized in that, Methods for filtering tag data with similarity greater than a preset similarity threshold and including them in a high-confidence tag pool include: For the label data in the low-confidence label set of each label category, semantic similarity calculation based on cosine value and character similarity calculation are performed respectively; wherein, the cosine vector value is obtained by semantic similarity calculation based on cosine value, and the character similarity value is obtained by character similarity calculation. We assign a weight of 50% to semantic similarity and 50% to character similarity, and use the cosine vector value and character similarity value as weights to obtain the overall similarity. Filter tag data whose overall similarity value is greater than the similarity threshold; Then, the label data will be categorized into the high-confidence label pool.

7. The method for extracting entity samples as described in claim 1, characterized in that, The method for preprocessing the acquired full tag data by segmentation and encoding according to tag category to obtain the tag matrix for each tag category includes: The acquired full set of tag data is segmented into words according to tag categories; Obtain the characters contained in the full data tags after word segmentation, and construct a text set from the characters; The text set is one-hot encoded in the word, sentence, paragraph, and article dimensions, and padding is performed to construct corresponding matrices at the four levels of word, sentence, paragraph, and article, thereby obtaining the word matrix, sentence matrix, paragraph matrix, and article matrix.

8. A system for extracting entity samples, characterized in that, The system includes: The full data label acquisition unit is used to classify the label data of entity samples and acquire the full label data of entity samples according to the classified label categories. The tag matrix acquisition unit is used to perform word segmentation and encoding preprocessing on the acquired full tag data according to the tag category to obtain the tag matrix for each tag category; The main feature acquisition unit is used to perform principal component analysis on the label matrix of each label category to obtain the main features corresponding to each label category; The high-confidence label pool acquisition unit is used to filter the label data of each label category according to the main features corresponding to each label category, and to select the label data that meets the preset standards as high-confidence labels; to generate a high-confidence label pool from the high-confidence label sets of each label category, and to use the high-confidence label pool for sample extraction.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps in the entity sample extraction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the entity sample extraction method as described in any one of claims 1 to 7.

Images