Sample cleaning method and apparatus, and electronic device and storage medium

Abstract

The present application relates to the technical field of artificial intelligence. Disclosed are a sample cleaning method and apparatus, and an electronic device and a storage medium. The method comprises: performing semantic clustering processing on samples in a corpus, so as to obtain sample sets of a plurality of clustering categories; determining label categories included in the sample set of each clustering category, and on the basis of the label categories, dividing the sample set of each clustering category into a plurality of sample sub-sets; and for the sample set of each clustering category, determining from among the plurality of sample sub-sets a target sample sub-set which has an annotation error, so as to implement a sample cleaning task for the corpus. By means of the technical solution provided in the present application, a sample that has an annotation error in a corpus can be relatively quickly located, and the accuracy of the determination of the sample which has the annotation error is relatively high, thereby effectively reducing the cost investments of time and labor.

Description

Sample cleaning method, device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application CN202410332295.6, entitled “A sample cleaning method, device, electronic device and storage medium” filed on March 22, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of artificial intelligence technology, and in particular to a sample cleaning method, device, electronic device and storage medium. Background Art

[0004] The quality of text annotations in a corpus significantly impacts the results of text models. High-quality corpus data can help achieve better business results after text model training. Therefore, text classification tasks typically require accurate and unique annotations for the text within the corpus. However, in real-world applications, due to the wide variety of sources and rapid growth of text within a corpus, inaccurate annotations and inconsistent annotation results for the same text can occur. Such annotations can negatively impact the effectiveness of model training. Therefore, corpus cleaning is necessary to improve corpus data quality and, in turn, enhance business results after model training.

[0005] The following methods are commonly used in the prior art to clean corpora: Method 1: Re-quality check all annotated texts. This method can identify incorrectly annotated samples, but it is equivalent to performing a second round of annotation, which is costly in terms of manpower and time. Method 2: The annotated texts are divided into batches, and a random sampling of each batch is inspected for annotation quality. This method avoids quality checking the entire text, but when the pass rate of each batch is low, it is equivalent to re-annotating the entire text, and the cost is comparable to Method 1. Method 3: Assuming that low-quality texts are prone to annotation errors, a classification model designed to identify text quality is used to remove low-quality texts from the corpus. This method can produce a corpus with fewer annotation errors, but it cannot determine whether high-quality, incorrectly annotated texts are incorrectly annotated. Since low-quality texts have certain business value, removing them does not meet business requirements, and training an additional classification model to determine text quality is also costly. Summary of the Invention

[0006] The present application provides a sample cleaning method, device, electronic device and storage medium, which can quickly locate incorrectly labeled samples in a semantic library. The accuracy of the identified incorrectly labeled samples is high, effectively reducing the time and manpower cost investment.

[0007] In a first aspect, the present application provides a method for cleaning a sample, the method comprising:

[0008] Perform semantic clustering on samples in the corpus to obtain sample sets of multiple cluster categories;

[0009] Determining the label categories contained in the sample set of each cluster category, and dividing the sample set of each cluster category into multiple sample subsets based on the label categories;

[0010] For the sample set of each cluster category, a target sample subset with labeling errors is determined from the multiple sample subsets, thereby achieving a sample cleaning task for the corpus.

[0011] Furthermore, the semantic clustering processing of samples in the corpus to obtain sample sets of multiple cluster categories includes: determining multiple clustering features and weights corresponding to each clustering feature; performing semantic clustering processing on the samples based on the multiple clustering features and the weights corresponding to each clustering feature to obtain sample sets of the multiple cluster categories; wherein the clustering features include at least one of text editing distance, keyword semantic similarity and text semantic similarity; the sample set includes multiple samples and annotated labels corresponding to each sample.

[0012] Furthermore, for the sample set of the current clustering category, determining the target sample subset with labeling errors from the multiple sample subsets includes: determining the number of samples in each sample subset; if the number of samples in the multiple sample subsets are the same, determining that there is no target sample subset with labeling errors in the sample set of the current clustering category; if the number of samples in the multiple sample subsets is different, determining the maximum number of samples and the minimum number of samples based on the sample data of the multiple sample subsets; and calculating whether each sample subset is a target sample subset with labeling errors based on the number of samples in each sample subset, the maximum number of samples and the minimum number of samples, thereby obtaining the target sample subset with labeling errors in the sample set of the current clustering category.

[0013] Furthermore, the step of calculating in sequence whether each sample subset is an incorrectly labeled target sample subset based on the number of samples in each sample subset, the maximum number of samples, and the minimum number of samples includes: for a current sample subset, calculating the error labeling probability of the current sample subset based on the number of samples in the current sample subset, the maximum number of samples, and the minimum number of samples; if the error labeling probability is greater than a first preset value, determining that the current sample subset is an incorrectly labeled target sample subset; and traversing the multiple sample subsets to obtain the error labeling probability of each sample subset, thereby obtaining whether each sample subset is an incorrectly labeled target sample subset.

[0014] Furthermore, the error labeling probability of the current sample subset is calculated based on the number of samples in the current sample subset, the maximum number of samples, and the minimum number of samples by the following formula:

[0015] In the formula, S represents the probability of incorrect labeling, i represents the index number of the current sample subset among multiple sample subsets, and m i represents the number of samples in the current sample subset, m1 represents the maximum number of samples, and m2 represents the minimum number of samples.

[0016] Furthermore, determining the target sample subset with labeling errors from the multiple sample subsets includes: determining the number of samples in each sample subset; if the number of samples is less than a second preset value, determining the sample subset corresponding to the number of samples as the target sample subset with labeling errors.

[0017] Furthermore, before performing semantic clustering processing on samples in the corpus to obtain sample sets of multiple cluster categories, the method further includes: performing format conversion on the samples in the corpus based on a preset format to obtain processed samples.

[0018] In a second aspect, the present application provides a sample cleaning device, the device comprising:

[0019] The sample clustering module is used to perform semantic clustering on samples in the corpus to obtain sample sets of multiple cluster categories;

[0020] A sample partitioning module, configured to determine the label categories contained in the sample set of each cluster category, and to partition the sample set of each cluster category into a plurality of sample subsets based on the label categories;

[0021] The sample cleaning module is used to determine, for the sample set of each cluster category, a target sample subset with labeling errors from the multiple sample subsets, thereby achieving a sample cleaning task for the corpus.

[0022] In a third aspect, the present application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the sample cleaning method described in any embodiment of the present application.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the sample cleaning method described in any embodiment of the present application when executed.

[0024] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer program instructions, which, when read and executed by a processor, execute the sample cleaning method described in any embodiment of the present application.

[0025] To address the shortcomings of the prior art in the background technology, an embodiment of the present application provides a sample cleaning method. Execution of this method can bring the following beneficial effects: the present application first clusters the samples in the semantic library into multiple sample sets, then divides each sample set into corresponding sample subsets according to the label categories it contains, and finally analyzes each sample subset to determine whether each sample subset has labeling errors. The present application can quickly locate incorrectly labeled samples in the semantic library, and the accuracy of the identified incorrectly labeled samples is high, effectively reducing the time and labor costs.

[0026] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the sample cleaning device or separately from the processor of the sample cleaning device, and this application does not limit this.

[0027] The description of the second, third and fourth aspects in this application can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third and fourth aspects can refer to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description.

[0029] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, and usage scenarios of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] FIG1 is a schematic diagram of a first flow chart of a sample cleaning method provided in an embodiment of the present application;

[0032] FIG2 is a second flow diagram of a sample cleaning method provided in an embodiment of the present application;

[0033] FIG3 is a schematic structural diagram of a sample cleaning device provided in an embodiment of the present application;

[0034] FIG4 is a block diagram of an electronic device for implementing a sample cleaning method according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0036] It should be noted that the terms "first", "second", "target" and "original" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including", "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Figure 1 is a schematic diagram of the first flow chart of a sample cleaning method provided in an embodiment of the present application. This embodiment is applicable to cleaning samples in a corpus. The sample cleaning method provided in this embodiment can be performed by a sample cleaning device provided in an embodiment of the present application. The device can be implemented in software and / or hardware and integrated into an electronic device that performs the method.

[0038] Referring to FIG1 , the method of this embodiment includes but is not limited to the following steps:

[0039] S110 , performing semantic clustering processing on samples in the corpus to obtain sample sets of multiple cluster categories.

[0040] The corpus is the corpus after the first round of annotation, and the sample set includes multiple samples and the annotation labels corresponding to each sample.

[0041] Furthermore, since there may be samples of different formats in the corpus, before semantic clustering is performed on the samples in the corpus to obtain sample sets of multiple cluster categories, it also includes: converting the samples in the corpus into a unified format based on a preset format to obtain processed samples, thereby making the corpus into samples to be clustered, and the samples are presented in the form of text plus labels.

[0042] Specifically, semantic clustering is performed on samples in a corpus to obtain sample sets of multiple cluster categories, including: determining multiple cluster features and the weight corresponding to each cluster feature; performing semantic clustering on the samples based on the multiple cluster features and the weight corresponding to each cluster feature to obtain sample sets of multiple cluster categories. The cluster features include at least one of text edit distance, keyword semantic similarity, and text semantic similarity; and the sample set includes multiple samples and annotated labels corresponding to each sample. According to a preferred embodiment of the present application, the cluster features may include, for example, text edit distance, keyword semantic similarity, and text semantic similarity.

[0043] In an embodiment of a content security business scenario, the weight of the text edit distance can be set to 0.3, the weight of the keyword semantic similarity can be set to 0.3, and the weight of the text semantic similarity can be set to 0.4. The clustering features and corresponding weights of this embodiment can be adjusted according to different business scenarios. Compared with directly using text semantics for clustering, this clustering method has higher accuracy and is suitable for content security business scenarios.

[0044] Optionally, the text edit distance and semantic similarity score can be used to semantically cluster the samples in the corpus using, for example, the K-Nearest Neighbor algorithm (kNN) or other clustering methods. The more accurate the clustering method, the further it can narrow the scope of samples to be quality checked in the semantic library, reducing the quality inspection cost investment.

[0045] For example, suppose a corpus contains 10,000 samples, each consisting of text and a corresponding label, assuming the labels are "a, b, c, d, e, f." Different weights are assigned to the text edit distance, keyword semantic similarity, and text semantic similarity. Semantic clustering generates a set of samples in 100 cluster categories, numbered 0 to 99.

[0046] S120 : Determine the label categories included in the sample set of each cluster category, and divide the sample set of each cluster category into multiple sample subsets based on the label categories.

[0047] The number of sample subsets is the same as the number of label categories contained in the sample set, and one label category corresponds to one sample subset.

[0048] For example, assume that there are 500 samples in the sample set labeled 0, and there are five label categories, namely "a, b, c, d, e". Then these 500 samples are divided into five sample subsets according to the label category.

[0049] S130 : For each cluster category sample set, determine a target sample subset with labeling errors from multiple sample subsets, thereby achieving a sample cleaning task for the corpus.

[0050] In this embodiment of the present application, for a sample set of the current cluster category, a target sample subset with labeling errors is determined from multiple sample subsets in the sample set of the current cluster category; after traversing the sample sets of all cluster categories, the target sample subset with labeling errors in each sample set can be obtained, thereby achieving the sample cleaning task of the corpus. The target sample subset is the sample to be quality-checked in the semantic library.

[0051] Specifically, determining a target sample subset with labeling errors from multiple sample subsets includes: determining the number of samples in each sample subset; if the sample number is less than a second preset value, determining the sample subset corresponding to the sample number as the target sample subset with labeling errors.

[0052] In the embodiment of the present application, it is generally believed in the art that texts with similar semantics are likely to have the same labels, so the samples in the corpus can be clustered, and then the distribution of the number of samples of each label in each cluster can be counted.

[0053] It is generally believed in the art that the text corresponding to a label with a small number of samples in a cluster is likely to be an incorrectly labeled text and needs to be quality checked. The second preset value can be set according to actual application requirements.

[0054] For example, suppose that statistics show that there are 500 samples in the sample set labeled 0, and there are five label categories, namely "a, b, c, d, e". Among them, the number of samples in the sample subset with label a is 300, the number of samples in the sample subset with label b is 100, the number of samples in the sample subset with label c is 80, the number of samples in the sample subset with label d is 10, and the number of samples in the sample subset with label e is 10. At this time, within this cluster category, the samples corresponding to the d label and the e label are likely to be incorrectly labeled samples, and their quality should be checked. Based on the same method, after traversing the sample sets with cluster categories from 0 to 99, samples with a high probability of being incorrectly labeled can be screened out.

[0055] The technical solution provided in this embodiment obtains sample sets of multiple cluster categories by semantic clustering the samples in the corpus; determines the label categories contained in the sample set of each cluster category, and divides the sample set of each cluster category into multiple sample subsets based on the label categories; for the sample set of each cluster category, determines the target sample subset with labeling errors from the multiple sample subsets, thereby achieving the sample cleaning task of the corpus. This application first clusters the samples in the semantic library into multiple sample sets, and then divides each sample set into corresponding sample subsets according to the label categories it contains, and finally analyzes each sample subset to determine whether each sample subset has labeling errors. This application can quickly locate the samples with labeling errors in the semantic library, and the accuracy of the determined labeling error samples is high, which effectively reduces the cost investment in time and manpower.

[0056] The following further describes the sample cleaning method provided by the embodiment of the present application. Figure 2 is a schematic diagram of the second flow chart of a sample cleaning method provided by the embodiment of the present application. The embodiment of the present application is optimized based on the above embodiments. Specifically, the optimization is as follows: for the sample set of the current cluster category, this embodiment provides a detailed explanation of the process of determining the target sample subset with labeling errors from multiple sample subsets.

[0057] Referring to FIG2 , the method of this embodiment includes but is not limited to the following steps:

[0058] S210: Determine the number of samples in each sample subset.

[0059] In an embodiment of the present application, semantic clustering is performed on samples in a corpus to obtain sample sets of multiple cluster categories. The label categories contained in the sample sets of each cluster category are determined, and the sample sets of each cluster category are divided into multiple sample subsets based on the label categories. For each sample set of the cluster category, the number of samples in each sample subset is determined, and it is determined whether the number of samples in each sample subset is the same.

[0060] S220: If the sample numbers of the multiple sample subsets are the same, determine that there is no target sample subset with an incorrect labeling in the sample set of the current cluster category.

[0061] In the embodiment of the present application, when the number of samples in multiple sample subsets is the same, it indicates that the error labeling probability of the corresponding sample set is 0, indicating that there are no samples with error labeling in the sample set.

[0062] S230: If the sample numbers of the multiple sample subsets are different, determine the maximum sample number and the minimum sample number based on the sample data of the multiple sample subsets.

[0063] In an embodiment of the present application, when the number of samples in multiple sample subsets is different, it is determined which label category sample subset has the largest number of samples to obtain the largest number of samples; it is determined which label category sample subset has the smallest number of samples to obtain the smallest number of samples.

[0064] S240 . For the current sample subset, calculate the error labeling probability of the current sample subset based on the number of samples, the maximum number of samples, and the minimum number of samples in the current sample subset.

[0065] Specifically, the error labeling probability of the current sample subset is calculated based on the number of samples, the maximum number of samples, and the minimum number of samples in the current sample subset using the following formula:

[0066] Where S represents the probability of mislabeling (equivalent to the label error rate score), which is the normalized fraction of the number of labels in the sample set of each cluster category, i represents the index number of the current sample subset in multiple sample subsets, and m i represents the number of samples in the current sample subset, m1 represents the maximum number of samples, and m2 represents the minimum number of samples.

[0067] The larger the value of S corresponding to a sample subset, the greater the probability that the sample subset has an incorrect label (that is, the sample subset is the target sample subset), and vice versa. The probability of an incorrect label for a sample subset is related to the number of label categories in the sample subset.

[0068] S250: If the error labeling probability is greater than a first preset value, determine that the current sample subset is a target sample subset with an error labeling.

[0069] The first preset value can be set according to different actual application scenarios. According to a preferred embodiment of the present application, in a business security scenario, the first preset value can be set to 0.8, for example. Experiments have shown that the range of samples to be quality-checked determined by the traditional method accounts for 50% to 100% of the total number of semantic library samples. In a business security scenario, a quality check is performed on a subset of samples with a probability of incorrect labeling greater than 0.8. In the end, only a small number of samples need to be checked (about 10% of the total number of semantic library samples, saving more than 50% of labor costs compared to traditional methods). The corpus can finally achieve the same sample accuracy as the traditional method (generally an accuracy rate of more than 99.5%), and the cost investment in cleaning the corpus is significantly reduced.

[0070] S260 , after traversing multiple sample subsets, obtain the error labeling probability of each sample subset, thereby obtaining the target sample subset with labeling errors in the sample set of the current cluster category.

[0071] In the embodiment corresponding to FIG1 , a sample subset is considered to have labeling errors when the number of samples in the subset is less than a second preset value. Since the number of label categories in each clustering category sample set is different, it is difficult to uniformly set the second preset value for each clustering category sample set. Therefore, the method of calculating the probability of incorrect labeling of a sample subset in this embodiment can also be used to determine whether the sample subset has labeling errors.

[0072] The technical solution provided by this embodiment determines the number of samples in each sample subset; if the number of samples in multiple sample subsets is the same, it is determined that there is no target sample subset with labeling errors in the sample set of the current cluster category; if the number of samples in multiple sample subsets is different, the maximum number of samples and the minimum number of samples are determined based on the sample data of multiple sample subsets; for the current sample subset, the error labeling probability of the current sample subset is calculated based on the number of samples in the current sample subset, the maximum number of samples and the minimum number of samples; if the error labeling probability is greater than a first preset value, it is determined that the current sample subset is a target sample subset with labeling errors; after traversing multiple sample subsets, the error labeling probability of each sample subset is obtained, thereby obtaining the target sample subset with labeling errors in the sample set of the current cluster category. This application determines whether each sample subset has labeling errors by calculating the error labeling probability of each sample subset. Compared with the traditional sample cleaning method, it can reduce the number of target sample subsets (i.e., samples to be quality inspected); this application can also quickly locate the samples with labeling errors in the semantic library, and the accuracy of the determined error labeling samples is high, which effectively reduces the cost of time and manpower.

[0073] FIG3 is a schematic diagram of the structure of a sample cleaning device provided in an embodiment of the present application. As shown in FIG3 , the device 300 may include:

[0074] The sample clustering module 310 is used to perform semantic clustering on samples in the corpus to obtain sample sets of multiple cluster categories;

[0075] A sample partitioning module 320 is configured to determine a label category included in a sample set of each cluster category, and to partition the sample set of each cluster category into a plurality of sample subsets based on the label category;

[0076] The sample cleaning module 330 is configured to determine, for the sample set of each cluster category, a target sample subset with labeling errors from the multiple sample subsets, thereby achieving a sample cleaning task for the corpus.

[0077] Furthermore, the above-mentioned sample clustering module 310 can be specifically used to: determine multiple clustering features and the weight corresponding to each clustering feature; perform semantic clustering processing on the samples based on the multiple clustering features and the weight corresponding to each clustering feature to obtain a sample set of the multiple cluster categories; wherein the clustering feature includes at least one of text editing distance, keyword semantic similarity and text semantic similarity; the sample set includes multiple samples and the annotation label corresponding to each sample.

[0078] Furthermore, the above-mentioned sample cleaning module 330 can be specifically used to: determine the number of samples in each sample subset for the sample set of the current clustering category; if the number of samples in the multiple sample subsets is the same, determine that there is no target sample subset with labeling errors in the sample set of the current clustering category; if the number of samples in the multiple sample subsets is different, determine the maximum number of samples and the minimum number of samples based on the sample data of the multiple sample subsets; based on the number of samples in each sample subset, the maximum number of samples and the minimum number of samples, calculate in turn whether each sample subset is a target sample subset with labeling errors, thereby obtaining the target sample subset with labeling errors in the sample set of the current clustering category.

[0079] Furthermore, the above-mentioned sample cleaning module 330 can also be specifically used to: for the current sample subset, calculate the error labeling probability of the current sample subset based on the number of samples in the current sample subset, the maximum number of samples and the minimum number of samples; if the error labeling probability is greater than a first preset value, determine that the current sample subset is a target sample subset with an error labeling; traverse the multiple sample subsets to obtain the error labeling probability of each sample subset, thereby obtaining whether each sample subset is a target sample subset with an error labeling.

[0080] Furthermore, the sample cleaning module 330 may be further configured to calculate the error labeling probability of the current sample subset based on the number of samples in the current sample subset, the maximum number of samples, and the minimum number of samples using the following formula:

[0081] In the formula, S represents the probability of incorrect labeling, i represents the index number of the current sample subset among multiple sample subsets, and m i represents the number of samples in the current sample subset, m1 represents the maximum number of samples, and m2 represents the minimum number of samples.

[0082] Furthermore, the sample cleaning module 330 may be specifically configured to: determine the number of samples in each sample subset; and if the number of samples is less than a second preset value, determine that the sample subset corresponding to the number of samples is a target sample subset with labeling errors.

[0083] Furthermore, the above-mentioned sample cleaning device may further include: a sample processing module;

[0084] The sample processing module is configured to perform format conversion on the samples in the corpus based on a preset format to obtain processed samples before performing semantic clustering on the samples in the corpus to obtain sample sets of multiple cluster categories.

[0085] The sample cleaning device provided in this embodiment can be applied to the sample cleaning method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0086] Fig. 4 is a block diagram of an electronic device for realizing a method for cleaning a sample of an embodiment of the present application. Electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. Electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships and their functions are merely examples and are not intended to limit the implementation of the present application described herein and / or required.

[0087] As shown in FIG4 , the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 and a random access memory (RAM) 13, that is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0088] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0089] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the sample cleaning method.

[0090] In some embodiments, the cleaning method of the sample can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the cleaning method of the sample described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the cleaning method of the sample in any other appropriate manner (for example, by means of firmware).

[0091] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0095] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0096] A computing system can include clients and servers. These clients and servers are typically remote from each other and typically interact via a communication network. The relationship between the client and server is established through programs running on their respective computers, with these programs acting in a client-server relationship. The server can be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0097] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. For example, those skilled in the art can use the various forms of processes shown above, reorder, add, or delete steps; and can perform the steps described in the present application in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present application can be achieved, and this document does not limit them here.

[0098] The above specific embodiments do not limit the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A sample cleaning method, characterized in that: The method comprises: Perform semantic clustering on samples in the corpus to obtain sample sets of multiple cluster categories; Determining the label categories contained in the sample set of each cluster category, and dividing the sample set of each cluster category into multiple sample subsets based on the label categories; For the sample set of each cluster category, a target sample subset with labeling errors is determined from the multiple sample subsets, thereby achieving a sample cleaning task for the corpus.

2. The sample cleaning method according to claim 1, characterized in that: The semantic clustering process is performed on the samples in the corpus to obtain sample sets of multiple cluster categories, including: Determine multiple clustering features and the weight corresponding to each clustering feature; Performing semantic clustering processing on the samples based on the multiple clustering features and the weight corresponding to each clustering feature to obtain sample sets of the multiple cluster categories; The clustering feature includes at least one of text edit distance, keyword semantic similarity and text semantic similarity; and the sample set includes multiple samples and a label corresponding to each sample.

3. The sample cleaning method according to claim 1, characterized in that: For the sample set of the current cluster category, determining a target sample subset with labeling errors from the multiple sample subsets includes: Determine the number of samples for each sample subset; If the sample numbers of the multiple sample subsets are the same, determining that there is no target sample subset with an incorrect labeling in the sample set of the current cluster category; If the sample numbers of the multiple sample subsets are different, determining a maximum sample number and a minimum sample number based on the sample data of the multiple sample subsets; Based on the number of samples in each sample subset, the maximum number of samples and the minimum number of samples, it is calculated in sequence whether each sample subset is a target sample subset with incorrect labeling, thereby obtaining a target sample subset with incorrect labeling in the sample set of the current cluster category.

4. The sample cleaning method according to claim 3, characterized in that: The calculating, based on the number of samples in each sample subset, the maximum number of samples, and the minimum number of samples, in sequence whether each sample subset is a target sample subset with incorrect labeling includes: For a current sample subset, calculating an error labeling probability of the current sample subset based on the number of samples in the current sample subset, the maximum number of samples, and the minimum number of samples; If the error labeling probability is greater than a first preset value, determining that the current sample subset is a target sample subset with an error labeling; The multiple sample subsets are traversed to obtain the error labeling probability of each sample subset, thereby determining whether each sample subset is a target sample subset with an error labeling.

5. The sample cleaning method according to claim 4, characterized in that: The error labeling probability of the current sample subset is calculated based on the number of samples in the current sample subset, the maximum number of samples, and the minimum number of samples using the following formula: In the formula, S represents the probability of incorrect labeling, i represents the index number of the current sample subset among multiple sample subsets, and m i represents the number of samples in the current sample subset, m1 represents the maximum number of samples, and m2 represents the minimum number of samples.

6. The sample cleaning method according to claim 1, characterized in that: The determining of a target sample subset with labeling errors from the multiple sample subsets includes: Determine the number of samples for each sample subset; If the number of samples is less than a second preset value, the sample subset corresponding to the number of samples is determined to be a target sample subset with labeling errors.

7. The sample cleaning method according to claim 1, characterized in that: Before performing semantic clustering on samples in the corpus to obtain sample sets of multiple cluster categories, the method further includes: The samples in the corpus are format-converted based on a preset format to obtain processed samples.

8. A sample cleaning device, characterized in that: The device comprises: The sample clustering module is used to perform semantic clustering on samples in the corpus to obtain sample sets of multiple cluster categories; A sample partitioning module, configured to determine the label categories contained in the sample set of each cluster category, and to partition the sample set of each cluster category into a plurality of sample subsets based on the label categories; The sample cleaning module is used to determine, for the sample set of each cluster category, a target sample subset with labeling errors from the multiple sample subsets, thereby achieving a sample cleaning task for the corpus.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the sample cleaning method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the sample cleaning method according to any one of claims 1 to 7 when executed.

11. A computer program product, characterized in that The method comprises computer program instructions, and when the computer program instructions are read and executed by a processor, the method for cleaning a sample according to any one of claims 1 to 7 is executed.

Images