Method for detecting data anomalies

Abstract

One or more embodiments of the specification provide a data anomaly detection method, the method comprising: for any test sample in a test set, obtaining a preset number of nearest neighbor samples similar to the test sample in a support set, wherein the test set is a set of test samples to be subjected to anomaly detection, and the support set is a set of labeled samples pre-labeled by manual labeling; establishing a probability distribution function according to data corresponding to a target field in the nearest neighbor samples; calculating a confidence of to-be-detected data corresponding to the target field in the test sample through the probability distribution function; and judging whether the target field in the test sample is abnormal according to the confidence.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of data processing technology, and in particular to a method for detecting data anomalies. Background Technology

[0002] With the advent of the digital intelligence era, more and more data processing processes can be implemented using deep learning algorithms without human intervention. In applying these algorithms to process data, a large amount of training data is needed to refine the deep learning model. If the training data cannot cover all possible application scenarios of the model, or if the objects processed by the model include various forms, the model's output results may become abnormal. Currently, the detection of abnormal model output results mainly relies on human intervention. This includes, for example, user feedback, manual analysis of feedback results, and processing of manually selected target samples to generate new training data for iterative optimization of the model.

[0003] Because the aforementioned optimization process requires significant manual intervention, errors can occur throughout the entire anomaly processing phase. Furthermore, the detection of data anomalies relies on user feedback, leading to delays in anomaly handling. Additionally, manual anomaly detection is costly; if the detection focus deviates from key anomalies, it can result in numerous duplicate / similar sample labeling issues during data processing, consuming substantial human resources. Therefore, identifying samples that existing models cannot correctly process using specific algorithms, and then using these samples for model iteration and algorithm updates, is a pressing issue that needs to be addressed. Summary of the Invention

[0004] In view of this, one or more embodiments of this specification provide a method for detecting data anomalies.

[0005] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of this specification, a method for detecting data anomalies is provided, comprising:

[0007] For any test sample in the test set, a preset number of neighboring samples similar to the test sample in the support set are obtained. The test set is a set of test samples to be detected for anomalies, and the support set is a set of labeled samples that have been manually annotated in advance.

[0008] Based on the data corresponding to the target field in the nearest neighbor samples, establish a probability distribution function;

[0009] The confidence level of the data to be detected in the test sample corresponding to the target field is calculated using the probability distribution function.

[0010] Based on the confidence level, determine whether there are any anomalies in the target field of the test sample.

[0011] According to a second aspect of one or more embodiments of this specification, a method for detecting data anomalies is provided, applied to a scenario of detecting data anomalies in recognized cards and documents output by an OCR recognition model, the method comprising:

[0012] For any identified card or document in the test set, obtain a preset number of neighboring card or document documents similar to the identified card or document document in the support set. The test set is a set of identified card or document documents to be detected for anomalies; the support set is a set of card or document documents that have been manually labeled in advance.

[0013] Based on the data corresponding to the target field in the neighboring cards and tickets, establish a probability distribution function;

[0014] The confidence level of the data to be detected in the identified cards and tickets corresponding to the target field is calculated using the probability distribution function.

[0015] Based on the confidence level, determine whether there are any anomalies in the target fields of the identified cards and tickets.

[0016] According to a third aspect of one or more embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first or second aspect.

[0017] According to a fourth aspect of one or more embodiments of this specification, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the method described in the first or second aspect.

[0018] In the technical solution provided in this specification, by obtaining neighboring samples similar to the test sample from the support set, the influence of labeled samples with low similarity to the test sample on the anomaly detection results of the test sample is avoided, thereby improving the accuracy of anomaly detection. Furthermore, by performing anomaly detection by field and obtaining the confidence level corresponding to each field, the anomaly detection results are output in the form of fields. The above approach can quickly locate the abnormal fields, making it easier to process the fields with anomalies. Attached Figure Description

[0019] Figure 1This is a schematic diagram of a data anomaly detection device architecture provided in an exemplary embodiment of this specification;

[0020] Figure 2 This is a schematic flowchart of a data anomaly detection method provided in an exemplary embodiment of this specification;

[0021] Figure 3 This is a schematic diagram of a support set provided in an exemplary embodiment of this specification;

[0022] Figure 4 This is a schematic diagram of another data anomaly detection method provided in an exemplary embodiment of this specification;

[0023] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this specification;

[0024] Figure 6 This is a schematic diagram of a data anomaly detection device provided in an exemplary embodiment of this specification;

[0025] Figure 7 This is a schematic diagram of another data anomaly detection device provided in an exemplary embodiment of this specification. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0027] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0028] With the advent of the digital intelligence era, more and more data processing processes can rely on deep learning algorithms. For example, the process of generating text from images using OCR (Optical Character Recognition) technology eliminates the need for manual reading of the text in the image. Instead, the image is input into a pre-trained machine learning model, and the text is obtained from the model's output. However, the training process of machine learning models relies on a large amount of training data. Even with weakly supervised learning, manual annotation of some samples is still required. Therefore, determining the appropriate number of manually labeled samples is crucial to avoid compromising the performance of the machine learning model while reducing the amount of training data. Furthermore, since each sample contributes differently to the model, selecting the samples for annotation is also a critical consideration.

[0029] In existing technologies, when selecting samples that need to be manually labeled, manual screening is usually adopted. During the application process, the machine learning model obtains user feedback on anomalies in the identification, and relies on manual statistical analysis of the distribution results of the obtained anomaly feedback. Then, based on the distribution results, the manually selected target samples are processed to complete the manual labeling, generating new training data. The new training data is added to the training of the machine learning model to complete the iterative optimization of the model.

[0030] Because the aforementioned optimization process requires significant manual intervention, errors can occur throughout the entire anomaly handling phase. Furthermore, the need for user feedback to detect anomalies leads to delays in anomaly processing. Additionally, manual anomaly detection is costly; if the detection focus deviates from the key anomalies, it may result in the labeling of numerous duplicate or similar samples, wasting substantial human resources without achieving satisfactory training results. Therefore, identifying samples that existing models cannot correctly process using specific algorithms, and then using these samples for model iteration and algorithm updates, is a pressing issue that needs to be addressed.

[0031] To address this issue, this specification provides a method for detecting data anomalies. It uses a manually labeled set of samples as the support set. When performing anomaly detection on the selected test samples in the test set, it uses the nearest neighbor samples from the support set that are similar to the test samples as the basis for comparison, instead of using all labeled samples in the support set. This reduces the number of comparison samples and avoids the influence of labeled samples from the support set that have significant differences in form or low similarity with the test samples on the anomaly detection results, thus improving the accuracy of anomaly detection. Furthermore, by performing anomaly detection by field and obtaining the confidence score for each field, the anomaly detection results are output in field format. This approach allows for rapid identification of anomalous fields, facilitating the processing of fields with anomalies.

[0032] The data anomaly detection methods proposed in this manual are described below. Please refer to... Figure 1 , Figure 1 This is a schematic diagram illustrating the architecture of a data anomaly detection device as an exemplary embodiment of this specification.

[0033] like Figure 1 As shown, the data anomaly detection device may include: a server 11, a network 12, and at least one terminal 13.

[0034] Server 11 can be a physical server containing an independent host, or it can be a virtual server hosted by a host cluster. During operation, server 11 can be configured with a data anomaly detection device, which can be implemented in software and / or hardware to provide a service for detecting data anomalies. This device detects whether any field in the test sample is abnormal by comparing the test sample in the test set with the nearest neighbor samples that are similar to the test sample in the support set.

[0035] Terminal 13 refers to an electronic device that a user can use to initiate a request for data anomaly detection of a test sample image. This electronic device can be a mobile phone, desktop computer, tablet, laptop, or PDA (Personal Digital Assistants), wearable device (such as smart glasses, VR glasses, etc.), or a scanner or digital camera with OCR recognition capabilities, or a dedicated OCR recognition device for cards and documents, such as a passport reader or driver's license reader. This specification does not limit this to any particular embodiment. The electronic device may include an image acquisition device for acquiring images of the target object for OCR recognition, or it may receive images of the target object acquired by other devices via a communication connection and perform OCR recognition.

[0036] In an exemplary embodiment of this specification, terminal 13 can implement OCR recognition functionality for a target object. Specifically, terminal 13 implements OCR recognition functionality for a target object based on a machine learning model deployed on it, such as OCR recognition of train tickets, extraction of ticket information, and formatted output records. The terminal sends the output of the aforementioned machine learning model as a test sample to the server 11 for anomaly detection. The server 11 optimizes the OCR function based on the anomaly detection results. For example, for scenarios involving new train ticket formats, or situations where the existing machine learning model cannot accurately recognize a ticket even if it is not a new format, resulting in data anomalies, test samples can be manually labeled, and the labeled samples can be used to iteratively update the aforementioned machine recognition model, thereby dynamically and proactively achieving model iteration, continuously optimizing and adapting to format changes. Of course, the data anomaly detection method proposed in this specification can also be applied to data anomaly detection in other application scenarios besides OCR recognition, and this specification does not impose specific limitations on this.

[0037] The network 12 for interaction between server 11 and terminal 13 can include various types of wired or wireless networks.

[0038] The following is combined Figure 2 This document describes a method for detecting data anomalies. Figure 2 This is a flowchart illustrating a data anomaly detection method provided in an exemplary embodiment. Figure 2 As shown, the method may include the following steps:

[0039] S201, for any test sample in the test set, obtain a preset number of neighboring samples similar to the test sample in the support set, wherein the test set is a set of test samples to be detected for anomalies, and the support set is a set of pre-labeled samples that have been manually annotated.

[0040] The test samples are those requiring anomaly detection, while the nearest neighbor samples selected from the labeled samples in the support set serve as the basis for anomaly detection comparison. For any test sample in the test set, a predetermined number of near neighbor samples similar to that test sample are obtained from the support set. The purpose of obtaining near neighbor samples is to avoid the influence of labeled samples in the support set that have significant differences in form or low similarity to the test sample on the anomaly detection results of the test sample during subsequent comparisons, thereby reducing the accuracy of anomaly detection.

[0041] In an exemplary embodiment of this specification, the selection of the nearest neighbor samples can be accomplished through the following steps:

[0042] Based on the field data contained in the test samples and labeled samples, generate corresponding feature vectors;

[0043] Calculate the distance between the feature vector corresponding to the test sample and the feature vector of the labeled sample, and obtain a preset number of nearest neighbor samples that are closest to the test sample.

[0044] In the above method, the test sample and the labeled sample are first represented by feature vectors, which can be generated based on the field data contained in the test sample and the labeled sample.

[0045] For example, in an exemplary embodiment of this specification, it is assumed that the test sample is a card or document output by a machine learning model for OCR recognition. Since the data contained in this card or document exists in a structured form of key-value pairs, a test sample will contain data in multiple fields, where any one field includes: category information, location information, and identification information. For example, assuming a test sample contains the field "Gender: Female", its category information is "Gender", the location information indicates the position of the field in the test sample, for example, the field is in the second line of the text generated by the test sample, or the coordinates of the field in the original object that has not been OCR recognized; the identification information is "Female". To facilitate comparison, all data included in the test sample and the labeled sample are statistically analyzed according to category, and the data of each category is arranged in a predetermined order. A feature vector for each sample is generated using the above method. For example, for a sample containing data categories including gender, identity, age, and document number, a feature vector can be formed in the order of gender, name, age, and document number to represent the sample.

[0046] After representing the samples in both the test set and the support set as feature vectors, the similarity between the feature vector of the test sample and the feature vectors of each labeled sample in the support set is calculated. In an exemplary embodiment of this specification, a cosine similarity algorithm can be used to calculate the cosine distance between the feature vector of the test sample and the feature vectors of the labeled samples in the support set to characterize the similarity between them. The closer the cosine distance between the feature vector of the test sample and the feature vector of a certain labeled sample, the higher the similarity between the test sample and the labeled sample. Using the above method, a predetermined number of nearest neighbor samples with the highest similarity to the test sample can be obtained in the support set. Of course, other calculation methods can also be used to calculate the similarity between the feature vector of the test sample and the feature vectors of the labeled samples in the support set, and this specification does not impose specific limitations on this.

[0047] After obtaining the nearest neighbor samples of the test sample, proceed to the next step.

[0048] S202, Based on the data corresponding to the target field in the nearest neighbor samples, establish a probability distribution function.

[0049] S203, calculate the confidence level of the data to be detected in the test sample corresponding to the target field using the probability distribution function.

[0050] For example, assuming the target field is "gender," we can statistically analyze the distribution of "gender" data in the nearest neighbor samples to generate a probability distribution function for that field. By substituting the test sample into the probability distribution function obtained from the nearest neighbor samples, we can calculate the confidence level of the target field in the test sample. A higher confidence level for the target field in the test sample indicates that it better conforms to the distribution pattern obtained from the nearest neighbor samples, meaning the likelihood of anomalies in that target field is lower. Conversely, a lower confidence level indicates a greater difference in distribution trend between the target field and the corresponding field in the nearest neighbor samples, meaning a higher likelihood of anomalies.

[0051] In an exemplary embodiment of this specification, the target field has different descriptive data across multiple dimensions. When establishing the probability distribution function, a probability distribution function corresponding to each dimension can be established based on the descriptive data of the target field in each dimension. In this specification, the descriptive data of the target field in any dimension is referred to as the criterion for that target field in that dimension. For example, taking the gender field in an ID card as an example, for the "gender" field, there are criteria based on the dimension of the frequency of category occurrence and the dimension of the number of recognition results. A corresponding probability distribution function is established for each of the above multiple criteria, and the confidence level of the target field of the test sample corresponding to each dimension is obtained through the probability distribution function corresponding to each dimension. The anomaly of the target field is comprehensively evaluated based on multiple criteria. When the target field is abnormal, the descriptive data in one dimension may happen to conform to the probability distribution function generated by the nearest neighbor samples based on that dimension, but the descriptive data in other dimensions may not conform to the probability distribution function generated based on that dimension. For example, regarding the "gender" field, suppose the test sample identifies the erroneous field "gender = good person". In this case, the anomaly cannot be detected in the category occurrence count dimension. However, in the number of identification results dimension, the appearance of "good person" clearly changes the number of identification results from a single character to two characters. Therefore, the anomaly of this field can be detected. By establishing a multi-dimensional probability distribution function for the same target field, the accuracy of data anomaly detection can be effectively improved.

[0052] In an exemplary embodiment of this specification, the probability distribution function may be in the form of a Gaussian distribution function, a Poisson distribution function, or other distribution functions suitable for application. This specification does not impose any specific limitations on this.

[0053] Specifically, assuming a test sample is used, the occurrence frequency and number of recognition results of the gender field in a preset number of nearest neighbor samples are obtained, and the statistical results of the above information are subjected to Gaussian distribution statistics. Since the gender field in the ID card appears only once, and the recognition result is either "male" or "female," both are single characters, the number of recognition results is 1. Based on the above background, for the gender field, when the similarity of the preset number of nearest neighbor samples is high, the occurrence frequency dimension may fit a Gaussian distribution with a mean of 1.0 and a variance of 0.0; while the number of recognition results dimension may fit a Gaussian distribution with a mean of 1.0 and a variance of 0.0. The probability distribution function results of the above two dimensions indicate that the gender field can only have one detection result, and the corresponding recognition result should only be one character. By substituting the occurrence frequency and number of recognition results of the gender field of the test sample into the corresponding probability distribution function and calculating the confidence level under the above criteria, it can be determined whether the field is abnormal.

[0054] In an exemplary embodiment of this specification, based on the confidence levels obtained for the target field according to the criteria based on different dimensions, an average confidence level for the target field can be calculated. Furthermore, reasonable weights can be assigned to the confidence levels obtained based on each criterion according to their importance, and a weighted confidence level corresponding to the target field can be calculated based on the weights of each criterion and the confidence levels of the target field obtained based on each criterion. The average confidence level or the overall confidence level is then used to determine whether an anomaly exists in the target field.

[0055] S204, Based on the confidence level, determine whether there is an anomaly in the target field of the test sample.

[0056] In an exemplary embodiment of this specification, an abnormal confidence threshold can be determined empirically. When the confidence of a target field is lower than the threshold, the target field is determined to be abnormal.

[0057] In another exemplary embodiment of this specification, a comprehensive confidence score for the test sample is generated based on the confidence scores of the data to be detected corresponding to each field in the entire test sample. Then, based on the comprehensive confidence score, it is determined whether the test sample as a whole exhibits any anomalies. For test samples represented using feature vectors, the comprehensive confidence score of the test sample can be represented by a feature matrix composed of the feature vectors of each field. The comprehensive confidence score corresponding to the entire test sample can be obtained by performing the calculations described above using the feature matrix.

[0058] In one exemplary embodiment of this specification, location information and other information can be added to the criteria based on different dimensions to improve the accuracy of data anomaly detection. Furthermore, when data of the same category appears multiple times in the same sample, introducing location information can distinguish between data of the same category appearing in different fields.

[0059] In an exemplary embodiment of this specification, the samples in the test set are derived from test samples that require data anomaly detection, output by a pre-trained machine model.

[0060] For example, suppose the pre-trained machine learning model is a structure detection model used for OCR recognition. This model is used to recognize characters in a preset object and output the corresponding text data. During the OCR recognition process of the aforementioned structure detection model, anomalies may occur in the OCR output due to differences in the carrier of the target object (e.g., recognizing paper materials in electronic images) and the different forms in which the information is recorded (e.g., the same information is located in different objects at different positions). To detect whether there are anomalies in the OCR output and to process the anomaly-prone objects accordingly, the OCR-recognized results are used as test samples and placed into a test set.

[0061] In the above embodiments, it is assumed that the support set contains labeled samples of multiple versions of a preset object. For example, if the preset object is a train ticket, there are two versions of the ticket: red and blue. The field data of the two versions of the ticket differs. For example, the blue version of the ticket includes the pinyin of the departure station, but the red version does not. If the average statistical result is used when detecting abnormal data, assuming the test sample is the blue version of the ticket, the probability distribution function for the departure station pinyin field may differ significantly from the actual result due to the influence of the red version of the ticket. When there are many red version ticket samples in the support set, the probability distribution for the departure station pinyin field may be biased towards the result that the field is not present. In this case, the blue version of the ticket used as the test sample will be judged as having abnormal data. However, in this specification, by obtaining the nearest neighbor samples in the support set, the influence of labeled samples with large version differences from the test sample on the detection results of the test sample can be effectively avoided.

[0062] In an exemplary embodiment of this specification, the samples in the test set may also be manually labeled samples. In this case, the above-described data anomaly detection method can be used to correct errors in the labeled samples. When any labeled sample used as a test sample is determined to be anomaly by the above method, it proves that the labeling information of the labeled sample may be incorrect, and the manually labeled information in the sample with anomalies needs to be modified. By placing labeled samples into the test set, the method can detect manually labeled samples with errors, and by modifying the samples with erroneous labels, the quality of the labeled data can be improved.

[0063] After completing the data anomaly detection, a large number of anomaly samples can be obtained. These anomaly samples may be homogeneous problems caused by the same model defects. To avoid duplicate labeling and increased costs, a portion of representative anomaly samples can be selected through active learning for manual labeling. The selected anomaly samples are those whose anomaly types are easiest to distinguish. Manually labeling these anomaly samples and then adding the labeled samples to the support set can expand the application scope of the data anomaly detection method provided in this manual.

[0064] Alternatively, in another exemplary embodiment of this specification, the manually labeled anomalous samples can also be used for iterative training of a machine learning model applicable to the support set. For example, for a machine learning model for OCR recognition, labeling the anomalous samples and using them in the training process of the model can expand the application scope of the data anomaly detection method provided in this specification.

[0065] In an exemplary embodiment described herein, the active learning method described above employs the following method for screening anomalous samples:

[0066] The abnormal samples are classified according to the abnormal fields present in the abnormal samples; the abnormal proportion corresponding to each category is obtained according to the number of abnormal samples in each category; and abnormal samples are extracted according to the abnormal proportion for manual annotation.

[0067] For example, in a test set, the abnormal samples can be categorized by the abnormal field: "Name + Gender" is abnormal, "Age" is abnormal, and "Gender" is abnormal. The ratio of abnormal samples in each category is 1:2:2, and the total number of abnormal samples is 10,000. Now, we need to select 1,000 samples for manual annotation. Based on the above ratio, we would select 200 abnormal samples with abnormal "Name + Gender", 400 abnormal samples with abnormal "Age" and 400 abnormal samples with abnormal "Gender" for manual annotation.

[0068] Of course, other search methods can also be used for sampling outliers, such as minimum confidence sampling, edge sampling, and entropy sampling. Taking minimum confidence sampling as an example, this search method uses the minimum confidence of each sample as an indicator of uncertainty. For example, the overall confidence of the test samples in the above embodiment can be obtained. For samples with lower overall confidence, it proves that their similarity to their nearest neighbors in the support set is lower and the difference is greater. Therefore, they need more manual annotation. The outliers are arranged in order of overall confidence. When there are 10,000 outliers and 1,000 samples need to be selected for manual annotation, the 1,000 outliers with the lowest overall confidence are selected for manual annotation. Different search methods can be selected for outlier sampling depending on the application scenario, and this specification does not impose specific restrictions on this.

[0069] For ease of understanding, this specification provides the following specific embodiments: The test samples in the test set are samples output by a machine learning model used for OCR recognition after recognizing a certain type of document. The support set contains 10,000 labeled samples each of two different versions of the document, designated as Version 1 and Version 2, respectively. Figure 3 As shown. Both of the above-mentioned document formats include the fields of name, gender, and age, and each field appears only once.

[0070] Assume that test sample 1 is closer to version 1, but its age field contains the data "Age = Male". Additionally, test sample 2 exists in the test set, which is even closer to version 2, but also includes a "Date of Birth" field. Test sample 2 represents a new version of this document.

[0071] When the age field in test sample 1 is used as the target field for anomaly detection, 1000 nearest neighbor samples are obtained from the support set for this target field. Based on the dimensions of "character type in the recognition result" and "number of characters in the recognition result," a probability distribution function for the target field under these two different dimensions is established according to the nearest neighbor samples, as follows: Figure 3 The probability distribution functions 1 and 2 are shown in the figure.

[0072] Since age is represented by Arabic numerals in Format 1, while the target field recognition result in Test Sample 1 is "Male," which is a Chinese character rather than an Arabic numeral, the confidence level for the target field in Test Sample 1 in the "Character Types in Recognition Result" dimension is 0, assuming no errors in the labeled samples in the support set. As for the "Number of Characters in Recognition Result" dimension, since age recognition information may range from single digits to three digits, the probability distribution function established based on the target field in the "Number of Characters in Recognition Result" dimension in neighboring samples has little reference value for judging data anomalies in this target field. In this case, other dimensions can be selected to establish corresponding probability distribution functions, or the weight of "Number of Characters in Recognition Result" can be reduced when calculating the weighted confidence level of the target field. Because Test Sample 1 has an anomaly in the age field, Test Sample 1 is an anomalous sample.

[0073] For test sample 2, although its nearest neighbor in the support set is version 2, when the "date of birth" field is used as the target field, since version 2 does not contain data for this field, the confidence level for this field is 0, assuming that there are no errors in the labeled samples in the support set, regardless of the dimension used. Therefore, test sample 2 is also an anomalous sample.

[0074] Assuming the test set includes only the two types of anomalies mentioned above, with a total of 10,000 test samples, where the ratio of anomaly type 1 to anomaly type 2 is 1:9, after anomaly detection in this test set, 1,000 anomaly samples with anomalies in the "age" field and 9,000 anomaly samples with anomalies in the "date of birth" field will be obtained. If 1,000 of these are selected for manual annotation, then 100 anomaly samples with anomalies in the "age" field and 900 anomaly samples with anomalies in the "date of birth" field will be selected for manual annotation.

[0075] For anomalous samples with abnormal "date of birth" fields, since the anomalies occurred due to the format update to Format 3, these anomalous samples can be manually labeled and added to the support set as new labeled samples. When test sample 3, belonging to Format 3, undergoes anomaly detection, its nearest neighbors are newly added labeled samples belonging to Format 3 in the support set. Therefore, when anomaly detection is performed on test sample 3 based on its nearest neighbors, test sample 3 is determined to be free of anomalies according to Format 3. In this case, for the support set of the data anomaly detection method, the introduction of Format 3 effectively expands the applicability of the method. Test sample 3 belonging to Format 3 will not be considered anomaly, greatly avoiding duplicate labeling of samples with the same anomaly type.

[0076] Simultaneously, manually labeled anomalous samples can be used for iterative training of the machine learning model for OCR recognition. Since the new labeled samples include new layouts, using these samples as training data for the machine learning model is equivalent to adding a new user scenario during the model training process. The updated machine learning model obtained after iterative training can be better suited for recognizing documents of layout 3.

[0077] An exemplary embodiment of this specification also provides a method for detecting data anomalies in recognized cards and documents output by an OCR recognition model, such as... Figure 4 As shown, it includes the following steps:

[0078] S401, for any identified card or ticket in the test set, obtain a preset number of neighboring card or ticket similar to the identified card or ticket in the support set. The test set is a set of identified card or ticket documents to be detected for anomalies; the support set is a set of card or ticket documents that have been manually labeled in advance.

[0079] S402, establish a probability distribution function based on the data corresponding to the target field in the neighboring cards and tickets;

[0080] S403, calculate the confidence level of the data to be detected in the identified card and document corresponding to the target field using the probability distribution function; determine whether there is an anomaly in the target field of the identified card and document based on the confidence level.

[0081] In an exemplary embodiment of this specification, the above-described method of obtaining a preset number of neighboring cards and documents similar to the identified cards and documents in the support set can be achieved through the following steps:

[0082] Based on the field data contained in the identified and labeled cards and tickets, generate corresponding feature vectors;

[0083] Calculate the distance between the feature vector corresponding to the identified card and the feature vector of the labeled card, and obtain a preset number of neighboring cards that are closest to the identified card.

[0084] The target field can have different descriptive data in multiple dimensions. In an exemplary embodiment of this specification, a probability distribution function corresponding to each dimension can be established based on the descriptive data of the target field in each dimension, and the confidence level of the data to be detected in the identified card and ticket corresponding to the target field in each dimension can be obtained through the probability distribution function corresponding to each dimension.

[0085] In another exemplary embodiment of this specification, a comprehensive confidence level of the identified card and document can be generated based on the confidence level of the data to be detected corresponding to each field in the identified card and document; and based on the comprehensive confidence level, it can be determined whether there is any abnormality in the overall identified card and document.

[0086] In an exemplary embodiment of this specification, cards and documents with anomalies in at least one field can be identified as anomalous cards and documents, and at least a portion of these anomalous cards and documents can be manually annotated. The manually annotated anomalous cards and documents can then be placed into a support set; or, the manually annotated anomalous cards and documents can be used for iterative training of the model used for OCR recognition. In this embodiment, after manual annotation, the anomalous cards and documents detected by the above method can be added to the support set as updated manually annotated cards and documents, thereby increasing the number of manually annotated cards and documents in the support set and improving the accuracy of the data anomaly detection method. Alternatively, the anomalous cards and documents detected by the above method can be manually annotated and used for iterative training of the model, iteratively updating the model used for OCR recognition and further improving the model's ability to understand cards and documents.

[0087] When manually labeling at least a portion of abnormal cards and receipts, the following steps can be taken: classify the abnormal cards and receipts according to the abnormal fields present in them; obtain the abnormality ratio corresponding to each category based on the number of abnormal cards and receipts in each category; and extract abnormal cards and receipts for manual labeling according to the abnormality ratio.

[0088] For specific embodiments of the data anomaly detection methods for recognized cards and documents output by OCR recognition models listed above, please refer to [the relevant documentation]. Figure 2 The specific embodiments of the method shown are for understanding purposes only and will not be elaborated upon here.

[0089] Figure 5 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of this specification. Please refer to... Figure 5 At the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, memory 508, and non-volatile memory 510. It may also include other hardware required for various functions. The processor 402 reads the corresponding computer program from the non-volatile memory 510 into the memory 508 and then runs it, forming a data anomaly detection device at the logical level. Of course, besides the software implementation, one or more embodiments in this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0090] Corresponding to the embodiments of the above methods, this specification also provides a data anomaly detection device, such as... Figure 6 As shown, the device may include:

[0091] The first acquisition unit 610 is used to acquire a preset number of neighboring samples similar to the test sample in the support set for any test sample in the test set, wherein the test set is a set of test samples to be detected for anomalies, and the support set is a set of labeled samples that have been manually labeled in advance.

[0092] The first establishment unit 620 is used to establish a probability statistical function based on the data corresponding to the target field in the nearest neighbor samples;

[0093] The first calculation unit 630 is used to calculate the confidence level of the data to be detected in the test sample corresponding to the target field through the probability statistics function;

[0094] The first judgment unit 640 is used to determine whether there is an anomaly in the target field of the test sample based on the confidence level.

[0095] Optionally, the first acquisition unit 610 may be specifically used for:

[0096] Based on the field data contained in the test samples and labeled samples, generate corresponding feature vectors;

[0097] Calculate the distance between the feature vector corresponding to the test sample and the feature vector of the labeled sample, and obtain a preset number of nearest neighbor samples that are closest to the test sample.

[0098] Optionally, the target field has different descriptive data in multiple dimensions, and the first establishment unit 620 can be specifically used for:

[0099] Based on the descriptive data of the target field in each dimension, establish probability statistical functions corresponding to each dimension;

[0100] The first computing unit 630 can be specifically used for:

[0101] The confidence levels of the test data corresponding to the target field in the test sample for each dimension are obtained by using the probability statistics functions corresponding to each dimension.

[0102] Optionally, the device further includes:

[0103] The first generation unit 650 is used to generate the comprehensive confidence level of the test sample based on the confidence level of the data to be detected corresponding to each field in the test sample.

[0104] The first overall judgment unit 660 is used to determine whether there is an anomaly in the overall test sample based on the overall confidence level.

[0105] Optionally, the test samples in the test set include at least one of the following: test samples that need to be detected for data anomalies, output by a pre-trained machine learning model, and pre-labeled samples that have been manually annotated.

[0106] Optionally, the machine learning model is a model for OCR recognition of a preset object, and the support set contains labeled samples of multiple versions of the preset object.

[0107] Optionally, the device further includes:

[0108] The first selection unit 670 is used to identify samples with anomalies in at least one field as abnormal samples, and to select at least a portion of the abnormal samples for manual annotation.

[0109] The first abnormal sample processing unit 680 is used to put manually labeled abnormal samples into a support set; or, to use manually labeled abnormal samples for iterative training of a machine learning model applicable to the support set.

[0110] Optionally, the first selection unit 670 may be specifically used for:

[0111] The abnormal samples are classified according to the abnormal fields present in the abnormal samples;

[0112] The abnormality ratio corresponding to each category is obtained based on the number of abnormal samples in each category;

[0113] Abnormal samples are extracted according to the stated abnormality ratio for manual labeling.

[0114] Corresponding to the embodiments of the above methods, this specification also provides another data anomaly detection device, applied to scenarios involving the detection of data anomalies in recognized cards and documents output by an OCR recognition model, such as... Figure 7 As shown, the device may include:

[0115] The second acquisition unit 710 is used to acquire, for any identified card or document in the test set, a preset number of neighboring card or document documents similar to the identified card or document document in the support set. The test set is a set of identified card or document documents to be detected for anomalies. The support set is a set of card or document documents that have been manually labeled in advance.

[0116] The second establishment unit 720 is used to establish a probability distribution function based on the data corresponding to the target field in the neighboring card documents;

[0117] The second calculation unit 730 is used to calculate the confidence level of the data to be detected in the identified card and ticket corresponding to the target field through the probability distribution function;

[0118] The second judgment unit 740 is used to determine whether there is an anomaly in the target field of the identified card or document based on the confidence level.

[0119] Optionally, the second acquisition unit 710 may be specifically used for:

[0120] Based on the field data contained in the identified and labeled cards and tickets, generate corresponding feature vectors;

[0121] Calculate the distance between the feature vector corresponding to the identified card and the feature vector of the labeled card, and obtain a preset number of neighboring cards that are closest to the identified card.

[0122] Optionally, the device may further include:

[0123] The second selection unit 750 is used to identify cards and tickets with at least one abnormal field as abnormal cards and tickets, and to select at least a portion of the abnormal cards and tickets for manual annotation.

[0124] The second abnormal sample processing unit 760 is used to put manually annotated abnormal cards and documents into the support set; or, to use manually annotated abnormal cards and documents for iterative training of the model used for OCR recognition.

[0125] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0126] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0127] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0128] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0129] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0130] In one or more embodiments of this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0131] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this specification. The singular forms “a,” “described,” and “the” used in one or more embodiments of this specification and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0132] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0133] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A method of detecting data anomalies, characterized by, The method is applied to a data anomaly detection scene of an OCR-recognized card, certificate or bill output by an OCR recognition model, and the method comprises the following steps: For any recognized card, certificate or bill in a test set, a preset number of neighbor cards, certificates or bills similar to the recognized card, certificate or bill are obtained from a support set, the test set is a set of recognized cards, certificates or bills to be subjected to anomaly detection, and the support set is a set of labeled cards, certificates or bills pre-labeled by manual labeling; A probability distribution function is established according to data corresponding to a target field in the neighbor cards, certificates or bills; The confidence of to-be-detected data corresponding to the target field in the recognized card, certificate or bill is calculated through the probability distribution function; Whether the target field in the recognized card, certificate or bill is abnormal is determined according to the confidence.

2. The method of claim 1, wherein, The obtaining of the preset number of neighbor cards, certificates or bills similar to the recognized card, certificate or bill from the support set comprises the following steps: Feature vectors are generated according to field data contained in the recognized card, certificate or bill and the labeled card, certificate or bill; The distance between the feature vector corresponding to the recognized card, certificate or bill and the feature vector of the labeled card, certificate or bill is calculated to obtain the preset number of neighbor cards, certificates or bills closest to the recognized card, certificate or bill.

3. The method of claim 1, wherein, The target field has different description data in multiple dimensions, respectively, and the establishment of the probability distribution function according to the data corresponding to the target field in the neighbor cards, certificates or bills comprises the following steps: The probability distribution functions corresponding to the multiple dimensions are respectively established based on the description data of the target field in the multiple dimensions; The confidence of to-be-detected data corresponding to the target field in the recognized card, certificate or bill is obtained through the probability distribution function, comprising the following steps: The confidence of to-be-detected data corresponding to the target field in the recognized card, certificate or bill corresponding to each dimension is respectively obtained through the probability distribution functions corresponding to each dimension.

4. The method of claim 1, wherein, Further comprising: A comprehensive confidence of the recognized card, certificate or bill is generated according to the confidence of to-be-detected data corresponding to each field in the recognized card, certificate or bill; Whether the recognized card, certificate or bill as a whole is abnormal is determined according to the comprehensive confidence.

5. The method of claim 1, wherein, The recognized cards, certificates or bills in the test set comprise at least one of the following: recognized cards, certificates or bills output by a pre-trained machine learning model and needing to be subjected to data anomaly detection and labeled samples pre-labeled by manual labeling.

6. The method of claim 5, wherein, The machine learning model is a model for performing OCR recognition on a preset object, and the support set contains labeled samples of multiple formats of the preset object.

7. The method of claim 1, wherein, Further comprising: At least one card, certificate or bill with an abnormal field is determined as an abnormal card, certificate or bill, and at least a part of the abnormal card, certificate or bill is selected for manual labeling; The manually labeled abnormal card, certificate or bill is put into the support set, or the manually labeled abnormal card, certificate or bill is used for iterative training of the model for OCR recognition.

8. The method of claim 7, wherein, The selection of at least a part of the abnormal card, certificate or bill for manual labeling comprises the following steps: The abnormal card, certificate or bill is classified according to the field with an abnormality in the abnormal card, certificate or bill; An abnormality proportion corresponding to each category is obtained according to the number of abnormal card, certificate or bills in each category. According to the abnormal proportion, an abnormal card certificate bill is extracted for manual marking.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the steps of the method of any one of claims 1-8.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the program, implements the steps of the method of any one of claims 1-8.

Images