A deep feature-based abnormal data elimination method, system, device and storage medium

Abstract

This invention discloses a method, system, device, and storage medium for anomaly removal based on deep features. The method includes: constructing a shadow pattern annotation dataset based on multimodal data from an industrial site and an industrial target detection model; the multimodal data includes image data in different formats; preprocessing the shadow pattern annotation dataset to obtain a preprocessed shadow pattern annotation dataset; inputting the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain an original feature set; and using an unsupervised anomaly detection algorithm based on the original feature set to identify and remove anomalous annotation samples in the preprocessed shadow pattern annotation dataset, thus obtaining a target training dataset. This method combines an industrial target detection model to construct a shadow pattern annotation dataset and filters out anomalous annotation samples through feature extraction and anomaly detection, enabling rapid identification of anomalous data.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of anomaly detection technology, and in particular to an anomaly data removal method, system, device and storage medium based on deep features. Background Technology

[0002] During model training, the accuracy of data annotation directly impacts the model's final performance. Especially in industrial settings, the large amounts of inspection data flowing back from actual production lines or environments often contain various annotation anomalies, such as mislabeling and boundary deviations. Traditional methods relying on manual screening of anomalies are inefficient, costly, and struggle to guarantee annotation consistency. Therefore, automatically and quickly identifying anomaly-labeled data from massive datasets is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This invention provides a method, system, device, and storage medium for anomaly data removal based on deep features, in order to solve the problem that existing technologies cannot automatically and quickly identify anomaly labeled data.

[0004] According to one aspect of the present invention, a method for anomaly removal based on deep features is provided, the method comprising: Based on multimodal data from industrial sites and industrial target detection models, a shadow pattern annotation dataset is constructed. The multimodal data includes image data in different formats, and the shadow pattern annotation dataset includes image data and the inference annotation results corresponding to each image data. The shadow pattern annotation dataset is preprocessed to obtain the preprocessed shadow pattern annotation dataset; The preprocessed shadow pattern annotation dataset is input into a deep feature extraction network for feature extraction to obtain the original feature set. Based on the original feature set, an unsupervised anomaly detection algorithm is used to identify and remove anomalous labeled samples in the preprocessed shadow pattern labeled dataset, thus obtaining the target training dataset.

[0005] According to another aspect of the present invention, an anomaly removal system based on deep features is provided, the system comprising: The shadow data acquisition module is used to construct a shadow pattern annotation dataset based on multimodal data from industrial sites and industrial target detection models. The multimodal data includes image data in different formats, and the shadow pattern annotation dataset includes image data and inference annotation results corresponding to each image data. The preprocessing module is used to preprocess the shadow pattern annotation dataset to obtain the preprocessed shadow pattern annotation dataset. The feature extraction module is used to input the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain the original feature set; The data cleaning module is used to identify and remove abnormal labeled samples in the preprocessed shadow pattern labeled dataset based on the original feature set using an unsupervised anomaly detection algorithm, thereby obtaining the target training dataset.

[0006] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: at least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the deep feature-based anomaly removal method according to any embodiment of the present invention.

[0007] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the deep feature-based anomaly removal method according to any embodiment of the present invention.

[0008] This invention discloses a method, system, device, and storage medium for anomaly removal based on deep features. The method includes: constructing a shadow pattern annotation dataset based on multimodal data from an industrial site and an industrial target detection model; the multimodal data includes image data in different formats; preprocessing the shadow pattern annotation dataset to obtain a preprocessed shadow pattern annotation dataset; inputting the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain an original feature set; and using an unsupervised anomaly detection algorithm based on the original feature set to identify and remove anomalous annotation samples in the preprocessed shadow pattern annotation dataset, thereby obtaining a target training dataset. This method constructs an automatic and efficient anomaly annotation detection mechanism, constructs a shadow pattern annotation dataset in conjunction with an industrial target detection model, and filters out anomalous annotation samples through feature extraction and anomaly detection. It can quickly identify and remove problematic data, thereby optimizing annotation quality, improving the robustness and accuracy of model training, and solving the problem of existing technologies being unable to automatically and quickly identify anomalous annotation data.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating an anomaly data removal method based on deep features provided in Embodiment 1 of the present invention. Figure 2 A flowchart illustrating an anomaly removal method based on deep features provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an anomaly data removal system based on deep features provided in Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device using the deep feature-based anomaly data removal method according to an embodiment of the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention. It should be understood that the various steps described in the method embodiments of the present invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0013] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, any variations of the terms "comprising" and "having," etc., are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0017] Example 1 Figure 1 This is a flowchart illustrating an anomaly removal method based on deep features provided in Embodiment 1 of the present invention. This method is applicable to filtering out anomaly data in a dataset to obtain a clean dataset. This method can be executed by an anomaly removal system based on deep features, wherein the system can be implemented by software and / or hardware and is generally integrated on an electronic device. In this embodiment, the electronic device includes, but is not limited to, devices such as computers.

[0018] like Figure 1 As shown, an anomaly data removal method based on deep features provided in Embodiment 1 of the present invention includes the following steps: S110. Based on multimodal data from industrial sites and industrial target detection models, construct a shadow pattern annotation dataset; the multimodal data includes image data in different formats, and the shadow pattern annotation dataset includes image data and inference annotation results corresponding to each image data.

[0019] The industrial site can refer to any location where industrial activities take place, and the industrial site can vary depending on the application scenario. For example, an industrial site can be a factory production line, equipment maintenance site, manufacturing workshop, or new energy site. Multimodal data can include image data in different formats. Specifically, multimodal data can include at least one of Red-Green-Blue (RGB) images, depth maps, and normal vector maps. This image data can be images of various equipment and personnel in the industrial site in operation or at rest, or it can be an overall environmental image of the industrial site; this embodiment does not limit this. The industrial target detection model can refer to a model that performs anomaly detection on objects to be detected in an industrial scene. The shadow pattern annotation dataset can refer to a dataset constructed and annotated based on shadow patterns. The shadow pattern annotation dataset can include the original image data and the inference annotation results corresponding to each image data. The inference annotation results of the image data can be annotated using the industrial target detection model.

[0020] In this embodiment, a shadow pattern annotation dataset can be constructed based on multimodal data from the industrial site and an industrial target detection model. For example, multimodal data can be input into an industrial target detection model, and the data in the multimodal data can be annotated by the industrial target detection model. The shadow pattern annotation dataset can then be constructed based on the annotation results.

[0021] In one embodiment, constructing a shadow pattern annotation dataset based on multimodal data from an industrial site and an industrial target detection model includes: collecting multimodal data from an industrial site; inputting the multimodal data into an industrial target detection model to obtain inference annotation results; and forming a shadow pattern annotation dataset based on the multimodal data and the inference annotation results.

[0022] Among them, the inference annotation result can be the prediction result obtained by the model based on the input data. Specifically, the inference annotation result can be the bounding box annotation information of the objects in the image. The bounding box annotation information can include the location information of the object, as well as the object's category, confidence level, and other information.

[0023] In this embodiment, multimodal data from the industrial site can be collected by setting up sensors at the industrial site. The multimodal data is then input into the industrial target detection model to obtain the inference annotation results of each image data. Finally, the original image data and the corresponding inference annotation results of each image data in the multimodal data are used as samples in the shadow pattern annotation dataset.

[0024] This embodiment supports multimodal data input, including RGB images, depth maps, and normal vector maps, and can extract discriminative features for different data formats, enhancing the system's applicability in diverse scenarios.

[0025] S120. Preprocess the shadow pattern annotation dataset to obtain the preprocessed shadow pattern annotation dataset.

[0026] In this embodiment, after obtaining the shadow pattern annotation dataset, the sample data in the shadow pattern annotation dataset can be preprocessed by scaling, normalization, etc., to obtain the preprocessed shadow pattern annotation dataset.

[0027] In one embodiment, preprocessing the shadow pattern annotation dataset to obtain a preprocessed shadow pattern annotation dataset includes: uniformly scaling the image data in the shadow pattern annotation dataset to a fixed size to obtain uniform-sized image data; normalizing the pixel values ​​of the uniform-sized image data to obtain normalized image data; converting the inference annotation results corresponding to each image data into a normalized coordinate format to obtain normalized inference annotation results; and using the normalized image data and the normalized inference annotation results as the preprocessed shadow pattern annotation dataset.

[0028] Normalized coordinate format refers to the format in which the original pixel coordinates in an image are converted into standardized coordinates.

[0029] In this embodiment, for the image data in the shadow pattern annotation dataset, the image data can be uniformly scaled to a fixed size to obtain image data of uniform size. For the image data of uniform size, its pixel values ​​can be normalized to obtain normalized image data. In order to ensure that the inference annotation results correspond to the converted image data, for the inference annotation results corresponding to each image data, the inference annotation results can be converted into a normalized coordinate format to obtain normalized inference annotation results. Then, the normalized image data and the normalized inference annotation results are used as the preprocessed shadow pattern annotation dataset.

[0030] S130. Input the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain the original feature set.

[0031] In this context, a deep feature extraction network can be a network model composed of a multi-layer neural network structure. The original feature set can refer to the set of features extracted from various image data.

[0032] In this embodiment, the preprocessed shadow pattern annotation dataset can be input into a deep feature extraction network for feature extraction, and an original feature set can be constructed based on the extraction results of each sample data.

[0033] In one embodiment, the step of inputting the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain an original feature set includes: inputting the preprocessed shadow pattern annotation dataset into a deep feature extraction network, wherein the deep feature extraction network adopts a convolutional neural network or a visual converter architecture; based on the deep feature extraction network, performing deep feature extraction on each sample in the shadow pattern annotation dataset to obtain a deep feature vector corresponding to each sample; and constructing an original feature set based on the deep feature vectors corresponding to all samples.

[0034] Here, Convolutional Neural Network (CNN) refers to a deep neural network that includes structures such as convolutional layers, pooling layers, and activation layers. Vision Transformer (ViT) refers to a visual model based on the Transformer architecture. Deep feature vectors can be high-dimensional features obtained after feature extraction from samples. Samples can refer to each image data in a shadow pattern annotation dataset.

[0035] In this embodiment, the preprocessed shadow pattern annotation dataset can be input into a deep feature extraction network. Based on the deep feature extraction network, deep features can be extracted from each sample in the shadow pattern annotation dataset to obtain the deep feature vector corresponding to each sample. Based on the deep feature vectors corresponding to all samples, the original feature set can be constructed.

[0036] The deep feature extraction network in this embodiment can receive and process multimodal labeled data (such as images, depth maps, normal vector maps, etc.), and automatically extract high-dimensional deep features for each labeled sample using pre-trained or task-adapted deep learning models (such as convolutional neural networks or feature encoding networks). The extracted features can effectively express the key semantic or structural information of the sample, forming the basic representation for subsequent anomaly detection.

[0037] Furthermore, after obtaining the original feature set, each deep feature vector in the original feature set can be normalized to eliminate the differences in dimensions and scales between different feature dimensions, thus obtaining a normalized feature set.

[0038] S140. Based on the original feature set, an unsupervised anomaly detection algorithm is used to identify and remove abnormal labeled samples in the preprocessed shadow pattern labeled dataset to obtain the target training dataset.

[0039] Unsupervised anomaly detection algorithms can be a type of machine learning method used to identify anomalies or outliers in a dataset. Anomaly-labeled samples can refer to samples whose inference labeling results may be anomalous.

[0040] In this embodiment, the original feature set can be analyzed using an unsupervised anomaly detection algorithm to identify anomalous labeled samples in the preprocessed shadow pattern labeled dataset. Removing the anomalous labeled samples will yield the target training dataset.

[0041] This fully automated anomaly labeling and identification process automatically extracts deep features from labeled data using deep learning networks and detects and removes anomalies by analyzing feature distribution. No manual intervention is required, significantly improving the efficiency and consistency of data cleaning. Furthermore, its modular design separates deep feature extraction from anomaly detection, allowing users to flexibly replace different feature extraction networks or anomaly detection algorithms (such as traditional statistical methods or deep learning methods) according to task requirements, enhancing the system's customizability and iteration convenience.

[0042] Furthermore, after obtaining the target training dataset, this embodiment can train the industrial target detection model based on the target training dataset to obtain a new industrial target detection model. In subsequent iterations, a shadow pattern annotation dataset can be constructed based on the new industrial target detection model.

[0043] This invention provides a method for anomaly removal based on deep features, comprising: constructing a shadow pattern annotation dataset based on multimodal data from an industrial site and an industrial target detection model; the multimodal data including image data in different formats; preprocessing the shadow pattern annotation dataset to obtain a preprocessed shadow pattern annotation dataset; inputting the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain an original feature set; and using an unsupervised anomaly detection algorithm based on the original feature set to identify and remove anomalous annotation samples in the preprocessed shadow pattern annotation dataset, thereby obtaining a target training dataset. This method constructs an automatic and efficient anomaly annotation detection mechanism, combines an industrial target detection model to construct a shadow pattern annotation dataset, and filters out anomalous annotation samples through feature extraction and anomaly detection. It can quickly identify and remove problematic data, thereby optimizing annotation quality, improving the robustness and accuracy of model training, and solving the problem of existing technologies being unable to automatically and quickly identify anomalous annotation data.

[0044] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0045] In one embodiment, the step of identifying and removing anomalous labeled samples in the preprocessed shadow pattern labeled dataset based on the original feature set using an unsupervised anomaly detection algorithm to obtain the target training dataset includes: analyzing the distribution of each sample in the feature space based on the original feature set using an unsupervised anomaly detection algorithm to identify anomalous labeled samples in the preprocessed shadow pattern labeled dataset that deviate from the main sample distribution; and removing the anomalous labeled samples from the preprocessed shadow pattern labeled dataset to obtain the target training dataset.

[0046] The feature space can be a vector space composed of all feature dimensions. The sample distribution can be the distribution of normal samples in the feature space.

[0047] In this embodiment, based on the original feature set, an unsupervised anomaly detection algorithm can be used to analyze the distribution of each sample in the feature space, identify the abnormal labeled samples in the preprocessed shadow pattern labeled dataset that deviate from the main distribution of the samples, and remove these abnormal labeled samples from the preprocessed shadow pattern labeled dataset to obtain the target training dataset.

[0048] In one embodiment, the unsupervised anomaly detection algorithm includes the Isolation Forest algorithm and / or kernel density estimation method.

[0049] Among them, the Isolation Forest algorithm can be considered an unsupervised anomaly detection algorithm based on the idea of ​​isolation. This algorithm recursively constructs multiple isolated trees to form a forest through random feature selection and random segmentation values. Kernel density estimation methods can be nonparametric statistical methods used to estimate unknown probability density functions from sample data.

[0050] In this embodiment, the unsupervised anomaly detection algorithm can be the Isolation Forest algorithm, the kernel density estimation method, or both algorithms can be used simultaneously as the unsupervised anomaly detection algorithm.

[0051] This embodiment is based on deep features extracted by a deep feature extraction network. By analyzing the distribution of features in the feature space, it can automatically identify anomalous labeled samples that deviate from the main distribution. It supports a variety of anomaly detection strategies, including traditional density-based methods (such as local outlier factor and isolated forest) as well as deep learning-based anomaly detection models (such as autoencoder reconstruction error detection and energy models). Users can choose or replace detection methods according to the characteristics of the task. The detected anomalous labels will be automatically marked or removed, outputting a clean and highly consistent labeled dataset for subsequent model training or evaluation.

[0052] For example, when an unsupervised anomaly detection algorithm uses the Isolation Forest algorithm, the Isolation Forest algorithm constructs multiple isolation trees on the original feature set (or the normalized feature set). It obtains the anomaly score by calculating the path length required for each sample in the feature set to be isolated, and judges the samples with scores higher than a preset threshold as anomalies.

[0053] When an unsupervised anomaly detection algorithm adopts a detection method based on kernel density estimation, it can use kernel density estimation to model the probability density distribution of normal labeled samples in the feature space based on the original feature set (or the normalized feature set), calculate the log probability density value of each sample in the feature set under the distribution, and determine the samples with probability density values ​​lower than the preset dynamic threshold as anomalies.

[0054] In one embodiment, after identifying the abnormal labeled samples, the method further includes: generating an audit report based on the abnormal score of the abnormal labeled samples, the reason for the abnormality determination, and the original multimodal data corresponding to the abnormal labeled samples; and performing manual review and model defect analysis based on the audit report.

[0055] Among them, the anomaly score can be the degree of anomaly of the anomaly-labeled sample, and the anomaly determination reason can be the reason why the sample is abnormal.

[0056] In this embodiment, after identifying anomalous labeled samples, an audit report can be generated based on the anomalous scores, reasons for anomaly determination, and the corresponding original multimodal data of the anomalous labeled samples. This audit report can then be used for manual review and model defect analysis. For example, the automatically removed anomalous labeled samples and their corresponding original multimodal data, along with their anomalous scores and reasons for determination, can be combined to generate a visual audit report for algorithm engineers to conduct manual review and model defect analysis.

[0057] Based on the technical solutions of the above embodiments, this invention provides several specific implementation methods.

[0058] As one specific implementation method of this embodiment. Figure 2 A flowchart illustrating an anomaly removal method based on depth features provided in this embodiment of the invention is shown below. Figure 2 As shown, the method includes: Step (1): Obtain the shadow pattern annotation dataset. The shadow pattern annotation dataset is obtained by inputting multimodal data collected from the industrial field into a pre-trained industrial target detection model for inference. The multimodal data can include at least one of RGB images, depth maps, and normal vector maps. The shadow pattern data includes the input data of the industrial target detection model and its corresponding automatically generated inference annotation results.

[0059] The pre-trained industrial target detection model can be an initial model trained on historical labeled data, while the shadow mode data is its inference output on new unlabeled or weakly labeled data, used to simulate the model's labeling behavior in actual deployment.

[0060] Step (2): Preprocess the shadow pattern data. Scale the images to a fixed size, normalize the pixel values, and convert all inference-generated bounding box annotations to a normalized coordinate format.

[0061] Feature normalization can be performed using the Z-Score standardization method, the formula of which is: ; in, This represents a value in one dimension of the original feature vector. This is the mean of that dimension over the entire feature set. Standard deviation, This is the normalized value.

[0062] Step (3): Build and initialize a deep feature extraction network that uses a convolutional neural network or visual Transformer architecture and whose input is adapted to multimodal data.

[0063] The weights of the deep feature extraction network can be initialized with the backbone network weights of a pre-trained industrial target detection model and then frozen or fine-tuned during this cleaning process.

[0064] Step (4): Input the preprocessed shadow pattern data into the deep feature extraction network to extract the corresponding deep feature vector for each sample in the dataset, forming the original feature set.

[0065] Step (5): Normalize each feature vector in the original feature set to eliminate the differences in dimensions and scales between different feature dimensions, and obtain the normalized feature set.

[0066] Step (6): Based on the normalized feature set, an unsupervised anomaly detection algorithm is used to analyze the distribution of samples in the feature space and automatically identify anomaly labeled samples that deviate from the main distribution density. Anomaly labels may include erroneous bounding boxes or category labels generated by model false detections, false misses, or inaccurate localization.

[0067] Step (7): Remove the abnormal labeled samples identified in step (6) from the shadow pattern labeled dataset to generate a clean and highly consistent training dataset; Step (8): Use a clean training dataset to retrain or incrementally fine-tune the industrial target detection model to optimize and improve model performance.

[0068] The method in this embodiment focuses on using deep learning and feature distribution analysis to automatically identify and remove outlier samples in labeled data, thereby improving data quality and the reliability of subsequent model training. Deployed as a closed-loop learning component, this method continuously collects shadow pattern data from the production environment, periodically and automatically performs anomaly cleaning and triggers iterative retraining of the model, achieving continuous optimization of model performance.

[0069] Example 2 Figure 3 This is a schematic diagram of an anomaly removal system based on deep features provided in Embodiment 2 of the present invention. The system is applicable to filtering out anomaly data in a dataset to obtain a clean dataset. The system can be implemented by software and / or hardware and is generally integrated into an electronic device.

[0070] like Figure 3 As shown, the system includes: The shadow data acquisition module 210 is used to construct a shadow pattern annotation dataset based on multimodal data from the industrial site and an industrial target detection model; the multimodal data includes image data in different formats, and the shadow pattern annotation dataset includes image data and inference annotation results corresponding to each image data; Preprocessing module 220 is used to preprocess the shadow pattern annotation dataset to obtain a preprocessed shadow pattern annotation dataset; Feature extraction module 230 is used to input the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain the original feature set; The data cleaning module 240 is used to identify and remove abnormal labeled samples in the preprocessed shadow pattern labeled dataset based on the original feature set using an unsupervised anomaly detection algorithm, thereby obtaining the target training dataset.

[0071] This embodiment provides an anomaly removal system based on deep features, comprising: a shadow data acquisition module for constructing a shadow pattern annotation dataset based on multimodal data from an industrial site and an industrial target detection model; the multimodal data includes image data in different formats, and the shadow pattern annotation dataset includes image data and inference annotation results corresponding to each image data; a preprocessing module for preprocessing the shadow pattern annotation dataset to obtain a preprocessed shadow pattern annotation dataset; a feature extraction module for inputting the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain an original feature set; and a data cleaning module for using an unsupervised anomaly detection algorithm to identify and remove anomalous annotation samples in the preprocessed shadow pattern annotation dataset based on the original feature set, thereby obtaining a target training dataset. This system constructs an automatic and efficient anomaly annotation detection mechanism, constructs a shadow pattern annotation dataset in conjunction with an industrial target detection model, and filters out anomalous annotation samples through feature extraction and anomaly detection. It can quickly identify and remove problematic data, thereby optimizing annotation quality, improving the robustness and accuracy of model training, and solving the problem of existing technologies being unable to automatically and quickly identify anomalous annotation data.

[0072] Furthermore, the shadow data acquisition module 210 includes: Collect multimodal data from industrial sites; The multimodal data is input into the industrial target detection model to obtain the inference and annotation results; A shadow pattern annotation dataset is formed based on the multimodal data and the inference annotation results.

[0073] Furthermore, the preprocessing module 220 includes: The image data in the shadow pattern annotation dataset is uniformly scaled to a fixed size to obtain image data of uniform size; The pixel values ​​of the uniform-sized image data are normalized to obtain normalized image data; The inference annotation results corresponding to each of the image data are converted into normalized coordinate format to obtain normalized inference annotation results; The normalized image data and the normalized inference annotation results are used as the preprocessed shadow pattern annotation dataset.

[0074] Furthermore, the feature extraction module 230 includes: The preprocessed shadow pattern annotation dataset is input into a deep feature extraction network, which adopts a convolutional neural network or a visual converter architecture. Based on the deep feature extraction network, deep feature extraction is performed on each sample in the shadow pattern annotation dataset to obtain the deep feature vector corresponding to each sample. Based on the deep feature vectors corresponding to all samples, the original feature set is constructed.

[0075] Furthermore, the data cleaning module 240 includes: Based on the original feature set, an unsupervised anomaly detection algorithm is used to analyze the distribution of each sample in the feature space, and to identify the abnormal labeled samples that deviate from the main sample distribution in the preprocessed shadow pattern labeled dataset. The abnormal labeled samples are removed from the preprocessed shadow pattern labeled dataset to obtain the target training dataset.

[0076] Furthermore, the unsupervised anomaly detection algorithm includes the isolated forest algorithm and / or kernel density estimation method.

[0077] Furthermore, after identifying anomalous labeled samples, the data cleaning module 240 also includes: An audit report is generated based on the anomaly scores, anomaly determination reasons, and the original multimodal data corresponding to the anomaly-labeled samples. Based on the audit report, a manual review and model defect analysis were conducted.

[0078] The above-described anomaly removal system based on depth features can execute the anomaly removal method based on depth features provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0079] Example 3 Figure 4 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0080] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0081] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0082] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as anomaly removal methods based on deep features.

[0083] In some embodiments, the depth-feature-based anomaly removal method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the depth-feature-based anomaly removal method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the depth-feature-based anomaly removal method by any other suitable means (e.g., by means of firmware).

[0084] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0085] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0086] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0087] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0088] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0089] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0090] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0091] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for anomaly removal based on deep features, characterized in that, The method includes: Based on multimodal data from industrial sites and industrial target detection models, a shadow pattern annotation dataset is constructed. The multimodal data includes image data in different formats, and the shadow pattern annotation dataset includes image data and the inference annotation results corresponding to each image data. The shadow pattern annotation dataset is preprocessed to obtain the preprocessed shadow pattern annotation dataset; The preprocessed shadow pattern annotation dataset is input into a deep feature extraction network for feature extraction to obtain the original feature set. Based on the original feature set, an unsupervised anomaly detection algorithm is used to identify and remove anomalous labeled samples in the preprocessed shadow pattern labeled dataset, thus obtaining the target training dataset.

2. The method according to claim 1, characterized in that, The shadow pattern annotation dataset, constructed based on multimodal data from industrial sites and industrial target detection models, includes: Collect multimodal data from industrial sites; The multimodal data is input into the industrial target detection model to obtain the inference and annotation results; A shadow pattern annotation dataset is formed based on the multimodal data and the inference annotation results.

3. The method according to claim 1 or 2, characterized in that, The preprocessing of the shadow pattern annotation dataset to obtain the preprocessed shadow pattern annotation dataset includes: The image data in the shadow pattern annotation dataset is uniformly scaled to a fixed size to obtain image data of uniform size; The pixel values ​​of the uniform-sized image data are normalized to obtain normalized image data; The inference annotation results corresponding to each of the image data are converted into normalized coordinate format to obtain normalized inference annotation results; The normalized image data and the normalized inference annotation results are used as the preprocessed shadow pattern annotation dataset.

4. The method according to claim 1, characterized in that, The preprocessed shadow pattern annotation dataset is input into a deep feature extraction network for feature extraction to obtain an original feature set, including: The preprocessed shadow pattern annotation dataset is input into a deep feature extraction network, which adopts a convolutional neural network or a visual converter architecture. Based on the deep feature extraction network, deep feature extraction is performed on each sample in the shadow pattern annotation dataset to obtain the deep feature vector corresponding to each sample. Based on the deep feature vectors corresponding to all samples, the original feature set is constructed.

5. The method according to claim 1, characterized in that, Based on the original feature set, an unsupervised anomaly detection algorithm is used to identify and remove anomalous labeled samples in the preprocessed shadow pattern labeled dataset, resulting in the target training dataset, including: Based on the original feature set, an unsupervised anomaly detection algorithm is used to analyze the distribution of each sample in the feature space, and to identify the abnormal labeled samples that deviate from the main sample distribution in the preprocessed shadow pattern labeled dataset. The abnormal labeled samples are removed from the preprocessed shadow pattern labeled dataset to obtain the target training dataset.

6. The method according to claim 5, characterized in that, The unsupervised anomaly detection algorithm includes the isolated forest algorithm and / or kernel density estimation method.

7. The method according to claim 5, characterized in that, After identifying anomalous labeled samples, the method further includes: An audit report is generated based on the anomaly scores, anomaly determination reasons, and the original multimodal data corresponding to the anomaly-labeled samples. Based on the audit report, a manual review and model defect analysis were conducted.

8. An anomaly removal system based on deep features, characterized in that, The system includes: The shadow data acquisition module is used to construct a shadow pattern annotation dataset based on multimodal data from industrial sites and industrial target detection models. The multimodal data includes image data in different formats, and the shadow pattern annotation dataset includes image data and inference annotation results corresponding to each image data. The preprocessing module is used to preprocess the shadow pattern annotation dataset to obtain the preprocessed shadow pattern annotation dataset. The feature extraction module is used to input the preprocessed shadow pattern annotation dataset into a deep feature extraction network for feature extraction to obtain the original feature set; The data cleaning module is used to identify and remove abnormal labeled samples in the preprocessed shadow pattern labeled dataset based on the original feature set using an unsupervised anomaly detection algorithm, thereby obtaining the target training dataset.

9. An electronic device, characterized in that, The device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the deep feature-based anomaly removal method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the deep feature-based anomaly removal method according to any one of claims 1-7.

Images