Industrial controller screen display anomaly detection method
Through the industrial controller screen display abnormality detection method of deep learning model and multi-model comprehensive judgment, the problems of low detection accuracy, low efficiency and high hardware cost in the prior art are solved, and efficient detection of various scenarios is achieved, reducing the Hardware and environmental requirements.
Patent Information
- Application Number
- CN202411302838.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2025-01-07
AI Technical Summary
The existing industrial screen display abnormality detection technology has low accuracy, low efficiency and depends on labor, especially in scenarios where negative samples are difficult to collect, with poor results, high hardware costs and strict environmental requirements.
Using the industrial controller screen display anomaly detection method based on deep learning, through image acquisition, processing and normalization, combined with three unsupervised deep learning models and bagging modules of Padim, Patchcore and Fastflow, comprehensive multi-model judgment and dynamic adjustment Threshold, reducing hardware and environmental requirements.
It improves the detection effect of a variety of positive and negative samples, reduces hardware costs, improves detection accuracy and efficiency, and reduces dependence on negative samples. It is suitable for commercial industrial environments.
Smart Images
Figure CN119273636A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of industrial screen production detection, and in particular to a method for detecting abnormal display on an industrial controller screen. Background Art
[0002] The biggest feature of smart factories is that they require fewer people and have a high degree of automation. The construction of factories needs to be deeply integrated with artificial intelligence technology to break through the bottleneck of automation and improve efficiency. As the production capacity of factories continues to increase, a large amount of manpower is needed to conduct product quality inspections to prevent defective products from entering the market.
[0003] At present, most industrial anomaly detection focuses on areas such as product surface defects and breakage. For example, "A printed mobile phone case surface defect detection device, system and method based on machine vision and deep learning". For display screens, most of the work is to detect anomalies in physical devices such as liquid crystal wafers, such as "A method and system for detecting LCD screen defects based on deep learning" and "A method for detecting display screen lighting defects based on deep learning". For packaged products, display content anomalies, such as screen flower screens, horizontal and vertical stripes caused by abnormal leakage, and incorrect displayed content. Due to the variety of display anomalies, the difficulty in collecting negative samples, and the serious imbalance of positive and negative samples, it currently relies mainly on manual detection, which restricts the degree of automation of smart factories.
[0004] Existing technologies are all targeted at scenarios with fixed positive samples, and the effect is poor when applied to scenarios with diverse positive and negative samples. The so-called diversity of positive samples means that the content displayed on the screens of different specific products on the same production line is different. The current deep learning-based solutions rely on supervised models and require the collection of a large number of negative samples, which cannot be applied to scenarios where negative samples are difficult to collect. At the same time, existing technologies have high environmental requirements and require hardware support such as black boxes, professional industrial cameras, and professional light sources, which are costly. Summary of the invention
[0005] In view of the problems of low accuracy, low efficiency, high labor cost, etc. of screen display abnormality detection, the present invention provides a method for detecting industrial controller screen display abnormality based on deep learning.
[0006] The solution adopted by the present invention to solve the technical problem is: a method for detecting abnormal display on the screen of an industrial controller, comprising the following steps:
[0007] Step 1: Image acquisition
[0008] Capture controller images through a camera;
[0009] Step 2: Image Processing
[0010] The controller display screen area is cropped out of the acquired image, the cropped image is converted into RGB color space, and the converted image is normalized;
[0011] Step 3: Screen abnormality detection
[0012] The normalized image is judged for abnormality by the abnormality detection algorithm, and the abnormality detection algorithm outputs the detection result.
[0013] The anomaly detection algorithm adopts three models to comprehensively judge whether the content of the display screen is abnormal. The three models include Padim, patchcore and Fastflow. At the same time, the anomaly detection algorithm also includes a bagging module; the normalized image is first passed through two unsupervised deep learning models of Padim and patchcore trained with positive samples, and the model detection results are sent to the bagging module; when the two models simultaneously judge that the image is abnormal, the display is judged to be abnormal, and when both models judge that it is normal, it is judged that the display is normal; when one model judges that it is normal and the other model judges that it is abnormal, the normalized image enters the trained Fastflow unsupervised deep learning model for judgment. When Fastflow judges that it is abnormal, the algorithm judges that the screen display is abnormal, otherwise it is judged that the display is normal.
[0014] Furthermore, the model judgment threshold and comprehensive judgment detection structure are set through the bagging module, where the threshold of the Patchcore model is set to be lower than 0.5; the threshold of the Padim model is set to be 0.5;
[0015] The Fastflow model samples dynamic thresholds, which change with the environment. The specific strategy is:
[0016] The threshold of the Fastflow model is dynamically adjusted based on the weighted scores of the Patchcore model and the Padim model. The higher the scores of the Patchcore model and the Padim model, the higher the threshold of the Fastflow model, and vice versa. The specific formula is as follows:
[0017]
[0018] Where threshold fastflow Indicates the threshold of the Fastflow model, result padim Indicates the value output by the padim model, result patchcore It represents the output value of the patchcore model, and α and β represent weights.
[0019] Furthermore, in the model training phase of the anomaly detection algorithm, the image is in the RGB domain and is a normalized image. The image normalization method is
[0020] x′=(x-min(x)) / (max(x)-min(x))
[0021] Where X is the original data.
[0022] Furthermore, the image acquisition uses an industrial camera to acquire the image of the controller, the industrial camera is fixed vertically above the controller to be detected, the industrial camera is connected to an edge computing device, the edge computing device is connected to a host computer, the host computer controls the detection to start, the edge computing device performs the detection according to the host computer control signal, and sends the detection result to the host computer. The anomaly detection algorithm is deployed on the edge computing device.
[0023] Beneficial effects of the present invention: The industrial controller screen display anomaly detection method provided by the present invention can solve the reliance on negative samples in the existing detection technology, and only requires positive samples for unsupervised deep learning model training. Improve the detection effect of various scenes of positive and negative samples to reach commercial level. At the same time, reduce the dependence on the environment, only need ordinary industrial cameras, normal indoor environment, indoor ambient lights to meet the requirements, and reduce hardware costs. Solve the problems of low accuracy, low efficiency, high labor costs and so on of screen display anomaly detection.
[0024] The industrial controller screen display anomaly detection method of the present invention has relatively low requirements for the deployment environment and can save hardware costs. The technical solution of comprehensive judgment of multiple models and dynamic thresholds can ensure the recall rate and improve the accuracy, thereby improving the overall detection effect. At the same time, the unsupervised deep learning model is adopted to reduce the dependence on negative samples. The algorithm adopts deep learning, which enables it to process more complex and diverse images, increasing the application scope of the technical solution. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a schematic diagram of the deployment of abnormality detection equipment displayed on the industrial controller screen.
[0026] Figure 2 It is a frame view of the industrial controller screen showing the anomaly detection process.
[0027] Figure 3 It is a screen anomaly detection process frame view.
[0028] Figure 4 This is the anomaly detection algorithm box view. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below.
[0030] Embodiment 1: The present invention implements a device, system and method for detecting screen display anomalies of industrial controllers based on deep learning, which solves the problems of low accuracy, low efficiency and high labor costs in screen display anomaly detection.
[0031] The industrial controller screen display abnormality detection method of the present invention provides a related screen detection environment through system deployment, such as Figure 1 As shown, the system deployment mode adopts edge computing, that is, the entire detection system runs on the edge end, and the edge end only exchanges simple detection results and signals with the host computer to reduce dependence on the network. The industrial camera is installed vertically above the controller, 30-40cm away from the screen. The camera field of view can complete the capture of the entire controller display. The edge computing device is connected to the camera via USB, and the edge end is connected to the host computer via a wireless network.
[0032] like Figure 2 As shown, the industrial controller screen display abnormality detection method is controlled by the host computer to control the detection start time. After the edge end receives the detection start signal, it performs the detection operation and sends the detection result to the host computer.
[0033] like Figure 3 As shown in the figure, the screen display anomaly detection process is as follows: when the anomaly detection starts, the camera captures the image, crops the display area from the captured image, converts the cropped image to the RGB color domain, and normalizes the converted image. Because the anomaly detection algorithm uses RGB domain images and normalized images during the model training phase, the image pre-processing during the deployment phase should be consistent with the training phase. The normalization method is
[0034] x′=(x-min(x)) / (max(x)-min(x))
[0035] Where X is the original data.
[0036] The normalized image enters the anomaly detection algorithm to determine whether it is abnormal, and the algorithm outputs the detection result.
[0037] like Figure 4 As shown, the anomaly detection algorithm described in the present invention uses three models to comprehensively judge whether the content of the display screen is abnormal. The normalized model first passes through two unsupervised deep learning models, Padim and patchcore, which are trained with positive samples, and the model detection results are sent to the bagging module. When the two models judge that the image is abnormal at the same time, the algorithm determines that the display is abnormal. When they are judged as normal at the same time, the algorithm determines that the display is normal. When one model judges that it is normal and the other judges that it is abnormal, the normalized image enters the trained Fastflow unsupervised deep learning model for judgment. When Fastflow judges that it is abnormal, the algorithm determines that the screen display is abnormal, otherwise it is judged that the display is normal.
[0038] Padim is a prototype-based unsupervised anomaly detection method that focuses on building a prototype representation of a "normal" sample during the training phase. Padim extracts image features using a pre-trained convolutional neural network (CNN) and transforms these features into a distribution in the prototype space. During the test phase, the model calculates the distance between the features of the input image and the prototype of the training set to evaluate the degree of anomaly. If the distance between the features of an image and the prototype is too large, the image is marked as anomaly.
[0039] PatchCore is an efficient unsupervised anomaly detection method that is particularly suitable for industrial defect detection. Similar to Padim, PatchCore also utilizes a pre-trained CNN to extract image features. However, instead of working directly on global features, PatchCore breaks down the image into multiple small blocks (patches) and extracts features from these patches. It then detects anomalies by calculating the nearest neighbor distance of each patch feature to a set of background features. A key advantage of PatchCore is its very fast processing speed because it avoids the time-consuming online learning process.
[0040] FastFlow is another efficient unsupervised anomaly detection framework that combines the concepts of manifold learning and self-supervised learning. The core idea of FastFlow is to find a manifold mapping function on the data manifold so that the distribution of normal samples on the manifold remains consistent, while abnormal samples deviate from this distribution. To achieve this, FastFlow uses a special loss function that encourages the model to learn a manifold structure that can distinguish between normal and abnormal samples. Like PatchCore, FastFlow also emphasizes fast detection capabilities while maintaining high detection accuracy.
[0041] The above three models have their own characteristics and may show different performance in different application scenarios. Among them, Padim may be more suitable for situations that require higher accuracy and can tolerate longer processing time; while PatchCore and FastFlow are more suitable for real-time or high-speed detection needs.
[0042] Detailed explanation of the Bagging module. The bagging module is mainly used to set the model judgment threshold and comprehensively judge the detection structure. For images, the closer the value of the model output is to 1, the more likely the model is abnormal, and the closer it is to 0, the more likely it is normal. Industrial anomaly detection is generally oriented towards full recall rate, that is, all abnormal screens must be detected. This requires that the abnormal judgment threshold cannot be set too high, but a lower threshold will increase the number of normal screens judged as abnormal screens, reducing the accuracy of detection. In order to improve the accuracy while ensuring the recall rate, the idea of bagging is introduced, and multiple models are comprehensively judged. The threshold determination is as follows:
[0043] 1. Set a lower threshold for the Patchcore model, the threshold is lower than 0.5, to ensure the recall rate, ensuring that at least one model will not miss abnormal situations;
[0044] 2. Set a normal threshold for the Padim model, and set the threshold to 0.5 to improve the accuracy;
[0045] 3. Because the environment of the entire system is relatively open, changes in the environment will cause some differences between the images actually collected by the camera and the distribution of the training data. This leads to the fact that when the actual deployment is carried out, if the environment is close to the training set, the overall model output score is low, and if the distribution is very different from the training set, the overall model output score is high. Therefore, the Fastflow model samples a dynamic threshold, and the threshold changes with the environment. The specific strategy is,
[0046] The threshold of the Fastflow model is dynamically adjusted based on the weighted scores of the Patchcore model and the Padim model. The overall solution is that the higher the scores of the first two models, the higher the threshold of the Fastflow model, and vice versa. The specific formula is as follows:
[0047]
[0048] Where threshold fastflow Indicates the threshold of the Fastflow model, result padim Indicates the value output by the padim model, result patchcore It represents the output value of the patchcore model, and α and β represent weights.
[0049] Furthermore, the three unsupervised deep learning models used in the anomaly detection algorithm of the present invention are not limited to the three models of Padim, patchcore, and Fastflow. Other types of unsupervised models can be used, and supervised models can also be used. The parameters and specific values of the dynamic threshold part need to be adjusted according to the actual situation.
[0050] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
Claims
1. A method for detecting abnormal display on an industrial controller screen, characterized in that: Step 1: Image acquisition Capture controller images through a camera; Step 2: Image Processing The controller display screen area is cropped out of the acquired image, the cropped image is converted into the RG B color gamut, and the converted image is normalized; Step 3: Screen abnormality detection The normalized image is judged for abnormality by the abnormality detection algorithm, and the abnormality detection algorithm outputs the detection result. The anomaly detection algorithm adopts three models to comprehensively judge whether the content of the display screen is abnormal. The three models include Padim, patchcore and FastflOW. At the same time, the anomaly detection algorithm also includes a bagging module; the normalized image is first passed through two unsupervised deep learning models of Padim and patchcore trained with positive samples, and the model detection results are sent to the bagging module; when the two models judge that the image is abnormal at the same time, the display is judged to be abnormal, and when they are judged to be normal at the same time, the display is judged to be normal; when one model judges to be normal and the other model judges to be abnormal, the normalized image enters the trained Fastflow unsupervised deep learning model for judgment. When Fastflow judges to be abnormal, the algorithm judges that the screen display is abnormal, otherwise it is judged to be normal.
2. The method for detecting abnormality of screen display of an industrial controller according to claim 1, characterized in that: The model judgment threshold and comprehensive judgment detection structure are set through the bagging module. The threshold of the Patchcore model is set to be lower than 0.5; the threshold of the Padim model is set to 0.5; the Fastflow model samples the dynamic threshold, and the threshold changes with the environment. The specific strategy is as follows: The threshold of the Fastflow model is dynamically adjusted based on the weighted scores of the Patchcore model and the Padim model. The higher the scores of the Patchcore model and the Padim model, the higher the threshold of the Fastflow model, and vice versa. The specific formula is as follows: Where threshold fasflow Indicates the threshold of the Fastflow model, result padim Indicates the value output by the padim model, result patch core It represents the output value of the patchcore model, and α and β represent weights.
3. The method for detecting abnormality of screen display of industrial controller according to claim 1, characterized in that: In the model training phase of the anomaly detection algorithm, the image is in the RG B domain and is a normalized image. The image normalization method is x′=(x-min(x)) / (max(x)-min(x)) Where X is the original data.
4. The method for detecting abnormality of screen display of an industrial controller according to claim 1, characterized in that: The image acquisition uses an industrial camera to acquire the image of the controller. The industrial camera is vertically fixed above the controller to be detected. The industrial camera is connected to an edge computing device, and the edge computing device is connected to a host computer. The host computer controls the detection to start. The edge computing device performs the detection according to the host computer control signal and sends the detection result to the host computer.
5. The method for detecting abnormality of screen display of industrial controller according to claim 4, characterized in that: The anomaly detection algorithm is deployed on edge computing devices.