An abnormality monitoring method for an instrument panel

Abstract

The application discloses an abnormality monitoring method of an instrument and meter cabinet, which comprises collecting instrument and meter cabinet data, obtaining an extended image data set, uniformly marking target image data and position signals in the extended image data set, forming a marking file, constituting a target detection network, saving mAP values and weight files of the network, reading real-time videos, calling network weight w1 to select a target frame, determining an abnormality type, coordinates of an abnormality image and confidence information according to data results. The application can not only realize monitoring and fault diagnosis of equipment, provide convenience for remote operation and maintenance protection, but also provide a basis and premise for future abnormality trend and even abnormality prediction by forming data characteristics for a long period of time.

Description

Technical Field

[0001] This invention belongs to the field of control system monitoring, and more specifically relates to a method for anomaly monitoring of an instrument cabinet. Background Technology

[0002] Instrument cabinets are used in many fields for equipment installation and control, serving as indispensable central control components in systems such as subways, power grids, railways, and buildings. Therefore, monitoring instrument cabinet faults is a necessary requirement. Currently, many systems employ intelligent technologies that primarily collect and analyze sensor data. For example, they might obtain instrument readings and trigger alarms if the readings are too high or if indicator light malfunctions are detected. However, they lack continuous analysis over extended periods, and the analytical models and methods are also inadequate. Furthermore, methods for summarizing and generalizing patterns based on this continuous analysis are missing. This is precisely why long-term maintenance-free operation is a crucial technological approach. Such intelligent methods cannot truly meet the need to replace manual labor; they cannot completely replace manual work with machines.

[0003] Meanwhile, equipment failures vary from device to device, and most devices do not have a high failure rate. Existing technologies mostly involve periodic, short-term, and fixed-point inspections. Repeated inspections of a large number of health cabinets can easily lead to a waste of resources and make it impossible to analyze and predict abnormal conditions of the cabinets. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes an anomaly monitoring method for instrument cabinets. This method not only identifies defects in the instrument cabinets but also performs intelligent analysis of anomaly types, thereby achieving the purpose of safety protection.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution: A method for anomaly monitoring of an instrument cabinet includes the following steps: (1) Collect data from the instrument cabinet and obtain an expanded image dataset; (2) The target image data and location signals in the expanded image dataset are uniformly labeled to form a label file, which constitutes the target detection network. The mAP value and weight file of the network are saved. When the mAP value reaches the usability of the network, the network architecture and model are no longer adjusted, and the network weights are saved. ; (3) Read real-time video and call network weights Select the target area and determine the anomaly type, coordinates of the anomaly image, and confidence level information based on the data results.

[0006] The method for determining the exception type is as follows: Let the target position i be denoted as ,in, The x-coordinate of the target location The vertical coordinate of the target position. For width, For height, Category labels for the target The probability that the system considers it to be the target; when the aspect ratio... Greater than the threshold At that time, the digit recognition network is invoked to obtain numerical results, and anomalies are determined based on the numerical values. The steps for the digit recognition network to determine the numerical results are as follows: collecting individual digit images to form a dataset, training a digit recognition network and saving its weights. The image to be tested is converted to grayscale and then subjected to grayscale projection. The processed image is then divided into n parts. When calling We obtain the numbers from n images, and then recombine them to obtain the numerical result. Aspect Ratio Not greater than the threshold At that time, the image form is distinguished according to the area, and the image data is input into the anomaly judgment network to determine whether it is an anomaly. The image form includes pointer, button, indicator light, handle and defect.

[0007] The algorithm for anomaly detection networks can employ methods such as individual network training, fusion network training, or image processing. The individual network training method includes the following steps: collecting normal and abnormal images of pointers, buttons, indicator lights, handles, and defects to train and form pointer networks, signal light networks, and defect networks, respectively, and performing data augmentation; calculating the weights of each network; and calling different network weights according to the image format to determine whether it is abnormal. The specific identification method is as follows: acquiring normal and abnormal samples, building a network model, and saving the weights of the network model when the accuracy of the network model reaches 99% or higher; real-time detection of video images, and after obtaining image data, calling the saved weights of the network model to determine whether the image data is abnormal; The fusion training network method includes the following steps: forming a comprehensive dataset by combining normal and abnormal images of pointers, buttons, indicator lights, handles, and defects, and performing data augmentation, corresponding to normal and abnormal labels, training the network and saving the weights, and calling the weights on the target location image to determine whether it is abnormal; Image processing methods include the following steps: The system performs preprocessing and thresholding on pointer, handle, and button images, emphasizes button height, fits the slope of pointer and handle, and makes judgments based on whether the slope of the real-time pointer and handle images or the height of the buttons are abnormal. Perform color analysis on the traffic light images and determine whether there are any abnormalities based on the real-time traffic light colors; The defect image is preprocessed, the defect area is calculated, and whether it is a defect is determined based on the defect area of ​​the real-time defect image; The image preprocessing includes grayscale conversion and binarization.

[0008] The specific steps for training an anomaly detection network are as follows: Let Q be the part remaining after removing the target from the original background. The resulting abnormal data image is processed by 11 methods: rotation, brightness increase, brightness decrease, filtering, noise addition, cropping, blurring, masking, binarization, LBP transformation, and Gabor transformation. The 11 processed result images are input into n anomaly judgment models. If all 11 outputs of each model are abnormal, the model is considered abnormal. If more than n / 2 models consider it abnormal, then the data image is considered abnormal data. The results of these 11 abnormal data are combined with multiple original background images to expand the image dataset as new samples.

[0009] The method for obtaining anomalies in the expanded image dataset is as follows: The original data is inspected in real time. If anomalies are found, the abnormal images are accumulated. When a certain amount of data is accumulated, it is placed in a discriminative network to determine that the accumulated data are all abnormal data, which is denoted as B. Assuming the original sample database is A, and the initial metric is the accuracy 'a' obtained through training, B is uploaded to expand the original database to form a new dataset A+B. The datasets A+B, A, and B are trained respectively, and the resulting accuracy values ​​are denoted as a1, a2, and a3. If both a1 and a3 are greater than a2, the original sample database is updated to A+B, and the metric is updated to the maximum value of a1 and a3; otherwise, no update is performed.

[0010] The image dataset is expanded using data augmentation methods, and the model is trained using a semi-supervised approach. The data augmentation methods include the following steps: (1) Obtain the width of the original image pic1 and height Define the original target in the original image as The four variables represent its horizontal coordinate, vertical coordinate, width, and height, respectively. (2) Randomly perform scaling operations on the original image and the original target and repeat the process multiple times to achieve data augmentation processing; The shrinking operation includes the following steps: Select a random number less than 1. As a scaling factor, the original target is reduced to a size with a width and height of [missing values]. , The new image pic2, the x-axis range of pic2 is The range of the vertical axis is Generate a space with a width and height of respectively. Image pic3, at location Place pic2 there. Then in pic3, the position of the target image pic2 is updated to... Repeat the above steps multiple times to obtain different enhanced images, and then store them in the database. The zoom-in operation is as follows: Take a value greater than 1 and less than 1 random numbers As a magnification ratio, let the original target be magnified to have a width and height of [missing information]. , The new image pic4, with its initial position randomly generated within the original image pic1. Place pic4; repeat the above steps multiple times to obtain different enhanced images, and then put them into the database; The training method for semi-supervised methods includes the following steps: (1) Construct two channels for training: the first is the original target, and the second is the image generated by the downsizing operation; (2) After training, the original target is used to obtain feature map fe1. The range of the horizontal coordinates of the position of fe1 is... The range of the vertical axis is ; (3) Train the image of the original target after it has been reduced in size to obtain the feature map fe2; (4) Using the Roi-Align method, the features of the corresponding region are extracted on the feature map fe1 based on the coordinate position of the original target, and the corresponding target feature is obtained as aim1; (5) The position of the target in the original image is updated to the position of pic2. Using the Roi-Align method, features of the corresponding region are extracted from the feature map fe2 based on the coordinate position of the target in the scaled-down image, and the corresponding target feature is obtained as aim2. (6) Fix the size of aim1 and aim2 by imresize operation, add a similarity loss of aim1 and aim2 to the loss function of the whole network, define the cross-correlation value or flatten the image into row vectors, and calculate the sum of absolute errors between row vectors; (7) Complete the training according to the above steps.

[0011] Drawing an abnormal trend curve based on the aforementioned anomaly includes the following steps: For abnormal numbers, 0 represents a high value, 1 represents a low value, and 2 represents a normal value. A continuous curve is created to obtain a graph in the form of numbers. For images that are defective, the historical data is plotted as a curve, with 0 for defective and 1 for non-defective. For other anomalies, six anomaly forms are defined, with 0 representing only one indicator light failure, 1 representing multiple indicator light failures, 2 representing abnormal handle position, 3 representing both 0 and 2, 4 representing both 1 and 2, and 5 representing a logical anomaly. The above data are plotted as a curve.

[0012] Preferably, a secure encryption scheme is adopted for the abnormal image, a watermark is embedded in the abnormal image, the watermark image is selected according to the pixel values ​​of the abnormality and Canny edge detection, a deep learning network is constructed according to the embedded watermark image to realize the discrimination of the embedded watermark frame, and a watermark extraction algorithm is used to extract the watermark.

[0013] A secure encryption scheme includes the following steps: Encryption process: (11) Collect abnormal images as training samples; (12) Number the five types of abnormalities: pointer, button, indicator light, handle, and defect, and label them as 1, 2, 3, 4, and 5; (13) Collect a large number of watermarked images, solve the image after Canny edge detection, solve the pixel sum for the image after Canny edge detection, divide the watermarked images into 5 groups according to the pixel sum, and denot them as A, B, C, D, E in descending order; (14) Establish a mapping relationship between the order of A, B, C, D, E and the abnormal labels 1, 2, 3, 4, 5. The mapping relationship is a non-sequential mapping. (15) When the video is detected in real time, if an anomaly is found in a certain frame, the corresponding image group to be embedded with the watermark is selected according to the sequence number of the anomaly. Two images are selected from the corresponding group watermark and the LSB algorithm is used to embed the video watermark. The center position of the two images is a random position near the center coordinate of the abnormal image. The decryption process employs deep learning algorithms and LSB to extract the video watermark and obtain information. (21) Select several images containing abnormal scenes to embed watermarks, select several images containing abnormal scenes to not embed watermarks, and select several images without watermarks and without abnormal scenes. The training set consists of three parts. (22) Convolutional neural networks are built on the training set of the first part, and the VGG16 deep learning algorithm is used to train the above three parts. The network performance is obtained through the LOSS curve and accuracy curve, and the weight file is saved. (23) When the video is decrypted, the video is converted into frames, each frame is verified, and the weights of the above steps are called. If the frame is determined by the network to be an image with an embedded watermark and containing an abnormal scene, the LSB algorithm will be used to extract the watermark. (24) After the watermark is extracted, solve the Canny edge detection image, solve the pixel sum of the image after the Canny edge detection, and determine which category (A, B, C, D, E) the image belongs to according to the size of the pixel sum; (25) Obtain the anomaly label based on the mapping relationship; (26) Embed watermarks into abnormal images according to the mapping relationship to expand the original training set, perform target detection on these abnormalities, and obtain network weights; (27) If the anomaly number obtained when the current detection frame is input to the network weight in step 26 is consistent with that in step 25, then the decoding work is completed; if they are inconsistent or no watermark is extracted, then the video is determined to be unencrypted. Preferably, the step of determining an anomaly in the pointer image format includes: (1) To build a network for object detection, collect pointer images and label the pointer positions to obtain a dataset, and train it using Faster R-CNN; (2) When a real-time image is input, the network in step 1 is called to obtain the coordinates of the pointer, and the pointer image is saved separately as an image; (3) Construct datasets with separate pointer graphs and pointer degrees, construct AlexNet networks on the datasets, and implement the mapping between separate pointer graphs and pointer degrees; (4) For the graph obtained in step 2, call the network in step 3 to obtain the pointer degree P1; (5) For the pointer graph obtained in 2, the hough transformation operation is performed to obtain the largest line segment. The slope corresponding to the largest line segment is calculated. Based on the mapping relationship between slope and degree, the pointer degree p2 is obtained. (6) Define the accuracy of the pointer degree obtained by deep learning as acc1, and the accuracy obtained by hough as acc2; (7) The final pointer degree is p1 (acc1 / (acc1+acc2))+p2 (acc2 / (acc1+acc2)).

[0014] The advantages of this invention are as follows: It designs a fully automatic, all-weather fault monitoring process, which monitors different types of images (such as pointers, indicator lights, defects, numerical dials, handles, etc.) in real time, collects images of normal and fault displays to form an anomaly dataset, uses a deep learning model to quantify each value, builds a network model, makes up for the lack of samples in the real environment, and performs more accurate real-time anomaly detection, effectively achieving automatic protection; and it protects the original real video from tampering, so that the video data can be restored after adding a watermark, thereby verifying whether the video has been tampered with.

[0015] A novel data augmentation network based on model validation is proposed. After data self-assertion, it is input into multiple node models for validation. After multi-party authentication confirms that the data is an anomaly, it is augmented into the anomaly dataset. This invention not only enables equipment monitoring and fault diagnosis, providing convenience for remote operation and maintenance protection, but also provides a foundation and premise for future abnormal trends and even anomaly prediction by forming data characteristics over a long period of time. Attached Figure Description

[0016] Figure 1 This is a flowchart of the present invention; Figure 2 A flowchart of a digital recognition network is established for Embodiment 1 of the present invention; Figure 3 Flowchart for establishing an expanded image dataset in Embodiment 1 of the present invention; Figure 4 An anomaly detection network flowchart is established for Embodiment 1 of the present invention; Figure 5 This is a flowchart of the anomaly detection network training in Embodiment 1 of the present invention. Detailed Implementation

[0017] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0018] Example 1 This embodiment discloses a method for anomaly monitoring of an instrument cabinet, including the following steps: (1) Collect a large amount of instrument cabinet data and perform data augmentation on it, such as adding noise, removing noise, changing brightness, changing contrast, rotating, cropping, mixup, binarization, color space changes, image LBP, Gabor changes, etc., to obtain a new expanded image dataset.

[0019] (2) Label the target image data in the expanded image dataset to form a labeled JSON file, construct the target detection network. The backbone network can be AlexNet, VGG, GoogleNet, YOLO series detection networks, SSD network, FasterECNN network, etc., or you can build your own network or set up a semi-supervised network. Save the network's mAP value and weight file. When the mAP value reaches the usability of the network, no further adjustments will be made to the network architecture and model. Save the network weights. .

[0020] (3) Read real-time video and call network weights The target is selected by bounding box, and the target can be individually generated into a JPG image and stored in the cache area for further detection.

[0021] (4) Based on the results obtained from the above network, the region of interest is obtained, and then the values ​​of horizontal and vertical coordinates and length and width can be obtained. When the length and width ratio is large, it is considered to be in the form of digital display. The digital display recognition network is called to obtain the digital results, and the abnormality is judged based on the values.

[0022] The following is a configurable recognition method: The target position i is represented as... ,in, The x-coordinate of the target location The vertical coordinate of the target position. For width, For height, Category labels for the target The probability that the system considers it to be the target; when the aspect ratio... Greater than the threshold At that time, the digit recognition network is invoked to obtain numerical results, and the presence of anomalies is determined based on the values; aspect ratio Not greater than the threshold At that time, the image form is distinguished according to the area, and the image data is input into the anomaly judgment network to determine whether it is an anomaly. The image form includes pointer, button, indicator light, handle and defect.

[0023] In this embodiment, the distinction between image formats is defined as follows: If area satisfy The image is in pointer form; If area satisfy The image is in the form of a traffic light; If area satisfy The image is in the form of a defect; Otherwise, record it as another exception.

[0024] The steps for a digit recognition network to determine numerical results are as follows: collect individual digit images to form a dataset, train a digit recognition network such as BP, SVM, or CNN, and save its weights. The image to be tested is converted to grayscale and then subjected to grayscale projection. The processed image is then divided into n parts. When calling We obtain the numbers from n images, and then recombine them to obtain the numerical result.

[0025] There are three methods for developing anomaly detection networks: training the network separately, training the network in a fusion manner, and using image processing methods.

[0026] The method employs a separate network training approach, with the following steps: Normal and abnormal images of pointers, buttons, indicator lights, handles, and defects are collected and trained to form pointer networks, indicator light networks, and defect networks, respectively. Data augmentation is then performed, such as adding noise, removing noise, changing brightness, changing contrast, rotating, cropping, mixup, binarization, color space changes, image LBP, and Gabor changes, resulting in a new dataset. The weights of each network are calculated, and different network weights are applied based on the image format to determine whether it is abnormal. Specifically, the identification method involves: acquiring normal and abnormal samples, building a network model, and saving the network model weights when the accuracy reaches over 99%; real-time detection of video images, obtaining image data, and then using the saved network model weights to determine whether the image data is abnormal.

[0027] The fusion training network method includes the following steps: forming a comprehensive dataset by combining normal and abnormal images of pointers, buttons, indicator lights, handles, and defects, and performing data augmentation, corresponding to normal and abnormal labels, training the network and saving the weights, and calling the weights on the target location image to determine whether it is abnormal.

[0028] Image processing methods, including the following steps: The image preprocessing and thresholding of pointer, handle and button images (image preprocessing includes grayscale and binarization) are performed to highlight button height, fit the slope of pointer and handle, and make judgments based on whether the slope of the real-time pointer and handle image or the height of the button is abnormal. Perform color analysis on the traffic light images and determine whether there are any abnormalities based on the real-time traffic light colors; The defect image is preprocessed, the defect area is calculated, and whether it is a defect is determined based on the defect area of ​​the real-time defect image.

[0029] The specific steps for training an anomaly detection network are as follows: Let Q be the part remaining after removing the target from the original background. Then, apply 11 enhancement methods to the obtained abnormal data image, including rotation, brightness increase, brightness decrease, filtering, noise addition, cropping, blurring, masking, binarization, LBP transformation, and Gabor transformation. Input the 11 processed result images into n anomaly judgment models. If all 11 outputs of each model are abnormal, then the model is considered abnormal. If more than n / 2 models consider it abnormal, then the data image is considered abnormal data, and these 11 results are used as new samples to expand the image dataset.

[0030] The method for obtaining abnormal image data in the augmented image dataset is as follows: The original data is inspected in real time. If anomalies are found, the abnormal images are accumulated. When a certain amount of data is accumulated, it is placed in a discriminative network to determine that the accumulated data are all abnormal data, which is denoted as B. Assuming the original sample database is A, and the initial metric is the accuracy 'a' obtained through training, B is uploaded to expand the original database to form a new dataset A+B. The datasets A+B, A, and B are trained respectively, and the resulting accuracy values ​​are denoted as a1, a2, and a3. If both a1 and a3 are greater than a2, the original sample database is updated to A+B, and the metric is updated to the maximum value of a1 and a3; otherwise, no update is performed.

[0031] To expand the image dataset, data augmentation methods are used, and semi-supervised model training is employed. The data augmentation methods include the following steps: Get the width of the original image pic1 and height Define the original target in the original image as The four variables represent its horizontal coordinate, vertical coordinate, width, and height, respectively. Randomly scale the original image and the original target multiple times to perform data augmentation processing. The shrinking operation includes the following steps: Select a random number less than 1. As a scaling factor, the original target is reduced to a size with a width and height of [missing values]. , The new image pic2, the x-axis range of pic2 is The range of the vertical axis is Generate a space with a width and height of respectively. Image pic3, at location Place pic2 there. Then in pic3, the position of the target image pic2 is updated to... Repeat the above steps 5 times to obtain different enhanced images, and then put them into the database. The zoom-in operation is as follows: Take a value greater than 1 and less than 1 random numbers As a magnification ratio, let the original target be magnified to have a width and height of [missing information]. , The new image pic4, with its initial position randomly generated within the original image pic1. Place pic4; repeat the above steps multiple times to obtain different enhanced images, and then put them into the database.

[0032] The training method of the semi-supervised method includes the following steps: Two channels are constructed for training: the first is the original target, and the second is the image generated by the downsizing operation. After training, the original target is used to obtain feature map fe1, where the x-coordinate range of fe1 is... The range of the vertical axis is ; The feature map fe2 is obtained by training on the scaled-down image of the original target. The Roi-Align method is adopted to extract the features of the corresponding region on the feature map fe1 based on the coordinate position of the original target, and the corresponding target feature is obtained as aim1; The target's position in the original image is updated to the position in pic2. Using the Roi-Align method, features of the corresponding region are extracted from the feature map fe2 based on the coordinate position of the target in the scaled-down image, and the corresponding target feature is obtained as aim2. Apply the imresize operation to fix the size of aim1 and aim2, add a similarity loss of aim1 and aim2 to the loss function of the entire network, define the cross-correlation value or flatten the image into row vectors, and calculate the sum of absolute errors between row vectors; Complete the training by following the steps outlined above.

[0033] Example 2 Based on Embodiment 1, this invention also discloses a monitoring method capable of plotting an anomaly trend curve based on the anomaly, specifically including the following steps: For abnormal numbers, 0 represents a high value, 1 represents a low value, and 2 represents a normal value. A continuous curve is created to obtain a graph in the form of numbers. For images that are defective, the historical data is plotted as a curve, with 0 for defective and 1 for non-defective. For other anomalies, six anomaly forms are defined, with 0 representing only one indicator light failure, 1 representing multiple indicator light failures, 2 representing abnormal handle position, 3 representing both 0 and 2, 4 representing both 1 and 2, and 5 representing a logical anomaly. The above data are plotted as a curve.

[0034] Example 3 The present invention also discloses an encryption algorithm that can take a secure encryption scheme for abnormal images when anomalies are detected in the above embodiments 1 and 2, embed watermarks into abnormal images, select watermark images based on the pixel values ​​of anomalies and Canny edge detection, construct a deep learning network based on the embedded watermark images, realize the discrimination of embedded watermark frames, and use a watermark extraction algorithm to extract watermarks.

[0035] A secure encryption scheme includes the following steps: Encryption process: (1) Collect abnormal images as training samples; (2) Number the five types of abnormalities: pointer, button, indicator light, handle, and defect, and label them as 1, 2, 3, 4, and 5; (3) Collect 100 watermark images, then solve the Canny edge detection image for these 100 watermark images, solve the pixel sum for the Canny edge detection image, and divide the watermark images into 5 groups according to the pixel sum, and denoted as A, B, C, D, E in descending order; (4) For the order of A, B, C, D, E and the abnormal labels 1, 2, 3, 4, 5, a mapping relationship is established. For security, a non-sequential mapping is adopted, that is, it is not 1 and A, 2 and B and so on. For example, the mapping method of 2 and A, 3 and B, 4 and C, 5 and D, 1 and E can be adopted, or other mappings can be used as the key for transmission between the two. (5) When the video is detected in real time, if an anomaly is found in a certain frame, the corresponding image group to be embedded with the watermark is selected according to the sequence number of the anomaly. Two images are selected from the corresponding group watermark and the LSB algorithm is used to embed the video watermark. LSB stands for Least Significant Bit. The center position of the two images is a random position near the center coordinate of the abnormal image, which can facilitate decryption.

[0036] The decryption process employs deep learning algorithms and LSB to extract the video watermark and obtain information. (1) Select several images containing abnormal scenes to embed watermarks, select several images containing abnormal scenes to not embed watermarks, and select several images without watermarks and without abnormal scenes. The training set consists of three parts. (2) Convolutional neural networks are built on the training set of the first part. Deep learning algorithms such as LeNet, AlexNet, and VGG16 can be used to train the above three parts. The network performance is obtained through the LOSS curve and accuracy curve, and the weight file is saved. (3) When the video is decrypted, the video is converted into frames, each frame is verified, and the weights of the above steps are called. If the frame is determined by the network to be an image with an embedded watermark and containing an abnormal scene, the LSB algorithm will be used to extract the watermark. (4) After the watermark is extracted, solve the Canny edge detection image, solve the pixel sum of the image after the Canny edge detection, and determine which category (A, B, C, D, E) the image belongs to according to the size of the pixel sum. (5) Obtain the anomaly label based on the mapping relationship; (6) Embed watermarks into abnormal images according to the mapping relationship to expand the original training set, perform target detection on these abnormalities, and obtain network weights; (7) If the anomaly number obtained when the current detection frame is input to the network weight in step 26 is consistent with that in step 25, then the decoding work is completed; if they are inconsistent or no watermark is extracted, then the video is determined to be unencrypted.

[0037] Example 4 This invention also discloses a specific method for determining anomalies in pointer image formats, comprising the following steps: (1) To build a network for object detection, collect pointer images and label the pointer positions to obtain a dataset, and train it using Faster R-CNN; (2) When a real-time image is input, the network in step 1 is called to obtain the coordinates of the pointer, and the pointer image is saved separately as an image; (3) Construct datasets with separate pointer graphs and pointer degrees, construct AlexNet networks on the datasets, and implement the mapping between separate pointer graphs and pointer degrees; (4) For the graph obtained in step 2, call the network in step 3 to obtain the pointer degree P1; (5) For the pointer graph obtained in 2, the hough transformation operation is performed to obtain the largest line segment. The slope corresponding to the largest line segment is calculated. Based on the mapping relationship between slope and degree, the pointer degree p2 is obtained. (6) Define the accuracy of the pointer degree obtained by deep learning as acc1, and the accuracy obtained by hough as acc2; (7) The final pointer degree is p1 (acc1 / (acc1+acc2))+p2 (acc2 / (acc1+acc2)).

[0038] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for anomaly monitoring of an instrument cabinet, characterized in that, Includes the following steps: (1) Collect data from the instrument cabinet and obtain an expanded image dataset; (2) The target image data and location signals in the expanded image dataset are uniformly labeled to form a label file, which constitutes the target detection network. The mAP value and weight file of the network are saved. When the mAP value reaches the usability of the network, the network architecture and model are no longer adjusted, and the network weights are saved. ; (3) Read real-time video and call network weights Select the target area and determine the anomaly type, coordinates of the anomaly image, and confidence level information based on the data results; A secure encryption scheme is adopted for abnormal images, and a watermark is embedded in the abnormal images. The watermark image is selected based on the pixel values ​​of the abnormality and Canny edge detection. Then, a deep learning network is constructed based on the embedded watermark image to realize the discrimination of the embedded watermark frame, and a watermark extraction algorithm is used to extract the watermark. The security encryption scheme includes the following steps: Encryption process: (11) Collect abnormal images as training samples; (12) Number the five types of abnormalities: pointer, button, indicator light, handle, and defect, and label them as 1, 2, 3, 4, and 5; (13) Collect a large number of watermarked images, solve the image after Canny edge detection, solve the pixel sum for the image after Canny edge detection, divide the watermarked images into 5 groups according to the pixel sum, and denot them as A, B, C, D, E in descending order; (14) Establish a mapping relationship between the order of A, B, C, D, E and the abnormal labels 1, 2, 3, 4, 5. The mapping relationship is a non-sequential mapping. (15) When the video is detected in real time, if an anomaly is found in a certain frame, the corresponding image group to be embedded with the watermark is selected according to the sequence number of the anomaly. Two images are selected from the corresponding group watermark and the LSB algorithm is used to embed the video watermark. The center position of the two images is a random position near the center coordinate of the abnormal image. The decryption process employs deep learning algorithms and LSB to extract the video watermark and obtain information. (21) Select several images containing abnormal scenes to embed watermarks, select several images containing abnormal scenes to not embed watermarks, and select several images without watermarks and without abnormal scenes. The training set consists of three parts. (22) Convolutional neural networks are built on the training set of the first part, and the VGG16 deep learning algorithm is used to train the above three parts. The network performance is obtained through the LOSS curve and accuracy curve, and the weight file is saved. (23) When the video is decrypted, the video is converted into frames, each frame is verified, and the weights of the above steps are called. If the frame is determined by the network to be an image with an embedded watermark and containing an abnormal scene, the LSB algorithm will be used to extract the watermark. (24) After the watermark is extracted, solve the Canny edge detection image, solve the pixel sum of the image after the Canny edge detection, and determine which category (A, B, C, D, E) the image belongs to according to the size of the pixel sum; (25) Obtain the anomaly label based on the mapping relationship; (26) Embed watermarks into abnormal images according to the mapping relationship to expand the original training set, perform target detection on these abnormalities, and obtain network weights; (27) If the anomaly number obtained when the current detection frame is input to the network weight in step 26 is consistent with that in step 25, then the decoding work is completed; if they are inconsistent or no watermark is extracted, then the video is determined to be unencrypted.

2. The method for abnormal monitoring of an instrument cabinet according to claim 1, characterized in that, The method for determining the anomaly type is as follows: Let the target position i be denoted as ,in, The x-coordinate of the target location The vertical coordinate of the target position. For width, For height, Category labels for the target The probability that the system considers it to be the target; when the aspect ratio... Greater than the threshold At that time, the digit recognition network is invoked to obtain numerical results, and anomalies are determined based on the numerical values. The steps for the digit recognition network to determine the numerical results are as follows: collecting individual digit images to form a dataset, training a digit recognition network and saving its weights. The image to be tested is converted to grayscale and then subjected to grayscale projection. The processed image is then divided into n images, with each image numbered sequentially. When calling We obtain the numbers from n images, and then recombine them to obtain the numerical result. Aspect Ratio Not greater than the threshold At that time, the image form is distinguished according to the area, and the image data is input into the anomaly judgment network to determine whether it is an anomaly. The image form includes pointer, button, indicator light, handle and defect.

3. The method for anomaly monitoring of an instrument cabinet according to claim 1, characterized in that, The algorithm for the anomaly detection network can be trained using a standalone network method, a fusion training network method, or an image processing method. The separately trained network method includes the following steps: collecting normal and abnormal images of pointers, buttons, indicator lights, handles, and defects respectively to train and form pointer networks, signal light networks, and defect networks, and performing data augmentation; calculating the weights of each network; calling different network weights according to the image form to determine whether it is abnormal; the specific identification method is: acquiring normal and abnormal samples, building a network model; when the accuracy of the network model reaches more than 99%, saving the weights of the network model. Real-time detection of video images; after obtaining image data, call the weights of the saved network model to determine whether the image data is abnormal. The fusion training network method includes the following steps: forming a comprehensive dataset from normal and abnormal images of pointers, buttons, indicator lights, handles, and defects, and performing data augmentation, corresponding to normal and abnormal labels, training the network and saving the weights, and calling the weights on the target location image to determine whether it is abnormal; The image processing method includes the following steps: The system performs preprocessing and thresholding on pointer, handle, and button images, emphasizes button height, fits the slope of pointer and handle, and makes judgments based on whether the slope of the real-time pointer and handle images or the height of the buttons are abnormal. Perform color analysis on the traffic light images and determine whether there are any abnormalities based on the real-time traffic light colors; The defect image is preprocessed, the defect area is calculated, and whether it is a defect is determined based on the defect area of ​​the real-time defect image; The image preprocessing includes grayscale conversion and binarization.

4. The method for abnormal monitoring of an instrument cabinet according to claim 1, characterized in that, The specific steps for training the anomaly detection network are as follows: Let Q be the part remaining after removing the target from the original background. The resulting abnormal data image is processed by 11 methods: rotation, brightness increase, brightness decrease, filtering, noise addition, cropping, blurring, masking, binarization, LBP transformation, and Gabor transformation. The 11 processed result images are input into n anomaly judgment models. If all 11 outputs of each model are abnormal, the model is considered abnormal. If more than n / 2 models consider it abnormal, then the data image is considered abnormal data. The results of these 11 abnormal data are combined with multiple original background images to expand the image dataset as new samples.

5. The method for anomaly monitoring of an instrument cabinet according to claim 1, characterized in that, The method for obtaining anomalies in the augmented image dataset is as follows: The original data is inspected in real time. If anomalies are found, the abnormal images are accumulated. When a certain amount of data is accumulated, it is placed in a discriminative network to determine that the accumulated data are all abnormal data, which is denoted as B. Assuming the original sample database is A, and the initial metric is the accuracy 'a' obtained through training, B is uploaded to expand the original database to form a new dataset A+B. The datasets A+B, A, and B are trained respectively, and the resulting accuracy values ​​are denoted as a1, a2, and a3. If both a1 and a3 are greater than a2, the original sample database is updated to A+B, and the metric is updated to the maximum value of a1 and a3; otherwise, no update is performed.

6. The method for anomaly monitoring of an instrument cabinet according to claim 1, characterized in that, The expanded image dataset is augmented using data augmentation methods, and the model is trained using a semi-supervised approach. The data augmentation methods include the following steps: (1) Obtain the width of the original image pic1 and height Define the original target in the original image as The four variables represent its horizontal coordinate, vertical coordinate, width, and height, respectively. (2) Randomly perform scaling operations on the original image and the original target and repeat the process multiple times to achieve data augmentation processing; The shrinking operation includes the following steps: Select a random number less than 1. As a scaling factor, the original target is reduced to a size with a width and height of [missing values]. , The new image pic2, the x-axis range of pic2 is The range of the vertical axis is Generate a space with a width and height of respectively. Image pic3, at location Place pic2 there. Then in pic3, the position of the target image pic2 is updated to... Repeat the above steps multiple times to obtain different enhanced images, and then store them in the database. The zoom-in operation is as follows: Take a value greater than 1 and less than 1 random numbers As a magnification ratio, let the original target be magnified to have a width and height of [missing information]. , The new image pic4, with its initial position randomly generated within the original image pic1. Place pic4; repeat the above steps multiple times to obtain different enhanced images, and then put them into the database; The training method of the semi-supervised method includes the following steps: (1) Construct two channels for training: the first is the original target, and the second is the image generated by the downsizing operation; (2) After training, the original target is used to obtain feature map fe1. The range of the horizontal coordinates of the position of fe1 is... The range of the vertical axis is ; (3) Train the image of the original target after it has been reduced in size to obtain the feature map fe2; (4) Using the Roi-Align method, the features of the corresponding region are extracted on the feature map fe1 based on the coordinate position of the original target, and the corresponding target feature is obtained as aim1; (5) The position of the target in the original image is updated to the position of pic2. Using the Roi-Align method, features of the corresponding region are extracted from the feature map fe2 based on the coordinate position of the target in the scaled-down image, and the corresponding target feature is obtained as aim2. (6) Fix the size of aim1 and aim2 by imresize operation, add the similarity loss of aim1 and aim2 to the loss function of the whole network, define the cross-correlation value or flatten the image into row vectors, and calculate the absolute error between the row vectors. (7) Complete the training according to the above steps.

7. The method for anomaly monitoring of an instrument cabinet according to claim 1, characterized in that, Based on the anomaly, an anomaly trend curve is plotted, including the following steps: For abnormal numbers, 0 represents a high value, 1 represents a low value, and 2 represents a normal value. A continuous curve is created to obtain a graph in the form of numbers. For images that are defective, the historical data is plotted as a curve, with 0 for defective and 1 for non-defective. For other anomalies, six anomaly forms are defined, with 0 representing only one indicator light failure, 1 representing multiple indicator light failures, 2 representing abnormal handle position, 3 representing both 0 and 2, 4 representing both 1 and 2, and 5 representing a logical anomaly. The above data are plotted as a curve.

8. The method for abnormal monitoring of the instrument cabinet according to claim 2, characterized in that, The steps to determine if a pointer image is abnormal include: (1) To build a network for object detection, collect pointer images and label the pointer positions to obtain a dataset, and train it using Faster R-CNN; (2) When a real-time image is input, the network in step 1 is called to obtain the coordinates of the pointer, and the pointer image is saved separately as an image; (3) Construct datasets with separate pointer graphs and pointer degrees, construct AlexNet networks on the datasets, and implement the mapping between separate pointer graphs and pointer degrees; (4) For the graph obtained in step 2, call the network in step 3 to obtain the pointer degree P1; (5) For the pointer graph obtained in 2, the hough transformation operation is performed to obtain the largest line segment. The slope corresponding to the largest line segment is calculated. Based on the mapping relationship between slope and degree, the pointer degree p2 is obtained. (6) Define the accuracy of the pointer degree obtained by deep learning as acc1, and the accuracy obtained by hough as acc2; (7) The final pointer degree is p1 (acc1 / (acc1+acc2))+p2 (acc2 / (acc1+acc2)).

Images