A yolov4-based device state light identification system and method

Abstract

The application discloses a device state lamp identification system and method based on Yolov4, and relates to the technical field of intelligent device identification.The method comprises the following steps: controlling a robot to collect a computer room device state lamp image, performing Mosaic data enhancement and labeling on the image, and constructing a data set; constructing a state lamp identification model, wherein the state lamp identification model comprises an input layer, a backbone network, a fusion network and a prediction output; improving a loss function by using CIOU Loss and DIOU_NMS to improve regression accuracy; training the state lamp identification model, introducing self-adversarial training and cross-small-batch standardization, and obtaining a weight file; and deploying the trained state lamp identification model to identify a device state lamp image.The application solves the problems of poor detection effect of current target detection algorithms on multiple targets and small targets by constructing a state lamp identification model, and has the advantages of high real-time identification efficiency, good small target identification effect and high identification accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of equipment intelligent identification, and particularly relates to a device state lamp identification system and method based on Yolov4. BACKGROUND

[0002] There are a large number of indicator lights in an industrial environment to determine whether each device is working normally. When the indicator light shows a certain state, it indicates that a fault has occurred and needs to be handled by the staff in a timely manner. Therefore, it is necessary to monitor the state of the indicator light in real time, and long-term monitoring in traditional manual monitoring is a heavy and tedious task. When there are many indicator lights, manual monitoring is almost impossible to achieve comprehensive and accurate monitoring, and abnormal situations are few. Therefore, manual monitoring will cause great waste of manpower and low efficiency. For traditional industrial indicator light state monitoring, not only is it labor-intensive, but also it is easy to cause false negatives due to the negligence of the monitoring personnel, so the monitoring efficiency is low.

[0003] With the development of deep learning in computer vision in recent years, many scholars are using neural networks to solve target detection and other related problems, and good results have been achieved. Currently, there are two main algorithms for detecting state lights: RCNN and YOLO3. R-CNN has redundant calculations because R-CNN generates candidate regions first and then performs convolution on the regions. The candidate regions overlap to some extent, causing repeated operations when extracting features with CNN. In addition, R-CNN stores the extracted features and then uses SVM for classification, which requires more storage space. Since CNN feature extraction is used, scaling operations need to be performed on the candidate boxes, but in reality, the selected boxes have various sizes, which can cause target distortion. The Yolo3 algorithm as a target detection algorithm has the problems of long training time, large hardware consumption, poor detection effect for multiple targets and small targets, etc. SUMMARY

[0004] The purpose of the present application is to provide a device state lamp identification system and method based on Yolov4 to solve the problems in the prior art.

[0005] To achieve the above purpose, the present application provides the following technical scheme: a device state lamp identification method based on Yolov4, the method comprising:

[0006] controlling a robot to collect equipment state lamp images in a machine room, and performing Mosaic data enhancement and labeling on the images to construct a data set;

[0007] The state lamp recognition model comprises an input layer, a backbone network, a fusion network and a prediction output; the input layer is used for inputting a device state lamp image; the backbone network comprises a CBM module and a CSP module, and is used for preliminary feature extraction; the fusion network comprises an SPP module and a PANet module, and is used for multi-level feature fusion; the prediction output generates a target detection result, and the target detection result comprises position and category information of the state lamp;

[0008] The loss function is improved by using CIOU Loss and DIOU_NMS, and the regression accuracy is improved.

[0009] Based on the data set, the state lamp recognition model is trained, self-adversarial training and cross-mini-batch standardization are introduced, the training process is optimized, and a weight file is obtained.

[0010] The trained state lamp recognition model is deployed to recognize device state lamp images.

[0011] According to the above scheme, the Mosaic data enhancement comprises:

[0012] Four images are randomly selected from the data set, and annotation information of each image is obtained; the annotation information comprises target frame coordinates and category labels;

[0013] Each selected image is respectively processed by flipping, scaling and color gamut changing;

[0014] The four processed images are spliced into one large image according to four directions to generate a composite image containing multiple targets and multiple backgrounds;

[0015] According to the position of the spliced large image, the annotation frame of each image is adjusted to ensure that the annotation frame position of each target in the spliced image is accurate, and an enhanced image is obtained.

[0016] According to the above scheme, the backbone network is based on the CSPDarknet53 structure and comprises a CBM module and a CSP module.

[0017] The backbone network receives the image input by the input layer, performs preliminary feature processing through the CBM module, and then sequentially passes through five CSP modules for stepwise down-sampling; a fixed-size convolution kernel is provided before each CSP module, and the step is set to a specific value to ensure that the size of the feature map gradually decreases while deeper semantic information is extracted; after the CBM module and the CSP module, preliminary feature extraction is completed, and a feature map with certain semantic information is output.

[0018] According to the scheme, the CBM module comprises a convolutional layer, a batch normalization layer and a Mish activation function; the convolutional layer extracts basic features in the image by performing a convolution operation on the input image through a convolution kernel; the batch normalization layer performs normalization processing on the features output by the convolutional layer; and the Mish activation function introduces a non-linear factor and enhances gradient flow, so that the state light recognition model learns the complex relationship between features.

[0019] The Mish activation function formula is as follows:

[0020] Mish(x)=x·tanh(ln(1+e x ));

[0021] wherein Mish(x) represents an output value after being processed by the Mish activation function; x represents an input of the activation function; tanh represents a hyperbolic tangent function; the Mish activation function can provide a small gradient when the input value is small, avoiding gradient explosion; and the Mish activation function can provide a large gradient when the input value is large, avoiding gradient disappearance, which helps the neural network to better learn complex data patterns and features.

[0022] The CSP module divides the feature mapping of the base layer into two parts: one part is continuously extracted through residual block stacking to extract high-level semantic features, and the other part is directly connected to the final output through a small amount of processing; the two parts of features are merged through a cross-stage hierarchical structure to reduce repeated calculation of gradient information.

[0023] A plurality of anchor boxes of different sizes are preset on each feature map for detecting targets of different sizes; the sizes of the anchor boxes are obtained through clustering analysis according to the size distribution of the targets in the data set, so as to ensure that the model can effectively detect small targets, medium targets and large targets; through the fusion and detection of multi-scale feature maps, the adaptability of the model to multi-scale targets is improved.

[0024] DropBlock is used to randomly mask continuous blocks of the feature map, so as to force the network to rely on other parts for prediction, thereby improving the generalization ability of the model, and DropBlock is a regularization method for alleviating overfitting.

[0025] According to the scheme, the fusion network comprises an SPP module and a PANet module.

[0026] The SPP module performs different scale maximum pooling operations on the input feature map respectively, processes the feature map using multiple specifications of pooling kernels, expands the perception range of the features, and obtains context feature information covering different scales; and the feature maps of different scales are spliced to generate a feature representation with rich context information.

[0027] The PANet module efficiently extracts and fuses multi-level feature maps through a bidirectional feature transmission mechanism from top to bottom and from bottom to top, and a tensor connection feature fusion mode.

[0028] According to the above scheme, the top-down includes that the PANet module receives multi-level feature maps from the backbone network and the SPP module, and the multi-level feature maps have different resolutions and semantic information; through an upsampling operation, strong semantic information of a high-level feature map is transmitted to a low-level feature map along a top-down path, so that the low-level feature map fuses high-level semantic information;

[0029] The bottom-up includes that the PANet module starts from a low-level feature map and transmits strong positioning information to a high-level feature map; through a convolution operation, positioning information of the low-level feature map is fused with semantic information of the high-level feature map to generate an intermediate feature map with strong semantic and positioning capabilities;

[0030] The bidirectional feature transmission mechanism includes that a bottom-up path transmits low-level positioning information to a high-level feature map, which is combined with semantic information transmitted by a top-down path to form a bidirectional flow of multi-level features;

[0031] The tensor connection includes that the PANet module uses tensor connection to deeply fuse feature maps generated by the top-down and bottom-up paths.

[0032] According to the above scheme, the data set is divided into a training set, a test set and a validation set;

[0033] The training set is used to train the state light recognition model; the validation set is used to evaluate the performance of the state light recognition model in combination with a loss and an accuracy; and the test set is used to independently evaluate the generalization ability of the state light recognition model after the training of the state light recognition model is completed;

[0034] In each training iteration, an adversarial sample is generated through self-adversarial training, and the detection ability of the state light recognition model is optimized using the adversarial sample; features are standardized through cross-mini batch standardization to ensure the stability of the training process; and an improved loss function of CIOU Loss and DIOU_NMS is combined to optimize the regression accuracy and prediction frame screening effect of the state light recognition model.

[0035] The CIOU is as follows:

[0036]

[0037] Wherein, CIoU represents a complete intersection-over-union ratio, which is an index for measuring the overlapping relationship between the prediction box and the real box in target detection; IOU represents an intersection-over-union ratio, which is used to represent the ratio of the intersection area of the prediction box and the real box to the union area; p represents the Euclidean distance; b represents the center point coordinate of the prediction box; gt represents the real value; b gt represents the center point coordinate of the real box; c represents the diagonal distance of the minimum closed region capable of containing the prediction box and the real box; a represents a weight parameter; v represents a parameter for measuring the length-width consistency ratio, which is used to evaluate the similarity of the length-width ratio of the prediction box and the real box;

[0038] The loss function LOSS based on CIoU is calculated CIOU , and the formula is as follows:

[0039]

[0040] Wherein, LOSS CIOU represents a loss function based on CIoU, which is used to optimize the regression relationship between the prediction box and the real box during the training of the target detection model, so that the speed and accuracy of the prediction box regression are higher;

[0041] When the loss value no longer significantly decreases after being evaluated by the verification set, it is determined that the state light recognition model converges, the training iteration is terminated, and the weight file is obtained.

[0042] According to the above scheme, the self-adversarial training includes a first stage and a second stage;

[0043] The first stage includes, during the training process of the state light recognition model, performing an adversarial attack on the input device state light image, and generating an adversarial sample by adjusting the image pixel value;

[0044] The second stage includes using the generated adversarial sample as input to train the state light recognition model in a normal manner, so that the state light recognition model can detect the target in the adversarial sample.

[0045] The cross-small-batch normalization includes dividing the training set into multiple small batches of data, each small batch of data containing part of the device state light image and corresponding label information; calculating the mean and variance of the features in each small batch of data, and collecting statistical data across multiple small batches of data in the training set; using the collected statistical data to standardize the features, ensuring the stability of the feature distribution and accelerating the convergence of the state light recognition model.

[0046] According to the above scheme, the weight file stores the connection weights and biases of each part of the state light recognition model, and determines the feature extraction and prediction ability of the model on the input data; by loading the weight file, the state light recognition model can efficiently recognize the device state light image and output the position and category information of the target.

[0047] A device state light recognition system based on Yolov4, comprising a data acquisition module, a state light recognition module, a data storage module and a user interaction module.

[0048] The data acquisition module comprises an image shooting module and a data preprocessing module; the image shooting module is used to control the movement of the robot and the adjustment of the lifting rod, position the device to be detected, and collect the computer room device state light image; the data preprocessing module is used to perform data enhancement and labeling on the collected image, and construct a data set;

[0049] The data storage module is used to store and manage the collected image data, the labeling file, the data set, the training log, the weight file and the warning record, provide data backup and recovery, and ensure data security;

[0050] The state light recognition module comprises a model construction module, a training optimization module and a model deployment module; the model construction module is used to construct a state light recognition model, the state light recognition model comprises an input layer, a backbone network, a fusion network and a prediction output; the training optimization module uses the training set, uses CIOU Loss and DIOU_NMS to improve the loss function, introduces self-adversarial training and cross-small-batch standardization, trains the state light recognition model, and generates a weight file; the model deployment module deploys the trained state recognition model, detects the input device state light image in real time, and outputs the position and category information of the state light;

[0051] The user interaction module comprises a user interface module and a warning module; the user interface module displays the recognition result and the warning information of the device state light, provides an operation interface for training and deploying the state light recognition model, generates a visual report, and displays the system running state and the recognition effect; the warning module receives the recognition result of the model deployment module, judges whether the color and state of the state light are normal, triggers the warning mechanism when detecting that the state light has an abnormal state, sends the warning information to the user interface, records the warning information, and generates a log file.

[0052] Compared with the prior art, the beneficial effects of the present application are:

[0053] 1. The present application is based on the CSPDarknet53 structure, combined with the CBM module and the CSP module, which reduces the repeated calculation of gradient information and improves the feature extraction efficiency;

[0054] 2. This invention introduces the SPP module and adopts an improved PANet module, which significantly improves the detection accuracy;

[0055] 3. This invention uses CIOU Loss and DIOU_NMS to improve the loss function, thereby enhancing regression accuracy and prediction box selection performance;

[0056] 4. This invention achieves efficient and accurate identification of device status lights through an automated and intelligent process, thereby improving the efficiency of status light identification.

[0057] 5. This invention combines an early warning module to detect abnormal status lights in real time and trigger an early warning mechanism, effectively preventing equipment failures and ensuring the normal operation of equipment in the computer room. Attached Figure Description

[0058] Fig. 1 This is a flowchart illustrating the steps of a device status light recognition method based on Yolov4 according to the present invention.

[0059] Fig. 2 This is a schematic diagram of the structure of a device status light recognition system based on Yolov4 according to the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Example: Figs. 1-2 As shown, the present invention provides a technical solution, a method for identifying device status lights based on Yolov4, the method comprising:

[0062] S1. Control the robot to collect images of the status lights of the equipment in the computer room, and perform Mosaic data augmentation and annotation on the images to build a dataset;

[0063] Specifically, robots equipped with high-resolution industrial cameras are used to collect images of the status lights on the equipment in the server room. For example, the industrial camera has a resolution of 1920×1080, a frame rate of 30fps, and good light adaptability, enabling it to clearly capture images of the equipment status lights under different lighting conditions. The robot moves to the target cabinet using a path planning algorithm, adjusts the lifting rod to the position of the equipment to be inspected, and collects images of the equipment status lights. For example, approximately 13,000 images were collected, covering different equipment and different states (such as green light, red light, yellow light, and off light).

[0064] Further, the collected images are subjected to Mosaic data enhancement; four images are randomly selected from the dataset, for example: images A, B, C and D; and the annotation information of each image is obtained; the annotation information includes target box coordinates and class labels; each selected image is subjected to flipping, scaling and color gamut change processing, for example: the scaling ratio ranges from 0.8 to 1.2, the brightness adjustment range is -20% to +20%, and the contrast adjustment range is -15% to +15%; the four processed images are spliced into one large image according to the top-left, top-right, bottom-left and bottom-right directions, for example: image A is placed in the top-left corner, image B is placed in the top-right corner, image C is placed in the bottom-left corner, and image D is placed in the bottom-right corner; a composite image containing multiple targets and multiple backgrounds is generated; according to the position of the spliced large image, the annotation box of each image is adjusted to ensure the accuracy of the annotation box position of each target in the spliced image, and an enhanced image is obtained; for example, the annotation box coordinates of image A are adjusted from (x1, y1, x2, y2) to (x1', y1', x2', y2'), to ensure the accuracy of the annotation box position of each target in the spliced image; and the state lamp in the image is labeled using an image labeling software.

[0065] Further, a folder VOC2007 containing a dataset package is provided in the root directory of the YOLOV4 folder, and the VOC2007 contains three sub-folders, namely Annotations, ImageSets and Images, wherein the Images folder stores all the dataset photos of the state lamp used for training, the Annotiations folder stores the.xml files corresponding to each labeled photo, and the sub-folders in the ImageSets folder store the training set and test set image information.

[0066] S2, a state lamp recognition model is constructed, the state lamp recognition model comprising an input layer, a backbone network, a fusion network and a prediction output; the input layer is used to input a device state lamp image, the backbone network comprises a CBM module and a CSP module, and is used for preliminary feature extraction; the fusion network comprises an SPP module and a PANet module, and is used for multi-level feature fusion; the prediction output generates a target detection result, and the target detection result comprises position and category information of the state lamp;

[0067] Specifically, the backbone network based on the CSPDarknet53 structure includes a CBM module and a CSP module; the backbone network receives an image input by the input layer, performs preliminary feature processing through the CBM module, and then sequentially performs step-by-step down-sampling through five CSP modules; a fixed 3x3 convolution kernel is provided before each CSP module, and the stride is set to a specific value to ensure that the size of the feature map gradually decreases while deeper semantic information is extracted; for example: if the input image is 608x608, after passing through the five CSP modules, the change rule of the feature map is: 608-304-152-76-38-19; finally, a 19x19 feature map is output; after passing through the CBM module and the CSP module, preliminary feature extraction is completed, and a feature map with certain semantic information is output.

[0068] Further, the CBM module includes a convolution layer, a batch normalization layer and a Mish activation function; the convolution layer extracts basic features in the input image through convolution operation of the convolution kernel; the batch normalization layer performs normalization processing on the features output by the convolution layer; for example: convolution kernel size: 3x3, stride: 1, padding: 1; the Mish activation function introduces a non-linear factor to enhance the gradient flow and enable the state light recognition model to learn the complex relationship between features;

[0069] The formula of the Mish activation function is as follows:

[0070] Mish(x)=x·tanh(ln(1+e x ));

[0071] wherein, Mish(x) represents the output value processed by the Mish activation function; x represents the input of the activation function; tanh represents the hyperbolic tangent function; the Mish activation function can provide a small gradient when the input value is small to avoid gradient explosion, and can provide a large gradient when the input value is large to avoid gradient disappearance, which helps the neural network to better learn complex data patterns and features;

[0072] Further, the CSP module divides the feature map of the base layer into two parts: one part is continuously extracted by the residual block stack to extract high-level semantic features, and the other part is directly connected with the final output through a small amount of processing; through the cross-stage hierarchy, the two parts of the feature are combined to reduce the repeated calculation of gradient information; on each feature map, a plurality of anchor boxes of different sizes are preset for detecting targets of different sizes; the size of the anchor box is obtained by clustering analysis according to the size distribution of the targets in the data set, so as to ensure that the model can effectively detect small targets, medium targets and large targets; through the fusion and detection of multi-scale feature maps, the adaptability of the model to multi-scale targets is improved; DropBlock is used to randomly shield continuous area blocks in the feature map, so as to force the network to rely on other parts for prediction, thereby improving the generalization ability of the model;

[0073] Specifically, the fusion network includes an SPP module and a PANet module; the SPP module performs different scale maximum pooling operations on the input feature map, for example: the size of the pooling kernel is 1×1, 5×5, 9×9 and 13×13; the feature map is processed by using a plurality of specifications of the pooling kernel, the perception range of the feature is expanded, and context feature information covering different scales is obtained, for example: the input feature map of 13×13 is pooled using a 5×5 size pooling kernel, padding=2, and the pooled feature map is still 13×13 in size; the feature maps of different scales are spliced to generate a feature representation with rich context information; the PANet module efficiently extracts and fuses multi-level feature maps through a top-down and bottom-up bidirectional feature transmission mechanism and a tensor connection feature fusion method;

[0074] Further, the top-down includes that the PANet module receives multi-level feature maps from the backbone network and the SPP module, and the multi-level feature maps have different resolutions and semantic information; through an upsampling operation, strong semantic information of a high-level feature map is transmitted to a low-level feature map along a top-down path, so that the low-level feature map fuses the high-level semantic information; the bottom-up includes that the PANet module starts from a low-level feature map and transmits strong positioning information to a high-level feature map; through a convolution operation, the positioning information of the low-level feature map is fused with the semantic information of the high-level feature map to generate an intermediate feature map with strong semantic and strong positioning capabilities; the bidirectional feature transmission mechanism includes that the low-level positioning information is transmitted to the high-level feature map through the bottom-up path, and is combined with the semantic information transmitted through the top-down path to form a bidirectional flow of multi-level features; the tensor connection includes that the PANet module uses tensor connection to deeply fuse the feature maps generated by the top-down and bottom-up paths.

[0075] S3, using CIOU Loss and DIOU_NMS to improve the loss function and improve the regression accuracy;

[0076] Specifically, the improved loss function combined with CIOU Loss and DIOU_NMS optimizes the regression accuracy and prediction frame screening effect of the state light recognition model; the positioning loss uses CIOU, the NMS of the prediction frame screening becomes DIOU_NMS, IOU is used as the regression optimization loss, and CIOU considers the distance, overlap rate and scale between the target and the anchor, so that the target regression becomes more stable

[0077] The CIOU is as follows:

[0078]

[0079] Wherein, CIOU represents the complete intersection over union, which is an index for measuring the overlap relationship between the prediction frame and the real frame in target detection; IOU represents the intersection over union, which is used to represent the ratio of the intersection area of the prediction frame and the real frame to the union area; ρ represents the Euclidean distance; b represents the center point coordinates of the prediction frame; gt represents the real value; b gt represents the center point coordinates of the real frame; c represents the diagonal distance of the smallest closed region that can contain the prediction frame and the real frame; α represents the weight parameter; v represents the parameter for measuring the length-width consistency ratio, which is used to evaluate the similarity of the length-width ratio of the prediction frame and the real frame.

[0080] Further, the loss function LOSS CIOU based on CIOU is calculated as follows:

[0081]

[0082] Wherein, LOSS CIOU represents the loss function based on CIOU, which is used to optimize the regression relationship between the prediction frame and the real frame during the training of the target detection model, so that the speed and accuracy of the prediction frame regression are higher.

[0083] S4, based on the data set, training the state light recognition model, introducing self-antagonistic training and cross-small batch standardization, optimizing the training process, obtaining the weight file;

[0084] Specifically, the data set is divided into a training set, a test set and a validation set; the training set is used to train the state light recognition model; the validation set is used to evaluate the performance of the state light recognition model in combination with the loss and the accuracy; the test set is used to independently evaluate the generalization ability of the state light recognition model after the training of the state light recognition model is completed.

[0085] Further, in each training iteration, an adversarial sample is generated by self-adversarial training, and the detection capability of the state light recognition model is optimized using the adversarial sample; the features are standardized by cross-mini batch standardization to ensure the stability of the training process; the cross-mini batch standardization includes dividing the training set into multiple mini batch data, for example: each mini batch contains 32 images; each mini batch data contains part of the device state light image and the corresponding label information; the mean and variance of the features are calculated in each mini batch data, and the statistical data is collected across multiple mini batch data in the training set; the features are standardized using the collected statistical data to ensure the stability of the feature distribution and accelerate the convergence of the state light recognition model;

[0086] Further, the self-adversarial training includes a first stage and a second stage; the first stage includes, during the training process of the state light recognition model, performing adversarial attack on the input device state light image, generating an adversarial sample by adjusting the pixel value of the image; for example: adjusting the pixel value of image A from (100, 150, 200) to (90, 135, 180); the second stage includes using the generated adversarial sample as input to train the state light recognition model in a normal manner, so that the state light recognition model can detect the target in the adversarial sample;

[0087] Further, when the loss value no longer significantly decreases after evaluation using the verification set, it is determined that the state light recognition model converges, the training iteration is terminated, and a weight file is obtained;

[0088] Further, the weight file stores the connection weights and biases of each part of the state light recognition model, which determines the feature extraction and prediction capability of the model on input data; by loading the weight file, the state light recognition model can efficiently recognize the device state light image and output the position and category information of the target.

[0089] S5, deploy the trained state light recognition model to recognize device state light images;

[0090] Specifically, the input device state light image, for example, has a size of 416x416, after model inference, the final rectangular prediction frame is obtained by normalization and non-maximum suppression, which frames the device state light, and the prediction result is the position of the framed device light in the prediction image. The color of the frame is the color of the state light; for example, the position of the detected state light is (200, 300, 250, 350), the category is "green light", and the frame color is green. This is only an example and is not limited.

[0091] The application provides another technical solution, a device state lamp identification system based on Yolov4, which comprises a data acquisition module, a state lamp identification module, a data storage module and a user interaction module.

[0092] The data acquisition module comprises an image shooting module and a data preprocessing module; the image shooting module is used for controlling robot movement and lifting rod adjustment, positioning the device to be detected, and collecting the state lamp image of the equipment room device; and the data preprocessing module is used for data enhancement and labeling of the collected image, and constructing a data set.

[0093] The data storage module is used for storing and managing the collected image data, the labeling file, the data set, the training log, the weight file and the early warning record, providing data backup and recovery, and ensuring data security.

[0094] The state lamp identification module comprises a model construction module, a training optimization module and a model deployment module; the model construction module is used for constructing a state lamp identification model, wherein the state lamp identification model comprises an input layer, a backbone network, a fusion network and a prediction output; the training optimization module uses a training set, uses CIOU Loss and DIOU_NMS to improve the loss function, introduces self-adversarial training and cross-small-batch standardization, trains the state lamp identification model, and generates a weight file; and the model deployment module deploys the trained state identification model, performs real-time detection on the input device state lamp image, and outputs the position and category information of the state lamp.

[0095] The user interaction module comprises a user interface module and an early warning module; the user interface module displays the identification result and early warning information of the device state lamp, provides an operation interface for training and deploying the state lamp identification model, generates a visual report, and shows the system running state and identification effect; and the early warning module receives the identification result of the model deployment module, judges whether the color and state of the state lamp are normal, triggers an early warning mechanism when detecting that the state lamp has an abnormal state, sends early warning information to the user interface, records the early warning information, and generates a log file.

[0096] The application provides another technical solution, in the equipment room, a device state lamp identification system based on Yolov4 is deployed, which is used for real-time monitoring of the device state lamp in the equipment room and timely discovery of device abnormalities.

[0097] The trained model is deployed on a high-performance server in the equipment room and is connected with the data acquisition module in real time, and when a new device state lamp image is input, the state lamp identification model can quickly respond and complete image analysis within 0.8 seconds, and accurately output the position and category information of the state lamp.

[0098] The early warning module monitors the recognition result output by the model deployment module at all times, and judges whether the color and state of the state light are normal according to a preset rule, for example: when the state light is green, it is a normal state; when the state light is orange, it is a slight abnormal state; when the state light is red, it is a moderate abnormal state; when the state light is not on, it is an abnormal state of the state light or the device; this is only an example and is not limiting, when the state light of a server is recognized as red by the state light recognition model, the early warning module immediately captures the state light abnormal information and triggers the early warning mechanism;

[0099] After triggering the early warning, the early warning module quickly displays the abnormal information of the device in the form of a prominent red pop-up window on the user interface module, and the abnormal information includes the device location, state light abnormal condition and the like;

[0100] The early warning module records the early warning information in detail, including the early warning time, abnormal device information, recognition result and the like, and generates a complete log file, which is stored in the data storage module, facilitating subsequent query and analysis.

[0101] It is apparent for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims involved.

Claims

1. A Yolov4-based device status light recognition method, characterized in that: The method comprises: controlling the robot to collect the equipment state lamp image of the machine room, and performing Mosaic data enhancement and labeling on the image to construct a data set; constructing a state lamp recognition model, the state lamp recognition model comprising an input layer, a backbone network, a fusion network and a prediction output; the input layer is used to input the equipment state lamp image, the backbone network comprises a CBM module and a CSP module, and is used for preliminary feature extraction; the fusion network comprises an SPP module and a PANet module, and is used for multi-level feature fusion; the prediction output generates a target detection result, and the target detection result comprises position and category information of the state lamp; the backbone network is based on a CSPDarknet53 structure and comprises the CBM module and the CSP module; the backbone network receives the image input by the input layer, performs preliminary feature processing through the CBM module, and then sequentially passes through five CSP modules for step-by-step down-sampling; a fixed-size convolution kernel is arranged in front of each CSP module, and a stride is set to a specific value to ensure that the size of the feature map gradually decreases and deeper semantic information is extracted; the CBM module and the CSP module are used to complete preliminary feature extraction, and a feature map with certain semantic information is output; the CBM module comprises a convolution layer, a batch normalization layer and a Mish activation function; the convolution layer extracts basic features in the input image through a convolution kernel; the batch normalization layer performs normalization processing on the features output by the convolution layer; and the Mish activation function introduces a nonlinear factor to enhance the gradient flow and enable the state lamp recognition model to learn the complex relationship between features; the CSP module divides the feature mapping of the base layer into two parts: one part is continuously extracted through residual block stacking to extract high-level semantic features, and the other part is directly connected to the final output through a small amount of processing; the two parts of features are combined through a cross-stage hierarchical structure to reduce repeated calculation of gradient information; CIOU Loss and DIOU_NMS are used to improve the loss function and improve the regression accuracy; based on the data set, the state lamp recognition model is trained, self-adversarial training and cross-small-batch standardization are introduced to optimize the training process, and a weight file is obtained; the data set is divided into a training set, a test set and a validation set; the state lamp recognition model is trained using the training set; the performance of the state lamp recognition model is evaluated by combining the loss and the accuracy rate using the validation set; and the test set is used to independently evaluate the generalization ability of the state lamp recognition model after the training of the state lamp recognition model is completed; in each training iteration, an adversarial sample is generated through self-adversarial training, and the detection ability of the state lamp recognition model is optimized using the adversarial sample; the features are standardized through cross-small-batch standardization to ensure the stability of the training process; and the improved loss function of CIOU Loss and DIOU_NMS is used to optimize the regression accuracy and prediction box screening effect of the state lamp recognition model. When the loss value no longer significantly decreases after being evaluated by the verification set, it is determined that the state light recognition model converges, the training iteration is terminated, and a weight file is obtained; The self-adversarial training includes a first stage and a second stage; The first stage includes, during the training of the state light recognition model, performing an adversarial attack on an input device state light image, generating an adversarial sample by adjusting image pixel values; The second stage includes, using the generated adversarial sample as input, training the state light recognition model in a normal manner, so that the state light recognition model can detect the target in the adversarial sample; The cross-mini-batch standardization includes dividing the training set into multiple mini-batch data, each containing part of the device state light image and corresponding label information; calculating the mean and variance of the features in each mini-batch data, and collecting statistical data across multiple mini-batch data in the training set; using the collected statistical data to standardize the features, ensuring the stability of the feature distribution and accelerating the convergence of the state light recognition model; Deploy the trained state light recognition model to recognize device state light images.

2. The Yolov4-based device status light recognition method of claim 1, wherein: The Mosaic data enhancement includes: Randomly selecting four images from the data set and obtaining the label information of each image; the label information includes target box coordinates and class labels; Each selected image is respectively processed by flipping, scaling, and color gamut changing; The processed four images are spliced into one large image according to four directions to generate a composite image containing multiple targets and backgrounds; According to the position of the spliced large image, adjust the label box of each image to ensure the accuracy of the label box position of each target in the spliced image, and obtain the enhanced image.

3. The device state light recognition method based on Yolov4 according to claim 1, characterized in that: The fusion network includes an SPP module and a PANet module; The SPP module performs different scale maximum pooling operations on the input feature map respectively, uses multiple specifications of pooling kernels to process the feature map respectively, expands the perception range of the feature, and obtains context feature information covering different scales; Different scale feature maps are spliced to generate a feature representation with rich context information; The PANet module efficiently extracts and fuses multi-level feature maps through a bidirectional feature transmission mechanism from top to bottom and from bottom to top, and a feature fusion method of tensor connection.

4. The device state light recognition method based on Yolov4 according to claim 3, characterized in that: The top-down includes that the PANet module receives multi-level feature maps from the backbone network and the SPP module, the multi-level feature maps have different resolutions and semantic information; through upsampling operation, the strong semantic information of high-level feature maps is transmitted to low-level feature maps step by step along the top-down path, so that the low-level feature maps fuse high-level semantic information. The bottom-up includes that the PANet module starts from a low-layer feature map and transmits strong positioning information to a high layer step by step; the positioning information of the low-layer feature map is fused with semantic information of a high-layer feature map through convolution operation to generate an intermediate feature map with strong semantic and strong positioning ability; The bidirectional feature transmission mechanism includes that a bottom-up path transmits low-layer positioning information to a high-layer feature map, and the positioning information is combined with semantic information transmitted by a top-down path to form bidirectional flow of multi-level features; The tensor connection includes that the PANet module adopts tensor connection to deeply fuse feature maps generated by the top-down and bottom-up paths.

5. The device state lamp recognition method based on Yolov4 according to claim 1, characterized in that: The weight file stores the connection weights and biases of each part of the state lamp recognition model, determines the feature extraction and prediction ability of the model on the input data, and through the loading of the weight file, the state lamp recognition model can efficiently recognize the device state lamp image and output the position and category information of the target.

6. A Yolov4-based device status light identification system applied to the Yolov4-based device status light identification method of any one of claims 1-5, characterized in that: The system comprises a data acquisition module, a state lamp recognition module, a data storage module and a user interaction module. The data acquisition module comprises an image shooting module and a data preprocessing module; the image shooting module is used to control the movement of the robot and the adjustment of the lifting rod, position the device to be detected, and acquire the computer room device state lamp image; the data preprocessing module is used to perform data enhancement and labeling on the acquired image, and construct a data set; The data storage module is used to store and manage the acquired image data, the labeling file, the data set, the training log, the weight file and the early warning record, provide data backup and recovery, and ensure data security; The state lamp recognition module comprises a model construction module, a training optimization module and a model deployment module; the model construction module is used to construct a state lamp recognition model, the state lamp recognition model comprises an input layer, a backbone network, a fusion network and a prediction output; the training optimization module uses the training set, uses CIOU Loss and DIOU_NMS to improve the loss function, introduces self-adversarial training and cross-small-batch standardization, trains the state lamp recognition model, and generates a weight file; the model deployment module deploys the trained state recognition model, performs real-time detection on the input device state lamp image, and outputs the position and category information of the state lamp; The user interaction module comprises a user interface module and an early warning module; the user interface module displays the recognition result and early warning information of the device state lamp, provides an operation interface for training and deploying the state lamp recognition model, generates a visual report, and displays the system running state and recognition effect; the early warning module receives the recognition result of the model deployment module, judges whether the color and state of the state lamp are normal, triggers the early warning mechanism when detecting that the state lamp has an abnormal state, sends the early warning information to the user interface, records the early warning information, and generates a log file.

Images