Digital side platform image recognition method and system based on YOLOv5 model

Abstract

The invention discloses a digital side platform image recognition method and system based on a YOLOv5 model. The method comprises the steps that an original image is collected through sensing equipment installed on a substation site; preprocessing the collected original image to obtain a preprocessed image; inputting the preprocessed image into a YOLOv5 model deployed on a side platform for prediction; screening and optimizing the predicted result to obtain a final recognition result; the invention provides an intelligent identification scheme capable of meeting the requirements of high speed, high precision and high adaptability at the same time, the response speed and identification accuracy of video monitoring and intelligent inspection can be improved, and the real-time performance and reliability requirements of few people or unattended operation of the transformer substation are met.

Description

TECHNICAL FIELD

[0001] The application belongs to the image recognition technical field of machine learning, and particularly relates to a digital edge platform image recognition method and system based on a YOLOv5 model. BACKGROUND

[0002] The substation can carry out video monitoring, meter / switch intelligent inspection, panoramic AR monitoring, operation safety control and other operations through image recognition technology, and carry out targeted intelligent operation and maintenance work. At present, the mainstream methods of image recognition mainly include traditional image processing method, two-stage algorithm based on candidate region and single-stage algorithm based on regression. Although the traditional image processing method has strong interpretability and low data requirement, it is seriously dependent on environmental light, weather and shooting angle. When the light changes, the characteristics may be completely invalid, and the false and missed reporting rates are high. Although the two-stage algorithm based on candidate region has high accuracy, its slow reasoning speed cannot meet the demand of real-time video analysis, and the computing power requirement of the edge computing device is too high. The single-stage algorithm based on regression achieves the best balance among speed, accuracy and model size, and is especially suitable for the real-time and lightweight deployment requirements of the substation. Therefore, there is an urgent need for an image recognition method suitable for the digital edge platform (intelligent edge platform) of the substation, which is real-time, reliable and easy to deploy.

[0003] Since there are few studies on digital edge platform image recognition in the substation environment, there is a lack of related image analysis and data processing methods, so the image recognition technology in other scenes is analyzed. For example, the patent CN120147275A obtains multi-modal sensing data through an image and data acquisition module, combines a defect recognition and positioning module of an improved ResNet-50 network and a YOLOv5 target detection algorithm, and significantly improves the accuracy of defect recognition and the adaptability to complex defect characteristics. For example, the patent CN117523486A discloses an icing detection method based on multi-source information fusion, which intelligently identifies the icing area by fusing infrared and visible light images through a CNN model, estimates the icing thickness, and realizes intelligent monitoring of the icing of the power transmission line. In summary, an image recognition model suitable for the background of the substation edge platform needs to be proposed. SUMMARY

[0004] The purpose of the present application is to provide a digital edge platform image recognition method and system based on a YOLOv5 model, which can improve the detection efficiency.

[0005] Technical scheme: the digital edge platform image recognition method based on the YOLOv5 model comprises:

[0006] Step 1: collect the original image through the sensing device installed on the substation site; the original image comprises a visible light image and an infrared image;

[0007] Step 2: Preprocess the collected original image to obtain a preprocessed image;

[0008] Step 3: input the preprocessed image into a YOLOv5 model deployed on an edge platform (smart edge platform) for prediction;

[0009] Step 4: screen and optimize the predicted results to obtain the final recognition results.

[0010] Further, the step 1 includes: collecting images covering all scenes, all devices and all states of the substation, using a labeling tool to draw a bounding box for each target object in the image and label the class, and outputting a dataset containing image path, bounding box coordinates and class label.

[0011] Further, the step 2 includes: performing standardization processing on the original image to adapt to model input and improve robustness; specifically, for infrared images, performing non-uniformity correction and temperature drift compensation; for visible light images, performing distortion correction, brightness normalization and weighted guided filter denoising processing, and performing Mosaic-9 splicing, HSV color space disturbance and random rotation and affine transformation on the visible light images to generate augmented samples.

[0012] Further, the step 3 includes: inputting the preprocessed image into the YOLOv5 network for forward propagation to obtain the original prediction tensor; specifically, dividing the input preprocessed image into grid, the number of rows and columns of the grid, and each grid is responsible for predicting multiple bounding boxes;

[0013] The predicted results include: locating and identifying the devices in the image, judging the device state, reading the meter value or segmenting the defect area.

[0014] Further, the step 4 includes:

[0015] First, the offset predicted by the model is decoded into the actual image coordinates bounding box by boundary box decoding; the decoding formula is as follows:

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] wherein, is a Sigmoid function to compress any input value between (0, 1); and is the coordinate of the current grid point relative to the top-left corner of the feature map; and is the width and height of the anchor box; is the step size, which represents the ratio of the input image size to the current feature map size;

[0022] Secondly, confidence filtering is performed, the decoded bounding box and the corresponding comprehensive confidence score of the bounding box are input, a confidence threshold is set, and all prediction boxes with a score lower than the threshold are determined as unreliable detection and discarded directly. The comprehensive confidence calculation formula is as follows:

[0023] ;

[0024] wherein, is the target score, is the target probability, which represents the probability of the existence of a target in the box. The closer the target probability is to 1, the greater the possibility of the existence of an object in the box; is the class score, is the maximum class probability, which represents the probability that the model considers the target to belong to the most likely class;

[0025] Finally, through non-maximum suppression NMS, the prediction box set after confidence filtering is input, and the following operations are performed for each class:

[0026] Step 4.1: The filtered prediction boxes are sorted in descending order of confidence;

[0027] Step 4.2: The box with the highest confidence is selected and placed in the final result list;

[0028] Step 4.3: Calculate the intersection over union IoU of the box with the highest confidence and all remaining boxes;

[0029] Step 4.4: Delete all boxes with an IoU greater than NMS;

[0030] Step 4.5: Repeat steps 4.2 to 4.4 from the remaining boxes until all boxes are processed.

[0031] Further, it further includes step 5: converting the final recognition result into business usable information, and comparing it with the preset rules. If it is judged as an exception, a structured alarm information is generated.

[0032] Further, step 6: according to the result of step 5, different data storage and uploading strategies are adopted to optimize bandwidth and storage resources; in particular:

[0033] If it is judged to be normal, the recognition result and the thumbnail are only stored locally on the edge platform for a certain period of time, which is used for historical query and trend analysis, and does not occupy the uplink bandwidth.

[0034] If it is judged to be abnormal, only the alarm information and the related evidence pictures or videos are selectively uploaded to the cloud center platform; the operation and maintenance personnel confirm or mark the alarm result as a false alarm through the interface, and the artificial processing result is taken as the true value label and returned to the corresponding edge node.

[0035] Based on the same inventive concept, the application also provides a digital edge platform image recognition system based on a YOLOv5 model, comprising:

[0036] A collection module is configured to collect original images through a sensing device installed on a substation site; the original images include visible light images and infrared images;

[0037] A preprocessing module is configured to preprocess the collected original images to obtain preprocessed images;

[0038] A prediction module is configured to input the preprocessed images into a YOLOv5 model deployed on an edge platform for prediction;

[0039] An identification module is configured to filter and optimize the predicted results to obtain final recognition results.

[0040] Based on the same inventive concept, the application also provides a computing device, comprising one or more processors, one or more memories, and one or more programs, the programs being stored in the memories and configured to be executed by the processors, and the programs being loaded into the processors to implement the steps of the digital edge platform image recognition method based on the YOLOv5 model according to any one of the above.

[0041] Based on the same inventive concept, the application also provides a storage medium, the storage medium storing a computer program, the computer program comprising program instructions, the program instructions causing the processor to execute the steps of the digital edge platform image recognition method based on the YOLOv5 model according to any one of the above when executed by the processor.

[0042] Beneficial effects: Compared with the prior art, the present application realizes millisecond-level inference speed while ensuring high detection accuracy (high mAP) by adopting an advanced single-stage YOLOv5 algorithm, making it possible to analyze the video stream returned by the video monitoring system in real time and issue immediate alarms, greatly improving the timeliness and early warning capability of intelligent inspection of substations; at the same time, in view of the characteristics of limited computing power of the digital edge platform of the substation, the advantages of the lightweight version of the YOLOv5 model are fully utilized; the model is small in size and high in computing efficiency, and can be efficiently deployed on resource-constrained edge computing devices, effectively reducing the dependence on hardware costs; compared with the complex scheme of deploying multiple special models for different tasks (such as meter identification and switch state judgment), the present application can simultaneously and efficiently complete the identification and state analysis of multiple devices using a single YOLOv5 model; such unified model architecture greatly simplifies system design, reduces resource consumption, realizes an efficient operation and maintenance mode of "one scan, multiple diagnosis", and improves the efficiency and maintainability of the overall system. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A method flowchart of an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0045] As shown in the accompanying Figure 1 The digital edge platform image recognition method based on the YOLOv5 model of the present embodiment includes:

[0046] Step 1: Collecting original images through a sensing device installed on the substation site; the original images include visible light images and infrared images;

[0047] Step 2: Preprocessing the collected original images to obtain preprocessed images;

[0048] Step 3: Inputting the preprocessed images into the YOLOv5 model deployed on the edge platform for prediction;

[0049] Step 4: Screening and optimizing the predicted results to obtain the final recognition results.

[0050] Specifically, step 1 collects a large number of images covering various scenes (different light, weather, angle) of the substation, various devices (meters, switches, insulators, etc.) and various states (normal, abnormal). Use a labeling tool (such as LabelImg) to draw a bounding box for each target object in the image and label the class. Finally, output an image path, bounding box coordinates and category labels.

[0051] Step 2, standardize the input image to adapt to the model input and improve robustness. According to the image modalities, differential preprocessing operations are performed. For infrared images, non-uniformity correction and temperature drift compensation are performed. For visible light images, distortion correction, brightness normalization, and weighted guided filter denoising are performed. For visible light images, Mosaic-9 splicing, HSV color space disturbance, and random rotation and affine transformation are performed to generate augmented samples, which improves image quality from the source.

[0052] Step 3, the preprocessed image is input into the YOLOv5 network for forward propagation to obtain the original prediction tensor. The core idea is to divide the input image into S x S grids, and each grid is responsible for predicting multiple bounding boxes.

[0053] Step 4, decode the original output of the model into intuitive detection results, and finally filter out clean, accurate, and non-redundant detection results to provide high-quality input for subsequent business logic (such as alarm, record).

[0054] First, the bounding box decoding converts the model's predicted offset into actual image coordinates . The decoding formula is as follows:

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] In the formula, is the Sigmoid function, which compresses any input value to between (0, 1), as shown in equation (5). grid_x, grid_y are the coordinates of the current grid point relative to the top-left corner of the feature map. anchor_w, anchor_h are the width and height of the anchor box. stride is the step size, which refers to the ratio of the input image size to the current feature map size.

[0061] Secondly, confidence filtering, input the decoded large number of bounding boxes and their corresponding comprehensive confidence score, set a confidence threshold (such as conf_threshold = 0.25). All the prediction boxes with a score lower than this threshold are determined as "unreliable detection" and discarded directly. The confidence calculation formula is as follows:

[0062] ;

[0063] In the formula, t_o is the target score, so σ(t_o) is the target probability, indicating the probability of "there is a target in the box". The closer this value is to 1, the greater the possibility of having an object in the box. c_i is the class score, and max(σ(c_i)) is the maximum class probability, indicating the probability that the model considers the target to belong to the most likely class.

[0064] Finally, through non-maximum suppression (NMS), input the remaining, relatively high-quality prediction box set after confidence filtering, and perform the following operations for each class separately:

[0065] S1: Sort these boxes by confidence from high to low.

[0066] S2: Select the highest confidence box and put it into the "final result" list.

[0067] S3: Calculate the IoU of this highest score box with all remaining boxes.

[0068] S4: Delete all boxes with an IoU exceeding the NMS threshold (such as nms_threshold = 0.45). Because these boxes will be considered as repeated detection of the same target as the current optimal box.

[0069] S5: Repeat steps S2 to S4 from the remaining boxes until all boxes are processed.

[0070] The IoU mentioned in the third and fourth steps is the intersection over union, the ratio of the intersection area to the union area, to reflect whether two boxes are the same target. If the IoU of two boxes is high, it means that their positions and sizes are very close, and it is very likely that they are detecting the same object. Since there is already a box with the highest confidence, other high overlap boxes should be deleted.

[0071] The embodiment also includes step 5: converting the post-processed result into business usable information and determining whether an alarm is needed according to the rules. Compare the recognition result with the preset rules (such as: is the meter reading over the threshold? Is the state of the knife gap inconsistent with the expectation? Is the device temperature abnormal?), and once it is determined to be abnormal, generate structured alarm information immediately.

[0072] Step 6, under normal conditions, the recognition result and the thumbnail are only stored locally on the edge platform for a certain period of time for historical query and trend analysis, without occupying the uplink bandwidth. Under abnormal conditions, only the alarm information and the related evidence pictures / videos are selectively uploaded to the cloud center platform. This ensures that the cloud only receives the most critical information for global monitoring and decision-making. The operation and maintenance personnel can confirm the alarm result or mark it as a false alarm through the interface. The artificial feedback result will be returned to the corresponding edge node as a high-quality true label.

[0073] Based on the same inventive concept, the embodiment also provides a digital edge platform image recognition system based on a YOLOv5 model, comprising:

[0074] The acquisition module is configured to acquire original images through a sensing device installed on the substation site; the original images include visible light images and infrared images;

[0075] The preprocessing module is configured to preprocess the acquired original images to obtain preprocessed images;

[0076] The prediction module is configured to input the preprocessed images into the YOLOv5 model deployed on the edge platform for prediction;

[0077] The recognition module is configured to filter and optimize the predicted results to obtain final recognition results.

[0078] Based on the same inventive concept, the embodiment also provides a computing device, comprising one or more processors, one or more memories, and one or more programs stored in the memories and configured to be executed by the processors, and the programs loaded into the processors implement the steps of the digital edge platform image recognition method based on the YOLOv5 model according to any one of the above.

[0079] Based on the same inventive concept, the embodiment also provides a storage medium, which stores a computer program, and the computer program comprises program instructions, which, when executed by a processor, cause the processor to perform the steps of the digital edge platform image recognition method based on the YOLOv5 model according to any one of the above.

[0080] The present application overcomes the defects of the existing image recognition method in the complex environment of the substation, such as poor real-time performance, insufficient reliability, and difficulty in lightweight deployment, and provides an intelligent recognition scheme that can simultaneously meet the requirements of high speed, high precision, and high adaptability, to improve the response speed and recognition accuracy of video monitoring and intelligent inspection, meet the real-time and reliability requirements of the substation with few or no personnel on duty, and provide a lightweight and efficient model to reduce the dependence on the computing power of the edge device and ensure the efficient and stable operation of the method on the digital edge platform of the substation.

Claims

1. A digital edge platform image recognition method based on a YOLOv5 model, characterized in that, The method comprises the following steps: Step 1: collecting original images through sensing equipment installed on the site of the substation; the original images comprise visible light images and infrared images; Step 2: pre-processing the collected original images to obtain pre-processed images; Step 3: inputting the pre-processed images into a YOLOv5 model deployed on an edge platform for prediction; Step 4: screening and optimizing the predicted results to obtain final recognition results.

2. The digitalized edge platform image recognition method based on the YOLOv5 model according to claim 1, characterized in that, The step 1 comprises: collecting images covering all scenes, all equipment and all states of the substation, using a labeling tool to draw a bounding box for each target object in the images and label the class, and outputting a data set comprising an image path, bounding box coordinates and class labels.

3. The digitalized edge platform image recognition method based on the YOLOv5 model according to claim 1, characterized in that, The step 2 comprises: performing standardization processing on the original images to adapt to model input and improve robustness; specifically, for infrared images, performing non-uniformity correction and temperature drift compensation; for visible light images, performing distortion correction, brightness normalization and weighted guided filter denoising processing, and generating augmented samples by performing Mosaic-9 splicing, HSV color space disturbance and random rotation and affine transformation on the visible light images.

4. The digitalized edge platform image recognition method based on the YOLOv5 model according to claim 1, characterized in that, The step 3 comprises: inputting the preprocessed image into a YOLOv5 network to perform forward propagation to obtain an original prediction tensor; specifically, the preprocessed image is divided into a plurality of grids , The number of rows and columns of the grid, and each grid is responsible for predicting a plurality of bounding boxes. The predicted results comprise: locating and identifying the equipment in the images, judging the equipment state, reading the meter value or segmenting the defect area.

5. The digitalized edge platform image recognition method based on the YOLOv5 model according to claim 1, characterized in that, The step 4 comprises: First, the model-predicted offsets are decoded into actual image coordinates by bounding box decoding The decoding formula is as follows:​ ； ； ； ； ； where, is a Sigmoid function used to compress any input value between (0, 1); and is the coordinate of the current grid point relative to the top-left corner of the feature map; and are the width and height of the anchor box; is the step size used to represent the ratio of the input image size to the current feature map size; Secondly, confidence filtering is performed, the decoded bounding box and the comprehensive confidence score corresponding to the bounding box are input, a confidence threshold is set, all prediction boxes with a score lower than the threshold are determined as unreliable detection and are discarded directly; the comprehensive confidence calculation formula is as follows: ； In the formula, is a target score, is a target probability, indicating the probability of the existence of a target in the frame, and the closer the target probability is to 1, the greater the possibility of the existence of an object in the frame; is a category score, is a maximum category probability, used to indicate the probability that the model considers the target to belong to the most likely category; Finally, repeated detection is eliminated through non-maximum suppression NMS, the prediction box set after confidence filtering is input, and the following operations are performed for each class: Step 4.1: sorting the filtered prediction boxes in descending order of confidence; Step 4.2: selecting the box with the highest confidence and placing it in the final result list; Step 4.3: calculating the intersection over union IoU of the box with the highest confidence and all remaining boxes; Step 4.4: deleting all boxes with an IoU higher than NMS; Step 4.5: repeating steps 4.2 to 4.4 from the remaining boxes until all boxes are processed.

6. The digitalized edge platform image recognition method based on the YOLOv5 model according to claim 1, characterized in that, Step 5: converting the final recognition results into information usable for business and comparing them with preset rules, if it is judged as abnormal, generating structured alarm information.

7. The digitalized edge platform image recognition method based on the YOLOv5 model according to claim 6, characterized in that, Step 6: according to whether the result judged in step 5 is normal, different data storage and uploading strategies are adopted to optimize bandwidth and storage resources; specifically: If it is judged as normal, the recognition results and thumbnails are only stored locally on the edge platform for a certain period of time for historical query and trend analysis, without occupying uplink bandwidth; If it is judged as abnormal, only the alarm information and related evidence pictures or videos are selectively uploaded to the cloud center platform; the operation and maintenance personnel confirm or mark the alarm results as false alarms through the interface, and the artificial processing results are used as true labels and are returned to the corresponding edge node.

8. A digitalized side platform image recognition system based on a YOLOv5 model, characterized in that, The method comprises the following steps: A collection module is configured to collect original images through sensing equipment installed on the site of the substation; the original images comprise visible light images and infrared images; A preprocessing module is configured to preprocess the collected original image to obtain a preprocessed image. A prediction module is configured to input the preprocessed image into a YOLOv5 model deployed on an edge platform to perform prediction. An identification module is configured to filter and optimize the predicted result to obtain a final identification result.

9. A computing device, comprising: The program is loaded into the processor to implement the steps of the YOLOv5 model-based digital edge platform image identification method according to any one of claims 1 to 7. The storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by the processor, the processor performs the steps of the YOLOv5 model-based digital edge platform image identification method according to any one of claims 1 to 7.

10. A storage medium, characterized by ​

Images