Visual intelligent identification system of electric power inspection robot

Abstract

A visual intelligent identification system of an electric power inspection robot comprises a service module and a visual processing module, the service module is linked with an HTTP client, the visual processing module configures algorithm parameters and network parameters through a yaml module, after the service module receives a request sent by the HTTP client, a data preprocessing module in the service module analyzes the request sent by an HTTP protocol, and the request is sent to the HTTP client through the yaml module. And the visual processing module receives the request and classifies the request according to the task type, calls a corresponding processing function according to the request type to carry out image processing, returns a detection result to the service module through a POST request, and sends the detection result to the HTTP client for cross-platform monitoring through the service module. According to the method, interference of backlight, blurring, distortion and the like of an original image is eliminated through an image preprocessing algorithm based on machine vision, meanwhile, a calculation method of a Darknet layer and a loss function of a neural network is improved, and the detection accuracy of the network in a complex environment and on a small target is ensured.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology, and in particular to a visual intelligent recognition system for power inspection robots. Background Technology

[0002] With the rapid development and continuous expansion of the power industry, inspection tasks are becoming increasingly heavy. Traditional power inspections mainly rely on manual labor, requiring inspectors to observe with the naked eye or use handheld devices to read and record data. Power inspection robots, however, combine robotics and computer vision technologies, enabling autonomous navigation, obstacle avoidance, and the use of various sensors and detection equipment to monitor and diagnose power facilities in real time.

[0003] The visual recognition component is a crucial part of power inspection robots. Through cameras and image processing technology, it reads various instruments in power facilities, identifies equipment defects, and monitors personnel operations. However, this field still faces numerous technical challenges and limitations. For example, the variety of instruments is vast, and they are often subject to interference from oil, dust, and lighting conditions. Current technologies often only enable simple image acquisition and preliminary processing, making it difficult to accurately read and record instrument data. More complex defects in power equipment, such as cracks, corrosion, and deformation, are difficult to identify. Furthermore, the lack of real-time analysis and compliance assessment of personnel actions makes it difficult to promptly detect and correct safety hazards during inspections.

[0004] In summary, existing power inspection robots are insufficient in the field of visual algorithms to meet the requirements of high precision, high efficiency, and high safety in power inspection. Summary of the Invention

[0005] This invention provides a visual intelligent recognition system for power inspection robots, aiming to overcome the shortcomings of existing technologies, improve the efficiency and accuracy of power inspection, and reduce inspection costs and safety risks.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A visual intelligent recognition system for power inspection robots includes a business module and a vision processing module. The business module is linked to an HTTP client. The vision processing module configures algorithm parameters and network parameters through a YAML module. After receiving a request from the HTTP client, the data preprocessing module within the business module parses the request sent by the HTTP protocol and classifies the request according to the task type. The vision processing module calls the corresponding processing function to perform image processing according to the request type. The vision processing module returns the detection result to the business module via a POST request, and then sends it to a cross-platform listening HTTP client through the business module.

[0008] The visual processing module includes an image reading module and an image preprocessing module. The image reading module uses OpenCV to read images captured by the gimbal camera. The image preprocessing module uses an adaptive sharpening algorithm and a contrast adjustment algorithm based on histogram equalization to enhance the contrast of local features in the image. After the image reading module reads the image, the image preprocessing module performs contrast enhancement processing on the local features of the image.

[0009] The processing functions called by the vision processing module include a detection network based on the MSE loss function and a detection network based on the attention mechanism.

[0010] The detection network based on the MSE loss function incorporates the mean squared error (MSE) loss function as a component of the YOLOv8 loss function, combining it with other loss functions to form a composite loss function. The MSE loss function-based detection network preprocesses the instrument image, enhancing the pointer and dashboard scale features. It uses the U-net segmentation network to segment and mark the scale lines and pointer, then obtains the position of the scale ring, mapping each feature point in the ring to a rectangle to complete the pointer scale expansion. Based on the expanded pointer scale, it calculates the positional relationship between the pointer, the minimum scale line, and the maximum scale line, ultimately calculating the final instrument reading.

[0011] The aforementioned attention-based detection network first enhances the detection effect of defects, substation safety equipment, and vehicle license plates by adding an attention mechanism to the neural network. It also uses image processing and machine learning algorithms to accurately identify and classify the status of equipment, or monitors personnel operation behavior and personnel safety through network and machine learning technology, and reports abnormal behavior to the front end in real time to ensure the safety and standardization of the inspection process.

[0012] The beneficial effects of this invention are as follows: This invention eliminates interference from backlighting, blurring, and distortion in the original image through a machine vision-based image preprocessing algorithm. Simultaneously, it improves the calculation method of the Darknet layer and loss function in the neural network, ensuring the network's accuracy in complex environments and for detecting small targets. Furthermore, by adding an attention mechanism to the neural network, it enhances the detection of defects (flames, smoke, missing insulators, scratches and damage to floating objects and instruments, leaks, etc.), substation safety equipment (substation cabinets, fire extinguishers, fire fighting equipment, etc.), and vehicle license plates. Image processing and machine learning algorithms are used to accurately identify and classify the status of the equipment. Through neural networks and machine learning technology, it achieves monitoring of personnel operational behavior (wearing proper attire, smoking, making or receiving phone calls, etc.) and personnel safety (falling over, climbing over guardrails, trespassing), promptly reporting abnormal behavior to the front end to ensure safety and standardization during inspections. Attached Figure Description

[0013] Figure 1 This is a system framework diagram of the present invention;

[0014] Figure 2 This is a system flowchart of the present invention;

[0015] Figure 3 This is a comparison image of the pointer instrument before and after preprocessing according to the present invention;

[0016] Figure 4 This is an unfolded view of the pointer instrument ring of the present invention;

[0017] Figure 5 This is a comparison image of the original and preprocessed images of the digital instrument of the present invention under low light conditions;

[0018] Figure 6 This is a comparison image of the original and preprocessed images of the digital instrument of the present invention under high light conditions;

[0019] Figure 7 This is a comparison image of the original image and the preprocessed image when the digital instrument of this invention is out of focus;

[0020] Figure 8 This is a comparison image of the original image and the preprocessed image when the digital instrument of this invention is blurred;

[0021] Figure 9 This is a comparison image of the original and preprocessed images of the digital instrument display of the present invention when the color is light.

[0022] Figure 10 This is a comparison image of the original and preprocessed images of the digital instrument of the present invention when it is functioning normally;

[0023] The following will describe in detail, with reference to the accompanying drawings, embodiments of the invention. Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0025] A visual intelligent recognition system for power inspection robots includes a business module and a vision processing module. The business module connects to an HTTP client, while the vision processing module configures algorithm and network parameters via a YAML module. The YAML module supports various data types and data structures. Upon receiving a request from the HTTP client, the business module's data preprocessing module parses the request and categorizes it according to task type. The vision processing module then calls the corresponding processing function for image processing based on the request type. The vision processing module returns the detection results to the business module via a POST request, which in turn sends the data to a cross-platform listening HTTP client. Since the data transmitted to the robot's business terminal needs to be in JSON format, and the system needs to parse the JSON upon receiving it, convert the detection result structure back to JSON format after image processing, and then send it to the business terminal via a POST request, this system uses the jsoncpp library for serialization and deserialization.

[0026] The visual processing module includes an image reading module and an image preprocessing module. The image reading module uses OpenCV to read images captured by the gimbal camera. The image preprocessing module employs an adaptive sharpening algorithm and a histogram equalization-based contrast adjustment algorithm to enhance the contrast of local image features. After the image reading module reads the image, the image preprocessing module further enhances the contrast of local image features. Image sharpening can dynamically adjust the sharpening intensity according to the characteristics of different regions in the image. For example, sharpening is enhanced in edge regions while sharpening is reduced in flat regions to avoid noise amplification. This enhances edge details while preserving smooth areas, resulting in a smoother and more natural effect. Histogram equalization enhances the contrast of local image regions and prevents noise amplification by limiting the upper limit of contrast; simultaneously, an adaptive strategy is used to adjust the contrast, effectively avoiding the noise amplification problem caused by equalization.

[0027] The processing functions called by the vision processing module include a detection network based on the MSE loss function and a detection network based on the attention mechanism. The vision processing module handles three categories of data: instrument readings, defect detection, and personnel safety regulations. After task parsing, image processing is performed along three routes, and the results are finally returned uniformly. The system processing flow is as follows: Figure 2 As shown, the data preprocessing module parses the requests sent via the HTTP protocol and categorizes them into instrument readings, defect detection, and personnel safety regulations based on the task type. The overall architecture encapsulates the logic for handling each task type into corresponding classes and functions using Docker containers, making it independent. In this way, the main processing function only needs to call the appropriate processing function based on the request type.

[0028] The detection network based on the Mean Squared Error (MSE) loss function incorporates the MSE loss function as a component of the YOLOv8 loss function, combining it with other loss functions to form a composite loss function. This maintains the overall performance of YOLOv8 in object detection tasks while introducing the advantages of the MSE loss function, such as computational simplicity, ease of optimization, and intuitive error measurement. The MSE loss function only needs to calculate the average of the sum of squares of the differences between the predicted and true values, which helps reduce the computational complexity of the model. Furthermore, for the instrument detection task involved in this system, the network should focus on the deviation between the detection box and the true value; the MSE loss function directly reflects the overall error between the predicted and true values. The detection network based on the MSE loss function preprocesses the instrument image, overcoming the inaccuracy caused by lighting issues and strengthening the features of pointers and dashboard scales, making them easier to identify in complex environments. The U-net segmentation network is used to segment and mark the scale lines and pointer. Then, the position of the scale ring is obtained, and each feature point in the ring, such as the scale lines and pointer, is mapped into a rectangle to complete the pointer scale expansion. Based on the result of the pointer scale expansion, the positional relationship between the pointer, the minimum scale line, and the maximum scale line is calculated, and the final instrument reading is calculated.

[0029] The aforementioned attention-based detection network first enhances the detection effect of defects, substation safety equipment, and vehicle license plates by adding an attention mechanism to the neural network. It also uses image processing and machine learning algorithms to accurately identify and classify the status of equipment, or monitors personnel operation behavior and personnel safety through network and machine learning technology, and reports abnormal behavior to the front end in real time to ensure the safety and standardization of the inspection process.

[0030] The main function of attention mechanisms is to enable models to focus on key parts of the input, thereby improving the ability to recognize important features. They also have the advantages of being easy to integrate, easy to extend, and having lightweight modules to reduce computational overhead.

[0031] By introducing an attention mechanism, YOLOv8 can better capture key information in images, thereby improving the accuracy of object detection. Especially in the detection of equipment defects and personnel safety regulations, the attention mechanism helps the model distinguish between the target and the background, reducing false positives and false negatives; it also enables the detection network to better learn feature representations in images, thus improving the model's generalization ability.

[0032] Example 1: Instrument readings

[0033] The system uses the YOLOv8 network for target detection, replacing the C3 module in YOLOv5 with the C2f module at its core, while retaining the SPPF module. An anchorless detection head is also employed. The system introduces the Mean Square Error (MSE) function to effectively improve accuracy; the MSE function formula is as follows: The function curve of MSE is smooth, continuous, and differentiable everywhere. Moreover, the gradient decreases as the error decreases, which is beneficial for the rapid convergence to the minimum value during training.

[0034] The characteristic of squared error is that when the difference between the predicted and true values ​​is greater than 1, the error increases; when the difference is less than 1, the error decreases. This is determined by the property of squares. MSE (Mean Squared Error) penalizes larger errors (>1) more and smaller errors (<1) less. From a training perspective, the model will be more biased towards points with larger penalties, assigning them greater weights, thereby achieving faster convergence.

[0035] A. Instrument Image Preprocessing: To address the issue of uneven lighting and enhance the features of the instrument image, image preprocessing is performed. An image preprocessing algorithm is employed to resolve recognition inaccuracies caused by uneven lighting, while simultaneously enhancing the features of the pointer and dashboard scale, making them easier to correctly identify in complex environments. The preprocessing time is approximately 50-100ms on the CPU. Figure 3 The comparison images of the pointer instrument before and after image preprocessing are shown below. Figure 3 Group (a) is the original image. Figure 3 Group (b) shows the illumination balance and feature enhancement diagrams. Figure 3 (c) is a bilateral denoised image), and the time consumed by the pointer instrument image preprocessing is shown in Table 1;

[0036] Table 1 Average time consumption of instrument image preprocessing algorithm

[0037]

[0038] Pointer instrument image reading: First, extract the scale lines and pointer using semantic segmentation, then use a torus unfolding method. The torus unfolded image is shown below. Figure 4 As shown, unfold the dial into a straight line, calculate the positional relationship between the pointer and the scale lines, and calculate the reading.

[0039] Semantic segmentation can divide an image into multiple regions, each corresponding to a specific object or category in the image. In pointer instrument reading recognition, semantic segmentation technology is used to extract the region where the pointer and scale are located, and it has the advantage of being highly robust to interference factors such as lighting and occlusion.

[0040] The circular expansion method is an image processing-based pointer instrument reading recognition method. This method simplifies the positioning and recognition of pointers and scales by expanding the circular instrument image into a rectangular image.

[0041] B. Digital Instrument Image Preprocessing: The reading of digital instruments is divided into two steps: digital region extraction and OCR character recognition. The process of digital instrument image recognition is: YOLO digital region extraction → digital instrument image preprocessing → PaddleOCR character recognition.

[0042] Digital instrument image preprocessing utilizes a pixel enhancement algorithm based on corner and feature points to increase the contrast of the digits while reducing the contrast of the background. This improves the subsequent OCR digit recognition performance, removes interference from the environment and shooting conditions, and the preprocessing time is approximately 50ms on the CPU. It also addresses issues such as uneven lighting and focus issues during shooting, and enhances the edge color features of the digits. A comparison of the digital instrument image before and after preprocessing is shown below. Figures 5-10 As shown ( Figure 5 (a) is the original image. Figure 5 (b) is the preprocessed image); the time taken for digital instrument image preprocessing is shown in Table 2;

[0043] Table 2 Average time consumption of instrument image preprocessing algorithm

[0044]

[0045] Digital instrument image readings:

[0046] Within the existing OCR recognition framework, digital instrument recognition is divided into two sets of algorithms for LCD dials and digital tube dials. After target detection in the digital area, the image is preprocessed for digital instrument recognition, and then pixel segmentation in HSV space and OUST algorithm segmentation are performed. Finally, the segmented binarized image is subjected to OCR recognition, and the OCR recognition result is filtered and traversed to obtain the final output number. The time consumption for digital instrument image recognition is shown in Table 3.

[0047] Table 3 Average time consumption of instrument image algorithm

[0048]

[0049]

[0050] Example 2: Equipment Defect and Personnel Monitoring:

[0051] The problems faced in defect detection include: missed detections due to distance during personnel safety compliance inspections and suspended matter detection; insufficient equipment defect samples; to address these issues, a small target detection layer and a CSP module were added to the CSPDarknet layer of the original YOLOv8 model to enhance the connection between different feature layers, and an attention mechanism was introduced to enhance the model's feature extraction capabilities.

[0052] The invention has been described above with reference to the accompanying drawings. Obviously, the specific implementation of the invention is not limited to the above-described manner. Any improvements made using the inventive concept and technical solution, or direct application to other situations without modification, are all within the scope of protection of the invention.

Claims

1. A visual intelligent recognition system for power inspection robots, characterized in that, It includes a business module and a vision processing module. The business module connects to an HTTP client. The vision processing module configures algorithm and network parameters through a YAML module. After receiving a request from the HTTP client, the data preprocessing module within the business module parses the request sent by the HTTP protocol and classifies the request according to the task type. The vision processing module calls the corresponding processing function to perform image processing according to the request type. The vision processing module returns the detection results to the business module via a POST request, and then sends them to the HTTP client for cross-platform monitoring through the business module.

2. The power inspection robot visual intelligent recognition system according to claim 1, characterized in that, The visual processing module includes an image reading module and an image preprocessing module. The image reading module uses OpenCV to read images captured by the gimbal camera. The image preprocessing module uses an adaptive sharpening algorithm and a contrast adjustment algorithm based on histogram equalization to enhance the contrast of local features in the image. After the image reading module reads the image, the image preprocessing module performs contrast enhancement processing on the local features of the image.

3. The power inspection robot visual intelligent recognition system according to claim 2, characterized in that, The processing functions called by the vision processing module include a detection network based on the MSE loss function and a detection network based on the attention mechanism.

4. The power inspection robot visual intelligent recognition system according to claim 3, characterized in that, The detection network based on the MSE loss function incorporates the mean squared error (MSE) loss function as a component of the YOLOv8 loss function, combining it with other loss functions to form a composite loss function. The MSE loss function-based detection network preprocesses the instrument image, enhancing the pointer and dashboard scale features. It uses the U-net segmentation network to segment and mark the scale lines and pointer, then obtains the position of the scale ring, mapping each feature point in the ring to a rectangle to complete the pointer scale expansion. Based on the expanded pointer scale, it calculates the positional relationship between the pointer, the minimum scale line, and the maximum scale line, ultimately calculating the final instrument reading.

5. The power inspection robot visual intelligent recognition system according to claim 4, characterized in that, The aforementioned attention-based detection network first enhances the detection effect of defects, substation safety equipment, and vehicle license plates by adding an attention mechanism to the neural network. It also uses image processing and machine learning algorithms to accurately identify and classify the status of equipment, or monitors personnel operation behavior and personnel safety through network and machine learning technology, and reports abnormal behavior to the front end in a timely manner to ensure the safety and standardization of the inspection process.

Images