Photovoltaic cleaning robot vision guidance posture correction and safety control method

Abstract

This invention belongs to the field of non-electric variable regulation or control systems, and discloses a visually guided posture correction and safety control method for a photovoltaic cleaning robot, which includes the following steps: S1: visual detection model construction and feature extraction; S2: visually guided dual closed-loop PID posture correction method; S3: joint identification of edges and gaps by ultrasonic and visual sensors. Step S3 includes using ultrasonic sensors at the front and rear of the robot to detect the distance information of the front and rear areas in real time during the robot's operation along the photovoltaic panel surface, and combining the visual sensor to collect and identify the image of the front area; when an abnormal distance is detected, the front ultrasonic sensor triggers the suspected edge area, and then the current abnormal area is reviewed by vision, and the detection results of the ultrasonic sensor are combined for joint judgment to distinguish whether the abnormal area is the edge of the photovoltaic panel or the gap between the panels, so as to prevent the robot from falling from the edge area of ​​the photovoltaic panel.

Description

Technical Field

[0001] This invention belongs to the field of non-electric variable regulation or control systems, and specifically relates to a method for visual guidance posture correction and safety control of a photovoltaic cleaning robot. Background Technology

[0002] With the large-scale construction and application of photovoltaic power generation systems, the impact of dust, bird droppings, and other adhering pollutants on the surface of photovoltaic modules on power generation efficiency and system stability is becoming increasingly prominent. To reduce manual cleaning costs and improve operation and maintenance efficiency, photovoltaic cleaning robots are gradually becoming important equipment for the intelligent operation and maintenance of photovoltaic power plants. Among them, photovoltaic cleaning robots, due to their low center of gravity, strong adhesion ability, and good adaptability to complex terrain, are well-suited for complex operating environments such as rooftop distributed photovoltaic arrays.

[0003] In actual operation on photovoltaic panel surfaces, photovoltaic cleaning robots still face many technical challenges. Firstly, photovoltaic modules are typically laid at a certain angle, and due to factors such as installation errors, support deformation, foundation settlement, and long-term environmental loads, the panel surface often experiences continuous, minute attitude changes in both the longitudinal and lateral directions. When the robot travels long distances on the tilted photovoltaic panel surface, it is prone to heading deviations due to changes in friction conditions between the tracks and the panel surface, localized slippage, and inconsistent drive mechanisms. This causes the cleaning path to deviate from the preset trajectory, affecting the integrity of cleaning coverage and the continuity of operations.

[0004] Traditional mobile robot attitude control relies heavily on IMU sensors or wheel speed-odometers for heading estimation. However, IMU sensors are susceptible to zero drift due to temperature drift, mechanical vibration, and integration errors during long-term operation. Wheel speed-odometers are also prone to cumulative errors due to track slippage and changes in adhesion conditions. This makes it difficult for robots to maintain stable attitude control accuracy over long periods in scenarios involving long-distance straight-line movement through photovoltaic arrays. Meanwhile, when robots approach the edges of photovoltaic panels, they typically rely on ultrasonic distance sensors for boundary detection. However, factors such as gaps between panels, edge shadows, and surface reflections can easily cause distance anomalies, leading to edge misjudgments or false alarms related to gaps, thus affecting safe operation.

[0005] Therefore, there is an urgent need to propose a visual guidance posture correction method suitable for complex operating scenarios of photovoltaic cleaning robots. This method would identify the structural features of the photovoltaic panel surface through vision and assist the IMU sensor in correcting the robot's posture online, while ensuring the robot's autonomous operation safety and operational reliability. Summary of the Invention

[0006] The purpose of this invention is to provide a visual guidance posture correction and safety control method for photovoltaic cleaning robots, which can make the visual detection model more accurate, improve the posture control precision, and ensure the safe operation of the robot.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: a method for visual guidance posture correction and safety control of a photovoltaic cleaning robot, comprising the following steps: S1: Visual Detection Model Construction and Feature Extraction During the operation of the photovoltaic cleaning robot, the image of the photovoltaic panel surface is collected in real time using a vision sensor. The image is then input into an improved RT-DETR vision detection model for detection. The RT-DETR vision detection model is used to identify the edges of the photovoltaic panel, the gaps between panels, and the surface grid lines. This obtains one or more feature information from the category, position, and orientation of the currently collected photovoltaic panel surface image. This information is then used as the visual input for subsequent attitude correction and safety control. S2: Visually Guided Dual-Loop PID Attitude Correction Method Using the main direction of the grid lines on the photovoltaic module surface as a visual reference, the yaw output of the IMU is corrected online. Then, a cascaded dual closed-loop control is formed by the attitude outer loop and the speed inner loop to achieve stable attitude control of the photovoltaic cleaning robot on the tilted photovoltaic panel surface. The attitude outer loop, as the main control loop, is responsible for converting the deviation of visual feedback into the target speed command; the speed inner loop, as the execution loop, is responsible for correcting the motor speed deviation in real time. The two loops work together. S3: Combined ultrasound and vision recognition for edges and gaps During the operation of the photovoltaic cleaning robot along the surface of the photovoltaic panel, ultrasonic sensors installed at the front and rear of the robot are used to detect the distance information between the front and rear areas in real time. Combined with a vision sensor, the robot performs image acquisition and recognition of the area in front. When an abnormal distance is detected, the front ultrasonic sensor first triggers the suspected edge area, then the vision sensor verifies and identifies the current abnormal area, and finally the detection results of the rear ultrasonic sensor are combined to make a joint judgment to distinguish whether the current abnormal area is the edge of the photovoltaic panel or the gap between the panels. This helps to suppress misjudgment of the gap between the panels and prevent the photovoltaic cleaning robot from falling from the edge area of ​​the photovoltaic panel.

[0008] In another implementation method, the specific processing procedure in step S1 is as follows: S11: Image Preprocessing Two cameras, positioned above and below the photovoltaic cleaning robot, simultaneously capture images of the photovoltaic panel surface. The images include the edges and gaps of the photovoltaic panel in the area directly in front of the robot, captured by the front camera, and the grid line area of ​​the photovoltaic module in the direction of the robot's movement, captured by the bottom camera. The captured images are transmitted to the main controller, which sequentially performs median filtering for noise reduction, adaptive histogram equalization for enhancement, scale unification, and tensor quantization on the raw images to obtain standardized images suitable for input to the improved RT-DETR visual inspection model. S12: Backbone Network Feature Extraction The standardized image obtained in step S11 is input into the backbone network of the improved RT-DETR visual detection model to extract multi-scale feature information from the photovoltaic panel surface image. S13: Feature Enhancement and Fusion The multi-scale feature information output in step S12 is input into the integrated enhanced Head module to perform global context modeling and cross-scale fusion of features at different levels to obtain a fused feature map, thereby improving the model's ability to recognize multi-scale and multi-morphological targets. S14: Detection Prediction The fused feature map obtained in step S13 is input into the detection head to classify and predict the target related to the edge area of ​​the photovoltaic panel, the gap area between the panels and the grid line, and output the bounding box parameters, target confidence and target category information of the corresponding target. Assume the detection head is at time The nth detected target in the output is: , Where, x n (k) represents the x-coordinate of the center point of the target's bounding box, y n (k) represents the ordinate of the center point of the target bounding box. n (k) represents the width of the target bounding box, h n (k) represents the length of the bounding box, s n (k) represents the target confidence level, c n (k) represents the target category information.

[0009] S15: Post-processing of test results The detection results output in step S14 are decoded and filtered. The normalized bounding boxes are mapped back to the original image coordinate system, and valid detection results are filtered according to a preset confidence threshold to obtain the final target box coordinates, target category, and target confidence information, which serve as the visual input for subsequent robot posture correction and safety control. The post-processing includes inverse transformation of predicted box coordinates, category resolution, and confidence filtering. S16: Target Feature Information Output and Feature Code Generation Based on the valid detection results obtained in step S15, the category features, location features, and orientation-related features of edge targets, gap targets, and grid line targets are extracted respectively, and the corresponding target feature information or feature codes are generated.

[0010] In another implementation, the specific processing procedure in step S11 is as follows: The original image is first subjected to multi-kernel (3×3 and 5×5 kernels; the 5×5 kernel is used first to filter out large-particle noise, and then the 3×3 kernel is used for finer processing. By fusing filtering operators of different scales, the random noise from the sensor is suppressed while preserving the edge features of the photovoltaic panel to the maximum extent) median filtering to suppress noise interference. Then, adaptive histogram equalization is performed to enhance the local contrast of edges, seams, and grid line areas. Finally, the image is scaled to a uniform input scale and normalized. , in, For a moment The original images obtained, The image data is normalized; the normalized image is then converted to... The format is then added to obtain the model input tensor. The batch dimension is the first dimension of the tensor shape, representing the number of samples processed in parallel at one time.

[0011] In another implementation, the specific processing procedure in step S12 is as follows: The backbone network first extracts basic features through convolutional layers, normalization layers, and activation layers; then, it obtains feature maps at different scales through hierarchical convolutional downsampling; the convolutional neural network in the backbone of the visual model is the Ghost_HGBlock module, and a lightweight backbone network is constructed by combining the DWConv deep convolutional structure during the downsampling stage to improve the model's real-time performance while ensuring feature representation capability; the backbone network finally outputs shallow detail feature maps, mid-level semantic feature maps, and high-level abstract feature maps.

[0012] In another implementation, the specific processing procedure in step S13 is as follows: After the high-level semantic features are output from the backbone network, a SimAM parameterless attention module is introduced for enhancement. Then, the global feature dependencies are modeled through the AIFI lightweight Transformer encoder. Multi-scale feature fusion is achieved through upsampling, skip connections, concatenation, and RepC3 structure integrated in the Head module. SimAM attention enhancement is further introduced on the fused shallow detail feature map, mid-level semantic feature map, and high-level abstract feature map to obtain a fused feature map that combines detailed information and global semantic information.

[0013] In another implementation, the specific processing procedure in step S14 is as follows: To improve the accuracy of bounding box regression, SIoU loss is adopted in the improved RT-DETR visual detection model, and a bounding box regression loss function is constructed: ,in, For location regression loss term, The bounding box shape and orientation constraint loss term is used to simultaneously constrain the overlap, center distance, and shape difference between the predicted box and the ground truth box, so as to improve the detection box localization accuracy and convergence speed.

[0014] In another implementation, the specific processing procedure in step S16 is as follows: For the edge region and inter-panel gap region of photovoltaic panels, the target category, bounding box coordinates, length, gap width, and relative position parameters are extracted. The model output category label is then mapped to the predefined target type to generate the corresponding category code. For the photovoltaic module grid line area, the corresponding detection area or pixel distribution information is output, providing input for the calculation of the principal direction angle of the white grid line using the second-order moment principal axis direction detection method in subsequent step S22; the category is encoded according to the preset encoding rules. The target feature code is generated by combining parameters such as target length, gap width, and relative coordinates, and serves as the input basis for robot path planning, posture correction, safety judgment, and operation and maintenance early warning.

[0015] In another implementation, the specific processing procedure in step S2 is as follows: S21: Acquisition of inertial attitude state The robot's current yaw angle, attitude angle, and angular velocity information are collected by an IMU sensor as real-time inertial attitude state quantities during the robot's movement. The IMU sensor is used for attitude detection during the robot's straight-line movement and angle control during the robot's turning process, so that the robot can complete the turning action at a set angle. S22: Visual Yaw Angle Construction During the robot's movement, a camera mounted on the bottom of the photovoltaic cleaning robot vertically captures images of the grid structure on the surface of the photovoltaic panel. The white grid area in the image is extracted, and the principal axis direction of the current white grid line is calculated using a second-order moment principal axis direction detection method. Then, the white grid lines are aligned along their main axis. Oriented to the preset standard straight line By comparing the results, the visually measured yaw angle of the robot is obtained; S23: Visual-assisted inertial correction After the IMU sensor completes zero-point self-calibration, the bottom vision sensor detects the surface image of the photovoltaic panel in a local area in front of the robot's current direction of travel, based on the main direction of the white grid lines. Oriented to the preset standard straight line Deviation between To determine whether the robot's overall posture has been corrected; when When the robot's overall posture has been corrected, the robot will now move in a straight line along the surface of the photovoltaic panel. when If this indicates that the robot is still yawing, the overall posture of the robot will be further corrected based on the visual detection results. In the above way, the real-time detection results of the white grid lines in the local area in front of the robot can be used by the vision sensor to continuously obtain the visual yaw angle of the robot relative to the preset standard straight line, thereby realizing real-time yaw detection and attitude correction during the robot's movement. S24: Attitude outer loop control Based on the current attitude correction stage of the robot, the corresponding yaw error is selected as the attitude loop input, and the attitude loop input is input into the attitude outer loop PID controller after angle normalization processing to obtain the steering control quantity of the robot's driving direction. S25: Speed ​​Inner Loop Execution Control Based on the steering control quantity, the target speeds of the left and right tracks are generated. Combined with the actual rotation speeds of the left and right tracks fed back by the Hall sensor, the speed inner loop incremental PID controller outputs the drive control quantities of the left and right DC servo motors to achieve robot posture correction and stable driving.

[0016] In another implementation, the specific processing procedure in step S21 is as follows: Let the robot's inertial yaw output at time k be: , The angular velocity output is: , Under discrete sampling conditions, the inertial yaw angle update relationship is as follows: , in: Let be the yaw angle output by the IMU sensor at time k; Let be the angular velocity output by the IMU sensor at time k; To control the sampling period; During the robot's movement, when the IMU sensor detects the robot's yaw angle... When a change occurs, indicating a deviation in the robot's current posture, the main controller will adjust the current inertial yaw angle. As the first stage attitude deviation, it is input into the attitude outer loop PID controller in step S24 for correction to eliminate IMU sensor zero drift and instantaneous bias error; after correction, the IMU sensor output is made to meet the requirements. ; After the above processing, the zero-position state output by the IMU sensor serves as the inertial correction benchmark for the robot's current posture, providing a reference for subsequent visual verification.

[0017] In another implementation, the specific processing procedure in step S22 is as follows: 1) Extract the set of white grid line pixels Assume the visual sensor is at time... The white grid line area on the surface of the photovoltaic panel was detected, and its pixel set was extracted as follows: , Here It could be: pixels on a white grid mask; 2) Calculate the center of the white grid line point set. Define the center of the point set as: , 3) Construct the second-order moment matrix Construct the covariance matrix based on the set of white grid line points: , in: , ,

[0018] 4) Find the principal direction angle of the white grid lines. The direction angle of the white grid line is obtained from the principal axis direction of the point set, denoted as: , Its calculation formula can be written as: , this It refers to the overall main direction of the white grid lines currently detected by vision.

[0019] 5) Compare with the preset standard straight line direction Let the direction angle of the preset standard straight line be: , The deviation angle between the direction of the white grid line detected by vision and the standard straight line can be defined as: , Since the straight line has 180° symmetry, it is recommended to normalize it using the minimum included angle; it can be written as: , The obtained deviation angle always falls on: .

[0020] In another implementation, the specific processing procedure in step S24 is as follows: When the IMU sensor detects that the robot is yawing and has not yet completed zero-position self-calibration, the inertial yaw angle is used. As input to the attitude loop, the first attitude deviation is constructed: , The main controller corrects the robot's current inertial yaw state based on the first attitude deviation until the IMU sensor output meets the following: , After the IMU sensor completes zero-point self-calibration, step S23, visual deviation calibration, is initiated; at this time, the visual deviation obtained in step S22 is used as the basis for the calibration. As input to the attitude loop, construct the second attitude deviation: , When the visual detection result satisfies: When the robot's overall posture has been corrected, it indicates that: , When the visual detection result satisfies: When the robot still has an overall yaw error after the IMU sensor completes zero-position self-calibration, the main controller continues to perform attitude outer loop control based on the second attitude deviation. Considering the principle of minimum turning angle when the yaw angle crosses the boundary, the attitude deviation is normalized: , The normalized attitude deviation is input into the outer loop PID controller to obtain the steering control input: , in: This is the steering control quantity output from the outer attitude loop; , , These are the proportional, integral, and derivative coefficients of the attitude outer loop PID, respectively.

[0021] In another implementation, the specific processing procedure in step S25 is as follows: Let the robot's base forward speed be: , Let the distance between the centers of the left and right tracks be D. Then, based on the steering control quantity output from the attitude outer loop... Generate the target speed of the left and right tracks: , , in: , These represent the target speeds of the left and right tracks, respectively. The actual speeds of the left and right tracks, as measured by the Hall sensor, are as follows: , , The speed errors of the left and right tracks are respectively: , , The speed inner loop uses incremental PID control, and the control increment for the left track is: , The right track control increment is: , The outputs of the left and right motor drives are as follows: , , in: , These are the control quantities output to the left and right DC servo motor drivers, respectively.

[0022] In another implementation, the specific processing procedure in step S3 is as follows: S31: Front and rear distance information is acquired synchronously with the front image. Ultrasonic sensors installed at the front and rear of the photovoltaic cleaning robot acquire real-time distance data between the areas in front of and behind the robot; at the same time, visual sensors simultaneously collect images of the photovoltaic panel surface in front of the robot for subsequent verification and identification of abnormal areas. S32: Preliminary detection of anomalies in the front ultrasonic sensor The main controller analyzes the distance data input by the front ultrasonic sensor in real time. When the distance exceeds the set threshold, it determines that there is a suspected boundary abnormality area in front of the robot, that is, the area in front may be the gap or edge of the photovoltaic panel. At this time, the visual verification process is triggered. S33: Visual Verification Recognition When step S32 determines that there is a suspected boundary anomaly area ahead, the main controller acquires the image ahead corresponding to the current anomaly moment and inputs the image into the visual detection model to identify the photovoltaic panel edge area, inter-panel gap area and normal panel area in the image, and outputs the bounding box, category label and confidence of the corresponding target to obtain the visual recognition result at the current moment. S34: Jointly determine edges or gaps The main controller combines the abnormal state of the front ultrasonic waves, the visual recognition results, and the abnormal state of the rear ultrasonic waves to jointly determine the current abnormal area. When the front ultrasonic data is abnormal and the visual detection indicates an edge, the current abnormal area is determined to be the true edge area of ​​the photovoltaic panel. When the front ultrasonic data is abnormal and the visual detection indicates that there is a photovoltaic panel in front, it is determined that the current distance abnormality is not directly caused by the true edge, but by the gap between the panels or local ranging disturbance, and the robot can continue to move. Furthermore, if the abnormal state of the front ultrasonic waves continues after the robot continues to move, and the rear ultrasonic data is abnormal at this time, it is determined that there is a true edge risk at the current position of the robot. This indicates that the visual recognition may be misjudged due to obstacle obstruction, reflection interference, or limited local field of view. In this case, the current area should be re-determined as an edge risk area. S35: Safety Control Execution Based on the joint judgment result of step S34, corresponding control is executed. When the current area is determined to be the actual edge area of ​​the photovoltaic panel, the robot is controlled to immediately perform deceleration, stop, reverse, or turn to avoid the edge of the photovoltaic panel. When the current area is determined to be an abnormal distance caused by a gap between panels or a normal panel surface, false alarms caused by the gap between panels are suppressed, and the robot is controlled to maintain normal driving. When the visual verification result is a gap or a normal panel surface, but the abnormal distance at the front is not resolved and the ultrasonic sensor at the rear detects the abnormal distance at the same time, the robot is controlled to immediately stop driving to achieve redundant safety protection in the case of visual misjudgment.

[0023] The beneficial effects of this invention are as follows: 1. This invention reduces computational overhead by lightweighting and optimizing the original RT-DETR model, thereby improving the detection accuracy, positioning robustness, and real-time processing capability of targets such as photovoltaic panel edges, inter-panel gaps, and grid lines. This makes it more suitable for online visual inspection during the operation of photovoltaic cleaning robots.

[0024] 2. By combining the visual recognition of the yaw angle with the yaw angle output of the IMU sensor, and then forming a cascaded dual closed-loop control through the attitude outer loop and the velocity inner loop, the photovoltaic cleaning robot can achieve stable attitude control on the tilted photovoltaic panel surface.

[0025] 3. This invention uses an improved visual detection algorithm to accurately identify the edges of photovoltaic panels and the gaps between panels, and combines sensor detection results with visual recognition results to divide the area between panels; based on the area division results, the robot's driving mode is determined, thereby achieving effective control of the robot's operating state and ensuring its safety during operation on the photovoltaic panel surface. Attached Figure Description

[0026] Figure 1 Schematic diagram of the improved RT-DETR detection network structure; Figure 2 A schematic diagram of the visually guided dual-closed-loop PID attitude correction method; Figure 3 Schematic diagram of the correction process of the photovoltaic cleaning robot; Figure 4 Schematic diagram of the overall structure of the photovoltaic cleaning robot; Figure 5 Schematic diagram of the internal upper structure of the photovoltaic cleaning robot; Figure 6 Schematic diagram of the lower internal structure of the photovoltaic cleaning robot; Figure 7 Schematic diagram of the bottom structure of the photovoltaic cleaning robot; Figure 8 Actual original example image of edges and gaps; Figure 9 Example image of edge and gap detection results; Figure 10 A schematic diagram showing the actual testing effect of the photovoltaic cleaning robot running along the surface of an L-shaped photovoltaic panel. Figure 11 Online detection system flowchart; Figure 12 Schematic diagram of an online detection system for photovoltaic panel edges and gaps; Figure 13 A schematic diagram illustrating the visual detection yaw angle effect of a photovoltaic cleaning robot as it operates at its initial position on the surface of a photovoltaic panel. Figure 14 A schematic diagram illustrating the visual detection yaw angle effect of a photovoltaic cleaning robot running at the center of a photovoltaic panel surface. Figure 15 A schematic diagram illustrating the visual detection yaw angle effect of a photovoltaic cleaning robot as it moves to its final position on the photovoltaic panel surface. The components include: 1. Front-facing camera, 2. Rear ultrasonic sensor, 3. Universal joint, 4. Water pipe connector (inner and outer), 5. Aerospace plug, 6. Trip lock, 7. Main controller, 8. Motor driver, 9. Right-side drive motor, 10. Left-side drive motor, 11. Roller brush motor, 12. IMU sensor, 13. Bottom camera, and 14. Front ultrasonic sensor. Detailed Implementation

[0027] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings: In this embodiment, the improved RT-DETR vision algorithm is mainly used to identify and extract features from the edge area, inter-panel gap area and grid line area of ​​the photovoltaic panel surface, providing visual input for the posture correction in the subsequent step S2 and the safety determination in the subsequent step S3.

[0028] This section provides a further explanation of the improved RT-DETR visual detection network. This network adopts a two-stage lightweight structure, consisting of a Backbone and a Head module. It abandons the independent Neck design, integrating feature encoding, multi-scale fusion, and detection decoding into a unified Head module. When a 640×640×3 image is input into the detection network, the image undergoes a complete process from image reading and preprocessing to feature extraction, global interaction, enhancement, fusion, end-to-end detection, and post-processing. Each stage is completed collaboratively by multiple customized modules. The specific process is as follows: First, the main controller receives images of the photovoltaic panel synchronously acquired by the robot's front and bottom cameras. The images are in BGR format by default and are uniformly adjusted to a standard size of 640×640×3. The pixel values ​​of the images are normalized, mapping the numerical range from [0,255] to the [0,1] interval. Format conversion is then performed, transforming the HWC layout into a CHW layout adapted for deep learning frameworks. Finally, the data is converted to PyTorch tensor format and batch dimensions are added, resulting in standardized data with a shape of [1,3,640,640], meeting the input requirements of the improved RT-DETR model.

[0029] After entering the model inference stage, multi-scale feature extraction is first performed by the lightweight improved Backbone. The model uses the HGStem module (the HGStem module is the entry Stem module of the high-performance backbone network, containing convolutional groups with fast downsampling and channel adjustment for efficient feature extraction in the early stages) as the initial feature extraction unit to quickly compress the image resolution and extract basic edge and texture features; the main body of the network completely replaces the original HGBlock with the Ghost_HGBlock lightweight module (combining Ghost...) The module's lightweight HGBlock, used to reduce computation and parameter count, significantly reduces model parameters and computational overhead while maintaining feature representation capabilities. Combined with DWConv deep convolution (DWConv is the channel-wise spatial convolution part of depthwise separable convolution, used to perform spatial convolution on each input channel individually, greatly reducing parameter count and computational cost), multi-level downsampling is achieved. This reduces feature map resolution and computation while preserving key local details such as edges and grid lines as much as possible. SimAM parameterless attention mechanism is embedded at the end of the highest layer of the network (assigning weights to each neuron based on an energy function, enhancing salient features without additional learnable parameters), adaptively strengthening feature weights for key structural regions such as photovoltaic panel edges, inter-panel seams, and grid lines. The entire backbone network outputs three layers of effective multi-scale feature maps: P3 (1 / 8 downsampling, shallow detail features), P4 (1 / 16 downsampling, mid-level semantic features), and P5 (1 / 32 downsampling, high-level abstract features), providing rich feature support for subsequent detection.

[0030] After backbone network feature extraction, the data is directly input into the integrated enhanced Head module, which integrates four core functions: Transformer global encoding, FPN-PAN bidirectional feature pyramid, multi-layer attention enhancement, and RT-DETR decoder. First, a 1×1 convolution is used to uniformly reduce the dimensionality of the three-layer multi-scale features, lowering computational overhead. Then, an AIFI (an intra-scale feature interaction module based on an attention-based Transformer Encoder, used for global context modeling within a single scale to enhance feature representation) lightweight Transformer encoder is connected to model global feature dependencies and capture long-range structural features of the photovoltaic panel. A bidirectional feature fusion pyramid is constructed using Upsample, skip connections, Concat (tensor concatenation, connecting feature maps of different branches or scales along the channel dimension), and a RepC3 structure (RepC3 is a reparameterized C3 module; during training, it is a multi-branch structure to improve performance; during inference, it can be merged into a single path to improve speed). This pyramid fuses high-level semantic information from top to bottom and enhances shallow detail features from bottom to top, achieving efficient complementarity of multi-scale features.

[0031] To further enhance the representation of key features, the SimAM attention module is applied to the effective output features of the P3, P4, and P5 layers after feature fusion. This module performs layered enhancement on detailed, semantic, and abstract features, significantly improving the perception of small targets, weak edges, and low-contrast defects. Finally, the triple-enhanced P3, P4, and P5 feature maps are input into the RTDETRDecoder end-to-end decoder, directly outputting the target detection results without the need for non-maximum suppression post-processing.

[0032] After inference, a lightweight post-processing decoding operation is performed: the predicted bounding box parameters output by the model are converted into the detection bounding box coordinates in the original image coordinate system, and the target category and confidence score are resolved. Thanks to the end-to-end design of RT-DETR, the redundant computation of traditional NMS is eliminated, significantly improving the inference speed. Finally, the optimal detection result is directly output, and each target includes accurate location information, category label (photovoltaic panel edge and gap), and detection confidence score.

[0033] Figure 12 The image showcases the detection performance of edges and seams in photovoltaic panels. Each column displays the target type, original image, model output, and a magnified view. Overall, this improved RT-DETR network demonstrates excellent performance in edge and seam detection. Photovoltaic panel edge: The positioning is continuous and complete, and the detection frame closely matches the actual edge direction; Inter-board gaps: Stable identification of narrow gaps and low-contrast gaps, with no breaks or missed detections; Grid line area: can be accurately identified simultaneously, without interfering with gaps and edges, and is clearly classified.

[0034] In summary, this improved RT-DETR visual detection network significantly enhances the detection accuracy and robustness for slender, weakly textured targets such as photovoltaic panel edges, seams, and grid lines through its lightweight Ghost_HGBlock structure, DWConv depth downsampling, AIFI global encoding, and multi-layer SimAM attention enhancement. The model combines lightweight design with real-time performance, enabling edge alignment, seam recognition, and autonomous walking decision-making on photovoltaic cleaning robots, thus meeting the needs of automated cleaning operations.

[0035] Figure 10 The system demonstrates its detection application in a real photovoltaic scenario: it can accurately identify the edges and gaps between photovoltaic panels, providing a reliable basis for robot movement correction and path planning; it achieves collaborative edge gap detection and cleaning operations, ensuring that the robot moves smoothly and cleans accurately along the gaps between panels, avoiding damage to the photovoltaic panels.

[0036] Table 1 below compares the performance metrics of the original RT-DETR with the improved model in this paper: The improved model significantly improves the mAP (mean accuracy) for edge and gap detection, greatly improves the recognition accuracy for slender, weak edges and narrow gaps, significantly reduces the number of parameters and computation, has a faster inference speed, lower resource consumption on embedded platforms, and significantly reduces the false detection rate and false negative rate for edge and gap detection, thus having higher practicality and reliability.

[0037] Table 1

[0038] In summary, this improved RT-DETR visual inspection network achieves a synergistic improvement in both accuracy and efficiency through four optimization strategies: the Ghost_HGBlock lightweight structure, DWConv deep convolution, AIFI global encoding, and hierarchical SimAM attention enhancement. The model can accurately identify the edges of photovoltaic panels and gaps between panels, with particularly outstanding performance in detecting elongated edges and low-contrast gaps. It outperforms the original model in terms of lightweight design, real-time performance, and detection accuracy. Highly compatible with the onboard embedded platform of photovoltaic cleaning robots, it provides reliable visual support for autonomous robot movement, path planning, and precise cleaning, meeting the needs of automated inspection and cleaning operations.

[0039] like Figure 2The diagram illustrates a visually guided dual-closed-loop PID attitude correction method. In this embodiment, the machine attitude correction process is divided into two stages: the first stage is attitude deviation detection and correction based on IMU sensors; the second stage is attitude verification and further correction based on visual recognition results. Both stages utilize the same control chain, meaning the attitude deviation obtained in each stage is input into the outer-loop PID controller for calculation. Then, based on the steering control output from the outer loop, the speed of the left and right DC servo motors is adjusted by the inner-loop PID controller, thereby achieving robot attitude correction and stable movement.

[0040] Figure 3 This diagram illustrates the robot's attitude correction during operation. The bottom camera vertically captures the grid structure (thin black lines) on the photovoltaic panel surface, while the robot's own coordinate axes (red dashed lines) serve as reference lines. When the robot veers off course due to friction or unevenness on the panel, the vision and IMU sensors work together to correct its heading in a timely manner, ensuring the robot travels in a straight line along the photovoltaic panel grid and preventing it from falling.

[0041] The specific details of machine attitude control are as follows: Phase 1: Acquiring IMU yaw angle and inertial null self-calibration phase The IMU sensor detects the robot's yaw status in real time. When yaw is detected, the main controller first enters the inertial zero-position self-calibration stage, using the current inertial yaw angle as the calibration value, until the IMU sensor output meets the zero-position reference requirement. The IMU sensor is not only used for attitude detection during the robot's straight-line travel, but also for angle control during the robot's turning process, so that the robot can complete the turning action at a preset angle.

[0042] Let the robot's inertial yaw output at time k be:

[0043] The angular velocity output is:

[0044] Under discrete sampling conditions, the inertial yaw angle update relationship is as follows:

[0045] in: Let be the yaw angle output by the IMU sensor at time k; Let be the angular velocity output by the IMU sensor at time k; To control the sampling period.

[0046] When the robot is in the first stage of attitude correction, the attitude outer loop input error is taken as:

[0047] Considering the principle of minimum turning angle when the yaw angle crosses the boundary, the attitude deviation is normalized and then input into the outer loop PID controller to obtain the steering control quantity:

[0048] in, This is the steering control quantity output from the outer attitude loop; , , These are the proportional, integral, and derivative coefficients of the attitude outer loop PID, respectively.

[0049] Based on the steering control input from the outer attitude loop, the target speeds for the left and right tracks are generated: ,

[0050] in, The robot's baseline forward speed, The distance between the centers of the left and right tracks. , The target speeds for the left and right tracks are set separately. Subsequently, Hall effect sensors provide real-time feedback on the actual speeds of the left and right tracks. The speed inner-loop PID controller outputs the drive control quantities for the left and right motors based on the error between the target speed and the actual speed, enabling the robot to complete the first stage of attitude correction.

[0051] After the first stage of calibration is completed, the IMU sensor output should meet the following requirements:

[0052] At this point, the zero-position state output by the IMU sensor serves as the inertial reference benchmark for subsequent visual verification.

[0053] Phase Two: Visual-Assisted Posture Correction like Figure 13 , 14 As shown in Figure 15, the robot's current angle is obtained by identifying the white grid lines on the photovoltaic panel surface in the real operating environment by the front-facing camera. This angle is compared with a standard straight line to obtain the robot's current yaw angle. After the attitude loop deviation is corrected by the IMU, further verification is performed, and the attitude is corrected.

[0054] After completing the first stage of correction, the vision sensor acquires an image of the photovoltaic panel surface in a local area in front of the robot, identifies the white grid line area in the image, calculates the principal direction of the white grid line using the second-order moment principal axis direction detection method, and compares the principal direction with the preset standard straight line direction to obtain the robot's current visual deviation angle.

[0055] in, The principal direction angle of the white grid lines obtained by visual detection. To preset the standard straight line direction angle, This represents the second-stage pose deviation obtained from visual recognition.

[0056] The main controller is based on the visual deviation angle The results of the first stage of correction were reviewed; when: When the robot's overall posture has been corrected, the robot continues to move in a straight line along the surface of the photovoltaic panel. when: When this occurs, it indicates that the robot still has an overall yaw after completing the first stage of correction. At this point, the main controller will adjust the visual deviation angle. The second-stage attitude deviation is input to the same attitude outer-loop PID controller for calculation, resulting in a new steering control quantity. This quantity is then adjusted by the speed inner-loop PID controller to regulate the speed of the left and right DC servo motors, thereby achieving the second-stage attitude correction.

[0057] When the robot is in the second stage of attitude correction, the attitude outer loop input error is taken as: Similarly, after normalizing the attitude deviation, it is input into the attitude outer loop PID controller to obtain the steering control quantity. Then, based on the steering control quantity, the target speeds of the left and right tracks are generated, and combined with the actual speeds of the left and right tracks fed back by the Hall sensors, the speed inner loop PID controller outputs the drive control quantities of the left and right motors, so that the overall posture of the robot is further inclined towards the target direction.

[0058] Subsequently, the target speeds of the left and right tracks are generated based on the steering control quantity, and the target speeds of the left and right tracks are input into the motor driver. Combined with the actual speeds of the left and right tracks fed back by the Hall sensor, the speed inner loop PID controller outputs the drive control quantities of the left and right DC servo motors. Finally, the robot's posture correction and stable driving are achieved by adjusting the speed of the left and right DC servo motors.

[0059] In this embodiment, attitude control is a continuous closed-loop control process. During robot operation, the IMU continuously outputs inertial attitude information, the vision sensor continuously acquires images of the photovoltaic panel surface, and the Hall sensor continuously feeds back the left and right motor speed information. The main controller executes the following process cyclically according to a preset control cycle: First, it determines whether the robot is yawing based on the inertial yawing angle output by the IMU sensor; when yawing is detected, it uses the inertial yawing angle... The attitude deviation is input to the outer loop PID controller in the first stage, and then further processed by the inner loop PID controller to perform differential speed adjustment of the left and right tracks, completing the first stage of correction; subsequently, the vision sensor identifies the direction of the grid lines on the photovoltaic panel surface and constructs the visual deviation angle. The main controller verifies the first-stage correction result based on the visual deviation angle. When the visual deviation angle is not zero, the visual deviation angle is input as the second-stage attitude deviation into the same attitude outer loop PID controller, and the left and right track differential adjustment is performed through the speed inner loop PID controller to complete the second-stage correction. During the continuous movement of the robot, the above process is repeated to achieve continuous correction and stable control of the robot's movement path.

[0060] In the second stage, the principal axis direction of the grid lines on the photovoltaic panel surface is identified using the second-order moment principal axis direction detection method.

[0061] During robot operation, the vision sensor continuously acquires images of white grid lines within a local area in front of the robot and extracts the set of pixels corresponding to these white grid lines. These pixels exhibit a geometric characteristic of extending along the grid line direction and contracting vertically. Therefore, by constructing the second-order moment matrix of the pixel set and determining its direction of maximum variance, the principal axis direction of the white grid line can be obtained. Then, the principal axis direction... Oriented to the preset standard straight line Compare and obtain the current deviation angle. .

[0062] 1) Extract the set of white grid line pixels Assume the visual sensor is at time... The white grid line area on the surface of the photovoltaic panel was detected, and its pixel set was extracted as follows:

[0063] Here It could be: pixels on a white grid mask; 2) Calculate the center of the white grid line point set. Define the center of the point set as:

[0064] 3) Construct the second-order moment matrix Construct the covariance matrix based on the set of white grid line points:

[0065] in: , ,

[0066] 4) Find the principal direction angle of the white grid lines. The direction angle of the white grid line is obtained from the principal axis direction of the point set, denoted as:

[0067] Its calculation formula can be written as:

[0068] this It refers to the overall main direction of the white grid lines currently detected by vision.

[0069] 5) Compare with the preset standard straight line direction Let the direction angle of the preset standard straight line be:

[0070] The deviation angle between the direction of the white grid line detected by vision and the standard straight line can be defined as:

[0071] Because the straight direction has To maintain symmetry, it is recommended to normalize the angle to the minimum. This can be written as:

[0072] The resulting deviation angle always falls on: .

[0073] Furthermore, the attitude outer-loop PID controller in the first and second stages will be explained in more detail: The main controller generates target speeds for the left and right tracks based on the steering control output of the attitude outer loop PID controller in the first and second stages, combined with the baseline forward speed set by the robot. This allows the robot to adjust its heading by differential speed between the left and right tracks while maintaining its overall forward trend.

[0074] In this embodiment, the attitude outer-loop PID controller outputs a steering control quantity based on the robot's current attitude deviation:

[0075] in, Indicates the robot at a certain moment The steering correction force required to eliminate the current attitude deviation. This steering control value does not directly drive the motor, but serves as the basis for the target speed distribution between the left and right tracks.

[0076] Let the robot's base forward speed be:

[0077] Let the distance between the centers of the left and right tracks be:

[0078] Then, based on the steering control quantity output from the attitude outer loop... The generated left and right track target speeds are as follows: ,

[0079] in: The target speed for the left track; The target speed for the right track; This is the baseline forward speed of the robot when it is moving normally in a straight line. The distance between the centers of the left and right tracks; The steering control quantity output by the attitude outer loop PID controller is shown in Table 2 below.

[0080] Table 2

[0081] Therefore, by differentially distributing the speeds of the left and right track targets, the robot can maintain its overall forward motion while simultaneously correcting its current yaw state in real time. The speed difference between the left and right track targets is: .

[0082] Furthermore, the speed inner loop PID controller in the first and second stages will be explained in more detail: 1. Hall effect sensor provides feedback on actual speed and constructs speed error. The left and right DC servo motors are each equipped with a Hall sensor to collect the rotational speed information of the left and right track drive motors in real time. The pulse signals detected by the Hall sensors are transmitted to the main controller, which calculates the actual operating speed of the left and right tracks based on the pulse signals, the sampling period, and the motor transmission parameters.

[0083] In this embodiment, let the Hall sensor be in the first... The number of pulses detected from the left and right motors within each control cycle are as follows:

[0084] Let the number of Hall pulses per revolution of the left and right motors be respectively:

[0085] Let the equivalent circumferences of the left and right track drive wheels be respectively:

[0086] The sampling period is controlled as follows:

[0087] The actual speeds of the left and right tracks, calculated from the Hall pulse signals, are as follows: ,

[0088] in: For a moment Actual speed of the left track; For a moment Actual speed of the right track; , These represent the number of pulses collected by the left and right Hall sensors during the current control cycle, respectively. , These represent the number of Hall pulses per revolution of the left and right motors, respectively. , These are the equivalent circumferences of the left and right track drive wheels, respectively. To control the sampling period.

[0089] In the control link, the main controller has already implemented the steering control input based on the attitude outer loop output. and robot reference forward speed Generate the target speed of the left and right tracks: ,

[0090] in, This is the distance between the centers of the left and right tracks.

[0091] Subsequently, the main controller compares the target speeds of the left and right tracks generated by the attitude outer loop with the actual speeds of the left and right tracks fed back by the Hall sensors, respectively, to obtain the speeds of the left and right tracks at time [time value missing]. Speed ​​error: ,

[0092] in: This refers to the left track speed error. The right track speed error is shown in Table 3 below.

[0093] Table 3

[0094] Furthermore, to meet the calculation requirements of the subsequent incremental PID controller for historical error terms, it is also necessary to save the speed errors of the left and right tracks at the previous moment and the two moments before that, i.e.: This is so that the inner loop control increments for the left and right track speeds can be constructed separately in the next step.

[0095] It should be noted that the outer attitude loop is used to calculate the required steering control amount based on the robot's current attitude deviation, thereby determining the target speed that the left and right tracks should reach; while the inner speed loop is used to dynamically correct the drive output of the left and right motors based on the difference between the target speed and the actual speed, so as to ensure that the left and right tracks can accurately track the target speed given in step four.

[0096] 2. The speed loop PID output drives the voltage and drives the DC servo motor. The internal speed loop of the motor inputs the left and right track speed errors obtained in the previous step into the corresponding speed inner loop PID controllers. The speed inner loop PID controllers output the drive control quantities of the left and right DC servo motors according to the current speed error and its historical changes, so that the actual speed of the left and right tracks can accurately track the target speed generated in step four, thereby realizing robot posture correction and stable driving.

[0097] In this embodiment, the speed inner loop preferably adopts incremental PID control. Let the left and right tracks be at time... The speed errors are as follows:

[0098] According to the incremental PID control law, the control increment of the left track is: The right track control increment is: , in: For the left track at time Control increment; For the right track at time Control increment; , , These are the proportional, integral, and derivative coefficients of the speed inner loop incremental PID controller.

[0099] Furthermore, the drive outputs of the left and right motors are updated as follows: ,

[0100] in: For a moment The control input is sent to the left-side DC servo motor driver; For a moment The control input is sent to the DC servo motor driver on the right.

[0101] The control quantity , This can be represented as a voltage control command, a PWM duty cycle control quantity, or an equivalent motor drive signal. After receiving the control quantity output by the main controller, the motor driver drives the left and right DC servo motors respectively, thereby adjusting the speed and torque output of the left and right tracks, as shown in Table 4 below.

[0102] Table 4

[0103] In this embodiment, after the left and right motor drive control quantities output by the speed inner loop incremental PID controller are applied to the left and right DC servo motors respectively, the actual speeds of the left and right tracks will continuously approach the target speeds of the left and right tracks generated by the attitude outer loop, that is: ,

[0104] Therefore, the speed difference between the left and right tracks can accurately reflect the steering control requirements given by the outer attitude ring, thereby gradually reducing the robot's yaw angle and tending towards the target direction, thus realizing real-time correction of the robot's attitude angle.

[0105] It should be noted here that the attitude outer loop is responsible for calculating the degree of steering correction the robot should make based on the current attitude deviation, thus obtaining the steering control quantity. The speed inner loop is responsible for further outputting the drive control quantities for the left and right motors based on the error between the target speed and the actual speed of the left and right tracks, ensuring that the motor system can accurately execute the control requirements proposed by the attitude outer loop. Through the cascaded dual closed-loop control of the attitude outer loop and the speed inner loop, the speed tracking accuracy, anti-interference ability, and attitude correction stability of the robot during its operation on the photovoltaic panel surface can be effectively improved.

[0106] During operation on the surface of photovoltaic panels, the photovoltaic cleaning robot first uses the front ultrasonic sensor 14 to perform preliminary detection of abnormal distances in front. When an abnormal distance is detected, visual recognition is used to verify whether it is an edge. If both the front ultrasonic sensor and the rear ultrasonic sensor 2 detect an abnormal distance, it is determined to be an edge, thereby suppressing misjudgments of gaps between panels and ensuring safety control against falls in the edge areas of photovoltaic panels.

[0107] The specific implementation process is as follows: Phase 1: Abnormal Ultrasonic Distance Detection First, place the photovoltaic cleaning robot at any starting position on the photovoltaic panel, start the robot and cleaning mechanism, and make the robot move along the surface of the photovoltaic panel to perform cleaning operations. During the operation of the robot, the left and right ultrasonic sensors set at the front of the robot collect distance information of the area in front in real time. The main controller analyzes the distance data output by the ultrasonic sensors in real time. When the distance in front exceeds the preset boundary threshold, it determines that there is a suspected abnormal area in front of the robot and triggers the visual verification process.

[0108] Phase Two: Visual Verification and Recognition like Figure 10As shown, during robot operation, the vision sensor synchronously acquires images of the photovoltaic panel surface in front of the robot. The controller receives and processes the image data according to a preset sampling period. When the ultrasonic sensor detects an abnormal distance, the main controller acquires the image corresponding to the current abnormal moment and inputs it into the improved RT-DETR visual detection model. The model identifies the edge region of the photovoltaic panel, the gap region between panels, and the normal panel surface region in the image, and outputs the bounding box, category label, and confidence score of the corresponding target. The controller performs visual verification of the current abnormal region based on the category label and detection results. If the visual determination of the area in front is a photovoltaic panel, the distance abnormality is caused by the gap between panels, and the robot can move safely. If the visual determination of the area in front is an edge, the robot stops immediately.

[0109] Furthermore, if the front ultrasonic sensor detects an abnormal distance, the vision system will perform a further verification. If the vision verification result is a gap, the robot will continue to move. However, if the abnormal distance of the front ultrasonic sensor has not been resolved, and the rear ultrasonic sensor simultaneously detects an abnormal distance, the robot will immediately stop moving. During visual recognition, there may be misjudgments due to obstacles in front, as shown in Table 5 below.

[0110] Table 5

[0111] Furthermore, after identifying edges and gaps and confirming safe operating conditions, visual information can also be used to assist in identifying areas on the photovoltaic panel surface that need cleaning. The controller can distinguish between cleaned and uncleaned areas on the photovoltaic panel surface based on brightness distribution, texture differences, or stain coverage characteristics in the image; when multiple directions meet the conditions for safe continued operation, the direction of the uncleaned area can be prioritized to continue operation, thereby improving the continuity and coverage efficiency of subsequent cleaning.

[0112] like Figures 4 to 7 As shown, the photovoltaic cleaning robot system in this embodiment mainly includes a vision perception module, a cleaning module, a motor drive module, an inertial measurement module, and a main control system. Each module works collaboratively to transmit information to the main control system, which then issues commands to coordinate the work of each module, achieving stable and safe operation of the photovoltaic cleaning robot. The coordinated operation control method is as follows: 1. Path correction module Two cameras are mounted on the main body of the photovoltaic cleaning robot. The front camera 1 is mounted on the top of the robot body to detect contaminated areas, edges, and crevices. Image information is transmitted to the main control system for determining the robot's safe driving area. The bottom camera 13 is mounted vertically downwards on the bottom of the main body to acquire real-time images of the photovoltaic panel grid lines. An IMU sensor 12 is located in the center of the robot body to detect the robot's heading angle, rotational angular velocity, etc., in real-time. The main controller 7 processes the grid line images acquired by the bottom camera 13 to determine the main axis orientation angle. principal axis direction angle Angle of direction from the preset standard straight line The deviation angle is obtained by comparison. yaw angle of IMU sensor Perform correction to obtain the corrected yaw angle. The main controller corrects the yaw angle. The target speeds of the right drive motor 9 and the left drive motor 10 are calculated, and instructions are sent to the motor driver 8 via RS485 communication. The internal speed loop of the motor adjusts the speed so that the robot can achieve adaptive correction and avoid deviation of the cleaning path.

[0113] 2. Cleaning module A universal joint 3 is installed on the upper part of the robot body for connecting to an external water source, preventing the robot from getting tangled in the external water pipe during cleaning. The water pipe connectors 4 connect to the other end of the universal joint 3, supplying water to the spraying system at the robot's head. Water is transported through the water pipe to the fan-shaped nozzles, where it is sprayed over a large area under pressure. Simultaneously, the roller brush motor 11 drives the roller brush to rotate via a pulley structure. By spraying water to wet the dust on the photovoltaic panel surface, the rotating roller brush cleans the dust, and finally, a scraper removes the water and dust, achieving deep cleaning of the photovoltaic panel surface and effectively improving power generation efficiency.

[0114] 3. Safety Detection Module The rear ultrasonic sensor 2 is installed on the left and right sides of the bottom of the robot. The rear ultrasonic sensor 2 detects the distance between the bottom of the robot and the surface of the photovoltaic panel in real time and transmits the data to the main control system in real time. When the data value exceeds the set safe distance value, it means that the robot is passing through the gap between the panels or has reached the edge. At this time, combined with the situation in the front area observed in real time by the front camera 1, it is determined whether it is safe to pass through, thus ensuring the safety of the robot.

[0115] 4. Quick-release connection device The robot body and cleaning brush head are connected via a release lock 6, the data communication cable and power cable are connected via an aerospace plug 5, and the water supply is connected via an internal and external water pipe connector 4. The robot body and cleaning brush head are connected via a quick-release mechanism, allowing for disassembly during transport and facilitating easy handling and installation.

[0116] 5. Main control system The main controller 7 is the core control unit of the robot. Based on an embedded computing platform, it runs multi-threaded tasks and is responsible for collecting and centrally processing data from modules such as vision images, ultrasonic sensors, motors, and IMU sensors. This enables the modules to cooperate with each other and controls the coordinated work of each module by issuing commands.

[0117] like Figure 11 , 12 As shown, this embodiment provides an online detection system for the edges and gaps of photovoltaic panels. This online detection system is used to identify the edge areas and gap areas between photovoltaic panels in real time and output corresponding detection results, providing a basis for the safe driving control of robots. The online detection system can be deployed on a host computer or a main controller, acquiring images of the area in front of the photovoltaic panel through a vision acquisition device and performing online analysis and processing of the image data. The online detection system includes an initialization and parameter configuration module, an image acquisition module, a detection model module, a result judgment module, a display output module, and a data management module.

[0118] Specifically, the initialization and parameter configuration module is used to initialize the system's operating state, load the detection model, and set detection parameters; the image acquisition module is used to acquire real-time image or video data of the area in front of the robot; the detection model module is used to process the acquired image data based on the target detection algorithm to extract target area features and complete the detection of targets at the edge of the photovoltaic panel and in the gap between the panels; the result determination module is used to generate target category, target location, and confidence information based on the output results of the detection model, and to determine whether the current area belongs to the edge area of ​​the photovoltaic panel or the gap area between the panels; the display output module is used to visualize and display the detection results; and the data management module is used to record, statistically analyze, store, and export the detection results.

[0119] In this embodiment, the system is first initialized through the initialization and parameter configuration module, and the detection parameters are set according to the actual working conditions. Then, the image acquisition module acquires real-time image data of the area in front of the robot and inputs the image data into the detection model module for online recognition. After the detection model module completes the target detection, it transmits the detection result to the result judgment module. When the judgment result is the edge of the photovoltaic panel, the edge detection signal is output; when the judgment result is the gap between the panels, the gap detection signal is output. The display output module displays the bounding box, category label, and confidence information of the recognition result. The data management module classifies, statistically analyzes, saves, and exports the current detection result and the historical detection result.

[0120] Furthermore, the main controller further determines the current area based on the edge detection signal or gap detection signal output by the result determination module, and in combination with the distance detection information from the ultrasonic sensor, to distinguish between real edges and gaps between boards. When it is determined to be a real edge area, the controller controls the robot to perform deceleration, stopping, reversing, or turning to avoid it. When it is determined to be a gap between boards, the controller suppresses false alarms and controls the robot to continue moving.

[0121] Through the above methods, the online detection system can realize real-time detection, result display, and data management of the edges and gaps of photovoltaic panels, thereby improving the safety and reliability of the robot during operation.

[0122] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for visual guidance posture correction and safety control of a photovoltaic cleaning robot, characterized in that, It includes the following steps: S1: Visual Detection Model Construction and Feature Extraction During the operation of the photovoltaic cleaning robot, the visual sensor is used to collect images of the photovoltaic panel surface in real time, and the images are input into the improved RT-DETR visual inspection model for detection. The improved RT-DETR visual inspection model is used to identify the photovoltaic panel edges, gaps between panels, and surface grid lines, and obtain one or more feature information of the category, position, and orientation of the currently collected photovoltaic panel surface image. This information is used as the visual input for subsequent posture correction and safety control. S2: Visually Guided Dual-Loop PID Attitude Correction Method Using the main direction of the grid lines on the surface of the photovoltaic module as a visual reference benchmark, the yaw output of the IMU is corrected online. Then, through the attitude outer loop and the velocity inner loop, a cascaded dual closed-loop control is formed to achieve stable attitude control of the photovoltaic cleaning robot on the tilted photovoltaic panel surface. S3: Combined ultrasound and vision recognition for edges and gaps During the operation of the photovoltaic cleaning robot along the surface of the photovoltaic panel, ultrasonic sensors installed at the front and rear of the robot are used to detect the distance information between the front and rear areas in real time. Combined with a vision sensor, the robot performs image acquisition and recognition of the area in front. When an abnormal distance is detected, the front ultrasonic sensor first triggers the suspected edge area, then the vision sensor verifies and identifies the current abnormal area, and finally the detection results of the rear ultrasonic sensor are combined to make a joint judgment to distinguish whether the current abnormal area is the edge of the photovoltaic panel or the gap between the panels. This helps to suppress misjudgment of the gap between the panels and prevent the photovoltaic cleaning robot from falling from the edge area of ​​the photovoltaic panel.

2. The method for visual guidance posture correction and safety control of photovoltaic cleaning robots according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11: Image Preprocessing Two cameras, positioned above and below the photovoltaic cleaning robot, simultaneously capture images of the photovoltaic panel surface. The images include the edges and gaps of the photovoltaic panel in the area directly in front of the robot, captured by the front camera, and the grid line area of ​​the photovoltaic module in the direction of the robot's movement, captured by the bottom camera. The captured images are transmitted to the main controller, which sequentially performs median filtering for noise reduction, adaptive histogram equalization for enhancement, scale unification, and tensor quantization on the raw images to obtain standardized images suitable for input to the improved RT-DETR visual inspection model. S12: Backbone Network Feature Extraction The standardized image obtained in step S11 is input into the backbone network of the improved RT-DETR visual detection model to extract multi-scale feature information from the photovoltaic panel surface image. S13: Feature Enhancement and Fusion The multi-scale feature information output in step S12 is input into the integrated enhanced Head module to perform global context modeling and cross-scale fusion of features at different levels to obtain a fused feature map, thereby improving the model's ability to recognize multi-scale and multi-morphological targets. S14: Detection Prediction The fused feature map obtained in step S13 is input into the detection head to classify and predict the target related to the edge area of ​​the photovoltaic panel, the gap area between the panels and the grid line, and output the bounding box parameters, target confidence and target category information of the corresponding target. Assume the detection head is at time The output of the first The detection targets are: , Where, x n (k) represents the x-coordinate of the center point of the target's bounding box, y n (k) represents the ordinate of the center point of the target bounding box. n (k) represents the width of the target bounding box, h n (k) represents the length of the bounding box, s n (k) represents the target confidence level, c n (k) represents target category information; S15: Post-processing of test results The detection results output in step S14 are decoded and filtered. The normalized bounding boxes are mapped back to the original image coordinate system, and the valid detection results are filtered according to the preset confidence threshold to obtain the final target box coordinates, target category and target confidence information, which serve as the visual input for subsequent robot posture correction and safety control. S16: Target Feature Information Output and Feature Code Generation Based on the valid detection results obtained in step S15, the category features, location features, and orientation-related features of edge targets, gap targets, and grid line targets are extracted respectively, and the corresponding target feature information or feature codes are generated.

3. The method for visual guidance posture correction and safety control of photovoltaic cleaning robots according to claim 2, characterized in that, In step S11, the specific processing procedure is as follows: The original image is first subjected to multi-kernel median filtering to suppress noise interference; then adaptive histogram equalization is performed to enhance the local contrast of edges, seams, and grid line areas; finally, the image is scaled to a uniform input scale and normalized. , in, For a moment The original images obtained, The image data is normalized; the normalized image is then converted to... The format is then added, and the batch dimension is added to obtain the model input tensor.

4. The method for visual guidance posture correction and safety control of photovoltaic cleaning robots according to claim 2, characterized in that, In step S12, the specific processing procedure is as follows: The backbone network first extracts basic features through convolutional layers, normalization layers, and activation layers; then it obtains feature maps at different scales through hierarchical convolutional downsampling; the convolutional neural network in the Backbone of the visual model is the Ghost_HGBlock module, and a lightweight backbone network is constructed by combining the DWConv deep convolutional structure during the downsampling stage; the backbone network finally outputs shallow detail feature maps (P3), mid-level semantic feature maps (P4), and high-level abstract feature maps (P5).

5. The method for visual guidance posture correction and safety control of photovoltaic cleaning robots according to claim 4, characterized in that, In step S13, the specific processing procedure is as follows: After the high-level semantic features are output from the backbone network, a SimAM parameterless attention module is introduced for enhancement. Then, the global feature dependencies are modeled through the AIFI lightweight Transformer encoder. Multi-scale feature fusion is achieved through upsampling, skip connections, concatenation, and the RepC3 structure integrated in the Head module. SimAM attention enhancement is further introduced on the fused shallow detail feature map (P3), mid-level semantic feature map (P4), and high-level abstract feature map (P5) to obtain a fused feature map that combines detailed information and global semantic information.

6. The method for visual guidance posture correction and safety control of photovoltaic cleaning robots according to claim 2, characterized in that, In step S14, the specific processing procedure is as follows: In the improved RT-DETR visual detection model, SIoU loss is employed, and a bounding box regression loss function is constructed: , in, For location regression loss term, The bounding box shape and orientation constraint loss term is used to simultaneously constrain the overlap, center distance, and shape difference between the predicted box and the ground truth box, so as to improve the detection box localization accuracy and convergence speed.

7. The method for visual guidance posture correction and safety control of photovoltaic cleaning robots according to claim 2, characterized in that, In step S16, the specific processing procedure is as follows: For the edge region and inter-panel gap region of photovoltaic panels, the target category, bounding box coordinates, length, gap width, and relative position parameters are extracted. The model output category label is then mapped to the predefined target type to generate the corresponding category code. For the photovoltaic module grid line area, the corresponding detection area or pixel distribution information is output, providing input for the calculation of the principal direction angle of the white grid line using the second-order moment principal axis direction detection method in subsequent step S22; the category is encoded according to the preset encoding rules. The target feature code is generated by combining parameters such as target length, gap width, and relative coordinates, and serves as the input basis for robot path planning, posture correction, safety judgment, and operation and maintenance early warning.

8. The method for visual guidance posture correction and safety control of a photovoltaic cleaning robot according to claim 7, characterized in that, In step S2, the specific processing procedure includes the following steps: S21: Acquisition of inertial attitude state The robot's current yaw angle, attitude angle, and angular velocity information are collected by an IMU sensor as real-time inertial attitude state quantities during the robot's movement. The IMU sensor is used for attitude detection during the robot's straight-line movement and angle control during the robot's turning process, so that the robot can complete the turning action at a set angle. S22: Visual Yaw Angle Construction During the robot's movement, a camera mounted on the bottom of the photovoltaic cleaning robot vertically captures images of the grid structure on the photovoltaic panel surface. The white grid area is extracted from the image, and the principal axis direction of the current white grid line is calculated using a second-order moment principal axis direction detection method. Then, the white grid lines are aligned along their main axis. Oriented to the preset standard straight line By comparing the results, the visually measured yaw angle of the robot is obtained; S23: Visual-assisted inertial correction After the IMU sensor completes zero-point self-calibration, the bottom vision sensor detects the surface image of the photovoltaic panel in a local area in front of the robot's current direction of travel, based on the main direction of the white grid lines. Oriented to the preset standard straight line Deviation between To determine whether the robot's overall posture has been corrected; when When the robot's overall posture has been corrected, the robot will now move in a straight line along the surface of the photovoltaic panel. when If this indicates that the robot is still yawing, the overall posture of the robot will be further corrected based on the visual detection results. S24: Attitude outer loop control Based on the current attitude correction stage of the robot, the corresponding yaw error is selected as the attitude loop input, and the attitude loop input is input into the attitude outer loop PID controller after angle normalization processing to obtain the steering control quantity of the robot's driving direction. S25: Speed ​​Inner Loop Execution Control Based on the steering control quantity, the target speeds of the left and right tracks are generated. Combined with the actual rotation speeds of the left and right tracks fed back by the Hall sensor, the speed inner loop incremental PID controller outputs the drive control quantities of the left and right DC servo motors to achieve robot posture correction and stable driving.

9. The method for visual guidance posture correction and safety control of a photovoltaic cleaning robot according to claim 8, characterized in that, In step S21, the specific processing procedure is as follows: Let the robot's inertial yaw output at time k be: , The angular velocity output is: , Under discrete sampling conditions, the inertial yaw angle update relationship is as follows: , in: Let be the yaw angle output by the IMU sensor at time k; Let be the angular velocity output by the IMU sensor at time k; To control the sampling period; During the robot's movement, when the IMU sensor detects the robot's yaw angle... When a change occurs, indicating a deviation in the robot's current posture, the main controller will adjust the current inertial yaw angle. As the first stage attitude deviation, it is input into the attitude outer loop PID controller in step S24 for correction to eliminate IMU sensor zero drift and instantaneous bias error; after correction, the IMU sensor output is made to meet the requirements. .

10. The method for visual guidance posture correction and safety control of a photovoltaic cleaning robot according to claim 8, characterized in that, In step S22, the specific processing procedure is as follows: 1) Extract the set of white grid line pixels Assume the visual sensor is at time... The white grid line area on the surface of the photovoltaic panel was detected, and its pixel set was extracted as follows: , Yes: Pixels on the white grid line mask; 2) Calculate the center of the white grid line point set. Define the center of the point set as: , 3) Construct the second-order moment matrix Construct the covariance matrix based on the set of white grid line points: ,in: , , , 4) Find the principal direction angle of the white grid lines. The direction angle of the white grid line is obtained from the principal axis direction of the point set, denoted as: , Its calculation formula can be written as: , The overall main direction of the white grid lines currently detected by vision; 5) Compare with the preset standard straight line direction Let the direction angle of the preset standard straight line be: , The deviation angle between the direction of the white grid line detected by vision and the standard straight line is defined as: , The straight line has 180° symmetry, and after normalization to the minimum included angle, it can be written as: , The obtained deviation angle always falls on: .

11. The method for visual guidance posture correction and safety control of a photovoltaic cleaning robot according to claim 8, characterized in that, In step S24, the specific processing procedure is as follows: When the IMU sensor detects that the robot is yawing and has not yet completed zero-position self-calibration, the inertial yaw angle is used. As input to the attitude loop, the first attitude deviation is constructed: , The main controller corrects the robot's current inertial yaw state based on the first attitude deviation until the IMU sensor output meets the following: , After the IMU sensor completes zero-point self-calibration, step S23, visual deviation calibration, is initiated; at this time, the visual deviation obtained in step S22 is used as the basis for the calibration. As input to the attitude loop, construct the second attitude deviation: , When the visual detection result satisfies: When the robot's overall posture has been corrected, it indicates that: , When the visual detection result satisfies: When the robot still has an overall yaw error after the IMU sensor completes zero-position self-calibration, the main controller continues to perform attitude outer loop control based on the second attitude deviation. Considering the principle of minimum turning angle when the yaw angle crosses the boundary, the attitude deviation is normalized: ， The normalized attitude deviation is input into the outer loop PID controller to obtain the steering control input: ， in: This is the steering control quantity output from the outer attitude loop; , , These are the proportional, integral, and derivative coefficients of the attitude outer loop PID, respectively.

12. The method for visual guidance posture correction and safety control of a photovoltaic cleaning robot according to claim 8, characterized in that, In step S25, the specific processing procedure is as follows: Let the robot's base forward speed be: , Let the distance between the centers of the left and right tracks be D. Then, based on the steering control quantity output from the attitude outer loop... Generate the target speed of the left and right tracks: ， ， in: , These are the target speeds for the left and right tracks, respectively. The actual speeds of the left and right tracks, as measured by the Hall sensor, are as follows: , , The speed errors of the left and right tracks are respectively: , ， The speed inner loop uses incremental PID control, and the control increment for the left track is: ， The right track control increment is: ， The outputs of the left and right motor drives are as follows: , ， in: , These are the control quantities output to the left and right DC servo motor drivers, respectively.

13. The method for visual guidance posture correction and safety control of a photovoltaic cleaning robot according to claim 1, characterized in that, In step S3, the specific processing procedure is as follows: S31: Front and rear distance information is acquired synchronously with the front image. Ultrasonic sensors installed at the front and rear of the photovoltaic cleaning robot acquire real-time distance data between the areas in front of and behind the robot; at the same time, visual sensors simultaneously collect images of the photovoltaic panel surface in front of the robot for subsequent verification and identification of abnormal areas. S32: Preliminary detection of anomalies in the front ultrasonic sensor The main controller analyzes the distance data input by the front ultrasonic sensor in real time. When the distance exceeds the set threshold, it determines that there is a suspected boundary abnormality area in front of the robot, that is, the area in front may be the gap or edge of the photovoltaic panel. At this time, the visual verification process is triggered. S33: Visual Verification Recognition When step S32 determines that there is a suspected boundary anomaly area ahead, the main controller acquires the image ahead corresponding to the current anomaly moment and inputs the image into the visual detection model to identify the photovoltaic panel edge area, inter-panel gap area and normal panel area in the image, and outputs the bounding box, category label and confidence of the corresponding target to obtain the visual recognition result at the current moment. S34: Jointly determine edges or gaps The main controller combines the abnormal state of the front ultrasonic waves, the visual recognition results, and the abnormal state of the rear ultrasonic waves to jointly determine the current abnormal area. When the front ultrasonic data is abnormal and the visual detection shows an edge, the current abnormal area is determined to be the real edge area of ​​the photovoltaic panel. When the front ultrasonic data is abnormal and the visual detection shows a photovoltaic panel in front, it is determined that the current distance abnormality is not directly caused by the real edge, but by the gap between the panels or local distance measurement disturbance, and the robot can continue to move. S35: Safety Control Execution Based on the joint judgment result of step S34, corresponding control is executed. When the current area is determined to be the actual edge area of ​​the photovoltaic panel, the robot is controlled to immediately perform deceleration, stop, reverse, or turn to avoid the edge of the photovoltaic panel. When the current area is determined to be an abnormal distance caused by a gap between panels or a normal panel surface, false alarms caused by the gap between panels are suppressed, and the robot is controlled to maintain normal driving. When the visual verification result is a gap or a normal panel surface, but the abnormal distance at the front is not resolved and the ultrasonic sensor at the rear detects the abnormal distance at the same time, the robot is controlled to immediately stop driving to achieve redundant safety protection in the case of visual misjudgment.

Images