Photovoltaic cleaning robot sensor data fusion and obstacle avoidance method and system

Abstract

The application provides a photovoltaic cleaning robot sensor data fusion and obstacle avoidance method and system, relates to the technical field of photovoltaic cleaning robots, and comprises the following steps: acquiring multi-source sensor data, adopting a Gaussian mixture model to perform probability distribution modeling, utilizing a Kalman filtering algorithm of an adaptive covariance matrix to realize data fusion, constructing a three-dimensional semantic map and labeling obstacle information, combining a global path planning and a local obstacle avoidance layer fuzzy logic controller to dynamically adjust a motion trajectory, and realizing smooth obstacle avoidance of the photovoltaic cleaning robot. The application improves the environmental perception ability and obstacle avoidance efficiency of the photovoltaic cleaning robot, reduces the collision risk, and guarantees the safety and efficiency of photovoltaic panel cleaning operation.

Description

Technical Field

[0001] This invention relates to photovoltaic cleaning robot technology, and more particularly to a method and system for sensor data fusion and obstacle avoidance in photovoltaic cleaning robots. Background Technology

[0002] With the rapid development of the photovoltaic industry and the continuous expansion of photovoltaic power plants, the cleaning and maintenance of photovoltaic panels has become a key factor affecting power generation efficiency. Traditional manual cleaning methods are not only costly and inefficient, but also pose safety hazards. Therefore, photovoltaic cleaning robots are widely used as an automated cleaning solution. However, photovoltaic power plants have complex and variable environments with a variety of obstacles, including photovoltaic supports, cables, dust accumulation areas, and various temporary obstacles. This places high demands on the sensor data fusion and obstacle avoidance capabilities of photovoltaic cleaning robots.

[0003] Currently, most photovoltaic cleaning robots on the market rely on a single sensor for environmental perception, resulting in incomplete data collection and difficulty in accurately identifying various obstacles in complex environments. In terms of sensor data processing, most employ simple data stitching or weighted averaging methods, lacking the ability to adapt to data noise and environmental changes, leading to insufficient fusion accuracy and affecting the accuracy of subsequent decisions.

[0004] Existing photovoltaic cleaning robots typically employ simple stop or detour strategies for obstacle avoidance, lacking a deep understanding of the environment and intelligent decision-making capabilities. Traditional obstacle avoidance algorithms often consider only a single factor, such as obstacle distance, while ignoring comprehensive factors such as relative speed and cleaning efficiency. This results in a rigid obstacle avoidance process, and may even lead to repeated adjustments and uneven paths, severely impacting cleaning efficiency and quality.

[0005] The unique surface characteristics of photovoltaic panels and the complexity of the installation environment make it difficult for existing obstacle avoidance technologies to balance cleaning efficiency and safety. Current technologies lack methods to effectively combine global path planning with local obstacle avoidance, resulting in slow or over-reactions when faced with sudden obstacles, which affects cleaning efficiency and may damage the photovoltaic panels. Summary of the Invention

[0006] This invention provides a method and system for sensor data fusion and obstacle avoidance in photovoltaic cleaning robots, which can solve the problems in the prior art.

[0007] A first aspect of the present invention,

[0008] Provides sensor data fusion and obstacle avoidance methods for photovoltaic cleaning robots, including:

[0009] Acquire multi-source sensor data of the photovoltaic cleaning robot, including surface image data, three-dimensional point cloud data, and attitude data;

[0010] The probability distribution of the multi-source sensor data is modeled based on the Gaussian mixture model, and the multi-source data is fused using the Kalman filter algorithm with adaptive covariance matrix. The adaptive covariance matrix is ​​updated in real time according to the noise characteristics of the multi-source sensor data and environmental changes.

[0011] A three-dimensional semantic map is constructed based on the fused multi-source sensor data, and the location, type, and size information of obstacles are marked in the three-dimensional semantic map;

[0012] Obstacle avoidance path planning is performed on the three-dimensional semantic map, which includes a global path planning layer and a local obstacle avoidance layer. The global path planning layer generates an initial path based on the Kalman filter algorithm, and the local obstacle avoidance layer uses a fuzzy logic controller to dynamically adjust the motion trajectory according to the obstacle distance, relative speed and photovoltaic panel boundary constraints to achieve smooth obstacle avoidance.

[0013] Based on the initial path achieved after smooth obstacle avoidance, the photovoltaic cleaning robot is controlled to move along the set path and detect obstacles ahead in real time. When an obstacle is detected, a local obstacle avoidance strategy is executed by combining the robot's current state and obstacle feature information.

[0014] The probability distribution of the multi-source sensor data is modeled based on a Gaussian mixture model, and the multi-source data fusion is performed using an adaptive covariance matrix Kalman filter algorithm, including:

[0015] The multi-source sensor data is constructed into a dataset, and a mixture model with multiple Gaussian components is constructed for the dataset. Each Gaussian component includes mixture weights, a mean vector, and a covariance tensor.

[0016] The parameters of the mixture model are iteratively optimized using the expectation-maximization algorithm. The expected values ​​of the latent variables are obtained by calculating the posterior probability. The mixture weights, the mean vector, and the covariance tensor are updated based on the expected values ​​of the latent variables to obtain the innovative sequence.

[0017] Based on the innovative sequence, an adaptive factor is calculated, and the adaptive factor is multiplied by the adaptive covariance matrix to obtain the corrected adaptive covariance matrix through a Kalman filter algorithm.

[0018] The Kalman filter algorithm is corrected using the corrected prediction error covariance to obtain the corrected Kalman gain; the state estimate and error covariance are updated based on the corrected Kalman gain to achieve adaptive fusion of the multi-source sensor data.

[0019] The features of the photovoltaic panel surface image data and the features of the 3D point cloud data in the multi-source sensor data are used to calculate the probability distribution. The weighting coefficients of the feature data are determined based on the results of the probability distribution calculation, including:

[0020] Multi-scale decomposition is performed on the photovoltaic panel surface image data in the multi-source sensor data to obtain image features, which include edge features, texture features, and local gradient features; geometric feature extraction is performed on the three-dimensional point cloud data in the multi-source sensor data to obtain point cloud features, which include normal vector features, curvature features, and local depth variation features.

[0021] The image probability distribution value is obtained by differential calculation based on the deviation between the image features and their mean values, and the point cloud probability distribution value is obtained by weighted calculation based on the deviation between the point cloud features and their geometric mean values.

[0022] The weighting coefficient is determined based on the ratio of the covariance between the image features and the point cloud features to the probability distribution values ​​of the image and the point cloud.

[0023] The global path planning layer generates an initial path based on the Kalman filter algorithm, and the local obstacle avoidance layer uses a fuzzy logic controller to dynamically adjust the motion trajectory according to obstacle distance, relative speed, and photovoltaic panel boundary constraints to achieve smooth obstacle avoidance, including:

[0024] Acquire historical motion trajectory data of the photovoltaic cleaning robot, and generate initial state prediction results based on the historical motion trajectory data;

[0025] The initial state prediction result is optimized using the Kalman filter algorithm to obtain the optimized state prediction result;

[0026] The optimized state prediction results are input into the path planning model. The cost value of different paths is calculated based on path length, turning angle, obstacle distance and energy consumption. The calculation process includes: multiplying each parameter by its corresponding weight coefficient and summing the results to obtain the comprehensive cost value. The path with the smallest comprehensive cost value is selected as the candidate path.

[0027] The candidate paths are smoothed, and their feasibility is evaluated based on path smoothness, safe distance, and task time. The optimal path is then selected as the global planning path.

[0028] The obstacle avoidance parameters are obtained by acquiring real-time obstacle distance information, relative speed information, and boundary constraint information around the photovoltaic cleaning robot, and by combining the obstacle distance, relative speed, and boundary distance normalized with the global planning path and then performing a weighted summation.

[0029] The speed correction value is obtained by multiplying the obstacle avoidance parameter by a preset speed adjustment coefficient, and smooth obstacle avoidance is achieved based on the obstacle avoidance parameter and the speed correction value.

[0030] An adaptive factor is introduced into the initial state prediction result, and the state transition matrix and observation matrix are dynamically adjusted according to the prediction error. The Kalman filter algorithm is then used to optimize the initial state prediction result, resulting in the following optimized state prediction result:

[0031] Obtain the system state vector, which includes position coordinates, orientation angle, linear velocity, and angular velocity. Use expectation operation to calculate the expectation value of the error product between the system state vector and the state estimation vector to obtain the prediction error covariance matrix.

[0032] The ratio of the prediction error covariance matrix to the theoretical measurement residual covariance matrix is ​​calculated to obtain the adaptive factor;

[0033] The adaptive factors are used to construct a diagonal adaptive weight matrix according to the dimension of the system state vector, and the diagonal elements of the diagonal adaptive weight matrix are the adaptive factors corresponding to different state components.

[0034] The diagonal adaptive weight matrix and the state transition matrix are multiplied to obtain the adjusted state transition matrix. The diagonal adaptive weight matrix is ​​multiplied with the difference between the current state vector and the predicted state value to obtain the prediction error compensation term.

[0035] Multiply the state vector from the previous time step by the adjusted state transition matrix and add it to the prediction error compensation term to obtain the optimized state prediction result.

[0036] Based on the initial path achieved after smooth obstacle avoidance, the photovoltaic cleaning robot is controlled to move along the set path and detect obstacles ahead in real time. When an obstacle is detected, a local obstacle avoidance strategy is executed by combining the robot's current state and obstacle feature information, including:

[0037] The initial path is used as the set path, and the photovoltaic cleaning robot is controlled to move along the set path. During the movement of the photovoltaic cleaning robot, the current position coordinates, current orientation angle and current speed are obtained in real time, and the position coordinates and envelope radius of the obstacles in front are detected in real time.

[0038] The position error is calculated based on the current position coordinates and the position coordinates of the corresponding path points of the set path, and the angle error is calculated based on the current orientation angle and the orientation angle of the corresponding path points, thus constructing a pose error vector;

[0039] Based on the pose error vector, the linear velocity control quantity is obtained by multiplying the position error with the first control gain parameter, and the angular velocity control quantity is obtained by multiplying the position error with the second control gain parameter.

[0040] The relative position vector is obtained by subtracting the position coordinates of the obstacle from the current position coordinates, and the relative distance is obtained by calculating the magnitude of the relative position vector.

[0041] Determine whether the relative distance satisfies the safety distance constraint, wherein the safety distance constraint is that the relative distance is greater than the minimum safety distance, the sum of the envelope radius and the robot radius;

[0042] When the safety distance constraint is met, the linear velocity control value and the angular velocity control value remain unchanged; when the safety distance constraint is not met, a local obstacle avoidance strategy is executed.

[0043] The position error is calculated based on the current position coordinates and the position coordinates of the corresponding path point on the set path; the angle error is calculated based on the current orientation angle and the orientation angle of the corresponding path point; and the pose error vector is constructed by:

[0044] Obtain the robot's current position coordinates and current orientation angle, and find the path point on the preset path that is closest to the current position coordinates as the corresponding path point;

[0045] Calculate the position error between the current position coordinates and the position coordinates of the corresponding path point in the robot's local coordinate system, and calculate the angle error between the current orientation angle and the orientation angle of the corresponding path point; combine the position error and the angle error to form a pose error vector.

[0046] A second aspect of the present invention,

[0047] Provides a sensor data fusion and obstacle avoidance system for photovoltaic cleaning robots, including:

[0048] The first unit is used to acquire multi-source sensor data of the photovoltaic cleaning robot, including surface image data, three-dimensional point cloud data, and attitude data.

[0049] The second unit is used to model the probability distribution of the multi-source sensor data based on the Gaussian mixture model, and to perform multi-source data fusion using the Kalman filter algorithm with adaptive covariance matrix. The adaptive covariance matrix is ​​updated in real time according to the noise characteristics of the multi-source sensor data and environmental changes.

[0050] The third unit is used to construct a three-dimensional semantic map based on the fused multi-source sensor data, and to mark the location, type and size information of obstacles in the three-dimensional semantic map;

[0051] The fourth unit is used to perform obstacle avoidance path planning on the three-dimensional semantic map. The three-dimensional semantic map includes a global path planning layer and a local obstacle avoidance layer. The global path planning layer generates an initial path based on the Kalman filter algorithm, and the local obstacle avoidance layer uses a fuzzy logic controller to dynamically adjust the motion trajectory according to the obstacle distance, relative speed and photovoltaic panel boundary constraints to achieve smooth obstacle avoidance.

[0052] The fifth unit is used to control the photovoltaic cleaning robot to move along the set path according to the initial path after achieving the smooth obstacle avoidance, and to detect obstacles in front in real time. When an obstacle is detected, the robot's current state and obstacle feature information are combined to execute a local obstacle avoidance strategy.

[0053] A third aspect of the present invention,

[0054] An electronic device is provided, comprising:

[0055] processor;

[0056] Memory used to store processor-executable instructions;

[0057] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0058] Fourth aspect of the present invention,

[0059] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0060] The beneficial effects of this application are as follows:

[0061] By employing an adaptive covariance matrix Kalman filter algorithm to fuse multi-source sensor data, high-precision perception of the environment surrounding the photovoltaic cleaning robot is achieved. This effectively reduces the limitations and uncertainties of a single sensor, improves the accuracy and robustness of environmental perception, and enables the robot to acquire more reliable environmental information in complex and ever-changing photovoltaic panel environments.

[0062] Constructing a 3D semantic map based on fused multi-source sensor data can not only accurately label the location, type, and size of obstacles, but also provide a comprehensive environmental understanding for subsequent path planning. This enhances the robot's ability to recognize its working environment, provides a solid foundation for intelligent decision-making, and effectively improves the accuracy of obstacle recognition.

[0063] By adopting a hierarchical obstacle avoidance path planning strategy, combining global path planning and the collaborative work of local obstacle avoidance layers, and using a fuzzy logic controller to dynamically adjust the motion trajectory based on obstacle distance, relative speed and photovoltaic panel boundary constraints, smooth obstacle avoidance is achieved. This not only ensures the continuity and efficiency of cleaning operations, but also effectively avoids collisions with obstacles, improving the safety and adaptability of the photovoltaic cleaning robot in complex environments. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the sensor data fusion and obstacle avoidance method for a photovoltaic cleaning robot according to an embodiment of the present invention.

[0065] Figure 2 This is a schematic diagram illustrating the sensor data processing accuracy under different anomaly rates in an embodiment of the present invention.

[0066] Figure 3 This is a schematic diagram comparing the detection accuracy under different pollution coverage rates in an embodiment of the present invention;

[0067] Figure 4 This is a flowchart illustrating the adaptive Kalman filter algorithm of an embodiment of the present invention.

[0068] Figure 5 This diagram illustrates the impact of the obstacle avoidance gain parameter K on system performance according to an embodiment of the present invention. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0071] Figure 1 This is a flowchart illustrating the sensor data fusion and obstacle avoidance method for a photovoltaic cleaning robot according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0072] Acquire multi-source sensor data of the photovoltaic cleaning robot, including surface image data, three-dimensional point cloud data, and attitude data;

[0073] The probability distribution of the multi-source sensor data is modeled based on the Gaussian mixture model, and the multi-source data is fused using the Kalman filter algorithm with adaptive covariance matrix. The adaptive covariance matrix is ​​updated in real time according to the noise characteristics of the multi-source sensor data and environmental changes.

[0074] A three-dimensional semantic map is constructed based on the fused multi-source sensor data, and the location, type, and size information of obstacles are marked in the three-dimensional semantic map;

[0075] Obstacle avoidance path planning is performed on the three-dimensional semantic map, which includes a global path planning layer and a local obstacle avoidance layer. The global path planning layer generates an initial path based on the Kalman filter algorithm, and the local obstacle avoidance layer uses a fuzzy logic controller to dynamically adjust the motion trajectory according to the obstacle distance, relative speed and photovoltaic panel boundary constraints to achieve smooth obstacle avoidance.

[0076] Based on the initial path achieved after smooth obstacle avoidance, the photovoltaic cleaning robot is controlled to move along the set path and detect obstacles ahead in real time. When an obstacle is detected, a local obstacle avoidance strategy is executed by combining the robot's current state and obstacle feature information.

[0077] In one optional implementation, the probability distribution modeling of the multi-source sensor data is performed based on a Gaussian mixture model, and the multi-source data fusion is performed using an adaptive covariance matrix Kalman filter algorithm, including:

[0078] The multi-source sensor data is constructed into a dataset, and a mixture model with multiple Gaussian components is constructed for the dataset. Each Gaussian component includes mixture weights, a mean vector, and a covariance tensor.

[0079] The parameters of the mixture model are iteratively optimized using the expectation-maximization algorithm. The expected values ​​of the latent variables are obtained by calculating the posterior probability. The mixture weights, the mean vector, and the covariance tensor are updated based on the expected values ​​of the latent variables to obtain the innovative sequence.

[0080] Based on the innovative sequence, an adaptive factor is calculated, and the adaptive factor is multiplied by the adaptive covariance matrix to obtain the corrected adaptive covariance matrix through a Kalman filter algorithm.

[0081] The Kalman filter algorithm is corrected using the corrected prediction error covariance to obtain the corrected Kalman gain; the state estimate and error covariance are updated based on the corrected Kalman gain to achieve adaptive fusion of the multi-source sensor data.

[0082] In this embodiment, a fusion method based on Gaussian mixture model and adaptive covariance matrix Kalman filtering is proposed to address the problem of multi-source sensor data fusion. This method models the probability distribution of multi-source sensor data using a Gaussian mixture model and then uses an adaptive covariance matrix Kalman filtering algorithm to achieve accurate fusion of the multi-source data.

[0083] Preprocessing and constructing data from multiple sensors. Assuming data from different sensors includes information such as position, velocity, and acceleration, this data is synchronized according to timestamps and constructed into a unified dataset. This dataset can be represented as D = {x1, x2, ..., x...} n}, where each data point x□ is a multidimensional vector containing various measurement values ​​collected by the sensor.

[0084] For the dataset described above, a mixture model with K Gaussian components is constructed. Each Gaussian component includes three key parameters: mixture weights π_k, mean vector μ_k, and covariance tensor Σ_k. The mixture weights represent the importance of the Gaussian component in the overall model, satisfying that the sum of all weights is 1. The mean vector represents the center of the data distribution, and the covariance tensor describes the dispersion of the data and the correlation between different dimensions.

[0085] In practical applications, the appropriate number of Gaussian components can be selected based on the data complexity. For example, for a multi-sensor fusion system for UAVs, 3-5 Gaussian components can be selected to model data from GPS, IMU, vision, and other sensors. Initially, the data can be grouped using the K-means clustering method, and the initial mean vector and covariance tensor can be calculated based on each group. The mixing weights can be set according to the proportion of data points in each group.

[0086] To optimize the parameters of the mixture model, an iterative Expectation-Maximization (EM) algorithm is employed. This algorithm consists of an expectation step (E-step) and a maximization step (M-step). In the E-step, the posterior probability of each data point belonging to each Gaussian component is calculated, which is the expected value of the latent variable z. For a data point x□, its posterior probability belonging to the k-th Gaussian component can be calculated using the current parameters.

[0087] In the M-step, the parameters of the mixture model are updated based on the calculated posterior probabilities. The updated mixture weights are the average posterior probabilities of each data point belonging to that component; the updated mean vector is the weighted average of all data points according to their posterior probabilities; and the updated covariance tensor takes into account the deviation between the data points and the mean vector. Through multiple iterations of the E-step and M-step, the parameters of the mixture model gradually converge to their optimal values.

[0088] After completing the Gaussian mixture model, the innovation sequence of the data is obtained. The innovation sequence represents the difference between the actual observations and the model predictions, and is a key indicator in Kalman filtering. Based on the statistical characteristics of the innovation sequence, an adaptive factor can be calculated. This adaptive factor reflects the degree of matching between the current model and the actual observations; the value of the adaptive factor increases when the deviation is large, and decreases when the deviation is small.

[0089] For example, if a certain UAV system integrates GPS and IMU data, and the innovative sequence shows large fluctuations, it indicates that there may be GPS signal interference in the current environment. In this case, the adaptive factor will be increased accordingly to reduce the dependence on GPS information and enhance the weight of IMU data in the fusion result.

[0090] An adaptive mechanism is introduced based on the traditional Kalman filter algorithm. The calculated adaptive factor is multiplied by the covariance matrix to obtain the corrected adaptive covariance matrix. This correction process takes into account the real-time characteristics of the data, enabling the filtering algorithm to dynamically adjust the confidence level for different sensor data.

[0091] For example, when data from a certain sensor exhibits abnormal fluctuations, its corresponding covariance component will be appropriately amplified to reduce the sensor's influence on the fusion result. In practice, reasonable upper and lower thresholds can be set to ensure that the adaptive factor fluctuates within an effective range, avoiding over-adjustment that could lead to system instability.

[0092] The Kalman filter algorithm is corrected using a modified prediction error covariance. In traditional Kalman filtering, the gain matrix determines the degree of adoption of observational information. By introducing an adaptive covariance matrix, a corrected Kalman gain is obtained. This gain can more reasonably balance prior information and new observation data, improving the accuracy of state estimation.

[0093] The state estimate and error covariance are updated based on the corrected Kalman gain. The update of the state estimate comprehensively considers both prior predictions and observational information, while the update of the error covariance reflects the uncertainty of the estimation results. Through this process, adaptive fusion of multi-source sensor data is achieved.

[0094] In practical applications, this method has been successfully applied to multi-sensor fusion in intelligent driving systems. In one test, the system fused data from LiDAR, millimeter-wave radar, and cameras. When the vehicle was driving in a tunnel, the quality of camera data deteriorated due to changes in lighting conditions. The system automatically increased the adaptive factor corresponding to the camera data, while also increasing the reliance on LiDAR and millimeter-wave radar data, ensuring the accuracy of position estimation and keeping the positioning error within 10 centimeters.

[0095] The core advantage of this method lies in its ability to adaptively adjust the fusion strategy based on data characteristics, exhibiting strong robustness to anomalous data. Furthermore, the introduction of a Gaussian mixture model enables the method to handle non-Gaussian distributed sensor data, expanding its application scenarios. Through an adaptive covariance matrix Kalman filtering algorithm, accurate fusion of multi-source sensor data is achieved, improving system performance.

[0096] Figure 2 This is a schematic diagram illustrating the sensor data processing accuracy under different anomaly rates in an embodiment of the present invention:

[0097] This graph compares the processing accuracy of three different filtering schemes under varying proportions of abnormal sensor data. The horizontal axis represents the proportion of abnormal sensor data, ranging from 5% to 50%; the vertical axis represents the processing accuracy as a percentage. The graph contains three curves representing the performance of our proposed solution (triangle marker), standard Kalman filter (circle marker), and particle filter (square marker). When the proportion of abnormal sensor data is 5%, all three schemes achieve high processing accuracy: our proposed solution approximately 98%, standard Kalman filter approximately 95%, and particle filter approximately 93%. As the proportion of abnormal data increases, the performance of all three schemes declines, but at different rates. When the proportion of abnormal data reaches 50%, our proposed solution still maintains approximately 87% accuracy, while standard Kalman filter drops to approximately 59%, and particle filter drops to approximately 55%. Overall, our proposed solution exhibits the strongest robustness, with the slowest rate of accuracy decline and the flattest curve as the proportion of abnormal data increases. The performance degradation of standard Kalman filter and particle filter is more significant, especially after the proportion of abnormal data exceeds 25%, where the accuracy reduction is more pronounced. This demonstrates that the technical solution has significant advantages in handling high proportions of abnormal data and is more suitable for applications in scenarios where sensors may generate a large amount of abnormal data.

[0098] In one optional implementation, the features of the photovoltaic panel surface image data and the features of the 3D point cloud data in the multi-source sensor data are used to calculate a probability distribution, and the weighting coefficients of the feature data are determined based on the result of the probability distribution calculation, including:

[0099] Multi-scale decomposition is performed on the photovoltaic panel surface image data in the multi-source sensor data to obtain image features, which include edge features, texture features, and local gradient features; geometric feature extraction is performed on the three-dimensional point cloud data in the multi-source sensor data to obtain point cloud features, which include normal vector features, curvature features, and local depth variation features.

[0100] The image probability distribution value is obtained by differential calculation based on the deviation between the image features and their mean values, and the point cloud probability distribution value is obtained by weighted calculation based on the deviation between the point cloud features and their geometric mean values.

[0101] The weighting coefficient is determined based on the ratio of the covariance between the image features and the point cloud features to the probability distribution values ​​of the image and the point cloud.

[0102] Edge Feature Extraction: The Sobel operator is used to process the image, extracting edge information in the horizontal and vertical directions. The gradient magnitude of each pixel is calculated, and points with gradient magnitudes greater than a preset threshold of 25 are marked as edge points. For a 512×512 pixel photovoltaic panel image, approximately 2000-3000 effective edge feature points can typically be extracted. Texture Feature Extraction: The Local Binary Pattern (LBP) algorithm is applied, using 8×8 pixel blocks as the basic unit, to statistically analyze the grayscale variation patterns of pixels within each block. For each block, the grayscale difference between the center pixel and its 8 surrounding pixels is calculated, forming an 8-bit binary code, which is then converted into a decimal value as the texture feature value of that block. In standard test samples, the texture feature value typically ranges from 0 to 255. Local Gradient Feature Extraction: A gradient direction histogram is calculated within a 16×16 pixel sliding window. The 360-degree gradient direction is divided into 8 equal intervals, and the frequency of gradient occurrence within each interval is statistically analyzed, forming an 8-dimensional feature vector. For minor defects on the surface of photovoltaic panels, local gradient features typically exhibit significant frequency anomalies within the 45-135 degree range.

[0103] The features extracted at the three scales were normalized and merged to form a complete feature set of the photovoltaic panel surface image.

[0104] Normal vector feature extraction: For each point, select its 30 nearest neighbors, fit a local plane using principal component analysis, and calculate the normal vector. The normal vector is represented by a three-dimensional unit vector, pointing outwards from the point cloud. For flat areas, the normal vector direction is highly consistent; for defective areas, the normal vector direction changes significantly. Curvature feature extraction: Calculate the local curvature of each point based on the aforementioned neighborhood points. Specifically, this is obtained by calculating the ratio of the average distance from the neighborhood point to the fitted plane to the neighborhood radius. In practical applications, the curvature value of a normal photovoltaic panel surface is usually below 0.001, while the curvature value of a defective area can reach 0.01-0.05. Local depth variation features: Calculate the standard deviation of the target point and its neighborhood points along the z-axis (depth) direction. For minor damage to photovoltaic panels, the local depth variation is usually within the range of 0.5-2 mm; for severe damage, the variation can reach more than 5 mm.

[0105] After feature extraction is completed, all feature values ​​are standardized so that they are distributed in the [0,1] interval, which facilitates subsequent fusion processing.

[0106] Calculate the mean of each type of image feature. For example, for edge features, calculate the average gradient magnitude of all extracted edge points; for texture features, calculate the average texture feature value of all blocks; for local gradient features, calculate the average gradient frequency over eight directional intervals. Calculate the deviation of each feature from its mean. Taking edge features as an example, if the gradient magnitude of a point is 75 and the mean is 50, the deviation is 25. Normalize the deviation and apply a Gaussian function to convert it into a probability value. The larger the deviation, the lower the probability value, indicating a higher probability of an anomaly at that feature point. Combine the probability values ​​of the three types of features to obtain the final image probability distribution. In practical applications, the merging weights for edge features, texture features, and local gradient features are set to 0.4, 0.3, and 0.3, respectively.

[0107] Calculate the mean values ​​of various point cloud geometric features. For example, the mean value of normal vector features is obtained by calculating the average direction of all normal vectors; the mean value of curvature features is the average of the curvature values ​​of all points; and the mean value of depth variation features is the average of the local depth standard deviations of all points. Calculate the weighted deviation of each feature from its mean. For normal vector features, calculate the angle between the current normal vector and the mean normal vector; for curvature and depth variation features, directly calculate the difference from the mean. Based on the importance of the three types of geometric features, assign weight coefficients of 0.35, 0.35, and 0.3, and calculate the sum of weighted deviations. Apply an exponential decay function to convert the weighted deviations into probability values, obtaining the point cloud probability distribution.

[0108] Calculate the covariance between image features and point cloud features. Covariance calculation is based on the pairing of corresponding feature points, determining the correspondence through the spatial location of the feature points. For a 512×512 resolution image and point cloud data containing approximately 100,000 points, typically about 5,000-8,000 effective feature points can be paired. Calculate the ratio of the image probability distribution value to the point cloud probability distribution value. Ideally, if both types of features have equal descriptive power for the target region, the ratio should be close to 1; if the image features are more accurate, the ratio should be greater than 1; if the point cloud features are more accurate, the ratio should be less than 1. Determine the final weighting coefficients based on the covariance and the probability distribution ratio. A larger covariance indicates higher consistency between the two types of features; the degree to which the probability distribution ratio deviates from 1 reflects the difference in accuracy between the two types of features.

[0109] In practical applications, for the detection of defects on the surface of photovoltaic panels, when detecting edge damage, the image feature weight coefficient is usually set to 0.65-0.75, and the point cloud feature weight coefficient is 0.25-0.35; when detecting small depressions on the surface, the image feature weight coefficient is usually set to 0.4-0.5, and the point cloud feature weight coefficient is 0.5-0.6.

[0110] The weighting coefficients determined by the above method enable effective fusion of photovoltaic panel surface image data and 3D point cloud data, improving the accuracy and reliability of defect detection. For a standard 1.5m × 1m photovoltaic panel, this method can detect surface defects as small as 2mm in diameter, with an accuracy rate exceeding 95%.

[0111] Figure 3 This is a schematic diagram comparing the detection accuracy under different pollution coverage rates in embodiments of the present invention:

[0112] This graph compares the detection accuracy of three different technologies under varying pollution coverage conditions. The horizontal axis represents pollution coverage, ranging from 5% to 50%; the vertical axis represents detection accuracy as a percentage. The graph contains three curves representing the performance of this technology (triangle markers), area image features (circle markers), and point cloud features only (square markers), respectively. At an initial pollution coverage of 5%, the detection accuracies of the three technologies are: approximately 97.5% for this technology, approximately 96% for area image features, and approximately 91% for point cloud features only. As the pollution coverage increases, the performance curves of all three technologies show an initial upward trend followed by a downward trend, but the peak positions and magnitudes differ. This technology reaches its highest accuracy of approximately 99% at a pollution coverage of 25%, area image features peak at approximately 97% at 15%, and point cloud features only peak at approximately 95% at 25%. When the pollution coverage rate continued to increase to 50%, the performance of the three schemes all declined to varying degrees: the performance of this scheme dropped to approximately 96.5%, still maintaining a high level; the performance of area image features dropped to approximately 90.2%; and the performance of point cloud features alone dropped to approximately 91%. Overall, this scheme demonstrated the best performance at all pollution coverage rates, and its performance curve was more stable, indicating that the scheme has better robustness and adaptability, especially in maintaining a high detection accuracy under high pollution coverage conditions.

[0113] In one optional implementation, the global path planning layer generates an initial path based on the Kalman filter algorithm, and the local obstacle avoidance layer uses a fuzzy logic controller to dynamically adjust the motion trajectory according to obstacle distance, relative speed, and photovoltaic panel boundary constraints to achieve smooth obstacle avoidance, including:

[0114] Acquire historical motion trajectory data of the photovoltaic cleaning robot, and generate initial state prediction results based on the historical motion trajectory data;

[0115] The initial state prediction result is optimized using the Kalman filter algorithm to obtain the optimized state prediction result;

[0116] The optimized state prediction results are input into the path planning model. The cost value of different paths is calculated based on path length, turning angle, obstacle distance and energy consumption. The calculation process includes: multiplying each parameter by its corresponding weight coefficient and summing the results to obtain the comprehensive cost value. The path with the smallest comprehensive cost value is selected as the candidate path.

[0117] The candidate paths are smoothed, and their feasibility is evaluated based on path smoothness, safe distance, and task time. The optimal path is then selected as the global planning path.

[0118] The obstacle avoidance parameters are obtained by acquiring real-time obstacle distance information, relative speed information, and boundary constraint information around the photovoltaic cleaning robot, and by combining the obstacle distance, relative speed, and boundary distance normalized with the global planning path and then performing a weighted summation.

[0119] The speed correction value is obtained by multiplying the obstacle avoidance parameter by a preset speed adjustment coefficient, and smooth obstacle avoidance is achieved based on the obstacle avoidance parameter and the speed correction value.

[0120] Acquire historical motion trajectory data of the photovoltaic cleaning robot. This data includes the robot's position, velocity, acceleration, and angular velocity over a past period. For example, the system can collect the robot's motion state over the last 10 seconds, recording data every 100 milliseconds to form a historical trajectory containing 100 data points. This historical data is used to predict the robot's initial state, including its current position coordinates, direction of movement, and velocity.

[0121] The Kalman filter algorithm is used to optimize the initial state prediction results. This algorithm, based on the robot's motion and observation models, continuously optimizes the state estimate through an iterative prediction-update process. Specifically, the system first calculates the predicted state value for the current moment based on historical data, and then corrects the prediction result by incorporating actual sensor measurements. For example, when the historical trajectory shows the robot moving northeast at a speed of 0.5 m / s, the Kalman filter predicts the next position based on this trend and corrects the predicted position using measured data from GPS or a visual positioning system, ultimately obtaining a more accurate state estimate.

[0122] After inputting the optimized state prediction results into the path planning model, the system begins to calculate the cost of different paths. Path cost calculation considers four key factors: path length, turning angle, obstacle distance, and energy consumption. The system assigns a weight coefficient to each factor; for example, path length has a weight of 0.3, turning angle has a weight of 0.2, obstacle distance has a weight of 0.4, and energy consumption has a weight of 0.1. For each possible path, the system standardizes each parameter, multiplies it by its corresponding weight, and sums the results to obtain the comprehensive cost. For example, a path 10 meters long (standardized value 0.5), with an average turning angle of 15 degrees (standardized value 0.3), a minimum obstacle distance of 1.5 meters (standardized value 0.7), and an estimated energy consumption of 2 kilojoules (standardized value 0.4), has a comprehensive cost of 0.5 × 0.3 + 0.3 × 0.2 + 0.7 × 0.4 + 0.4 × 0.1 = 0.51. The system compares the cost of all paths and selects the path with the lowest cost as the candidate path.

[0123] Smoothing candidate paths is a crucial step in ensuring stable robot motion. The system uses Bézier curve interpolation to smooth path inflection points, replacing sharp turns in the original path with smooth curves. The smoothed path is evaluated based on three criteria: path smoothness (measured by the rate of change of curvature, with a threshold set at 0.2 / meter), safe distance (maintaining a distance of at least 0.8 meters from obstacles), and task time (not exceeding 1.2 times the preset cleaning time). If a candidate path does not meet these conditions, the system selects the path with the second lowest cost for evaluation until the optimal path that meets the criteria is found as the global planning path.

[0124] During actual operation, the system acquires obstacle information around the photovoltaic cleaning robot in real time. This information comes from lidar, ultrasonic sensors, or vision systems, including obstacle distance (measurement range 0-5 meters, accuracy ±2 centimeters), relative speed (measurement range ±2 meters / second, accuracy ±0.1 meters / second), and photovoltaic panel boundary constraints (distance from the edge not less than 0.2 meters). The system processes this information in conjunction with global path planning. First, it normalizes the obstacle distance, relative speed, and boundary distance to a range of 0-1. Then, the system assigns weights to these three parameters (obstacle distance 0.5, relative speed 0.3, boundary constraint 0.2), and obtains the obstacle avoidance parameters through weighted summation.

[0125] For example, when an obstacle is detected 2 meters ahead with a relative speed of 0.5 m / s (the obstacle is approaching) and a distance of 0.5 meters from the edge of the photovoltaic panel, the normalized parameters are as follows: obstacle distance 0.4 (2 / 5 = 0.4), relative speed 0.75 (higher value due to proximity), and boundary constraint 0.6 (0.5 / 0.83 = 0.6, assuming the maximum boundary distance is 0.83 meters). The weighted obstacle avoidance parameters are 0.4 × 0.5 + 0.75 × 0.3 + 0.6 × 0.2 = 0.53.

[0126] Multiplying the obstacle avoidance parameter by a preset speed adjustment coefficient (usually set to 1.5) yields a speed correction value of 0.795. Based on the magnitude of the obstacle avoidance parameter and the speed correction value, the system dynamically adjusts the robot's trajectory: when the obstacle avoidance parameter is less than 0.3, the robot maintains its original speed and direction; when the obstacle avoidance parameter is between 0.3 and 0.7, the robot decelerates and fine-tunes its direction to increase the distance to the obstacle; when the obstacle avoidance parameter is greater than 0.7, the robot significantly decelerates (down to a maximum of 30% of normal speed) and makes a large-angle turn (up to 45 degrees) to avoid the obstacle.

[0127] In this way, the photovoltaic cleaning robot can dynamically adjust its movement trajectory based on real-time environmental information while maintaining the overall direction of global path planning, thus achieving smooth obstacle avoidance and effectively avoiding collisions with obstacles. At the same time, it maintains the continuity and stability of the movement trajectory, improving the efficiency and safety of cleaning operations.

[0128] In one optional implementation, an adaptive factor is introduced into the initial state prediction result, and the state transition matrix and observation matrix are dynamically adjusted according to the prediction error. A Kalman filter algorithm is then used to optimize the initial state prediction result, resulting in the optimized state prediction result, which includes:

[0129] Obtain the system state vector, which includes position coordinates, orientation angle, linear velocity, and angular velocity. Use expectation operation to calculate the expectation value of the error product between the system state vector and the state estimation vector to obtain the prediction error covariance matrix.

[0130] The ratio of the prediction error covariance matrix to the theoretical measurement residual covariance matrix is ​​calculated to obtain the adaptive factor;

[0131] The adaptive factors are used to construct a diagonal adaptive weight matrix according to the dimension of the system state vector, and the diagonal elements of the diagonal adaptive weight matrix are the adaptive factors corresponding to different state components.

[0132] The diagonal adaptive weight matrix and the state transition matrix are multiplied to obtain the adjusted state transition matrix. The diagonal adaptive weight matrix is ​​multiplied with the difference between the current state vector and the predicted state value to obtain the prediction error compensation term.

[0133] Multiply the state vector from the previous time step by the adjusted state transition matrix and add it to the prediction error compensation term to obtain the optimized state prediction result.

[0134] Obtain the initial state prediction results. These results are based on a traditional prediction model and include information such as position coordinates, orientation angle, linear velocity, and angular velocity. For example, for a mobile robot, its initial state might be: position coordinates (10.2 m, 15.7 m), orientation angle 45 degrees, linear velocity 1.5 m / s, and angular velocity 0.2 radians / s. These initial prediction values ​​may contain some errors and require further optimization.

[0135] Obtain the system state vector, which includes position coordinates, orientation angle, linear velocity, and angular velocity. Taking a mobile robot as an example, the system state vector can be represented as [x, y, θ, v, ω], where x and y are position coordinates, θ is the orientation angle, v is the linear velocity, and ω is the angular velocity. Assume the system state vector at the current moment is [10.5 m, 15.9 m, 46 degrees, 1.55 m / s, 0.22 radians / s].

[0136] The prediction error covariance matrix is ​​obtained by calculating the expected value of the product of the errors between the system state vector and the state estimation vector through expectation operation. Specifically, the system first calculates the difference between the system state vector and the state estimation vector, then calculates the product of these differences, and finally takes the expected value to obtain the prediction error covariance matrix. Taking the mobile robot mentioned above as an example, if the state estimation vector is [10.2 m, 15.7 m, 45 degrees, 1.5 m / s, 0.2 radians / s], and the difference is [0.3 m, 0.2 m, 1 degree, 0.05 m / s, 0.02 radians / s], then the prediction error covariance matrix can be calculated.

[0137] Obtain the theoretical measurement residual covariance matrix, which reflects the noise level during the measurement process. For a mobile robot, assuming a position measurement accuracy of ±0.1 meters, an angle measurement accuracy of ±0.5 degrees, a velocity measurement accuracy of ±0.05 meters / second, and an angular velocity measurement accuracy of ±0.01 radians / second, the theoretical measurement residual covariance matrix can be constructed accordingly.

[0138] The adaptation factor is obtained by calculating the ratio of the prediction error covariance matrix to the theoretical measurement residual covariance matrix. The ratio is calculated element-wise, that is, each element in the prediction error covariance matrix is ​​divided by the corresponding element in the theoretical measurement residual covariance matrix. For example, the adaptation factor for position x might be 3.0, for position y it might be 2.0, for the orientation angle it might be 2.0, for the linear velocity it might be 1.0, and for the angular velocity it might be 2.0.

[0139] The adaptive factors are used to construct a diagonal adaptive weight matrix according to the dimension of the system state vector. The diagonal elements of this matrix are the adaptive factors corresponding to different state components. For the mobile robot mentioned above, the diagonal elements of the diagonal adaptive weight matrix are [3.0, 2.0, 2.0, 1.0, 2.0], representing the adaptive weights of position x, position y, orientation angle, linear velocity, and angular velocity.

[0140] The adjusted state transition matrix is ​​obtained by multiplying the diagonal adaptive weight matrix with the state transition matrix using matrix multiplication. The state transition matrix describes the transition of the system from the previous state to the current state. For example, for a uniform motion model, the state transition matrix may include a time interval term. Assuming the time interval is 0.1 seconds, the adjusted state transition matrix will reflect the influence of the adaptive weights of each state variable on the transition process.

[0141] Calculate the difference between the current state vector and the predicted state value, for example, [0.3 m, 0.2 m, 1 degree, 0.05 m / s, 0.02 radians / s]. Then, perform matrix multiplication on the diagonal adaptive weight matrix and this difference to obtain the prediction error compensation term, such as [0.9 m, 0.4 m, 2 degrees, 0.05 m / s, 0.04 radians / s].

[0142] The state vector from the previous moment is multiplied on the left by the adjusted state transition matrix, and then added to the prediction error compensation term to obtain the optimized state prediction result. For example, if the state vector from the previous moment is [10.0 m, 15.5 m, 44 degrees, 1.5 m / s, 0.2 radians / s], the adjusted state transition matrix might yield the basic prediction value [10.15 m, 15.65 m, 44.5 degrees, 1.5 m / s, 0.2 radians / s]. After adding the prediction error compensation term, the final optimized state prediction result is [11.05 m, 16.05 m, 46.5 degrees, 1.55 m / s, 0.24 radians / s].

[0143] This adaptive Kalman filtering method effectively improves the accuracy of state prediction by dynamically adjusting the state transition matrix and introducing a prediction error compensation term, making it particularly suitable for scenarios with high uncertainty in state changes. In practical applications, the method can be adjusted according to the specific characteristics of the system to obtain the best prediction results.

[0144] This method has been successfully applied to the navigation systems of various mobile robots. For example, in an unmanned transport vehicle project, after adopting this method, the position prediction error was reduced from an average of 0.5 meters to 0.15 meters, and the orientation angle prediction error was reduced from an average of 2 degrees to 0.6 degrees, significantly improving the accuracy and reliability of the transport task.

[0145] Figure 4 Here is a flowchart illustrating the adaptive Kalman filter algorithm of this invention:

[0146] This diagram illustrates a flowchart of system state prediction and optimization, primarily describing the implementation steps of the adaptive Kalman filter algorithm. The system acquires state vectors, including key parameters such as position coordinates, orientation angle, linear velocity, and angular velocity. The expected value of the error product between these system state vectors and the state estimation vector is calculated through expectation operations, thus obtaining the prediction error covariance matrix. Next, the system calculates the ratio of the prediction error covariance matrix to the theoretical measurement residual covariance matrix, thereby obtaining the adaptive factor. A diagonal adaptive weight matrix is ​​constructed based on the dimensions of the system state vectors, where the diagonal elements are the adaptive factors corresponding to different state components. Based on this, the diagonal adaptive weight matrix is ​​multiplied by the state transition matrix to calculate the adjusted state transition matrix. Simultaneously, the diagonal adaptive weight matrix is ​​multiplied by the difference between the current state vector and the predicted state value to obtain the prediction error compensation term. The state vector from the previous time step is left-multiplied by the adjusted state transition matrix and added to the prediction error compensation term, finally yielding the optimized state prediction result. This complete process demonstrates the characteristic of the adaptive Kalman filter algorithm to improve prediction accuracy through dynamic adjustment.

[0147] In one optional implementation, based on the initial path after achieving smooth obstacle avoidance, the photovoltaic cleaning robot is controlled to move along a set path and obstacles ahead are detected in real time. When an obstacle is detected, a local obstacle avoidance strategy is executed by combining the robot's current state and obstacle feature information, including:

[0148] The initial path is used as the set path, and the photovoltaic cleaning robot is controlled to move along the set path. During the movement of the photovoltaic cleaning robot, the current position coordinates, current orientation angle and current speed are obtained in real time, and the position coordinates and envelope radius of the obstacles in front are detected in real time.

[0149] The position error is calculated based on the current position coordinates and the position coordinates of the corresponding path points of the set path, and the angle error is calculated based on the current orientation angle and the orientation angle of the corresponding path points, thus constructing a pose error vector;

[0150] Based on the pose error vector, the linear velocity control quantity is obtained by multiplying the position error with the first control gain parameter, and the angular velocity control quantity is obtained by multiplying the position error with the second control gain parameter.

[0151] The relative position vector is obtained by subtracting the position coordinates of the obstacle from the current position coordinates, and the relative distance is obtained by calculating the magnitude of the relative position vector.

[0152] Determine whether the relative distance satisfies the safety distance constraint, wherein the safety distance constraint is that the relative distance is greater than the minimum safety distance, the sum of the envelope radius and the robot radius;

[0153] When the safety distance constraint is met, the linear velocity control value and the angular velocity control value remain unchanged; when the safety distance constraint is not met, a local obstacle avoidance strategy is executed.

[0154] After smoothly avoiding obstacles, the robot moves along the set path and detects obstacles in front of it in real time. When an obstacle is detected, the robot executes a local obstacle avoidance strategy by combining the robot's current state and the obstacle's characteristic information.

[0155] In practical applications, an initial path is used as the set path, and the photovoltaic cleaning robot is controlled to move along this set path. The initial path is obtained through offline planning, taking into account the layout characteristics of the photovoltaic array and known fixed obstacles, such as brackets and junction boxes. This initial path is represented by a series of path points, each containing position coordinates (x, y) and orientation angle θ.

[0156] During the movement of the photovoltaic cleaning robot, a combination of onboard sensors acquires the robot's current position coordinates (x current, y current), current orientation angle θ current, and current velocity v current in real time. Position information can be obtained via GPS, RTK, or visual odometry; the orientation angle can be measured via IMU or electronic compass; and the velocity can be calculated by an encoder. Simultaneously, detection devices such as LiDAR, ultrasonic sensors, or depth cameras detect the position coordinates (x obstacle, y obstacle) and the obstacle envelope radius r in real time. The envelope radius represents the size of the obstacle and is estimated using the detected obstacle point cloud or contour.

[0157] To maintain the robot's movement along the set path, pose error needs to be calculated and controlled. The position error is calculated based on the coordinates of the current position and the corresponding path point. The corresponding path point can be the closest point on the set path to the current position, or a preview point (a path point a certain distance ahead of the current position). The position error *eposition* is represented as the Euclidean distance between the current position coordinates and the corresponding path point coordinates. Simultaneously, the angle error *eangle* is calculated based on the current orientation angle and the orientation angle of the corresponding path point. The angle error is the difference between the two angles and is normalized to the range [-π, π]. The position error and angle error together form the pose error vector [eposition, eangle].

[0158] Based on the pose error vector, the linear velocity control command v is obtained by multiplying the position error by the first control gain parameter K_position, and the angular velocity control command ω is obtained by multiplying the angle error by the second control gain parameter K_angle. In a specific embodiment, K_position is set to 0.5 and K_angle is set to 1.2. When the position error is large, the robot will decelerate and adjust its orientation; when the angle error is large, the robot will increase its steering force to quickly adjust its orientation.

[0159] The relative position vector [x_obstacle - x_current, y_obstacle - y_current] is obtained by subtracting the obstacle's position coordinates from the current position coordinates. The relative distance d_relative is then calculated by determining the magnitude of this relative position vector. The relative distance represents the distance from the robot's center to the obstacle's center.

[0160] Determine if the relative distance satisfies the safe distance constraint. The safe distance constraint is defined as the sum of the relative distance *drelative* (the minimum safe distance *dsafe*), the obstacle envelope radius *robstacle*, and the robot radius *rrobot*. In a specific example, *dsafe* is set to 0.3 meters, the photovoltaic cleaning robot radius *rrobot* is 0.4 meters, and the detected obstacle envelope radius *robstacle* is 0.2 meters. Then the safe distance constraint is *drelative* > (0.3 + 0.2 + 0.4) = 0.9 meters.

[0161] When the safe distance constraint is met, the linear velocity control command v and the angular velocity control command ω remain unchanged, and the robot continues to travel along the set path. When the safe distance constraint is not met, a local obstacle avoidance strategy is executed.

[0162] The local obstacle avoidance strategy employs a variant of the artificial potential field method, superimposing obstacle avoidance control variables on top of the original control variables. The obstacle avoidance control variables are inversely proportional to the relative distance and related to the direction of the relative position vector. Specifically, the obstacle avoidance linear velocity control variable *v_obstacle_avoidance* is the negative value of *K_obstacle_avoidance* multiplied by (1 / d_relative - 1 / (d_safe + r_obstacle + r_robot)) multiplied by the projection of the unit vector of the relative position vector onto the robot's forward direction, where *K_obstacle_avoidance* is the obstacle avoidance gain parameter, set to 0.8. The obstacle avoidance angular velocity control variable *ω_obstacle_avoidance* is the projection of the unit vector of the relative position vector onto the robot's left side, multiplied by *K_rotation*, where *K_rotation* is set to 1.5.

[0163] The control commands are v command final = v command + v obstacle avoidance and ω command final = ω command + ω obstacle avoidance. This method enables the robot to gradually decelerate and circumvent obstacles when approaching them, and resume traveling along the set path after moving away from them.

[0164] If the robot deviates too far from the set path (e.g., more than 1.5 meters), a path replanning mechanism will be triggered to generate a new local path to return to the original set path. The replanning uses the Rapid Exploration Random Tree (RRT) algorithm, which considers the current position and obstacle distribution to plan a collision-free path back to the appropriate point on the original set path.

[0165] To handle situations with multiple obstacles, an obstacle avoidance control variable is calculated for each detected obstacle, and the vector sum of these variables is taken as the final obstacle avoidance control variable. This method can handle multi-obstacle scenarios in complex environments.

[0166] Actual tests show that in a standard photovoltaic array environment, this method enables the photovoltaic cleaning robot to travel at an average speed of 0.5 m / s, successfully avoiding various obstacles, including temporarily placed tools and falling objects. The maximum distance deviating from the set path does not exceed 0.8 meters, the path tracking accuracy is over 90%, and the obstacle avoidance success rate reaches 98%.

[0167] Compared to existing technologies, traditional obstacle avoidance methods for photovoltaic cleaning robots typically employ simple stop-and-go strategies or preset obstacle avoidance patterns, lacking smoothness and adaptability. These methods often require a complete stop when encountering obstacles before executing predefined obstacle avoidance actions, resulting in discontinuous movement, low efficiency, and difficulty in adapting to complex and changing obstacle environments.

[0168] Figure 5 This is a schematic diagram illustrating the impact of the obstacle avoidance gain parameter K on system performance according to an embodiment of the present invention.

[0169] This graph shows the relationship between three different indicators and the obstacle avoidance gain parameter K. The horizontal axis represents the range of K from 0.2 to 2.0, the vertical axis represents accuracy as a percentage on the left and speed as a percentage on the right. The graph contains three curves: path tracking accuracy (triangle marker), obstacle avoidance success rate (circle marker), and average speed (square marker). As the value of K increases, the obstacle avoidance success rate (circle) gradually rises from around 85%, reaching a peak of approximately 99% between K = 1.2 and 1.6, and then slightly decreases. Path tracking accuracy (triangle) shows a similar trend, starting to rise from around 82%, reaching a peak of approximately 97% at K = 1.2, and then slowly decreasing. The average speed (square) shows a continuous downward trend, decreasing from an initial 0.55 m / s to around 0.32 m / s. Overall, it can be observed that the system performance is optimal when the K value is around 1.2, at which point both obstacle avoidance success rate and path tracking accuracy reach high levels, while maintaining a moderate speed. This indicates that the choice of obstacle avoidance gain parameter K requires a trade-off between safety (obstacle avoidance success rate), accuracy (tracking precision), and efficiency (driving speed).

[0170] The method proposed in this application improves upon the following aspects: First, it adopts a continuous control strategy, organically combining path tracking control and obstacle avoidance control, enabling the robot to smoothly avoid obstacles without needing to stop completely; second, it introduces safety distance constraints based on obstacle characteristics, making obstacle avoidance behavior more reasonable; and third, it designs an adaptive control parameter adjustment mechanism that can dynamically adjust the obstacle avoidance intensity according to the obstacle distance. These improvements enable the photovoltaic cleaning robot to complete cleaning tasks more intelligently and efficiently, reducing time losses caused by obstacle avoidance, improving overall operational efficiency, and ensuring the safety of equipment and photovoltaic panels.

[0171] In one optional implementation, the position error is calculated based on the current position coordinates and the position coordinates of the corresponding path point on the set path, and the angle error is calculated based on the current orientation angle and the orientation angle of the corresponding path point. Constructing the pose error vector includes:

[0172] Obtain the robot's current position coordinates and current orientation angle, and find the path point on the preset path that is closest to the current position coordinates as the corresponding path point;

[0173] Calculate the position error between the current position coordinates and the position coordinates of the corresponding path point in the robot's local coordinate system, and calculate the angle error between the current orientation angle and the orientation angle of the corresponding path point; combine the position error and the angle error to form a pose error vector.

[0174] The position error is calculated based on the current position coordinates and the position coordinates of the corresponding path points on the set path. Then, the angle error is calculated based on the current orientation angle and the orientation angle of the corresponding path points. Finally, the pose error vector is constructed.

[0175] The photovoltaic cleaning robot is equipped with multiple positioning sensors, including a GPS receiver, an inertial measurement unit, and an encoder. Data from these sensors is fused using a Kalman filter to obtain the robot's current position coordinates (x-current, y-current) and current orientation angle θ in the global coordinate system. The position coordinate accuracy is ±2 cm, and the orientation angle accuracy is ±1 degree.

[0176] The path is generated by the path planning module and represented as a series of discrete path points. Each path point contains three key pieces of information: x-coordinate, y-coordinate, and orientation angle. For example, the i-th path point can be represented as (x-path i, y-path i, θ-path i). In practical applications, the distance between adjacent path points is usually set to 10 centimeters to ensure path smoothness and control precision.

[0177] Find the path point on the preset path that is closest to the current position coordinates and use it as the corresponding path point. The specific steps are: traverse all path points on the preset path, calculate the Euclidean distance from each path point to the current position, and select the path point with the smallest distance as the corresponding path point.

[0178] Assuming a path contains N path points, for each path point (x path i, y path i), calculate the distance d_i between it and the current position (x current, y current), i.e., d_i is equal to sqrt((x path i x current)^2 + (y path i y current)^2), where sqrt represents the square root. Find the index j that minimizes d_i; the corresponding path point is (x path j, y path j, θ path j).

[0179] A local search strategy is employed. Based on the robot's movement speed and control cycle, a search window can be defined, searching only within a certain range near the previous corresponding path point. For example, if the robot's maximum speed is 0.5 m / s and the control cycle is 0.1 sec, the search window can be set to 10 points before and after the previous corresponding path point.

[0180] Assuming the current location coordinates are (2.35, 1.47), the nearest path point is calculated to be the 56th point, with coordinates (2.31, 1.52) and an orientation angle of 0.78 radians. This point is the corresponding path point.

[0181] Calculate the position and angle errors. Since robot motion control is typically performed in the robot's local coordinate system, it is necessary to transform the position error from the global coordinate system to the robot's local coordinate system. The origin of the robot's local coordinate system is located at the robot's center, the x-axis is along the robot's forward direction, and the y-axis is perpendicular to the x-axis and points to the left.

[0182] Calculate the position error vector in the global coordinate system, which is the coordinates of the corresponding path point minus the current position coordinates, to obtain (x path jx current, y path jy current); perform coordinate system transformation to transform the vector from the global coordinate system to the robot's local coordinate system.

[0183] Coordinate system transformation requires the use of a rotation matrix, which is constructed based on the robot's current orientation angle θ. Specifically, let the position error vector in the global coordinate system be (global x-error, global y-error), then the position error in the robot's local coordinate system is (local x-error, local y-error), calculated as follows: local x-error equals global x-error multiplied by cos(θ_current) plus global y-error multiplied by sin(θ_current); local y-error equals -global x-error multiplied by sin(θ_current) plus global y-error multiplied by cos(θ_current).

[0184] The position error vector in the global coordinate system is (-0.04, 0.05). Assuming the current orientation angle is 0.7 radians, the position error in the robot's local coordinate system is (0.006, 0.063), which means that the robot needs to move forward 0.006 meters and to the left 0.063 meters to return to the preset path.

[0185] Calculating the angle error is relatively simple; just subtract the current orientation angle from the orientation angle of the corresponding path point. That is, the angle error θ equals θ_path_j minus θ_current. To ensure that the angle error is within the range of [-π, π], normalization is required: if the angle error θ is greater than π, subtract 2π; if the angle error θ is less than -π, add 2π.

[0186] The orientation angle of the corresponding path point is 0.78 radians, and the current orientation angle is 0.7 radians. Therefore, the angle error is 0.08 radians, which means that the robot needs to rotate counterclockwise by 0.08 radians to match the orientation of the preset path.

[0187] The position error and angle error are combined into a pose error vector, i.e., [x-error local, y-error local, θ-angle error]. In the above example, the pose error vector is [0.006, 0.063, 0.08]. This vector will serve as the input to the subsequent path tracking controller, used to generate control commands to make the robot move along a preset path.

[0188] Use lookup tables or spatial index structures to accelerate the search for the nearest path point. For example, using a KD-tree data structure can reduce the search complexity from O(n) to O(log n), where n is the number of path points. For long paths, this optimization can significantly reduce computation time.

[0189] Traditional methods for calculating pose error typically use the error in the global coordinate system directly, or simply take the vertical distance between the current position and the path as the position error. These methods suffer from insufficient accuracy and unstable control when dealing with complex paths or high-speed motion.

[0190] The improvements in this application are as follows: First, the position error calculation in the robot's local coordinate system is introduced, which is more in line with the actual needs of robot motion control; second, the nearest path point is used as a reference instead of a simple vertical projection point, which can better handle path segments with large curvature; and third, the angle error is normalized to avoid the jump problem when the angle crosses the ±π boundary.

[0191] These improvements make the construction of the pose error vector more accurate and stable, providing more reliable input information for the path tracking control of the photovoltaic cleaning robot. Practical applications show that the pose error vector constructed using this method can improve the robot's path tracking accuracy by more than 30%, especially in the tracking performance during turns and speed changes, which is significantly better than traditional methods, greatly improving the efficiency and quality of photovoltaic cleaning operations.

[0192] A second aspect of the present invention,

[0193] Provides a sensor data fusion and obstacle avoidance system for photovoltaic cleaning robots, including:

[0194] The first unit is used to acquire multi-source sensor data of the photovoltaic cleaning robot, including surface image data, three-dimensional point cloud data, and attitude data.

[0195] The second unit is used to model the probability distribution of the multi-source sensor data based on the Gaussian mixture model, and to perform multi-source data fusion using the Kalman filter algorithm with adaptive covariance matrix. The adaptive covariance matrix is ​​updated in real time according to the noise characteristics of the multi-source sensor data and environmental changes.

[0196] The third unit is used to construct a three-dimensional semantic map based on the fused multi-source sensor data, and to mark the location, type and size information of obstacles in the three-dimensional semantic map;

[0197] The fourth unit is used to perform obstacle avoidance path planning on the three-dimensional semantic map. The three-dimensional semantic map includes a global path planning layer and a local obstacle avoidance layer. The global path planning layer generates an initial path based on the Kalman filter algorithm, and the local obstacle avoidance layer uses a fuzzy logic controller to dynamically adjust the motion trajectory according to the obstacle distance, relative speed and photovoltaic panel boundary constraints to achieve smooth obstacle avoidance.

[0198] The fifth unit is used to control the photovoltaic cleaning robot to move along the set path according to the initial path after achieving the smooth obstacle avoidance, and to detect obstacles in front in real time. When an obstacle is detected, the robot's current state and obstacle feature information are combined to execute a local obstacle avoidance strategy.

[0199] A third aspect of the present invention,

[0200] An electronic device is provided, comprising:

[0201] processor;

[0202] Memory used to store processor-executable instructions;

[0203] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0204] Fourth aspect of the present invention,

[0205] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0206] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A photovoltaic cleaning robot sensor data fusion and obstacle avoidance method, characterized in that, The application relates to a photovoltaic panel cleaning robot and a method for adaptive fusion of multi-source sensor data. The method comprises the following steps: acquiring multi-source sensor data of a photovoltaic panel cleaning robot, wherein the multi-source sensor data comprises surface image data, three-dimensional point cloud data and attitude data; modeling the multi-source sensor data based on a Gaussian mixture model and performing multi-source data fusion by using a Kalman filtering algorithm with an adaptive covariance matrix, which comprises the following steps: constructing the multi-source sensor data into a data set and constructing a mixture model with multiple Gaussian components for the data set, wherein each Gaussian component comprises a mixture weight, a mean vector and a covariance tensor; iteratively optimizing the parameters of the mixture model by using an expectation maximization algorithm, obtaining the expected value of the hidden variable by calculating the posterior probability, updating the mixture weight, the mean vector and the covariance tensor based on the expected value of the hidden variable, and obtaining an innovation sequence; calculating an adaptive factor based on the innovation sequence, multiplying the adaptive factor by an adaptive covariance matrix to obtain a modified adaptive covariance matrix by using a Kalman filtering algorithm; correcting the Kalman filtering algorithm by using the modified prediction error covariance to obtain a corrected Kalman gain, updating the state estimation value and the error covariance based on the corrected Kalman gain, and realizing adaptive fusion of the multi-source sensor data, wherein the photovoltaic panel surface image data in the multi-source sensor data is subjected to multi-scale decomposition to obtain image features, the image features comprising edge features, texture features and local gradient features; the three-dimensional point cloud data in the multi-source sensor data is subjected to geometric feature extraction to obtain point cloud features, the point cloud features comprising normal vector features, curvature features and local depth variation features; obtaining image probability distribution values by differentiating the image features from their feature mean values, and obtaining point cloud probability distribution values by weighting the point cloud features from their geometric feature mean values; determining a weight coefficient according to the covariance between the image features and the point cloud features and the ratio of the image probability distribution values and the point cloud probability distribution values, wherein the adaptive covariance matrix is updated in real time according to the noise characteristics and environmental changes of the multi-source sensor data; constructing a three-dimensional semantic map based on the fused multi-source sensor data, and labeling the position, type and size information of obstacles in the three-dimensional semantic map; performing obstacle avoidance path planning on the three-dimensional semantic map, wherein the three-dimensional semantic map comprises a global path planning layer and a local obstacle avoidance layer, the global path planning layer generates an initial path based on the Kalman filtering algorithm, and the local obstacle avoidance layer dynamically adjusts a motion trajectory according to obstacle distance, relative speed and photovoltaic panel boundary constraints by using a fuzzy logic controller to realize smooth obstacle avoidance; controlling the photovoltaic panel cleaning robot to move along a set path according to the initial path after realizing the smooth obstacle avoidance, and detecting obstacles in front in real time, wherein when an obstacle is detected, a local obstacle avoidance strategy is executed in combination with the current state of the robot and the characteristic information of the obstacle.

2. The method of claim 1, wherein, The global path planning layer generates an initial path based on the Kalman filtering algorithm, and the local obstacle avoidance layer dynamically adjusts the motion trajectory according to the obstacle distance, relative speed and photovoltaic panel boundary constraint by using a fuzzy logic controller to achieve smooth obstacle avoidance, including: Obtain historical motion trajectory data of the photovoltaic cleaning robot, and generate an initial state prediction result based on the historical motion trajectory data; Optimize the initial state prediction result by using a Kalman filtering algorithm to obtain an optimized state prediction result; Input the optimized state prediction result into a path planning model, calculate the generation value of different paths according to the path length, turning angle, obstacle distance and energy consumption, and the calculation process includes: multiplying each parameter by the corresponding weight coefficient and summing to obtain a comprehensive generation value, and selecting the path with the minimum comprehensive generation value as the candidate path; Smooth the candidate path, evaluate the feasibility of the candidate path according to the path smoothness, safety distance and task time, and select the optimal path as the global planning path; Obtain real-time obstacle distance information, relative speed information and boundary constraint information around the photovoltaic cleaning robot, normalize the obstacle distance, relative speed and boundary distance after combining the global planning path, and then perform weighted summation to obtain obstacle avoidance parameters; Multiply the obstacle avoidance parameters by a preset speed adjustment coefficient to obtain a speed correction value, and realize smooth obstacle avoidance based on the obstacle avoidance parameters and the speed correction value.

3. The method of claim 2, wherein, Introduce an adaptive factor to the initial state prediction result, dynamically adjust the state transition matrix and the observation matrix according to the prediction error, optimize the initial state prediction result by using a Kalman filtering algorithm, and obtain an optimized state prediction result, including: Obtain a system state vector, the system state vector includes position coordinates, orientation angle, linear velocity and angular velocity, calculate the expected value of the error product of the system state vector and the state estimation vector by using expectation operation, and obtain a prediction error covariance matrix; Calculate the ratio of the prediction error covariance matrix and the theoretical measurement residual covariance matrix to obtain an adaptive factor; Construct a diagonal adaptive weight matrix according to the dimension of the system state vector, the diagonal elements of the diagonal adaptive weight matrix are adaptive factors corresponding to different state components; Perform matrix multiplication operation on the diagonal adaptive weight matrix and the state transition matrix to obtain an adjusted state transition matrix, and perform matrix multiplication operation on the diagonal adaptive weight matrix and the difference between the current time state vector and the state prediction value to obtain a prediction error compensation term; Multiply the state vector at the last time by the adjusted state transition matrix, and add the prediction error compensation term to obtain an optimized state prediction result.

4. The method of claim 1, wherein, According to the initial path after realizing the smooth obstacle avoidance, control the photovoltaic cleaning robot to move along the set path, and detect the front obstacle in real time, when detecting the obstacle, combine the current state of the robot and the obstacle feature information, and execute a local obstacle avoidance strategy, including: The initial path is taken as a set path, and the photovoltaic cleaning robot is controlled to move along the set path; during the movement of the photovoltaic cleaning robot, the current position coordinates, the current orientation angle and the current speed are acquired in real time, and the position coordinates and the envelope radius of the front obstacle are detected in real time; A position error is calculated according to the current position coordinates and the position coordinates of a corresponding path point of the set path, an angle error is calculated according to the current orientation angle and the orientation angle of the corresponding path point, and a pose error vector is constructed; Based on the pose error vector, a linear velocity control quantity is obtained from the product of the position error and a first control gain parameter, and an angular velocity control quantity is obtained from the product of the position error and a second control gain parameter; The position coordinates of the obstacle are subtracted from the current position coordinates to obtain a relative position vector, and a relative distance is calculated by taking the modulus of the relative position vector; It is judged whether the relative distance satisfies a safety distance constraint, and the safety distance constraint is that the relative distance is greater than the sum of a minimum safety distance, the envelope radius and a robot radius; When the safety distance constraint is satisfied, the linear velocity control quantity and the angular velocity control quantity are kept unchanged; when the safety distance constraint is not satisfied, a local obstacle avoidance strategy is executed.

5. The method of claim 4, wherein, The pose error vector is constructed by calculating a position error according to the current position coordinates and the position coordinates of a corresponding path point of the set path, and calculating an angle error according to the current orientation angle and the orientation angle of the corresponding path point, and includes: The current position coordinates and the current orientation angle of the robot are acquired, and a path point closest to the current position coordinates on a preset path is taken as a corresponding path point; The position error of the current position coordinates and the position coordinates of the corresponding path point in a robot local coordinate system is calculated, and the angle error is calculated by taking the included angle between the current orientation angle and the orientation angle of the corresponding path point; the position error and the angle error are combined to form a pose error vector.

6. Photovoltaic cleaning robot sensor data fusion and obstacle avoidance system for implementing the method according to any one of the preceding claims 1-5, characterized in that, It includes: The first unit is configured to acquire multi-source sensor data of the photovoltaic cleaning robot, and the multi-source sensor data includes surface image data, three-dimensional point cloud data and attitude data; The second unit is configured to model a probability distribution of the multi-source sensor data based on a Gaussian mixture model, and perform multi-source data fusion by using a Kalman filtering algorithm with an adaptive covariance matrix, wherein the adaptive covariance matrix is updated in real time according to noise characteristics of the multi-source sensor data and environmental changes; The third unit is configured to construct a three-dimensional semantic map according to the fused multi-source sensor data, and mark position, type and size information of obstacles in the three-dimensional semantic map; The fourth unit is configured to plan an obstacle avoidance path for the three-dimensional semantic map, and the three-dimensional semantic map includes a global path planning layer and a local obstacle avoidance layer, wherein the global path planning layer generates an initial path based on the Kalman filtering algorithm, and the local obstacle avoidance layer dynamically adjusts a movement trajectory according to obstacle distance, relative speed and photovoltaic panel boundary constraints by using a fuzzy logic controller, to realize smooth obstacle avoidance. A fifth unit is configured to control the photovoltaic cleaning robot to move along the set path according to the initial path after the smoothing and obstacle avoidance, and to detect obstacles in front in real time, and when an obstacle is detected, to execute a local obstacle avoidance strategy in combination with the current state of the robot and the characteristic information of the obstacle.

7. An electronic device, comprising: Comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the method of any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1 to 5.

Images