Posture control method and device of photovoltaic cleaning robot

Abstract

A pose control method and device of a photovoltaic cleaning robot are disclosed. The method comprises: acquiring distance measurement values of a plurality of ranging sensors and joint state information of a mechanical arm; determining overall distance deviation information and pose deviation information based on the plurality of distance measurement values; constructing a current state vector based on the plurality of distance measurement values, the overall distance deviation information, the pose deviation information, and the joint state information; inputting the current state vector into a preset pose control strategy network to determine a joint angular velocity control instruction of the mechanical arm, wherein the preset pose control strategy network is a continuous action control network trained based on reinforcement learning; and controlling the mechanical arm to move based on the joint angular velocity control instruction to adjust an end pose of a roller brush end effector relative to a photovoltaic module surface, so that the roller brush end effector maintains a target relative pose relative to the photovoltaic module surface. The application can improve the accuracy of pose control of the photovoltaic cleaning robot in a complex environment.

Description

Technical Field

[0001] This application relates to the field of photovoltaic cleaning robot control technology, and in particular to a posture control method and device for a photovoltaic cleaning robot. Background Technology

[0002] With the rapid development of the photovoltaic industry, the demand for cleaning and maintenance of photovoltaic modules in large-scale photovoltaic power plants is increasing. Photovoltaic modules are exposed to the outdoor environment for extended periods, and their surfaces are prone to accumulating dust, sand, and other pollutants. If these are not removed promptly, they will affect the light reception efficiency of the photovoltaic modules, thereby reducing power generation efficiency. However, manual cleaning of photovoltaic modules is labor-intensive and inefficient. Fixed spraying methods are heavily reliant on water resources, and reciprocating scrubbing relies on fixed tracks, making it difficult to adapt to the complex working conditions of photovoltaic power plants, such as uneven ground, varying module tilt angles, and module installation errors. To improve the automation level of photovoltaic module cleaning, existing robotic arm cleaning equipment uses a robotic arm to drive a roller brush end effector to clean the surface of the photovoltaic modules. However, in actual operation, changes in the attitude of the moving chassis, changes in the tilt angle of the photovoltaic module surface, uneven ground, and changes in the relative attitude between the roller brush and the module surface can cause the roller brush to struggle to maintain a suitable contact distance and parallel state. Therefore, how to enable photovoltaic cleaning robots to adaptively adjust the attitude of the roller brush end effector in complex working environments to maintain a stable target relative posture with the photovoltaic module surface is a problem worthy of attention. Summary of the Invention

[0003] In view of this, embodiments of this application provide a posture control method and apparatus for a photovoltaic cleaning robot, aiming to improve the accuracy of posture control of the photovoltaic cleaning robot in complex environments. A first aspect provides a posture control method for a photovoltaic cleaning robot, the photovoltaic cleaning robot including a robotic arm with multiple joints, a roller brush end effector disposed at the end of the robotic arm, and multiple ranging sensors disposed on the roller brush end effector, wherein the multiple ranging sensors are respectively arranged at different ranging positions of the roller brush end effector. The posture control method includes: acquiring distance measurement values ​​collected by the multiple ranging sensors respectively, the distance measurement values ​​being used to characterize the relative distance between the roller brush end effector and the surface of the photovoltaic module at the corresponding ranging position; acquiring joint state information of the robotic arm, the joint state information including the joint angle and joint angular velocity of each joint; acquiring the roller brush end effector... The system collects overall distance and attitude deviation information between the actuator and the photovoltaic module surface; constructs the current state vector of the robotic arm based on multiple distance measurements, overall distance deviation information, attitude deviation information, and joint state information; inputs the current state vector into a preset attitude control strategy network to determine the joint angular velocity control command of the robotic arm, wherein the preset attitude control strategy network is a continuous motion control network trained based on context-aware domain reinforcement learning; controls the movement of the robotic arm based on the joint angular velocity control command to adjust the end attitude of the brush end effector relative to the photovoltaic module surface, so that the brush end effector maintains the target relative pose with respect to the photovoltaic module surface.

[0004] Optionally, obtaining overall distance deviation information and attitude deviation information between the end effector of the roller brush and the surface of the photovoltaic module includes: determining overall distance deviation information and attitude deviation information between the end effector of the roller brush and the surface of the photovoltaic module based on multiple distance measurements, wherein the attitude deviation information includes a first attitude deviation determined based on the difference between distance measurements at different distance measurement positions in a first direction, and a second attitude deviation determined based on the difference between distance measurements at different distance measurement positions in a second direction.

[0005] Optionally, based on multiple distance measurements, the overall distance deviation information and attitude deviation information between the brush end effector and the photovoltaic module surface are determined, including: performing median filtering on the distance measurements collected by each ranging sensor at multiple consecutive sampling times to determine the corresponding median filtering result; performing Kalman filtering on the median filtering result to determine multiple filtered distance values; determining the overall distance deviation information based on the difference between the average of the multiple filtered distance values ​​and the target distance; determining the first attitude deviation based on the difference between the filtered distance values ​​at different ranging positions in the first direction; and determining the second attitude deviation based on the difference between the filtered distance values ​​at different ranging positions in the second direction.

[0006] Optionally, based on multiple distance measurements, overall distance deviation information, posture deviation information, and joint state information, a current state vector of the robotic arm is constructed, including: normalizing multiple distance measurements to determine distance state information; normalizing the joint angles of each joint to determine joint angle state information; normalizing the joint angular velocities of each joint to determine joint angular velocity state information; and constructing the current state vector of the robotic arm based on the distance state information, overall distance deviation information, posture deviation information, joint angle state information, and joint angular velocity state information.

[0007] Optionally, the joint angular velocity control commands include angular velocity commands corresponding to each joint of the robotic arm; inputting the current state vector into a preset attitude control strategy network to determine the joint angular velocity control commands of the robotic arm includes: inputting the current state vector into the preset attitude control strategy network to obtain a continuous motion vector; and performing amplitude limiting processing on the continuous motion vector based on a preset angular velocity range to determine the angular velocity commands corresponding to each joint.

[0008] Optionally, the preset attitude control strategy network includes an actor network and a critic network. The actor network is configured to output continuous action vectors based on the current state vector. The critic network is configured to determine the state-action value corresponding to the current state vector and the continuous action vector based on the current state vector and the continuous action vector. During the training of the preset attitude control strategy network, the network parameters of the actor network and the critic network are updated based on a reward function, which includes distance maintenance reward, attitude alignment reward, stability control reward, and efficiency reward. Among them, the distance maintenance reward is determined based on the distance deviation between the end effector of the brush and the surface of the photovoltaic module, the attitude alignment reward is determined based on the attitude deviation information, the stability control reward is determined based on the action amplitude of the continuous action vector, and the efficiency reward is determined based on whether the overall distance deviation information and attitude deviation information meet the preset error conditions.

[0009] Optionally, the actor network includes a context encoder and a master policy network. The context encoder is configured to generate a context vector based on historical state vectors, historical continuous action vectors, and the state vector at the next moment within a preset historical time period. The context vector is used to characterize the dynamic characteristics of the current working environment. The current state vector is input into a preset attitude control policy network to obtain continuous action vectors, including: obtaining the context vector output by the context encoder; concatenating the current state vector and the context vector to obtain an extended state vector; and inputting the extended state vector into the master policy network to obtain continuous action vectors. The training of the context encoder and the master policy network includes: simulation environment training and real environment parameter updates. In the simulation environment training, the parameters of the photovoltaic module and sensor parameters are updated. At least one of the parameters and environmental parameters is randomized, and the context encoder and main policy network are trained based on the randomized simulation environment. In the real environment parameter update, the network parameters obtained by training in the simulation environment are transferred to the photovoltaic cleaning robot, while keeping the network parameters of the context encoder and some network parameters of the main policy network unchanged. Based on the training data collected in the real working environment, the network parameters of at least one fully connected layer in the main policy network near the output end are updated. Among them, the photovoltaic module parameters include at least one of the following: photovoltaic module tilt angle, photovoltaic module radius of curvature and surface friction coefficient; the sensor parameters include at least one of the following: measurement delay, measurement noise and data loss rate; and the environmental parameters include at least one of the following: ground unevenness and base vibration amplitude.

[0010] Optionally, the preset attitude control strategy network is trained in an increasing manner, including: a basic task training phase, a perturbation task training phase, and an extreme task training phase. In the basic task training phase, the preset attitude control strategy network is trained based on a first working condition, which includes: the photovoltaic module is placed horizontally, there is no uneven ground, and the sensor noise is less than a preset noise threshold. In the perturbation task training phase, the preset attitude control strategy network is trained based on a second working condition, which introduces at least one perturbation factor among the photovoltaic module tilt angle, ground unevenness, and sensor noise compared to the first working condition. In the extreme task training phase, the preset attitude control strategy network is trained based on a third working condition, where the perturbation range of the perturbation factor in the third working condition is greater than that in the second working condition. When the success rate of the preset attitude control strategy network in multiple consecutive training rounds of the current training phase meets the target success rate threshold of the corresponding training phase, the next training phase begins.

[0011] Optionally, the robotic arm is a six-degree-of-freedom robotic arm, including a base rotation joint, a shoulder joint, an elbow joint, a wrist pitch joint, a wrist flip joint, and an end effector rotation joint; multiple ranging sensors include four ultrasonic ranging sensors, which are respectively set at different ranging positions of the brush end effector, and the four ranging positions are arranged at intervals in the first direction and the second direction.

[0012] Secondly, a posture control device for a photovoltaic cleaning robot is provided. The photovoltaic cleaning robot includes a robotic arm with multiple joints, a roller brush end effector disposed at the end of the robotic arm, and multiple ranging sensors disposed on the roller brush end effector. The multiple ranging sensors are respectively arranged at different ranging positions on the roller brush end effector. The posture control device includes: a distance acquisition unit for acquiring distance measurement values ​​collected by the multiple ranging sensors, the distance measurement values ​​representing the relative distance between the roller brush end effector and the surface of the photovoltaic module at the corresponding ranging position; a state acquisition unit for acquiring joint state information of the robotic arm, including joint angles and joint angular velocities of each joint; and a state acquisition unit for acquiring the joint state information of the roller brush end effector. The system includes: an overall distance deviation information and an attitude deviation information between the manipulator and the photovoltaic module surface; a construction unit for constructing a current state vector based on multiple distance measurements, overall distance deviation information, attitude deviation information, and joint state information; a control command determination unit for inputting the current state vector into a preset attitude control strategy network to determine the joint angular velocity control commands of the manipulator, wherein the preset attitude control strategy network is a continuous motion control network trained based on context-aware domain reinforcement learning; and an adjustment unit for controlling the movement of the manipulator based on the joint angular velocity control commands to adjust the end effector attitude of the brush end effector relative to the photovoltaic module surface, so that the brush end effector maintains the target relative pose with respect to the photovoltaic module surface.

[0013] Compared with the prior art, the above-described technical solution based on the concept of this application can achieve the following beneficial effects: (1) This application utilizes a deep reinforcement learning algorithm, enabling the robotic arm of the cleaning robot to autonomously learn control strategies to adapt to different working conditions without requiring manual adjustment of control parameters. Faced with uncertainties such as changes in the tilt angle of photovoltaic modules, uneven ground, and brush wear, the system can automatically adjust its control strategy to maintain good cleaning performance. Furthermore, it offers a fast response speed. The reinforcement learning strategy employs a continuous motion space design, allowing the controller to directly output smooth joint speed commands, avoiding delays caused by discrete motion switching.

[0014] (2) This application provides rich distance information through multiple ranging sensors. Combined with the powerful feature extraction capability of deep neural networks, the system can accurately perceive the relative attitude between the end effector of the roller brush and the photovoltaic module, and achieve high-precision position control.

[0015] (3) This application effectively suppresses abrupt changes in the joint motion of the robotic arm through the stable control term in the reward function, making the robotic arm move smoothly and reducing mechanical shock and wear. At the same time, the reinforcement learning strategy has strong anti-interference ability and can still operate stably even when there is a certain amount of measurement noise in the ranging sensor. Attached Figure Description

[0016] The following is a brief introduction to the accompanying drawings used in the description of the embodiments of this application: Figure 1 A flowchart illustrating the attitude control method of a photovoltaic cleaning robot provided in some embodiments of this application is shown. Figure 2 This document illustrates a flowchart of a method for determining overall distance deviation information and attitude deviation information between a brush end effector and the surface of a photovoltaic module, provided in some embodiments of this application. Figure 3 A flowchart illustrating a method for determining a continuous action vector provided in some embodiments of this application is shown. Figure 4 A schematic diagram of the posture control device for a photovoltaic cleaning robot provided in some embodiments of this application is shown. Detailed Implementation

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, examples of implementation methods of this application will be described below with reference to the accompanying drawings. The accompanying drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without creative effort. Adjustments and improvements made without departing from the concept of this application are all within the protection scope of this application.

[0018] To keep the drawings simple, each figure only schematically shows the parts relevant to the embodiment, and they do not represent the actual structure of the product. In addition, for the sake of clarity and ease of understanding, some figures only schematically show parts of components with the same structure or function, and there may actually be more or fewer components with the same structure or function.

[0019] In this application, unless otherwise expressly specified and limited, ordinal numbers, such as "first" and "second," are used only to distinguish and describe related objects, and should not be construed as indicating or implying the relative importance or order between related objects; furthermore, they do not represent the quantity of related objects. "Multiple" includes two or more, and other quantifiers are similar.

[0020] Photovoltaic module cleaning robots are crucial equipment in the automated operation and maintenance systems of large-scale photovoltaic power plants. They primarily move between photovoltaic arrays via a mobile chassis carrying a cleaning mechanism, using roller brushes or spray systems to remove dust, sand, and other deposits from the photovoltaic module surfaces. In existing engineering applications, manual cleaning is inefficient, labor-intensive, and ill-suited to the continuous maintenance requirements of large-scale photovoltaic power plants. Fixed spray systems rely on water resources, limiting their application in water-scarce areas or large-scale power plants. Reciprocating scrubbing robots typically operate on fixed tracks or preset paths, resulting in relatively fixed movement trajectories and cleaning postures. When photovoltaic modules exhibit differences in installation angle, localized deformation, variations in surface height, or uneven ground, it is difficult to adapt to the actual posture of the module surface in real time, easily leading to insufficient cleaning in certain areas or abnormal cleaning pressure.

[0021] To address these issues, some existing equipment uses robotic arms to drive brush end effectors for cleaning photovoltaic (PV) modules. The robotic arm offers high degrees of freedom, allowing for adjustments to the position and orientation of the brush end effector to accommodate PV modules of varying heights, tilt angles, or arrangements. However, variations in the installation tilt angle of PV modules, the flatness of their surfaces, and the height differences between module arrays can occur. Furthermore, the mobile chassis supporting the robotic arm is affected by uneven ground and track vibrations during operation, causing continuous changes in the posture of the robotic arm base and the brush end effector. If the brush end effector only moves along a preset trajectory, it's difficult to ensure it maintains a suitable relative distance and parallelism with the PV module surface throughout the cleaning process. If the distance between the brush and the PV module surface is too large, the brush cannot fully contact the contaminants, leading to incomplete cleaning; if the distance is too small, the brush may exert excessive force on the module surface, increasing the risk of scratching. In view of this, this application proposes a posture control method and device for a photovoltaic cleaning robot, aiming to achieve adaptive adjustment of the posture of the end effector of the roller brush under complex operating conditions such as changes in the tilt angle of photovoltaic modules, uneven ground, sensor noise, and chassis vibration, thereby improving the distance maintenance capability, posture alignment capability, and motion stability during the cleaning process of photovoltaic modules.

[0022] The following description is in conjunction with the accompanying drawings: Figure 1 The diagram illustrates a flowchart of an attitude control method for a photovoltaic cleaning robot according to some embodiments of this application. The photovoltaic cleaning robot includes a robotic arm with multiple joints, a roller brush end effector disposed at the end of the robotic arm, and multiple ranging sensors disposed on the roller brush end effector. The multiple ranging sensors are respectively arranged at different ranging positions on the roller brush end effector. The attitude control method includes: S110: Acquire distance measurement values ​​collected by multiple ranging sensors respectively. The distance measurement values ​​are used to characterize the relative distance between the brush end effector and the surface of the photovoltaic module at the corresponding ranging position. S120: Obtain the joint status information of the robotic arm, including the joint angle and joint angular velocity of each joint; S130: Obtain overall distance deviation information and attitude deviation information between the end effector of the roller brush and the surface of the photovoltaic module; S140: Construct the current state vector of the robotic arm based on multiple distance measurements, overall distance deviation information, posture deviation information, and joint state information; S150: Input the current state vector into the preset attitude control strategy network to determine the joint angular velocity control command of the robotic arm. The preset attitude control strategy network is a continuous motion control network trained based on context-aware domain reinforcement learning. S160: Controls the movement of the robotic arm based on joint angular velocity control commands to adjust the end posture of the brush end effector relative to the photovoltaic module surface, so that the brush end effector maintains the target relative posture with respect to the photovoltaic module surface.

[0023] In the above embodiments, the robotic arm of the photovoltaic cleaning robot can be used to drive the end effector of the roller brush to adjust its position and attitude in space, enabling the roller brush end effector to clean the surface of the photovoltaic module. Multiple ranging sensors can be arranged at different ranging positions on the roller brush end effector to sense the relative distance between the roller brush end effector and the photovoltaic module surface from multiple locations. The ranging position can be the installation location of the ranging sensors on the roller brush end effector. Since the roller brush end effector often has a certain length and width, by setting multiple ranging sensors at different ranging positions, the distance differences between the roller brush end effector and the photovoltaic module surface at different locations can be obtained, thus providing a data basis for determining whether the roller brush end effector is too close, too far, or tilted relative to the photovoltaic module surface. During actual cleaning, the photovoltaic module may have an installation tilt angle and local height differences. Real-time distance measurements from multiple ranging sensors allow the robot control unit to promptly perceive the relative changes between the roller brush end effector and the photovoltaic module surface, providing input for the adaptive attitude adjustment of the robotic arm.

[0024] Joint angles characterize the current form of the robotic arm, while joint angular velocities characterize its current motion state. Since the brush end effector is located at the end of the robotic arm, the angles and angular velocities of each joint directly affect its spatial position, orientation, and subsequent motion trend. Therefore, when performing attitude control, the robotic arm's own joint state information can be incorporated. This allows the control strategy to understand not only the relative distance between the end effector and the photovoltaic module but also the current motion structure and state of the robotic arm. Using joint state information as input to the attitude control process improves the continuity and stability of the robotic arm's attitude adjustment.

[0025] Overall distance deviation information can be used to characterize the degree to which the brush end effector is generally too close or too far from the photovoltaic module surface. For example, if the overall distance measurements at multiple ranging positions are greater than the expected distance, it indicates that the brush end effector may be generally far from the photovoltaic module surface, potentially resulting in insufficient cleaning. Conversely, if the overall distance measurements at multiple ranging positions are less than the expected distance, it indicates that the brush end effector may be generally too close to the photovoltaic module surface, potentially increasing the risk of pressure or scratches on the module surface. Attitude deviation information can be used to characterize the degree of tilt or non-parallelism of the brush end effector relative to the photovoltaic module surface.

[0026] The current state vector is a data set representing the current control state of the photovoltaic cleaning robot. It includes distance measurements collected by the ranging sensor, overall distance and posture deviation information determined based on these measurements, and the joint angles and angular velocities of the robotic arm. By constructing the current state vector, the relative pose between the end effector of the brush and the surface of the photovoltaic module, the current configuration of the robotic arm, and the current motion trend of the robotic arm can be uniformly expressed as input data suitable for the control strategy. The preset posture control strategy network is a pre-trained neural network model used for posture control decisions. For example, through context-aware reinforcement learning, it utilizes the interaction between state, action, and feedback reward to enable the control strategy to learn the control actions to be taken in different states during training, thus forming a continuous motion control network. In this embodiment, the joint angular velocity control command can be used to control the movement speed of each joint of the robotic arm. Therefore, using a continuous motion control network is beneficial for outputting smooth joint movement commands and reducing robotic arm jitter or impact caused by discontinuous action switching. After the current state vector is input into the preset attitude control strategy network, the network can output joint angular velocity control commands corresponding to each joint of the robotic arm based on the distance measurement value, overall distance deviation, attitude deviation, and joint state reflected by the current state vector. These joint angular velocity control commands can be used to indicate the direction and speed of movement of each joint of the robotic arm within the current control cycle, causing the end effector of the robotic arm to move in a direction that reduces overall distance and attitude deviation. The continuous motion control network trained using reinforcement learning can learn the mapping relationship between complex states and control actions during the training process, thereby better adapting to complex factors such as changes in photovoltaic module attitude, chassis attitude fluctuations, and the nonlinear motion characteristics of the robotic arm.

[0027] The target relative pose can include the desired relative distance and relative orientation between the brush end effector and the photovoltaic module surface. This target relative pose is not limited to a single distance value between the brush end effector and the photovoltaic module surface; it can also include a parallel or pre-set tilted state of the brush end effector relative to the photovoltaic module surface. During the cleaning process, the robotic arm can drive the joints to move according to joint angular velocity control commands, thereby changing the position and orientation of the brush end effector in space, gradually reducing the overall distance and orientation deviation between the brush end effector and the photovoltaic module surface.

[0028] In this application, multiple ranging sensors continuously collect distance measurements between the brush end effector and the photovoltaic module surface. The robotic arm continuously provides joint state information. The control system constructs a current state vector based on the distance measurements and joint state information, and outputs joint angular velocity control commands through a preset attitude control strategy network. After the robotic arm executes the joint angular velocity control commands, the relative pose between the brush end effector and the photovoltaic module surface changes. The multiple ranging sensors continue to collect new distance measurements in the next control cycle, thereby achieving continuous adaptive adjustment of the brush end effector's attitude.

[0029] In some embodiments of this application, step S130 obtains overall distance deviation information and attitude deviation information between the brush end effector and the photovoltaic module surface, including: determining overall distance deviation information and attitude deviation information between the brush end effector and the photovoltaic module surface based on multiple distance measurements, wherein the attitude deviation information includes a first attitude deviation determined based on the difference between distance measurements at different distance measurement positions in a first direction, and a second attitude deviation determined based on the difference between distance measurements at different distance measurement positions in a second direction.

[0030] In the above embodiments, the overall distance deviation information can be used to guide the robotic arm to adjust the end effector of the brush to move closer to or further away from the photovoltaic module surface, and the posture deviation information can be used to guide the robotic arm to adjust the tilt angle or direction of the end effector of the brush, making it more parallel to the photovoltaic module surface. By considering both the overall distance deviation and posture deviation simultaneously, the problem of ignoring local tilt while only maintaining the average distance can be avoided, thereby improving cleaning uniformity and operational safety. The first direction and the second direction can be understood as two different directions formed by the distribution of different distance measuring positions on the end effector of the brush. They can be determined according to the structure of the end effector of the brush and the arrangement of the distance measuring sensors. For example, the first direction can correspond to the length direction of the end effector of the brush, and the second direction can correspond to the width direction or other installation directions different from the first direction. For example, by comparing the distance measurement values ​​of different distance measuring positions in the first direction, it can be determined whether the end effector of the brush is tilted relative to the photovoltaic module surface in the first direction; by comparing the distance measurement values ​​of different distance measuring positions in the second direction, it can be determined whether the end effector of the brush has a posture deviation in the second direction. Therefore, the overall distance deviation information and attitude deviation information describe the relative pose state between the brush end effector and the photovoltaic module surface from the perspectives of overall distance and local tilt, respectively.

[0031] Figure 2 This document illustrates a flowchart of a method for determining overall distance deviation information and attitude deviation information between a brush end effector and the surface of a photovoltaic module, provided in some embodiments of this application. The method includes: S210: Perform median filtering on the distance measurement values ​​collected by each ranging sensor at multiple consecutive sampling times, and determine the corresponding median filtering result; S220: Perform Kalman filtering based on the median filtering results to determine multiple filter distance values; S230: Determine the overall distance deviation information based on the difference between the average of multiple filtered distance values ​​and the target distance; S240: Determine the first attitude deviation based on the difference between the filtered distance values ​​at different ranging positions in the first direction; S250: Determine the second attitude deviation based on the difference between the filtered distance values ​​at different ranging positions in the second direction.

[0032] In this embodiment, the distance measurements collected by the ranging sensor can undergo median filtering and Kalman filtering before entering the subsequent attitude control process to reduce the impact of outliers and random noise on the deviation information determination result. Median filtering is mainly used to suppress abrupt outliers. For each ranging sensor, the distance measurements collected at multiple consecutive sampling times can be selected as a sampling window, and the multiple distance measurements within this sampling window are sorted, with the median value being used as the median filtering result. In one specific implementation, the distance measurements collected continuously by each ranging sensor can be sorted, and the median value can be used as the valid input, thereby reducing the impact of impulse noise or instantaneous abnormal ranging values ​​on subsequent filtering and attitude determination. After determining the median filtering result, Kalman filtering can be performed based on the median filtering result. For any ranging sensor, a state-space model can be established with reference to Formula 1: Formula 1 Where, x k Let x represent the true distance state at time k. k-1 z represents the true distance state at time k-1. k The measured value input to the Kalman filter at time k represents the value, which can be the median filtering result mentioned above; A represents the state transition matrix, H represents the measurement matrix, and w k Indicates process noise, v k This represents measurement noise. Process noise and measurement noise can be set according to the measurement characteristics of the ranging sensor and the operating environment. Under this model, the Kalman filter can combine the distance state estimation result of the previous moment and the median filtering result of the current moment to recursively obtain the distance state estimation value of the current moment. This distance state estimation value can then be used as the filtered distance value of the corresponding ranging sensor.

[0033] After obtaining multiple filtered distance values, the overall distance deviation information can be determined based on the difference between the average of these filtered distance values ​​and the target distance. The target distance is the expected distance or equivalent distance that the brush end effector should maintain relative to the photovoltaic module surface during cleaning. The average of the multiple filtered distance values ​​characterizes the overall distance of the brush end effector relative to the photovoltaic module surface. If the average value is greater than the target distance, it indicates that the brush end effector is far from the photovoltaic module surface; if the average value is less than the target distance, it indicates that the brush end effector is close to the photovoltaic module surface. The overall distance deviation information can thus reflect the degree to which the brush end effector is close to or far from the photovoltaic module surface.

[0034] Furthermore, to characterize the attitude deviation of the brush end effector relative to the photovoltaic module surface, attitude deviation information can be determined based on the difference in filtered distance values ​​in different directions. When the difference in filtered distance values ​​corresponding to different ranging positions in the first direction is large, it indicates that the brush end effector has an attitude deviation relative to the photovoltaic module surface in the first direction; when the difference in filtered distance values ​​corresponding to different ranging positions in the second direction is large, it indicates that the brush end effector has an attitude deviation relative to the photovoltaic module surface in the second direction. Thus, the first attitude deviation and the second attitude deviation can be used to reflect the attitude deviation of the brush end effector in different directions, enabling the subsequent attitude control process to not only adjust the brush end effector to move closer to or further away from the photovoltaic module surface based on the overall distance deviation information, but also to adjust the tilt state of the brush end effector based on the first attitude deviation and the second attitude deviation.

[0035] In some embodiments of this application, the current state vector of the robotic arm is constructed based on multiple distance measurements, overall distance deviation information, posture deviation information, and joint state information. This includes: normalizing multiple distance measurements to determine distance state information; normalizing the joint angles of each joint to determine joint angle state information; normalizing the joint angular velocities of each joint to determine joint angular velocity state information; and constructing the current state vector of the robotic arm based on the distance state information, overall distance deviation information, posture deviation information, joint angle state information, and joint angular velocity state information.

[0036] In this embodiment, the current state vector can be used as input to a preset attitude control strategy network, reflecting the relative distance and attitude between the brush end effector and the photovoltaic module surface, as well as the motion state of the robotic arm itself. Since distance measurements, joint angles, and joint angular velocities have different dimensions and numerical ranges, these data can be normalized before constructing the current state vector to map them to a numerical range suitable for network input, thereby improving the stability of the network input data. For multiple distance measurements, a max-min normalization method can be used, and the normalized distance value can be referenced in the following formula 2: Formula 2 Where, d norm This represents the normalized distance value, where d represents the distance measurement value collected by the ranging sensor. min This represents the preset minimum distance measurement value, d. max This indicates the preset maximum distance measurement value. For example, the minimum distance measurement value d... min It can be set to 40 mm, with a maximum ranging value d. max This can be set to 400 mm to correspond to the effective measurement range of the rangefinder. Through the above normalization process, the distance measurement value can be mapped to the zero-to-one interval, thereby forming distance status information.

[0037] For the joint angles of each joint, since the range of motion of different joints of the robotic arm may be different, normalization is performed according to the angle range of each joint to ensure that the angle information of different joints has a uniform data scale in the current state vector. For the joint angular velocity of each joint, normalization is performed according to the corresponding angular velocity range of each joint to obtain the joint angular velocity state information, so that the preset attitude control strategy network can generate joint angular velocity control commands in combination with the current motion trend of the robotic arm.

[0038] After completing the above normalization process, the distance state information, overall distance deviation information, attitude deviation information, joint angle state information, and joint angular velocity state information can be combined in a preset order to construct the current state vector. This allows the preset attitude control strategy network to directly obtain control deviation information related to distance maintenance and attitude alignment while receiving the distance state and joint state. This is achieved using distance information from four ranging sensors and the current angle values ​​θ of six joints. d And the current angular velocity values ​​ω of the 6 joints d For example, the constructed state vector s t The total dimensions are 16, as shown in Formula 3 below: Formula 3 The numbers d1 to d4 can also be assigned based on the location of the ranging sensor. For example, the sensor located at the top left or bottom left corner would be numbered d1. L1 and d L2 The sensors in the upper right and lower right corners are numbered d. R1 and d R2 .

[0039] In some embodiments of this application, the joint angular velocity control command includes angular velocity commands corresponding to each joint of the robotic arm; inputting the current state vector into a preset attitude control strategy network to determine the joint angular velocity control command of the robotic arm includes: inputting the current state vector into the preset attitude control strategy network to obtain a continuous motion vector; and performing amplitude limiting processing on the continuous motion vector based on a preset angular velocity range to determine the angular velocity command corresponding to each joint.

[0040] The continuous motion vector output by the preset attitude control strategy network can be used to characterize the motion control quantities of each joint of the robotic arm within the current control cycle. The end effector attitude of the brush end effector needs to be adjusted through the coordinated motion of multiple joints. Therefore, the continuous motion vector can include multiple motion components corresponding one-to-one with each joint, each motion component used to determine the angular velocity command of the corresponding joint. Thus, the preset attitude control strategy network can directly output continuous numerical motion results to drive the robotic arm's motion based on the distance state, attitude deviation, and joint state reflected in the current state vector. This allows for a detailed characterization of the magnitude and direction of the motion speed of each joint in different states, enabling smooth attitude adjustment of the robotic arm towards the brush end effector. For example, taking a six-DOF robotic arm as an example, the continuous motion vector a... t This can be expressed by Formula 4: Formula 4 The continuous motion vector can include angular velocity control values ​​for the six joints, from ωt1 to ωt6. Due to limitations imposed by factors such as the motor, mechanical structure, and cleaning operation stability, the angular velocities of each joint in the robotic arm cannot be increased indefinitely. Therefore, after obtaining the continuous motion vector, it can be limited based on a preset angular velocity range, setting the allowable output angular velocity range for each joint. For example, this can be limited to ±2.5 radians per second. When a motion component in the continuous motion vector exceeds the upper limit of the corresponding joint's angular velocity, the motion component is adjusted to that upper limit; when a motion component is below the lower limit of the corresponding joint's angular velocity, the motion component is adjusted to that lower limit; when a motion component is within the preset angular velocity range of the corresponding joint, the motion component can be retained.

[0041] In some embodiments of this application, the preset attitude control strategy network includes an actor network and a critic network; the actor network is configured to output a continuous action vector based on the current state vector; the critic network is configured to determine the state-action value corresponding to the current state vector and the continuous action vector based on the current state vector and the continuous action vector; during the training of the preset attitude control strategy network, the network parameters of the actor network and the critic network are updated based on a reward function, which includes a distance maintenance reward, an attitude alignment reward, a stability control reward, and an efficiency reward; wherein, the distance maintenance reward is determined based on the distance deviation between the end effector of the brush and the surface of the photovoltaic module, the attitude alignment reward is determined based on the attitude deviation information, the stability control reward is determined based on the action amplitude of the continuous action vector, and the efficiency reward is determined based on whether the overall distance deviation information and the attitude deviation information meet the preset error conditions.

[0042] In this embodiment, the preset posture control strategy network can employ a reinforcement learning structure combining an actor network and a critic network. The actor network outputs continuous motion vectors based on the current state vector, which can be further used to determine the angular velocity commands for each joint of the robotic arm. The critic network outputs state-action values ​​based on the current state vector and the continuous motion vectors. These state-action values ​​evaluate the contribution of executing the corresponding continuous motion vectors in the current state to achieving the target relative pose. Therefore, during training, the critic network's evaluation of the state-action values ​​can guide the actor network to update its motion output strategy. For example, the actor network may include an input layer, at least one hidden layer, and an output layer. The input layer receives the current state vector, and the output layer outputs the continuous motion vectors. The critic network may also include an input layer, at least one hidden layer, and an output layer. The input layer receives the current state vector and the continuous motion vectors, and the output layer outputs the state-action values.

[0043] To enable the preset attitude control policy network to learn a control policy that simultaneously satisfies the requirements of distance preservation, attitude alignment, motion stability, and fast convergence, the overall reward function R shown in Equation 5 can be used during training. t : Formula 5 Among them, R distance Indicates a distance-maintaining reward; R alignment Indicates the pose alignment reward; R stability Indicates stable control of rewards; R efficiency The distance retention reward R represents the efficiency reward; α1, α2, and α3 represent the weight coefficients of the distance retention reward, attitude alignment reward, and stable control reward, respectively. By adjusting these weight coefficients, the weight proportions of different control objectives during training can be changed. distance Posture alignment reward Ralignment Stable control reward R stability It can be determined using Formula 6: Formula 6 Where, d avg d represents the average distance values ​​corresponding to multiple ranging sensors. target This represents the target distance. The distance retention reward is related to the deviation between the average distance and the target distance, when d avg The closer to d target The higher the distance maintenance reward, the lower the reward; conversely, the higher the deviation between the two, the lower the reward. This distance maintenance reward guides the preset attitude control strategy network to output a continuous motion vector that brings the brush end effector closer to the target distance. left and d right d can represent the average distance values ​​of the distance sensors located on different sides of the first direction. up and d down These can represent the average distance values ​​of the ranging sensors located on different sides of the second direction. This attitude alignment reward characterizes the attitude deviation of the brush end effector relative to the photovoltaic module surface in different directions. The smaller the distance difference in the first or second direction, the closer the brush end effector is to the target attitude, and the higher the attitude alignment reward. 'a' represents the continuous motion vector, ||a|| 2 This represents the squared norm of the continuous motion vector. The stability control reward is negatively correlated with the motion amplitude; the larger the amplitude of the continuous motion vector, the lower the stability control reward. This stability control reward can suppress violent movements of the robotic arm joints, reduce the impact of the brush end effector on the photovoltaic module surface, and lower mechanical wear and energy consumption. Efficiency reward R efficiency: You can refer to the following formula 7: Formula 7 ε represents the error tolerance threshold, and β represents the efficiency bonus value. When the distance error between the brush end effector and the photovoltaic module surface is less than the error tolerance threshold, a positive efficiency bonus is given to guide the preset attitude control strategy network to learn a control strategy that quickly enters and maintains a stable cleaning state. In a specific example, α1 can be 1.0, α2 can be 1.5, α3 can be 0.1, and β can be 10. The above parameters are only examples and can be adjusted according to the brush material, robotic arm motion performance, cleaning pressure requirements, and photovoltaic module surface safety requirements.

[0044] In some embodiments of this application, the actor network includes a context encoder and a master policy network; the context encoder is configured to generate a context vector based on the historical state vector, the historical continuous action vector and the state vector at the next moment within a preset historical period, and the context vector is used to characterize the dynamic characteristics of the current working environment; the current state vector is input into the preset attitude control policy network to obtain the continuous action vector. Figure 3 The following is a flowchart illustrating a method for determining a continuous action vector provided in some embodiments of this application. The method includes: S310: Obtain the context vector output by the context encoder; S320: Concatenate the current state vector and the context vector to obtain the extended state vector; S330: Input the extended state vector into the main policy network to obtain continuous action vectors; wherein, the training of the context encoder and the main policy network includes: simulation environment training and real environment parameter update; S340: In simulation environment training, at least one of the photovoltaic module parameters, sensor parameters and environmental parameters is randomized, and the context encoder and master policy network are trained based on the randomized simulation environment. S350: In the real environment parameter update, the network parameters obtained by training in the simulation environment are transferred to the photovoltaic cleaning robot. The network parameters of the context encoder and some network parameters of the main strategy network remain unchanged. Based on the training data collected in the real working environment, the network parameters of at least one fully connected layer near the output end in the main strategy network are updated. Among them, the photovoltaic module parameters include at least one of the following: photovoltaic module tilt angle, photovoltaic module radius of curvature and surface friction coefficient; the sensor parameters include at least one of the following: measurement delay, measurement noise and data loss rate; and the environmental parameters include at least one of the following: ground unevenness and base vibration amplitude.

[0045] In this embodiment, the context encoder is used to extract the dynamic characteristics of the current operating environment from the historical control process. These dynamic characteristics may include, but are not limited to, sensor noise levels, photovoltaic module surface friction conditions, base vibration conditions, and the degree to which ground unevenness affects the robot arm's end-effector posture. Since these factors are difficult to measure directly in actual operation, they can be indirectly characterized through the relationship between the historical state vector, the historical continuous action vector, and the next-moment state vector. For example, the input to the context encoder can be a sequence consisting of the states, actions, and next states from the past N time steps, where τ... t Formula 8 can be used as a reference: Formula 8 Where, τ tLet s represent the historical sequence corresponding to time t, N represent the number of samples within the preset historical time period, and s t-N To s t a represents the state vector corresponding to different control moments. t-N To a t-1 This represents the continuous action vectors corresponding to different control moments. This historical sequence reflects the influence of these continuous action vectors on state changes under a specific operating environment. The process of the context encoder generating context vectors based on the historical sequence can be represented as z. t =Encoder(τ t Among them, z t Let represent the context vector at time t, and Encoder() represent the context encoder. The context encoder extracts temporal features from the historical sequence, providing supplementary input to the main policy network that reflects the differences in the current operating environment. After obtaining the context vector, the current state vector and the context vector can be concatenated to obtain the extended state vector 'a'. t =π(s t ,z t ), where a t Let z represent the current state vector. t Let π represent the context vector, and π() represent the main policy network. The main policy network is based on the extended state vector a. t The output continuous motion vectors enable the actor network to jointly determine the joint motion control quantities of the robotic arm based on the current state and environmental characteristics.

[0046] In some implementations, the context encoder can employ a bidirectional long short-term memory network and output a latent vector of a preset dimension as the context vector. This network structure can simultaneously utilize sequential information from previous and subsequent events in the historical sequence, improving its ability to characterize environmental dynamics such as sensor noise, friction variations, and base vibration. During simulation environment training, the photovoltaic module parameters, sensor parameters, and environmental parameters can be randomized, allowing the context encoder and the master policy network to learn control strategies under different operating conditions. By changing the photovoltaic module parameters, sensor parameters, or environmental parameters during training, the network can be exposed to multiple possible operating environments, reducing the control strategy's dependence on a single simulation condition.

[0047] The loss functions of the context encoder and the main policy network during training in a simulation environment can be expressed by Equation 9: Formula 9 Where L represents the total loss function, L policy L represents the policy loss, used to update the action output capability of the main policy network; encoderL represents the encoder prediction loss, used to constrain the context encoder's ability to characterize randomized physical parameters or environmental characteristics; decoder λ1 and λ2 represent the auxiliary reconstruction loss, used to enhance the ability of the context vector to express the relationship of historical state changes; λ1 and λ2 represent the loss balance coefficients. Using the above loss function, the context encoder can learn to extract operational environment dynamics information from historical control sequences while training the main policy network.

[0048] In updating parameters in a real-world environment, the network parameters of the context encoder and some network parameters of the master policy network can be kept unchanged. Only the network parameters of at least one fully connected layer in the master policy network closest to the output are updated. This approach can adjust the motion output using real-world operational data while preserving the general state representation and control capabilities learned in simulation training. This makes the continuous motion vectors more adaptable to the actual robotic arm, sensor, and photovoltaic module environment.

[0049] In this way, the actor network can further introduce a context vector on top of the current state vector, so that the action output of the main strategy network no longer depends solely on the current state information, but also incorporates the dynamic characteristics of the working environment reflected by historical state changes. Therefore, even under conditions such as changes in photovoltaic module tilt angle, sensor noise variations, uneven ground, or base vibration, the preset attitude control strategy network can still output continuous action vectors that are more adapted to the current environment based on the extended state vector, improving the environmental adaptability and stability of the brush end effector's attitude control.

[0050] In some embodiments of this application, the preset attitude control strategy network is trained in an increasing manner, including: a basic task training phase, a perturbation task training phase, and an extreme task training phase. In the basic task training phase, the preset attitude control strategy network is trained based on a first working condition, which includes: the photovoltaic module is placed horizontally, there is no uneven ground, and the sensor noise is less than a preset noise threshold. In the perturbation task training phase, the preset attitude control strategy network is trained based on a second working condition, which introduces at least one of the following perturbation factors relative to the first working condition: photovoltaic module tilt angle, ground unevenness, and sensor noise. In the extreme task training phase, the preset attitude control strategy network is trained based on a third working condition, where the perturbation range of the perturbation factor in the third working condition is greater than that in the second working condition. When the success rate of the preset attitude control strategy network in multiple consecutive training rounds of the current training phase meets the target success rate threshold of the corresponding training phase, the next training phase begins.

[0051] During the basic task training phase, the relative relationship between the brush end effector and the photovoltaic module surface is relatively stable. The preset attitude control strategy network mainly learns the basic rules of outputting joint angular velocity control commands based on distance measurement values ​​and joint state information, so that the brush end effector can approach the target distance and maintain a relatively parallel attitude.

[0052] During the perturbation training phase, the second operating condition introduces a certain range of perturbation factors compared to the first operating condition. For example, the tilt angle of the photovoltaic module can be randomly set within the range of 0 to 10 degrees, the ground unevenness can vary randomly within the range of 0 mm to 30 mm, and the sensor measurement noise can be randomly added within a preset noise range. This phase is used to enable the preset attitude control strategy network to learn the attitude control strategy under non-ideal conditions, so that it can still output relatively stable joint angular velocity control commands when the photovoltaic module has a tilt angle, the moving chassis is affected by ground undulations, or the ranging sensor has a certain amount of noise.

[0053] In the extreme mission training phase, the perturbation range of the disturbance factors in the third working condition is greater than that in the second working condition. For example, the tilt angle of the photovoltaic module can be randomly set within the range of 0 to 30 degrees, the ground unevenness can vary randomly within the range of 0 mm to 100 mm, and the sensor measurement noise can be randomly added over a wider range. This phase is used to enhance the adaptability of the preset attitude control strategy network to extreme working conditions, enabling it to maintain the target relative pose of the brush end effector to the photovoltaic module surface even when the photovoltaic module attitude changes significantly, the ground undulations are obvious, or the sensor measurement noise increases significantly.

[0054] To ensure a smoother transition between different training phases, some implementations may include a transition period between adjacent training phases. During this transition period, the operating parameters of the previous and subsequent training phases can be gradually changed using linear interpolation. For example, the tilt angle range of the photovoltaic modules, the ground unevenness range, or the sensor noise range can be gradually increased over a preset number of training rounds, allowing the preset attitude control strategy network to gradually adapt to more complex training conditions.

[0055] In some implementations, when entering the next training phase, some experience data from the previous training phase can be retained and used together with newly acquired experience data in the next training phase for training. This reduces the risk that the preset attitude control strategy network will forget the basic control capabilities it has learned after entering the new training phase, so that it can continue to maintain its control capabilities for basic distance maintenance and attitude alignment tasks while adapting to more difficult working conditions.

[0056] In some embodiments of this application, the robotic arm is a six-degree-of-freedom robotic arm, including a base rotation joint, a shoulder joint, an elbow joint, a wrist pitch joint, a wrist flip joint, and an end effector rotation joint; the multiple ranging sensors include four ultrasonic ranging sensors, which are respectively disposed at different ranging positions of the brush end effector, and the four ranging positions are spaced apart in the first direction and the second direction.

[0057] Figure 4 This diagram illustrates the structure of a posture control device for a photovoltaic cleaning robot according to some embodiments of this application. The posture control device 400 is applied to a photovoltaic cleaning robot, which includes a robotic arm with multiple joints, a roller brush end effector disposed at the end of the robotic arm, and multiple ranging sensors disposed on the roller brush end effector. The multiple ranging sensors are respectively arranged at different ranging positions on the roller brush end effector. The posture control device 400 includes: a distance acquisition unit 410, used to acquire distance measurement values ​​collected by the multiple ranging sensors, the distance measurement values ​​being used to characterize the relative distance between the roller brush end effector and the surface of the photovoltaic module at the corresponding ranging position; a state acquisition unit 420, used to acquire joint state information of the robotic arm, including the joint angle and joint angular velocity of each joint; and to acquire the distance between the roller brush end effector and the surface of the photovoltaic module. The system includes: an overall distance deviation information and an attitude deviation information between the photovoltaic module surfaces; a construction unit 430, used to construct the current state vector of the robotic arm based on multiple distance measurements, overall distance deviation information, attitude deviation information, and joint state information; a control command determination unit 440, used to input the current state vector into a preset attitude control strategy network to determine the joint angular velocity control command of the robotic arm, wherein the preset attitude control strategy network is a continuous motion control network trained based on context-aware domain reinforcement learning; and an adjustment unit 450, used to control the movement of the robotic arm based on the joint angular velocity control command to adjust the end attitude of the brush end effector relative to the photovoltaic module surface, so that the brush end effector maintains the target relative pose with respect to the photovoltaic module surface.

[0058] Based on the same technical concept, this application also provides a computer-readable storage medium storing instructions thereon, which, when read by a processor, execute the posture control method of the photovoltaic cleaning robot provided in the above embodiments.

[0059] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail or in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Furthermore, the above embodiments can be freely combined as needed.

Claims

1. A posture control method for a photovoltaic cleaning robot, characterized in that, The photovoltaic cleaning robot includes a robotic arm with multiple joints, a roller brush end effector disposed at the end of the robotic arm, and multiple ranging sensors disposed on the roller brush end effector, wherein the multiple ranging sensors are respectively arranged at different ranging positions on the roller brush end effector, and the attitude control method includes: The distance measurement values ​​collected by the multiple ranging sensors are obtained respectively. The distance measurement values ​​are used to characterize the relative distance between the brush end effector and the surface of the photovoltaic module at the corresponding ranging position. The joint state information of the robotic arm is obtained, including the joint angle and joint angular velocity of each joint. Obtain the overall distance deviation information and attitude deviation information between the end effector of the roller brush and the surface of the photovoltaic module; Based on multiple distance measurements, the overall distance deviation information, the posture deviation information, and the joint state information, the current state vector of the robotic arm is constructed. The current state vector is input into a preset attitude control strategy network to determine the joint angular velocity control command of the robotic arm. The preset attitude control strategy network is a continuous motion control network trained based on context-aware domain reinforcement learning. The movement of the robotic arm is controlled based on the joint angular velocity control command to adjust the end posture of the brush end effector relative to the surface of the photovoltaic module, so that the brush end effector maintains the target relative posture with respect to the surface of the photovoltaic module.

2. The attitude control method according to claim 1, characterized in that, The step of obtaining the overall distance deviation information and attitude deviation information between the end effector of the roller brush and the surface of the photovoltaic module includes: Based on multiple distance measurements, overall distance deviation information and attitude deviation information between the brush end effector and the surface of the photovoltaic module are determined. The attitude deviation information includes a first attitude deviation determined based on the difference between distance measurements at different distance measurement positions in a first direction, and a second attitude deviation determined based on the difference between distance measurements at different distance measurement positions in a second direction.

3. The attitude control method according to claim 2, characterized in that, The determination of overall distance deviation information and attitude deviation information between the end effector of the roller brush and the surface of the photovoltaic module based on multiple distance measurements includes: The distance measurement values ​​collected by each of the ranging sensors at multiple consecutive sampling times are subjected to median filtering to determine the corresponding median filtering result; Based on the median filtering results, Kalman filtering is performed to determine multiple filtering distance values; The overall distance deviation information is determined based on the difference between the average of the multiple filtered distance values ​​and the target distance; The first attitude deviation is determined based on the difference between the filtered distance values ​​at different ranging positions located in the first direction; The second attitude deviation is determined based on the difference between the filtered distance values ​​at different ranging positions in the second direction.

4. The attitude control method according to claim 1, characterized in that, The construction of the current state vector of the robotic arm based on multiple distance measurements, overall distance deviation information, posture deviation information, and joint state information includes: The distance measurements are normalized to determine the distance status information; The joint angles of each joint are normalized to determine the joint angle state information; The joint angular velocities of each joint are normalized to determine the joint angular velocity state information; Based on the distance state information, the overall distance deviation information, the posture deviation information, the joint angle state information, and the joint angular velocity state information, the current state vector of the robotic arm is constructed.

5. The attitude control method according to claim 1, characterized in that, The joint angular velocity control command includes the angular velocity command corresponding to each joint of the robotic arm; The step of inputting the current state vector into a preset attitude control strategy network to determine the joint angular velocity control command of the robotic arm includes: The current state vector is input into the preset attitude control strategy network to obtain a continuous action vector; The continuous motion vector is limited based on a preset angular velocity range to determine the angular velocity command corresponding to each joint.

6. The attitude control method according to claim 5, characterized in that, The preset posture control strategy network includes an actor network and a critic network; The actor network is configured to output the continuous motion vector based on the current state vector; The critic network is configured to determine the state-action value corresponding to the current state vector and the continuous action vector based on the current state vector and the continuous action vector; During the training of the preset posture control strategy network, the network parameters of the actor network and the critic network are updated based on a reward function, which includes distance preservation reward, posture alignment reward, stability control reward and efficiency reward. The distance maintenance reward is determined based on the distance deviation between the end effector of the roller brush and the surface of the photovoltaic module; the attitude alignment reward is determined based on the attitude deviation information; the stability control reward is determined based on the motion amplitude of the continuous motion vector; and the efficiency reward is determined based on whether the overall distance deviation information and the attitude deviation information meet the preset error conditions.

7. The attitude control method according to claim 6, characterized in that, The actor network includes a context encoder and a master policy network; The context encoder is configured to generate a context vector based on the historical state vector, historical continuous action vector and the state vector at the next moment within a preset historical period. The context vector is used to characterize the dynamic characteristics of the current working environment. The step of inputting the current state vector into the preset attitude control strategy network to obtain a continuous action vector includes: Obtain the context vector output by the context encoder; The current state vector and the context vector are concatenated to obtain the extended state vector; The extended state vector is input into the main policy network to obtain the continuous action vector; The training of the context encoder and the master policy network includes: simulation environment training and real environment parameter updating; in the simulation environment training, at least one of the photovoltaic module parameters, sensor parameters and environmental parameters is randomized, and the context encoder and the master policy network are trained based on the randomized simulation environment. In the real environment parameter update, the network parameters obtained by training in the simulation environment are transferred to the photovoltaic cleaning robot, while keeping the network parameters of the context encoder and some network parameters of the main strategy network unchanged. Based on the training data collected in the real working environment, the network parameters of at least one fully connected layer in the main strategy network near the output end are updated. The photovoltaic module parameters include at least one of the following: photovoltaic module tilt angle, photovoltaic module radius of curvature and surface friction coefficient; the sensor parameters include at least one of the following: measurement delay, measurement noise and data loss rate; and the environmental parameters include at least one of the following: ground unevenness and base vibration amplitude.

8. The attitude control method according to claim 7, characterized in that, The preset attitude control strategy network is trained in an increasing manner, including: a basic task training phase, a perturbation task training phase, and an extreme task training phase. In the basic task training phase, the preset attitude control strategy network is trained based on the first working condition, which includes: the photovoltaic module is placed horizontally, there is no uneven ground, and the sensor noise is less than a preset noise threshold. During the training phase of the disturbance task, the preset attitude control strategy network is trained based on the second working condition, which introduces at least one disturbance factor among photovoltaic module tilt angle, ground unevenness and sensor noise relative to the first working condition. During the extreme task training phase, the preset attitude control strategy network is trained based on the third working condition, where the perturbation range of the perturbation factor in the third working condition is greater than the perturbation range of the perturbation factor in the second working condition. Specifically, when the success rate of the preset attitude control strategy network in multiple consecutive training rounds of the current training phase meets the target success rate threshold of the corresponding training phase, it enters the next training phase.

9. The attitude control method according to any one of claims 1 to 8, characterized in that, The robotic arm is a six-degree-of-freedom robotic arm, including a base rotation joint, a shoulder joint, an elbow joint, a wrist pitch joint, a wrist flip joint, and an end effector rotation joint; The plurality of ranging sensors include four ultrasonic ranging sensors, which are respectively disposed at different ranging positions of the roller brush end actuator, and the four ranging positions are arranged at intervals in the first direction and the second direction.

10. A posture control device for a photovoltaic cleaning robot, characterized in that, The photovoltaic cleaning robot includes a robotic arm with multiple joints, a roller brush end effector disposed at the end of the robotic arm, and multiple ranging sensors disposed on the roller brush end effector, wherein the multiple ranging sensors are respectively arranged at different ranging positions on the roller brush end effector, and the attitude control device includes: The distance acquisition unit is used to acquire distance measurement values ​​collected by the multiple ranging sensors respectively. The distance measurement values ​​are used to characterize the relative distance between the end effector of the roller brush and the surface of the photovoltaic module at the corresponding ranging position. The status acquisition unit is used to acquire the joint status information of the robotic arm, including the joint angle and joint angular velocity of each joint; and to acquire the overall distance deviation information and attitude deviation information between the end effector of the roller brush and the surface of the photovoltaic module. The construction unit is used to construct a current state vector based on multiple distance measurement values, the overall distance deviation information, the posture deviation information, and the joint state information; The control command determination unit is used to input the current state vector into a preset attitude control strategy network to determine the joint angular velocity control command of the robotic arm, wherein the preset attitude control strategy network is a continuous motion control network trained based on context-aware domain reinforcement learning. An adjustment unit is used to control the movement of the robotic arm based on the joint angular velocity control command, so as to adjust the end posture of the brush end effector relative to the surface of the photovoltaic module, so that the brush end effector maintains a target relative posture with respect to the surface of the photovoltaic module.

Images