A photovoltaic power station cleaning method, device, equipment and readable storage medium

Abstract

This application discloses a method, apparatus, equipment, and readable storage medium for cleaning photovoltaic power plants, including: generating a first reference trajectory with a timestamp for a drone and a second reference trajectory with a timestamp for a legged robot; wherein the timestamps of the drone and the legged robot are synchronized; acquiring the current states of the drone and the legged robot; using a model predictive controller to predict control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current state; and controlling the legged robot to release or retrieve water pipes accordingly through the control commands to clean the photovoltaic power plant. This application achieves precise matching between the task trajectory and the executed actions, ensuring the accuracy, safety, and efficient coordination of water pipe operations.

Description

Technical Field

[0001] This application relates to the field of photovoltaic operation and maintenance technology, and in particular to a method, apparatus, equipment and computer-readable storage medium for cleaning photovoltaic power plants. Background Technology

[0002] The accumulation of contaminants near the surface of photovoltaic modules reduces their light transmittance and triggers hot spot effects, thus significantly impacting the power generation performance of photovoltaic power plants. Manual cleaning of modules is inefficient, ineffective, and poses safety risks for personnel. Photovoltaic cleaning robots are energy-intensive and expensive, hindering their widespread adoption. Furthermore, the pressure exerted by the robot's weight on the photovoltaic modules and the vibrations generated during operation may damage the module surface and shorten their lifespan.

[0003] Currently, drone cleaning is being gradually promoted and applied. Its cleaning process does not involve direct contact with components, resulting in high cleaning efficiency and low labor costs. For tank-type cleaning drones, the drone's payload capacity limits the amount of water it can carry and the weight of the water pump, and the round trip for water retrieval is time-consuming, thus limiting its promotion and application. Tethered cleaning drones, due to the gaps between component arrays, may have their tethered water pipes entangled with components and supports. This usually requires dedicated personnel to promptly pull the tethered water pipes, resulting in high workload, high labor costs, and a higher risk due to proximity to the operating drone.

[0004] In summary, how to effectively solve the problems of high labor intensity, high labor costs, and high risk associated with the current photovoltaic power station cleaning methods, which involve close proximity to drones, is an urgent issue that needs to be addressed by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a photovoltaic power plant cleaning method that achieves precise matching between task trajectory and execution action, ensuring the accuracy, safety, and efficient coordination of water pipe operations; another purpose of this application is to provide a photovoltaic power plant cleaning device, equipment, and computer-readable storage medium.

[0006] To solve the above-mentioned technical problems, this application provides the following technical solution: A method for cleaning a photovoltaic power station, comprising: A first reference trajectory with a timestamp is generated for the UAV, and a second reference trajectory with a timestamp is generated for the legged robot; wherein the timestamp of the UAV is synchronized with the timestamp of the legged robot; Obtain the current state of the drone and the legged robot; The model predictive controller predicts control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current state. The control commands control the legged robot to release or retrieve the water pipes to clean the photovoltaic power station.

[0007] In one specific embodiment of this application, obtaining the current state of the drone and the legged robot includes: Obtain the relative pose between the drone and the legged robot; Obtain the absolute global pose of the legged robot; The model prediction controller uses the legged robot dynamics model and the UAV dynamics model to predict the cooperative state of the UAV and the legged robot. The relative pose, the absolute global pose, and the cooperative state are fused using a preset state estimation algorithm to obtain a globally consistent pose of the UAV and the legged robot; wherein, the globally consistent pose includes the three-dimensional position of the legged robot, the pose quaternion of the legged robot, the three-dimensional position of the UAV, and the pose quaternion of the UAV. The globally consistent pose is determined as the current state.

[0008] In one specific embodiment of this application, obtaining the absolute global pose of the legged robot includes: The first pose of the legged robot in the previous time step is obtained, and the particle cloud is initialized according to the first pose. The initialized particle cloud is then determined as the particle cloud of the previous time step. Obtain the displacement difference and rotation difference from the previous time step to the current time step; Using the adaptive Monte Carlo localization algorithm, based on the displacement difference and the rotation difference, the running model is used to predict the second pose of each particle in the particle cloud at the current time step, according to the displacement difference and the rotation difference. The absolute global pose of the legged robot is determined based on each second pose.

[0009] In one specific embodiment of this application, determining the absolute global pose of the legged robot based on each second pose includes: A likelihood field representing the probability of occurrence of a measurement point is generated based on each second pose; wherein, the measurement point is the point where the scanning ray hits the obstacle; Acquire the current scan data and convert the current scan data to the global map coordinate system according to each second pose to obtain each query location point; wherein, the current scan data includes the current measured distance corresponding to each current scan ray; Find the probability value of each current scanning ray at each query position point from the likelihood field; wherein each probability value is the probability that the end point of each current scanning ray appears at each query position point when it is in each second pose. Calculate the particle weights corresponding to the particles in each second pose based on the probability values. The particles in each second pose are resampled according to their respective weights to obtain the resampled particles. The absolute global pose of the legged robot is determined based on the second pose corresponding to each resampled particle.

[0010] In one specific embodiment of this application, obtaining the relative pose between the drone and the legged robot, and determining the absolute global pose of the legged robot based on the second poses corresponding to each resampled particle, includes: The relative pose between the drone and the legged robot is calculated using a visual algorithm. The predicted global pose of the legged robot is determined based on the second pose corresponding to each resampled particle. Obtain the global position provided by the UAV positioning system, and calculate the visual global pose of the legged robot based on the relative pose and the global position; The preset state estimation algorithm is used to perform state estimation correction on the predicted global pose and the visual global pose, and the absolute global pose is output.

[0011] In one specific embodiment of this application, generating a first reference trajectory with a timestamp for a drone and generating a second reference trajectory with a timestamp for a legged robot includes: The global optimal path for the UAV and the global optimal path for the legged robot are calculated based on a preset optimization algorithm. The global optimal path of the UAV and the global optimal path of the legged robot are spatiotemporally synchronized and their feasibility is verified to obtain the first reference trajectory and the second reference trajectory.

[0012] In one specific embodiment of this application, it further includes: When an abnormality is detected in the release or dragging of the water pipe, an abnormality report is made.

[0013] A photovoltaic power plant cleaning device includes: A reference trajectory generation module is used to generate a first reference trajectory with a timestamp for the UAV and a second reference trajectory with a timestamp for the legged robot; wherein the timestamp of the UAV is synchronized with the timestamp of the legged robot; The current state acquisition module is used to acquire the current state of the drone and the legged robot; The control command prediction module is used to predict control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory and the current state using a model prediction controller. The photovoltaic power station cleaning module is used to control the legged robot to release or retract water pipes according to the control commands, so as to clean the photovoltaic power station.

[0014] A photovoltaic power plant cleaning device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the photovoltaic power plant cleaning method as described above.

[0015] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the photovoltaic power plant cleaning method described above.

[0016] The photovoltaic power station cleaning method provided in this application includes: generating a first reference trajectory with a timestamp for a drone and generating a second reference trajectory with a timestamp for a legged robot; wherein the timestamp of the drone is synchronized with the timestamp of the legged robot; obtaining the current state of the drone and the legged robot; using a model predictive controller to predict control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory and the current state; and controlling the legged robot to release or retrieve water pipes accordingly through the control commands to clean the photovoltaic power station.

[0017] As described in the above technical solution, by generating timestamped reference trajectories for the legged robot and the drone, the current states of the drone and the legged robot are obtained. A model predictive controller then predicts control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current state. These control commands then control the legged robot to release or retrieve the water pipes, thereby completing the cleaning of the photovoltaic power station. By using the legged robot to drag, release, and retrieve the water pipes, and by jointly planning the spatiotemporal aspects of the drone and the legged robot, precise matching of task trajectories and execution actions is achieved, ensuring the accuracy, safety, and efficient collaboration of the water pipe operation.

[0018] Accordingly, this application also provides a photovoltaic power plant cleaning device, equipment, and computer-readable storage medium corresponding to the above-mentioned photovoltaic power plant cleaning method, which have the above-mentioned technical effects, and will not be repeated here. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating one implementation of the photovoltaic power station cleaning method in this application. Figure 2 This is a flowchart illustrating another implementation of the photovoltaic power station cleaning method in this application. Figure 3 This is a schematic diagram illustrating the starting position of cleaning the first row of components in an embodiment of this application; Figure 4 This is a schematic diagram of a cleaning method for the first row of components according to an embodiment of this application; Figure 5 This is a schematic diagram illustrating the end position of cleaning the first row of components in an embodiment of this application; Figure 6 This is a schematic diagram of a component moving to the second row in an embodiment of this application; Figure 7 This is a schematic diagram illustrating the starting position of cleaning the second row of components in an embodiment of this application; Figure 8 This is a schematic diagram of a cleaning process for the second row of components according to an embodiment of this application; Figure 9 This is a schematic diagram of a dynamic real-time optimization in an embodiment of this application; Figure 10 This is a flowchart illustrating a real-time accessibility verification process in an embodiment of this application. Figure 11 This is a diagram of a hierarchical closed-loop data fusion control architecture in an embodiment of this application; Figure 12 This is a schematic diagram of a multi-mode adaptive mechanism in an embodiment of this application; Figure 13 This is a schematic diagram of the pose tracking process of a drone and a legged robot in an embodiment of this application; Figure 14 This is a structural block diagram of a photovoltaic power station cleaning device according to an embodiment of this application; Figure 15 This is a structural block diagram of a photovoltaic power station cleaning device according to an embodiment of this application; Figure 16 This is a schematic diagram of the specific structure of a photovoltaic power station cleaning device provided in an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0023] See Figure 1 , Figure 1 This is a flowchart illustrating one implementation of a photovoltaic power plant cleaning method in this application. The method may include the following steps: S101: Generate a first reference trajectory with timestamps for the UAV and a second reference trajectory with timestamps for the legged robot.

[0024] The timestamps of the drones and the legged robots are synchronized.

[0025] When cleaning a photovoltaic power station is required, a drone and a legged robot can be used in collaboration. The legged robot can be a robotic dog. A first reference trajectory with a timestamp is generated for the drone, and a second reference trajectory with a timestamp is generated for the legged robot. The timestamps of the drone and the legged robot are synchronized.

[0026] S102: Obtain the current state of the drone and legged robot.

[0027] When using a drone and a legged robot to clean a photovoltaic power station in a coordinated manner, the current state of the drone and the legged robot is obtained. The current state can be determined based on the relative pose between the drone and the legged robot, the absolute global pose of the legged robot, and the coordinated state of the drone and the legged robot.

[0028] S103: The model predictive controller predicts control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current state.

[0029] After generating a timestamped first reference trajectory for the UAV and a timestamped second reference trajectory for the legged robot, and obtaining the current states of both the UAV and the legged robot, a model predictive controller is used to predict control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current states. Dynamic constraints may include steering capability, attitude limitations, etc.

[0030] S104: Control the legged robot to release or retrieve water pipes via control commands to clean the photovoltaic power station.

[0031] After predicting and obtaining control commands that meet the dynamic constraints, the legged robot is controlled to release or retrieve the water pipes accordingly, thereby cleaning the photovoltaic power station.

[0032] As described in the above technical solution, by generating timestamped reference trajectories for the legged robot and the drone, the current states of the drone and the legged robot are obtained. A model predictive controller then predicts control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current state. These control commands then control the legged robot to release or retrieve the water pipes, thereby completing the cleaning of the photovoltaic power station. By using the legged robot to drag, release, and retrieve the water pipes, and by jointly planning the spatiotemporal aspects of the drone and the legged robot, precise matching of task trajectories and execution actions is achieved, ensuring the accuracy, safety, and efficient collaboration of the water pipe operation.

[0033] It should be noted that, based on the above embodiments, this application also provides corresponding improvement solutions. In subsequent embodiments, steps that are the same as or corresponding to those in the above embodiments can be referred to each other, and the corresponding beneficial effects can also be referred to each other. These improvements will not be elaborated upon in the following improved embodiments.

[0034] See Figure 2 , Figure 2 This is another implementation flowchart of the photovoltaic power plant cleaning method in this application embodiment. The method may include the following steps: S201: Calculate the global optimal path for the UAV and the global optimal path for the legged robot based on the preset optimization algorithm.

[0035] The globally optimal path for the UAV and the globally optimal path for the legged robot are calculated based on preset optimization algorithms. These preset optimization algorithms may include Minimum Control Effort Trajectory Optimization (MINCO) and nonlinear programming algorithms.

[0036] See Figure 3, Figure 3 This is a schematic diagram illustrating the cleaning of a power plant component according to an embodiment of this application. See also... Figure 4 , Figure 4 This is a schematic diagram illustrating a method for cleaning the first row of components according to an embodiment of this application. See also... Figure 5 , Figure 5 This is a schematic diagram illustrating the cleaning of the end position of the first row of components in an embodiment of this application. See also... Figure 6 , Figure 6 This is a schematic diagram of a component moving to the second row in an embodiment of this application. See also... Figure 7 , Figure 7 This is a schematic diagram illustrating the starting position of cleaning the second row of components in an embodiment of this application. See also... Figure 8 , Figure 8 This is a schematic diagram of a method for cleaning the second row of components according to an embodiment of this application. In the application scenario of cleaning photovoltaic modules using a tethered drone, the robot dog is equipped with a linear guide rail mechanism to drag the water pipe during its movement, and release or retrieve a fixed length of water pipe upon reaching the target point. The robot dog's main control and underlying control board serve as the operating carriers of the core control program of the collaborative architecture. The main control handles core logic such as path planning, task scheduling, and data fusion, while the underlying control board drives the linear guide rail, force sensor, encoder, and other execution components, matching the hardware logic of the collaborative architecture's "main control coordination + underlying execution." Simultaneously, the robot dog's tension detection and water pipe length measurement functions provide crucial feedback for the closed-loop control of the collaborative architecture, ensuring that the collaborative system can adjust its operational strategy in real time, guaranteeing the safety, accuracy, and efficiency of the overall operation.

[0037] S202: Perform spatiotemporal synchronization and feasibility verification on the global optimal path of the UAV and the global optimal path of the legged robot to obtain the first reference trajectory and the second reference trajectory.

[0038] The timestamps of the drones and the legged robots are synchronized.

[0039] After calculating the globally optimal paths for both the UAV and the legged robot, spatiotemporal synchronization and feasibility verification were performed on these paths to obtain a first reference trajectory and a second reference trajectory. By calculating and verifying the globally optimal paths for both the UAV and the legged robot, the feasibility of the obtained first and second reference trajectories was ensured.

[0040] Legged robots can use robot dogs to perform static task planning in parallel collaboration between drones and robot dogs. The static task planning process can include environmental benchmark establishment and position calibration, high-precision spatiotemporal benchmark (GNSS-RTK + time synchronization), and initial task trajectory planning.

[0041] Environmental baseline establishment and position calibration may include the acquisition of point cloud files and orthorectified image files. This can be achieved using a drone equipped with a visible light camera, solid-state LiDAR, or other data acquisition capabilities. The drone should possess centimeter-level positioning (RTK positioning function) and high-precision inertial navigation. Settings include (laser pulse emission frequency) sampling frequency, scanning mode, echo mode, laser lateral overlap rate, and visible light lateral overlap rate. Before data acquisition, inertial navigation unit calibration is performed to adjust inertial navigation accuracy. After data acquisition, the raw point cloud data undergoes preprocessing such as data parsing, noise reduction / filtering / downsampling, gimbal attitude correction and coordinate transformation, point cloud registration and mapping, and is exported as a post-processed .pcd file. High-resolution RGB images with POS data (position + attitude) from the drone aerial photography are stored. Aerial photos with POS data are imported using software tools (Pix4D, ContextCapture, etc.), aerial triangulation calculations and orthorectification are performed, and the orthorectified images are stitched together and output as GeoTIFF files.

[0042] High-precision spatiotemporal references can include high-precision GNSS positioning modules for both drones and robot dogs, each equipped with PPS signal output and combined with RTK technology, achieving centimeter-level positioning accuracy. The drone uses its built-in high-precision GNSS module to acquire its latitude and longitude coordinates and outputs PPS pulse signals and GPRMC statements via its interface. PPS provides nanosecond-level precision synchronization to the whole second, while GPRMC contains complete UTC time information (year, month, day, hour, minute, second), providing a highly reliable timestamp. The robot dog uses its built-in or extended GNSS module (such as the u-blox NEO-F9P) to acquire its latitude and longitude coordinates. Using the GNSS module's PPS output pin, it connects the pulse signal to the GPIO input pin of the robot dog's main control system, accurately timestamping the sensor data. The PPS itself does not contain a specific time; by matching it with the GPRMC statement received via the serial port, it achieves high-precision synchronization aligned with UTC time, using this time source as the robot dog's data timestamp. Global Navigation Satellite Systems broadcast high-precision atomic clock time signals, serving as a globally shared and extremely stable public time reference. Both PPS signals originate from the same satellite atomic clock time signal, with a time error within tens of nanoseconds.

[0043] Initial mission trajectory planning can utilize mission planning software (such as QGroundControl or DJI Pilot 2) within a common global coordinate system (WGS84 latitude and longitude). The UAV's flight path is planned, including parameters such as latitude, longitude, altitude, speed, and hovering time for the first, last, and turning waypoints of each row of components to be cleaned within the photovoltaic array. Based on the UAV's flight speed and the Euclidean distance between waypoints, the initial static expected arrival time for each waypoint is calculated. Based on the GeoTIFF orthophoto, a cooperative optimal path is planned for the robot dog within the nearest passable component gap along the drone's flight path. The static expected arrival time at each waypoint is calculated based on the robot dog's ground path length and speed range. Adjust the robot dog's movement speed to ensure that the expected arrival time of the drone and the robot dog at the corresponding waypoints is consistent. This achieves static time alignment during the task planning phase.

[0044] See Figure 9 , Figure 9 This is a schematic diagram of dynamic real-time optimization in an embodiment of this application. The basic preparatory work for synchronous operations needs to be optimized in real-time under dynamic conditions. Real-time optimization under dynamic conditions may include operation status monitoring (real-time feedback of speed differences), collaborative response difference testing and modeling, and model predictive controller (MPC) design.

[0045] The operational status monitoring process can include real-time speed monitoring and command issuance, using the FRD UAV body coordinate system uniformly. The coordinate transformation method is to obtain the speed in the NED coordinate system (a local geographic coordinate system with the UAV's location as the origin) through the UAV API. , , ].

[0046] The drone's velocity in the NED coordinate system can be converted to the FRD body coordinate system using the drone's yaw angle. In photovoltaic cleaning scenarios, the drone's attitude stability is minimal, and in practical applications, the simplified calculation formula only relates to the yaw angle, ignoring the effects of pitch and roll angles. The formula is as follows: ; ; ; in, This represents the velocity component of the UAV in the north direction in the NED coordinate system, with north being positive. This represents the velocity component of the UAV in the east direction in the NED coordinate system, with eastward being positive. Let be the vertical (downward) velocity component of the UAV in the NED coordinate system, with descent being positive. Yaw is the UAV's yaw angle. This represents the velocity component of the UAV in the nose direction within the FRD UAV body coordinate system, with positive for forward. This represents the velocity component of the UAV in the fuselage direction within the FRD UAV body coordinate system, with the right side being positive. This represents the vertical velocity component of the UAV in the FRD UAV body coordinate system, with downward velocity being positive.

[0047] The robot dog calculates its speed using an encoder or IMU. The robot dog's speed in the FLU body coordinate system can be obtained via the API, with the X-axis pointing directly in front of the robot dog's head, the Y-axis pointing to the left side of the robot dog, and the Z-axis pointing directly above the robot dog. To unify the robot dog's FLU body coordinate system speed to the UAV's FRD body coordinate system, a positive Y-axis and Z-axis transformation is performed, using the following formula: ; ; ; in, This represents the velocity component of the robot dog in the direction directly in front of its head in the FLU body coordinate system, with "directly in front" being positive. This represents the velocity component of the robot dog in the body direction within the FLU body coordinate system, with the left side being positive. This represents the vertical velocity component of the robot dog in the FLU body coordinate system, with upward being positive. This represents the velocity component of the robot dog in the direction directly in front of its head, within the FRD drone's body coordinate system, with "directly in front" being positive. This represents the velocity component of the robot dog along the fuselage direction in the FRD drone's body coordinate system, with the right side being positive. This represents the vertical velocity component of the robot dog in the FRD drone's body coordinate system, with downward velocity being positive.

[0048] The collaborative response difference testing and modeling process may involve addressing the response differences between the power systems of the drone and the robot dog, requiring speed range matching, calibration of their speed-command mapping relationship, and conducting the following speed response difference tests: (1) Acceleration test: Test and statistically analyze the response time of both from rest to different target velocities, and plot the acceleration curves.

[0049] (2) Deceleration / Braking Test: Test and statistically analyze the braking time and distance from different speeds to a stop. The braking distance is based on the original position data on the trajectory during the braking time. The Haversine formula was used to calculate the geodetic distance between each adjacent original location point. Then, the total braking distance is obtained by summing the geodetic distances between all adjacent points on the trajectory during the braking time.

[0050] (3) Speed ​​command-actual speed mapping: Send fixed different speed commands, measure and statistically analyze the final stable actual speed of both.

[0051] (4) Delay test: Test and count the communication and actuator delays from the issuance of the instruction to the start of the response.

[0052] (5) Record the above test dataset, including timestamp t and the speed command sent. ,speed Raw location data (latitude, longitude, and altitude) Raw location data After converting to the WGS84 geodetic coordinate system, similar to the velocity conversion method, the position status can be obtained by uniformly converting to the FRD UAV body coordinate system. .

[0053] MPC utilizes dynamic predictive models for feedforward control, reducing the differences in the power systems of the drone and the robot dog. It compensates for the latency in command transmission and execution by the robot dog, generating smooth and feasible acceleration commands by optimizing control constraints and penalty terms in the problem.

[0054] The design of a model predictive controller may include the following parts: (1) Establish a prediction model Based on the test dataset of speed response differences between drones and robotic dogs, a first-order inertial fitting is performed on the speed response characteristics of the drones / robotic dogs. The fitting formula structure is as follows: ; in, Represents the steady-state gain, derived from the stable value of the velocity curve. calculate, ; The speed command at time t; Let t be the actual output speed of the drone / robot dog at time t; Let be the time constant, representing the rate at which the speed increases from the moment the instruction is issued. The time required.

[0055] Transform the continuous-time model into the discrete-time model required for MPC: ; in, The inertia coefficient represents the weight of the current velocity's influence on the next velocity; it can be calculated separately for drones and robotic dogs. and ; The control gain coefficient represents the control weight of the current speed command on the speed at the next moment. It can be calculated separately for drones and robot dogs. and ; This is the MPC sampling time.

[0056] Therefore, the dynamic prediction model for UAVs is: ; in, Let be the speed of the drone at time t. It is the speed command at time t.

[0057] Therefore, the dynamic prediction model for the robot dog is: ; in, Let be the speed of the robot dog at time t. It is the speed command at time t.

[0058] (2) State-space model of cooperative system

[0059] Combining the two models into an augmented state-space model: ; in, The fourth-order augmented inertial coefficient matrix represents the position-velocity state autoregressive relationship between the UAV and the robot dog, with the states of the two decoupled. This is a 4×2 augmented control gain coefficient matrix, representing the control effect of the speed command on each state; Let t represent the position and state of the UAV at time t; Let t be the position and state of the robot dog at time t.

[0060] (3) Define the optimization problem

[0061] At each time step k, MPC needs to solve the following optimization problem: A. Optimization Objectives The objective is to minimize the cost function. Includes tracking error, control variables, and time synchronization parameters: ; Among them, the tracking error term Used to penalize deviations between the predicted trajectory and the desired trajectory, ensuring relative position synchronization throughout the entire prediction time domain from i=0 to... Summation above: ; Control Quantity Items Used to penalize excessively large control commands and smooth the motion, starting from the current time k, throughout the entire prediction time domain i=0 to... Summation above: ; Time synchronization item Its core function is to quantify time synchronization deviation; the larger the deviation, The larger the value, the higher the cost function. The larger the overall size, the more preferentially the optimizer will adjust the robot dog's control commands, making... Ensure real-time alignment.

[0062] Based on the static alignment of arrival times during the static task planning phase, the MPC controller needs to dynamically adjust the robot dog's movement speed to ensure that both arrive at the same work area simultaneously, achieving dual synchronization of spatial location and time node: ; in, The predicted position state of the UAV at time k+i; The predicted position state of the UAV at time k+i; To optimize the velocity command at time k+i; The prediction step size, i.e., the time domain included in the prediction. The number of steps, where i=0 is the current step. This is the final step in predicting the time domain; These are control weighting coefficients used to balance tracking accuracy and control strength; It is the time synchronization weighting coefficient; For the drone to reach the Real-time estimated time of waypoints; For the robot dog to reach the first Real-time estimated time of waypoints; This refers to the offset distance that needs to be maintained between the movement trajectories of the drone and the robot dog.

[0063] B. Constraints

[0064] Input constraints include: the maximum speed of the drone. Maximum acceleration The maximum speed of the robot dog Maximum acceleration .

[0065] C. Implementation process of rolling optimization and compensation

[0066] At time k, MPC acquires the current states of the drone and the robot dog, and derives the augmented state vector. : ; Based on the current state and the dynamic prediction models of the drone and robot dog respectively, MPC predicts the future of both. Position and velocity trajectory within a step.

[0067] Maintaining the desired relative distance between the drone and the robot dog To optimize the objective, a control term is used to penalize excessively large robot dog control commands to ensure smooth motion. At the same time, a time synchronization term is introduced to jointly construct the model predictive control optimization cost function.

[0068] MPC obtains the future by solving an optimization problem. Optimal control command sequence for each step .

[0069] MPC only transmits the control command from the first moment of the control sequence. The command is sent to the robot dog for execution. At the next time step k+1, MPC, based on the new and more accurate state, repeats the above rolling optimization process, outputs the robot dog's current control command, and executes it, continuously ensuring collaborative synchronization.

[0070] S203: Obtain the relative pose between the drone and the legged robot.

[0071] When using drones and legged robots to clean photovoltaic power plants in a coordinated manner, the relative poses between the drones and legged robots are obtained.

[0072] S204: Obtain the absolute global pose of the legged robot.

[0073] Obtain the absolute global pose of the legged robot.

[0074] In one specific embodiment of this application, step S204 may include the following steps: Step 1: Obtain the first pose of the legged robot in the previous time step, initialize the particle cloud based on the first pose, and determine the initialized particle cloud as the particle cloud of the previous time step. Step 2: Obtain the displacement difference and rotation difference from the previous time step to the current time step; Step 3: Using the adaptive Monte Carlo localization algorithm, based on the displacement difference and rotation difference, predict the second pose of each particle in the particle cloud at the current time step by running the model; Step 4: Determine the absolute global pose of the legged robot based on each second pose.

[0075] For ease of description, the four steps above can be combined for explanation.

[0076] The algorithm acquires the first pose of the legged robot at the previous time step and initializes the particle cloud based on this pose. This initialized particle cloud is then designated as the particle cloud of the previous time step. The displacement and rotation differences from the previous to the current time step are obtained. Using an adaptive Monte Carlo localization algorithm, based on these displacement and rotation differences, the algorithm predicts the second pose of each particle in the particle cloud from the previous time step at the current time step using a running model. The absolute global pose of the legged robot is then determined based on these second poses. By utilizing the adaptive Monte Carlo localization algorithm to predict the second pose of each particle in the particle cloud from the previous time step at the current time step using a running model, the absolute global pose of the legged robot is determined, improving both the efficiency and accuracy of absolute global pose determination.

[0077] In one specific embodiment of this application, determining the absolute global pose of the legged robot based on each second pose may include the following steps: Step 1: Generate a likelihood field representing the probability of occurrence of the measurement point based on each second pose; where the measurement point is the point where the scanning ray hits the obstacle; Step 2: Obtain the current scan data and convert it to the global map coordinate system according to each second pose to obtain each location point to be queried; wherein, the current scan data includes the current measured distance corresponding to each current scan ray; Step 3: Find the probability value of each current scanning ray at each query position point from the likelihood field; where each probability value is the probability that the end point of each current scanning ray appears at each query position point when it is in each second pose. Step 4: Calculate the particle weights corresponding to the particles in each second pose based on the probability values; Step 5: Resample the particles in each second pose according to their weights to obtain the resampled particles; Step 6: Determine the absolute global pose of the legged robot based on the second pose corresponding to each resampled particle.

[0078] For ease of description, the above six steps can be combined for explanation.

[0079] After predicting the second pose of each particle in the particle cloud at the previous time step using the running model, a likelihood field representing the probability of the occurrence of a measurement point is generated based on each second pose. The measurement point is the point where the scanning ray hits the obstacle. Current scanning data is acquired, including the current measurement distance for each current scanning ray. This data is then converted to the global map coordinate system based on each second pose to obtain the query location points. The probability values ​​for each current scanning ray at each query location point are retrieved from the likelihood field. Each probability value represents the probability that the endpoint of each current scanning ray in each second pose appears at that query location point. Particle weights are calculated for each particle in each second pose based on these probability values. The particles in each second pose are then resampled based on these weights to obtain resampled particles. The absolute global pose of the legged robot is determined based on the second poses corresponding to each resampled particle. By resampling based on particle weights and determining the absolute global pose of the legged robot based on the second poses corresponding to each resampled particle, the accuracy of the determined absolute global pose is further improved.

[0080] In one specific embodiment of this application, step S203 may include the following steps: The relative pose between the drone and the legged robot is calculated using visual algorithms; Accordingly, determining the absolute global pose of the legged robot based on the second pose corresponding to each resampled particle can include the following steps: Step 1: Calculate the relative pose between the drone and the legged robot using visual algorithms; Step 2: Determine the predicted global pose of the legged robot based on the second pose corresponding to each resampled particle; Step 3: Obtain the global position provided by the UAV positioning system, and calculate the visual global pose of the legged robot based on the relative pose and the global position; Step 4: Use a preset state estimation algorithm to perform state estimation correction on the predicted global pose and the visual global pose, and output the absolute global pose.

[0081] For ease of description, the four steps above can be combined for explanation.

[0082] The relative pose between a drone and a legged robot is calculated using a visual algorithm. Particles in their respective second poses are resampled according to their weights. After obtaining the resampled particles, the relative pose between the drone and the legged robot is calculated again using the visual algorithm. The predicted global pose of the legged robot is determined based on the second poses corresponding to each resampled particle. The global position provided by the drone's localization system is obtained, and the visual global pose of the legged robot is calculated based on the relative pose and global position. A preset state estimation algorithm is used to perform state estimation correction on the predicted global pose and the visual global pose, outputting the absolute global pose. By using a visual algorithm to calculate the relative pose between the drone and the legged robot, calculating the visual global pose of the legged robot based on the relative pose and global position, and then using a preset state estimation algorithm to perform state estimation correction on the predicted global pose and the visual global pose, the accuracy of the obtained absolute global pose is further improved.

[0083] The preset state estimation algorithm can be an Extended Kalman Filter (EKF), an Unscented Kalman Filter (UKF), an Error-State Kalman Filter (ESKF), or an Incremental Smoothing and Mapping (iSAM2) algorithm based on factor graphs.

[0084] S205: Using a model predictive controller based on the dynamics models of a legged robot and an unmanned aerial vehicle (UAV), predict the cooperative state of the UAV and the legged robot.

[0085] A model predictive controller is used to predict the cooperative state of a drone and a legged robot based on a legged robot dynamics model and a drone dynamics model.

[0086] S206: The relative pose, absolute global pose and cooperative state are fused using a preset state estimation algorithm to obtain the globally consistent pose of the UAV and the legged robot.

[0087] Globally consistent pose includes the 3D position of the legged robot, the pose quaternion of the legged robot, the 3D position of the UAV, and the pose quaternion of the UAV.

[0088] After obtaining the relative pose between the UAV and the legged robot and the absolute global pose of the legged robot, and predicting the cooperative state of the UAV and the legged robot, a preset state estimation algorithm is used to fuse the relative pose, absolute global pose, and cooperative state to obtain the globally consistent pose of the UAV and the legged robot. The globally consistent pose includes the 3D position of the legged robot, the pose quaternion of the legged robot, the 3D position of the UAV, and the pose quaternion of the UAV.

[0089] S207: Determine the global consistent pose as the current state.

[0090] After obtaining the globally consistent poses of the UAV and the legged robot, these poses are determined as the current state. By fusing the relative pose, absolute global pose, and cooperative state using a pre-defined state estimation algorithm, a globally consistent pose is obtained, including the 3D position and attitude quaternions of both the legged robot and the UAV. This improves the comprehensiveness of the determined current states of the UAV and the legged robot.

[0091] S208: The model predictive controller predicts control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current state.

[0092] See Figure 10 , Figure 10 This is a flowchart illustrating a real-time accessibility verification process according to an embodiment of this application. The real-time accessibility verification process may include environment modeling (OctoMap generation and point cloud processing), map transmission, robot dog localization (AMCL fusion), and robot dog visual tracking and target locking.

[0093] 1. Environment Modeling (OctoMap Generation and Point Cloud Processing): Due to the large volume of raw point cloud data, direct transmission requires high bandwidth. The point cloud file is converted to OctoMap, and the root node of an octree is determined based on the spatial extent of the work area. The work area is then recursively divided into 3D voxels of different resolutions using octrees. A 3D voxel is the smallest spatial unit in OctoMap. Only voxels containing obstacles are recursively subdivided to a preset minimum resolution, while larger voxels are retained in uniform areas. Each voxel is labeled as "occupied (with obstacles)," "idle (no obstacles)," or "unknown (unobserved)." This results in a smaller map size that directly includes obstacle information, making it more suitable for path planning.

[0094] This implementation uses the PCL library and ROS framework to convert point clouds into OctoMap. ROS package code is written to read local .pcd format point cloud files using the PCL library, parse them into point cloud data structures, encapsulate them into the ROS standard PointCloud2 message type, and publish them to a specified topic (e.g., / cloud_in). After the octomap_server node subscribes to this topic, it automatically completes the voxelization and octree construction of the point cloud, recursively divides the work area into 3D voxels according to a preset resolution, and labels the "occupied / idle / unknown" status of each voxel. Finally, a lightweight OctoMap is generated for use by the path planning modules of drones and robotic dogs.

[0095] 2. Map transmission: 1) Route 1: Direct real-time communication for power plant scenarios with stable networks The drone and robot dog are connected to the same local area network (such as a power plant's dedicated Wi-Fi), and the OctoMap map is transmitted in real time using a "topic publish / subscribe" mode based on the ROS framework.

[0096] On the drone side: After the octomap_server node completes the conversion of point cloud to OctoMap, it will continuously and periodically publish the / octomap_binary topic. This topic is the default OctoMap binary publication topic for the octomap_serve node. The message type is octomap_msgs::Octomap, which encapsulates the octomap octree structure, voxel resolution, and core information such as "occupied / idle / unknown" status of OctoMap. Moreover, the binary encoding format greatly reduces the transmission volume, which is suitable for the real-time communication needs of power plant scenarios.

[0097] On the robot dog side: It listens for the / octomap_binary topic published by the drone through the ROS subscription mechanism. After receiving an octomap_msgs::Octomap type message, it calls the octomap_msgs::Octomap service interface provided by the local octomap_server node to parse the received binary OctoMap data and load it into the robot dog's local map server to complete the synchronization of the environmental map and provide a unified environmental benchmark for the robot dog's path planning.

[0098] 2) Route Two: File sharing for power plant scenarios with unstable networks

[0099] (1) Save map file: On the drone side, use the octomap_server command to save the built OctoMap as a file (usually in .bt or .ot format).

[0100] (2) File transfer: Transfer the my_map.bt file to the host computer (onboard computer) of the robot dog by means of cloud storage or USB flash drive copy.

[0101] 3. Robot dog positioning (AMCL fusion): When the robot dog starts its navigation function, it loads the OctoMap file by specifying the path in the launch file. After loading, it uses the Adaptive Monte Carlo positioning algorithm (AMCL) to fuse motion predictions from the leg odometry and updates from LiDAR scanning measurements, thereby determining its optimal pose from the prior OctoMap. The specific process is as follows: (1) Initialization: The robot dog's starting position is known (at a fixed starting point). In RViz, a rough initial pose is manually given through "2D Pose Estimate" to initialize the particle cloud to a small area near the starting point, thus accelerating algorithm convergence.

[0102] (2) Prediction stage: During the robot dog's movement, AMCL uses motion data output from the leg odometry system, specifically the displacement and rotation differences (Δx, Δy, Δθ) from the previous moment to the current moment. It predicts the movement and diffusion of the particle cloud using a motion model, simulating positional uncertainty. For each particle, if it was at position P0 in the previous moment, its updated position is P0 + (Δx, Δy, Δθ).

[0103] (3) Measurement update phase: Step 1: Before actual localization begins, generate a likelihood field based on OctoMap processing: 1) Binarization map: Convert OctoMap or point cloud map into a binary obstacle map (obstacles are 1, and empty areas are 0).

[0104] 2) Distance transformation: Calculate the Euclidean distance from each grid cell in the binary map to the nearest obstacle.

[0105] 3) Distance value to probability value conversion: The distance value from the voxel center to the nearest point cloud obstacle is converted into an occupancy probability value. The distance value represents the spatial distance between the voxel and the obstacle surface, and the probability value quantifies the likelihood that the voxel is "occupied" by the obstacle (value range 0~1). An exponential decay function is used for the conversion. ; Where P is the probability value of the voxel being occupied; d is the distance from the voxel center to the nearest point cloud point. This represents the maximum occupancy probability. The attenuation coefficient is adjusted according to the obstacle accuracy in the power plant scenario. The smaller the distance value (closer to the obstacle), the closer the occupancy probability value is to 1, and it is judged as "occupied" state; the larger the distance value (farther from the obstacle), the closer the occupancy probability value is to 0, and it is judged as "idle" state.

[0106] Finally, a likelihood field (probability heatmap) is generated to characterize the probability of the occurrence of laser measurement points (a laser measurement point is the point on the obstacle where each laser ray hits). The higher the heat in the graph, the higher the probability of observing a laser measurement point at that point.

[0107] Step 2: Online calculation of particle weights during the localization process: LiDAR outputs scanning data in real time , ( Let k be the measured distance of the k-th ray. For a single laser scan, the weight of each particle is calculated using the following steps: 1) Coordinate transformation: Assuming the pose represented by the particle is .

[0108] laser scanning point Transform from the lidar coordinate system to the global map coordinate system.

[0109] The transformation formula is: ; ; in, It is the angle between the k-th ray and the direction of the machine dog head.

[0110] 2) Probability query: Query in the likelihood field The corresponding probability value , Indicates the pose The endpoint of the kth ray appears at this position. The probability of.

[0111] 3) Weight calculation: K laser beams are emitted at once. Assuming independent measurements, log-likelihood is used to avoid numerical underflow. The weighting formula is as follows: ; The particle weight is positively correlated with the laser scanning matching degree; the higher the matching degree, the greater the weight.

[0112] (4) Resampling: Based on the particle weights, resampling is performed, low-weight particles are eliminated, and high-weight particles are replicated, keeping the total number of particles unchanged; after iteration, the particle cloud gradually gathers near the robot dog's actual position.

[0113] (5) Output pose estimation: Calculate the weighted average pose of the particle cloud (or select the pose of the particle with the highest weight) as the current optimal pose of the robot dog, and publish the absolute pose (xamcl, yamcl, θamcl) at a frequency of 10~20Hz in ROS topics (such as / amcl_pose) for the navigation stack to call.

[0114] 4. Visual tracking and target locking of the robot dog: (1) Preparation: Visual tagging: Attach the AprilTag digital QR code to the drone tripod.

[0115] Software preparation: Run ROS on the robot dog's built-in main controller, use the readily available apriltag_ros or aruco_ros package to identify tags, and write PID control nodes to issue speed commands to the robot dog.

[0116] Training data: Collect images of cleaning drone operations with visual markers from different angles, under different lighting conditions, at different distances, and against different backgrounds, and manually annotate them.

[0117] Data augmentation: flipping, rotating, changing brightness, etc.

[0118] (2) Training the detection model: Object detection model: A lightweight YOLOv8n real-time object detection model is used. It runs in real time on the robot dog's main control system.

[0119] Model optimization: Use tools such as TensorRT to accelerate neural network inference.

[0120] Tracking and detection scheme: Based on detection, the ByteTrack tracking algorithm is used to maintain the target's ID in the video stream by utilizing motion information and appearance features, ensuring tracking stability and reducing computational load (it is not necessary to run a complete detection in every frame). The target is then retrieved and associated again after it disappears briefly.

[0121] (2) Perception node: The robot dog uses an onboard camera (RGB camera) to input the camera image stream and identify and track the drone's visual markers in real time.

[0122] Run a ROS node on the robot dog. The node: Subscribe to the camera image topic ( / camera / image_raw) and load the trained detection model.

[0123] Inference is performed on each frame of the image to obtain the detection box.

[0124] The center point coordinates of the detection box ( , The height h of the bounding box is posted to a new topic (e.g., / uav_detection).

[0125] (3) Control node: Another ROS node subscribes to / uav_detection: 1) State estimation: Based on the Boolean value indicating the presence of a visual marker and the coordinates of the detection box, the distance and orientation of the drone relative to the robot dog are estimated. Using computer vision algorithms, the precise 3D position, orientation, and distance of the marker relative to the camera are calculated.

[0126] The distance is estimated based on the known physical dimensions of the cleaning drone (e.g., wheelbase) and its pixel size in the image.

[0127] Calculate the (X, Y, Z) offset and velocity of the cleaning drone relative to the robot dog. 6-DOF pose (3D translation (x, y, z) and 3D rotation (roll, pitch, yaw)).

[0128] 2) PID control: Calculation error: Lateral error: The horizontal distance of the digital QR code from the center of the image reflects whether the robot dog is facing the drone's digital QR code.

[0129] Distance error: The deviation between the actual distance between the robot dog and the drone and the expected distance, reflecting the spacing deviation between the robot dog and the drone.

[0130] The robot dog's internal PID controller: Control the robot dog to rotate left and right, keeping the drone in the center of the screen. Control the robot dog to move forward and backward, maintaining the distance from the drone. The controller input is the desired following distance and the currently detected actual position of the drone.

[0131] The controller outputs the speed commands (forward / backward speed, left / right translation speed, rotation speed) executed by the robot dog.

[0132] Two types of calculation errors are input as closed-loop feedback signals to the internal PID controller. The PID controller calculates the control quantity based on the error and eliminates the error. The lateral channel PID outputs a rotation control quantity based on the lateral error, ensuring the drone remains centered in the frame. The distance channel PID outputs a forward / backward motion control quantity based on the distance error, ensuring the robot dog maintains the desired distance from the drone. This ensures the digital QR code remains centered in the frame, guaranteeing the robot dog faces the drone and maintains a fixed distance.

[0133] (4) The calculated speed command (Twis message) is published to the robot dog's control topic (e.g., / cmd_vel). The robot dog receives the / cmd_vel topic and executes the movement.

[0134] S209: Control the legged robot to release or retrieve water pipes via control commands to clean the photovoltaic power station.

[0135] By planning matching, spatiotemporally related routes for both drones and legged robots, they can execute autonomously according to the plan, synchronize in time, and arrive at key points on the path simultaneously, thereby releasing or retrieving the water pipe accordingly.

[0136] S210: When an abnormality in releasing or dragging the water pipe is detected, an abnormality report shall be made.

[0137] During the process of controlling the legged robot to release or retrieve the water pipe via control commands, the robot performs release or dragging detection. When an abnormality in the release or dragging of the water pipe is detected, an anomaly report is generated. By reporting abnormalities in the release or dragging of the water pipe, timely early warning of abnormal situations is achieved.

[0138] Security anomaly handling mechanisms may include the following parts: (1) When the force sensor detects a clamping force of ≥8kg, the slide rail will stop immediately, the clamp will remain in place, and the abnormality will be reported to the main controller / UAV to avoid damaging the water pipe.

[0139] (2) When the dual force sensor detects a tension of ≥20kg, the robot dog immediately stops moving, the slide rail remains clamped, the drone stops operating and alarms to prevent overload or water pipe from falling off.

[0140] (3) When the force sensor detects that the tension drops to ≤0.5kg, the robot dog stops, the slide rail releases the clamp (or holds), and reports the water pipe detachment fault to avoid ineffective operation.

[0141] (4) Encoder metering L n When the distance reaches N+0.01m, the slide rail immediately stops extending and retracting to ensure precise extension and retraction.

[0142] (5) If the clamping operation fails (clamping force is not 3-5kg, water pipe falls off), the task scheduling node will trigger a retry (3 times). Before each retry, the slide rail position will be repositioned or calibrated. (6) If the length of the extension and retraction is less than N meters (error exceeds the threshold), after the slide rail stops, the encoder data is reread and calibrated, and the extension and retraction are performed again. N is the distance between the end point of the cleaning route of the current row of components being cleaned and the starting point of the cleaning route of the next row of components to be cleaned. The static task planning is calculated and sent to the main control system in advance.

[0143] (7) If the retry fails 3 times in a row, immediately report the abnormality to the UAV and the host computer, terminate the current component operation, and terminate the task.

[0144] It should be noted that the threshold values ​​of the parameters corresponding to the above parts can be set and adjusted according to the actual situation, and this application embodiment does not limit this.

[0145] To achieve high-precision and robust collaborative operation between the UAV and the robot dog, a hierarchical closed-loop data fusion control architecture is constructed. The first layer is global spatiotemporal joint path planning, which generates a globally optimal, timestamped reference trajectory offline. This provides the UAV and robot dog with a time-synchronized, globally optimal path benchmark, avoiding local optima traps, but it struggles to handle dynamic environmental disturbances. The second layer is Model Predictive Control (MPC), which uses the global reference trajectory and the robot's current state as input, performing rolling optimization within the prediction time domain to output smooth control commands that satisfy dynamic constraints. This achieves short-term trajectory tracking and dynamic error compensation, but is limited by the prediction field of view and model accuracy. The third layer is robot dog visual tracking, which identifies the AprilTag markers on the UAV and calculates the UAV's 3D relative position and attitude with the robot dog in real time. This offers advantages such as centimeter-level accuracy, low latency, no cumulative error, and independence from global positioning, providing high-precision relative state observation.

[0146] Building upon this layered architecture, an Extended Kalman Filter (EKF) is introduced to construct a data fusion layer. This layer fuses high-frequency relative measurements from visual tracking, AMCL global localization results, and MPC predicted states, outputting a high-precision pose estimate above 30Hz. The MPC controller, based on the fused optimal state and the global spatiotemporal trajectory, solves a constrained optimization problem and outputs control commands. Simultaneously, it feeds back the predicted state to the fusion layer, forming a closed-loop structure of global planning—data fusion—real-time control—visual feedback. This architecture combines global optimality with real-time disturbance compensation capabilities, supports multi-mode adaptive switching and graceful degradation in the event of sensor failure, ultimately achieving high-precision, robust, and active cooperative tracking in the UAV-robot dog system.

[0147] Static task planning under parallel collaboration provides a global, optimal, and time-synchronized reference trajectory. It generates separate paths for the drone and the robot dog, with each point on the path having a timestamp. While globally optimal, it avoids local optima traps, but cannot handle real-time dynamic disturbances such as newly added obstacles.

[0148] The MPC controller is responsible for short-term, real-time trajectory tracking and dynamic compensation. Based on the input global reference trajectory and current state (position, velocity), it outputs smooth, feasible control commands that satisfy dynamic constraints. It can anticipate dynamics several steps ahead, proactively compensate for discrepancies, and handle constraints. However, the number of prediction steps is short, and it depends on the accuracy of the model.

[0149] The robot dog visual tracking provides high-precision, low-latency relative state measurement between the UAV and the robot dog. It outputs the UAV's three-dimensional relative position and relative velocity relative to the robot dog. The accuracy is extremely high (centimeter-level), does not rely on global positioning systems such as GPS, and has no cumulative error. However, it depends on the field of view and may be subject to occlusion. The robot dog visual tracking is highly coupled with a hierarchical closed-loop data fusion control architecture: visual tracking provides centimeter-level, low-latency, and error-free relative pose measurement between the UAV and the robot dog, serving as the core observation input for the EKF data fusion layer, while also providing high-precision deviation feedback for MPC real-time trajectory tracking and time synchronization constraints; the global spatiotemporal flight path provides a timestamped optimal reference trajectory; MPC is responsible for short-term dynamic compensation; and visual tracking is responsible for measuring actual collaborative deviations. Together, these three elements form a complete closed loop of "global planning—data fusion—real-time control—visual feedback," enabling high-precision and robust spatiotemporal collaborative operation between the UAV and the robot dog.

[0150] See Figure 11 , Figure 11 This is a diagram of a hierarchical closed-loop data fusion control architecture in an embodiment of this application.

[0151] (1) First layer: Global path planning layer (offline / low frequency).

[0152] The "time-synchronized dual-track" (drone + robot dog) generated by the global path planning layer (offline / low-frequency online) directly provides core task instructions for the robot dog's operation. The robot dog's navigation node receives the target point coordinates (the designated location for releasing / retrieving the water pipe) issued by the planning layer. At the same time, the planning layer optimizes the dragging path based on parameters such as the robot dog's traction force (≥20kg) and the water pipe load (1kg / m), avoiding working conditions such as steep slopes and obstacles that may cause the traction force to exceed the limit, ensuring that the path planning is compatible with the robot dog's hardware performance and operational requirements. The N-value (the distance between the two rows of component cleaning routes) is also issued synchronously by the planning layer, serving as the core control basis for the length of water pipe deployment and retrieval.

[0153] 1) Input: Task objectives: start / end point, task type, performance metrics; Environment map: static obstacles, terrain information, feasible area (generated based on OctoMap); Constraints: Based on the equipment parameters such as the steering motor, communication module, and camera parameters of the drone and robot dog, combined with field tests and verification, kinematic constraints (maximum speed, acceleration), dynamic constraints (steering ability, attitude limitation), cooperative constraints (relative distance, communication range, field of view constraint), and operational constraints (traction force threshold, load parameters, and the baseline value N for the retraction and extension length) are obtained.

[0154] A. Kinematic constraints (maximum velocity, acceleration) need to be obtained through a combination of equipment parameter calibration and experimental testing.

[0155] Robot dog: The theoretical maximum speed and acceleration (initial reference values) given in the equipment manual are corrected to the actual achievable kinematic limits through actual field testing (such as the influence of ground friction and slope, which may cause the actual maximum speed to be lower than the theoretical value).

[0156] Unmanned Aerial Vehicles (UAVs): By combining the parameters of the UAV's power system (motor power, propeller efficiency), hovering and constant speed flight tests are conducted to determine its maximum level flight speed, vertical speed and corresponding acceleration constraints in three-dimensional space, so as to avoid insufficient power that would prevent the trajectory from being executed.

[0157] B. Dynamic constraints (steering capability, attitude limitations) are obtained through dynamic modeling and experimental calibration.

[0158] Steering capability (robot dog): Based on the robot dog's chassis structure and steering motor performance, a dynamic model is established. The maximum steering angular velocity and steering angle range are obtained through steering tests to ensure that the trajectory steering command is within the robot dog's mechanical limits.

[0159] Attitude Limitations (Drone + Robot Dog): The drone collects flight attitude data through IMU sensors to calibrate its maximum roll and pitch angles (to prevent instability). The robot dog calibrates the fuselage attitude limits through joint sensors to ensure that the attitude angles required for the trajectory do not exceed the mechanical structure's tolerance.

[0160] C. Some of the collaborative constraints (relative distance, communication range, and field of view constraints) can be obtained directly, while others require experimental calibration.

[0161] Communication range: Refer to the technical parameters of the communication modules (such as WiFi and LoRa) between the drone and the robot dog to determine the maximum communication distance as the upper limit of the collaborative constraint.

[0162] Relative distance: Combining the field of view of the vision system and task requirements (such as collaborative work distance), a safe relative distance range is determined through experiments (ensuring effective visual recognition while avoiding collisions).

[0163] Field of view constraints: By calibrating the camera parameters (focal length, field of view angle) of the robot dog, the maximum distance and angle range of the drone within the camera's field of view are determined to ensure that the drone is always within the visually recognizable area of ​​the robot dog during the trajectory.

[0164] D. Job constraints.

[0165] The maximum traction force threshold of the robot dog (≥20kg) should be avoided when planning the dragging path. Steep slopes, muddy terrain, and other terrains that may cause the traction force to exceed the limit should be avoided to prevent the robot dog from being overloaded or the water pipe from falling off.

[0166] The water pipe load parameter (1kg / m) + the linear guide rail rated load (≥2kg / m) limits the maximum length of the water pipe to be dragged in a single operation, thus preventing deformation of the guide rail mechanism and failure of the drive system.

[0167] The baseline value N for the water pipe extension / retraction length is used to plan the segmented displacement of the release / retraction operation, ensuring that the extension / retraction length is accurately N meters (the distance between the two rows of component routes) for each extension / retraction operation, and to connect multiple rows of component cleaning tasks.

[0168] 2) Processing procedure: Trajectory generation: Calculate the globally optimal path based on optimization algorithms (such as MINCO, nonlinear programming).

[0169] Spatiotemporal synchronization: Ensure that the two trajectories of the drone and the robot dog are strictly matched in time and space to guarantee coordination and consistency.

[0170] Feasibility verification: Check whether the trajectory meets all preset constraints to ensure that the trajectory is executable.

[0171] The specific verification steps are as follows: A. Single-device constraint verification: The trajectories of the drone and the robot dog are independently verified to ensure that each device's trajectory meets its own kinematic and dynamic constraints. This includes: Velocity / acceleration verification: Extract the velocity and acceleration values ​​of each timestamp on the trajectory and compare them with the calibrated maximum velocity and acceleration. If there are nodes that exceed the constraints, the trajectory is smoothed and corrected. Attitude / steering verification: Extract the attitude angles and steering angular velocities on the trajectory, compare them with the calibrated attitude limits and maximum steering capability, eliminate trajectory segments that exceed the constraints, and replan the path for that segment; Mechanical limit verification: Based on the mechanical structure parameters of the equipment, verify that the required range of motion (such as the joint angle of the robot dog, the propeller speed of the drone) does not exceed the mechanical tolerance range.

[0172] B. Cooperative Constraint Verification: Verify the cooperative consistency of the two trajectories to ensure they meet the cooperative constraint requirements. This includes: Relative distance verification: Calculate the relative distance between the drone and the robot dog at each timestamp to ensure that it is within the preset safe relative distance range, neither exceeding the communication range nor falling below the safe collision distance; Field of view constraint verification: Based on the poses of the UAV and robot dog on the trajectory, calculate the position of the UAV within the field of view of the robot dog's camera to ensure that the UAV is always in the visually recognizable area and avoid visual loss leading to collaborative failure. Time synchronization verification: Verify that the timestamps of the two tracks are completely synchronized to ensure that the pose and speed of the drone and the robot dog match at the same time point, and avoid coordination deviation caused by time misalignment.

[0173] C. Environment Adaptation Verification: Based on the input environment map (static obstacles, terrain information), verify that the trajectory path does not cross static obstacles or exceed the feasible area, and adapt to the terrain slope (e.g., the slope of the robot dog's trajectory does not exceed its climbing ability).

[0174] D. Redundancy verification and correction: For trajectory segments that exceed the constraints found during verification, a trajectory smoothing algorithm (such as B-spline interpolation) is used for correction. After correction, the above verification process is re-executed until the trajectory meets all constraints. If the constraints still cannot be met after correction, the global path is regenerated and the planning algorithm parameters (such as path node density) are adjusted.

[0175] E. Mechanism Adaptability Verification: After generating the trajectory, verify simultaneously whether the robot dog's gait speed is adapted to the uniform dragging requirement (0.2-0.5m / s); whether the linear slide rail extension speed matches the robot dog's movement speed (avoiding water pipe bending and sudden tension changes); whether the extension / retraction length N matches the slide rail travel and the robot dog's positioning accuracy (≤5cm); if the verification fails, automatically optimize the trajectory parameters to ensure hardware compatibility.

[0176] Obtaining constraints must be completed in advance (experimental calibration, parameter configuration) as a prerequisite for global path planning. Feasibility verification and mechanism adaptability verification must be completed after trajectory generation and before output to the MPC controller. These are key steps to ensure trajectory execution and avoid equipment damage or collaborative failure. They work in conjunction with subsequent multi-mode adaptive mechanisms to further enhance system robustness.

[0177] 3) Output: A. Under ideal conditions with no real-time errors, the offline low-frequency generated global optimal reference trajectory and the time-synchronized dual trajectory. and The specific form is as follows: Drone reference trajectory: ; Robot dog reference trajectory (strictly synchronized with the drone): ; in, , The three-dimensional positions of the drone and the robot dog; , For the attitude quaternions of the drone and the robot dog; , Linear velocity for the drone and the robot dog; , The angular velocities of the drone and the robot dog; For timestamps.

[0178] and It serves as the input to the MPC controller, providing the robot dog and drone with a globally optimal, spatiotemporally synchronized ideal target trajectory, guiding the MPC's tracking direction.

[0179] B. Output machine dog operation segmentation instructions, including: Precise coordinates of the robot dog's target point (release / retrieval operation position A / B, corresponding to the initial position of the linear guide rail); Segmented drag / drop commands (e.g., drag to target point → pause → execute N-meter release / retraction → drag to the next row starting point); The extension / retraction direction of the linear guide rail and the displacement threshold (corresponding to the release / retraction operation logic).

[0180] (2) Second layer: Multi-source data fusion layer (30Hz+ high frequency).

[0181] The multi-source data fusion layer's core data sources include hardware sensor data from the robot dog, such as tension data from the robot dog platform's force sensors and the single-axis force sensor of the linear guide mechanism, as well as displacement (pipe length) data from the servo motor encoder. This data is fused with the UAV's GPS pose and the robot dog's AMCL positioning data to output a high-frequency, globally consistent pose, providing accurate data support for target point arrival determination at the robot dog's navigation nodes and command issuance at the task scheduling nodes. Simultaneously, the fusion layer's adaptive weighting mechanism dynamically adjusts the fusion strategy based on real-time data from the force sensors and encoders, ensuring positioning and measurement accuracy during robot dog operations.

[0182] 1) Input source: A. Vision system (30-60Hz): Provides centimeter-level relative pose between the drone and the robot dog. Its advantages are high frequency, high precision, and no cumulative error. Its limitation is that it depends on the field of view and may be occluded (based on AprilTag recognition).

[0183] B. AMCL positioning (10-20Hz): Provides the robot dog's absolute global pose. Its advantages are global consistency and stability. Its disadvantages are that it may drift and has a low frequency (based on OctoMap positioning).

[0184] C. MPC Prediction Status (20-30Hz): The MPC controller predicts and outputs short-term actual state estimates (including error compensation and real-time motion) based on the dynamic models of the robot dog and the drone, providing model-based short-term cooperative state prediction for both the robot dog and the drone, and helping to improve the fusion accuracy. It is based on the dynamic model of a robot dog and a drone, predicting their coordinated state in the short term (within the MPC prediction time domain). The core focus is on the robot dog's state (MPC primarily controls the robot dog to track the drone), while simultaneously predicting the drone's state (to calculate relative pose error and maintain spatiotemporal synchronization), ensuring consistency in coordinated control. The corresponding EKF state vector is fed back to the second EKF fusion layer to assist in correcting the fusion state. This improves pose estimation accuracy and provides short-term prediction basis for MPC's own rolling optimization, specifically in the following form: ; in, For location indicators (the three-dimensional spatial positions of the robot dog and the drone, respectively); These are attitude indicators (roll angle, pitch angle, yaw angle, describing the attitude state of both). It is a linear velocity index (the speed at which both move in three dimensions). This refers to the angular velocity index (the rotational speed of both around the three-dimensional coordinate axes).

[0185] D. Homework feedback data: Linear guide rail servo encoder: motor rotation count, real-time displacement (corresponding to water pipe extension / retraction length L) n This allows for precise calibration of the robot dog's positioning error and supplementation of global pose information in the "displacement dimension".

[0186] Dual-force sensor data: platform force sensor (drag tension) and slide rail force sensor (clamping force), real-time monitoring of clamping force (3-5kg) and drag tension (≤20kg), serving as the core basis for fusion weight adjustment.

[0187] Slide rail mechanism status: clamp opening and closing status, push rod in place signal, verify whether the operation execution is in place, and avoid false status of "positioning in place but mechanism not in place".

[0188] 2) Fusion mechanism: Extended Kalman Filter (EKF).

[0189] (20-30Hz) Feedback is sent to the EKF multi-source data fusion layer, where it is fused with the relative pose of the vision system and the absolute pose of the robot dog from AMCL, to correct the globally consistent pose of the fused state. This improves the real-time performance and accuracy of pose estimation.

[0190] Adaptive weights: visual weights (Image quality, QR code visibility); AMCL weights (Location covariance, environmental characteristics); MPC weights (Determined by model confidence and tracking error); Force / displacement weight coefficient: When the dragging tension approaches the 20kg threshold or the water pipe length measurement data fluctuates greatly, the GPS / visual weight is automatically reduced and the force sensor / encoder weight is increased to avoid operational errors caused by positioning errors.

[0191] Abnormal data filtering module: For abnormal data from force sensors (clamping force exceeding 8kg, sudden drop in tension) and encoders (displacement jump), a filtering threshold is set to prevent abnormal data from interfering with the fusion results and affecting the robot dog's navigation and task scheduling.

[0192] 3) Output: High-frequency state estimation (30Hz+) refers to the globally consistent pose of the robot dog and the drone in their coordinated states (the core being the robot dog's state, synchronously correlated with the drone's state, adapting to the requirements of coordinated control). (When the vision system is working normally, it can achieve centimeter-level accuracy). ; in, This represents the robot dog's three-dimensional position. Let be the quaternion of the robot dog's pose; This represents the three-dimensional position of the drone; Let be the attitude quaternion of the drone.

[0193] Simultaneously outputs integrated operational status data, including pipe clamping status (normal / abnormal, clamping force value); and extension / retraction length measurement results (L). n The difference between N and N); the coordination status between the robot dog and the slide rail (whether they are synchronized, whether the mechanism is in position). This provides accurate job execution feedback for the model prediction control layer and task scheduling nodes.

[0194] (3) Third layer: Model prediction control layer (high frequency decision).

[0195] The Model Predictive Control (MPC) layer outputs control commands that directly drive the robot dog's actions. Based on the current pose output by the fusion layer and the reference trajectory of the global planning layer, the MPC controller issues drag, stop, release, and retraction commands to the robot dog's task scheduling node. Simultaneously, it optimizes control commands by considering parameters such as the robot dog's traction force threshold (20kg) and the linear guide rail's load capacity, ensuring uniform dragging and stable tension. When the MPC controller detects excessive dragging tension or abnormal water pipe conditions, it immediately triggers an emergency stop command, forming a dual safety guarantee with the water pipe dragging, release, and retraction operation process.

[0196] 1) Input: Reference trajectory: , (From the global path planning layer); Current status: (From the multi-source data fusion layer).

[0197] 2) Control objectives: The control objective is achieved through the control commands (driving the robot dog to move) output by the MPC and the predicted state (feedback optimization), which matches the output content in step 5) below, forming a correspondence of "objective → means of implementation".

[0198] A. Main objective: Maintain a specific relative pose .

[0199] B. Secondary objective: Track the global reference trajectory, maintain coordinated time synchronization, and satisfy all kinematic, dynamic, and cooperative constraints.

[0200] C. Work control objectives: Tension stability control: Stabilize the dragging tension within a safe range (≤20kg, avoid overload), and synchronously match the clamping force (3-5kg), which is achieved by adjusting the robot dog's gait speed and the slide rail extension speed; Precise control of extension and retraction length: L measured by an encoder n For feedback, control the extension and retraction displacement of the slide rail to ensure that the length of each release / retraction is accurately N meters, with an error of ≤ ±0.01m; Mechanism-coordinated control: Coordinates the robot dog's movement speed with the linear guide rail's movement rhythm to prevent the robot dog from moving too fast or too slow, which could cause abnormal stress on the water pipe or jamming of the guide rail.

[0201] 3) MPC optimization issues:

[0202] Constraints: State constraints: ; Control constraints: ; Safe distance: ; Clamping force constraint: 3kg≤clamping force≤5kg (safe range), maximum clamping force≤8kg; Dragging tension constraint: real-time tension ≤ 20kg (maximum traction force of the robot dog), breaking force threshold ≤ 0.5kg; Displacement measurement constraints: extension / retraction length L≤N (segmented operation), encoder measurement error ≤±0.01m.

[0203] 4) Control process: Loop begins (33ms control cycle); Get the current state after fusion ; Calculate the relative pose error: ; MPC optimization solution: ; Send control commands to the underlying controller; Waiting for the next control cycle, then repeating the process.

[0204] 5) Output: Control commands: Robot dog movement commands (Forward, Lateral, and Rotational Speed), clamp opening and closing commands (clamping / releasing), servo push rod extension and retraction commands (forward / backward, displacement N meters), and slide rail positioning confirmation commands (synchronized with the robot dog's stop command) are output to the robot dog's underlying controller. Predicted state This is used for model prediction updates in the multi-source data fusion layer (EKF), forming a closed loop. Unlike the dual trajectories output by the global planning layer, this is the short-term actual state predicted by MPC based on a dynamic model (high-frequency generation, no independent timestamps, corresponding to the EKF state vector).

[0205] The output of MPC (control commands + predicted states) matches the control objective (2). The control objective is the desired effect, and the output is the method to achieve the objective and the feedback: control commands. It is directly used to control the robot dog, achieving the primary and secondary objectives of maintaining relative pose and tracking the global trajectory. (Predicted state) Feedback is sent to the fusion layer to help improve the accuracy of state estimation, indirectly ensuring that the control objective is achieved. The two work together to achieve the control objective.

[0206] (4) Execution control layer (machine dog + sliding rail collaborative execution).

[0207] A. Optimization of the robot dog's execution layer.

[0208] Gait control: Define the gait parameters for uniform dragging (speed 0.2-0.5m / s) and the requirements for stable standing posture (lock gait during operation), and match them with the MPC command closed loop to ensure smooth dragging and stable working posture; Status feedback upgrade: In addition to pose and traction, "work progress feedback" (such as whether the target point has been reached, whether the clamping / release has been completed) has been added and uploaded to the fusion layer and task scheduling node simultaneously.

[0209] B. Linear slide rail mechanism execution layer.

[0210] The execution layer is carried out by the robot dog's underlying control board, and its core responsibilities and logic include: Command reception and parsing: Receives slide rail control commands issued by MPC / task scheduling nodes and parses them into the extension and retraction of servo push rods and the opening and closing of clamps; Hardware drive and closed loop: drive servo electric push rod (to realize displacement measurement) and V-shaped clamp (to realize clamping force control), and use force sensor / encoder to provide real-time feedback of status, forming a small closed loop of "command → execution → feedback"; Safety emergency stop response: When receiving an emergency stop signal from the robot dog's main control (tension ≥ 20kg, clamping force exceeding 8kg), immediately stop the push rod action, maintain the clamp state, and avoid secondary damage; ROS node adaptation: encapsulated as a ROS node ( / dog / slide_control), it communicates with the task scheduling node and MPC controller through ROS topics / services, and maintains communication consistency with the collaborative architecture.

[0211] A coordinated triggering mechanism ensures synchronized movements of the drone, robot dog, and sliding rail mechanism. Upon reaching the target point, the following synchronized actions are triggered: robot dog stops → sliding rail clamp positions → sliding rail retraction / release → after retraction / release, robot dog continues with the next drag segment. The drone synchronously monitors the status of the robot dog and the sliding rail, based on L... n Measurement results and tension data are used to dynamically adjust the flight trajectory and mission scheduling, connecting with the multi-row component cleaning process. This achieves deep adaptation between hardware and logic.

[0212] See Figure 12 , Figure 12 This is a schematic diagram of a multi-mode adaptive mechanism in an embodiment of this application. The multi-mode adaptive mechanism serves as a robustness guarantee and dynamic adaptation supplement to the three-layer architecture of the global path planning layer, multi-source data fusion layer, and model prediction and control layer. It is deeply coupled with and collaborates with the three-layer architecture, with the core connections as follows: Time-synchronized dual trajectories output by the global path planning layer and As the "global benchmark" of the multi-mode adaptive mechanism, regardless of which mode is switched to (high precision, robust positioning, dynamic obstacle avoidance, etc.), the system ultimately takes the pre-planned trajectory as the target. In the dynamic obstacle avoidance mode, the system returns to the original trajectory after bypassing the obstacle. In the robust positioning mode, the system also needs to maintain tracking of the original trajectory to ensure that the global collaborative direction does not deviate.

[0213] The core of the multi-mode adaptive mechanism relies on the EKF fusion filter in the multi-source data fusion layer, and mode switching directly determines the input source and fusion logic of the multi-source data fusion layer: In high-precision mode, EKF fuses visual, AMCL, GPS, and MPC prediction states; in robust positioning mode, EKF only fuses AMCL and leg odometry; in GPS rejection mode, EKF does not fuse GPS data, relying only on visual relative pose and MPC prediction states; simultaneously, mode switching dynamically adjusts the adaptive weights of EKF (e.g., when vision is lost), Reset to zero , This ensures that the pose of the fused output matches the requirements of the current mode.

[0214] The MPC controller is the "execution core" and "trigger node" of the multi-mode adaptive mechanism: ① The dynamic obstacle avoidance mode is directly triggered by the MPC (the MPC detects dynamic obstacles, causing the pre-planned route to fail), and the MPC performs local replanning to avoid the obstacle; ② In all modes, the MPC receives the pose after fusion from the multi-source data fusion layer. The output control commands drive the robot dog, adapting to different tracking requirements (such as high-precision mode pursuing centimeter-level tracking, and robust positioning mode prioritizing motion continuity); ③ MPC output prediction status Feedback is sent to the multi-source data fusion layer to assist EKF fusion under various modes and improve pose estimation accuracy.

[0215] After system startup, three key threads—AMCL (Machine Dog Global Pose), UAV GPS (Target Global Pose), and Vision System (Relative Pose)—run in parallel, with all perception information input into the EKF fusion filter. Mode selection and switching use the detection results of the Vision System (whether the AprilTag QR code is visible) as the main switch, combined with the UAV's GPS signal status and dynamic obstacle detection results, to achieve dynamic adaptive switching between four modes, ensuring stable collaboration of the system in complex environments.

[0216] The collaborative architecture's multi-mode adaptive mechanism (using the vision system as a switch) directly adapts to the robot dog's operating scenarios. When the vision system (AprilTag) is functioning normally, the system is in high-precision tracking mode, allowing the robot dog to accurately receive instructions from the drone, complete target point positioning, and precisely execute the retraction and extension actions of the linear guide rail mechanism, ensuring encoder measurement accuracy. When vision is lost or GPS is rejected, the system switches to robust positioning / degradation mode. The robot dog relies on AMCL positioning, leg-type odometers, and combined with tension detection and length measurement data from the linear guide rail to maintain operational stability and avoid errors in hose retraction and extension length or dragging failures due to positioning deviations.

[0217] See Figure 13 , Figure 13 This is a schematic diagram illustrating the pose tracking process of a drone and a legged robot in an embodiment of this application. The adaptive mode includes: 1) High-precision operation mode (normal vision + stable GPS, core operation scenario): Triggering condition: The robot dog's camera successfully recognizes the AprilTag digital QR code visual mark on the drone's tripod, indicating that the vision system is working properly; Working principle: The visual algorithm provides the transformation from QR code to camera. Based on known external parameters of the camera and the robot dog body calibration The robot dog's pose relative to the drone was calculated. ; Using the global position provided by the drone's GPS, the robot dog's visual global pose is calculated. , This represents the pose composition operator. This represents the global position of the robot dog. This represents the global position of the drone.

[0218] Fusion logic: The global pose of the robot dog calculated by vision and the pose predicted by AMCL are input into the EKF fusion filter. By comparing the differences between the two, the state estimation is corrected, and the high-frequency (30Hz+) and drift-resistant accurate pose is output. The positioning frequency is raised to the frequency of the vision system, realizing smooth, high-frequency and high-precision tracking of the UAV-robot dog.

[0219] 2) Robust positioning mode (visual loss, downgraded adaptation): Triggering conditions: The AprilTag QR code flies out of the camera's field of view, is obstructed, or is lost due to poor lighting conditions; Working principle: Visual algorithms cannot provide relative pose measurement updates, and the EKF fusion filter only fuses data from AMCL and leg odometry; Collaborative support: AMCL provides absolute position reference to avoid cumulative errors; the leg odometry provides high-frequency relative motion estimation in a short time to maintain the continuity of the robot dog's movement; MPC continues to track the pre-planned route based on the fused global positioning and internal model until visual recognition is restored, at which point the system immediately switches back to high-precision tracking mode.

[0220] 3) Dynamic obstacle avoidance mode (emergency mode): Triggering condition: MPC receives the fused optimal pose and detects that the pre-planned route has failed due to dynamic obstacles; Working principle: If no dynamic obstacle is detected, the system maintains the original route and tracks stably; if a dynamic obstacle is detected, MPC starts the local replanning algorithm to complete the obstacle avoidance, and automatically returns to the original pre-planned route after the obstacle avoidance is completed. Key safeguards: The entire process relies on continuous pose information provided by the vision system or fusion positioning to ensure that the collaborative relationship between the drone and the robot dog is not interrupted.

[0221] 4) GPS Deny Downgrade Mode (Extreme Robust Mode): Triggering condition: The drone's GPS signal fails, and it cannot provide global pose reference; Working principle: The system automatically degrades, and the robot dog relies entirely on the relative pose measured by the vision system to track the drone itself as the moving reference frame, without relying on global positioning. Core value: As a robust guarantee for the system in GPS-denied environments (such as indoors or obstructed areas), it ensures that collaborative tracking tasks are not interrupted.

[0222] 5) Precision clamping mode (prioritizing force control accuracy during water pipe clamping): Triggering conditions: When the robot dog arrives at the work position A / B and is ready to clamp the water pipe, the task scheduling node issues a "clamping command" and automatically switches to "precise clamping mode" to prioritize ensuring the accuracy of clamping force and water pipe position. Working principle: The vision system locks the position of the water pipe, the force sensor reads the value in real time, and the servo push rod finely adjusts the position of the clamp to ensure a clamping force of 3-5kg, so as to avoid injury or insecure clamping. 6) Deployment / Retraction Verification Mode (During the release / retrieval phase, length measurement is prioritized): Triggering conditions: When performing release / retraction operations or sliding rail extension / retraction, and the task scheduling node issues a "retraction / release command", switch to "retraction / release length verification mode" to prioritize ensuring accurate displacement measurement. Working principle: The encoder-measured L is used as the core feedback, and vision assists in positioning. If the difference between L and N exceeds the threshold (±0.01m), the slide rail is stopped immediately and the displacement is calibrated.

[0223] This application clarifies the core selection requirements for the linear guide rail mechanism and the legged robot platform, as well as the deployment of the ROS distributed software architecture. It establishes the full-process execution logic for pipe clamping, uniform-speed dragging, segmented release, and retrieval, and designs corresponding safety and anomaly handling mechanisms. By constructing parallel collaborative static task planning and dynamic real-time optimization strategies, the design and modeling of the Model Predictive Controller (MPC) are completed, achieving precise matching between task trajectory and execution actions. A collaborative hierarchical closed-loop data fusion control architecture for UAVs and legged robots is built, encompassing four layers: global path planning, multi-source data fusion, model predictive control, and execution control. Combined with a multi-mode adaptive mechanism, the system's robustness is improved, adapting to different operating scenarios and complex environments. By clarifying the full-link coordination and execution logic of the collaborative hierarchical closed-loop control, a collaborative closed loop of "planning-control-execution" is established, achieving deep adaptation between hardware and control logic. This ensures the accuracy, safety, and collaborative efficiency of pipe operations, providing engineering reference and technical support for UAVs and legged robots to collaboratively perform similar dragging and retrieval tasks.

[0224] Corresponding to the above method embodiments, this application also provides a photovoltaic power station cleaning device. The photovoltaic power station cleaning device described below can be referred to in correspondence with the photovoltaic power station cleaning method described above.

[0225] See Figure 14 , Figure 14 This is a structural block diagram of a photovoltaic power station cleaning device according to an embodiment of this application. The device may include: The reference trajectory generation module 11 is used to generate a first reference trajectory with timestamps for the UAV and a second reference trajectory with timestamps for the legged robot; wherein the timestamps of the UAV and the legged robot are synchronized. Current state acquisition module 12 is used to acquire the current state of the drone and the legged robot; The control command prediction module 13 is used to predict control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory and the current state using the model prediction controller; The photovoltaic power station cleaning module 14 is used to control a legged robot to release or recycle water pipes through control commands in order to clean the photovoltaic power station.

[0226] As described in the above technical solution, by generating timestamped reference trajectories for the legged robot and the drone, the current states of the drone and the legged robot are obtained. A model predictive controller then predicts control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current state. These control commands then control the legged robot to release or retrieve the water pipes, thereby completing the cleaning of the photovoltaic power station. By using the legged robot to drag, release, and retrieve the water pipes, and by jointly planning the spatiotemporal aspects of the drone and the legged robot, precise matching of task trajectories and execution actions is achieved, ensuring the accuracy, safety, and efficient collaboration of the water pipe operation.

[0227] In one specific embodiment of this application, the current state acquisition module 12 may include: The relative pose acquisition submodule is used to acquire the relative pose between the drone and the legged robot; The absolute global pose acquisition submodule is used to acquire the absolute global pose of the legged robot. The cooperative state prediction submodule is used to predict the cooperative state of the UAV and the legged robot based on the dynamics model of the legged robot and the dynamics model of the UAV using the model prediction controller. The global consistent pose acquisition submodule is used to fuse the relative pose, absolute global pose, and cooperative state using a preset state estimation algorithm to obtain the global consistent pose of the UAV and the legged robot. The global consistent pose includes the 3D position of the legged robot, the pose quaternion of the legged robot, the 3D position of the UAV, and the pose quaternion of the UAV. The current state determination submodule is used to determine the globally consistent pose as the current state.

[0228] In one specific embodiment of this application, the absolute global pose acquisition submodule may include: The previous time step particle cloud determination unit is used to obtain the first pose of the legged robot in the previous time step, initialize the particle cloud based on the first pose, and determine the initialized particle cloud as the previous time step particle cloud. The displacement difference and rotation difference acquisition unit is used to acquire the displacement difference and rotation difference from the previous time step to the current time step; The second pose prediction unit is used to predict the second pose of each particle in the particle cloud in the previous time step in the current time step by using the adaptive Monte Carlo localization algorithm based on the displacement difference and rotation difference and by running the model. The absolute global pose determination unit is used to determine the absolute global pose of the legged robot based on each second pose.

[0229] In one specific embodiment of this application, the absolute global pose determination unit may include: The likelihood field generation subunit is used to generate a likelihood field representing the probability of occurrence of the measurement point based on each second pose; where the measurement point is the point where the scanning ray hits the obstacle; The sub-unit for obtaining the location point to be queried is used to acquire the current scan data and convert the current scan data to the global map coordinate system according to each second pose to obtain each location point to be queried; wherein, the current scan data includes the current measurement distance corresponding to each current scan ray; The probability value lookup subunit is used to find the probability value corresponding to each current scanning ray at each query position point from the likelihood field; wherein, each probability value is the probability that the end point of each current scanning ray appears at each query position point when it is in each second pose. The particle weight calculation subunit is used to calculate the particle weight corresponding to each particle in each second pose based on each probability value. After resampling, the particles obtain sub-units, which are used to resample the particles in each second pose according to the weight of each particle to obtain the resampled particles. The absolute global pose determination subunit is used to determine the absolute global pose of the legged robot based on the second pose corresponding to each resampled particle.

[0230] In one specific embodiment of this application, the relative pose acquisition submodule is specifically a module that uses a visual algorithm to calculate the relative pose between the UAV and the legged robot; The absolute global pose determination subunit is specifically used to determine the predicted global pose of the legged robot based on the second pose corresponding to each resampled particle; obtain the global position provided by the UAV positioning system, and calculate the visual global pose of the legged robot based on the relative pose and the global position; use a preset state estimation algorithm to perform state estimation correction on the predicted global pose and the visual global pose, and output the absolute global pose.

[0231] In one specific embodiment of this application, the reference trajectory generation module 11 may include: The global optimal path calculation submodule is used to calculate the global optimal path for the UAV and the global optimal path for the legged robot based on a preset optimization algorithm. The reference trajectory acquisition submodule is used to perform spatiotemporal synchronization and feasibility verification of the global optimal path of the UAV and the global optimal path of the legged robot, and obtain the first reference trajectory and the second reference trajectory.

[0232] In one specific embodiment of this application, the device may further include: The anomaly reporting module is used to report anomalies when abnormalities are detected in the release or dragging of the water pipe.

[0233] For the method embodiments described above, see [link to relevant documentation]. Figure 15 , Figure 15 This is a schematic diagram of the photovoltaic power plant cleaning equipment provided in this application. The equipment may include: Memory 332 is used to store computer programs; The processor 322 is used to execute a computer program to implement the steps of the photovoltaic power plant cleaning method described in the above method embodiment.

[0234] For details, please refer to Figure 16 , Figure 16 This is a schematic diagram illustrating the specific structure of a photovoltaic power station cleaning device provided in this embodiment. The photovoltaic power station cleaning device can vary significantly due to differences in configuration or performance. It may include a processor (central processing unit, CPU) 322 (e.g., one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. The memory 332 can be for short-term or long-term storage. The program stored in the memory 332 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute the series of instruction operations stored in the memory 332 on the photovoltaic power station cleaning device 301.

[0235] The photovoltaic power station cleaning equipment 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0236] The steps in the photovoltaic power plant cleaning method described above can be implemented by the structure of the photovoltaic power plant cleaning equipment.

[0237] Corresponding to the above method embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the following steps: A first reference trajectory with a timestamp is generated for the UAV, and a second reference trajectory with a timestamp is generated for the legged robot; wherein the timestamp of the UAV is synchronized with the timestamp of the legged robot; the current state of the UAV and the legged robot is obtained; the model predictive controller is used to predict control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory and the current state; the control commands are used to control the legged robot to release or retrieve the water pipes accordingly in order to clean the photovoltaic power station.

[0238] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0239] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0240] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses, devices, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0241] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for cleaning a photovoltaic power station, characterized in that, include: A first reference trajectory with a timestamp is generated for the UAV, and a second reference trajectory with a timestamp is generated for the legged robot; wherein the timestamp of the UAV is synchronized with the timestamp of the legged robot; Obtain the current state of the drone and the legged robot; The model predictive controller predicts control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory, and the current state. The control commands control the legged robot to release or retrieve the water pipes to clean the photovoltaic power station.

2. The photovoltaic power station cleaning method according to claim 1, characterized in that, Obtaining the current state of the drone and the legged robot includes: Obtain the relative pose between the drone and the legged robot; Obtain the absolute global pose of the legged robot; The model prediction controller uses the legged robot dynamics model and the UAV dynamics model to predict the cooperative state of the UAV and the legged robot. The relative pose, the absolute global pose, and the cooperative state are fused using a preset state estimation algorithm to obtain a globally consistent pose of the UAV and the legged robot; wherein, the globally consistent pose includes the three-dimensional position of the legged robot, the pose quaternion of the legged robot, the three-dimensional position of the UAV, and the pose quaternion of the UAV. The globally consistent pose is determined as the current state.

3. The photovoltaic power station cleaning method according to claim 2, characterized in that, Obtaining the absolute global pose of the legged robot includes: The first pose of the legged robot in the previous time step is obtained, and the particle cloud is initialized according to the first pose. The initialized particle cloud is then determined as the particle cloud of the previous time step. Obtain the displacement difference and rotation difference from the previous time step to the current time step; Using the adaptive Monte Carlo localization algorithm, based on the displacement difference and the rotation difference, the running model is used to predict the second pose of each particle in the particle cloud at the current time step, according to the displacement difference and the rotation difference. The absolute global pose of the legged robot is determined based on each second pose.

4. The photovoltaic power station cleaning method according to claim 3, characterized in that, Determining the absolute global pose of the legged robot based on each second pose includes: A likelihood field representing the probability of occurrence of a measurement point is generated based on each second pose; wherein, the measurement point is the point where the scanning ray hits the obstacle; Acquire the current scan data and convert the current scan data to the global map coordinate system according to each second pose to obtain each query location point; wherein, the current scan data includes the current measured distance corresponding to each current scan ray; Find the probability value of each current scanning ray at each query position point from the likelihood field; wherein each probability value is the probability that the end point of each current scanning ray appears at each query position point when it is in each second pose. Calculate the particle weights corresponding to the particles in each second pose based on the probability values. The particles in each second pose are resampled according to their respective weights to obtain the resampled particles. The absolute global pose of the legged robot is determined based on the second pose corresponding to each resampled particle.

5. The photovoltaic power station cleaning method according to claim 4, characterized in that, Obtaining the relative pose between the UAV and the legged robot, and determining the absolute global pose of the legged robot based on the second pose corresponding to each resampled particle, includes: The relative pose between the drone and the legged robot is calculated using a visual algorithm. The predicted global pose of the legged robot is determined based on the second pose corresponding to each resampled particle. Obtain the global position provided by the UAV positioning system, and calculate the visual global pose of the legged robot based on the relative pose and the global position; The preset state estimation algorithm is used to perform state estimation correction on the predicted global pose and the visual global pose, and the absolute global pose is output.

6. The photovoltaic power station cleaning method according to any one of claims 1 to 5, characterized in that, Generate a first reference trajectory with a timestamp for the drone and a second reference trajectory with a timestamp for the legged robot, including: The global optimal path for the UAV and the global optimal path for the legged robot are calculated based on a preset optimization algorithm. The global optimal path of the UAV and the global optimal path of the legged robot are spatiotemporally synchronized and their feasibility is verified to obtain the first reference trajectory and the second reference trajectory.

7. The photovoltaic power station cleaning method according to claim 1, characterized in that, Also includes: When an abnormality is detected in the release or dragging of the water pipe, an abnormality report is made.

8. A photovoltaic power station cleaning device, characterized in that, include: A reference trajectory generation module is used to generate a first reference trajectory with a timestamp for the UAV and a second reference trajectory with a timestamp for the legged robot; wherein the timestamp of the UAV is synchronized with the timestamp of the legged robot; The current state acquisition module is used to acquire the current state of the drone and the legged robot; The control command prediction module is used to predict control commands that satisfy dynamic constraints based on the first reference trajectory, the second reference trajectory and the current state using a model prediction controller. The photovoltaic power station cleaning module is used to control the legged robot to release or retract water pipes according to the control commands, so as to clean the photovoltaic power station.

9. A photovoltaic power station cleaning device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the photovoltaic power plant cleaning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the photovoltaic power plant cleaning method as described in any one of claims 1 to 7.

Images