A UAV river inspection method and system based on water environment factors

Abstract

The present invention discloses a method and system for river inspection using a drone based on water environment factors, which belongs to the field of river inspection technology and is used to solve the technical problem that the existing drone force / position control algorithm does not fully consider the influence of water environment factors on floating objects, resulting in poor judgment accuracy of the drone gripper control force and gripping position, and low efficiency in cleaning foreign objects from the river. The method includes: obtaining water environment data of the river to be inspected and foreign object data detected by the drone, and determining the force control amount and position control amount of the drone; updating the force control amount based on a real-time compensation mechanism for gripping force control based on the water environment; dynamically allocating the control amount of the updated force control amount and position control amount based on an adaptive error compensation mechanism to obtain the total control amount of the drone gripper; controlling the drone gripper to grip foreign objects based on the total control amount, and simultaneously performing grip correction and flight attitude correction on the drone through a large cloud model.

Description

Technical Field

[0001] The present invention relates to the technical field of river inspection, and in particular to a method and system for river inspection using an unmanned aerial vehicle (UAV) based on water environment factors. Background Art

[0002] With the development of the low-altitude economy in the smart water industry, drones are playing an increasingly important role in irrigation area inspections. Traditional river inspections rely primarily on manual inspections, which are subject to significant investment, low efficiency, and limited coverage. Floating debris in irrigation area rivers can include plastic bags, branches, crop residue, and household garbage. These objects not only detract from the aesthetics of the river but can also clog it, hindering water flow and even harming downstream aquatic ecosystems. Therefore, timely detection and removal of river debris is crucial for maintaining the normal operation of irrigation area water conservancy facilities and ensuring water quality.

[0003] The application of drone technology offers a new solution for river inspections, enabling rapid and comprehensive river surveys. However, when drones detect floating foreign objects in a river, efficient and safe handling remains a pressing challenge. Traditional mechanical gripping methods face numerous challenges, especially when foreign objects float on the water surface or are partially submerged. Traditional force / position control algorithms primarily consider workpiece quality and appearance. However, floating objects on the water surface are affected by water environmental factors such as buoyancy, resistance, and water velocity, and their appearance and quality vary. Improper application of control force can lead to breakage, falling, or displacement of floating objects, resulting in ineffective results. Furthermore, fluctuations in the water environment significantly influence the position of floating objects. Failure to fully account for these factors can lead to deviations in the force control and gripping position control of the drone's end gripper, hindering the capture of floating objects. Summary of the Invention

[0004] An embodiment of the present invention provides a drone river inspection method and system based on water environment factors, which is used to solve the following technical problems: the existing drone force / position control algorithm does not fully consider the impact of water environment factors on floating objects, resulting in poor judgment accuracy of the drone gripper control force and clamping position, and low efficiency in clearing foreign objects from the river.

[0005] The embodiment of the present invention adopts the following technical solutions:

[0006] In one aspect, an embodiment of the present invention provides a method for river inspection using a drone based on water environment factors, the method comprising: obtaining water environment data of the river to be inspected and data of foreign objects detected by the drone, and determining a force control amount and a position control amount of the drone based on the water environment data and the foreign object data;

[0007] updating the force control amount based on a real-time compensation mechanism of the grasping force control by the water environment;

[0008] Based on the adaptive error compensation mechanism, the updated force control variable and the position control variable are dynamically allocated to obtain the total control variable of the UAV gripper;

[0009] Based on the total control amount, the gripper of the drone is controlled to grip foreign objects, and at the same time, the drone is corrected for gripping and flight posture through a large cloud model.

[0010] In a feasible implementation, obtaining water environment data of the river to be inspected and data of foreign objects detected by the drone specifically includes:

[0011] Acquire real-time water environment data of the river to be inspected through a river water monitoring sensor in communication with the drone; wherein the water environment data includes at least: current river water density, current water flow resistance coefficient, current water flow velocity, and current acceleration of the river water relative to the foreign object;

[0012] The visual sensors and position sensors carried by the drone are used to detect floating foreign objects in the river channel to be inspected, and corresponding foreign object data is obtained; wherein the foreign object data includes at least: the position of the foreign object, the volume of river water displaced by the foreign object, the windward area of ​​the foreign object, and the speed of the foreign object relative to the water flow.

[0013] In a feasible implementation, determining the force control amount and position control amount of the drone based on the water environment data and the foreign object data specifically includes:

[0014] Calculating the theoretical buoyancy, river water resistance, water flow thrust, Reynolds stress, and fluid inertia of the foreign object in the river water based on the water environment data and the foreign object data, and determining the actual buoyancy of the foreign object in the river water based on the calculation results;

[0015] determining a force control amount of the drone according to the actual buoyancy of the foreign object in the river water and control parameters of the drone gripper;

[0016] The position control amount of the drone is determined according to the position of the foreign object and the control parameters of the drone gripper.

[0017] In a feasible implementation, the force control amount of the drone is determined based on the actual buoyancy of the foreign object in the river water and the control parameters of the drone gripper, specifically including:

[0018] according to , determine the force control amount of the UAV F c ;

[0019] in, Proportional gain for adaptive gripper force control of drone, Given the gripping force of the gripper, is the current gripping force of the gripper, F w is the actual buoyancy of the foreign object in the river water; is the differential coefficient of the gripper speed control, is the gripper speed control quantity, is the current speed of the gripper.

[0020] In a feasible implementation, determining the position control amount of the drone according to the position of the foreign object and the control parameters of the drone gripper specifically includes:

[0021] according to , determine the position control amount of the UAV P c ;

[0022] in, Adaptive proportional gain for the position control of the UAV gripper, Given the position of the gripper, is the current position of the gripper; is the differential coefficient of the gripper position control.

[0023] In a feasible implementation, the force control value is updated based on a real-time compensation mechanism of the water environment for the grasping force control, specifically including:

[0024] Based on the error value of the water environment data and the preset adjustment range, dynamically estimate the water environment data at the next moment to obtain an estimated value of the water environment data;

[0025] The calculation formula of the force control amount is updated by the water environment estimation value to obtain an improved force control amount formula: ;

[0026] in, Proportional gain for adaptive gripper force control of drone, Given the gripping force of the gripper, is the current gripping force of the gripper, F w is the actual buoyancy of the foreign body in the river water, is the estimated density of river water, is the estimated value of the water flow resistance coefficient, is the estimated value of water velocity, is the differential coefficient of the gripper speed control, is the gripper speed control quantity, is the current speed of the gripper;

[0027] The force control amount is recalculated by the improved force control amount formula to obtain the updated force control amount F c .

[0028] In a feasible implementation, based on the adaptive error compensation mechanism, the updated force control variable and the position control variable are dynamically allocated to obtain the total control variable of the drone gripper, specifically including:

[0029] according to , determine the real-time switching function of the proportional coefficient of force control amount and position control amount ;in, The value range of is [0,1], which is used to dynamically adjust the weight of the adaptive force / position hybrid control affected by the water environment; is the switching sensitivity coefficient, which is used to control the steepness of the switching of force / position hybrid control. is the weight coefficient, which is used to balance the mutual influence between the force control error and the position control error. F ce is the real-time error of the force control quantity, is the real-time error of the position control quantity;

[0030] according to , determine the total control amount of the drone gripper u c ;in, F c is the updated force control amount, P c is the position control amount;

[0031] When the error of the force control amount is large, the value of the real-time switching function approaches 1, and the drone gripper mainly compensates for the force control amount error. When the error of the position control amount is large, the value of the real-time switching function approaches 0, and the drone gripper mainly compensates for the position control amount error.

[0032] In a feasible embodiment, after dynamically allocating the updated force control variable and the position control variable based on the adaptive error compensation mechanism to obtain the total control variable of the drone gripper, the method further includes:

[0033] Based on the stability function considering water environment data , the stability of the control system of the UAV gripper is evaluated; among them, F ce is the real-time error of the force control quantity, is the real-time error of the position control quantity, is the comprehensive gain of proportional control and differential control, is the combined estimation error of the estimated values ​​of river water density, water flow resistance coefficient, and water flow velocity;

[0034] When the derivative of the stability function satisfies When , it is determined that the stability of the current control system meets the standard; is the decay rate, and >0.

[0035] In a feasible implementation, the drone is subjected to grasping correction and flight attitude correction through a large cloud model, specifically including:

[0036] While the drone is grabbing foreign objects, the flight data and gripper status data of the drone are obtained in real time and transmitted to the cloud-based large model;

[0037] Extracting the motion characteristics of the foreign object from the water environment data, the foreign object data, the flight data, and the gripper state data using the cloud-based large model;

[0038] Based on the motion characteristics, the cloud-based large model is called through the prompt word template to predict the motion trajectory of the foreign object in the future preset time period;

[0039] According to the motion trajectory, the flight direction and speed of the drone are dynamically adjusted to correct the grip of the drone gripper and compensate for the drop or displacement of foreign objects that occur during the initial pickup process;

[0040] After confirming that the foreign object has been successfully grasped, the drone's lift and yaw angle are dynamically adjusted based on the change in the drone's mass at the moment the foreign object leaves the water and the current wind speed to maintain the stability of the gripper.

[0041] Control the drone to fly towards the nearest foreign object disposal station and continuously perform flight attitude correction during the flight until the floating objects are transported to the foreign object disposal station.

[0042] On the other hand, an embodiment of the present invention further provides a drone river inspection system based on water environment factors, the system comprising:

[0043] The gripper control module is used to obtain water environment data of the river to be inspected and data on foreign objects detected by the drone, and determine the force control value and position control value of the drone based on the water environment data and foreign object data; update the force control value based on a real-time compensation mechanism for the water environment's gripping force control; and dynamically allocate the updated force control value and position control value based on an adaptive error compensation mechanism to obtain the total control value of the drone gripper;

[0044] The attitude correction module is used to control the gripper of the drone to grasp foreign objects based on the total control amount, and at the same time perform grasping correction and flight attitude correction on the drone through a large cloud model.

[0045] Compared with the existing technology, the embodiment of the present invention provides a drone river inspection method and system based on water environment factors, which has the following beneficial effects:

[0046] The present invention proposes a drone river inspection and floating object picking method that takes into account the influence of water environment factors. When floating foreign objects are detected in the river, the drone can automatically identify and locate the foreign objects, and use an improved force / position hybrid control strategy that takes into account water environment factors to pick up the foreign objects through the end gripper. If the initial grabbing of floating objects is not completed, the edge-end gripper can be assisted by a large cloud-based model to perform adaptive positioning and posture correction, and finally the foreign objects can be dropped to the nearest disposal station.

[0047] This invention fully considers the characteristics of floating objects and water environment, as well as the influence of water environment factors during UAV river inspection and floating object picking, and proposes an improved force / position hybrid compliant control method. It also uses a large model of cloud-edge collaboration to assist the secondary positioning of the gripper and the posture correction of the UAV to ensure the smooth completion of the grasping task, thereby helping the ecological construction of the irrigation area and the high-quality development of the low-altitude economy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments described in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0049] Figure 1 A flow chart of a method for river inspection using a drone based on water environment factors provided by an embodiment of the present invention;

[0050] Figure 2 A schematic structural diagram of a UAV river inspection system based on water environment factors provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0052] Currently, research on using drones, unmanned boats, and other auxiliary tools to inspect irrigation river channels and capture floating objects primarily focuses on mechanical structure design and optimization of control algorithms, such as eagle claw grippers and synovial control algorithms, to improve the gripper's stability and accuracy in capturing floating objects. However, due to the influence of water environmental factors, existing gripping technologies have certain limitations when dealing with complex water environments. For example, high water flow makes it difficult to accurately capture floating objects.

[0053] General control methods and multi-sensor fusion technologies are difficult to apply in this scenario. Traditional passive grasping mechanisms mainly rely on the adaptability of mechanical structures, and have low grasping accuracy and stability. In addition, existing grasping technologies are slightly lacking in real-time and flexibility, and it is difficult to make adaptive corrections based on the movement trajectory of floating objects and changes in the water environment. Improper force control can easily cause floating objects to be damaged, dropped, or shifted. The main reason for the above defects is that the key influencing factors of the river floating object grasping scenario are not fully considered. The key factor for the realization of this scenario lies in the control of the grasping force of the UAV's end gripper, and taking into account the position changes of floating objects caused by the flow of river water. Therefore, it is necessary to fully consider the influence of relevant water environment factors such as river water density, resistance, and flow rate.

[0054] In order to quickly and efficiently complete drone river inspection and foreign object cleaning tasks, the embodiment of the present invention provides a drone river inspection method based on water environment factors to address the problems of insufficient adaptability of the traditional control strategy of the drone end gripper to the water environment, poor robustness, and low control accuracy.

[0055] Figure 1 A flow chart of a method for river inspection using a drone based on water environment factors is provided in an embodiment of the present invention, such as Figure 1 As shown, the UAV river inspection method based on water environment factors specifically includes steps S101-S104:

[0056] S101. Obtain water environment data of the river to be inspected and data of foreign objects detected by the drone, and determine the force control amount and position control amount of the drone based on the water environment data and the foreign object data.

[0057] Specifically, floating objects on the surface of irrigation river channels mainly include food packaging bags, paper towels, branches, crop residues, domestic garbage and other items, which have different masses and buoyancy. Therefore, when the UAV's visual sensors and position sensors locate the approximate location of the foreign object and perform the initial grasping, the influence of the water environment must be fully considered, including factors such as water flow velocity, wind speed, and water surface resistance, which will affect the position and grasping force of the floating object. The purpose of considering the water environment is mainly to improve the limitations of traditional force control, and to define the transformation range of the UAV's end gripper force / position hybrid control, to switch reasonably, and to take into account the different shapes and masses of floating objects to accurately apply grasping pressure.

[0058] First, the real-time water environment data of the river to be inspected is obtained through the river water monitoring sensor connected to the drone. The water environment data includes at least: the current river water density, the current water flow resistance coefficient, the current water flow velocity, and the current acceleration of the river water relative to the foreign object.

[0059] Furthermore, the visual sensors and position sensors carried by the drone are used to detect floating foreign objects in the river channel to be inspected, and the corresponding foreign object data is obtained; wherein the foreign object data includes at least: the position of the foreign object, the volume of river water displaced by the foreign object, the windward area of ​​the foreign object, and the speed of the foreign object relative to the water flow.

[0060] Furthermore, based on the water environment data and foreign body data, the theoretical buoyancy of the foreign body in the river water, river water resistance, water flow thrust, Reynolds stress and fluid inertia are calculated respectively, and the actual buoyancy of the foreign body in the river water is determined based on the calculation results.

[0061] As a feasible implementation method, the actual buoyancy of floating foreign objects It can be expressed as: .

[0062] The first term on the right side of the equation (the part before the first plus sign) represents the theoretical buoyancy of the foreign matter to be cleaned. is the current river water density, is the acceleration due to gravity, is the volume of river water displaced by the floating objects. The second term (the part between the first plus sign and the second plus sign) represents the resistance of the river water. is the resistance coefficient of river water, is the frontal area of ​​the foreign body, is the speed of the foreign body relative to the water flow; the third term (the part between the second plus sign and the third plus sign) is the thrust of the water flow. is the current water velocity; the fourth term (the part between the third plus sign and the fourth plus sign) is the Reynolds stress term, is the Reynolds stress tensor, which represents the correlation of water velocity in different directions; the fifth term (the part after the fourth plus sign) is the fluid inertia term, is the acceleration of the river water relative to the foreign object.

[0063] The above five different forces are added together according to their vector sum to obtain the resultant force acting on the floating object on the water surface, which is the actual buoyancy mentioned above.

[0064] Furthermore, the above-mentioned buoyancy is mainly used for solving the control force of the UAV's end gripper in the force control mode, and the position control is mainly performed according to the positioning result of the position sensor. The traditional force / position hybrid control is mainly calculated based on the error between the given force / position information and the actual force / position information, and then the final control quantity is obtained according to a certain fixed proportional coefficient of force control and position control, but the environmental speed and force conditions change in real time. Therefore, the present invention further considers the water environment and proposes a water environment adaptive compensation mechanism. By considering the comprehensiveness of water influencing factors, the robustness of force control is improved, and the smooth switching of the force / position hybrid control mode is further realized.

[0065] Specifically, firstly, according to the actual buoyancy of foreign matter in river water F w As well as the control parameters of the drone gripper, the force control amount of the drone is determined.

[0066] As a feasible implementation method, according to , determine the force control amount of the UAV F c .in, Proportional gain for adaptive gripper force control of drone, Given the gripping force of the gripper, is the current gripping force of the gripper, F w is the actual buoyancy of the foreign object in the river water; is the differential coefficient of the gripper speed control, is the gripper speed control quantity, is the current speed of the gripper.

[0067] Furthermore, the position control amount of the drone is determined according to the position of the foreign object and the control parameters of the drone gripper.

[0068] As a feasible implementation method, according to , determine the position control amount of the UAV P c .in, Adaptive proportional gain for the position control of the UAV gripper, Given the position of the gripper, is the current position of the gripper; is the differential coefficient of the gripper position control.

[0069] S102: updating the force control value based on the real-time compensation mechanism of the grasping force control by the water environment.

[0070] Specifically, the water environment mainly affects the gripping force control of the UAV's end gripper. Therefore, it is necessary to further establish a real-time compensation mechanism for the water environment to force control, and to estimate the frequently changing water environment-related parameters such as river water density, resistance coefficient, and water flow velocity in real time, and substitute them into the force control expression for iterative update.

[0071] First, based on the error value of the water environment data and the preset adjustment range, the water environment data at the next moment is dynamically estimated to obtain the estimated value of the water environment data.

[0072] Among them, as a feasible implementation method, the river water density estimation value The calculation formula is: .in, is the initial estimate of the river water density, is the adjustment margin for the estimated river water density, is the adjustment sensitivity coefficient of river water density, is the buoyancy error.

[0073] Estimated value of water flow resistance coefficient The calculation formula is: .in, is the initial estimate of the river resistance, is the adjustment margin for the estimated value of river resistance, is the adjustment sensitivity coefficient of river water resistance, is the resistance error.

[0074] Estimated water velocity The calculation formula is: .in, is the initial estimate of the water velocity, is the adjustment margin for the estimated value of water velocity, is the adjustment sensitivity coefficient of water flow velocity, is the water flow thrust error.

[0075] Furthermore, the above water environment estimation value is substituted into the calculation formula of the gripper control force of the drone to update the calculation formula of the force control amount according to the water environment estimation value, and the improved force control amount formula is obtained: .in, Proportional gain for adaptive gripper force control of drone, Given the gripping force of the gripper, is the current gripping force of the gripper, is the estimated density of river water, is the estimated value of the water flow resistance coefficient, is the estimated value of water velocity, is the differential coefficient of the gripper speed control, is the gripper speed control quantity, is the current speed of the gripper.

[0076] Compared to the previous equation, this equation primarily updates the resultant force on floating objects in real time based on estimates of river water density, resistance, and current velocity, improving the accuracy of force control during the initial grasp. The water environment significantly impacts force control, and the initial grasp places higher demands on force control. Position control currently does not incorporate compensation, but this is considered during secondary positioning and pickup of floating objects affected by current and wind speeds.

[0077] Furthermore, the force control amount is recalculated by the above improved force control amount formula to obtain the updated force control amount F c .

[0078] S103. Based on the adaptive error compensation mechanism, dynamically distribute the updated force control variable and position control variable to obtain the total control variable of the drone gripper.

[0079] Specifically, when using drones to grip foreign objects, the switching between force and position control of the slag grip requires a certain degree of flexibility, otherwise the gripper will become stuck and inflexible. To achieve this smooth switching between force and position control, the present invention fully considers the real-time errors and dynamic responses of force and position control, and adopts a strategy of dynamic adaptive adjustment of control coefficients.

[0080] First, a real-time switching function was designed , used to determine the proportional coefficient of force control quantity and position control quantity. Among them, The value range of is [0,1], which is used to dynamically adjust the weight of the adaptive force / position hybrid control affected by the water environment; is the switching sensitivity coefficient, which is used to control the steepness of the switching of force / position hybrid control. is the weight coefficient, which is used to balance the mutual influence between the force control error and the position control error. F ce is the real-time error of the force control quantity, is the real-time error of the position control quantity.

[0081] Furthermore, according to the dynamic allocation equation , determine the total control amount of the drone gripper u c ;in, F cis the updated force control amount, P c is the position control quantity.

[0082] When the error of the force control amount is large, the real-time switching function The value of is close to 1, and the UAV gripper is mainly used to compensate for the force control error. When the position control error is large, the real-time switching function The value of approaches 0, and the UAV gripper is mainly used to compensate for the position control error.

[0083] At this point, the improved UAV terminal gripper force / position hybrid compliant control method for picking up floating objects considering the influence of water environment is basically completed. In order to ensure the stability of the control method, the Lyapunov method is introduced to analyze the system stability, and the influence of the estimation error of the water environment parameters is considered. The stability function as follows: .in, F ce is the real-time error of the force control quantity, is the real-time error of the position control quantity, is the comprehensive gain of proportional control and differential control, It is the comprehensive estimation error of the estimated value of river water density, estimated value of water flow resistance coefficient and estimated value of water flow velocity.

[0084] When the derivative of the stability function satisfies When , it is determined that the stability of the current control system meets the standard; is the decay rate, and >0 to ensure that the control system error index converges to 0, thereby ensuring the stability of the drone's initial capture of floating objects and laying the foundation for subsequent secondary corrections.

[0085] S104, based on the total control amount, the gripper of the drone is controlled to grip the foreign object, and at the same time, the gripping correction and flight attitude correction of the drone are performed through the cloud-based large model.

[0086] Specifically, as the drone grasps a foreign object, it acquires real-time flight data and gripper status data, which are then transmitted to a large cloud-based model. The cloud-based model then extracts the foreign object's motion characteristics from the water environment data, foreign object data, flight data, and gripper status data. Based on these motion characteristics, the cloud-based model is invoked using a prompt word template to predict the foreign object's trajectory over a predetermined time period.

[0087] Furthermore, according to the movement trajectory of the foreign object, the flight direction and speed of the drone are dynamically adjusted to correct the gripping of the drone gripper and compensate for the falling or displacement of the foreign object during the initial picking process.

[0088] After successfully grasping the foreign object, the drone's lift and yaw angle are dynamically adjusted based on the change in the drone's mass at the moment the foreign object leaves the water and the current wind speed to maintain gripper stability. The drone is then controlled to fly toward the nearest foreign object disposal station, continuously adjusting its flight attitude until the floating object is delivered to the disposal station.

[0089] As a feasible implementation method, after the drone's end gripper initially grabs a floating object based on the improved force / position hybrid compliant control that considers the water environment, the floating object is still affected by factors such as river water resistance, water flow velocity, and wind speed at the moment it leaves the water surface, making it prone to falling and shifting. Therefore, combined with the results of the force / position hybrid control strategy that considers the water environment in the first step, it is necessary to quickly perform a secondary positioning and pick up of the foreign object to avoid repeated calculations of the force / position hybrid control quantity. In addition, considering the changes in the overall mass of the quadcopter after picking up, its position and attitude information must be adaptively and dynamically updated to complete the final floating object picking task. The specific steps are as follows:

[0090] (1) Real-time data collection and transmission on the end side

[0091] Utilizing the drone's GPS and additional visual sensors, position sensors, and force sensors, the drone collects real-time information on the location and velocity of floating objects, as well as water environment parameters such as river resistance, current velocity, and density. This includes the object's initial position, shape, size, color, wind speed, current velocity, water temperature, and wave height, all recorded during the initial grab. Furthermore, considering the change in the drone's mass the moment the object leaves the water, the drone collects real-time data on its position, including pitch, yaw, and roll angles; gripper status data, including the gripper's opening and closing angle and its position relative to the object; and flight environment factors, such as airflow direction.

[0092] The data is transmitted to the cloud-based large model through the MAVLink protocol for computational processing, and the collected data is pre-processed by normalization, denoising, feature extraction, etc., so that the cloud-based large model can perform unified processing and analysis.

[0093] (2) Cloud-based large model correction instruction transmission

[0094] After data preprocessing is completed in the cloud, key features are further extracted, including the trajectory characteristics of floating objects and the actual impact of wind speed and water currents on floating objects. Combining historical data with current environmental conditions, a large model is used to predict the future trajectory of floating objects using a prompt word template. This model takes into account parameters such as displacement, velocity, and acceleration, influenced by factors such as the water environment.

[0095] The results are sent to the edge via the MAVLink protocol, dynamically adjusting the drone's flight direction and speed. This allows for secondary positioning of the drone's end gripper, quickly compensating for any drops or displacements of floating objects during the initial pickup. After ensuring successful secondary positioning of the floating object, the change in drone mass at the moment the object leaves the water, as well as the effects of wind speed, must be considered. When wind speeds are high and in the opposite direction of the gripper's motion, the large model issues a command to increase the drone's lift and adjusts the yaw angle to maintain gripper stability.

[0096] At the same time, the large model will integrate the above-mentioned real-time data to continuously adjust the overall control strategy of position and attitude, continuously adjust the flight position and attitude information, control the drone to fly to the nearest foreign object disposal station, and transport the floating objects to the disposal station, thereby completing the river inspection and foreign object cleaning tasks. Through the secondary positioning of foreign objects and the correction of the drone's flight attitude, it can be ensured that after the gripper clamps the foreign object, the drone can quickly adapt to the change in weight and the influence of various resistances on the clamped foreign object, and minimize the probability of the foreign object falling. And even if the foreign object really falls after the foreign object is clamped, the method provided by the present invention does not need to recalculate the gripper control force and control position. Instead, a rapid shift analysis is performed through the cloud-based large model to adjust the gripper's gripping position and control force at the fastest speed, so that the foreign object can be clamped again within a short time and a short distance, greatly improving the efficiency of foreign object cleaning.

[0097] In addition, the embodiment of the present invention also provides a UAV river inspection system based on water environment factors, such as Figure 2 As shown, the UAV river inspection system 200 based on water environment factors specifically includes:

[0098] The gripper control module 210 is configured to obtain water environment data of the river to be inspected and data on foreign objects detected by the drone, determine the force control value and position control value of the drone based on the water environment data and foreign object data, update the force control value based on a real-time compensation mechanism for gripping force control by the water environment, and dynamically allocate the updated force control value and position control value based on an adaptive error compensation mechanism to obtain the total control value of the drone gripper.

[0099] The attitude correction module 220 is used to control the gripper of the drone to grip foreign objects based on the total control amount, and at the same time perform grasping correction and flight attitude correction on the drone through a large cloud model.

[0100] In a feasible embodiment, the gripper control module 210 is also used to obtain real-time water environment data of the river channel to be inspected through a river water monitoring sensor connected to the drone for communication; wherein, the water environment data at least includes: current river water density, current water flow resistance coefficient, current water flow velocity and current river water acceleration relative to foreign objects; through the visual sensor and position sensor carried by the drone, floating foreign objects in the river channel to be inspected are detected, and corresponding foreign object data are obtained; wherein, the foreign object data at least includes: the position of the foreign object, the volume of river water displaced by the foreign object, the windward area of ​​the foreign object and the velocity of the foreign object relative to the water flow.

[0101] In a feasible embodiment, the gripper control module 210 is also used to calculate the theoretical buoyancy, river water resistance, water flow thrust, Reynolds stress and fluid inertia of the foreign object in the river water according to the water environment data and the foreign object data, and determine the actual buoyancy of the foreign object in the river water according to the calculation results; determine the force control amount of the drone according to the actual buoyancy of the foreign object in the river water and the control parameters of the drone gripper; and determine the position control amount of the drone according to the position of the foreign object and the control parameters of the drone gripper.

[0102] In a feasible embodiment, the attitude correction module 220 is also used to obtain the flight data and gripper status data of the drone in real time while the drone is grasping the foreign object, and transmit them to the cloud-based large model; through the cloud-based large model, the motion characteristics of the foreign object are extracted from the water environment data, the foreign object data, the flight data and the gripper status data; based on the motion characteristics, the cloud-based large model is called through the prompt word template to pass parameters to predict the motion trajectory of the foreign object in the future preset time period; according to the motion trajectory, the flight direction and flight speed of the drone are dynamically adjusted to perform grip correction on the drone gripper and compensate for the foreign object falling or shifting during the initial picking process; after confirming that the foreign object has been successfully grasped, the lift and yaw angle of the drone are dynamically adjusted based on the change in the drone's overall mass at the moment the foreign object leaves the water surface and the current wind speed to maintain the stability of the gripper; the drone is controlled to fly to the nearest foreign object disposal station, and the flight attitude correction is continuously performed during the flight until the floating object is transported to the foreign object disposal station.

[0103] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0104] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0105] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, the embodiments of this specification may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware. Furthermore, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0109] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0110] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0111] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0112] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0113] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A drone river inspection method based on water environment factors, characterized by: The method comprises: Obtaining water environment data of the river to be inspected and data of foreign objects detected by the drone, and determining force control and position control of the drone based on the water environment data and foreign object data; Based on the real-time compensation mechanism of the water environment for the grasping force control, the force control amount is updated, specifically including: Based on the error value of the water environment data and the preset adjustment range, dynamically estimate the water environment data at the next moment to obtain an estimated value of the water environment data; The calculation formula of the force control amount is updated by the water environment estimation value to obtain an improved force control amount formula: ; in, Proportional gain for adaptive gripper force control of drone, Given the gripping force of the gripper, is the current gripping force of the gripper, F w is the actual buoyancy of the foreign body in the river water, is the estimated density of river water, is the estimated value of the water flow resistance coefficient, is the estimated value of water velocity, is the differential coefficient of the gripper speed control, is the gripper speed control quantity, is the current speed of the gripper; The force control amount is recalculated by the improved force control amount formula to obtain the updated force control amount F c ; Based on the adaptive error compensation mechanism, the updated force control variable and the position control variable are dynamically allocated to obtain the total control variable of the UAV gripper, specifically including: according to , determine the real-time switching function of the proportional coefficient of force control amount and position control amount ;in, The value range of is [0,1], which is used to dynamically adjust the weight of the adaptive force / position hybrid control affected by the water environment; is the switching sensitivity coefficient, which is used to control the steepness of the switching of force / position hybrid control. is the weight coefficient, which is used to balance the mutual influence between the force control error and the position control error. F ce is the real-time error of the force control quantity, is the real-time error of the position control quantity; according to , determine the total control amount of the drone gripper u c ;in, F c is the updated force control amount, P c is the position control amount; When the error of the force control amount is large, the value of the real-time switching function approaches 1, and the UAV gripper mainly compensates for the force control amount error. When the error of the position control amount is large, the value of the real-time switching function approaches 0, and the UAV gripper mainly compensates for the position control amount error. Based on the total control amount, the gripper of the drone is controlled to grip foreign objects, and at the same time, the drone is corrected for gripping and flight posture through a large cloud model.

2. The method for river inspection using a drone based on water environment factors according to claim 1 is characterized in that: Obtain water environment data of the river to be inspected and data on foreign objects detected by the drone, including: Acquire real-time water environment data of the river to be inspected through a river water monitoring sensor in communication with the drone; wherein the water environment data includes at least: current river water density, current water flow resistance coefficient, current water flow velocity, and current acceleration of the river water relative to the foreign object; The visual sensors and position sensors carried by the drone are used to detect floating foreign objects in the river channel to be inspected, and corresponding foreign object data is obtained; wherein the foreign object data includes at least: the position of the foreign object, the volume of river water displaced by the foreign object, the windward area of ​​the foreign object, and the speed of the foreign object relative to the water flow.

3. The method for river inspection using a drone based on water environment factors according to claim 1 is characterized in that: Determine the force control amount and position control amount of the UAV based on the water environment data and the foreign object data, specifically including: Calculating the theoretical buoyancy, river water resistance, water flow thrust, Reynolds stress, and fluid inertia of the foreign object in the river water based on the water environment data and the foreign object data, and determining the actual buoyancy of the foreign object in the river water based on the calculation results; determining a force control amount of the drone according to the actual buoyancy of the foreign object in the river water and control parameters of the drone gripper; The position control amount of the drone is determined according to the position of the foreign object and the control parameters of the drone gripper.

4. The method for river inspection using a drone based on water environment factors according to claim 3 is characterized in that: The force control amount of the drone is determined based on the actual buoyancy of the foreign object in the river water and the control parameters of the drone gripper, specifically including: according to , determine the force control amount of the UAV F c ; in, Proportional gain for adaptive gripper force control of drone, Given the gripping force of the gripper, is the current gripping force of the gripper, F w is the actual buoyancy of the foreign object in the river water; is the differential coefficient of the gripper speed control, is the gripper speed control quantity, is the current speed of the gripper.

5. The method for river inspection using a drone based on water environment factors according to claim 3 is characterized in that: Determining the position control amount of the drone according to the position of the foreign object and the control parameters of the drone gripper, specifically including: according to , determine the position control amount of the UAV P c ; in, Adaptive proportional gain for the position control of the UAV gripper, Given the position of the gripper, is the current position of the gripper; is the differential coefficient of gripper position control; is the gripper speed control quantity, is the current speed of the gripper.

6. The method for river inspection using a drone based on water environment factors according to claim 1 is characterized in that: After dynamically allocating the updated force control variable and the position control variable based on the adaptive error compensation mechanism to obtain the total control variable of the drone gripper, the method further includes: Based on the stability function considering water environment data , the stability of the control system of the UAV gripper is evaluated; among them, F ce is the real-time error of the force control quantity, is the real-time error of the position control quantity, is the comprehensive gain of proportional control and differential control, is the combined estimation error of the estimated values ​​of river water density, water flow resistance coefficient, and water flow velocity; When the derivative of the stability function satisfies When , it is determined that the stability of the current control system meets the standard; is the decay rate, and >

0.

7. The method for river inspection using a drone based on water environment factors according to claim 1 is characterized in that: The drone is corrected for grasping and flight attitude using a large cloud-based model, specifically including: While the drone is grabbing foreign objects, the flight data and gripper status data of the drone are obtained in real time and transmitted to the cloud-based large model; Extracting the motion characteristics of the foreign object from the water environment data, the foreign object data, the flight data, and the gripper state data using the cloud-based large model; Based on the motion characteristics, the cloud-based large model is called through the prompt word template to predict the motion trajectory of the foreign object in the future preset time period; According to the motion trajectory, the flight direction and speed of the drone are dynamically adjusted to correct the grip of the drone gripper and compensate for the drop or displacement of foreign objects that occur during the initial pickup process; After confirming that the foreign object has been successfully grasped, the drone's lift and yaw angle are dynamically adjusted based on the change in the drone's mass at the moment the foreign object leaves the water and the current wind speed to maintain the stability of the gripper. Control the drone to fly towards the nearest foreign object disposal station and continuously perform flight attitude correction during the flight until the floating objects are transported to the foreign object disposal station.

8. A drone river inspection system based on water environment factors, using a drone river inspection method based on water environment factors as described in any one of claims 1 to 7, characterized in that: The system comprises: The gripper control module is used to obtain water environment data of the river to be inspected and data on foreign objects detected by the drone, and determine the force control value and position control value of the drone based on the water environment data and foreign object data; update the force control value based on a real-time compensation mechanism for the water environment's gripping force control; and dynamically allocate the updated force control value and position control value based on an adaptive error compensation mechanism to obtain the total control value of the drone gripper; The attitude correction module is used to control the gripper of the drone to grasp foreign objects based on the total control amount, and at the same time perform grasping correction and flight attitude correction on the drone through a large cloud model.

Images