An AI vision-based spraying unmanned aerial vehicle stability control method and system

Abstract

The application discloses a spraying unmanned aerial vehicle stability control method and system based on AI vision, and relates to the technical field of unmanned aerial vehicle stability control. The prior art has problems such as control lag and insufficient stability. The application comprises the following steps: collecting vision data and preprocessing to form a vision data sequence; performing AI vision feature extraction to form a vision feature sequence; constructing a dynamic space relation graph sequence according to the vision feature sequence; constructing a graph structure reserve calculation model based on the graph sequence, performing state evolution under the propagation constraint, and forming a reserve state sequence; mapping the spraying disturbance information into an input item to inject the evolution process, obtaining a disturbed reserve state sequence; calculating the reserve state change amount, combining the propagation constraint and the vision confidence, and generating a stability risk sequence; finally, constructing a control amount generation mapping to generate a posture control amount sequence, and performing closed-loop adjustment on the unmanned aerial vehicle. The application realizes early perception and adaptive adjustment of unstable trends, and improves the operation stability of the spraying unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of drone stability control technology, and in particular to a method and system for stability control of a painting drone based on AI vision. Background Technology

[0002] With the development of drone technology, spraying drones have been widely used in agricultural and forestry plant protection, surface coating, and maintenance operations. Existing stability control methods for spraying drones typically rely on inertial measurement units (IMUs), barometers, and other sensor information to construct attitude control models, and combine these with feedback control algorithms using fixed parameters to adjust roll, pitch, and yaw. Some technical solutions incorporate visual perception for target recognition or job positioning; however, visual information is mostly used as an auxiliary reference, primarily for path planning or job area identification, and a systematic modeling mechanism for the spatial relationships of spraying operations has not yet been established.

[0003] During spraying operations, changes in spray flow rate, spray reaction force, and load cause continuous changes in the spatial relationships and dynamic characteristics of the UAV. Existing technologies mostly treat these factors as independent disturbances and provide simple compensation, lacking the ability to model the propagation and evolution of disturbances within the spatial structure. Furthermore, existing stability control methods are typically based on local errors or instantaneous states, making it difficult to reflect the formation process of unstable trends in a timely manner. This results in stability control exhibiting lag and locality, making it difficult to adapt to the complex and dynamic changes in spatial relationships during spraying operations. Summary of the Invention

[0004] The technical problem to be solved and the technical task proposed by this invention is to improve and refine existing technical solutions, and to provide a stability control method and system for spray painting drones based on AI vision, so as to achieve stable control of spray painting drones. To this end, this invention adopts the following technical solution.

[0005] Firstly, a stability control method for a painting drone based on AI vision is provided, which includes the following steps: Visual data from the spraying drone is collected and preprocessed to form a visual data sequence; AI visual feature extraction is performed based on visual data sequences, and the sequences are arranged according to a unified time index to form a visual feature sequence; Construct a dynamic spatial relationship graph sequence based on visual feature sequences; A graph-structured reserve calculation model is constructed based on a dynamic spatial relationship graph sequence. Propagation constraints are determined, and under the constraints of the propagation constraints, the reserve state performs state evolution according to a unified time index to generate a reserve state sequence. Information on changes in spraying flow rate, spraying reaction force, and load during the spraying operation is obtained and mapped as spraying disturbance input terms into the state evolution process to obtain a reserve state sequence containing the effects of spraying disturbance. Based on the reserve state sequence, the change in reserve state under adjacent time indices is calculated, and a stability risk sequence is generated by combining the propagation constraint relationship and visual confidence features. Based on the stability risk sequence, a stability-sensitive control quantity generation mapping is constructed, control weights are assigned to each reserve state component, and an attitude control quantity sequence is generated. The attitude control sequence is input into the flight control actuator, and the roll, pitch and yaw of the painting drone are adjusted in a closed loop according to a unified time index to achieve stability control of the painting drone.

[0006] Optionally, the formation of the visual data sequence includes: During the spraying operation, the raw visual data of the spraying operation scene is continuously collected by the spraying drone, and a corresponding collection time mark is attached to each frame of raw visual data. Distortion correction processing is performed on the raw visual data, and the radial and tangential distortions caused by the imaging optical system are reverse-mapped and corrected based on the pre-calibrated imaging parameters. Based on the acquisition time stamp, time alignment processing is performed on the original visual data after distortion correction. The original visual data after time alignment is processed by scale normalization, which maps the image resolution, pixel spatial scale and brightness to a standardized numerical range. The raw visual data, after distortion correction, time alignment, and scale normalization, are arranged in a unified time index order to form a visual data sequence.

[0007] Optionally, the formation of the visual feature sequence includes: Read the visual data sequence and process the visual data sequence frame by frame according to a unified time index to obtain the visual data corresponding to each unified time index; AI visual feature extraction is performed based on the visual data corresponding to each unified time index to calculate the boundary features of the spraying target. During the AI ​​visual feature extraction process, the surface structure information of the sprayed target in the visual data is used to calculate the surface orientation features of the sprayed target. Based on visual data under adjacent unified time index, cross-time correlation processing is performed on the boundary features and surface orientation features of the sprayed target to calculate the relative pose features of the UAV. Based on the output stability of the sprayed target boundary features, sprayed target surface orientation features, and UAV relative pose features under a continuous unified time index, the visual confidence feature is calculated. The boundary features of the sprayed target, the orientation features of the sprayed target surface, the relative pose features of the UAV, and the visual confidence features are arranged in a unified time index order to form a visual feature sequence.

[0008] Optionally, the construction of the dynamic spatial relationship graph sequence includes: Read the visual feature sequence and parse it according to the unified time index to obtain the sprayed target boundary features, sprayed target surface orientation features and UAV relative pose features corresponding to each unified time index; For each unified time index, a mapping process is performed based on the boundary features of the sprayed target, the orientation features of the sprayed target surface, and the relative pose features of the UAV to construct a set of nodes; For a set of nodes under the same unified time index, perform mapping processing based on the spatial relative relationships between the nodes in the set to construct a connection set; The set of nodes and the set of connections built under the same unified time index are combined to generate a corresponding dynamic spatial relationship graph. The dynamic spatial relationship graphs corresponding to each unified time index are arranged in the order of the unified time index to form a sequence of dynamic spatial relationship graphs.

[0009] Optionally, the formation of the reserve state sequence includes: Read the sequence of dynamic spatial relationship graphs and parse the sequence of dynamic spatial relationship graphs time by time according to the unified time index to obtain the set of nodes and the set of connections corresponding to each unified time index; Under the adjacent unified time index, based on whether the spatial relative relationship between each node in the node set has changed, it is determined whether the reserve state is allowed to be propagated between nodes, and propagation constraint relationship is generated; The propagation constraint relationship is constrained based on the connection set; Under adjacent unified time index, the propagation of reserve states that satisfy propagation constraints is constrained according to the direction of change of spatial relative relationship between nodes; Under the combined constraints of propagation constraints and direction constraints, the state evolution of the reserve state is performed according to a unified time index. The reserve states obtained through the state evolution under each unified time index are arranged in the order of the unified time index and correspond one-to-one with the dynamic spatial relationship diagram sequence to form a reserve state sequence.

[0010] Optionally, the generation of the reserve state sequence including the effects of spraying disturbance includes: During the spraying operation, the changes in spraying flow rate, spraying reaction force, and load are obtained according to a unified time index. Numerical scaling is applied to the changes in spray flow rate, spray reaction force, and load. Based on the changes in spraying flow rate, spraying reaction force, and load under the same unified time index, a mapping process is performed to map the changes in spraying flow rate, spraying reaction force, and load as spraying disturbance input items. According to the unified time index, the spraying disturbance input is injected into the state evolution process corresponding to the reserve state sequence; The reserve states after the spraying disturbance input are injected are arranged in a unified time index order to obtain a reserve state sequence that includes the effects of the spraying disturbance.

[0011] Optionally, the generation of the stability risk sequence includes: Based on the reserve state sequence that includes the impact of spraying disturbance, reserve states corresponding to adjacent time indices are selected according to a unified time index. For the reserve status corresponding to adjacent time indices, calculate the change in reserve status; By combining the propagation constraint relationships between nodes in the dynamic spatial relationship graph sequence, the changes in the reserve state are constrained. By combining the visual confidence features in the visual feature sequence, the change in the reserve state after processing by the propagation constraint relationship is corrected; Based on the changes in the reserve state after processing by propagation constraints and correction by visual confidence features, a stability risk sequence characterizing the attitude stability of the spraying UAV is generated according to a unified time index.

[0012] Optionally, the generation of the attitude control quantity sequence includes: Based on the stability risk sequence, obtain the stability risk value corresponding to each unified time index according to the unified time index; A stability-sensitive control quantity generation mapping is constructed based on the stability risk sequence, and a correspondence is established with the unified time index. The reserve state components corresponding to the stability risk sequence are selected according to the unified time index, and a mapping is generated based on the stability-sensitive control quantity to assign control weights to the reserve state components. Under each unified time index, attitude control quantities are generated based on control weights and reserve state components, and a corresponding relationship is established with the unified time index. The attitude control variables corresponding to each unified time index are arranged in the order of the unified time index to form an attitude control variable sequence.

[0013] Optionally, the closed-loop regulation includes: Based on the attitude control sequence, roll control, pitch control and yaw control are obtained according to the unified time index; The attitude control sequence is input into the flight control actuator of the painting drone, and the roll control, pitch control and yaw control are received according to a unified time index. Under each unified time index, the flight control actuator adjusts the roll of the painting drone according to the roll control quantity, adjusts the pitch of the painting drone according to the pitch control quantity, and adjusts the yaw of the painting drone according to the yaw control quantity. During the spraying operation, the roll, pitch, and yaw states of the spraying drone are obtained according to a unified time index, and the roll, pitch, and yaw states are fed back to the flight control actuator. The flight control actuator performs closed-loop adjustment of the roll, pitch, and yaw of the painting UAV according to the attitude control sequence and the feedback results of roll, pitch, and yaw states, in accordance with a unified time index.

[0014] Secondly, an AI vision-based stability control system for a painting drone is provided, comprising: The visual data processing module is used to collect visual data during the spraying operation and form a visual data sequence with a unified time index. The visual feature extraction module is used to extract the boundary features of the sprayed target, the orientation features of the surface of the sprayed target, the relative pose features of the UAV, and the visual confidence features from the visual data sequence. The spatial relationship graph construction module is used to construct a dynamic spatial relationship graph sequence based on visual features. The reserve calculation module is used to generate a reserve state sequence reflecting the spatial state evolution based on the dynamic spatial relationship graph sequence; The spraying disturbance injection module is used to introduce changes in spraying flow rate, spraying reaction force, and load into the reserve state evolution process; The stability risk generation module is used to generate a stability risk sequence that characterizes the attitude stability of the spraying drone. The attitude control generation module is used to generate an attitude control quantity sequence based on the stability risk sequence. The flight control execution module is used to perform closed-loop adjustment of the attitude of the painting drone based on the attitude control quantity sequence.

[0015] The beneficial effects of this invention are: This invention introduces a dynamic spatial relationship modeling mechanism based on artificial intelligence vision during the spraying operation, enabling a structured expression of the relationship between the spraying drone's operating environment and its own attitude. Compared to existing stability control methods that rely solely on inertia or local visual information, this invention maps the boundary features of the spraying target, the surface orientation features of the spraying target, and the relative pose features of the drone into a unified dynamic spatial relationship graph. This allows the constantly changing spatial relationships during the spraying operation to be continuously characterized in a graph structure, providing a clear and computable spatial constraint basis for subsequent state evolution and stability analysis, effectively improving the integrity and consistency of stability modeling.

[0016] Meanwhile, this invention constructs a graph-structured reserve calculation model based on the dynamic spatial relationship graph sequence, and introduces propagation constraints and directional constraints that are dynamically updated over time, so that the evolution process of the reserve state is strictly controlled by changes in spatial relative relationships and propagation direction constraints. This technique avoids the problem of unstructured propagation of state changes in traditional numerical models, allowing unstable trends to gradually emerge along the real spatial relationship path, thereby achieving early perception and continuous characterization of the attitude change trend of the spraying UAV. Furthermore, the changes in spraying flow rate, spraying reaction force, and load are mapped as spraying disturbance inputs and injected into the state evolution process, so that the spraying disturbance is no longer simply used as compensation for external noise, but directly participates in state evolution modeling, enhancing the system's adaptability to disturbances specific to spraying operations.

[0017] Furthermore, this invention introduces visual confidence features into the stability risk generation process to correct the changes in the reserve state after propagation constraint processing, thereby reducing the impact of visual instability or perception errors on the stability assessment results and improving the reliability of the stability risk sequence. Based on this, by constructing a stability-sensitive control quantity generation map, a correlation is established between the stability risk and the control weights of the reserve state components, enabling the attitude control quantity to adaptively adjust with changes in stability risk. This control mechanism avoids the problem of traditional control methods that only passively correct after significant attitude instability, achieving proactive and continuous closed-loop adjustment of the roll, pitch, and yaw of the painting UAV, thus significantly improving flight stability and operational reliability during the painting process. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of a stability control method for a spraying drone based on AI vision proposed in this invention; Figure 2This is a schematic diagram of the structure for constructing a dynamic spatial relationship graph sequence in the AI ​​vision-based stability control method for spraying drones proposed in this invention; Figure 3 This is a schematic diagram of the closed-loop attitude adjustment system of a painting drone in the stability control method based on AI vision proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] like Figures 1-3 As shown, a stability control method for a painting drone based on AI vision includes the following steps: Visual data from the spraying drone during the spraying operation is collected, and distortion correction, time alignment and scale normalization are performed on the visual data to form a visual data sequence with a corresponding unified time index. AI visual feature extraction is performed based on the visual data sequence to obtain the sprayed target boundary features, sprayed target surface orientation features, UAV relative pose features and visual confidence features corresponding to the unified time index, and arranged according to the unified time index to form a visual feature sequence; A sequence of dynamic spatial relationship graphs is constructed based on the visual feature sequence. Each time index corresponds to a dynamic spatial relationship graph. Each dynamic spatial relationship graph includes a set of nodes and a set of connections. The set of nodes is generated by mapping the boundary features of the sprayed target, the surface orientation features of the sprayed target, and the relative pose features of the UAV. The set of connections is generated by mapping the spatial relative relationships between the nodes in the set of nodes. A graph structure reserve calculation model is constructed based on the dynamic spatial relationship graph sequence. The propagation constraint relationship of the reserve state between nodes is determined according to the connection set. Under the constraint of the propagation constraint relationship, the reserve state performs state evolution according to a unified time index, generating a reserve state sequence that corresponds one-to-one with the dynamic spatial relationship graph sequence. Information on changes in spray flow rate, spray reaction force, and load during the spraying operation is obtained. This information is then mapped to spraying disturbance inputs. The spraying disturbance inputs are injected into the state evolution process corresponding to the reserve state sequence according to a unified time index, resulting in a reserve state sequence containing the effects of spraying disturbances. Based on the reserve state sequence containing the impact of spraying disturbance, the change in reserve state under adjacent time indices is calculated. Combined with the propagation constraint relationship between nodes in the dynamic spatial relationship graph sequence and the visual confidence feature in the visual feature sequence, a stability risk sequence characterizing the attitude stability of the spraying UAV is generated. A stability-sensitive control quantity generation mapping is constructed based on the stability risk sequence. Control weights are assigned to the reserve state components corresponding to the stability risk sequence to generate an attitude control quantity sequence corresponding to a unified time index. The attitude control quantity sequence includes roll control quantity, pitch control quantity, and yaw control quantity. The attitude control sequence is input into the flight control actuator of the painting drone, and the roll, pitch and yaw of the painting drone are adjusted in a closed loop according to a unified time index to achieve stability control of the painting drone during the painting operation.

[0021] In this embodiment, the formation of the visual data sequence includes: During the spraying operation, the raw visual data of the spraying operation scene is continuously collected by the spraying drone, and a corresponding collection time mark is attached to each frame of raw visual data. Distortion correction processing is performed on the raw visual data, and the radial and tangential distortions caused by the imaging optical system are reverse-mapped and corrected based on the pre-calibrated imaging parameters. The reverse mapping correction includes, during the image distortion correction process, using the ideal imaging plane as a reference, mapping each pixel position in the corrected image back to the corresponding pixel position in the original distorted image through a known imaging distortion model, and obtaining the corresponding pixel value from the original image to generate the image after distortion correction. Based on the acquisition time stamp, time alignment processing is performed on the original visual data after distortion correction, so that each frame of original visual data corresponds to a unified time index. The original visual data after time alignment is processed by scale normalization, which maps the image resolution, pixel spatial scale and brightness to a standardized numerical range. The raw visual data, after distortion correction, time alignment, and scale normalization, are arranged in a unified time index order to form a visual data sequence.

[0022] In this embodiment, the formation of the visual feature sequence includes: Read the visual data sequence and process the visual data sequence frame by frame according to the unified time index. Under the constraint of the unified time index, take a single frame of visual data as the smallest processing unit, and perform processing operations independently on each frame of visual data in the visual data sequence according to the time order. For each frame of processing result, retain the unified time index corresponding to it, and obtain the visual data corresponding to each unified time index. AI visual feature extraction is performed based on the visual data corresponding to each unified time index. Artificial intelligence visual analysis methods are used to process the single frame visual data. Structured visual features describing the spatial state of the spraying operation and the relative state of the drone are extracted from the visual data. Spraying target boundary features are calculated. The spraying target boundary features represent the boundary position distribution state of the spraying target in the visual data. During the AI ​​visual feature extraction process, the surface orientation features of the sprayed target are calculated based on the surface structure information of the sprayed target in the visual data. The surface orientation features of the sprayed target characterize the orientation distribution state of the sprayed target surface in the visual data. Based on visual data under adjacent unified time index, cross-time correlation processing is performed on the boundary features and surface orientation features of the spraying target to calculate the relative pose features of the UAV. The relative pose features of the UAV characterize the relative position and relative attitude changes of the spraying UAV under adjacent unified time index. Based on the output stability of the sprayed target boundary features, sprayed target surface orientation features, and UAV relative pose features under a continuous unified time index, the visual confidence feature is calculated. The boundary features of the sprayed target, the orientation features of the sprayed target surface, the relative pose features of the UAV, and the visual confidence features are arranged in a unified time index order to form a visual feature sequence.

[0023] In this embodiment, the construction of the dynamic spatial relationship graph sequence includes: Read the visual feature sequence and parse it according to the unified time index. Based on the unified time index, sort and read each group of visual features in the visual feature sequence in an orderly manner so that each group of visual features corresponds one-to-one with the corresponding time index. Obtain the spraying target boundary features, spraying target surface orientation features and UAV relative pose features corresponding to each unified time index. For each unified time index, a mapping process is performed based on the boundary features of the sprayed target, the orientation features of the sprayed target surface, and the relative pose features of the UAV to construct a node set. The nodes in the node set are generated by mapping the boundary features of the sprayed target, the orientation features of the sprayed target surface, and the relative pose features of the UAV one by one. For a set of nodes under the same unified time index, a mapping process is performed based on the spatial relative relationship between each node in the node set to construct a connection set. The spatial relative relationship is determined by the relative positional relationship, relative directional relationship, and relative change relationship between corresponding nodes in the node set under adjacent unified time indices. The set of nodes and the set of connections built under the same unified time index are combined to generate a corresponding dynamic spatial relationship graph. The dynamic spatial relationship graphs corresponding to each unified time index are arranged in the order of the unified time index to form a sequence of dynamic spatial relationship graphs.

[0024] In this embodiment, the formation of the reserve state sequence includes: Read the sequence of dynamic spatial relationship graphs and parse the sequence of dynamic spatial relationship graphs time by time according to the unified time index to obtain the set of nodes and the set of connections corresponding to each unified time index; Under adjacent unified time indices, based on whether the spatial relative relationship between each node in the node set changes, it is determined whether the reserve state is allowed to propagate between nodes, and a propagation constraint relationship for state evolution is generated. The propagation constraint relationship is a state propagation constraint relationship that is dynamically updated with the unified time index and is jointly determined by the change in the spatial relative relationship of the node set in the dynamic spatial relationship graph sequence under adjacent unified time indices. The propagation constraint relationship is constrained according to the connection set, such that the reserve state is allowed to propagate between corresponding nodes only when the spatial relative relationship of the nodes simultaneously changes and they belong to the directly connected nodes defined by the connection set. Under the adjacent unified time index, the propagation of the reserve state that satisfies the propagation constraint relationship is constrained according to the direction of change of the spatial relative relationship between nodes, so that the reserve state only propagates in the node set along the direction of change of the spatial relative relationship. Under the combined constraints of propagation constraints and direction constraints, the state evolution of the reserve state is performed according to the unified time index, so that the reserve state of each node in the node set under the current unified time index is determined by the reserve state of the node under the previous unified time index and the reserve states of the adjacent nodes that satisfy the propagation constraints. The reserve states obtained through the state evolution under each unified time index are arranged in the order of the unified time index and correspond one-to-one with the dynamic spatial relationship diagram sequence to form a reserve state sequence.

[0025] In this embodiment, the generation of the reserve state sequence including the effects of spraying disturbance includes: During the spraying operation, the changes in spraying flow rate, spraying reaction force, and load are obtained according to a unified time index, and the above parameters are matched one-to-one with the unified time index. The changes in spray flow rate, spray reaction force, and load are processed to unify the numerical scale, so that the changes in spray flow rate, spray reaction force, and load are represented on the same numerical scale. Based on the changes in spraying flow rate, spraying reaction force, and load under the same unified time index, a mapping process is performed to map the changes in spraying flow rate, spraying reaction force, and load as spraying disturbance input items. According to the unified time index, the spraying disturbance input item is injected into the state evolution process corresponding to the reserve state sequence, so that the spraying disturbance input item participates in the state evolution process under each unified time index. The reserve states after the spraying disturbance input are injected are arranged in a unified time index order to obtain a reserve state sequence that includes the effects of the spraying disturbance.

[0026] In this embodiment, the generation of the stability risk sequence includes: Based on the reserve state sequence that includes the impact of spraying disturbance, reserve states corresponding to adjacent time indices are selected according to a unified time index. For the reserve status corresponding to adjacent time indices, the change in the reserve status is calculated. The change in the reserve status is determined by the difference in the value of the reserve status at the corresponding node under adjacent time indices. Combining the propagation constraint relationships between nodes in the dynamic spatial relationship graph sequence, the change in the reserve state is constrained. The constraint processing includes limiting the change in the reserve state based on the determined propagation constraint relationships, so that only the change in the propagation constraint relationships participates in the subsequent stability risk generation process. By combining the visual confidence features in the visual feature sequence, the change in the reserve state after the propagation constraint relationship is corrected. After filtering and limiting the effective propagation range and direction according to the propagation constraint relationship, the remaining change in the reserve state is further adjusted by combining additional confidence or reliability information, so that the change more accurately reflects the real attitude stability change trend during the spraying operation. Based on the changes in the reserve state after processing by propagation constraints and correction by visual confidence features, a stability risk sequence characterizing the attitude stability of the spraying UAV is generated according to a unified time index.

[0027] In this embodiment, the generation of the attitude control quantity sequence includes: Based on the stability risk sequence, obtain the stability risk value corresponding to each unified time index according to the unified time index; A stability-sensitive control quantity generation map is constructed based on the stability risk sequence. The stability-sensitive control quantity generation map is used to characterize the correspondence between stability risk values ​​and control weights, and establishes a correspondence with a unified time index. The reserve state components corresponding to the stability risk sequence are selected according to the unified time index, and a mapping is generated based on the stability-sensitive control quantity. Control weights are assigned to the reserve state components. The allocation of control weights includes assigning different influence coefficients to each reserve state component used for control generation in the reserve state sequence according to the stability risk sequence, so that different reserve state components have different degrees of influence in the attitude control quantity generation process. Under each unified time index, attitude control quantities are generated based on control weights and reserve state components, and a corresponding relationship is established with the unified time index. The attitude control quantities corresponding to each unified time index are arranged in the order of the unified time index to form an attitude control quantity sequence, which includes roll control quantity, pitch control quantity and yaw control quantity.

[0028] In this embodiment, the closed-loop regulation includes: Based on the attitude control sequence, roll control, pitch control and yaw control are obtained according to the unified time index; The attitude control sequence is input into the flight control actuator of the painting drone, and the roll control, pitch control and yaw control are received according to a unified time index. Under each unified time index, the flight control actuator adjusts the roll of the painting drone according to the roll control quantity, adjusts the pitch of the painting drone according to the pitch control quantity, and adjusts the yaw of the painting drone according to the yaw control quantity. During the spraying operation, the roll, pitch, and yaw states of the spraying drone are obtained according to a unified time index, and the roll, pitch, and yaw states are fed back to the flight control actuator. Based on the attitude control sequence and feedback results of roll, pitch, and yaw states, the flight control actuator performs closed-loop adjustment of the roll, pitch, and yaw of the painting drone according to a unified time index. After inputting the pitch and yaw control quantities into the flight control actuator under the unified time index, the corresponding pitch and yaw states are obtained as feedback under subsequent unified time indexes. This allows the flight control actuator to continuously adjust the pitch and yaw of the painting drone based on the control quantities and feedback states, thereby achieving stable control of the painting drone during the painting operation.

[0029] A stability control system for a painting drone based on AI vision, comprising: The visual data processing module is used to collect visual data during the spraying operation and form a visual data sequence with a unified time index. The visual feature extraction module is used to extract the boundary features of the sprayed target, the orientation features of the surface of the sprayed target, the relative pose features of the UAV, and the visual confidence features from the visual data sequence. A spatial relationship graph construction module is used to construct a dynamic spatial relationship graph sequence based on the visual features; The reserve calculation module is used to generate a reserve state sequence reflecting the spatial state evolution based on the dynamic spatial relationship graph sequence; The spraying disturbance injection module is used to introduce changes in spraying flow rate, spraying reaction force, and load into the reserve state evolution process; The stability risk generation module is used to generate a stability risk sequence that characterizes the attitude stability of the spraying drone. The attitude control generation module is used to generate an attitude control quantity sequence based on the stability risk sequence. The flight control execution module is used to perform closed-loop adjustment of the attitude of the painting drone based on the attitude control quantity sequence.

[0030] Example 1: To verify the feasibility of this invention in practice, it was applied to the stability control process of a spraying drone in an agricultural plant protection operation scenario. This scenario is located in a contiguous crop planting area, characterized by inconsistent crop heights, significant surface undulations, and continuous pesticide output during spraying. In actual spraying operations, the drone needs to fly stably along the crop's direction of travel at low altitudes while maintaining the relative attitude relationship between the spraying direction and the crop surface to ensure uniform coverage and operational safety. During continuous spraying, as the pesticide is consumed, the load changes, and the spraying flow rate and reaction force continuously disturb the drone's attitude. Traditional stability control methods relying solely on inertial sensing and fixed control parameters struggle to detect these spatial changes in a timely manner, leading to attitude jitter and spraying direction deviation.

[0031] In this operational scenario, after takeoff, the painting drone enters an automated painting operation state, and the onboard vision equipment continuously collects visual information about the painting operation scene. The collected visual data first undergoes distortion correction, time alignment, and scale normalization to ensure consistency of visual information acquired at different times under a unified time index. Subsequently, the system performs artificial intelligence visual feature extraction based on the visual data, extracting the boundary features of the painting target, the surface orientation features of the painting target, and the relative pose features of the drone from single-frame visual data, and generating visual confidence features by combining the stability of the visual output over continuous time. Through the above processing, the spatial structure information of the painting operation area and the relative relationship between the drone and the work object are transformed into a visual feature sequence that changes continuously over time.

[0032] After obtaining the visual feature sequence, the system analyzes the visual features based on a unified time index and maps the boundary features of the sprayed target, the orientation features of the sprayed target surface, and the relative pose features of the UAV into a set of nodes. Simultaneously, it generates a connection set based on the spatial relationships between the nodes, thus constructing a dynamic spatial relationship graph sequence. This dynamic spatial relationship graph can intuitively reflect the changes in spatial relationships during the spraying operation, ensuring that the geometric relationship between the UAV and the sprayed target is no longer merely an implicit parameter but participates in subsequent calculations in a structured form.

[0033] Based on this, the system constructs a graph-structured reserve calculation model based on a dynamic spatial relationship graph sequence. During the spraying operation, as the UAV's attitude and the operating environment change, the spatial relative relationships between nodes at adjacent time indices will adjust. The system dynamically generates propagation constraints accordingly, limiting the propagation range and direction of the reserve state between nodes. Under the constraints of these propagation constraints, the reserve state undergoes state evolution in chronological order, allowing the impact of spatial relationship changes on state evolution to gradually emerge, thus providing a basis for the early identification of unstable trends.

[0034] Meanwhile, the system continuously acquires information on changes in spray flow rate, spray reaction force, and load during the spraying operation, and maps this information into spraying disturbance inputs that are then injected into the evolution process of the reserve state. In this way, spraying disturbances are no longer simply treated as external noise, but directly participate in state evolution modeling, enabling the reserve state to accurately reflect the impact of changes in spraying operation conditions on the stability of the UAV.

[0035] After the state evolution is complete, the system compares the reserve states under adjacent time indices, calculates the changes in the reserve states, and, in conjunction with the established propagation constraints in the dynamic spatial relationship graph, constrains the changes, retaining change information that conforms to the spatial propagation logic. Subsequently, the system introduces visual confidence features to correct the constrained changes, ensuring that visually reliable changes have a more reasonable influence in the stability assessment. Based on the corrected changes, the system generates a stability risk sequence according to a unified time index, reflecting the trend of the painting UAV's attitude stability changing with the operation process from a temporal perspective.

[0036] Based on the stability risk sequence, the system constructs a stability-sensitive control quantity generation map, establishing a correspondence between stability risks and reserve state components, and assigning different control weights to different reserve state components. In this way, the attitude control quantity can dynamically adjust its sensitivity according to changes in stability risk, enabling control behavior to respond at the initial stage of risk occurrence. Finally, the system generates an attitude control quantity sequence including roll, pitch, and yaw control quantities, and inputs these attitude control quantities into the flight control actuator of the painting UAV.

[0037] In the flight control actuator, attitude control variables are applied to the painting UAV according to a unified time index, continuously adjusting roll, pitch, and yaw in a closed loop. The system acquires the UAV's attitude state as feedback information under subsequent time indices, forming a complete closed-loop adjustment mechanism. During actual painting operations, it can be observed that the UAV maintains a relatively stable attitude change trend even under varying loads and continuous painting disturbances. The relative relationship between the painting direction and the surface of the workpiece remains stable, and the painting process exhibits good continuity. By comparing the attitude changes and painting coverage during the operation, it can be verified that this invention can effectively mitigate the impact of painting disturbances and changes in spatial relationships on UAV stability, demonstrating the practicality and reliability of this invention in painting operation stability control.

[0038] Table 1 Comparison of Experimental Data on Stability Control of Spraying Drones

[0039] As shown in Table 1, the mean square error of the attitude angle reveals that the traditional PID control method exhibits a relatively large error of 3.8 degrees during the spraying operation. This is primarily due to the fact that its control strategy does not consider changes in the spraying load and the dynamic changes in spatial relationships. Introducing visual assistance reduces the error to 2.6 degrees, but it still struggles to systematically model the disturbance propagation process. The method of this invention further reduces the error to 1.4 degrees, demonstrating that state evolution modeling based on dynamic spatial relationship diagrams and graph structure reserves can more accurately describe the attitude change trend.

[0040] Regarding the attitude fluctuation amplitude index, the traditional method achieves a fluctuation amplitude of 6.5 degrees under the continuous presence of spraying reaction force, while the method of this invention controls the fluctuation within 2.1 degrees, a reduction of more than 60%. This improvement stems from the restriction of the state evolution path by the propagation constraint relationship, avoiding the unconstrained diffusion of unstable trends in the system.

[0041] Regarding the disturbance recovery time, the traditional method requires about 1.9 seconds to recover stability after a change in spray flow rate, while the method of this invention shortens the recovery time to 0.6 seconds, indicating that the stability risk sequence can reflect the unstable trend in advance, allowing the control quantity to start adjusting before the attitude becomes obviously unstable.

[0042] The deviation of coating uniformity is an important indicator of work quality. Traditional methods have a deviation of nearly 18.7 percentage points, while the method of this invention reduces it to 6.9 percentage points, showing that stable posture control directly improves the coating effect.

[0043] In terms of overall stability score, the method of this invention achieved 85 points, which is higher than the comparison method. This fully demonstrates that the comprehensive application of AI vision, dynamic spatial relationship modeling and stability-sensitive control strategy has a stronger stability guarantee capability under complex spraying operation conditions.

[0044] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A stability control method for a spray painting drone based on AI vision, characterized in that, Includes the following steps: Visual data from the spraying drone is collected and preprocessed to form a visual data sequence; AI visual feature extraction is performed based on visual data sequences, and the sequences are arranged according to a unified time index to form a visual feature sequence; Construct a dynamic spatial relationship graph sequence based on visual feature sequences; A graph-structured reserve calculation model is constructed based on a dynamic spatial relationship graph sequence. Propagation constraints are determined, and under the constraints of the propagation constraints, the reserve state performs state evolution according to a unified time index to generate a reserve state sequence. Information on changes in spraying flow rate, spraying reaction force, and load during the spraying operation is obtained and mapped as spraying disturbance input terms into the state evolution process to obtain a reserve state sequence containing the effects of spraying disturbance. Based on the reserve state sequence, the change in reserve state under adjacent time indices is calculated, and a stability risk sequence is generated by combining the propagation constraint relationship and visual confidence features. Based on the stability risk sequence, a stability-sensitive control quantity generation mapping is constructed, control weights are assigned to each reserve state component, and an attitude control quantity sequence is generated. The attitude control sequence is input into the flight control actuator, and the roll, pitch and yaw of the painting drone are adjusted in a closed loop according to a unified time index to achieve stability control of the painting drone.

2. The stability control method for a painting drone based on AI vision according to claim 1, characterized in that, The formation of the visual data sequence includes: During the spraying operation, the raw visual data of the spraying operation scene is continuously collected by the spraying drone, and a corresponding collection time mark is attached to each frame of raw visual data. Distortion correction processing is performed on the raw visual data, and the radial and tangential distortions caused by the imaging optical system are reverse-mapped and corrected based on the pre-calibrated imaging parameters. Based on the acquisition time stamp, time alignment processing is performed on the original visual data after distortion correction. The original visual data after time alignment is processed by scale normalization, which maps the image resolution, pixel spatial scale and brightness to a standardized numerical range. The raw visual data, after distortion correction, time alignment, and scale normalization, are arranged in a unified time index order to form a visual data sequence.

3. The stability control method for a painting drone based on AI vision according to claim 1, characterized in that, The formation of the visual feature sequence includes: Read the visual data sequence and process the visual data sequence frame by frame according to a unified time index to obtain the visual data corresponding to each unified time index; AI visual feature extraction is performed based on the visual data corresponding to each unified time index to calculate the boundary features of the spraying target. During the AI ​​visual feature extraction process, the surface structure information of the sprayed target in the visual data is used to calculate the surface orientation features of the sprayed target. Based on visual data under adjacent unified time index, cross-time correlation processing is performed on the boundary features and surface orientation features of the sprayed target to calculate the relative pose features of the UAV. Based on the output stability of the sprayed target boundary features, sprayed target surface orientation features, and UAV relative pose features under a continuous unified time index, the visual confidence feature is calculated. The boundary features of the sprayed target, the orientation features of the sprayed target surface, the relative pose features of the UAV, and the visual confidence features are arranged in a unified time index order to form a visual feature sequence.

4. The stability control method for a painting drone based on AI vision according to claim 1, characterized in that, The construction of the dynamic spatial relationship graph sequence includes: Read the visual feature sequence and parse it according to the unified time index to obtain the sprayed target boundary features, sprayed target surface orientation features and UAV relative pose features corresponding to each unified time index; For each unified time index, a mapping process is performed based on the boundary features of the sprayed target, the orientation features of the sprayed target surface, and the relative pose features of the UAV to construct a set of nodes; For a set of nodes under the same unified time index, perform mapping processing based on the spatial relative relationships between the nodes in the set to construct a connection set; The set of nodes and the set of connections built under the same unified time index are combined to generate a corresponding dynamic spatial relationship graph. The dynamic spatial relationship graphs corresponding to each unified time index are arranged in the order of the unified time index to form a sequence of dynamic spatial relationship graphs.

5. The stability control method for a painting drone based on AI vision according to claim 1, characterized in that, The formation of the reserve state sequence includes: Read the sequence of dynamic spatial relationship graphs and parse the sequence of dynamic spatial relationship graphs time by time according to the unified time index to obtain the set of nodes and the set of connections corresponding to each unified time index; Under the adjacent unified time index, based on whether the spatial relative relationship between each node in the node set has changed, it is determined whether the reserve state is allowed to be propagated between nodes, and propagation constraint relationship is generated; The propagation constraint relationship is constrained based on the connection set; Under adjacent unified time index, the propagation of reserve states that satisfy propagation constraints is constrained according to the direction of change of spatial relative relationship between nodes; Under the combined constraints of propagation constraints and direction constraints, the state evolution of the reserve state is performed according to a unified time index. The reserve states obtained through the state evolution under each unified time index are arranged in the order of the unified time index and correspond one-to-one with the dynamic spatial relationship diagram sequence to form a reserve state sequence.

6. The stability control method for a painting drone based on AI vision according to claim 1, characterized in that, The generation of the reserve state sequence including the effects of spraying disturbances includes: During the spraying operation, the changes in spraying flow rate, spraying reaction force, and load are obtained according to a unified time index. Numerical scaling is applied to the changes in spray flow rate, spray reaction force, and load. Based on the changes in spraying flow rate, spraying reaction force, and load under the same unified time index, a mapping process is performed to map the changes in spraying flow rate, spraying reaction force, and load as spraying disturbance input items. According to the unified time index, the spraying disturbance input is injected into the state evolution process corresponding to the reserve state sequence; The reserve states after the spraying disturbance input are injected are arranged in a unified time index order to obtain a reserve state sequence that includes the effects of the spraying disturbance.

7. The stability control method for a painting drone based on AI vision according to claim 1, characterized in that, The generation of the stability risk sequence includes: Based on the reserve state sequence that includes the impact of spraying disturbance, reserve states corresponding to adjacent time indices are selected according to a unified time index. For the reserve status corresponding to adjacent time indices, calculate the change in reserve status; By combining the propagation constraint relationships between nodes in the dynamic spatial relationship graph sequence, the changes in the reserve state are constrained. By combining the visual confidence features in the visual feature sequence, the change in the reserve state after processing by the propagation constraint relationship is corrected; Based on the changes in the reserve state after processing by propagation constraints and correction by visual confidence features, a stability risk sequence characterizing the attitude stability of the spraying UAV is generated according to a unified time index.

8. The stability control method for a painting drone based on AI vision according to claim 1, characterized in that, The generation of the attitude control quantity sequence includes: Based on the stability risk sequence, obtain the stability risk value corresponding to each unified time index according to the unified time index; A stability-sensitive control quantity generation mapping is constructed based on the stability risk sequence, and a correspondence is established with the unified time index. The reserve state components corresponding to the stability risk sequence are selected according to the unified time index, and a mapping is generated based on the stability-sensitive control quantity to assign control weights to the reserve state components. Under each unified time index, attitude control quantities are generated based on control weights and reserve state components, and a corresponding relationship is established with the unified time index. The attitude control variables corresponding to each unified time index are arranged in the order of the unified time index to form an attitude control variable sequence.

9. The stability control method for a painting drone based on AI vision according to claim 1, characterized in that, The closed-loop regulation includes: Based on the attitude control sequence, roll control, pitch control and yaw control are obtained according to the unified time index; The attitude control sequence is input into the flight control actuator of the painting drone, and the roll control, pitch control and yaw control are received according to a unified time index. Under each unified time index, the flight control actuator adjusts the roll of the painting drone according to the roll control quantity, adjusts the pitch of the painting drone according to the pitch control quantity, and adjusts the yaw of the painting drone according to the yaw control quantity. During the spraying operation, the roll, pitch, and yaw states of the spraying drone are obtained according to a unified time index, and the roll, pitch, and yaw states are fed back to the flight control actuator. The flight control actuator performs closed-loop adjustment of the roll, pitch, and yaw of the painting UAV according to the attitude control sequence and the feedback results of roll, pitch, and yaw states, in accordance with a unified time index.

10. A stability control system for a painting drone based on AI vision, executing the stability control method for a painting drone based on AI vision as described in any one of claims 1 to 9, characterized in that, include: The visual data processing module is used to collect visual data during the spraying operation and form a visual data sequence with a unified time index. The visual feature extraction module is used to extract the boundary features of the sprayed target, the orientation features of the surface of the sprayed target, the relative pose features of the UAV, and the visual confidence features from the visual data sequence. A spatial relationship graph construction module is used to construct a dynamic spatial relationship graph sequence based on the visual features; The reserve calculation module is used to generate a reserve state sequence reflecting the spatial state evolution based on the dynamic spatial relationship graph sequence; The spraying disturbance injection module is used to introduce changes in spraying flow rate, spraying reaction force, and load into the reserve state evolution process; The stability risk generation module is used to generate a stability risk sequence that characterizes the attitude stability of the spraying drone. The attitude control generation module is used to generate an attitude control quantity sequence based on the stability risk sequence. The flight control execution module is used to perform closed-loop adjustment of the attitude of the painting drone based on the attitude control quantity sequence.

Images