High-altitude autonomous paint spraying method based on multi-rotor unmanned aerial vehicle

Abstract

The invention discloses a high-altitude autonomous paint spraying method based on a multi-rotor unmanned aerial vehicle, and relates to the technical field of high-altitude paint spraying, the multi-rotor unmanned aerial vehicle is provided with a binocular camera and spraying equipment, and the autonomous paint spraying method comprises the following steps: step S1, unmanned aerial vehicle self-inspection; s2, after self-inspection is qualified, preliminary spraying work is conducted according to a set paint spraying task route, obstacles are avoided in the flying process, and image information is collected; s3, the unmanned aerial vehicle flies back to the starting point where paint spraying work starts, the original route is repeated once, machine vision carrying a Yolov5 neural network is added, and paint spraying defects occurring during primary paint spraying work are checked; s4, a spraying coefficient is selected according to the defect type, and the regional paint spraying defect is improved; and S5, the unmanned aerial vehicle autonomously returns along the historical path. According to the high-altitude autonomous paint spraying method based on the multi-rotor unmanned aerial vehicle, the paint spraying effect can be evaluated and fed back in real time, the paint spraying effect is optimized, and the quality and effect of operation completion are guaranteed.

Description

Technical Field

[0001] The invention relates to the technical field of high-altitude autonomous painting, and in particular to a high-altitude autonomous painting method based on a multi-rotor unmanned aerial vehicle. Background Art

[0002] In the current field of drone painting technology, although a variety of equipment and systems have been developed to improve painting efficiency and quality, most existing technologies still have some limitations. Especially in terms of intelligence and automation, many drone painting equipment lacks fine control and real-time feedback mechanisms, which to some extent limits the optimization and improvement of painting effects.

[0003] Existing drone painting equipment mostly relies on preset programs and simple sensor feedback. When performing painting tasks, these devices often cannot effectively identify and adapt to complex working environments, such as surfaces of different materials and colors, irregular object shapes, and changes in ambient light. This may lead to problems such as uneven painting, missed spraying, or repeated spraying, which not only affects the final painting quality, but may also increase material waste and operating costs.

[0004] To solve these problems, the concept of intelligent visual painting drones with deep learning came into being. This drone is equipped with an advanced machine vision system that can capture and analyze image data during the painting process in real time. Through deep learning algorithms, the drone can identify the texture, color and shape of the spraying surface, thereby providing real-time evaluation and feedback on the painting effect.

[0005] The core advantage of the intelligent visual painting drone lies in its adaptive painting capability. Based on the deep learning model, the drone can adjust the painting parameters such as paint pressure, nozzle distance and spraying speed according to real-time feedback to ensure that every spraying achieves the best effect. In addition, the intelligent vision system can also identify and avoid obstacles to ensure the safe operation of the drone in complex environments.

[0006] The existing technologies are all based on the improvement and optimization of the hardware of the drone equipment. These improvements and optimizations have certainly promoted the advancement of drone painting technology, but they still cannot fundamentally eliminate the problems that may exist in drone painting operations, such as unevenness, missed spraying or repeated spraying. Summary of the invention

[0007] In response to the above-mentioned technical problems to be solved, the present invention provides a high-altitude autonomous painting method based on a multi-rotor drone, which can perform real-time evaluation and feedback on the painting effect, optimize the painting effect, and ensure the quality and effect of the completed operation.

[0008] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0009] A high-altitude autonomous painting method based on a multi-rotor drone, wherein the multi-rotor drone is provided with a binocular camera and a spraying device, and the autonomous painting method comprises the following steps:

[0010] Step S1, drone self-check;

[0011] Step S2, after the self-inspection is passed, the preliminary spraying work is carried out according to the set painting task route, obstacles are avoided during the flight, and image information is collected;

[0012] Step S3, the drone flies back to the starting point of the painting operation, repeats the original route, and adds machine vision equipped with Yolov5 neural network to check for paint defects that occurred during the initial painting operation;

[0013] Step S4, selecting a spraying coefficient according to the defect type to improve the regional spraying defects;

[0014] Step S5: The drone returns autonomously along the historical path.

[0015] As a further improvement of the above technical solution:

[0016] Preferably, in step S3, when a spraying defect is detected, the specific coordinates and actual area of ​​the actual target in the camera coordinate system are calculated according to the pixel coordinates and target contour in the image captured by the binocular camera, and the calculation process is as follows:

[0017] S3-1, first define three coordinate systems, namely the pixel coordinate system, the image coordinate system and the camera coordinate system, and the three share a common point P; the origin of the pixel coordinate system is O0, and the coordinates of P in the system are (u, v), which means that the pixel points occupied by P are in the uth column and the vth column in the image; the center of the image plane is the origin of the image coordinate system O1, let the coordinates of point P in the system be (x, y), and the coordinates of point O1 in the pixel coordinate system be (u0, v0), let d x d y As the physical size of the unit pixel on the horizontal axis and the vertical axis respectively, there is the following relationship between the two coordinate systems:

[0018]

[0019] It can be expressed as a homogeneous matrix:

[0020]

[0021] The inverse relationship is expressed as:

[0022]

[0023] S3-2, let point O be the optical center of the camera, Z cis the optical axis of the camera, optical axis Z c The point of intersection with the image plane is O1, which is the origin of the image coordinate system. Let the distance between OO1 be the focal length f of the camera, and the coordinate of point P in the camera coordinate system is (x c ,y c ,z c ), according to the triangle similarity principle, the transformation relationship from the camera coordinate system to the image coordinate system is:

[0024]

[0025] The following matrix is ​​used to represent it:

[0026]

[0027] S3-3, let the optical center of the left camera in the binocular camera be O l , the optical center of the right camera is O r , O l O r The distance between them is B, the focal lengths of the two cameras are the same as f, and they are on the same plane. The two cameras acquire image information and return the coordinates and type information of the identified defects after processing by the Yolov5 algorithm. When the two cameras identify the same defect, the imaging points of the defect on the two cameras are P and l (u l ,v) and P r (u r ,v), the horizontal coordinates are inconsistent, the pixel height and width of the defect are h and w respectively; let d x ,d y As the physical size of the unit pixel on the horizontal axis and the vertical axis respectively, according to the relationship between the pixel coordinate system and the image coordinate system, let P l and P r Mapped to P in the image coordinate system l ′ (x l ,y) and P r ′ (x r ,y), then:

[0028]

[0029] According to the relationship between the image coordinate system and the camera coordinate system, the following equation is satisfied:

[0030]

[0031] Let x c =x cl , D = x l -x r, D represents the disparity of two points in the image coordinate system, and the following formula is obtained:

[0032]

[0033] Get the three-dimensional coordinates P(x c ,y c ,z c ), the coordinate is established based on the optical center of the left camera; if the center of the left camera and the right camera is used as the reference, then for x c Calibrate and set the coordinates of point P after calibration to P(x ′ c ,y c ,z c ), the calibration process is as follows:

[0034]

[0035] P(x ′ c ,y c ,z c ) as the three-dimensional coordinates of the target defect to the camera;

[0036] S3-4, assuming that the area of ​​the defect detection frame in the camera coordinate system is S, and the height and width in the image coordinate system are H and W respectively, then the relationship is satisfied:

[0037]

[0038] Convert the height and width in the image coordinate system to the camera coordinate system to get the actual height and width, set as H ′ and W ′ , let the actual area be S ′ , satisfying the relationship:

[0039]

[0040] Substitution Get the final actual area:

[0041]

[0042] Preferably, in step S4, when improving the paint defects in the area, the drone hovers at a fixed point for spraying, and uses the optical flow method for auxiliary positioning to correct the position error according to the offset direction.

[0043] Preferably, in step S4, the method of auxiliary positioning is specifically:

[0044] Assume that the light intensity at a feature in the image is I(x,y,t), where (x,y) is the coordinate of the pixel occupied by the feature, and t represents the time dimension at this moment; between adjacent frames, there is a slight increment in the coordinates of the feature, set to (dx,dy), and the change between frames is set to dt. The light intensity does not change under slight changes, then the relationship is as follows:

[0045] I(x,y,t)=I(x+dx,y+dy,t+dt) (2)

[0046] Taylor expansion of I(x+dx,y+dy,t+dt) at the point (x,y,t) yields the following equation (3), where ξ is an infinitesimal term:

[0047]

[0048] Substitute equation (3) into the right side of equation (2) and divide both sides by dt to obtain equation (4):

[0049]

[0050] Let u, v be the velocity vectors of the optical flow along the x-axis and y-axis respectively, and we get:

[0051]

[0052] Order I x , I y , I t Respectively represent the partial derivatives of the grayscale of the pixel in the image along the X, Y, and T directions:

[0053]

[0054] Formula (4) can be written as:

[0055] I x u+I y v+I t =0 (5)

[0056] Among them, I x , I y , I t From the image data, we can find the light loss (u, v), get the position of the pixel in the next frame, and know the offset direction of the drone.

[0057] Compared with the prior art, the high-altitude autonomous painting method based on a multi-rotor drone provided by the present invention has the following advantages:

[0058] (1) The high-altitude autonomous painting method based on a multi-rotor drone of the present invention is equipped with a deep learning vision system. The system is based on the Yolov5 neural network model and uses a deep learning algorithm to achieve real-time recognition and classification of spray targets, thereby improving the accuracy and efficiency of spraying, reducing human operating errors, and ensuring the quality of spraying work, while increasing the operation speed and reducing labor costs.

[0059] (2) The high-altitude autonomous painting method based on a multi-rotor drone of the present invention has a real-time feedback and automatic adjustment mechanism: the system can respond to the detection results of the Yolov5 algorithm in real time, and automatically adjust the parameters of the drone painting operation to achieve closed-loop control.

[0060] (3) The high-altitude autonomous painting method based on a multi-rotor drone of the present invention, with high-precision autonomous navigation and positioning, combined with traditional GPS and optical flow methods, can realize accurate autonomous navigation and positioning of the drone in complex environments such as outdoor high altitudes, which can effectively improve the safety of operations, avoid collisions and accidental spraying, and at the same time improve the stability of the drone flight to ensure controllable flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a schematic diagram of the distance measurement of the binocular camera of the present invention.

[0062] Figure 2 This is a schematic diagram of the conversion between the image and the pixel coordinate system of the present invention.

[0063] Figure 3 Schematic diagram of the conversion between the image and the camera coordinate system of the present invention

[0064] Figure 4 It is a schematic diagram of the application of the present invention.

[0065] Figure 5 It is a schematic diagram of the drone of the present invention.

[0066] Figure 6 It is a work implementation flow chart of the present invention. DETAILED DESCRIPTION

[0067] The specific embodiments of the present invention are described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0068] like Figures 1 to 6As shown in , the high-altitude autonomous painting system based on a multi-rotor drone of the present invention includes a visual module, an obstacle avoidance module, a flight control module, an information processing module, an information interaction device, and a spraying control module, and the visual module, the obstacle avoidance module, the information processing module, the spraying control module, and the information interaction device are all connected to the flight control module. The drone of the present invention adopts the drone in the prior art, which is a relatively mature drone structure.

[0069] In this embodiment, the vision module includes a global positioning system (GPS) and a binocular vision unit. GPS is a device that uses satellite signals to determine the precise position of a receiver on the earth. It uses triangulation to calculate the distance between the receiver and at least four satellites to determine the three-dimensional position (latitude, longitude and altitude) of the receiver. GPS provides location information for drone flight and sends this information directly to the flight control module. It is the source of positioning information for drones.

[0070] The binocular vision unit consists of two cameras with fixed focal lengths. By measuring the pixel position of the target object in the captured image, it infers the relative position of the object and the drone, especially the drone and the paint module, and then makes fine position adjustments to achieve precise painting effects.

[0071] In this embodiment, the obstacle avoidance module includes a laser radar obstacle avoidance unit. The laser radar is used as a sensor of the drone, and its long detection distance and strong robustness can be used to execute obstacle avoidance and path planning algorithms. In this embodiment, a 360-degree single-line laser radar is used to detect obstacles within a 360-degree range and provide angle-distance information, and guide the drone to autonomously avoid obstacles based on the location information provided by GPS. The laser radar is connected to the onboard computer through a serial port, and the program on the onboard computer analyzes the point cloud information of the laser radar to achieve the obstacle avoidance function.

[0072] In this embodiment, the flight control module includes a main control chip and an onboard computer. The main control chip adopts an STM32F427 processor chip, which integrates sensors such as an attitude sensor, an air pressure sensor, and a magnetometer, and runs Ardupilot flight control software (with its own position correction PID algorithm). It can receive positioning information from GPS and run the PID algorithm to correct position errors in real time; at the same time, it receives control instructions from the onboard computer to complete the control of position, speed, and heading.

[0073] In this embodiment, the spray control module includes a spray head and a spray tank. The spray head can dynamically adjust parameters such as pressure to adjust different types of spray defects. The module is directly connected to the drone itself, and the drone can directly control the switch of the spray module. The operator can send commands to the drone on the ground, and the drone controls the spray module, so that the operator can indirectly remotely control the spray module.

[0074] The onboard computer uses the Cortex-A57 architecture chip as an embedded processor, equipped with a Linux operating system, and has a faster computing speed. The onboard computer is a place where multi-party information fusion and positioning data are solved. It is the brain of the drone flight control. The onboard computer is connected to the main control chip through a serial port to read various information of the main control chip. In this embodiment, the main control chip uses an STM32F427 chip. Information includes but is not limited to attitude, altitude, etc., and the onboard computer sends flight control instructions to the main control chip to control the position, speed, heading, altitude, etc. of the aircraft. The binocular camera is connected to the onboard computer, and the onboard computer performs image analysis and position solution. The binocular vision unit is connected to the information interaction device through an information processing module, and various information including visual positioning information, drone status information, etc. are released in real time. The spray control module is connected to the onboard computer, and the onboard computer is used to release the control spray module.

[0075] In this embodiment, the information processing module includes a data transmission module and an image transmission module; the data transmission module is connected to the ground terminal through a data transmission communication link, which is used for ground personnel to monitor the status of the drone in real time; the image transmission module is connected to the ground terminal through an image transmission communication link, which is used to transmit the images taken by the binocular vision unit to the information interaction device in real time.

[0076] In this embodiment, the information interaction device is composed of multiple components, including but not limited to a display screen, an information transmission unit, and a central processing unit. These devices can be in various forms such as a personal computer, a smart phone, a tablet computer, or a portable wearable device. The function of the display screen is to display key information, such as location data, real-time images captured by the drone camera, flight path information, and the current flight status of the drone. As the core component of the drone system, the information transmission unit is responsible for establishing a data transmission link between the drone and the ground control station, so that ground operators can monitor the operation of the drone in real time. The central processing unit (CPU) is the core of the entire computer system and is responsible for executing all information processing tasks and program operations.

[0077] The high-altitude autonomous painting method based on a multi-rotor drone of the present invention comprises the following steps:

[0078] Step S1, the drone starts self-checking. If there is no problem, it proceeds to the next step. If there is a problem, it repeats this step. When the number of self-checks reaches the set value, the drone issues an alarm to remind the operator to check.

[0079] Step S2, the drone takes off vertically and autonomously plans its flight according to the set painting task route. During the flight, the onboard computer reads the obstacle data of the laser radar obstacle avoidance unit, and the binocular vision unit reads the image information and transmits it to the information interaction device (here only the image information is obtained without solving it). The laser radar obstacle avoidance unit detects the target by emitting a laser beam, and forms a point cloud and obtains data by collecting the reflected beam to avoid obstacles.

[0080] The operator views the images returned by the drone in real time when it is flying at high altitude. When the drone is observed to fly to the starting point of the painting operation, the operator remotely operates the spray module through the information interaction device, opens the spray port, sprays paint according to the default spray pressure coefficient, and starts the painting operation along the preset route. When the entire route is completed, the operator remotely operates the spray module again to close the spray port, indicating that the preliminary spraying work is completed.

[0081] In step S3, the drone flies back to the starting point of the painting operation (i.e., the position where the spray port is first opened), repeats the original route, and adds machine vision equipped with Yolov5 neural network to check for paint defects that may occur during the initial painting work.

[0082] When training the model of the Yolov5 neural network, the possible defects in this embodiment are divided into five types, namely shadow, scratches, bubble, uneven_color, and texture_difference. The re-spraying pressure coefficient corresponding to each defect is inconsistent, corresponding to different re-spraying methods, which are set by the operator and will not be elaborated here.

[0083] When spraying defects are detected, the pixel coordinates and target outline in the image captured by the binocular camera are solved into the specific coordinates and actual area of ​​the actual target in the camera coordinate system. The solution process is as follows:

[0084] S3-1, first define three coordinate systems, namely the pixel coordinate system, the image coordinate system, and the camera coordinate system. The relationship between the three is given in the attached figure, and they all share a point P. The pixel coordinate system takes the upper left corner of the image as the origin O0, and the coordinates of P in the system are set to (u, v), indicating that the pixel points occupied by P are in the uth column and the vth column in the image. In addition, the center of the image plane is taken as the origin O1 of the image coordinate system, and the coordinates of point P in the system are set to (x, y), and the coordinates of point O1 in the pixel coordinate system are set to (u0, v0), and d x d y As the physical size of the unit pixel on the horizontal axis and the vertical axis respectively, there is the following relationship between the two coordinate systems:

[0085]

[0086] It can be expressed as a homogeneous matrix:

[0087]

[0088] Its inverse relationship is expressed as:

[0089]

[0090] S3-2, step S3-1 describes the relationship between the pixel coordinate system and the image coordinate system. Next, the relationship between the camera coordinate system and the image coordinate system is described. As shown in the figure, let point O be the optical center of the camera, Z c is the optical axis of the camera, optical axis Z c The intersection point with the image plane is O1, which is the origin of the image coordinate system. Let the distance between OO1 be the focal length f of the camera, and the coordinates of point P in the camera coordinate system be (x c ,y c ,z c ). According to the triangle similarity principle, the transformation relationship from the camera coordinate system to the image coordinate system is as follows:

[0091]

[0092] The following matrix is ​​used to represent it:

[0093]

[0094] S3-3, after knowing the relationship between the three coordinate systems, start the actual solution. Assume that the optical center of the left camera in the binocular camera is O l , the optical center of the right camera is O r , O l O rThe distance between them is B, and the focal lengths of both cameras are the same as f, and they are on the same plane, that is, the horizontal coordinates of their projection centers are equal. Both cameras can obtain image information and return the coordinates and type information of the identified defects after processing by the Yolov5 algorithm. When the two cameras identify the same defect, the imaging points of the defect on the two cameras are P and l (u l ,v) and P r (u r ,v), only the horizontal coordinates are inconsistent, which are the coordinates in the pixel coordinate system, and the pixel height and width occupied by the defect are h and w respectively. Let d x d y As the physical size of the unit pixel on the horizontal and vertical axes respectively. According to the relationship between the pixel coordinate system and the image coordinate system, let P l and P r Mapped to P in the image coordinate system l ′ (x l ,y) and P r ′ (x r ,y), then:

[0095]

[0096] At the same time, according to the relationship between the image coordinate system and the camera coordinate system, the following formula is satisfied:

[0097]

[0098] Let x c =x cl , D = x l -s r , D represents the disparity of two points in the image coordinate system, and the above equations are combined to obtain the final equation:

[0099]

[0100] At this point, the three-dimensional coordinates P(x c ,y c ,z c ), the coordinates are established based on the optical center of the left camera. If the center of the left and right cameras is used as the reference, s c After a slight calibration, let the coordinates of point P after calibration be P(x ′ c ,y c ,z c ), the calibration process is as follows:

[0101]

[0102] Finally, P(x ′ c ,y c ,z c ) as the 3D coordinates of the target defect to the camera.

[0103] S3-4, next calculate the actual area of ​​the defect. As defined in S3-3, the pixel height and width of the defect are h and w respectively. And let d x ,d y As the physical size of the unit pixel on the horizontal and vertical axes respectively, let the area of ​​the defect detection frame in the camera coordinate system be S, and the height and width in the image coordinate system be H and W respectively, then the relationship is satisfied:

[0104]

[0105] According to the triangle similarity principle, the height and width in the image coordinate system are converted to the camera coordinate system to obtain the actual height and width, which are set as H ′ and W ′ , and let the actual area be S ′ , satisfying the relationship:

[0106]

[0107] Substitution Get the final actual area:

[0108]

[0109] In step S4, the data calculated in the previous step is returned to the information processing module to guide the drone to align with the target area. After alignment, the drone autonomously selects the corresponding spray coefficient according to the defect type and the calculated actual area, and then turns on the nozzle to improve the paint defects in the corresponding area.

[0110] When improving paint defects in the corresponding areas, the drone needs to hover at a fixed point for spraying. However, when the drone is close to large buildings, the GPS signal it searches for is relatively weak, resulting in deviations in the drone's positioning, which in turn affects the stability of the drone's hovering and the fixed-point spraying effect. Therefore, the optical flow method is used visually to assist in positioning, helping the drone to correct the position error according to the offset direction when the position deviates.

[0111] The specific mathematical derivation process is as follows:

[0112] Assume that the light intensity at a feature in the image is I(x,y,t), where (x,y) is the coordinate of the pixel occupied by the feature, and t represents the time dimension at this moment; between adjacent frames, there is a slight increment in the coordinates of the feature, set to (dx,dy), and the change between frames is set to dt. The light intensity does not change under slight changes, then the relationship is as follows:

[0113] I(x,y,t)=I(x+dx,y+dy,t+dt) (2)

[0114] Taylor expansion of I(x+dx,y+dy,t+dt) at the point (x,y,t) yields the following equation (3), where ξ is an infinitesimal term:

[0115]

[0116] Substitute equation (3) into the right side of equation (2), ignore the infinitesimal terms, and divide both sides by dt to obtain equation (4):

[0117]

[0118] Let u, v be the velocity vectors of the optical flow along the x-axis and y-axis respectively, and we get:

[0119]

[0120] At the same time, let I x , I y , I t Respectively represent the partial derivatives of the grayscale of the pixel in the image along the X, Y, and T directions:

[0121]

[0122] In summary, formula (4) can be written as the following formula (5):

[0123] I x u+I y v+I t =0 (5)

[0124] Among them, I x , I y , I t All of them can be obtained from the image data. We need to find the light loss (u, v), but since there is only one formula but two unknowns, we cannot find the exact value of (u, v). Therefore, we can only introduce new constraints from different angles. At present, there are mature technologies in this area, such as gradient-based methods, matching-based methods, and energy-based methods. We will not go into details here, but just explain that there is a way to find a more accurate value of (u, v). Therefore, we can know the direction and speed of the drone's offset, which can assist the drone in positioning when spraying paint at a fixed point.

[0125] When the drone completes the planned painting path again, step S4 is completed, and the operator can also repeat steps S3 and S4 to ensure the completion of the task. If repetition is not required or has been completed, step S5 is entered.

[0126] Step S5: The drone returns autonomously along the historical path.

[0127] The present invention mainly adds the visual element of neural network so that the drone can change the spraying parameters in real time according to the identified spraying results when performing the painting work, thereby achieving more accurate spraying; secondly, the optical flow method is added in the vision, and the small differences between adjacent images sent back by the binocular camera can assist in judging the position offset of the drone, solve the problem of inaccurate ordinary GPS positioning near buildings, and thus perform calibration auxiliary positioning, improve the stability of the drone flying near large equipment, and better complete the painting operation.

[0128] The above implementation cases are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, they are not intended to limit the present invention. Therefore, any simple modification, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A high-altitude autonomous painting method based on a multi-rotor drone, wherein the multi-rotor drone is provided with a binocular camera and a spraying device, characterized in that: The autonomous painting method comprises the following steps: Step S1, drone self-check; Step S2, after the self-inspection is passed, the preliminary spraying work is carried out according to the set painting task route, obstacles are avoided during the flight, and image information is collected; Step S3, the drone flies back to the starting point of the painting operation, repeats the original route, and adds machine vision equipped with Yolov5 neural network to check for paint defects that occurred during the initial painting operation; Step S4, selecting a spraying coefficient according to the defect type to improve the regional spraying defects; Step S5: The drone returns autonomously along the historical path.

2. The high-altitude autonomous painting method based on a multi-rotor drone according to claim 1 is characterized in that: In step S3, when a spraying defect is detected, the pixel coordinates and target contour in the image captured by the binocular camera are solved into the specific coordinates and actual area of ​​the actual target in the camera coordinate system. The solving process is as follows: S3-1, first define three coordinate systems, namely the pixel coordinate system, the image coordinate system and the camera coordinate system, and the three share a common point P; the origin of the pixel coordinate system is O0, and the coordinates of P in the system are (u, v), which means that the pixel points occupied by P are in the uth column and the vth column in the image; the center of the image plane is the origin of the image coordinate system O1, let the coordinates of point P in the system be (x, y), and the coordinates of point O1 in the pixel coordinate system be (u0, v0), let d x d y As the physical size of the unit pixel on the horizontal and vertical axes respectively, there is the following relationship between the image coordinate system and the pixel coordinate system: It can be expressed as a homogeneous matrix: The inverse relationship is expressed as: S3-2, let point O be the optical center of the camera, Z c is the optical axis of the camera, optical axis Z c The point of intersection with the image plane is O1, which is the origin of the image coordinate system. Let the distance between OO1 be the focal length f of the camera, and the coordinate of point P in the camera coordinate system is (x c ,y c ,z c ), according to the triangle similarity principle, the transformation relationship from the camera coordinate system to the image coordinate system is: The following matrix is ​​used to represent it: S3-3, let the optical center of the left camera in the binocular camera be O l , the optical center of the right camera is O r , O l O r The distance between them is B, the focal lengths of the two cameras are the same as f, and they are on the same plane. The two cameras acquire image information and return the coordinates and type information of the identified defects after processing by the Yolov5 algorithm. When the two cameras identify the same defect, the imaging points of the defect on the two cameras are P and l (u l ,v) and P r (u r ,v), the horizontal coordinates are inconsistent, the pixel height and width of the defect are h and w respectively; let d x d y As the physical size of the unit pixel on the horizontal axis and the vertical axis respectively, according to the relationship between the pixel coordinate system and the image coordinate system, let P l and P r Mapped to P in the image coordinate system l ′ (x l ,y) and P r ′ (x r ,y), then: According to the relationship between the image coordinate system and the camera coordinate system, the following equation is satisfied: Let x c =x cl , D = x l -x r , D represents the disparity of two points in the image coordinate system, and the following formula is obtained: Get the three-dimensional coordinates P(x c ,y c ,z c ), the coordinate is established based on the optical center of the left camera; if the center of the left camera and the right camera is used as the reference, then for x c Calibrate and set the coordinates of point P after calibration to P(x ′ c ,y c ,z c ), the calibration process is as follows: P(x ′ c ,y c ,z c ) as the three-dimensional coordinates of the target defect to the camera; S3-4, assuming that the area of ​​the defect detection frame in the camera coordinate system is S, and the height and width in the image coordinate system are H and W respectively, then the relationship is satisfied: Convert the height and width in the image coordinate system to the camera coordinate system to get the actual height and width, set as H ′ and W ′ , let the actual area be S ′ , satisfying the relationship: Substitution Get the final actual area:

3. The high-altitude autonomous painting method based on a multi-rotor drone according to claim 2 is characterized in that: In step S4, when the paint defects in the area are improved, the drone hovers at a fixed point to spray, and the optical flow method is used for auxiliary positioning to correct the position error according to the offset direction.

4. The high-altitude autonomous painting method based on a multi-rotor drone according to claim 2 is characterized in that: In step S4, the method of auxiliary positioning is specifically as follows: Assume that the light intensity at a feature in the image is I(x,y,t), where (x,y) is the coordinate of the pixel occupied by the feature, and t represents the time dimension at this moment; between adjacent frames, there is a slight increment in the coordinates of the feature, set to (dx,dy), and the change between frames is set to dt. The light intensity does not change under slight changes, then the relationship is as follows: I(x,y,t)=I(x+dx,y+dy,t+dt) (2) Taylor expansion of I(x+dx,y+dy,t+dt) at the point (x,y,t) yields the following equation (3), where ξ is an infinitesimal term: Substitute equation (3) into the right side of equation (2) and divide both sides by dt to obtain equation (4): Let u, v be the velocity vectors of the optical flow along the x-axis and y-axis respectively, and we get: Order I x , I y , I t Respectively represent the partial derivatives of the grayscale of the pixel in the image along the X, Y, and T directions: Formula (4) can be written as: I x u+I y v+I t =0 (5) Among them, I x , I y , I t From the image data, we can find the light loss (u, v), get the position of the pixel in the next frame, and know the offset direction of the drone.

Images