An image quality optimization control method for covering an aerial photography area by a multi-UAV system

Abstract

The present invention relates to the field of coverage control and image quality optimization of unmanned aerial vehicles (UAVs), and discloses an image quality optimization control method for a multi-UAV system to cover an aerial photography area, which includes the following steps. Step S1, establish an image quality function, which is an image quality function constructed based on pixel density and camera viewing angle and is used to describe the coverage effect of the multi-UAV system in a convex polygon area. Step S2, divide the task area, and use an extended Voronoi segmentation method to divide the task area of each UAV. Step S3, design a distributed controller, and based on the designed distributed controller, achieve the local optimal deployment of the UAVs. The present invention introduces a new image quality function, combines pixel density and lens distortion, optimizes the coverage control of the multi-UAV system, solves the defect in the prior art that the characteristics of optical cameras are not fully considered, and significantly improves the quality of aerial photography images and the complete coverage of the target area.

Description

Technical Field

[0001] The present invention relates to the field of coverage control and image quality optimization of unmanned aerial vehicles, and specifically to an image quality optimization control method for covering an aerial photography area by a multi-unmanned aerial vehicle system. Background Art

[0002] Multi-unmanned aerial vehicle systems have been widely used in fields such as public safety, environmental monitoring, and disaster response in recent years. However, when existing multi-unmanned aerial vehicle systems perform area coverage, they face many technical challenges, including task image quality assessment and cooperative control of multi-unmanned aerial vehicles.

[0003] Traditional image acquisition methods, such as satellite remote sensing and manned helicopters, although they can provide certain area monitoring capabilities, are costly and have a slow response speed. In contrast, multi-unmanned aerial vehicle systems can be flexibly deployed to quickly provide high-resolution real-time images and effectively reduce operating costs. However, there are deficiencies in the multi-unmanned aerial vehicle coverage control of the existing technology. Although some studies have proposed optimization methods based on pixel information loss and strategies to avoid blind spots, the influence of the imaging angle of the optical camera on the image quality has been ignored, especially the influence of the change in the angle between the object and the camera optical axis on the image clarity and distortion. That is, when the multi-unmanned aerial vehicle system performs aerial photography of a convex polygon area, it is impossible to achieve full-area coverage and obtain high-quality images. The traditional methods have not fully considered the imaging characteristics of the optical camera, especially the influence of pixel density and camera viewing angle on the image quality, resulting in poor image quality and area coverage effect. During the cooperative coverage process of the multi-unmanned aerial vehicle system, problems such as blind spots, overlaps, and image distortion are likely to occur, and it is difficult to ensure the global image clarity and the integrity of the target area. Therefore, this application proposes an image quality optimization control method for covering an aerial photography area by a multi-unmanned aerial vehicle system. Summary of the Invention

[0004] The purpose of the present invention is to provide an image quality optimization control method for covering an aerial photography area by a multi-unmanned aerial vehicle system to solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: An image quality optimization control method for covering an aerial photography area by a multi-unmanned aerial vehicle system, including the following steps:

[0006] Step S1, establish an image quality function, an image quality function constructed based on pixel density and camera viewing angle, which is used to describe the coverage effect of the multi-unmanned aerial vehicle system in a convex polygon area;

[0007] Step S2, division of the task area, using an extended Voronoi division method to divide the task area of each unmanned aerial vehicle;

[0008] Step S3: Design of the distributed controller. Based on the designed distributed controller, the local optimal deployment of the UAVs is achieved.

[0009] Preferably, the establishment of the image quality function in step S1 is specifically as follows. For the th UAV, its image quality function for the task point is defined as follows:

[0010] (1.3)

[0011] where represents the area of the region contained in one pixel, represents the focal length of the camera lens, is a preset parameter of the camera, . The function represents the environmental information contained in a unit pixel. The function is used to measure the degree of lens distortion and represents the influence of the angle between the target point and the optical axis of the camera. The value of is inversely proportional to the image quality of the UAV for this point.

[0012] Preferably, for the division of the task area in step S2, UAVs are deployed in the convex polygon area . represents the set of horizontal coordinates of UAVs. The task area of the UAV

[0013] ,

[0014] where is the expanded area of the convex polygon area , and represents the expanded Voronoi partition of the convex polygon area .

[0015] Preferably, the design of the distributed controller in step S3 proposes a coverage quality function to evaluate the coverage effect of the multi-UAV system on the task area , which is defined as follows:

[0016] , (1.5)

[0017] where represents the importance weight of each point in the area , and is defined by equation (1.3).

[0018] Preferably, the design process of the distributed controller in step S3 is specifically as follows. First, take the partial derivative of equation (1.5) with respect to the horizontal position of the UAV to obtain: :

[0019] (1.6)

[0020] where represents the Voronoi partition neighbor set of the UAV . Further expand the first term on the right side of equation (1.6):

[0021] (1.7)

[0022] By solving the and terms that make up the , further simplify the second term on the right side of equation (1.7) and obtain the result:

[0023] (1.8)

[0024] Then solve the and terms that make up the , further simplify the third term on the right side of equation (1.7) and obtain the result:

[0025] (1.9)

[0026] Substitute equations (1.8) and (1.9) into equation (1.7) to get

[0027] (1.10)

[0028] For the second term on the right side of equation (1.6), further simplify to get:

[0029] (1.11)

[0030] Get:

[0031] (1.12)

[0032] where

[0033]

[0034] Take the partial derivative of equation (1.5) with respect to the altitude of the UAV to get

[0035] (1.13)

[0036] where

[0037]

[0038] Take the partial derivative of formula (1.5) with respect to the yaw angle of the UAV , and we get

[0039] (1.14).

[0040] As an optimization: in the design of the distributed controller in step S3, the following distributed controller is designed for the UAV :

[0041] (1.15)

[0042] where , , and are defined by formula (1.12), formula (1.13) and formula (1.14) respectively, .

[0043] The beneficial effects of the present invention compared with the prior art are as follows:

[0044] By introducing a new image quality function, the present invention can effectively optimize the attitude and position of the UAV, ensure high-quality complete images in the shooting area, and the image quality of the present invention is improved under the condition of the same height compared with the traditional method;

[0045] Using the extended Voronoi partition method, the present invention can divide the convex polygon area into multiple convex sub-areas, effectively avoid collisions between UAVs, improve the overall safety and efficiency of the system, not only ensure complete coverage of the mission area, but also facilitate the synthesis and processing of images taken by the multi-UAV system in the later stage;

[0046] The distributed controller designed by the present invention can achieve local optimal deployment of the UAV and adapt to changes in the mission area;

[0047] The coverage control method of the present invention reduces the dependence on manual intervention and reduces the operation cost. Compared with the traditional manual control and fixed camera network, the multi-UAV system using the present invention can be fully automated, thus reducing the overall mission cost;

[0048] By maintaining a safe distance between UAVs and avoiding collisions, the present invention ensures safety in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is the flow chart of the method of the present invention;

[0050] Figure 2 Schematic diagram of the UAV model and the optical camera structure;

[0051] Figure 3 is a convex polygon area of expansion area schematic diagram;

[0052] Figure 4 is a convex area of extended Voronoi partition;

[0053] Figure 5 Top view of the final deployment status of the UAV;

[0054] Figure 6 Top view of the final deployment status of the multi-UAV system. Detailed implementation manners

[0055] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0056] Embodiment

[0057] Please refer to Figure 1 , an image quality optimization control method for a multi-UAV system to cover an aerial photography area in the figure, including the following steps:

[0058] Step S1, establish an image quality function, an image quality function constructed based on pixel density and camera view angle, used to describe the coverage effect of the multi-UAV system in the convex polygon area;

[0059] Step S2, division of the task area, using the extended Voronoi partition method to divide the task area of each UAV;

[0060] Step S3, design of the distributed controller, based on the designed distributed controller, achieve the local optimal deployment of the UAVs.

[0061] In the present invention, and respectively represent one-dimensional and dimensional Euclidean spaces. For , represents the vector obtained by taking the absolute value of each element of the vector , represents each element of which is correspondingly less than The element of represents the modulus of the vector. represents a zero matrix of represents the identity matrix of order

[0062] In the present invention, consider a UAV model equipped with an optical camera, which is fixedly installed below the UAV, with its lens optical axis vertically downward and having a rectangular field of view. Set the symbol to represent the state of the th UAV, where represents the three-dimensional position of the camera focus. represents the horizontal coordinate of the camera focus. represents the height of the camera focus. represents the yaw angle of the UAV. represents the set of numbers of all UAVs. The rectangular field of view range of the th UAV is defined as follows:

[0063] (1.1)

[0064] where represents the half horizontal view angle and half vertical view angle of the camera. is the rotation matrix related to the yaw angle :

[0065]

[0066] Let represent the boundary of the UAV's field of view. represents the corresponding unit outer normal vector on this boundary, where , set , , and are unit vectors in fixed directions. To ensure the safe operating distance between UAVs, set the safe distance as . Figure 2 shows a schematic diagram of the UAV model and its optical camera structure.

[0067] The objective of the present invention is to achieve a control method for multi-UAV system cooperative coverage of a convex polygon area, where "coverage" means obtaining high-quality and complete images of the mission area through the optical cameras carried by UAVs. Assume that the mission area is a known convex polygon, and the UAVs can obtain the boundary information of this area. To achieve effective coverage of the mission area, the present invention sets the following control objectives:

[0068] The multi - UAV system acquires complete and high - quality images of the convex polygon mission area;

[0069] The UAVs always maintain a safe distance from each other to avoid collisions.

[0070] To achieve the above - mentioned coverage control objective, the present invention designs a first - order controller for each UAV, which is used to dynamically adjust the flight trajectory and attitude of the UAV to ensure full coverage of the mission area and optimization of image quality. Its form is as follows:

[0071] (1.2)

[0072] The present invention proposes a new image quality function to describe the coverage effect of the multi - UAV system in the convex polygon area. This image quality function mainly considers two factors: one is the area of the region contained in a unit pixel, and the other is the influence of lens distortion on image quality. In the present invention, further considering that when the camera height is fixed, the farther the target point is from the camera, the larger the angle between it and the camera optical axis, and the worse the imaging effect. Therefore, for the th UAV, its image quality function for the mission point is defined as follows:

[0073] (1.3)

[0074] Wherein, represents the area of the region contained in a pixel, represents the focal length of the camera lens, ( can be arbitrarily large) is a preset parameter of the camera, . The function represents the environmental information contained in a unit pixel. The smaller the value, the higher the image quality. The function is used to measure the degree of lens distortion, representing the influence of the angle between the target point and the camera optical axis. The larger the value, the larger the angle and the worse the imaging effect. Therefore, the smaller the value, the higher the image quality of the UAV for this point.

[0075] The above - mentioned image quality function can dynamically evaluate the coverage effect of the multi - UAV system in the mission area, optimize the position and attitude adjustment of the UAVs, and ensure high - quality image acquisition for each sub - area.

[0076] In response to the requirement of the multi - UAV system to cover the convex area, the present invention proposes an extended Voronoi partitioning method. This method not only realizes the reasonable division of the mission area, but also effectively avoids the collision problem between agents, and reserves enough boundary space for later image synthesis and processing.

[0077] First, the present invention defines a convex polygon region 's expansion region to ensure an appropriate distance between the boundary of the task region and the imaging boundary of the camera, and the specific definition is as follows:

[0078] Definition 1: For the convex polygon region , each edge is translated by a distance in the direction of its unit outer normal vector, and the intersection points of all the new lines where the new edges are located form a new convex polygon, which is called the expansion region of the convex polygon region , denoted as . Figure 3 shows a schematic diagram of the expansion region of the convex polygon region , where the yellow region is the expansion region , the black solid line represents the boundary of the original convex polygon region , and the black dashed line represents the boundary of the expansion region .

[0079] Based on the above definition of the expansion region, an extended Voronoi partitioning method is proposed, and the specific definition is as follows:

[0080] Definition 2: Deploy UAVs in the convex polygon region . Let represent the set of horizontal coordinates of the UAVs. The task region of the UAV is defined by Voronoi partitioning as:

[0081] ,

[0082] where is the expansion region of the convex polygon region , represents the extended Voronoi partitioning of the convex polygon region . Figure 4 shows the extended Voronoi partitioning diagram of the convex polygon region . The black solid line represents the Voronoi partitioning boundary of the region, and the black dashed line represents the boundaries of the Voronoi partitioning regions and 's expansion regions. represents the shared boundary of the regions and . represents The boundary corresponding to in the same way, represents the boundary corresponding to in.

[0083] The present invention ensures a reasonable division of the task area for each drone, avoids collisions between drones, and enhances the safety of the system and the stability of task execution by expanding the Voronoi segmentation method.

[0084] Before designing a controller for each drone, the present invention first gives the following related lemmas about drones. Lemma 1: For any , the point satisfies the relational expression

[0085] (1.4)

[0086] where is the boundary of the drone's field of view , , Equation (1.4) with respect to , and has partial derivatives of,

[0087]

[0088] Lemma 2: For any , drones and are Voronoi neighbors, then with respect to has a partial derivative that satisfies,

[0089] ,

[0090] where represents the unit outer normal vector at the boundary , , .

[0091] To evaluate the coverage effect of the multi - drone system on the task area , the present invention proposes a coverage quality function, which is defined as follows:

[0092] , (1.5)

[0093] where, represents the importance weight of each point in the area , is defined by Equation (1.3), The smaller the value, the better the coverage effect of the multi-UAV system on the area . Therefore, the goal of the present invention is to find the local minimum of this function to optimize the coverage effect.

[0094] For the UAV , the present invention designs a distributed controller and deploys it to the local optimal position of the mission area to find the local minimum of the coverage quality function (1.5). First, take the partial derivative of formula (1.5) with respect to the horizontal position of the UAV to obtain:

[0095] (1.6)

[0096] where represents the Voronoi partitioning neighbor set of the UAV . Further expand the first term on the right side of formula (1.6):

[0097] (1.7)

[0098] By solving the and constituted terms, further simplify the second term on the right side of formula (1.7) and obtain the result:

[0099] (1.8)

[0100] Then solve the and constituted terms, further simplify the third term on the right side of formula (1.7) and obtain the result:

[0101] (1.9)

[0102] Substitute formulas (1.8) and (1.9) into formula (1.7) to get

[0103] (1.10)

[0104] For the second term on the right side of formula (1.6), further simplify to get:

[0105] (1.11)

[0106] Get:

[0107] (1.12)

[0108] where

[0109]

[0110] Take the partial derivative of equation (1.5) with respect to the UAV altitude , and we get

[0111] (1.13)

[0112] where

[0113]

[0114] Take the partial derivative of equation (1.5) with respect to the UAV yaw angle , and we get

[0115] (1.14)

[0116] For the UAV the following distributed controller is designed:

[0117] (1.15)

[0118] where , , and are defined by equations (1.12), (1.13) and (1.14) respectively, .

[0119] The UAV driven by the controller (1.15), the entire multi-UAV system will be deployed to the local optimal position in the convex polygon area , so as to achieve complete high-quality coverage of this area, .

[0120] To demonstrate the effectiveness of the coverage control strategy proposed by the invention, the present invention will illustrate the effectiveness of the present invention through mathematical proof and numerical simulation respectively.

[0121] Mathematical proof

[0122] Consider the coverage quality function, let

[0123] (1.16)

[0124] hold, take the derivative of equation (1.16) with respect to time, and we get

[0125] .

[0126] If , this means , according to the LaSalle invariant set principle, the UAV will converge to the invariant set that satisfies .

[0127] Numerical simulation

[0128] We will verify the effectiveness of the proposed distributed controller through numerical simulation methods. The parameters of the camera are , , , , . The flight speed is , and the rotation speed is rad / s, and the time interval is s.

[0129] To demonstrate the effect of the image quality function proposed in the present invention, the first part of the simulation will be illustrated by a single agent covering a single convex region. Compared with the traditional image quality function that only considers pixel density, the image quality function in the present invention can place the target area at the center of the image, thereby reducing the influence of lens distortion on the graphic quality and achieving the purpose of improving the image quality.

[0130] The vertex coordinates of the convex polygon task area are, in counterclockwise order, , , , and . The convex polygon area 's expansion area 's parameter . The UAV 's initial state is .

[0131] Figure 5 Figure [X] shows the top view of the final deployment of the UAV under the action of different image quality functions and controllers, where the black solid line represents the boundary of the convex polygon task area , the black dashed line represents the boundary of its expansion area , the blue solid line represents the boundary of the UAV's field of view, and the red solid dot represents the position of the UAV. Figure 5 (a) represents the final deployment status of covering the task area using the traditional image quality function and controller, and Figure 5(b) represents the final deployment status of covering the task area using the image quality function (1.3) and controller (1.15).

[0132] Compared with Figure 5 (a), Figure 5(b) There is a certain distance between the field-of-view border of the drone and the task area, which is convenient for post-image cropping and processing. At the same time, Figure 5 the task area in (b) is placed closer to the optical axis in the drone's field of view, which can reduce the impact of lens distortion on the image quality.

[0133] Simulation 2

[0134] In the simulation of this part, the coverage of a convex polygon task area by a multi-drone system is considered. The vertex coordinates of the convex polygon task area are in counterclockwise order as , , , , , and , and the corresponding parameters of the extended Voronoi partition area are . The initial states of the drones are respectively , , , and . Figure 6 shows a top view of the final deployment state of the multi-drone system under different image quality functions and controllers. The black solid line represents the Voronoi partition boundary of the convex polygon task area , the black dashed line represents the extended Voronoi partition boundary of the area , the blue solid line represents the boundary of the drone's field of view, and the red solid dots represent the positions of the drones. Figure 6 (a) shows the final deployment state of covering the task area using the traditional image quality function and controller. Figure 6 (b) shows the final deployment state of covering the task area using the image quality function (1.3) and the controller (1.15).

[0135] From Figure 6 , it can be seen that the task area obtained using the image quality function (1.3) and the controller (1.15) will be placed at the center of the image, and at the same time, the distance between the drone's field-of-view border and the task area is reserved, which is convenient for later image synthesis and processing.

[0136] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0137] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An image quality optimization control method for covering an aerial photography area by a multi-UAV system, characterized in that, It includes the following steps: Step S1, establish an image quality function, which is an image quality function constructed based on pixel density and camera view angle and is used to describe the coverage effect of the multi-UAV system within the convex polygon area; Step S2, divide the task area. Use the extended Voronoi partitioning method to divide the task area of each UAV; Step S3, design a distributed controller. Based on the designed distributed controller, achieve the local optimal deployment of the UAVs; The specific establishment of the image quality function in step S1 is as follows. For the th drone, its image quality function for the task point is defined as follows: （1.3） Among them, represents the area of the region included in one pixel, represents the focal length of the camera lens, is a preset parameter of the camera, . The function represents the environmental information included in a unit pixel. The function is used to measure the degree of lens distortion and represents the influence of the angle between the target point and the optical axis of the camera. The value of is inversely proportional to the image quality of the UAV for this point; The division of the task area in step S2 is carried out in the convex polygon area Deploy drones, Indicates the set of horizontal coordinates of the drones. The task area of a drone is defined by Voronoi partitioning as: , Among them is a convex polygon region of the extended region indicating the extended Voronoi partition of the convex polygon region ; The design of the distributed controller in step S1 proposes a coverage quality function to evaluate the coverage effect of the multi-UAV system on the mission area which is defined as follows: ,（1.5） wherein, represents the importance weight of each point in the region defined by Equation (1.3); The design process of the distributed controller in step S3 is specifically as follows. First, take the partial derivative of formula (1.5) with respect to the horizontal position of the UAV to obtain: as follows: （1.6） Among them represents the Voronoi partition neighbor set of the UAV , expand the first term on the right side of equation (1.6): （1.7） By solving the and constituting the terms, the second term on the right side of equation (1.7) is simplified and the result is obtained: （1.8） Among them, is a convex polygon region 's extended region; represents the extended Voronoi partition of the convex polygon region ; represents the shared boundary of the regions and ; represents the boundary in corresponding to represents the boundary in corresponding to is the field of view of the UAV, is the field of view of the UAV 's boundary, represents the unit outer normal vector at the boundary Next, solve the and constituting the terms, simplify the third term on the right side of equation (1.7) and obtain the result: （1.9） Substitute equations (1.8) and (1.9) into equation (1.7) to obtain （1.10） For the second term on the right side of equation (1.6), simplify to get: （1.11） Obtain: （1.12） Where, is the rotation matrix related to the yaw angle , where s is the translation distance of each edge along the direction of its unit outer normal vector; Taking the partial derivative of equation (1.5) with respect to the UAV altitude , we get （1.13） Where, represent the half horizontal and half vertical viewing angles of the camera; Take the partial derivative of formula (1.5) with respect to the yaw angle of the UAV , and we get （1.14）。 2. The image quality optimization control method for covering an aerial photography area by a multi-UAV system according to claim 1, wherein: In the design of the distributed controller in step S3, for the drone The following distributed controller is designed: （1.15） where , , and are defined by equations (1.12), (1.13), and (1.14) respectively, .

Images