Image quality optimization control method for aerial photography area covered by multi-unmanned aerial vehicle system

Abstract

The invention relates to the field of unmanned aerial vehicle coverage control and image quality optimization, and discloses an image quality optimization control method for covering an aerial photography area by a multi-unmanned aerial vehicle system, which comprises the following steps: S1, establishing an image quality function based on pixel density and camera view angle, the description module is used for describing the coverage effect of the multi-unmanned-aerial-vehicle system in the convex polygon area; s2, task area division: dividing the task area of each unmanned aerial vehicle by adopting an extended Voronoi segmentation method; and S3, designing a distributed controller, and realizing local optimal deployment of the unmanned aerial vehicle based on the designed distributed controller. A new image quality function is introduced, and the coverage control of the multi-unmanned aerial vehicle system is optimized by combining the pixel density and the lens distortion, so that the defect that the characteristics of an optical camera are not fully considered in the prior art is overcome, and the optimal deployment of the unmanned aerial vehicle is realized. And the quality of the aerial image and the complete coverage of the target area are obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of coverage control and image quality optimization of unmanned aerial vehicles, and in particular to an image quality optimization control method for a multi-unmanned aerial vehicle system covering an aerial photography area. Background Art

[0002] In recent years, multi-UAV systems have been widely used in public safety, environmental monitoring, and disaster response. However, existing multi-UAV systems face many technical challenges when performing regional coverage, including mission image quality assessment and coordinated control of multiple UAVs.

[0003] Traditional image acquisition methods, such as satellite remote sensing and manned helicopters, can provide certain regional monitoring capabilities, but they are expensive and have a slow response speed. In contrast, multi-UAV systems can be flexibly deployed, quickly provide high-resolution real-time images, and effectively reduce operating costs. However, the existing technology has shortcomings in multi-UAV coverage control. Although some studies have proposed optimization methods based on pixel information loss and strategies to avoid blind spots, they ignore the impact of optical camera imaging angles on image quality, especially the impact of changes in the angle between the object and the camera optical axis on image clarity and distortion. That is, when performing aerial photography of convex polygonal areas, multi-UAV systems cannot achieve full area coverage and obtain high-quality images. Traditional methods fail to fully consider the imaging characteristics of optical cameras, especially the impact of pixel density and camera viewing angle on image quality, resulting in poor image quality and regional coverage. In the process of collaborative coverage, multi-UAV systems are prone to blind spots, overlaps, and image distortions, making it difficult to ensure global image clarity and the integrity of the target area. To this end, this application proposes an image quality optimization control method for multi-UAV systems covering aerial photography areas. Summary of the invention

[0004] The object of the present invention is to provide an image quality optimization control method for a multi-UAV system covering an aerial photography area, so as to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: an image quality optimization control method for a multi-UAV system covering an aerial photography area, comprising the following steps:

[0006] Step S1, establishing an image quality function, the image quality function constructed based on pixel density and camera viewing angle is used to describe the coverage effect of the multi-UAV system in the convex polygon area;

[0007] Step S2, division of the mission area, using the extended Voronoi segmentation method to divide the mission area of ​​each UAV;

[0008] Step S3, design of a distributed controller, based on the designed distributed controller, achieves the local optimal deployment of UAVs.

[0009] As a preferred embodiment, the step S1 of establishing the image quality function is specifically that, for the i-th drone, the image quality function of the task point q is defined as follows:

[0010]

[0011] Where a represents the area of ​​a pixel, b represents the focal length of the camera lens, m>0 is the preset parameter of the camera, i=1,2,...n., function a(bz i ) 2 Represents the environmental information contained in a unit pixel, function It is used to measure the degree of lens distortion and represents the influence of the angle between the target point q and the camera optical axis. i ,q) is inversely proportional to the image quality of the UAV at that point.

[0012] As a preferred embodiment: the division of the task area in step S2 is to deploy n drones in the convex polygon area Q, C = (c1, c2, ..., c n ) represents the horizontal coordinate set of n drones, drone p i The task area is defined by Voronoi segmentation as:

[0013]

[0014] Where i=1,2,...n. It is a convex polygon area Q i The s extension area, represents the extended Voronoi partitioning of the convex polygonal region Q.

[0015] As a preferred embodiment, the design of the distributed controller in step S3 proposes a coverage quality function to evaluate the coverage effect of the multi-UAV system on the mission area Q, which is defined as follows:

[0016]

[0017] Among them, 0<φ(q)≤1 represents the region The importance weight of each point in i ,q) is defined by formula (1.3).

[0018] As a preferred embodiment: the design process of the distributed controller in step S3 is as follows: first, the horizontal position c of the drone is calculated from equation (1.5): i The partial derivative of , we get:

[0019]

[0020] Where N i Indicates drone p i The Voronoi partition neighbor set is further expanded on the first term on the right side of equation (1.6):

[0021]

[0022] By solving the and Composed of The second term on the right side of equation (1.7) is further simplified and the result is:

[0023]

[0024] Then solve the problem and Composed of The third term on the right side of equation (1.7) is further simplified and the result is:

[0025]

[0026] Substituting equations (1.8) and (1.9) into equation (1.7), we get

[0027]

[0028] For the second term on the right side of equation (1.6), further simplification yields:

[0029]

[0030] have to:

[0031]

[0032] in,

[0033]

[0034] Calculate the height z of the drone from equation (1.5) i The partial derivative of , we get,

[0035]

[0036] in,

[0037]

[0038] Calculate the yaw angle ψ of the UAV from equation (1.5) i The partial derivative of , we get,

[0039]

[0040] As a preferred embodiment: in the design of the distributed controller in step S3, the drone p i The following distributed controller is designed:

[0041]

[0042] where α i >0, and Defined by equations (1.12), (1.13) and (1.14) respectively, i = 1, 2, ..., n.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] By introducing a new image quality function, the present invention can effectively optimize the posture and position of the drone to ensure high-quality and complete images of the shooting area. Compared with the traditional method, the image quality of the present invention is improved under the same height condition;

[0045] By using the extended Voronoi segmentation method, the present invention can divide the convex polygonal area into multiple convex sub-areas, effectively avoiding collisions between drones, improving the overall safety and efficiency of the system, and not only ensuring full coverage of the mission area, but also facilitating the synthesis and processing of images taken by the multi-drone system in the later stage;

[0046] The distributed controller designed by the present invention can realize the local optimal deployment of UAVs and adapt to changes in the mission area;

[0047] The coverage control method of the present invention reduces the reliance on manual intervention and reduces the operating cost. Compared with the traditional manual control and fixed camera network, the multi-UAV system of the present invention can be fully automated, thereby reducing the overall mission cost.

[0048] By maintaining a safe distance between drones and avoiding collisions, the present invention ensures safety in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of the method of the present invention;

[0050] Figure 2 This is a schematic diagram of the UAV model and optical camera structure;

[0051] Figure 3 is the s-extended area of ​​the convex polygon area Q Schematic diagram of

[0052] Figure 4 is the s-extended Voronoi partitioning of the convex region Q;

[0053] Figure 5 This is a bird's-eye view of the drone's final deployment status;

[0054] Figure 6 A bird's-eye view of the final deployment status of the multi-UAV system. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Example

[0057] See also Figure 1 , the image quality optimization control method for a multi-UAV system covering an aerial photography area shown in the figure includes the following steps:

[0058] Step S1, establishing an image quality function, the image quality function constructed based on pixel density and camera viewing angle is used to describe the coverage effect of the multi-UAV system in the convex polygon area;

[0059] Step S2, division of the mission area, using the extended Voronoi segmentation method to divide the mission area of ​​each UAV;

[0060] Step S3, design of a distributed controller, based on the designed distributed controller, achieves the local optimal deployment of UAVs.

[0061] In the present invention, and Represent one-dimensional and n-dimensional Euclidean space respectively. |a| represents the vector after taking the absolute value of each element of vector a, a≤b means that all elements of a correspond to elements less than b, ||a|| represents the modulus of the vector, 0 m×n represents an m×n zero matrix, I n Represents the n-th order identity matrix.

[0062] In the present invention, a drone model equipped with an optical camera is considered. The camera is fixedly installed below the drone, with its lens optical axis vertically downward and a rectangular field of view. The symbol is set represents the state of the i-th drone, where represents the three-dimensional position of the camera focus, Indicates the horizontal coordinate of the camera focus, z i represents the height of the camera focus, ψ irepresents the yaw angle of the drone, i=1,2,...,nN={1,2,...,n} represents the serial number set of all drones, and the rectangular field of view range B of the i-th drone i The definition is as follows:

[0063]

[0064] Where θ = [θ1θ2] Τ represents the half horizontal and half vertical viewing angles of the camera, R(ψ i ) is related to the yaw angle ψ i The associated rotation matrix:

[0065]

[0066] Assume l ik represents the boundary of the drone’s field of view, Represents the unit external normal vector corresponding to the boundary, where k = 1, 2, 3, 4, and n1 = [1 0] Τ ,n2=[0 1] Τ ,n3=[-1 0] Τ and n4 = [0 -1] Τ is a unit vector with a fixed direction. To ensure the safe operating distance between drones, the safety distance is set to r. Figure 2 A schematic diagram showing the drone model and its optical camera structure.

[0067] The goal of the present invention is to achieve a control method for collaboratively covering a convex polygonal area with multiple drone systems, where "coverage" refers to obtaining a high-quality, complete image of the mission area through the optical camera carried by the drone. Assume that the mission area is a known convex polygon, and the drone can obtain the boundary information of the area. In order to achieve effective coverage of the mission area, the present invention sets the following control objectives:

[0068] The multi-UAV system acquires complete, high-quality images of the convex polygonal mission area;

[0069] Drones always maintain a safe distance between each other to avoid collision.

[0070] In order to achieve the above coverage control objectives, the present invention designs a first-order controller for each drone to dynamically adjust the flight trajectory and attitude of the drone to ensure full coverage of the mission area and optimization of image quality. Its form is as follows:

[0071]

[0072] The present invention proposes a new image quality function to describe the coverage effect of a multi-UAV system in a convex polygonal area. The image quality function mainly considers two factors: one is the area contained in the unit pixel, and the other is the influence of lens distortion on the image quality. In the present invention, it is further considered that when the camera height is fixed, the farther the distance between the target point and the camera, the greater the angle between the target point and the camera optical axis, and the worse the imaging effect. Therefore, for the i-th UAV, its image quality function for the task point q is defined as follows:

[0073]

[0074] Where a represents the area of ​​a pixel, b represents the focal length of the camera lens, m>0 (m can be arbitrarily large) is the preset parameter of the camera, and i=1,2,...n. Function a(bz i ) 2 Represents the environmental information contained in a unit pixel. The smaller the value, the higher the image quality. It is used to measure the degree of lens distortion and indicates the influence of the angle between the target point q and the camera optical axis. The larger the value, the larger the angle and the worse the imaging effect. Therefore, f(p i The smaller the value of ,q), the higher the image quality of the point by the UAV.

[0075] The above image quality function can dynamically evaluate the coverage effect of the multi-UAV system in the mission area, optimize the position and attitude adjustment of the UAV, and ensure high-quality image acquisition in each sub-area.

[0076] Aiming at the requirement of multi-UAV system covering convex area, the present invention proposes an extended Voronoi segmentation method, which not only realizes the reasonable division of task area, but also effectively avoids the collision problem between intelligent agents and reserves sufficient boundary space for later image synthesis and processing.

[0077] First, the present invention defines an extension area s of the convex polygon area Q to ensure that the boundary of the task area and the imaging boundary of the camera maintain an appropriate distance, which is specifically defined as follows:

[0078] Definition 1: For a convex polygon region Q, after translating each edge along its unit external normal vector by a distance s, the intersection of all the new edges forms a new convex polygon, which is called the s-extended region of the convex polygon region Q, denoted by Figure 3 A schematic diagram showing the s-extension area of ​​the convex polygon area Q, where the yellow area is the s-extension area The black solid line represents the boundary of the original convex polygon region Q, and the black dotted line represents the extended region s. the border.

[0079] Based on the definition of the above s extension area, an extended Voronoi segmentation method is proposed, which is specifically defined as follows:

[0080] Definition 2: Deploy n drones in a convex polygon area Q, C = (c1, c2, ..., c n ) represents the horizontal coordinate set of n drones, drone p i The task area is defined by Voronoi segmentation as:

[0081]

[0082] in It is a convex polygon area Q i The s extension area, represents the extended Voronoi partitioning of the convex polygonal region Q, Figure 4 The extended Voronoi partitioning diagram of the convex polygon region Q is shown. The black solid line represents the Voronoi partitioning boundary of the Q region, and the black dotted line represents the Voronoi partitioning region Q. i and Q j The boundary of the s extension area, Indicates area Q i and Q j The shared borders of express Zhongyu The corresponding boundary, similarly, Indicates Q j Zhongyu The corresponding boundary.

[0083] The present invention ensures that the mission area of ​​each UAV is reasonably divided by extending the Voronoi segmentation method, while avoiding collisions between UAVs, thereby enhancing the security of the system and the stability of mission execution.

[0084] Before designing a controller for each drone, the present invention first gives the following lemmas about drones: Lemma 1: For any q∈l ik , point q satisfies the relation

[0085] (qc i ) T R(ψ i ) Τ n k =z i tanθ T |n k |. (1.4)

[0086] Among them l ik The drone’s field of view B i The boundary of, i = 1, 2, ..., n, k = 1, 2, 3, 4. (1.4) is about c i , zi and ψ i The partial derivatives of are,

[0087]

[0088] Lemma 2: For any Drone i and p j is a Veno neighbor, then q is about c i The partial derivatives of satisfy,

[0089]

[0090] in Representing Boundaries The unit external normal vector at position i,j=1,2,...,n,i≠j.

[0091] In order to evaluate the coverage effect of the multi-UAV system on the mission area Q, this paper proposes a coverage quality function, which is defined as follows:

[0092]

[0093] Among them, 0<φ(q)≤1 represents the region The importance weight of each point in i ,q) is defined by formula (1.3). The smaller the H value is, the better the coverage effect of the multi-UAV system on area Q is. Therefore, the goal of this invention is to find the local minimum value of this function to optimize the coverage effect.

[0094] Targeting drones i , the present invention designs a distributed controller and deploys it to the local optimal position of the mission area Q to obtain the local minimum of the coverage quality function (1.5). First, the horizontal position c of the UAV is calculated for (1.5) i The partial derivative of , we get:

[0095]

[0096] Where N i Indicates drone p i The Voronoi partition neighbor set is further expanded on the first term on the right side of equation (1.6):

[0097]

[0098] By solving the and Composed of The second term on the right side of equation (1.7) is further simplified and the result is:

[0099]

[0100] Then solve the problem and Composed of The third term on the right side of equation (1.7) is further simplified and the result is:

[0101]

[0102] Substituting equations (1.8) and (1.9) into equation (1.7), we get

[0103]

[0104] For the second term on the right side of equation (1.6), further simplification yields:

[0105]

[0106] have to:

[0107]

[0108] in,

[0109]

[0110] Calculate the height z of the drone from equation (1.5) i The partial derivative of , we get,

[0111]

[0112] in,

[0113]

[0114] Calculate the yaw angle ψ of the UAV from equation (1.5) i The partial derivative of , we get,

[0115]

[0116] For drones i The following distributed controller is designed:

[0117]

[0118] where α i >0, and Defined by equations (1.12), (1.13) and (1.14) respectively, i = 1, 2, ..., n.

[0119] Drone iDriven by the controller (1.15), the entire multi-UAV system will be deployed to the local optimal position in the convex polygonal area Q, thereby achieving complete high-quality coverage of the area, i = 1, 2, ..., n.

[0120] In order to demonstrate the effectiveness of the coverage control strategy proposed in the invention, the present invention will illustrate the effectiveness of the invention through mathematical proof and numerical simulation respectively.

[0121] Mathematical Proof

[0122] Consider the coverage quality function, let

[0123]

[0124] V ≥ 0 holds true, and the time derivative of equation (1.16) is obtained.

[0125]

[0126] If This means According to the LaSalle invariant set principle, the drone will converge to satisfy of the unchanged concentration.

[0127] Numerical simulation

[0128] We will verify the effectiveness of the proposed distributed controller through numerical simulation. The camera parameters are a = 10 -6 ,b=10 -2 ,θ1=45°,θ2=30°,m=10 17 . The flight speed is v = 3m / s, and the rotation speed is Time interval Δt=0.01s.

[0129] In order to demonstrate the effect of the image quality function proposed in the present invention, the simulation in the first part will be illustrated by a single agent covering a single convex area. Compared with the traditional image quality function that only considers pixel density, the image quality function in the present invention can place the target area in the center of the image, thereby reducing the impact of lens distortion on image quality and achieving the purpose of improving image quality.

[0130] The vertex coordinates of the convex polygon task area are [14 22] in counterclockwise order. Τ ,[14.7 17.8] Τ ,[22.7 17.6] Τ ,[25 21.8] Τ and [23 24] Τ. The parameter s in the s-extended region Q of the convex polygon region Q is 1. The initial state of the drone p is p = [10 15 20π / 6] Τ .

[0131] Figure 5 The figure shows the top view of the final deployment of the UAV under different image quality functions and controllers. The black solid line represents the boundary of the convex polygon mission area Q, the black dotted line represents the boundary of its s-extension area Q, the blue solid line represents the boundary of the UAV's field of view, and the red solid dot represents the position of the UAV. Figure 5 (a) shows the final deployment status of covering the task area Q using the traditional image quality function and controller, and Figure 5(b) shows the final deployment status of covering the task area using the image quality function (1.3) and controller (1.15).

[0132] Compared to Figure 5 (a) Figure 5 In (b), the UAV’s field of view frame is kept at a certain distance from the mission area, which is convenient for the later cropping and processing of the image. Figure 5 In (b), the task area Q is placed closer to the optical axis in the drone’s field of view, which can reduce the impact of lens distortion on image quality.

[0133] Simulation 2

[0134] In the simulation of this part, the coverage of the convex polygonal task area by the multi-UAV system is considered. The vertex coordinates of the convex polygonal task area are in counterclockwise order [27 27] Τ ,[39 23] Τ ,[53 30] Τ ,[54 38] Τ ,[48 46] Τ ,[35 53] Τ and [25 40] Τ , the corresponding extended Voronoi segmentation region parameter is s = 1, and the initial states of the drone are p1 = [34 35 20 0] Τ ,p2=[35 35 20 0] Τ ,p3=[36 35 20 0] Τ ,p4=[35 34 20 0] Τ and p5 = [36 34 20 0] Τ , Figure 6The top view of the final deployment state of the multi-UAV system under different image quality functions and controllers is shown, where the black solid line represents the Voronoi segmentation boundary of the convex polygonal task area Q, the black dashed line represents the extended Voronoi segmentation boundary of area Q, the blue solid line represents the boundary of the UAV field of view, and the red solid dot represents the position of the UAV. Figure 6 (a) shows the final deployment state of covering the mission area using traditional image quality functions and controllers. Figure 6 (b) shows the final deployment state covering the mission area using the image quality function (1.3) and the controller (1.15).

[0135] from Figure 6 As can be seen in the figure, the mission area obtained by using the image quality function (1.3) and the controller (1.15) will be placed in the center of the image, while keeping a distance between the drone's field of view border and the mission area, which is convenient for later image synthesis and processing.

[0136] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0137] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for optimizing the image quality of an aerial photography area covered by a multi-UAV system, characterized in that: The steps include: Step S1, establishing an image quality function, the image quality function constructed based on pixel density and camera viewing angle is used to describe the coverage effect of the multi-UAV system in the convex polygon area; Step S2, division of the mission area, using the extended Voronoi segmentation method to divide the mission area of ​​each UAV; Step S3, design of a distributed controller, based on the designed distributed controller, achieves the local optimal deployment of UAVs.

2. The image quality optimization control method for a multi-UAV system covering an aerial photography area according to claim 1, characterized in that: The step S1 of establishing the image quality function is specifically that, for the i-th drone, the image quality function of the task point q is defined as follows: Where a represents the area of ​​a pixel, b represents the focal length of the camera lens, m>0 is the preset parameter of the camera, i=1,2,...n., function a(bz i ) 2 Represents the environmental information contained in a unit pixel, function It is used to measure the degree of lens distortion and represents the influence of the angle between the target point q and the camera optical axis. i ,q) is inversely proportional to the image quality of the UAV at that point.

3. The image quality optimization control method for a multi-UAV system covering an aerial photography area according to claim 2, characterized in that: The task area is divided in step S2, and n drones are deployed in the convex polygon area Q, C = (c1, c2, ..., c n ) represents the horizontal coordinate set of n drones, drone p i The task area is defined by Voronoi segmentation as: Where i=1,2,...n. It is a convex polygon area Q i The s extension area, represents the extended Voronoi partitioning of the convex polygonal region Q.

4. The image quality optimization control method for a multi-UAV system covering an aerial photography area according to claim 3, characterized in that: The design of the distributed controller in step S3 proposes a coverage quality function to evaluate the coverage effect of the multi-UAV system on the mission area Q, which is defined as follows: Among them, 0<φ(q)≤1 represents the region The importance weight of each point in i ,q) is defined by formula (1.3).

5. The method for optimizing image quality of an aerial photography area covered by a multi-UAV system according to claim 4, characterized in that: The design process of the distributed controller in step S3 is as follows: first, the horizontal position c of the UAV is calculated from equation (1.5): i The partial derivative of , we get: Where N i Indicates drone p i The Voronoi partition neighbor set is further expanded on the first term on the right side of equation (1.6): By solving the and Composed of The second term on the right side of equation (1.7) is further simplified and the result is: Then solve the problem and Composed of The third term on the right side of equation (1.7) is further simplified and the result is: Substituting equations (1.8) and (1.9) into equation (1.7), we get For the second term on the right side of equation (1.6), further simplification yields: have to: in, Calculate the height z of the drone from equation (1.5) i The partial derivative of , we get, in, Calculate the yaw angle ψ of the UAV from equation (1.5) i The partial derivative of , we get, 6. The method for optimizing image quality of an aerial photography area covered by a multi-UAV system according to claim 5, characterized in that: In the design of the distributed controller in step S3, the UAV p i The following distributed controller is designed: where α i >0, and Defined by equations (1.12), (1.13) and (1.14) respectively, i = 1, 2, ..., n.

Images