A UAV image optimization system and method based on image matching

Abstract

The present invention relates to the field of unmanned aerial vehicle image technology, and in particular to a system and method for optimizing unmanned aerial vehicle images based on image matching. The optimization method comprises the following steps: respectively acquiring a plurality of images continuously taken by an unmanned aerial vehicle at each shooting position; judging and identifying shooting positions with shooting abnormalities and marking the features through feature deviation comparison; constructing a three-dimensional image corresponding to the image according to images taken at adjacent shooting positions of the unmanned aerial vehicle, analyzing the shooting angles of the unmanned aerial vehicle at two adjacent shooting positions, and obtaining the shooting angles of the unmanned aerial vehicle at the abnormal shooting position; analyzing the adjustment route and shooting angle change trend of the unmanned aerial vehicle, and obtaining the selection priority of a plurality of adjustment routes of the unmanned aerial vehicle in an abnormal flight section; acquiring in real time the images taken by the unmanned aerial vehicle at each shooting position during flight, analyzing to obtain a plurality of real-time abnormal flight sections, and adjusting the flight route of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle images, and in particular to a system and method for optimizing unmanned aerial vehicle images based on image matching. Background Art

[0002] Binocular stereo image matching algorithm is a computer vision algorithm used to process binocular stereo images. Its main purpose is to find the correspondence between the corresponding pixels in the left and right images by comparing their features, so as to calculate the depth information.

[0003] The images taken by the drone during flight are reconstructed in three dimensions by calculating the disparity information of each pixel through a binocular stereo image matching algorithm. However, this requires accurate calculation of the disparity information of the corresponding pixels in the image. During the flight of the drone, the captured images may be affected by various environmental factors, resulting in blurred images or abnormal exposure of some areas, making it impossible to obtain a complete and effective image. If the abnormal image cannot be repaired, the only option is to re-shoot it, but this cannot be completed quickly. Repeated shooting will consume a lot of energy and is also very inefficient. Summary of the invention

[0004] The purpose of the present invention is to provide a drone image optimization system and method based on image matching to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a drone image optimization method based on image matching, the optimization method comprising the following steps:

[0006] Step S100: obtaining a plurality of images continuously captured at each shooting position during the process of the drone collecting images of the target object along the flight route; performing feature deviation comparison on the plurality of images at each shooting position, identifying and marking the shooting positions with shooting abnormalities, and obtaining a plurality of abnormal shooting positions; wherein the number of images captured by the drone at one shooting position is greater than or equal to 2;

[0007] Step S200: acquiring the coordinate position of a shooting position adjacent to the abnormal shooting position and the image captured at the coordinate position, and determining the coordinate position of the target object in the image in the actual position; analyzing the shooting angles of the two shooting positions adjacent to the abnormal shooting position, and obtaining the shooting angle of the drone at the abnormal shooting position;

[0008] Step S300: obtaining a number of abnormal flight sections according to the distribution of abnormal shooting positions, analyzing the change process of the coordinate position of the drone in each abnormal flight section; analyzing the adjustment route and shooting angle change trend of the drone in each abnormal flight section, and obtaining the selection priority of the drone in the abnormal flight section;

[0009] Step S400: Acquire images taken by the UAV at various shooting positions during the flight in real time, analyze and obtain several real-time abnormal flight sections, and adjust the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process.

[0010] Furthermore, step S100 includes the following steps:

[0011] Step S101: Set any two images captured by the drone at the i-th shooting position during a flight as and , extract the image and images All feature points in the image include corner points, edges, SIFT feature points, SURF feature points, etc., to obtain the image The feature set and images The feature set ; Get feature set Any feature point in the image The coordinate position in the image is extracted The pixel point corresponding to the coordinate position described in the figure is used to determine whether the pixel point is a feature set. a feature point in ; if the pixel point is not a feature point, the i-th shooting position is marked as abnormal, and the abnormal shooting position set of the UAV in a flight process is obtained;

[0012] Step S102: Obtain a set of abnormal shooting positions during each flight of the drone, select an abnormal shooting position during any flight, set the abnormal shooting position in the flight process as the i-th shooting position, and count the number of abnormal shooting positions in all unit cycles. , the abnormal proportion of the i-th shooting position is ; Set an abnormal percentage threshold ,like , then the i-th shooting position is marked with features; by analyzing the abnormal proportion of each abnormal shooting position, abnormal shooting positions in some special cases can be screened out. Such abnormal shooting positions can be directly replaced according to historical shooting images and images taken at adjacent shooting positions, which can improve work efficiency to a certain extent.

[0013] Further, step S200 includes the following steps:

[0014] Step S201: set the i-th shooting position in a flight process of the drone to have a feature mark, obtain images captured at adjacent shooting positions, calculate the parallax of corresponding pixels in the two images using a binocular stereo image matching algorithm, and obtain the horizontal offset of the corresponding pixels; determine the horizontal offset of each pixel, obtain the depth information of each pixel, and determine the coordinate position of the target object corresponding to the captured image;

[0015] Step S202: Obtain the coordinate position of the drone at the i+1th shooting position, with the flight direction of the drone as one side and the line between the coordinate position of the drone and the coordinate position of the three-dimensional image as the other side, and the generated angle is the shooting angle of the drone at the i+1th shooting position;

[0016] Step S203: Obtain the shooting angles of the drone at the two shooting positions adjacent to the i-th shooting position, and set the shooting angle of the i-1-th shooting position to , the shooting angle of the i+1th shooting position is , calculate the average of the two shooting angles to get the shooting angle of the i-th shooting position .

[0017] Further, step S300 includes the following steps:

[0018] Step S301: Acquire all abnormal shooting positions of the UAV during a flight process. If two adjacent abnormal shooting positions are two adjacent shooting positions during the flight process, divide the two adjacent abnormal shooting positions into an abnormal flight section, and generate several abnormal flight sections during a flight process of the UAV;

[0019] Step S302: setting the first abnormal shooting position in an abnormal flight section as the starting shooting position and the last abnormal shooting position as the ending shooting position; extracting the coordinate position of the UAV at the starting shooting position and the coordinate position at the ending shooting position, establishing a sphere with the two coordinate positions as the two end points of a diameter, and using the surface of the sphere as the flight adjustment area of ​​the UAV in the abnormal flight section;

[0020] Step S303: Obtain images taken by the drone at various positions in the flight adjustment area, and set any two images at coordinate positions (x, y) as and , extract images respectively and images The number of common feature points in two feature sets is , according to the formula:

[0021]

[0022] in, For images The feature set of For images The feature set of is the number of feature points after the intersection of two feature sets; the image is calculated and images Similarity ; Set a similarity threshold ,like , then set the coordinate position (x, y) as the target position, and obtain the shooting angle of the coordinate position (x, y);

[0023] Step S304: Acquire all target positions in the flight adjustment area, use the coordinate position of the starting shooting position as the starting point of the UAV flight route, use the coordinate position of the last shooting position as the end point of the UAV flight route, connect all target positions, and obtain several adjustment routes of the UAV in the flight adjustment area;

[0024] Step S305: Set the shooting angle of each coordinate position in the jth adjustment route, where the shooting angle of the kth coordinate position is , the angle difference between the kth coordinate position and the k+1th coordinate position is calculated as ; Get all the angle differences of the jth adjustment route, calculate the average value, and get the average adjustment angle of the jth adjustment route: ;

[0025] Step S306: Obtain the flight distance of the jth adjusted route: , according to the formula:

[0026]

[0027] Among them, a and b are constant coefficients; the selection priority of the jth adjustment route is calculated; the selection priorities are sorted from large to small, and the adjustment route with the largest selection priority is selected as the planned flight route of the UAV in an abnormal flight section; the adjustment route of the UAV will be affected by the route distance and the change of shooting angle. The route distance will affect the efficiency of the UAV, and the large change of shooting angle will increase the difficulty of operation, which is not conducive to the shooting of the UAV. Therefore, it is necessary to select the most effective flight route from multiple adjustment routes, which can effectively improve work efficiency.

[0028] Furthermore, step S400 includes the following steps:

[0029] Step S401: acquiring two images taken at the i-th shooting position during the flight of the drone in real time, extracting feature sets of the two images, comparing the two feature sets to determine whether the i-th shooting position is an abnormal shooting position, and dividing the abnormal shooting positions during the flight of the drone into a number of real-time abnormal flight sections;

[0030] Step S402: Obtain several abnormal flight segments of the UAV in the historical flight process. If a real-time abnormal flight segment is the same as an abnormal flight segment, the time length of the same flight segment is t. According to the formula: ,in, is the duration of abnormal flight sections in historical flight processes, and the proportion of the same abnormal duration of real-time abnormal flight sections is calculated. ;

[0031] Step S403: Setting a percentage threshold ,like , the flight route of the abnormal flight section shall be adjusted according to the flight route adjustment priority; if , then obtain the flight adjustment area of ​​the real-time abnormal flight section, generate the flight route adjustment priority in the real-time abnormal flight section, and adjust the flight route of the UAV.

[0032] In order to better implement the above method, a UAV image optimization system based on image matching is also proposed. The optimization system includes an abnormal shooting position analysis module, a UAV data analysis module, a route priority selection module, and a route adjustment module;

[0033] The abnormal shooting position analysis module is used to obtain a number of images continuously captured at each shooting position when the drone collects images of the target object along the flight route; by comparing the feature deviations of the images at each shooting position, the shooting positions with shooting abnormalities are judged, identified and marked with features, and a number of abnormal shooting positions are obtained;

[0034] The drone data analysis module is used to obtain the coordinate position of the shooting position adjacent to the abnormal shooting position and the image captured at the coordinate position, and determine the coordinate position of the target object in the image in the actual position; analyze the shooting angles of the two shooting positions adjacent to the abnormal shooting position, and obtain the shooting angle of the drone at the abnormal shooting position;

[0035] The route priority selection module is used to obtain several abnormal flight sections according to the distribution of abnormal shooting positions, and analyze the change process of the coordinate position of the drone in each abnormal flight section; according to the images taken by the drone at each coordinate position, obtain several target positions and shooting angles of the target positions, analyze the adjustment route of the drone and the change trend of the shooting angle, and obtain the selection priority of several adjustment routes of the drone in the abnormal flight section;

[0036] The route adjustment module is used to obtain in real time the images taken by the UAV at various shooting positions during the flight, analyze and obtain several real-time abnormal flight sections, and adjust the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process.

[0037] Further, the abnormal shooting position analysis module includes a feature set generation unit and an abnormal marking unit;

[0038] The feature set generation unit is used to obtain a number of images taken simultaneously at various shooting positions during each flight of the drone in history, and compare the feature points between the feature sets of the several images; the abnormal marking unit is used to compare the feature points between the feature sets of the several images, analyze the differences between the feature points, and determine whether to mark the abnormal shooting positions with features.

[0039] Further, the drone data analysis module includes a target position determination unit and a shooting angle analysis unit;

[0040] The target position determination unit is used to obtain the coordinate position of the shooting position adjacent to the abnormal shooting position and the image captured at the coordinate position to determine the coordinate position of the target object in the image in actual orientation; the shooting angle analysis unit is used to analyze the shooting angles of the two shooting positions adjacent to the abnormal shooting position to obtain the shooting angle of the drone at the abnormal shooting position.

[0041] Further, the route priority selection module includes a position change analysis unit and a priority setting unit;

[0042] The position change analysis unit is used to obtain several abnormal flight sections according to the distribution of abnormal shooting positions, and analyze the change process of the coordinate position of the drone in each abnormal flight section; the priority setting unit is used to obtain several target positions and shooting angles of the target positions according to the images taken by the drone at each coordinate position, analyze the adjustment route of the drone and the change trend of the shooting angle, and obtain the selection priority of several adjustment routes of the drone in the abnormal flight section.

[0043] Further, the route adjustment module includes a real-time data analysis unit and an adjustment scheme determination unit;

[0044] The real-time data analysis unit is used to obtain in real time the images taken by the UAV at various shooting positions during the flight, and analyze them to obtain a number of real-time abnormal flight sections; the adjustment plan determination unit is used to adjust the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process.

[0045] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: (1) The present invention reduces the problem of image anomalies caused by various environmental factors in the images taken by the drone by adjusting the flight route of the drone, effectively optimizes the drone images, and ensures the quality of the images taken by the drone; (2) The present invention provides multiple routes for selection by adjusting the flight routes of consecutive abnormal flight sections, thereby ensuring image quality while selecting more effective and energy-saving routes, thereby improving the shooting efficiency of the drone; and the images of some abnormal shooting positions can be compensated by adjacent images, thereby improving work efficiency; (3) The present invention sets the priority of flight route adjustment, which can help staff effectively select appropriate routes for flight shooting, effectively shorten working time, and avoid the situation where repeated shooting still has abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0047] Figure 1 It is a schematic diagram of the steps of a UAV image optimization method based on image matching;

[0048] Figure 2 It is a structural diagram of a UAV image optimization system based on image matching. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] See also Figure 1 to Figure 2 The present invention provides a technical solution: a drone image optimization method based on image matching, the optimization method comprising the following steps:

[0051] Step S100: obtaining a plurality of images continuously captured at each shooting position during the process of the drone collecting images of the target object along the flight route; performing feature deviation comparison on a plurality of images at each shooting position, determining and identifying shooting positions with shooting abnormalities and marking the features, and obtaining a plurality of abnormal shooting positions;

[0052] Wherein, step S100 includes the following steps:

[0053] Step S101: Set any two images captured by the drone at the i-th shooting position during a flight as and , extract the image and images All the feature points in the image are obtained The feature set and images The feature set ; Get feature set Any feature point in the image The coordinate position in the image is extracted The pixel point corresponding to the coordinate position described in the figure is used to determine whether the pixel point is a feature set. a feature point in ; if the pixel point is not a feature point, the i-th shooting position is marked as abnormal, and the abnormal shooting position set of the UAV in a flight process is obtained;

[0054] Step S102: Obtain a set of abnormal shooting positions during each flight of the drone, select an abnormal shooting position during any flight, set the abnormal shooting position in the flight process as the i-th shooting position, and count the number of abnormal shooting positions in all unit cycles. , the abnormal proportion of the i-th shooting position is ; Set an abnormal percentage threshold ,like , then the i-th shooting position is marked with features.

[0055] Step S200: obtaining the coordinate position of a shooting position adjacent to the abnormal shooting position and the image captured at the coordinate position, and determining the coordinate position of the target object in the image in the actual position; analyzing the shooting angles of the two shooting positions adjacent to the abnormal shooting position, and obtaining the shooting angle of the drone at the abnormal shooting position;

[0056] Wherein, step S200 includes the following steps:

[0057] Step S201: set the i-th shooting position in a flight process of the drone to have a feature mark, obtain images captured at adjacent shooting positions, calculate the parallax of corresponding pixels in the two images using a binocular stereo image matching algorithm, and obtain the horizontal offset of the corresponding pixels; determine the horizontal offset of each pixel, obtain the depth information of each pixel, and determine the coordinate position of the target object corresponding to the captured image;

[0058] Step S202: Obtain the coordinate position of the drone at the i+1th shooting position, with the flight direction of the drone as one side and the line between the coordinate position of the drone and the coordinate position of the three-dimensional image as the other side, and the generated angle is the shooting angle of the drone at the i+1th shooting position;

[0059] Step S203: Obtain the shooting angles of the drone at the two shooting positions adjacent to the i-th shooting position, and set the shooting angle of the i-1-th shooting position to , the shooting angle of the i+1th shooting position is , calculate the average of the two shooting angles to get the shooting angle of the i-th shooting position .

[0060] Step S300: obtaining a number of abnormal flight sections according to the distribution of abnormal shooting positions, analyzing the change process of the coordinate position of the drone in each abnormal flight section; analyzing the adjustment route and shooting angle change trend of the drone in each abnormal flight section, and obtaining the selection priority of the drone in the abnormal flight section;

[0061] Wherein, step S300 includes the following steps:

[0062] Step S301: Acquire all abnormal shooting positions of the UAV during a flight process. If two adjacent abnormal shooting positions are two adjacent shooting positions during the flight process, divide the two adjacent abnormal shooting positions into an abnormal flight section, and generate several abnormal flight sections during a flight process of the UAV;

[0063] Step S302: setting the first abnormal shooting position in an abnormal flight section as the starting shooting position and the last abnormal shooting position as the ending shooting position; extracting the coordinate position of the UAV at the starting shooting position and the coordinate position at the ending shooting position, establishing a sphere with the two coordinate positions as the two end points of a diameter, and using the surface of the sphere as the flight adjustment area of ​​the UAV in the abnormal flight section;

[0064] Step S303: Obtain images taken by the drone at various positions in the flight adjustment area, and set any two images at coordinate positions (x, y) as and , extract images respectively and images The number of common feature points in two feature sets is , according to the formula:

[0065]

[0066] in, For images The feature set of For images The feature set of is the number of feature points after the intersection of two feature sets; the image is calculated and images Similarity ; Set a similarity threshold ,like , then set the coordinate position (x, y) as the target position, and obtain the shooting angle of the coordinate position (x, y);

[0067] Step S304: Acquire all target positions in the flight adjustment area, use the coordinate position of the starting shooting position as the starting point of the UAV flight route, use the coordinate position of the last shooting position as the end point of the UAV flight route, connect all target positions, and obtain several adjustment routes of the UAV in the flight adjustment area;

[0068] Step S305: Set the shooting angle of each coordinate position in the jth adjustment route, where the shooting angle of the kth coordinate position is , the angle difference between the kth coordinate position and the k+1th coordinate position is calculated as ; Get all the angle differences of the jth adjustment route, calculate the average value, and get the average adjustment angle of the jth adjustment route: ;

[0069] Step S306: Obtain the flight distance of the jth adjusted route: , according to the formula:

[0070]

[0071] Among them, a and b are constant coefficients; the selection priority of the jth adjustment route is calculated; the selection priorities are sorted from large to small, and the adjustment route with the largest selection priority is selected as the planned flight route of the UAV in an abnormal flight section.

[0072] Step S400: acquiring images taken by the UAV at various shooting positions during the flight in real time, analyzing and obtaining a number of real-time abnormal flight sections, and adjusting the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process;

[0073] Wherein, step S400 includes the following steps:

[0074] Step S401: acquiring two images taken at the i-th shooting position during the flight of the drone in real time, extracting feature sets of the two images, comparing the two feature sets to determine whether the i-th shooting position is an abnormal shooting position, and dividing the abnormal shooting positions during the flight of the drone into a number of real-time abnormal flight sections;

[0075] Step S402: Obtain several abnormal flight segments of the UAV in the historical flight process. If a real-time abnormal flight segment is the same as an abnormal flight segment, the time length of the same flight segment is t. According to the formula: ,in, is the duration of abnormal flight sections in historical flight processes, and the proportion of the same abnormal duration of real-time abnormal flight sections is calculated. ;

[0076] Step S403: Setting a percentage threshold ,like , the flight route of the abnormal flight section shall be adjusted according to the flight route adjustment priority; if , then obtain the flight adjustment area of ​​the real-time abnormal flight section, generate the flight route adjustment priority in the real-time abnormal flight section, and adjust the flight route of the UAV.

[0077] A drone image optimization system based on image matching, the optimization system includes an abnormal shooting position analysis module, a drone data analysis module, a route priority selection module, and a route adjustment module;

[0078] The abnormal shooting position analysis module is used to obtain a number of images continuously captured at each shooting position when the drone collects images of the target object along the flight route; by comparing the feature deviations of the images at each shooting position, the shooting positions with shooting abnormalities are judged, identified and marked with features, and a number of abnormal shooting positions are obtained;

[0079] The drone data analysis module is used to obtain the coordinate position of the shooting position adjacent to the abnormal shooting position and the image captured at the coordinate position, and determine the coordinate position of the target object in the image in the actual position; analyze the shooting angles of the two shooting positions adjacent to the abnormal shooting position, and obtain the shooting angle of the drone at the abnormal shooting position;

[0080] The route priority selection module is used to obtain several abnormal flight sections according to the distribution of abnormal shooting positions, and analyze the change process of the coordinate position of the drone in each abnormal flight section; according to the images taken by the drone at each coordinate position, obtain several target positions and shooting angles of the target positions, analyze the adjustment route of the drone and the change trend of the shooting angle, and obtain the selection priority of several adjustment routes of the drone in the abnormal flight section;

[0081] The route adjustment module is used to obtain in real time the images taken by the UAV at various shooting positions during the flight, analyze and obtain several real-time abnormal flight sections, and adjust the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process.

[0082] Among them, the abnormal shooting position analysis module includes a feature set generation unit and an abnormal marking unit;

[0083] The feature set generation unit is used to obtain a number of images taken simultaneously at various shooting positions during each flight of the drone in history, and compare the feature points between the feature sets of the several images; the abnormal marking unit is used to compare the feature points between the feature sets of the several images, analyze the differences between the feature points, and determine whether to mark the abnormal shooting positions with features.

[0084] Among them, the drone data analysis module includes a target position determination unit and a shooting angle analysis unit;

[0085] The target position determination unit is used to obtain the coordinate position of the shooting position adjacent to the abnormal shooting position and the image captured at the coordinate position to determine the coordinate position of the target object in the image in actual orientation; the shooting angle analysis unit is used to analyze the shooting angles of the two shooting positions adjacent to the abnormal shooting position to obtain the shooting angle of the drone at the abnormal shooting position.

[0086] Wherein, the route priority selection module includes a position change analysis unit and a priority setting unit;

[0087] The position change analysis unit is used to obtain several abnormal flight sections according to the distribution of abnormal shooting positions, and analyze the change process of the coordinate position of the drone in each abnormal flight section; the priority setting unit is used to obtain several target positions and shooting angles of the target positions according to the images taken by the drone at each coordinate position, analyze the adjustment route of the drone and the change trend of the shooting angle, and obtain the selection priority of several adjustment routes of the drone in the abnormal flight section.

[0088] Among them, the route adjustment module includes a real-time data analysis unit and an adjustment plan determination unit;

[0089] The real-time data analysis unit is used to obtain in real time the images taken by the UAV at various shooting positions during the flight, and analyze them to obtain a number of real-time abnormal flight sections; the adjustment plan determination unit is used to adjust the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process.

[0090] It is assumed that the drone has conducted three flight shootings in history, and the flight route and shooting content are the same; during the first flight, two images obtained from the second shooting position are obtained, feature points in the two images are extracted, and two corresponding feature sets are generated; a feature point in one of the images is selected, and the coordinate position in the other image is obtained according to the coordinate position of the feature point, but in the coordinate position in the other image, the pixel point is not a feature point, so the first shooting position needs to be marked as abnormal; all shooting positions with abnormal marks are obtained, among which the second, third, and fourth shooting positions are all abnormal shooting positions;

[0091] It is counted that the number of times the three shooting positions are abnormal shooting positions is 3 times during the three flights, so the abnormal proportion of the three shooting positions is 100%, and the abnormal proportion threshold is set to 90%. Therefore, the three shooting positions are marked with features.

[0092] The coordinate position of the drone at the fifth shooting position is obtained. The coordinate position of the three-dimensional image and the flight direction of the drone are constructed based on the captured image. The shooting angle of the drone is 90 degrees, so the shooting angle of the third shooting position is also 90 degrees.

[0093] According to the coordinate position of the second shooting position and the coordinate position of the fourth shooting position as the two end points of a diameter, a sphere is established, and the surface of the sphere is used as the flight adjustment area of ​​the UAV in the abnormal flight section; a coordinate position in the flight adjustment area is selected, and the image captured according to this coordinate position is obtained, and the number of identical feature points in the two feature sets is 50, one of which has 60 feature points and the other has 60 feature points. The number of feature points after the intersection of the two feature sets is 70, and the similarity of the two images is 71.4%. The similarity threshold is set to 70%, because 71.4%>70%, so this coordinate position is set as the target position, and the shooting angle is obtained as 90%; the shooting angle difference between two adjacent coordinate positions is 0;

[0094] The distance to obtain the flight route is 1m, and a=1, b=1 are set, so the priority of the flight route is 1, and the selection priority of all flight routes is obtained; sort them from large to small according to the selection priority, and select the adjustment route with the largest selection priority as the planned flight route of the UAV in an abnormal flight section.

[0095] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0096] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A drone image optimization method based on image matching, characterized in that: The optimization method comprises the following steps: Step S100: obtaining a plurality of images continuously captured at each shooting position during the process of the drone collecting images of the target object along the flight route; performing feature deviation comparison on a plurality of images at each shooting position, determining and identifying shooting positions with shooting abnormalities and marking the features, and obtaining a plurality of abnormal shooting positions; Step S200: obtaining the coordinate position of a shooting position adjacent to the abnormal shooting position and the image captured at the coordinate position, and determining the coordinate position of the target object in the image in the actual position; analyzing the shooting angles of the two shooting positions adjacent to the abnormal shooting position, and obtaining the shooting angle of the drone at the abnormal shooting position; Step S300: obtaining a number of abnormal flight sections according to the distribution of abnormal shooting positions, analyzing the change process of the coordinate position of the drone in each abnormal flight section; analyzing the adjustment route and shooting angle change trend of the drone in each abnormal flight section, and obtaining the selection priority of the drone in the abnormal flight section; Step S400: acquiring images taken by the UAV at various shooting positions during the flight in real time, analyzing and obtaining a number of real-time abnormal flight sections, and adjusting the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process; The step S300 includes the following steps: Step S301: Acquire all abnormal shooting positions of the UAV during a flight process. If two adjacent abnormal shooting positions are two adjacent shooting positions during the flight process, divide the two adjacent abnormal shooting positions into an abnormal flight section, and generate several abnormal flight sections during a flight process of the UAV; Step S302: setting the first abnormal shooting position in an abnormal flight section as the starting shooting position and the last abnormal shooting position as the ending shooting position; extracting the coordinate position of the UAV at the starting shooting position and the coordinate position at the ending shooting position, establishing a sphere with the two coordinate positions as the two end points of a diameter, and using the surface of the sphere as the flight adjustment area of ​​the UAV in the abnormal flight section; Step S303: Obtain images taken by the drone at various positions in the flight adjustment area, and set any two images at coordinate positions (x, y) as and , extract images respectively and images The number of common feature points in two feature sets is , according to the formula: ； in, For images The feature set of For images The feature set of is the number of feature points after the intersection of two feature sets; the image is calculated and images Similarity ; Set a similarity threshold ,like , then set the coordinate position (x, y) as the target position, and obtain the shooting angle of the coordinate position (x, y); Step S304: Acquire all target positions in the flight adjustment area, use the coordinate position of the starting shooting position as the starting point of the UAV flight route, use the coordinate position of the last shooting position as the end point of the UAV flight route, connect all target positions, and obtain several adjustment routes of the UAV in the flight adjustment area; Step S305: Set the shooting angle of each coordinate position in the jth adjustment route, where the shooting angle of the kth coordinate position is , the angle difference between the kth coordinate position and the k+1th coordinate position is calculated as ; Get all the angle differences of the jth adjustment route, calculate the average value, and get the average adjustment angle of the jth adjustment route: ; Step S306: Obtain the flight distance of the jth adjusted route: , according to the formula: ； Among them, a and b are constant coefficients; the selection priority of the jth adjustment route is calculated; the selection priorities are sorted from large to small, and the adjustment route with the largest selection priority is selected as the planned flight route of the UAV in an abnormal flight section.

2. The method for optimizing unmanned aerial vehicle images based on image matching according to claim 1, characterized in that: The step S100 includes the following steps: Step S101: Set any two images captured by the drone at the i-th shooting position during a flight as and , extract the image and images All the feature points in the image are obtained The feature set and images The feature set ; Get feature set Any feature point in the image The coordinate position in the image is extracted The pixel point corresponding to the coordinate position described in the figure is used to determine whether the pixel point is a feature set. a feature point in ; if the pixel point is not a feature point, the i-th shooting position is marked as abnormal, and the abnormal shooting position set of the UAV in a flight process is obtained; Step S102: Obtain a set of abnormal shooting positions during each flight of the drone, select an abnormal shooting position during any flight, set the abnormal shooting position in the flight process as the i-th shooting position, and count the number of abnormal shooting positions in all unit cycles. , the abnormal proportion of the i-th shooting position is ; Set an abnormal percentage threshold ,like , then the i-th shooting position is marked with features.

3. The method for optimizing unmanned aerial vehicle images based on image matching according to claim 2, characterized in that: The step S200 includes the following steps: Step S201: set the i-th shooting position in a flight process of the drone to have a feature mark, obtain images captured at adjacent shooting positions, calculate the parallax of corresponding pixels in the two images using a binocular stereo image matching algorithm, and obtain the horizontal offset of the corresponding pixels; determine the horizontal offset of each pixel, obtain the depth information of each pixel, and determine the coordinate position of the target object corresponding to the captured image; Step S202: Obtain the coordinate position of the drone at the i+1th shooting position, with the flight direction of the drone as one side and the line between the coordinate position of the drone and the coordinate position of the three-dimensional image as the other side, and the generated angle is the shooting angle of the drone at the i+1th shooting position; Step S203: Obtain the shooting angles of the drone at the two shooting positions adjacent to the i-th shooting position, and set the shooting angle of the i-1-th shooting position to , the shooting angle of the i+1th shooting position is , calculate the average of the two shooting angles to get the shooting angle of the i-th shooting position .

4. The method for optimizing unmanned aerial vehicle images based on image matching according to claim 3 is characterized in that: The step S400 includes the following steps: Step S401: acquiring two images taken at the i-th shooting position during the flight of the drone in real time, extracting feature sets of the two images, comparing the two feature sets to determine whether the i-th shooting position is an abnormal shooting position, and dividing the abnormal shooting positions during the flight of the drone into a number of real-time abnormal flight sections; Step S402: Obtain several abnormal flight segments of the UAV in the historical flight process. If a real-time abnormal flight segment is the same as an abnormal flight segment, the time length of the same flight segment is t. According to the formula: ,in, is the duration of abnormal flight sections in historical flight processes, and the proportion of the same abnormal duration of real-time abnormal flight sections is calculated. ; Step S403: Setting a percentage threshold ,like , the flight route of the abnormal flight section shall be adjusted according to the flight route adjustment priority; if , then obtain the flight adjustment area of ​​the real-time abnormal flight section, generate the flight route adjustment priority in the real-time abnormal flight section, and adjust the flight route of the UAV.

5. A drone image optimization system based on a drone image optimization method based on image matching according to any one of claims 1 to 4, characterized in that: The optimization system includes an abnormal shooting location analysis module, a drone data analysis module, a route priority selection module, and a route adjustment module; The abnormal shooting position analysis module is used to obtain a number of images continuously captured at each shooting position during the process of the drone collecting images of the target object along the flight route; by comparing the feature deviations of the images at each shooting position, the shooting positions with shooting abnormalities are judged, identified and feature marked, and a number of abnormal shooting positions are obtained; The drone data analysis module is used to obtain the coordinate position of the shooting position adjacent to the abnormal shooting position and the image captured at the coordinate position, and determine the coordinate position of the target object in the image in the actual position; analyze the shooting angles of the two shooting positions adjacent to the abnormal shooting position, and obtain the shooting angle of the drone at the abnormal shooting position; The route priority selection module is used to obtain several abnormal flight sections according to the distribution of abnormal shooting positions, and analyze the change process of the coordinate position of the drone in each abnormal flight section; obtain several target positions and shooting angles of the target positions according to the images shot by the drone at each coordinate position, analyze the adjustment route of the drone and the change trend of the shooting angle, and obtain the selection priority of several adjustment routes of the drone in the abnormal flight section; The route adjustment module is used to obtain in real time the images taken by the UAV at various shooting positions during the flight, analyze and obtain a number of real-time abnormal flight sections, and adjust the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process.

6. The UAV image optimization system according to claim 5, characterized in that: The abnormal shooting position analysis module includes a feature set generation unit and an abnormal marking unit; The feature set generation unit is used to obtain a number of images taken simultaneously at various shooting positions during each flight history of the drone, and compare the feature points between the feature sets of the several images; the abnormal marking unit is used to compare the feature points between the feature sets of the several images, analyze the differences between the feature points, and determine whether to perform feature marking on the abnormal shooting positions.

7. The UAV image optimization system according to claim 5, characterized in that: The drone data analysis module includes a target position determination unit and a shooting angle analysis unit; The target position determination unit is used to obtain the coordinate position of a shooting position adjacent to the abnormal shooting position, and the image captured at the coordinate position, to determine the coordinate position of the target object in the image in actual orientation; the shooting angle analysis unit is used to analyze the shooting angles of two shooting positions adjacent to the abnormal shooting position, to obtain the shooting angle of the drone at the abnormal shooting position.

8. The UAV image optimization system according to claim 5, characterized in that: The route priority selection module includes a position change analysis unit and a priority setting unit; The position change analysis unit is used to obtain several abnormal flight sections according to the distribution of abnormal shooting positions, and analyze the change process of the coordinate position of the drone in each abnormal flight section; the priority setting unit is used to obtain several target positions and shooting angles of the target positions according to the images taken by the drone at each coordinate position, analyze the adjustment route of the drone and the change trend of the shooting angle, and obtain the selection priority of several adjustment routes of the drone in the abnormal flight section.

9. The UAV image optimization system according to claim 5, characterized in that: The route adjustment module includes a real-time data analysis unit and an adjustment scheme determination unit; The real-time data analysis unit is used to obtain in real time the images taken by the UAV at various shooting positions during the flight, and analyze them to obtain a number of real-time abnormal flight sections; the adjustment scheme determination unit is used to adjust the flight route of the UAV according to the distribution of shooting positions between the real-time abnormal flight sections and the abnormal flight sections in the historical flight process.

Images