Secondary task planning method based on remote sensing shooting

Abstract

The invention discloses a secondary task planning method based on remote sensing shooting, and relates to the technical field of remote sensing shooting task planning, and the method comprises the steps: setting a target region corresponding to unqualified image data in an image data set of a completed shooting task in a first remote sensing shooting task as a target region of secondary task planning, setting shooting time of secondary task planning; adjusting shooting parameters of secondary task planning; performing a secondary task according to the shooting time and the shooting parameters; and carrying out image splicing on image data shot by the secondary task and qualified image data in the image data set. According to the method, secondary task planning is performed on the target area corresponding to the unqualified image data, so that the quality of the remote sensing shooting image data is improved; the compatibility between different image data is improved by performing image splicing on the image data shot by the secondary task and the qualified image data in the image data set.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing photography task planning, and in particular to a secondary task planning method based on remote sensing photography. Background Art

[0002] Remote sensing photography is a comprehensive technology for long-distance detection and identification of various surface objects and phenomena. It is widely used in agriculture, forestry, geology, oceanography, meteorology, environmental monitoring and other fields. Among them, remote sensing photography mission planning is an important part of remote sensing photography technology.

[0003] In the prior art, Chinese patent CN113220027A discloses a concave polygonal area drone track planning based on remote sensing tasks. This method performs track planning for multiple drones based on specific remote sensing task mapping requirements, provides specific aerial photography point information, and ensures the shortest total flight distance while ensuring full coverage of the observation area. Chinese patent CN112498745A discloses a low-orbit optical satellite stereo imaging autonomous planning method, which uses an orbit recursive algorithm to obtain the satellite's position and velocity information within 120 seconds before and after the satellite passes the imaging target position, and obtains the position point of the ±25° pitch angle required for the satellite to perform stereoscopic imaging of the target position through solution, thereby finally completing the satellite stereo imaging attitude planning.

[0004] In the actual process of remote sensing shooting tasks, due to environmental factors or shooting angles, there are often some image data of unqualified quality in a remote sensing shooting task. However, the above-mentioned existing technologies have not considered studying or solving this problem, and the quality of remote sensing image data is low; in addition, different remote sensing image data may have different shooting parameters, which makes it difficult to integrate and subsequently process different remote sensing image data. The above-mentioned existing technologies have not studied or solved this problem, and the compatibility between different image data is low. Summary of the invention

[0005] The embodiment of the present application solves the problems of low quality of remote sensing image data and low compatibility between different image data in the existing remote sensing photography technology by providing a secondary task planning method based on remote sensing photography.

[0006] In a first aspect, an embodiment of the present application provides a secondary task planning method based on remote sensing photography, comprising the following steps:

[0007] An image data set of a completed shooting task in a first remote sensing shooting task is obtained, and it is determined whether the quality of each image data in the image data set is qualified.

[0008] The target area corresponding to the unqualified image data in the image data set is set as the target area of ​​the secondary task planning, the priority of the secondary task planning of the target area is set, and the shooting time of the secondary task planning of the corresponding target area is set according to the priority.

[0009] The shooting parameters of the secondary task planning of the corresponding target area are adjusted according to the unqualified image data.

[0010] Perform a secondary task according to the shooting time and the shooting parameters.

[0011] Image stitching is performed on the image data captured by the secondary task and the qualified image data in the image data set.

[0012] In a possible implementation manner, determining whether the quality of each image data in the image data set is qualified includes:

[0013] A quality assessment is performed on each image data in the image data set. When the quality assessment result of an image data is higher than a preset quality assessment threshold, the image data is judged to be qualified image data. When the quality assessment result of an image data is lower than or equal to the preset quality assessment threshold, the image data is judged to be unqualified image data.

[0014] The quality assessment is image clarity assessment and / or image integrity assessment.

[0015] In a possible implementation manner, setting the priority of the secondary task planning of the target area, and setting the shooting time of the secondary task planning of the corresponding target area according to the priority includes:

[0016] The priority of the secondary task planning of the target area is set to be higher than the priority of the unfinished shooting tasks in the first remote sensing shooting task, and the priority of the secondary task planning of each target area is sorted according to the position of each target area.

[0017] The shooting time of the secondary task planning of the corresponding target area is set according to the priority of the secondary task planning of each target area after sorting.

[0018] In a possible implementation, the shooting parameters include: cloud cover parameters.

[0019] The step of adjusting the shooting parameters of the secondary task planning of the corresponding target area according to the unqualified image data includes:

[0020] Perform cloud cover analysis on the unqualified image data to obtain cloud cover data and occlusion degree data corresponding to the unqualified image data.

[0021] The cloud cover parameters of the secondary task planning of the corresponding target area are adjusted according to the cloud cover data and the occlusion degree data.

[0022] Obtain historical cloud cover data of the target area corresponding to the cloud cover parameter, and predict the cloud cover change trend of the corresponding target area in the future based on the historical cloud cover data.

[0023] The shooting time of the secondary task planning of the corresponding target area is adjusted according to the cloud amount parameter and the cloud amount change trend.

[0024] In a possible implementation, the shooting parameters include: a side swing angle parameter.

[0025] The step of adjusting the shooting parameters of the secondary task planning of the corresponding target area according to the unqualified image data includes:

[0026] Obtain the side swing angle data, shooting height data, pixel height data and ground resolution data of the unqualified image data, and obtain the building height data of the unqualified image data according to the side swing angle data, the shooting height data, the pixel height data and the ground resolution data.

[0027] The side swing angle parameter of the secondary mission planning of the corresponding target area is adjusted according to the building height data.

[0028] In a possible implementation manner, the performing image stitching on the image data captured by the secondary task and the qualified image data in the image data set includes:

[0029] Feature extraction is performed on the image data taken by the secondary task and the qualified image data in the image data set to obtain a feature set, wherein the feature set includes several feature subsets, and a feature subset includes: all features of an image data taken by the secondary task or all features of a qualified image data.

[0030] Feature matching is performed on adjacent feature subsets corresponding to every two adjacent regions to obtain a matching feature set, wherein the matching feature set includes a plurality of matching subsets, and a matching subset includes: the same features in two adjacent feature subsets.

[0031] The coordinates of the image data captured by the secondary task and the qualified image data in the image data set are reconstructed according to the matching feature set.

[0032] Image fusion is performed on the image data taken by the secondary task after coordinate reconstruction and the qualified image data after coordinate reconstruction.

[0033] In a possible implementation, the feature extraction uses a feature detection algorithm, which includes: a scale-invariant feature transformation algorithm, an accelerated robust feature algorithm, and a corner detection algorithm.

[0034] In a possible implementation, the feature matching adopts a similarity measurement algorithm, and the similarity measurement algorithm includes: Euclidean distance method, cosine similarity method, Manhattan distance method and Pearson correlation coefficient method.

[0035] In a possible implementation, reconstructing the coordinates of the image data captured by the secondary task and the qualified image data in the image data set according to the matching feature set includes:

[0036] Calculate the geographic coordinate transformation formula parameters between the adjacent area image data corresponding to each matching subset.

[0037] The coordinates of the image data captured by the secondary task and the qualified image data in the image data set are reconstructed according to the geographic coordinate transformation formula parameters.

[0038] In a possible implementation manner, the image fusion adopts image fusion technology, and the image fusion technology includes: Poisson fusion, Laplace pyramid fusion, multi-band fusion and feathering fusion.

[0039] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0040] The embodiment of the present application improves the quality of remote sensing image data by performing secondary task planning on the target area corresponding to unqualified image data; and improves the compatibility between different image data by performing image stitching on the image data taken by the secondary task and the qualified image data in the image data set.

[0041] By setting the priority of the secondary task planning of the target area and setting the shooting time of the secondary task planning of the corresponding target area according to the priority, the rationality of the secondary task planning is improved.

[0042] By adjusting the cloud cover parameters of the secondary mission planning and adjusting the shooting time of the secondary mission planning according to the cloud cover parameters and cloud cover change trends, the influence of cloud cover on the image quality of the secondary mission shooting is avoided and the rationality of the secondary mission planning is improved.

[0043] It is proposed to obtain building height data of unqualified image data according to the swing angle data, shooting height data, pixel height data and ground resolution data, and adjust the swing angle parameters of secondary task planning according to the building height data, thereby improving the quality of image data taken by the secondary task.

[0044] By combining feature extraction, feature matching, coordinate reconstruction and image fusion, the image data taken by the secondary task and the qualified image data in the image data set are stitched, which reduces the difficulty of processing between different image data and improves the accuracy of image stitching. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A schematic diagram of a flow chart of a secondary task planning method based on remote sensing photography provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a process for performing image stitching on image data captured for a secondary task and qualified image data in an image data set provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] like Figure 1 As shown, the embodiment of the present application provides a secondary task planning method based on remote sensing photography, comprising the following steps:

[0050] S110, obtaining an image data set of completed shooting tasks in the first remote sensing shooting task, and determining whether the quality of each image data in the image data set is qualified.

[0051] S120, setting the target area corresponding to the unqualified image data in the image data set as the target area for secondary task planning, setting the priority of the secondary task planning of the target area, and setting the shooting time of the secondary task planning of the corresponding target area according to the priority.

[0052] S130, adjusting the shooting parameters of the secondary task planning of the corresponding target area according to the unqualified image data.

[0053] S140: Perform a secondary task according to the shooting time and the shooting parameters.

[0054] S150, performing image stitching on the image data captured by the secondary task and the qualified image data in the image data set.

[0055] Specifically, in this embodiment, a remote sensing drone equipped with a shooting device is used to perform the first remote sensing shooting task and the secondary task. In other possible embodiments, a remote sensing satellite or other remote sensing platforms can also be used to perform the first remote sensing shooting task and the secondary task.

[0056] Exemplarily, in S110, determining whether the quality of each image data in the image data set is qualified includes:

[0057] A quality assessment is performed on each image data in the image data set. When the quality assessment result of an image data is higher than a preset quality assessment threshold, the image data is judged to be qualified image data. When the quality assessment result of an image data is lower than or equal to the preset quality assessment threshold, the image data is judged to be unqualified image data.

[0058] The quality assessment is image clarity assessment and / or image integrity assessment.

[0059] Specifically, in this embodiment, the grayscale variance algorithm in the image clarity assessment is used to perform quality assessment on each image data in the image data set. The specific process is as follows: convert the image data to be evaluated into a grayscale image; traverse each pixel in the grayscale image to obtain its grayscale value; calculate the grayscale average value of all pixels, calculate the sum of the square differences between the grayscale value of each pixel and the grayscale average value, divide the sum of the square differences by the total number of pixels in the grayscale image, and obtain the grayscale variance value; compare the grayscale variance value of each image data to be evaluated with a preset grayscale variance threshold value, when the grayscale variance value of an image data is higher than the preset grayscale variance threshold value, the image data is judged to be qualified image data, and when the grayscale variance value of an image data is lower than or equal to the preset grayscale variance threshold value, the image data is judged to be unqualified image data. The specific grayscale variance threshold value can be the grayscale variance value of a pre-prepared reference clear image, or the average value of the grayscale variance values ​​of multiple reference clear images, and can also be adjusted and set according to actual conditions.

[0060] In other possible embodiments, quality assessment may also adopt one of the energy gradient method, Tenegrad gradient method, and Laplace gradient method in image clarity assessment, or one of the structural similarity index, multi-scale structural similarity index, mean square error, and perceptual loss in image integrity assessment.

[0061] Exemplarily, in S120, setting the priority of the secondary task planning of the target area, and setting the shooting time of the secondary task planning of the corresponding target area according to the priority includes:

[0062] The priority of the secondary task planning of the target area is set to be higher than the priority of the unfinished shooting tasks in the first remote sensing shooting task, and the priority of the secondary task planning of each target area is sorted according to the position of each target area.

[0063] The shooting time of the secondary task planning of the corresponding target area is set according to the priority of the secondary task planning of each target area after sorting.

[0064] Specifically, in this embodiment, taking the example of 100 consecutive shooting areas set in the first remote sensing shooting task, the priorities of the 100 shooting areas are set from short to long according to the route lengths of the 100 shooting areas relative to the starting point in the planned route of the first remote sensing shooting task, and the smaller the priority number, the higher the priority; when setting the priority of the secondary task planning of the target area, the priority of the completed shooting area in the first remote sensing shooting task corresponding to the target area is assigned to the priority of the secondary task planning of the target area. For example, when the remote sensing platform is in a shooting area with a priority of 50 to perform a shooting task, the shooting areas with priorities of 1 to 49 are all completed shooting areas, and the shooting areas with priorities of 50 to 100 are all unfinished shooting areas, that is, shooting areas that have not completed the shooting task. When the priority of the completed shooting area in the first remote sensing shooting task corresponding to the target area is assigned to the priority of the secondary task planning of the target area, the priority of the secondary task planning of the target area must be within 1 to 49, that is, higher than the priority of 50 to 100 of the unfinished shooting tasks in the first remote sensing shooting task.

[0065] Specifically, in this embodiment, when the priority of the secondary task planning of each target area is sorted according to the position of each target area, the closer the distance between each target area and the current area, the higher the priority of the sorting. For example, when there are 5 target areas in the secondary task planning, and the priorities of the target areas before sorting are 22, 26, 31, 33, and 37 respectively, the priorities are sorted from near to far according to the distance between each target area and the current area, that is, the closer the distance, the higher the priority of the sorting.

[0066] In a possible embodiment, an optimization algorithm may be used to perform path optimization on all target areas of the secondary task planning, and the path length of each target area relative to the starting point of the optimization path is obtained when the shortest path is obtained. The target area with a shorter path length has a higher sorting priority.

[0067] In a possible embodiment, no sorting is required, which is equivalent to sorting according to the route length relative to the starting point in the planned route of the target area corresponding to the first remote sensing shooting task.

[0068] The shooting time of the secondary task planning of the corresponding target area is set according to the priority of the secondary task planning of each target area after sorting, that is, the higher the priority of the target area, the earlier the shooting time of the secondary task planning.

[0069] Exemplarily, in S130, the shooting parameters include: cloud cover parameters.

[0070] The step of adjusting the shooting parameters of the secondary task planning of the corresponding target area according to the unqualified image data includes:

[0071] Perform cloud cover analysis on the unqualified image data to obtain cloud cover data and occlusion degree data corresponding to the unqualified image data.

[0072] The cloud cover parameters of the secondary task planning of the corresponding target area are adjusted according to the cloud cover data and the occlusion degree data.

[0073] Obtain historical cloud cover data of the target area corresponding to the cloud cover parameter, and predict the cloud cover change trend of the corresponding target area in the future based on the historical cloud cover data.

[0074] The shooting time of the secondary task planning of the corresponding target area is adjusted according to the cloud amount parameter and the cloud amount change trend.

[0075] Specifically, in this embodiment, the cloud amount data corresponding to the unqualified image data is obtained by a cloud detection algorithm based on texture features. The process of the cloud detection algorithm based on texture features is as follows: texture features are extracted for an unqualified image data (in this embodiment, texture features are extracted based on texture variance, and cloud areas usually have lower texture variance); principal component analysis (PCA) is used to reduce the dimension and optimize the extracted texture features; a support vector machine classifier is used to classify and label the pixels of the unqualified image data according to the optimized texture features to obtain a cloud pixel area; the number of pixels in the cloud pixel area is divided by the total number of pixels of the unqualified image data to obtain the cloud amount data T of the unqualified image data, 0≤T≤1. In other possible embodiments, the cloud amount data corresponding to the unqualified image data can also be obtained by a cloud detection algorithm based on a threshold or based on machine learning.

[0076] In this embodiment, the occlusion degree data corresponding to the unqualified image data is obtained by using an occlusion degree assessment algorithm based on shadow detection. The process of the occlusion degree assessment algorithm based on shadow detection is as follows: convert the color space of an unqualified image data into HSV (hue, saturation, brightness); perform shadow feature extraction on the converted unqualified image data (in this embodiment, shadow feature extraction is performed according to the low brightness standard), and perform morphological processing (such as corrosion, expansion, etc.) to obtain a shadow feature area; divide the number of pixels in the shadow feature area by the total number of pixels of the unqualified image data to obtain the occlusion degree data Q of the unqualified image data, 0≤Q≤1. In other possible embodiments, the occlusion degree data corresponding to the unqualified image data can also be obtained by using an occlusion degree assessment algorithm based on machine learning.

[0077] In this embodiment, the cloud cover parameter of the secondary mission planning of the corresponding target area is adjusted according to the following formula:

[0078]

[0079] Wherein, T′ represents the cloud cover parameter of the secondary task planning, κ represents the cloud cover adjustment coefficient, 0≤κ≤0.2, and in this embodiment, κ is 0.1.

[0080] In this embodiment, the historical cloud data of the target area corresponding to the cloud amount parameter is obtained through the meteorological station (in this embodiment, the cloud amount numerical data within 72 hours, and the interval between adjacent data is 12 minutes), and the cloud amount change trend of the corresponding target area within the next 6 hours is predicted according to the historical cloud amount data through the trend model. In this embodiment, the trend model adopts the Winter addition model, which is as follows:

[0081]

[0082] Where t represents the current period, x t represents the historical cloud cover data of period t, i.e., the latest cloud cover data, l t represents the estimated level in period t, b t represents the estimated trend of period t, s t represents the estimated season of the tth period, α represents the horizontal smoothing parameter, which is between 0 and 1, β represents the trend smoothing parameter, which is between 0 and 1, γ represents the seasonal smoothing parameter, which is between 0 and 1, m represents the cycle length, which is 24 (i.e., 24 hours a day) in this embodiment, h represents the number of forecast lead periods, which is set to 0.5 hours for each period in this embodiment, h is 1, 2, 3, ... 12, the smaller the value of h, the higher the forecast accuracy, k is a parameter that changes with h, and x t+h It represents the cloud cover forecast data for the t+hth period, that is, the cloud cover forecast data for the hth period after the current period. Several periods of cloud cover forecast data constitute the cloud cover trend data.

[0083] In this embodiment, the time corresponding to the cloud amount prediction data lower than the cloud amount parameter in the cloud amount trend data is obtained, and the time closest to the current time is used as the shooting time of the secondary task planning of the corresponding target area. When there is a shooting time conflict between different target areas, the shooting time of the target area with relatively low priority is postponed until there is no shooting time conflict, and the postponed shooting time is still within the time corresponding to the cloud amount prediction data lower than the cloud amount parameter in the cloud amount trend data. If the postponed shooting time exceeds the time corresponding to the cloud amount prediction data lower than the cloud amount parameter in the cloud amount trend data, the cloud amount change trend of the corresponding target area within the next 6 hours is predicted again after a preset time (3 hours in this embodiment), and the shooting time is adjusted again according to the above process, and so on.

[0084] Exemplarily, in S130, the shooting parameters include: a side swing angle parameter.

[0085] The step of adjusting the shooting parameters of the secondary task planning of the corresponding target area according to the unqualified image data includes:

[0086] Obtain the side swing angle data, shooting height data, pixel height data and ground resolution data of the unqualified image data, and obtain the building height data of the unqualified image data according to the side swing angle data, the shooting height data, the pixel height data and the ground resolution data.

[0087] The side swing angle parameter of the secondary mission planning of the corresponding target area is adjusted according to the building height data.

[0088] Specifically, in this embodiment, the side swing angle data θ, shooting height data H and ground resolution data R of the unqualified image data can be directly obtained; the pixel height data (i.e., the pixel height of the highest building in the unqualified image data) needs to be obtained by performing image recognition on the unqualified image data.

[0089] In this embodiment, a neural network-based target detection algorithm is used to perform image recognition on unqualified image data to obtain the building outline, and then the pixel height of the highest building is obtained through the building outline. Specifically, one hundred remote sensing image data containing buildings and other objects are collected in advance, eighty of which are unqualified image data, and the highest building outline is marked for the one hundred remote sensing images, and randomly divided into training sets and test sets according to a ratio of 7:3. Select an existing target detection model (in this embodiment, the YOLOV5 model is selected, and in other possible embodiments, it can also be an SSD model, R-CNN model, etc.), set training parameters, including learning rate, batch size, number of iterations, etc., use the training set to train the target detection model, optimize the model parameters through the back propagation algorithm, and after iterative training for several times, obtain a trained target detection model, and use the test set to verify the performance of the trained target detection model. The trained and tested target detection model is used for actual unqualified image data to obtain the building outline, and the number of pixels in the height direction of the building outline is extracted, which is the pixel height data.

[0090] In this embodiment, the building height data H′ of the unqualified image data is obtained by the following formula:

[0091]

[0092] In the formula, H represents the shooting height data, represents pixel height data, R represents ground resolution data, and θ represents side swing angle data.

[0093] In this embodiment, the side swing angle parameter θ′ of the secondary mission planning of the target area is obtained by the following formula:

[0094]

[0095] Wherein, ρ represents the basic parameter of the roll angle, the unit is degree, 0≤ρ≤10, and ρ is 3 degrees in this embodiment; μ represents the roll angle weight coefficient, has no unit, 0.5≤μ≤3, and μ is 1 in this embodiment.

[0096] like Figure 2 As shown, exemplarily, in S150, the image stitching of the image data captured by the secondary task and the qualified image data in the image data set includes:

[0097] S151, extracting features from the image data captured by the secondary task and the qualified image data in the image data set to obtain a feature set, wherein the feature set includes a plurality of feature subsets, and a feature subset includes: all features of an image data captured by the secondary task or all features of a qualified image data.

[0098] S152, performing feature matching on adjacent feature subsets corresponding to every two adjacent regions to obtain a matching feature set, wherein the matching feature set includes a plurality of matching subsets, and a matching subset includes: the same features in two adjacent feature subsets.

[0099] S153, reconstructing the coordinates of the image data captured by the secondary task and the qualified image data in the image data set according to the matching feature set.

[0100] S154, performing image fusion on the image data captured by the secondary task after coordinate reconstruction and the qualified image data after coordinate reconstruction.

[0101] Exemplarily, in S151, the feature extraction adopts a feature detection algorithm, and the feature detection algorithm includes: a scale-invariant feature transformation algorithm, an accelerated robust feature algorithm, and a corner detection algorithm.

[0102] Specifically, in this embodiment, a scale-invariant feature transformation algorithm is used to perform feature extraction on the image data taken by the secondary task and the qualified image data in the image data set. The scale-invariant feature transformation algorithm can extract more feature points and is invariant to the scale, rotation and affine transformation of the image. It is also highly robust to noise and illumination changes, and can provide comprehensive support for subsequent feature matching, coordinate reconstruction and image fusion.

[0103] Specifically, each feature in the feature subset is represented in the form of a high-dimensional vector.

[0104] Exemplarily, in S152, the feature matching adopts a similarity measurement algorithm, and the similarity measurement algorithm includes: Euclidean distance method, cosine similarity method, Manhattan distance method and Pearson correlation coefficient method.

[0105] Specifically, in this embodiment, the Euclidean distance method is used to perform feature matching on adjacent feature subsets corresponding to every two adjacent regions, and the Euclidean distance between each two features of the adjacent feature subsets corresponding to every two adjacent regions is calculated. Two features whose Euclidean distance is less than a preset value are the same features.

[0106] Exemplarily, in S153, reconstructing the coordinates of the image data captured by the secondary task and the qualified image data in the image data set according to the matching feature set includes:

[0107] Calculate the geographic coordinate transformation formula parameters between the adjacent area image data corresponding to each matching subset.

[0108] The coordinates of the image data captured by the secondary task and the qualified image data in the image data set are reconstructed according to the geographic coordinate transformation formula parameters.

[0109] Specifically, in this embodiment, the affine transformation formula in the geographic coordinate transformation formula is used to calculate the geographic coordinate transformation formula parameters between the adjacent area image data corresponding to each matching subset. Affine transformation is a linear mapping that maintains the flatness and parallelism of the graph. Its basic formula is as follows:

[0110] X′=AX+BY+C

[0111] Y′=DX+EY+F

[0112] Wherein, (X′, Y′) represents the coordinates after affine transformation, (X, Y) represents the coordinates before affine transformation, and (A, B, C, D, E, F) represents the affine transformation parameters determined according to the coordinate system relationship between the adjacent area image data corresponding to each matching subset. In other possible embodiments, the projection transformation formula in the geographic coordinate transformation formula can also be used to calculate the geographic coordinate transformation formula parameters between the adjacent area image data corresponding to each matching subset.

[0113] In this embodiment, all qualified image data in the first remote sensing shooting task are defaulted to the same coordinate system, while the image data shot by the second task has a different side swing angle from the first remote sensing shooting task, so the coordinate system is different, and the image data shot by the second task needs to be reconstructed. When each image data shot by the second task has qualified image data of the first remote sensing shooting task in the adjacent area, each image data shot by the second task is converted into the coordinate system of qualified image data of the first remote sensing shooting task in the adjacent area according to the parameters of the geographic coordinate transformation formula; when several image data shot by the second task are continuous areas, that is, only the first and last areas of the continuous area have qualified image data of the first remote sensing shooting task in the adjacent area, firstly, the image data shot by the second task corresponding to the first and last areas in the continuous area are converted into the coordinate system of qualified image data of the first remote sensing shooting task in the adjacent area according to the parameters of the geographic coordinate transformation formula, and then the image data shot by the second task corresponding to the second, second to last, third, and third to last areas in the continuous area are reconstructed in turn until the coordinate reconstruction of all image data shot by the second task corresponding to the continuous area is completed.

[0114] In a possible embodiment, the qualified image data in the first remote sensing shooting task and the image data shot in the second task do not have the same coordinate system. Then the coordinate system of the first qualified image data is used as the reference coordinate system, and the qualified image data in the first remote sensing shooting task or the image data shot in the second task in the adjacent area of ​​the qualified image data are reconstructed in sequence until the coordinates of all the image data shot in the second task and all the qualified image data in the image data set are reconstructed.

[0115] Exemplarily, in S154, the image fusion adopts image fusion technology, and the image fusion technology includes: Poisson fusion, Laplace pyramid fusion, multi-band fusion and feathering fusion.

[0116] Specifically, in this embodiment, Poisson fusion is used to fuse the image data captured by the secondary task after coordinate reconstruction and the qualified image data after coordinate reconstruction. Poisson fusion can seamlessly fuse the two images, so that the final composite image looks natural and realistic. Since the image data captured by the secondary task after coordinate reconstruction and the qualified image data after coordinate reconstruction have been unified in the coordinate system, it is only necessary to directly use Poisson fusion to fuse the image data of two adjacent areas.

[0117] The embodiment of the present application improves the quality of remote sensing image data by performing secondary task planning on the target area corresponding to unqualified image data; and improves the compatibility between different image data by performing image stitching on the image data taken by the secondary task and the qualified image data in the image data set.

[0118] By setting the priority of the secondary task planning of the target area and setting the shooting time of the secondary task planning of the corresponding target area according to the priority, the rationality of the secondary task planning is improved.

[0119] By adjusting the cloud cover parameters of the secondary mission planning and adjusting the shooting time of the secondary mission planning according to the cloud cover parameters and cloud cover change trends, the influence of cloud cover on the image quality of the secondary mission shooting is avoided and the rationality of the secondary mission planning is improved.

[0120] It is proposed to obtain building height data of unqualified image data according to the swing angle data, shooting height data, pixel height data and ground resolution data, and adjust the swing angle parameters of secondary task planning according to the building height data, thereby improving the quality of image data taken by the secondary task.

[0121] By combining feature extraction, feature matching, coordinate reconstruction and image fusion, the image data taken by the secondary task and the qualified image data in the image data set are stitched, which reduces the difficulty of processing between different image data and improves the accuracy of image stitching.

[0122] The methods, systems or modules described in this application can be implemented in a computer-readable program code or in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (English: Application Specific Integrated Circuit; Abbreviation: ASIC), a programmable logic controller and an embedded microcontroller. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, AtmelAT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the system included in it for implementing various functions can also be regarded as a structure within the hardware component. Or even, a system for implementing various functions may be considered as both a software module implementing the method and a structure within a hardware component.

[0123] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0124] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0125] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A secondary task planning method based on remote sensing photography, characterized in that: The following steps are involved: Acquire an image data set of a completed shooting task in the first remote sensing shooting task, and determine whether the quality of each image data in the image data set is qualified; Setting the target area corresponding to the unqualified image data in the image data set as the target area for secondary task planning, setting the priority of the secondary task planning for the target area, and setting the shooting time of the secondary task planning for the corresponding target area according to the priority; Adjusting the shooting parameters of the secondary task planning of the corresponding target area according to the unqualified image data; Perform a secondary task according to the shooting time and the shooting parameters; Image stitching is performed on the image data captured by the secondary task and the qualified image data in the image data set.

2. The method for secondary task planning based on remote sensing photography according to claim 1, characterized in that: The determining whether the quality of each image data in the image data set is qualified comprises: Performing a quality assessment on each image data in the image data set, and when a quality assessment result of an image data is higher than a preset quality assessment threshold, judging the image data as qualified image data, and when a quality assessment result of an image data is lower than or equal to the preset quality assessment threshold, judging the image data as unqualified image data; The quality assessment is image clarity assessment and / or image integrity assessment.

3. The method for secondary task planning based on remote sensing photography according to claim 1, characterized in that: The step of setting the priority of the secondary task planning of the target area and setting the shooting time of the secondary task planning of the corresponding target area according to the priority includes: The priority of the secondary task planning of the target area is set to be higher than the priority of the unfinished shooting tasks in the first remote sensing shooting task, and the priority of the secondary task planning of each target area is sorted according to the position of each target area; The shooting time of the secondary task planning of the corresponding target area is set according to the priority of the secondary task planning of each target area after sorting.

4. The method for secondary task planning based on remote sensing photography according to claim 1, characterized in that: The shooting parameters include: cloud cover parameters; The step of adjusting the shooting parameters of the secondary task planning of the corresponding target area according to the unqualified image data includes: Performing cloud cover analysis on the unqualified image data to obtain cloud cover data and occlusion degree data corresponding to the unqualified image data; Adjust the cloud cover parameters of the secondary task planning of the corresponding target area according to the cloud cover data and the occlusion degree data; Acquire historical cloud cover data of the target area corresponding to the cloud cover parameter, and predict the cloud cover change trend of the corresponding target area in the future based on the historical cloud cover data; The shooting time of the secondary task planning of the corresponding target area is adjusted according to the cloud amount parameter and the cloud amount change trend.

5. The method for secondary task planning based on remote sensing photography according to claim 1, characterized in that: The shooting parameters include: side swing angle parameters; The step of adjusting the shooting parameters of the secondary task planning of the corresponding target area according to the unqualified image data includes: Obtaining the side swing angle data, shooting height data, pixel height data and ground resolution data of the unqualified image data, and obtaining the building height data of the unqualified image data according to the side swing angle data, the shooting height data, the pixel height data and the ground resolution data; The side swing angle parameter of the secondary mission planning of the corresponding target area is adjusted according to the building height data.

6. The method for secondary task planning based on remote sensing photography according to claim 1, characterized in that: The image stitching of the image data captured by the secondary task and the qualified image data in the image data set comprises: Performing feature extraction on the image data captured by the secondary task and the qualified image data in the image data set to obtain a feature set, wherein the feature set includes a plurality of feature subsets, and a feature subset includes: all features of an image data captured by the secondary task or all features of a qualified image data; Performing feature matching on adjacent feature subsets corresponding to every two adjacent regions to obtain a matching feature set, wherein the matching feature set includes a plurality of matching subsets, and a matching subset includes: the same features in two adjacent feature subsets; Reconstructing coordinates of the image data captured by the secondary task and the qualified image data in the image data set according to the matching feature set; Image fusion is performed on the image data taken by the secondary task after coordinate reconstruction and the qualified image data after coordinate reconstruction.

7. The method for secondary task planning based on remote sensing photography according to claim 6, characterized in that: The feature extraction adopts a feature detection algorithm, which includes: a scale-invariant feature transformation algorithm, an accelerated robust feature algorithm and a corner point detection algorithm.

8. The method for secondary task planning based on remote sensing photography according to claim 6, characterized in that: The feature matching adopts a similarity measurement algorithm, which includes: Euclidean distance method, cosine similarity method, Manhattan distance method and Pearson correlation coefficient method.

9. The method for secondary task planning based on remote sensing photography according to claim 6, characterized in that: The coordinate reconstruction of the image data captured by the secondary task and the qualified image data in the image data set according to the matching feature set comprises: Calculate the geographic coordinate transformation formula parameters between the adjacent area image data corresponding to each matching subset; The coordinates of the image data captured by the secondary task and the qualified image data in the image data set are reconstructed according to the geographic coordinate transformation formula parameters.

10. The method for secondary task planning based on remote sensing photography according to claim 6, characterized in that: The image fusion adopts image fusion technology, and the image fusion technology includes: Poisson fusion, Laplace pyramid fusion, multi-band fusion and feathering fusion.

Images