An automatic extraction method and system for the effective area of a UAV photogrammetry model

Abstract

The present invention relates to the technical field of unmanned aerial vehicles, and particularly to a method and system for automatically extracting the effective area of a model in unmanned aerial vehicle photogrammetry. The method includes: obtaining the pose information and object space elevation information of the unmanned aerial vehicle images, and constructing the maximum circumscribed rectangle of the experimental area; extracting elements as constraints to perform regional segmentation and spatial index construction on the maximum circumscribed rectangle of the experimental area; searching for the coverage area of the unmanned aerial vehicle images according to the spatial index, and calculating the influencing factors of the effective area of the unmanned aerial vehicle photogrammetry model; and extracting the final multi-element constrained effective area of the model according to the influencing factors. By analyzing and calculating the influencing factors of the effective area of the unmanned aerial vehicle photogrammetry model, the present invention constructs an effective area extraction model with multi-element constraints, automatically extracts the effective area of the unmanned aerial vehicle photogrammetry model, optimizes the model area in the three-dimensional reconstruction process, changes the previous strategy of manually trimming the effective area of the model after generating the model, and improves the reconstruction efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles (UAVs), and more particularly to techniques related to the three-dimensional model reconstruction of UAV photogrammetry, and in particular to a method and system for automatically extracting the effective area of a model in UAV photogrammetry. Background Art

[0002] With the rapid development of unmanned aerial vehicle (UAV) technology, UAVs have become an efficient and convenient platform for obtaining remote sensing data. UAV photogrammetry uses a UAV flight platform to carry one or more sensors to obtain ground object images of the experimental area from multiple angles, and uses a three-dimensional modeling algorithm for multi-view images to reconstruct the three-dimensional model of the experimental area, which has been widely used in fields such as real-scene three-dimensional construction, disaster emergency response, and cultural relic protection.

[0003] In order to ensure the integrity of the reconstructed model, the image data set collected in the field usually contains a large amount of redundant information, and the main sources of the redundant information are as follows: 1) The flight route planning expands outward on the basis of the experimental area boundary to ensure the integrity of the boundary. The expanded images will contain a large amount of redundant information outside the experimental area boundary while covering the boundary; 2) In order to obtain the texture information of the side of the ground object, the data set generally contains a large number of oblique images. The oblique images cover a large area and will contain both the ground objects inside the experimental area and a large amount of ground object information outside the experimental area boundary; 3) In order to improve the acquisition efficiency, a multi-lens camera (such as a five-lens camera) is often used for data acquisition. Due to the different orientations of the camera lenses, in the area around the experimental area boundary, some cameras are oriented outside the experimental area, and the captured images are also invalid images outside the experimental area. The redundant information causes the covered area of the image set to contain a large number of non-uniformly distributed small overlapping areas outside the experimental area, so there will be a large number of fragmented and incomplete invalid model areas outside the experimental area in the reconstructed model.

[0004] In view of the above situation, the existing methods mainly reconstruct all the areas covered by the image set as a whole, and then manually crop the reconstruction result to obtain the effective area of the three-dimensional model. The large number of invalid areas in the reconstructed model reduces the production efficiency on the one hand and affects the aesthetics of the model on the other hand. Therefore, a method and system for extracting the effective area of a model in UAV photogrammetry are needed to optimize the reconstruction result and improve the three-dimensional reconstruction efficiency. Summary of the Invention

[0005] In order to solve the problem that the UAV image set contains a large amount of redundant information, resulting in a large number of invalid areas outside the experimental area in the reconstructed model, the present invention constructs a method and system for automatically extracting the effective area of a model in UAV photogrammetry to improve the three-dimensional reconstruction efficiency.

[0006] The present invention is implemented by adopting the following technical solutions:

[0007] A method for automatically extracting the effective area of an unmanned aerial vehicle (UAV) photogrammetry model, comprising the following steps:

[0008] Obtain the pose information and object-space elevation information of the UAV images, and construct the maximum circumscribed rectangle of the experimental area;

[0009] Extract features as constraints, and perform regional segmentation and spatial index construction on the maximum circumscribed rectangle of the experimental area;

[0010] Find the coverage area of the UAV images according to the spatial index, and calculate the influencing factors of the effective area of the UAV photogrammetry model;

[0011] Extract the final effective area of the model with multi-factor constraints according to the influencing factors.

[0012] As a further solution of the present invention, the method for constructing the maximum circumscribed rectangle of the experimental area is:

[0013] Obtain the known aerial triangulation connection point information, and input the pose information of the UAV images;

[0014] Set the connection points P = {p1, p2, …, p n}, and find the position information of the connection point p i = {x i , y i , z i};

[0015] According to the maximum values x max , y max of the connection points on the X and Y axes and the minimum values x min , y min , construct the maximum circumscribed rectangle R MBR .

[0016] Furthermore, the pose information includes the image pose obtained from the aerial triangulation connection point information, and the position and attitude information in the flight control data.

[0017] Furthermore, the method for constructing the maximum circumscribed rectangle of the experimental area further includes: using the image ground projection polygon to construct the maximum circumscribed rectangle R MBR .

[0018] Furthermore, if the rough boundary of the experimental area is known, generate the maximum circumscribed rectangle through the rough boundary.

[0019] As a further solution of the present invention, when performing regional segmentation on the maximum circumscribed rectangle of the experimental area, it includes:

[0020] Perform clustering analysis on the object-space elevation information of the experimental area, and the number of clusters is k;

[0021] Define the height H of the experimental area as H = {h1, h2, …, h k}, where h k is the average elevation of each category;

[0022] Define the initial ground height Z G of the experimental area as h1. According to the size and focal length f of the image sensor, and based on the principle of similar triangles, calculate the projected quadrilateral area of the image set I = {i1, i2, …, i m} under vertical conditions, and calculate the average value S of the vertical projected area of the images;

[0023] Define the minimum segmentation area or set the minimum segmentation area S according to the characteristics of the experimental area min ;

[0024] Use the minimum segmentation area S min and the minimum number of connection points N min as constraints to perform regional segmentation and construct a spatial index for the maximum circumscribed rectangle R MBR of the experimental area.

[0025] Furthermore, the spatial index includes a grid spatial index, a binary tree index, and a quadtree index.

[0026] Furthermore, when the rectangle area or the number of connection points of the segmentation node is less than the threshold, stop the segmentation of the node, set the node that meets the constraints as the leaf node of the spatial index, and the maximum circumscribed rectangle R MBR of the experimental area after segmentation is R = {R1, R2, …, R n}.

[0027] As a further solution of the present invention, the method for calculating the influencing factors of the effective area of the UAV photogrammetry model is as follows:

[0028] Find the coverage area of the UAV image projected quadrilateral according to the spatial index;

[0029] Calculate the ratio of the intersection area between the leaf node rectangle of the coverage area and the projected quadrilateral to the rectangle area, and set it as the overlap degree of this node;

[0030] Calculate the overlap degree coverage of each image in turn, and count the influencing factors of each leaf node.

[0031] Furthermore, the influencing factors include the sum of rectangle overlap degrees O i , the leaf node rectangle area A i , and the number of connection points T i and other influencing factors.

[0032] Furthermore, the method for constructing the UAV image projected quadrilateral is as follows: According to the image ii Using the pose information and the initial height h1, the object-space projection of the image corner points is obtained by the collinearity equation, and the object-space projection quadrilateral Q of the image is constructed using the four corner points. i .

[0033] As a further aspect of the present invention, the method for extracting the effective region of the final multi-factor constrained model includes:

[0034] Construct an importance function W based on the influence of the influencing factors of each leaf node on the effective region of the model. i ;

[0035] According to the constructed importance function W i Extract the effective rectangles and find the union U of the effective rectangles;

[0036] Perform region fitting on the polygon U to extract the final effective region of the model.

[0037] The present invention also includes an automatic extraction system for the effective region of a UAV photogrammetry model. The automatic extraction system for the effective region of a UAV photogrammetry model uses the aforementioned method for automatically extracting the effective region of a UAV photogrammetry model to optimize the model region in the 3D reconstruction process. The automatic extraction system for the effective region of a UAV photogrammetry model includes a maximum circumscribed rectangle construction module, a rectangle segmentation module, an influencing factor calculation module, and an effective region extraction module.

[0038] The maximum circumscribed rectangle construction module is used to obtain the pose information and object-space elevation information of the UAV image and construct the maximum circumscribed rectangle of the experimental area.

[0039] The rectangle segmentation module is used to perform region segmentation and spatial index construction on the maximum circumscribed rectangle of the experimental area with multiple constraints.

[0040] The influencing factor calculation module is used to find the coverage area of the UAV image according to the spatial index and calculate the influencing factors of the effective region of the UAV photogrammetry model; and

[0041] The effective region extraction module is used to extract the final multi-factor constrained effective region of the model according to the influencing factors.

[0042] The present invention also includes a computer device, including: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the at least one processor to execute the method for automatically extracting the effective region of the UAV photogrammetry model.

[0043] The present invention further includes a computer-readable storage medium storing computer instructions for causing a computer to execute the method for automatically extracting the effective area of a drone photogrammetry model as described above.

[0044] The technical solution provided by the present invention has the following beneficial effects:

[0045] 1. The method for automatically extracting the effective area of a drone photogrammetry model provided by the present invention utilizes image pose and object elevation information, and can extract the effective area of a drone photogrammetry model at the beginning or during the three-dimensional reconstruction process, avoiding the reconstruction of invalid areas and improving the reconstruction efficiency and model aesthetics.

[0046] 2. The method for automatically extracting the effective area of a drone photogrammetry model provided by the present invention utilizes image pose and object elevation information to extract the effective area of a drone photogrammetry model, and can timely detect whether the data acquisition is sufficient. If some target areas do not meet the reconstruction conditions, on-site reshooting can be carried out.

[0047] 3. The method for extracting the effective area of a drone photogrammetry model provided by the present invention is applicable to image sets obtained by multiple platforms and multiple sensors, can adapt to terrain and object heights, and ensure the correctness and stability of the extraction results.

[0048] These aspects or other aspects of the present invention will be more clearly understood in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following will briefly introduce the drawings required for the description of the exemplary embodiments or related technologies. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0050] Figure 1 is a flowchart of a method for automatically extracting the effective area of a drone photogrammetry model of the present invention.

[0051] Figure 2 is a flowchart of constructing the maximum circumscribed rectangle in the method for automatically extracting the effective area of a drone photogrammetry model of the present invention.

[0052] Figure 3 is a flowchart of dividing the area by the maximum circumscribed rectangle in the method for automatically extracting the effective area of a drone photogrammetry model of the present invention.

[0053] Figure 4 Schematic diagram of maximum circumscribed rectangle segmentation in the method for automatically extracting the effective area of a UAV photogrammetry model according to the present invention.

[0054] Figure 5 Flowchart of calculating the influencing factors of the effective area in the method for automatically extracting the effective area of a UAV photogrammetry model according to the present invention.

[0055] Figure 6 Schematic diagram of the visualization display of the superposition degree O in the method for automatically extracting the effective area of a UAV photogrammetry model according to the present invention. i of the visualization display.

[0056] Figure 7 Schematic diagram of the visualization display of the number of connection points T in the method for automatically extracting the effective area of a UAV photogrammetry model according to the present invention. i of the visualization display.

[0057] Figure 8 Flowchart of extracting the effective area of the final multi-factor constraint model in the method for automatically extracting the effective area of a UAV photogrammetry model according to the present invention.

[0058] Figure 9 Schematic diagram of model area extraction in the method for automatically extracting the effective area of a UAV photogrammetry model according to the present invention.

[0059] Figure 10 System block diagram of the system for automatically extracting the effective area of a UAV photogrammetry model in an embodiment of the present invention. Detailed implementation manners

[0060] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0061] In some processes described in the specification, claims and above-mentioned drawings of the present invention, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear herein or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are of different types.

[0062] Next, in combination with the accompanying drawings in the exemplary embodiments of the present invention, the technical solutions in the exemplary embodiments of the present invention will be clearly and completely described. Obviously, the described exemplary embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0063] A method and system for automatically extracting the effective area of an unmanned aerial vehicle (UAV) photogrammetry model provided by the present invention analyze and calculate the influencing factors of the effective area of the UAV photogrammetry model, thereby constructing an effective area extraction model with multi-factor constraints, automatically extracting the effective area of the UAV photogrammetry model, optimizing the model area in the 3D reconstruction process, changing the previous strategy of manually cropping the effective area of the model after generating the model, and improving the reconstruction efficiency. The present invention can solve the problem that the UAV image set contains a lot of redundant information, resulting in a large number of invalid areas outside the experimental area in the reconstructed model, and improve the 3D reconstruction efficiency.

[0064] The following further illustrates the technical solutions of the present invention in combination with specific embodiments:

[0065] Refer to Figure 1 as shown Figure 1 is a flowchart of a method for automatically extracting the effective area of an unmanned aerial vehicle (UAV) photogrammetry model provided by the present invention. An embodiment of the present invention provides a method for automatically extracting the effective area of an unmanned aerial vehicle (UAV) photogrammetry model, including the following steps:

[0066] S1: Obtain the pose information and object space elevation information of the UAV images, and construct the maximum circumscribed rectangle of the experimental area.

[0067] It should be particularly noted that in this embodiment, as shown in Figure 2 the method for constructing the maximum circumscribed rectangle of the experimental area is as follows:

[0068] S101: Obtain the known aerial triangulation connection point information and input the pose information of the UAV images;

[0069] S102: Set the connection points P = {p1, p2,..., p n}, and find the position information of the connection point p i = {x i , y i , z i};

[0070] S103: According to the maximum values x max , y max of the connection points on the X and Y axes and the minimum values x min , y min, construct the maximum circumscribed rectangle R of the experimental area MBR .

[0071] In this embodiment, the pose information includes the image pose obtained from the aerial triangulation connection point information, and the position and attitude information in the flight control data.

[0072] When extracting the maximum circumscribed rectangle of the experimental area, the method for constructing the maximum circumscribed rectangle of the experimental area further includes: using the image ground projection polygon to construct the maximum circumscribed rectangle R MBR . If the rough boundary of the experimental area is known, generate the maximum circumscribed rectangle through the rough boundary.

[0073] The above-mentioned generation of the maximum circumscribed rectangle R of the experimental area based on the image position and attitude of the experimental area or the object space position information such as the ground connection points MBR When. This embodiment is known aerial triangulation connection point information, which is only a part of the embodiments of the present invention. Other model effective area influence factor information not included in the embodiments can be added or the model effective area influence factor information already existing in the embodiments can be deleted. Therefore, when extracting the maximum circumscribed rectangle of the experimental area, the process is as follows: input the pose information of the UAV image (including the image pose obtained from aerial triangulation, the position and attitude information in the flight control data, etc.), the connection points P = {p1, p2,..., p n}, find the connection point p i = {x i , y i , z i}, find the maximum values x max , y max and the minimum values x min , y min of the connection point p MBR on the X and Y axes, and then construct the maximum circumscribed rectangle R of the experimental area MBR ; or use the image ground projection polygon to construct the maximum circumscribed rectangle R

[0074] S2: Extract elements as constraints to perform regional segmentation and spatial index construction on the maximum circumscribed rectangle of the experimental area.

[0075] It should be noted in particular that, as shown in Figure 3 , when performing regional segmentation on the maximum circumscribed rectangle of the experimental area, it includes:

[0076] S201. Perform clustering analysis on the object elevation information of the experimental area, and the number of clusters is k;

[0077] S202. Define the height H of the experimental area = {h1, h2,..., h k}, h kis the average elevation for each category;

[0078] S203. Define the initial ground height Z of the experimental area G = h1. According to the image sensor size and focal length f, and based on the principle of similar triangles, find the projected quadrilateral area of the image set I = {i1, i2, …, i m} under vertical conditions, and find the average value S of the vertical projected area of the image;

[0079] S204. Define the minimum segmentation area or set the minimum segmentation area S according to the characteristics of the experimental area min ;

[0080] S205. Use the minimum segmentation area S min and the minimum number of connection points N min as constraints to perform regional segmentation and spatial index construction on the maximum circumscribed rectangle R of the experimental area MBR ;

[0081] In this embodiment, the spatial index includes but is not limited to grid spatial index, binary tree index, and quadtree index.

[0082] When the rectangle area or the number of connection points of the segmentation node is less than the threshold, stop the segmentation of the node, set the node that meets the constraints as the leaf node of the spatial index, and the maximum circumscribed rectangle R of the experimental area after segmentation MBR = {R1, R2, …, R n};

[0083] Therefore, in the rectangle segmentation and index construction of the experimental area with multiple constraints, the process is as follows: First, calculate the projected area of the UAV image using the pose information and object space elevation information of the image, and then set the minimum segmentation area S according to the calculation result min , if the connection point information of the experimental area is known, the minimum number of connection points N can be set min , and the same applies to other elements; finally, use the above threshold as a constraint to perform regional segmentation and spatial index construction on the maximum circumscribed rectangle R of the experimental area MBR , and set the nodes that meet the constraints as the leaf nodes of the spatial index.

[0084] Perform clustering analysis on the object space elevation information of the experimental area, with the number of clusters k, and then let the height H of the experimental area = {h1, h2, …, h k}, h i is the average elevation for each category. Let the initial ground height Z of the experimental area G = h1. According to the image sensor size and focal length f, based on the principle of similar triangles, find the image set I = {i1, i2, …, i m} The area of the projected quadrilateral under vertical conditions, and calculate the average value S of the vertical projection area of the image. Define the minimum segmentation area Or set S according to the characteristics of the experimental area min .

[0085] Use the minimum segmentation area S min And the minimum number of connection points N min And other elements as constraints for the maximum circumscribed rectangle R of the experimental area MBR Perform regional segmentation and spatial index construction (including grid spatial index, binary tree index, quadtree index, etc.). When the rectangle area or the number of connection points of the segmentation node is less than the threshold, stop the segmentation of the node, set the node as a leaf node, and after segmentation, the maximum circumscribed rectangle R of the experimental area MBR ={R1, R2, …, R n}}, and the segmentation example is Figure 4 As shown, the darker the color, the larger the value.

[0086] S3: Search for the coverage area of the UAV image according to the spatial index, and calculate the influencing factors of the effective area of the UAV photogrammetry model.

[0087] It should be noted that when extracting the influencing factors of the effective area of the UAV photogrammetry model, first search for the coverage area of the UAV image projection quadrilateral according to the spatial index, then calculate the ratio of the intersection area of the leaf node rectangle in the coverage area and the projection quadrilateral to the rectangle area, and set it as the overlap degree of the node. Next, calculate the overlap degree coverage of each image in turn. Finally, count the sum O of the rectangle overlap degrees of each leaf node i , the leaf node rectangle area A i , the number of connection points T i And other influencing factors.

[0088] In this embodiment, as shown in Figure 5 , the method for calculating the influencing factors of the effective area of the UAV photogrammetry model is as follows:[[]]

[0089] S301. Search for the coverage area of the UAV image projection quadrilateral according to the spatial index;

[0090] S302. Calculate the ratio of the intersection area of the leaf node rectangle in the coverage area and the projection quadrilateral to the rectangle area, and set it as the overlap degree of the node;

[0091] S303. Calculate the overlap degree coverage of each image in turn, and count the influencing factors of each leaf node.

[0092] In this embodiment, the influencing factors include the sum O of the rectangle overlap degrees i , the leaf node rectangle area A i And the number of connection points Ti influence factors including. The construction method of the UAV image projection quadrilateral is as follows: According to the pose information of image i i and the initial height h1, the object space projection of the image corner points is obtained by using the collinearity equation, and the object projection quadrilateral Q of the image is constructed by using the four corner points i .

[0093] When calculating the influence factors in the effective area of the experimental area model, according to the pose information of image i i and the initial height h1, the object space projection of the image corner points is obtained by using the collinearity equation, and the object projection quadrilateral Q of the image is constructed by using the four corner points i .

[0094] According to the spatial index, the leaf node rectangle R i intersecting with the projection quadrilateral Q is searched for in sequence, and the height h of the experimental area closest to the average height of the leaf node is searched for i , if the height h i is not equal to h1, then h i is used to recalculate the projection quadrilateral Q i (furthermore, if the terrain of the experimental area is relatively flat, that is, there is only height h1 after height clustering, there is no need to search for the closest height h i ); i )

[0095] According to the computational geometry theory, then calculate the intersection area of the leaf node rectangle R i and the projection quadrilateral Q i , and the ratio of the intersection area to the area A i of the rectangle R i is used as the overlap degree. Calculate the overlap degree coverage of each image in sequence, and count the sum O i of the rectangle overlap degrees of each leaf node. At the same time, count the area A i of each leaf node rectangle, the number of connection points T i and other related elements, and visually display O i and T i , as shown in Figure 6 and Figure 7 , the darker the color, the larger the value

[0096] S4: Extract the final multi-factor constrained model effective area according to the influence factors

[0097] It should be noted that when extracting the multi-factor constrained model effective area, according to the area A i of the leaf node rectangle, the number of connection points T i , the overlap degree O i and other elements' influence on the model effective area, construct the importance function W i, and then use methods such as region contraction or region growth to extract valid rectangles. Obtain the union U of the valid rectangles, and perform region fitting on the polygon U to extract the final model valid region.

[0098] In this embodiment, as shown in Figure 8 , the method for extracting the final model valid region with multi-factor constraints includes:

[0099] S401. Construct an importance function W according to the influence of the influencing factors of each leaf node on the model valid region i ;

[0100] S402. Extract valid rectangles according to the constructed importance function W i and obtain the union U of the valid rectangles;

[0101] S403. Perform region fitting on the polygon U to extract the final model valid region.

[0102] When extracting the model region with multi-factor constraints, the specific extraction process is as follows:

[0103] (1) Analyze the influence of factors such as the rectangle area A i , the number of connection points T i , and the overlap degree O i of each leaf node on the model valid region in turn. The influence functions of each factor are:

[0104]

[0105]

[0106]

[0107] According to the analysis, it can be obtained that the number of connection points T i and the overlap degree O i are positively correlated with the model valid region, and the rectangle area A i is negatively correlated. Therefore, construct an importance function W i :

[0108]

[0109] Where: W i can be determined using a more refined method, such as: calculating the change rate C i of the spatial adjacency overlap degree of the leaf node rectangles, the height consistency H i of the connection points, the model valid region index V i etc., and using machine learning-related algorithms in artificial intelligence to construct a multi-factor constraint model valid region function:

[0110]

[0111] W i = f(V i )

[0112] (2) According to W i Perform threshold segmentation or region contraction to extract the effective leaf node rectangle R of the experimental area i . Obtain the union U of the effective rectangles, perform region fitting on the polygon U, and extract the final model area. As Figure 9 shown, in the figure, the middle color area represents the effective area of the model.

[0113] The present invention provides a method for automatically extracting the effective area of a drone photogrammetry model, which can utilize image pose and object space elevation information, and can extract the effective area of the drone photogrammetry model at the beginning or in the middle process of 3D reconstruction, avoiding the reconstruction of invalid areas and improving the reconstruction efficiency and model aesthetics; by using image pose and object space elevation information, the effective area of the drone photogrammetry model can be extracted, and it can be timely found whether the data acquisition is sufficient. If some target areas do not meet the reconstruction conditions, on-site supplementary shooting can be carried out. It is applicable to image sets obtained by multiple platforms and multiple sensors, can adapt to terrain and object heights, and ensure the correctness and stability of the extraction results.

[0114] It should be understood that although the above is described in a certain order, these steps are not necessarily executed in the above order. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, a part of the steps in this embodiment may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0115] In one embodiment, as Figure 10 shown, a system for automatically extracting the effective area of a drone photogrammetry model is provided, including a maximum circumscribed rectangle construction module 100, a rectangle segmentation module 200, an influencing factor calculation module 300, and an effective area extraction module 400. Among them:

[0116] The maximum circumscribed rectangle construction module 100 is used to obtain the pose information and object space elevation information of the drone image, and construct the maximum circumscribed rectangle of the experimental area; when constructing the maximum circumscribed rectangle, obtain the known aerial triangulation connection point information and input the pose information of the drone image; set the connection point P = {p1, p2,..., p n} and search for the connection point pi = {x i , y i , z i} position information; According to the maximum values x max , y max on the X and Y axes of the connection points and the minimum values x min , y min , construct the maximum circumscribed rectangle R MBR of the experimental area. Among them, the method of constructing the maximum circumscribed rectangle of the experimental area also includes: using the image ground projection polygon to construct the maximum circumscribed rectangle R MBR . If the approximate boundary of the known experimental area is available, generate the maximum circumscribed rectangle through the approximate boundary.

[0117] The rectangle segmentation module 200 is used to perform region segmentation and spatial index construction on the maximum circumscribed rectangle of the experimental area under multiple constraints; when performing region segmentation on the maximum circumscribed rectangle of the experimental area, perform clustering analysis on the object space elevation information of the experimental area, and the number of clusters is k; define the height H of the experimental area = {h1, h2,..., h k}, h i is the elevation average value of each category; define the initial ground height Z G of the experimental area = h1, according to the image sensor size and focal length f, and according to the similar triangle principle, find the projected quadrilateral area of the image set I = {i1, i2,..., i m} under vertical conditions, and find the average value S of the image vertical projection area; define the minimum segmentation area or set the minimum segmentation area S min according to the characteristics of the experimental area; use the minimum segmentation area S min and the minimum number of connection points N min elements as constraints to perform region segmentation and spatial index construction on the maximum circumscribed rectangle R MBR of the experimental area.

[0118] The influencing factor calculation module 300 is used to find the coverage area of the UAV image according to the spatial index and calculate the influencing factors of the effective area of the UAV photogrammetry model; when extracting the influencing factors of the effective area of the UAV photogrammetry model, first find the coverage area of the UAV image projected quadrilateral according to the spatial index, then calculate the ratio of the intersection area of the coverage area leaf node rectangle and the projected quadrilateral to the rectangle area, and set it as the overlap degree of this node. Next, calculate the overlap degree coverage of each image in turn. Finally, count the sum O i of the rectangle overlap degrees of each leaf node, the leaf node rectangle area A i , the number of connection points T i and other influencing factors.

[0119] The effective area extraction module 400 is used to extract the final model effective area with multi-factor constraints according to the influencing factors. When extracting the model effective area with multi-factor constraints, according to the rectangular area A of the leaf node i , the number of connection points T i , the overlap degree O i and other factors affecting the model effective area, construct the importance function W i , and then use methods such as region contraction or region growth to extract the effective rectangle. Obtain the union U of the effective rectangles, and perform region fitting on the polygon U to extract the final model effective area.

[0120] In this embodiment, the automatic extraction system for the model effective area of unmanned aerial vehicle photogrammetry adopts the steps of the above-mentioned automatic extraction method for the model effective area of unmanned aerial vehicle photogrammetry when executing. Therefore, the operation process of the automatic extraction system for the model effective area of unmanned aerial vehicle photogrammetry in this embodiment will not be introduced in detail.

[0121] In one embodiment, in the embodiments of the present invention, a computer device is further provided, including at least one processor, and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor. When the instructions are executed by the at least one processor, the at least one processor executes the above-mentioned automatic extraction method for the model effective area of unmanned aerial vehicle photogrammetry. When the processor executes the instructions, it implements the steps in the above-mentioned method embodiments:

[0122] Obtain the pose information and object space elevation information of the unmanned aerial vehicle image, and construct the maximum circumscribed rectangle of the experimental area;

[0123] Extract elements as constraints, and perform region segmentation and spatial index construction on the maximum circumscribed rectangle of the experimental area;

[0124] Search for the coverage area of the unmanned aerial vehicle image according to the spatial index, and calculate the influencing factors of the effective area of the unmanned aerial vehicle photogrammetry model;

[0125] Extract the final model effective area with multi-factor constraints according to the influencing factors.

[0126] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium stores computer instructions for causing the computer to execute the above-mentioned automatic extraction method for the model effective area of unmanned aerial vehicle photogrammetry. The steps are as follows:

[0127] Obtain the pose information and object space elevation information of the unmanned aerial vehicle image, and construct the maximum circumscribed rectangle of the experimental area;

[0128] Extract elements as constraints to perform regional segmentation and spatial index construction on the maximum circumscribed rectangle of the experimental area;

[0129] Find the coverage area of the UAV image according to the spatial index, and calculate the influencing factors of the effective area of the UAV photogrammetry model;

[0130] Extract the final effective area of the multi-element constrained model according to the influencing factors.

[0131] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program characterized by computer instructions. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories.

[0132] Non-volatile memory can include read-only memory, magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory or dynamic random access memory, etc.

[0133] In summary, the technical solutions provided by the present invention have the following advantages:

[0134] 1. The method for automatically extracting the effective area of the UAV photogrammetry model provided by the present invention utilizes the image pose and object space elevation information, and can extract the effective area of the UAV photogrammetry model at the beginning or in the middle process of 3D reconstruction, avoiding the reconstruction of invalid areas and improving the reconstruction efficiency and model aesthetics.

[0135] 2. The method for automatically extracting the effective area of the UAV photogrammetry model provided by the present invention utilizes the image pose and object space elevation information to extract the effective area of the UAV photogrammetry model, and can timely discover whether the data acquisition is sufficient. If some target areas do not meet the reconstruction conditions, on-site supplementary shooting can be carried out.

[0136] 3. The method for extracting the effective area of the UAV photogrammetry model provided by the present invention is applicable to the image sets obtained by multiple platforms and multiple sensors, can adapt to the terrain and the height of ground objects, and ensure the correctness and stability of the extraction results.

[0137] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An automatic extraction method for the effective area of a drone photogrammetry model, characterized in that, Including: Obtain the pose information and object-space elevation information of the UAV images, and construct the minimum bounding rectangle of the experimental area; Extract elements as constraints, and perform regional segmentation and spatial index construction on the minimum bounding rectangle of the experimental area; Find the coverage area of the UAV image according to the spatial index, and calculate the influencing factors of the effective area of the UAV photogrammetry model. The influencing factors include the sum of rectangular overlap degrees O i , the rectangular area A of the leaf node i , and the number of connection points T i ; Extract the final model valid area with multi-element constraints according to the influencing factors; The process of performing regional segmentation and spatial index construction is as follows: First, calculate the projected area of the UAV image using the pose information of the image and the elevation information of the object space, and then set the minimum segmentation area S according to the calculation result. min , if the connection point information of the experimental area is known, set the minimum number of connection points N. min ; Finally, use the above thresholds as constraints for the maximum circumscribed rectangle R of the experimental area MBR to perform region segmentation and construct a spatial index, and set the nodes that meet the constraints as the leaf nodes of the spatial index.

2. The method for automatically extracting the effective area of the model in drone photogrammetry according to claim 1, wherein, The method for constructing the minimum bounding rectangle of the experimental area is: Obtain the known aerial triangulation connection point information and input the pose information of the UAV images; Set connection points P = {p1, p2, …, p n}, and find the position information of connection point p i = {x i , y i , z i}; According to the maximum values x max and y max of the connection points on the X and Y axes, and the minimum values x min and y min , construct the maximum circumscribed rectangle R MBR of the experimental area.

3. The method for automatically extracting the effective area of the model in drone photogrammetry according to claim 2, wherein The pose information includes the image pose obtained from the aerial triangulation connection point information, and the position and attitude information in the flight control data.

4. The method for automatically extracting the effective area of the model in drone photogrammetry according to claim 1, wherein, The method for constructing the maximum circumscribed rectangle of the experimental area further includes: using the image ground projection polygon to construct the maximum circumscribed rectangle R MBR .

5. The automatic extraction method for the effective area of the model in drone photogrammetry according to any one of claims 1-4, characterized in that, When performing regional segmentation on the minimum bounding rectangle of the experimental area, it includes: Perform cluster analysis on the object-space elevation information of the experimental area, with the number of clusters being k; Define the height H of the experimental area as {h1, h2, …, h k}, where h k is the average elevation of each category; Define the initial ground height Z of the experimental area G = h1. According to the size of the image sensor and the focal length f, and based on the principle of similar triangles, calculate the projected quadrilateral area of the image set I = {i1, i2, …, i m} under vertical conditions, and calculate the average value S of the vertical projected area of the images; Define the minimum segmentation area and 0 < r < 6, or set the minimum segmentation area S according to the characteristics of the experimental area min ; Using the minimum segmentation area S min and the minimum number of connection points N min as constraints, the maximum circumscribed rectangle R of the experimental area MBR is segmented and a spatial index is constructed.

6. The method for automatically extracting the effective area of the model in drone photogrammetry according to claim 5, characterized in that, When the rectangular area or the number of connection points of the splitting node is less than the threshold, stop splitting the node, set the node that meets the constraints as the leaf node of the spatial index, and the maximum circumscribed rectangle R of the experimental area after splitting MBR ={R1, R2, …, R n}.

7. The automatic extraction method for the effective area of the model in drone photogrammetry according to claim 6, characterized in that, The method for calculating the influencing factors of the valid area of the UAV photogrammetry model is: Search for the coverage area of the UAV image projection quadrilateral according to the spatial index; Calculate the ratio of the intersection area between the leaf node rectangle of the coverage area and the projection quadrilateral to the rectangle area, and set it as the overlap degree of this node; Calculate the overlap degree coverage of each image in sequence, and count the influencing factors of each leaf node.

8. The method for automatically extracting the effective area of the model in drone photogrammetry according to claim 7, wherein, The method for constructing the quadrilateral of the UAV image projection is as follows: According to the pose information of image i i and the initial height h1, the object-space projection of the image corner points is obtained by using the collinearity equation, and the object-space projection quadrilateral Q i of the image is constructed by using the four corner points.

9. The method for automatically extracting the effective area of the model in drone photogrammetry according to claim 8, characterized in that, The method for extracting the final model valid area with multi-element constraints includes: Construct an importance function W based on the influence of the influencing factors of each leaf node on the effective region of the model i ; According to the constructed importance function W i Extract valid rectangles and obtain the union U of the valid rectangles; Perform regional fitting on polygon U to extract the final model valid area.

10. An automatic extraction system for the effective area of a drone photogrammetry model, characterized in that, The automatic extraction system for the valid area of the UAV photogrammetry model uses the method for automatically extracting the valid area of the UAV photogrammetry model described in any one of claims 1-9 to extract the valid area of the UAV photogrammetry model and optimize the model area in the 3D reconstruction process; the automatic extraction system for the valid area of the UAV photogrammetry model includes: A minimum bounding rectangle construction module, which is used to obtain the pose information and object-space elevation information of the UAV images and construct the minimum bounding rectangle of the experimental area; A rectangle segmentation module, which is used to perform regional segmentation and spatial index construction on the minimum bounding rectangle of the experimental area with multiple constraints; An influencing factor calculation module, which is used to search for the coverage area of the UAV images according to the spatial index and calculate the influencing factors of the valid area of the UAV photogrammetry model; and A valid area extraction module, which is used to extract the final model valid area with multi-element constraints according to the influencing factors.

Images