Image pair selection method, device, equipment and storage medium

Abstract

The present application provides an image pair selection method, device, equipment and storage medium, which relate to the field of surveying and mapping technology. The method includes: obtaining the aerial triangulation results of the image data set collected for the target scene; selecting the candidate image pair sets corresponding to the multiple three-dimensional space units from the image data set according to the three-dimensional connection points of the multiple three-dimensional space units in the target scene; screening the candidate image pair sets corresponding to the multiple three-dimensional space units to generate image pairs of the target scene. The image pair selection method, device, equipment and storage medium proposed in the present application can reduce the redundancy of image data and improve the efficiency of reconstructing the dense point cloud of the target scene.

Description

Technical Field

[0001] The present application relates to the field of surveying and mapping technology, and in particular to an image pair selection method, device, equipment and storage medium. Background Art

[0002] Dense image matching is an important means to obtain dense point clouds in three-dimensional reconstruction based on oblique photography, and is also a hot issue in photogrammetry and computer vision.

[0003] For the reconstruction of three-dimensional models of large-scale target scenes, in the existing technology, a large amount of image data is collected through an unmanned aerial vehicle oblique photography platform, and a dense point cloud is generated based on all the collected image data. However, since the collected image data is large in size and has a lot of redundancy, the efficiency of generating dense point clouds for the entire scene is low.

[0004] Therefore, the image pairs can be screened before dense image matching, and the dense point cloud of the target scene can be reconstructed with as few image pairs as possible. Summary of the invention

[0005] The purpose of the present invention is to provide an image pair selection method, device, equipment and storage medium to address the problems in the prior art of large amount of image data, large amount of redundancy and low efficiency in reconstructing dense point clouds of target scenes.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides an image pair selection method, the method comprising:

[0008] Obtain aerial triangulation results of image datasets collected for target scenes;

[0009] According to the three-dimensional connection points of the multiple three-dimensional space units in the target scene, respectively selecting candidate image pair sets corresponding to the multiple three-dimensional space units from the image data set;

[0010] The candidate image pair sets corresponding to the multiple three-dimensional space units are screened to generate image pairs of the target scene.

[0011] Optionally, before selecting, from the image data set, candidate image pair sets corresponding to the multiple three-dimensional space units in the target scene according to the three-dimensional connection points of the multiple three-dimensional space units, the method further includes:

[0012] According to the three-dimensional bounding box range of the target scene, the target scene is divided into the multiple three-dimensional space units using a preset spacing, wherein the preset spacing is a three-dimensional spacing determined in advance according to a preset ground object resolution of the target scene.

[0013] Optionally, screening the candidate image pair sets corresponding to the multiple three-dimensional space units to generate the image pairs of the target scene includes:

[0014] Calculate the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs;

[0015] According to different image pairs in the candidate image pair set corresponding to different three-dimensional space units, for the coverage of multiple three-dimensional space units in the same target scene, the image pair smoothing items of adjacent three-dimensional space units are calculated;

[0016] The Markov random field energy equation for the target scene is solved according to the image pair data items of the multiple three-dimensional space units and the image pair smoothing items of the adjacent three-dimensional space units, and the solution set when the energy equation is minimized is the image pair of the target scene.

[0017] Optionally, calculating the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs includes:

[0018] Calculating the weight ratio of each image pair in each three-dimensional space unit according to the weights of all the image pairs and the ratio of the maximum weights of all the image pairs;

[0019] The image pair data item of each three-dimensional space unit is calculated according to the preset maximum weight ratio and the difference between the weight ratios.

[0020] Optionally, the calculating, according to different image pairs in the candidate image pair set corresponding to different three-dimensional space units, image pair smoothing items of adjacent three-dimensional space units for coverage of multiple three-dimensional space units in the same target scene includes:

[0021] Calculating the overlapping coverage ratio of the different image pairs according to the coverage areas of the different image pairs for the multiple three-dimensional space units in the same target scene and the ratio of the total coverage areas of the different three-dimensional space units;

[0022] According to a preset maximum coverage ratio threshold and a difference between the overlapping coverage ratios, a smoothing item of the image pairs of the adjacent three-dimensional space units is calculated.

[0023] Optionally, before calculating the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs, the method further includes:

[0024] Calculate the angle weight of each group of image pairs at each three-dimensional connection point in each three-dimensional space unit according to the angle between each group of image pairs and each three-dimensional connection point;

[0025] Calculating the scale weight of each group of image pairs at each connection point according to the scale value of each connection point on each group of image pairs;

[0026] Calculate the target weight of each group of image pairs at each connection point according to the angle weight of each group of image pairs at each three-dimensional connection point and the scale weight of each group of image pairs at each connection point;

[0027] The weight of each group of image pairs is calculated according to the average value of the target weight of each group of image pairs at each connection point.

[0028] Optionally, before calculating the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs, the method further includes:

[0029] Calculating the average angle of each group of image pairs according to the angle of each group of image pairs at each three-dimensional connection point;

[0030] Calculating an average scale ratio of each group of image pairs according to the scale ratios of each three-dimensional connection point on each group of image pairs;

[0031] The image pairs whose average angle exceeds a preset angle range and / or whose average scale ratio is less than a preset scale ratio threshold are eliminated from the candidate image pair set corresponding to each three-dimensional space unit.

[0032] In a second aspect, an embodiment of the present application provides an image pair selection device, comprising:

[0033] An acquisition module, used to obtain aerial triangulation results of an image data set collected for a target scene;

[0034] A selection module, configured to select, from the image data set, candidate image pair sets corresponding to the multiple three-dimensional space units according to the three-dimensional connection points of the multiple three-dimensional space units in the target scene;

[0035] The screening module is used to screen the candidate image pair sets corresponding to the multiple three-dimensional space units to generate image pairs of the target scene.

[0036] In a third aspect, an embodiment of the present application provides a computer device, comprising: a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor executes the computer program to perform the image pair selection method described in any one of the first aspects above.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image pair selection method described in any one of the first aspects are executed.

[0038] Compared with the prior art, this application has the following beneficial effects:

[0039] An image pair selection method, apparatus, device and storage medium provided in the embodiments of the present application obtain aerial triangulation results of an image data set collected for a target scene, select candidate image pair sets corresponding to multiple three-dimensional space units from the image data set according to the three-dimensional connection points of multiple three-dimensional space units in the target scene, screen the candidate image pair sets corresponding to the multiple three-dimensional space units, generate image pairs of the target scene, and reconstruct a dense point cloud of the target scene. The embodiments of the present application divide the target scene into multiple three-dimensional space units, screen the candidate image pair sets corresponding to the multiple three-dimensional space units, reduce the image data set of the target scene that can reconstruct the dense point cloud, reduce the redundancy of image data, reconstruct the dense point cloud of the target scene with less image data, and can reduce the efficiency of reconstructing the dense point cloud of the target scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A schematic diagram of a flow chart of an image pair selection method provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of a process for obtaining multiple three-dimensional space units provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of a process for generating an image pair of a target scene provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of a process for calculating image pair data items of each three-dimensional space provided in an embodiment of the present application;

[0045] Figure 5 A schematic diagram of a process for calculating smoothness items of image pairs of adjacent three-dimensional space units provided in an embodiment of the present application;

[0046] Figure 6 A schematic diagram of a process for calculating the weight of each group of image pairs provided in an embodiment of the present application;

[0047] Figure 7 A schematic diagram of a process for eliminating image pairs provided in an embodiment of the present application;

[0048] Figure 8 A schematic diagram of an image pair selection device provided in an embodiment of the present application;

[0049] Fig. 9 A schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0051] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0052] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0053] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0054] In order to reduce the redundancy of image data and improve the efficiency of reconstructing the dense point cloud of the target scene, the technical solution of the present application provides an image pair selection method. The following is an explanation of an image pair selection method provided in an embodiment of the present application in combination with multiple specific examples. Figure 1 A flowchart of an image pair selection method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the image pair selection method includes:

[0055] S101, obtaining an aerial triangulation result of an image data set collected for a target scene.

[0056] In the embodiment of the present application, the aerial triangulation may adopt simulated aerial triangulation or analytical aerial triangulation, which is not specifically limited in the embodiment of the present application.

[0057] By solving the aerial triangulation of the image data set collected for the target scene, the position and attitude information of each collected image and the sparse three-dimensional connection points of the target scene can be obtained.

[0058] Among them, the image dataset contains all the three-dimensional connection points in the target scene, so each observation image contains several three-dimensional connection points, and there are overlapping three-dimensional connection points between multiple observation images.

[0059] S102 , selecting candidate image pair sets corresponding to the multiple three-dimensional space units from the image data set according to the three-dimensional connection points of the multiple three-dimensional space units in the target scene.

[0060] Since each observation image contains several three-dimensional connection points, and there are overlapping three-dimensional connection points between multiple observation images, each three-dimensional connection point corresponds to at least two observation images, wherein at least two observation images contain the three-dimensional connection point.

[0061] For example, the observed image picture1 contains three-dimensional connection points dot1, dot2, and dot3, and the observed image picture2 contains connection points dot2, dot3, and dot4. Then, the three-dimensional connection point dot1 corresponds to the observed image picture1, the three-dimensional connection point 2 corresponds to the observed images picture1 and picture2, the three-dimensional connection point 3 corresponds to the observed images picture1 and picture2, and the three-dimensional connection point 4 corresponds to the observed image picture2.

[0062] The target scene is an actual scene in three-dimensional space. According to the division boundaries, the target scene can be divided into multiple three-dimensional space units, and the three-dimensional connection points in the target scene are also divided as the target scene is divided, that is, all three-dimensional connection points in multiple three-dimensional space units constitute the three-dimensional connection points of the entire target scene, wherein each three-dimensional space unit is divided into at least one three-dimensional connection point.

[0063] Since each three-dimensional connection point corresponds to at least two observation images, each three-dimensional space unit corresponds to at least two observation images, that is, there is a candidate observation image set in each three-dimensional space unit, wherein the observation images in the candidate observation image set are not repeated.

[0064] For example, as described above, the observation images corresponding to the three-dimensional connection point 3 are picture1 and picture2, and the observation image corresponding to the three-dimensional connection point 4 is picture2. If the three-dimensional connection point 3 and the three-dimensional connection point 4 are in the same three-dimensional space unit, and the three-dimensional space unit only contains the three-dimensional connection point 3 and the three-dimensional connection point 4, then the candidate observation image set in the three-dimensional space unit is {picture1, picture2}, among which, when the observation image picture2 appears twice, it is only recorded once in the candidate observation image set.

[0065] According to the candidate observation image set existing in each three-dimensional space unit selected from the image data set, the candidate image pair set corresponding to each three-dimensional space unit can be calculated and generated. Among them, it can be calculated and generated by an exhaustive method, and other methods can also be adopted, which are not specifically limited in the embodiments of the present application.

[0066] The candidate image pair may be a combination of any two observation images in the candidate observation image set, wherein the order of combining any two observation images is not restricted.

[0067] If there is a three-dimensional space unit, which can be denoted as c, c has a candidate observation image set which can be denoted as I(c) = {I 1 , I 2 , ..., I k}, where k is the number of candidate observation images in the three-dimensional space unit c, then the candidate image pair corresponding to the three-dimensional space unit c can be recorded as Pair(c) = {I i , I j}(I i , I j ∈I(c)∩i≠j), where the candidate image pair {I i , I j} can also be written as

[0068] For example, if the three-dimensional space unit c has a candidate observation image set I(c)={I 1 , I 2 , I 3}, that is, there are three candidate observation images, then the corresponding candidate image pair set is {{I 1 , I 2},{I 1 , I3},{I 2 , I 3}},Right now Among them, {I 1 , I 2} can also be expressed as {I 2 , I 1},{I 1 , I 3} can also be expressed as {I 3 , I 1},{I 2 , I 3} can also be expressed as {I 3 , I 2}.

[0069] S103, screening the candidate image pair sets corresponding to the multiple three-dimensional space units to generate image pairs of the target scene.

[0070] Each three-dimensional space unit corresponds to a candidate image pair set. The redundancy of the candidate image pairs in each candidate image pair set can be reduced by screening the candidate image pairs in each candidate image pair set. After the screening is completed, the image pairs of the target scene are generated according to each candidate image pair set corresponding to each screened three-dimensional space unit, and then the dense point cloud of the target scene is reconstructed by screening and merging the image pairs of the target scene.

[0071] In summary, an image pair selection method provided in an embodiment of the present application obtains an aerial triangulation result of an image data set collected for a target scene, selects a plurality of candidate image pair sets corresponding to the three-dimensional space units in the image data set according to the three-dimensional connection points of the plurality of three-dimensional space units in the target scene, screens the candidate image pair sets corresponding to the plurality of three-dimensional space units, generates image pairs of the target scene, and reconstructs a dense point cloud of the target scene. The embodiment of the present application reduces the image data set of the reconstructable dense point cloud of the target scene by dividing the target scene into a plurality of three-dimensional space units and screens the candidate image pair sets corresponding to the plurality of three-dimensional space units, thereby reducing the redundancy of image data, and reconstructing the dense point cloud of the target scene with less image data, thereby reducing the efficiency of reconstructing the dense point cloud of the target scene.

[0072] In the above Figure 1 Based on the image pair selection method shown, the embodiment of the present application also provides a method for realizing multiple three-dimensional space units. The embodiment is explained in detail below with reference to the accompanying drawings. Figure 2 A schematic diagram of a process for obtaining multiple three-dimensional space units provided in an embodiment of the present application, such as Figure 2As shown, in the above S102, before selecting candidate image pair sets corresponding to multiple three-dimensional space units from the image data set according to the three-dimensional connection points of multiple three-dimensional space units in the target scene, the method further includes:

[0073] According to the range of the three-dimensional bounding box of the target scene, the target scene is divided into a plurality of three-dimensional space units using a preset spacing, wherein the preset spacing is a three-dimensional spacing determined in advance according to a preset ground object resolution of the target scene.

[0074] like Figure 2 As shown, assuming that the top view of the three-dimensional bounding box range of the target scene is a square, there are multiple three-dimensional connection points in the square, which can be understood as the drone is above the target scene when shooting the target scene. Therefore, the square can be a top view of the target scene, wherein each connection point in the top view of the target scene contains an elevation value in the actual scene. The three-dimensional bounding box range of the target scene can also be a circle, a parallelogram, etc., which is not specifically limited in the embodiments of the present application.

[0075] like Figure 2 As shown, the square corresponding to the three-dimensional bounding box range can be divided into multiple rectangles according to the preset spacing, wherein each rectangle is a three-dimensional space unit, that is, the multiple rectangles can also be called multiple three-dimensional space units. Figure 2 The division is performed by the method shown. Of course, other preset spacings and other division methods can also be used, such as division into multiple triangles, etc. The division method is not specifically limited in the embodiments of the present application.

[0076] The preset spacing can be determined according to the scene object resolution, for example, it can be 100 times the scene object resolution. The scene object resolution can be determined according to the image data set. If the observed image is determined, the observed image has a corresponding resolution. Of course, the preset spacing can also be determined by other parameters, and the preset spacing can also have other calculation methods, which are not specifically limited in the embodiments of the present application.

[0077] An embodiment of the present application provides a method for obtaining multiple three-dimensional space units. The target scene is divided into multiple three-dimensional space units by a preset spacing through the three-dimensional bounding box range of the target scene, and then a set of candidate image pairs corresponding to the multiple three-dimensional space units can be screened, thereby reducing the image data set of the reconstructible dense point cloud of the target scene and reducing the redundancy of the image data.

[0078] In the above Figure 1Based on the image pair selection method shown, the embodiment of the present application further provides a method for implementing the above S103 of screening the candidate image pair set corresponding to multiple three-dimensional space units to generate image pairs of the target scene. The embodiment is specifically explained in conjunction with the accompanying drawings as follows. Figure 3 A schematic diagram of a process for generating an image pair of a target scene provided in an embodiment of the present application. Figure 3 As shown, the above S103 includes:

[0079] S301, calculating the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs.

[0080] In order to provide reference information for screening candidate image pairs, all image pairs in the candidate image pair set corresponding to each three-dimensional space unit can be calculated to obtain the weights of all image pairs, where the larger the weight, the smaller the cost of generating a dense point cloud of the target scene.

[0081] After calculating all the image pairs in the candidate image pair set corresponding to each three-dimensional space unit and obtaining the weights, a group of image pairs with the largest weights can be selected by comparison, and the normalization of other groups of image pairs can be achieved through this group of image pairs.

[0082] According to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs, the image pair data item of each three-dimensional space unit can be calculated, and the image pair data item is used to characterize the image quality of the candidate image pair set corresponding to each three-dimensional space unit.

[0083] S302 , according to different image pairs in the candidate image pair set corresponding to different three-dimensional space units, for the coverage of multiple three-dimensional space units in the same target scene, calculate the image pair smoothness items of adjacent three-dimensional space units.

[0084] Each three-dimensional space unit corresponds to its own candidate image pair set. According to different image pairs in the candidate image pair sets corresponding to different three-dimensional space units, the image pair smoothness items of adjacent three-dimensional space units can be calculated by counting the coverage of the different image pairs to multiple three-dimensional space units.

[0085] Specifically, a group of image pairs are selected from the respective candidate image pair sets corresponding to adjacent three-dimensional space units to form two different groups of image pairs, each group of image pairs contains at least one three-dimensional connection point. Therefore, each group of image pairs can correspond to at least one three-dimensional space unit. In the embodiment of the present application, the coverage range of different image pairs for multiple three-dimensional space units in the same target scene can be understood as the three-dimensional space units corresponding to different image pairs.

[0086] The image pair smoothness term can be calculated and obtained through the coverage range, where the image pair smoothness term is used to characterize the coverage contribution of different image pairs to multiple three-dimensional space units in the same target scene. The closer the three-dimensional space units covered by different image pairs are, the lower the cost of generating a dense point cloud of the target scene.

[0087] S303, solving the Markov random field energy equation for the target scene according to the image pair data items of multiple three-dimensional space units and the image pair smoothing items of adjacent three-dimensional space units, and obtaining the image pairs of the target scene as the solution set when the energy equation is minimized.

[0088] In the embodiment of the present application, the image pair screening problem can be transformed into a Markov random field problem, and then the Markov random field solution is used to screen the image pairs in the embodiment of the present application.

[0089] According to the image pair data items of multiple three-dimensional space units and the image pair smoothing items of adjacent three-dimensional space units, the energy equation constructed according to the Markov random field in the embodiment of the present application can be obtained, as shown in formula (1):

[0090]

[0091] Among them, c i represents a 3D space unit in the target scene, C represents a set of 3D space units, Represents a three-dimensional space unit c i There is a set of image pairs in Represents image pair In three-dimensional space unit c i The image pair data item in Represents image pair The image pair data items in all three-dimensional space units in the target scene can also be understood as image pair quality.

[0092] Among them, c j Represents another 3D space unit in the target scene, Represents a three-dimensional space unit c i A set of image pairs in Represents a three-dimensional space unit c j A set of image pairs in Represents two sets of images and In adjacent three-dimensional space units c i and c j The image pair smoothing term in , Represents two sets of images and Smooths the image pairs among all adjacent 3D spatial cells.

[0093] Among them, E(p) represents the energy of the selected image pair. Therefore, by solving the minimum E(p), the minimum set of image pairs required to reconstruct the entire target scene can be obtained.

[0094] The energy equation in the above formula (1) can be solved by belief propagation, greedy method, etc. In the embodiment of the present application, there is no specific limitation on the solution method.

[0095] An embodiment of the present application provides a method for generating image pairs of a target scene. The Markov random field energy equation for the target scene is solved through image pair data items of multiple three-dimensional space units and image pair smoothing items of adjacent three-dimensional space units. The solution set when the energy equation is minimized is the image pairs of the target scene. The image pair selection problem is converted into a Markov random field problem, and then solved by a Markov random field solution method, so as to achieve the purpose of reconstructing a complete dense point cloud of the target scene using as few image pairs as possible, thereby improving the efficiency of reconstructing the dense point cloud of the target scene.

[0096] In the above Figure 3 Based on the method for generating an image pair of a target scene shown in the figure, the embodiment of the present application further provides a method for calculating the image pair data item of each three-dimensional space unit in the above S103. The embodiment is explained in detail below with reference to the accompanying drawings. Figure 4 A schematic diagram of a process for calculating image pair data items in each three-dimensional space provided in an embodiment of the present application. Figure 4 As shown, in the above S301, according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs, the image pair data item of each three-dimensional space unit is calculated, including:

[0097] S401, calculating the weight ratio of each image pair in each three-dimensional space unit according to the weights of all image pairs and the ratio of the maximum weights of all image pairs.

[0098] The weight ratio of each image pair in each three-dimensional space unit can be expressed as in, Represents a three-dimensional space unit c i There is a set of image pairs in Represents image pair The weight, W max (c i ) represents the three-dimensional space unit c i The maximum value of all image pair weights in W max (c i) can be expressed as formula (2):

[0099]

[0100] S402, calculating the image pair data item of each three-dimensional space unit according to the preset maximum weight ratio and the difference between the weight ratios.

[0101] In the embodiment of the present application, the preset maximum weight ratio may be 1.0, and the image pair data item of each three-dimensional space unit may be expressed as formula (3):

[0102]

[0103] An embodiment of the present application provides a method for calculating image pair data items for each three-dimensional space. The image pair data items for each three-dimensional space unit are calculated based on a preset maximum weight ratio and a difference between the weight ratios, so as to characterize the image quality of a set of candidate image pairs corresponding to each three-dimensional space unit.

[0104] In the above Figure 3 Based on the method for generating an image pair of a target scene shown in the figure, the embodiment of the present application further provides a method for implementing the calculation of the smoothness term of the image pair of adjacent three-dimensional space units in the above S302. The embodiment is explained in detail below with reference to the accompanying drawings. Figure 5 A schematic diagram of a process for calculating the smoothness term of an image pair of adjacent three-dimensional space units provided in an embodiment of the present application. Figure 5 As shown, in the above S302, according to different image pairs in the candidate image pair set corresponding to different three-dimensional space units, for the coverage range of multiple three-dimensional space units in the same target scene, the image pair smoothing items of adjacent three-dimensional space units are calculated, including:

[0105] S501, calculating the overlapping coverage ratio of different image pairs according to the ratio of the same coverage area of ​​different image pairs for the three-dimensional space unit in the same target scene and the total coverage area of ​​different three-dimensional space units.

[0106] The same coverage area of ​​the three-dimensional space unit in the same target scene can be expressed as in, Represents image pair The collection of visible three-dimensional space units can be understood as the image pair There are multiple connection points in the image, and the multiple connection points are in at least one three-dimensional space unit, then the image pair Can correspond to at least one three-dimensional space unit, that is, image pair The collection of visible three-dimensional space cells. Represents image pair The set of visible three-dimensional space units, then Represents image pair and A collection of commonly visible three-dimensional space units, that is, the same coverage area of ​​three-dimensional space units.

[0107] The total coverage area of ​​different three-dimensional space units can be expressed as It can be understood that the image or The collection of visible three-dimensional space cells.

[0108] Therefore, the overlap ratio of different image pairs can be expressed as formula (4):

[0109]

[0110] S502, calculating the smoothing items of the image pairs of adjacent three-dimensional space units according to a preset maximum coverage ratio threshold and a difference between overlapping coverage ratios.

[0111] In the embodiment of the present application, the preset maximum coverage ratio threshold may be 1.0, and the image pair smoothing term of adjacent three-dimensional space units may be expressed as formula (5):

[0112]

[0113] An embodiment of the present application provides a method for calculating smoothness items of image pairs of adjacent three-dimensional space units. According to the same coverage area of ​​three-dimensional space units in the same target scene by different image pairs, the total coverage area of ​​different three-dimensional space units, and a preset maximum coverage ratio threshold, the smoothness items of image pairs of adjacent three-dimensional spaces are calculated to characterize the coverage contributions of different image pairs to multiple three-dimensional space units in the same target scene.

[0114] In the above Figure 3 Based on the method for generating an image pair of a target scene shown in the figure, the embodiment of the present application further provides a method for calculating the weight of each group of image pairs. The embodiment is explained in detail below in conjunction with the accompanying drawings. Figure 6 A schematic diagram of a process for calculating the weight of each image pair provided in an embodiment of the present application. Figure 6 As shown, before the above S301, according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs, before calculating the image pair data item of each three-dimensional space unit, the method further includes:

[0115] S601, calculating the angle weight of each image pair at each three-dimensional connection point according to the angle between each image pair and each three-dimensional connection point in each three-dimensional space unit.

[0116] The angle between each three-dimensional connection point in each three-dimensional space unit of each image pair can be obtained by calculating the intersection angle, and the intersection angle formed by the three-dimensional connection point and the photographic center of the two observation images in the image pair can be calculated. Among them, the three-dimensional connection point has three-dimensional coordinates, and the three-dimensional coordinates of the photographic center of the two observation images are determined as the observation images are taken. According to the three three-dimensional coordinates, the intersection angle formed by the three-dimensional connection point and the image pair can be calculated.

[0117] The angle weight of a set of image pairs at a three-dimensional connection point can be expressed as formula (6):

[0118] w angle =min(1,(angle / Angle min ) 1.5 ) Formula (6)

[0119] Where angle represents a three-dimensional connection point p and two observation images in the image pair (I i with I j ), Anglemin represents a preset minimum intersection angle threshold, which can be set to 12° in the embodiment of the present application, for example, and w angle Indicates that the set of images The angle weight at this 3D connection point.

[0120] S602: Calculate the scale weight of each image pair at each connection point according to the scale value of each three-dimensional connection point on each image pair.

[0121] A set of images The scale weight at a three-dimensional connection point p can be expressed as formula (7):

[0122]

[0123] Among them, s i and j Indicates that the 3D connection point p is in the image pair The scale value on the two images, w scale Represents a set of image pairs Scaling weight at a 3D connection point p.

[0124] The scale value of the three-dimensional connection point on the observation image can be obtained by dividing the distance between the three-dimensional connection point and the photographic center of the observation image by the focal length of the observation image, wherein the focal length of the observation image can be determined when the observation image is captured.

[0125] S603, calculating the target weight of each image pair at each connection point according to the angle weight of each image pair at each three-dimensional connection point and the scale weight of each image pair at each connection point.

[0126] The target weight of the image pair at the 3D connection point can be obtained through the angle weight and scale weight. The target weight of the image pair at the 3D connection point is expressed as formula (8):

[0127]

[0128] Among them, w angle Represents image pair The angle weight at the three-dimensional connection point p, Represents image pair The target weight at the 3D connection point p.

[0129] S604, calculating the weight of each group of image pairs according to the average value of the target weight of each group of image pairs at each connection point.

[0130] In each three-dimensional space unit, after calculating the target weight of each image pair at each three-dimensional connection point, the multiple target weights corresponding to each image pair are averaged to obtain the weight of each image pair.

[0131] An embodiment of the present application provides a method for calculating the weight of each group of image pairs. According to the angle weight and scale weight of each group of image pairs at each three-dimensional connection point, a target weight is calculated, and then the weight of each group of image pairs is calculated, which is used to calculate the image pair data item of each three-dimensional space unit.

[0132] In the above Figure 3 Based on the method for generating an image pair of a target scene shown in the figure, the embodiment of the present application further provides a method for eliminating an image pair. The embodiment is explained in detail below in conjunction with the accompanying drawings. Figure 7 A schematic diagram of a process for eliminating image pairs provided in an embodiment of the present application. Figure 7 As shown, before the above S301, according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs, before calculating the image pair data item of each three-dimensional space unit, the method further includes:

[0133] S701, calculating the average angle of each group of image pairs according to the angle of each group of image pairs at each three-dimensional connection point.

[0134] As described in S601, the angle between each image pair at each 3D connection point can be represented by an intersection angle. The intersection angles between each image pair at each 3D connection point are averaged to obtain the average intersection angle of each image pair, that is, the average angle.

[0135] S702: Calculate the average scale ratio of each group of image pairs according to the scale ratios of the three-dimensional connection points on each group of image pairs.

[0136] The scale ratio of each 3D tie point in each image pair is obtained by dividing the scale value of one observation image of each 3D tie point in a group of image pairs by the scale value of another observation image, and then dividing the scale value of one observation image of each 3D tie point in a group of image pairs by the scale value of another observation image, to obtain another scale ratio. By comparison, the smallest scale ratio is selected from the two scale ratios as the scale ratio of each 3D tie point in each image pair, and then the average scale ratio of each image pair is obtained.

[0137] S703: Eliminate image pairs whose average angle exceeds a preset angle range and / or whose average scale ratio is less than a preset scale ratio threshold from the candidate image pair set corresponding to each three-dimensional space unit.

[0138] In an embodiment of the present application, the preset angle range can be 3° to 45°, and the preset scale ratio threshold can be 0.75. If the average angle of each group of image pairs is not within the preset angle range, it means that the baseline of the image pair is too short or too long, and the image pair is invalid and should be eliminated. If the average scale ratio of each group of image pairs is less than the preset scale ratio threshold, it means that the scale difference between the two observed images of the image pair is too large, and the image pair is invalid and should be eliminated.

[0139] An embodiment of the present application provides a method for eliminating image pairs, which eliminates invalid image pairs and retains valid image pairs by comparing the average angle and the average scale ratio with a preset angle range and a preset scale ratio threshold. This can reduce the redundancy of image data, reconstruct a dense point cloud of the target scene with less image data, and improve the efficiency of reconstructing dense point clouds.

[0140] The following is a description of an image pair selection device and a computer device provided by the present application for execution. The specific implementation process and technical effects are described above and will not be repeated below.

[0141] Figure 8 A schematic diagram of an image pair selection device provided in an embodiment of the present application, such as Figure 8 As shown, the image contrast includes:

[0142] The acquisition module 801 is used to acquire the aerial triangulation result of the image data set collected for the target scene.

[0143] The selection module 802 is used to select candidate image pair sets corresponding to the multiple three-dimensional space units in the target scene from the image data set according to the three-dimensional connection points of the multiple three-dimensional space units.

[0144] The screening module 803 is used to screen the candidate image pair sets corresponding to the multiple three-dimensional space units to generate image pairs of the target scene to reconstruct the target scene.

[0145] Optionally, the selection module 802 is specifically used to divide the target scene into multiple three-dimensional space units according to the three-dimensional bounding box range of the target scene and using a preset spacing, wherein the preset spacing is a three-dimensional spacing determined in advance according to a preset object resolution of the target scene.

[0146] Optionally, the screening module 803 is specifically used to calculate the image pair data items of each three-dimensional space unit based on the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs; calculate the image pair smoothing items of adjacent three-dimensional space units for the coverage range of multiple three-dimensional space units in the same target scene based on different image pairs in the candidate image pair set corresponding to different three-dimensional space units; solve the Markov random field energy equation for the target scene based on the image pair data items of multiple three-dimensional space units and the image pair smoothing items of adjacent three-dimensional space units, and obtain the image pairs of the target scene as the solution set when the energy equation is minimized.

[0147] Optionally, the screening module 803 is specifically used to calculate the weight ratio of each image pair in each three-dimensional space unit based on the weights of all image pairs and the ratio of the maximum weights of all image pairs; and calculate the image pair data item of each three-dimensional space unit based on the preset maximum weight ratio and the difference between the weight ratios.

[0148] Optionally, the screening module 803 is specifically used to calculate the overlapping coverage ratio of different image pairs based on the same coverage area of ​​the three-dimensional space units in the same target scene by different image pairs and the ratio of the total coverage areas of different three-dimensional space units; and calculate the smoothing item of the image pairs of adjacent three-dimensional space units based on a preset maximum coverage ratio threshold and the difference between the overlapping coverage ratios.

[0149] Optionally, the screening module 803 is specifically used to calculate the angle weight of each image pair at each three-dimensional connection point according to the angle of each image pair to each three-dimensional connection point in each three-dimensional space unit; calculate the scale weight of each image pair at each connection point according to the scale value of each three-dimensional connection point on each image pair; calculate the target weight of each image pair at each connection point according to the angle weight of each image pair at each three-dimensional connection point and the scale weight of each image pair at each connection point; calculate the weight of each image pair according to the average value of the target weight of each image pair at each connection point.

[0150] Optionally, the screening module 803 is specifically used to calculate the average angle of each group of image pairs based on the angle of each group of image pairs at each three-dimensional connection point; calculate the average scale ratio of each group of image pairs based on the scale ratio of each three-dimensional connection point on each group of image pairs; and eliminate image pairs whose average angle exceeds a preset angle range and / or whose average scale ratio is less than a preset scale ratio threshold in the set of candidate image pairs corresponding to each three-dimensional spatial unit.

[0151] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program codes. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0152] Fig. 9 A schematic diagram of a computer device provided in an embodiment of the present application, which may be a computing device with computing processing capabilities.

[0153] The computer device includes: a processor 901 , a storage medium 902 , and a bus 903 . The processor 901 and the storage medium 902 are connected via the bus 903 .

[0154] The storage medium 902 is used to store programs, and the processor 901 calls the programs stored in the storage medium 902 to execute the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0155] Optionally, the present invention also provides a program product, such as a computer-readable storage medium, comprising a program, which is used to execute the above method embodiment when executed by a processor.

[0156] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0159] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0160] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for selecting an image pair, characterized in that: include: Obtain aerial triangulation results of image datasets collected for target scenes; According to the three-dimensional connection points of the multiple three-dimensional space units in the target scene, respectively selecting candidate image pair sets corresponding to the multiple three-dimensional space units from the image data set; Screening a set of candidate image pairs corresponding to the multiple three-dimensional space units to generate image pairs of the target scene; The step of screening the candidate image pair sets corresponding to the plurality of three-dimensional space units to generate the image pairs of the target scene includes: Calculate the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs; According to different image pairs in the candidate image pair set corresponding to different three-dimensional space units, for the coverage of multiple three-dimensional space units in the same target scene, the image pair smoothing items of adjacent three-dimensional space units are calculated; The Markov random field energy equation for the target scene is solved according to the image pair data items of the multiple three-dimensional space units and the image pair smoothing items of the adjacent three-dimensional space units, and the solution set when the energy equation is minimized is the image pair of the target scene.

2. The method according to claim 1, characterized in that: Before selecting candidate image pair sets corresponding to the multiple three-dimensional space units from the image data set according to the three-dimensional connection points of the multiple three-dimensional space units in the target scene, the method further includes: According to the three-dimensional bounding box range of the target scene, the target scene is divided into the multiple three-dimensional space units using a preset spacing, wherein the preset spacing is a three-dimensional spacing determined in advance according to a preset ground object resolution of the target scene.

3. The method according to claim 1, characterized in that The step of calculating the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs includes: Calculating the weight ratio of each image pair in each three-dimensional space unit according to the weights of all the image pairs and the ratio of the maximum weights of all the image pairs; The image pair data item of each three-dimensional space unit is calculated according to the preset maximum weight ratio and the difference between the weight ratios.

4. The method according to claim 1, characterized in that The method of calculating the image pair smoothness items of adjacent three-dimensional space units according to different image pairs in the candidate image pair set corresponding to different three-dimensional space units and the coverage range of multiple three-dimensional space units in the same target scene includes: Calculating the overlapping coverage ratio of the different image pairs according to the ratio of the same coverage area of ​​the three-dimensional space units in the same target scene by the different image pairs and the total coverage area of ​​the different three-dimensional space units; According to a preset maximum coverage ratio threshold and a difference between the overlapping coverage ratios, a smoothing item of the image pairs of the adjacent three-dimensional space units is calculated.

5. The method according to claim 1, characterized in that Before calculating the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs, the method further includes: Calculate the angle weight of each group of image pairs at each three-dimensional connection point in each three-dimensional space unit according to the angle between each group of image pairs and each three-dimensional connection point; Calculating the scale weight of each group of image pairs at each connection point according to the scale value of each three-dimensional connection point on each group of image pairs; Calculate the target weight of each group of image pairs at each connection point according to the angle weight of each group of image pairs at each three-dimensional connection point and the scale weight of each group of image pairs at each connection point; The weight of each group of image pairs is calculated according to the average value of the target weight of each group of image pairs at each connection point.

6. The method according to claim 5, characterized in that Before calculating the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs, the method further includes: Calculating the average angle of each group of image pairs according to the angle of each group of image pairs at each three-dimensional connection point; Calculating an average scale ratio of each group of image pairs according to the scale ratios of the three-dimensional connection points on each group of image pairs; The image pairs whose average angle exceeds a preset angle range and / or whose average scale ratio is less than a preset scale ratio threshold are eliminated from the candidate image pair set corresponding to each three-dimensional space unit.

7. An image pair selection device, characterized in that: include: An acquisition module, used to obtain aerial triangulation results of an image data set collected for a target scene; A selection module, configured to select, from the image data set, candidate image pair sets corresponding to the multiple three-dimensional space units according to the three-dimensional connection points of the multiple three-dimensional space units in the target scene; A screening module, used to screen the candidate image pair sets corresponding to the multiple three-dimensional space units to generate image pairs of the target scene; The screening module is specifically used for: Calculate the image pair data item of each three-dimensional space unit according to the weights of all image pairs in the candidate image pair set corresponding to each three-dimensional space unit and the maximum weight of all image pairs; According to different image pairs in the candidate image pair set corresponding to different three-dimensional space units, for the coverage of multiple three-dimensional space units in the same target scene, the image pair smoothing items of adjacent three-dimensional space units are calculated; The Markov random field energy equation for the target scene is solved according to the image pair data items of the multiple three-dimensional space units and the image pair smoothing items of the adjacent three-dimensional space units, and the solution set when the energy equation is minimized is the image pair of the target scene.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the image pair selection method described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is read and executed, the image pair selection method described in any one of claims 1 to 6 is implemented.

Images