Disordered image pose recovery method and device
By combining normalized segmentation of image similarity graphs and global pose recovery algorithms, the problem of excessive memory usage in large-scale image processing by traditional image pose recovery algorithms is solved, and efficient ultra-large-scale disordered image pose recovery is achieved.
Patent Information
- Application Number
- CN202510171744.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-17
- Publication Date
- 2025-05-16
AI Technical Summary
When traditional image pose recovery algorithms deal with large-scale disordered images, the memory usage is too high and cannot effectively deal with the problem of ultra-large-scale image pose recovery.
By normalizing the image similarity graph, it is divided into multiple sub-blocks, and the number of images in each sub-block is reduced. The image pose is restored by block by block, and the pose is merged through the image overlap relationship between sub-blocks, and finally the pose is optimized by global beam adjustment.
It reduces memory usage, realizes efficient processing of ultra-large-scale disordered image pose recovery, and solves the processing difficulties of traditional algorithms under limited memory.
Smart Images
Figure CN120014030A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of three-dimensional vision algorithms, and in particular to a method and device for recovering disordered image posture. Background Art
[0002] Image pose recovery is a basic algorithm in three-dimensional vision algorithms, and it is very important in many applications such as image three-dimensional reconstruction and image visual relocation. Traditional image pose recovery algorithms such as SfM (Structure from Motion) have been widely verified in practice and have evolved into many algorithm variants such as incremental SfM (Incremental Structure from Motion) and global SfM (Global Structure from Motion), but these algorithms have a high overall memory usage and cannot handle large-scale image pose recovery problems when computer memory is limited. With the rapid development of smart phones and drones, the cost of data acquisition is getting lower and lower, and the amount of data for many image-based three-dimensional vision tasks is increasing rapidly, with the number of images reaching hundreds of thousands or even millions. This patent proposes a new method for ultra-large-scale disordered image pose recovery. First, through the normalized segmentation (Normalized Cut) of the image similarity graph, large-scale data is automatically grouped, and the partial overlapping relationship of each sub-block is retained during the grouping process. The number of images in each sub-block is much smaller than the total number of images. Secondly, the image pose is restored block by block through the traditional global pose recovery algorithm to obtain the image pose in each sub-block. Third, through the image overlap relationship between sub-blocks, each sub-block is merged to obtain the initial value of all image poses. Finally, the global bundle adjustment algorithm is used to optimize all image poses. Since relatively accurate image poses have been obtained in each sub-block, the global bundle adjustment mainly solves the pose consistency problem of images in the overlapping area. Therefore, the number of equations in the global bundle adjustment can be greatly reduced, ultimately achieving the purpose of reducing memory usage and realizing large-scale disordered image pose recovery.
[0003] Therefore, in order to meet actual needs, a disordered image pose recovery technology is provided. Summary of the invention
[0004] In view of the defects existing in the prior art, the purpose of this application is to provide a method and device for restoring the posture of disordered images, which organizes and segments the disordered effects based on similarity, calculates and merges the image postures separately, ensures efficient image processing efficiency while reducing memory usage, and meets actual needs.
[0005] In order to achieve the above objectives, the technical solution adopted by this application is:
[0006] In a first aspect, the present application provides a method for restoring the posture of an unordered image, the method comprising the following steps:
[0007] Arrange multiple disordered images according to similarity to obtain a similarity graph arranged based on similarity;
[0008] Based on a normalized graph cut algorithm, segmenting the similarity graph into at least two image groups;
[0009] Based on the image pose recovery algorithm, the image pose corresponding to each of the image groups is calculated;
[0010] The image poses corresponding to each of the image groups are combined to obtain a final image pose.
[0011] On the basis of the above technical solution, each vertex in the similarity graph corresponds to one of the disordered images;
[0012] Two adjacent vertices in the similarity graph constitute an edge in the similarity graph, and an edge weight is configured based on the similarity between the disordered images corresponding to the two vertices.
[0013] On the basis of the above technical solution, the number of disordered images corresponding to each of the image groups is the same and is less than the total number of disordered effects;
[0014] The disordered image compositions between the image groups overlap.
[0015] On the basis of the above technical solution, the similarity graph is divided into at least two image groups based on the normalized graph cut algorithm, including the following steps:
[0016] Based on a normalized graph cut algorithm, segmenting the similarity graph into at least two initial image groups;
[0017] Based on each of the initial image groups, based on the similarity, the corresponding image group is integrated to obtain; wherein,
[0018] One of the initial image groups corresponds to one of the image groups;
[0019] There is no overlapping relationship between the disordered image components of each of the initial image groups
[0020] The disordered image compositions between the image groups overlap.
[0021] On the basis of the above technical solution, the method further comprises the following steps:
[0022] Based on the image pose recovery algorithm, parallel calculations are performed on each of the image groups to obtain the image pose corresponding to each of the image groups.
[0023] In a second aspect, the present application provides a disordered image posture recovery device, the device comprising:
[0024] A similarity map acquisition module is used to sort multiple disordered images according to similarity to obtain a similarity map arranged based on similarity;
[0025] An image segmentation module, used for segmenting the similarity graph into at least two image groups based on a normalized graph cut algorithm;
[0026] An image posture calculation module, which is used to calculate and obtain the image posture corresponding to each of the image groups based on an image posture recovery algorithm;
[0027] The image pose merging module is used to merge the image poses corresponding to each of the image groups to obtain a final image pose.
[0028] On the basis of the above technical solution, each vertex in the similarity graph corresponds to one of the disordered images;
[0029] Two adjacent vertices in the similarity graph constitute an edge in the similarity graph, and an edge weight is configured based on the similarity between the disordered images corresponding to the two vertices.
[0030] On the basis of the above technical solution, the number of disordered images corresponding to each of the image groups is the same and is less than the total number of disordered effects;
[0031] The disordered image compositions between the image groups overlap.
[0032] On the basis of the above technical solution, the image segmentation module is further used to segment the similarity graph into at least two initial image groups based on a normalized graph cut algorithm;
[0033] The image segmentation module is also used to integrate and obtain the corresponding image groups based on the similarity of each of the initial image groups; wherein,
[0034] One of the initial image groups corresponds to one of the image groups;
[0035] There is no overlapping relationship between the disordered image components of each of the initial image groups
[0036] The disordered image compositions between the image groups overlap.
[0037] On the basis of the above technical solution, the image pose calculation module is also used to perform parallel calculations on each of the image groups based on an image pose recovery algorithm to obtain the image pose corresponding to each of the image groups.
[0038] Compared with the prior art, the advantages of this application are:
[0039] This application organizes and segments disordered effects based on similarity, calculates and merges image poses separately, ensures efficient image processing efficiency while reducing memory usage, and meets actual needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0041] Figure 1 A flowchart of the steps of the method for restoring the disordered image posture according to an embodiment of the present application;
[0042] Figure 2 This is a principle flow chart of the method for restoring the disordered image posture according to an embodiment of the present application;
[0043] Figure 3 A schematic diagram of the steps of constructing an image similarity graph in the method for restoring the disordered image pose in an embodiment of the present application;
[0044] Figure 4 A schematic diagram of the steps of normalizing graph segmentation in the method for recovering the disordered image pose in an embodiment of the present application;
[0045] Figure 5 It is a schematic diagram of the steps of SfM recovery of image pose within a block in the disordered image pose recovery method of an embodiment of the present application;
[0046] Figure 6 A schematic diagram of the steps of block merging in the method for recovering the posture of disordered images in an embodiment of the present application;
[0047] Figure 7 This is a structural block diagram of a disordered image posture recovery device according to an embodiment of the present application. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0049] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.
[0050] The embodiments of the present application provide a method and device for restoring the posture of disordered images, which organize and segment disordered effects based on similarity, calculate and merge image postures separately, and ensure efficient image processing efficiency while reducing memory usage to meet actual needs.
[0051] In order to achieve the above technical effects, the overall idea of this application is as follows:
[0052] A method for restoring disordered image posture, the method comprising the following steps:
[0053] S1, sorting multiple disordered images according to similarity to obtain a similarity graph arranged based on similarity;
[0054] S2, based on the normalized graph cut algorithm, dividing the similarity graph into at least two image groups;
[0055] S3, based on the image posture recovery algorithm, calculate and obtain the image posture corresponding to each image group;
[0056] S4. Merge the image poses corresponding to each image group to obtain the final image pose.
[0057] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.
[0058] First, see Figures 1 to 6 As shown, the embodiment of the present application provides a method for restoring the posture of disordered images, and the method comprises the following steps:
[0059] S1, sorting multiple disordered images according to similarity to obtain a similarity graph arranged based on similarity;
[0060] S2, based on the normalized graph cut algorithm, dividing the similarity graph into at least two image groups;
[0061] S3, based on the image posture recovery algorithm, calculate and obtain the image posture corresponding to each image group;
[0062] S4. Merge the image poses corresponding to each image group to obtain the final image pose.
[0063] It should be noted that the memory usage of traditional image pose recovery algorithms during operation is positively correlated with the number of images. When processing ultra-large-scale image data sets, they often fail due to insufficient memory.
[0064] The technical solution of the embodiment of the present application proposes a "grouping-merging" motion recovery structure method, which is mainly used to solve the problem of insufficient memory in ultra-large-scale image pose recovery. The method first groups the images, and then divides the image similarity graph to achieve image grouping and clustering, so that the number of images contained in each group is small, and the image pose can be obtained through the traditional global motion recovery structure method;
[0065] In the process of grouping, it is ensured that each group has a part of common images for subsequent group merging. The main purpose of "grouping" is to enable the algorithm to obtain the initial values of all image poses within a limited memory space;
[0066] The second is block merging. The blocks can be roughly merged through the common images between groups. However, due to the initial pose values calculated in each sub-block and the calculation errors in each sub-block, the pose will be discontinuous, that is, there is a "stitching gap" between blocks. All poses need to be re-optimized through global bundle adjustment. During the optimization process, the focus is on overlapping images. Since the overall proportion of overlapping images is low, the global bundle adjustment can achieve higher efficiency and lower memory usage as a whole, thereby solving the high memory usage problem of traditional motion recovery structure algorithm.
[0067] In the embodiment of the present application, disordered effects are sorted and segmented based on similarity, and image poses are calculated and merged separately, thereby ensuring efficient image processing efficiency while reducing memory usage and meeting actual needs.
[0068] Furthermore, each vertex in the similarity graph corresponds to one of the disordered images;
[0069] Two adjacent vertices in the similarity graph constitute an edge in the similarity graph, and an edge weight is configured based on the similarity between the disordered images corresponding to the two vertices.
[0070] Furthermore, the number of disordered images corresponding to each of the image groups is the same and is less than the total number of disordered effects;
[0071] The disordered image compositions between the image groups overlap.
[0072] Furthermore, the similarity graph is segmented into at least two image groups based on a normalized graph cut algorithm, comprising the following steps:
[0073] Based on a normalized graph cut algorithm, segmenting the similarity graph into at least two initial image groups;
[0074] Based on each of the initial image groups, based on the similarity, the corresponding image group is integrated to obtain; wherein,
[0075] One of the initial image groups corresponds to one of the image groups;
[0076] There is no overlapping relationship between the disordered image components of each of the initial image groups
[0077] The disordered image compositions between the image groups overlap.
[0078] Furthermore, the method further comprises the following steps:
[0079] Based on the image pose recovery algorithm, parallel calculations are performed on each of the image groups to obtain the image pose corresponding to each of the image groups.
[0080] Based on the technical solution of the embodiment of the present application, the specific situation is as follows:
[0081] The first step is to construct the image similarity graph:
[0082] For large-scale disordered images, the similarity between images can be calculated through the image retrieval algorithm. The images are used as vertices in the graph. Two images with a certain similarity can form an edge in the graph, and the similarity can be used as the edge weight. Figure 3 As shown, each dot represents an image (8 images in total), and a similarity graph as shown on the right can be constructed by similarity calculation.
[0083] The second step is to normalize the graph and divide it into blocks:
[0084] After constructing the image similarity graph, the normalized graph cut algorithm can be used to evenly segment the block to obtain several sub-blocks, each of which contains less images than the total number of images. At the same time, it should be noted that when segmenting, a certain overlap relationship should be ensured between the blocks to facilitate subsequent block merging. Figure 4 As shown, the image can be divided into two groups (colored green and red respectively) by normalized graph cut, and considering the overlapping relationship of the blocks, the image can be further divided into two groups (1,2,3,5,6,7) and (2,3,4,6,7,8).
[0085] In the third step, SfM recovers the image pose within the block:
[0086] Through the traditional structure from motion algorithm, the image pose within the block can be restored. After the normalized graph cut algorithm, the large data set is cut into several small data sets with overlap. Each small data set can independently calculate the pose through the SfM algorithm. Multiple blocks can be calculated in parallel, which improves the efficiency of the algorithm to a certain extent. Figure 5As shown, the unordered image of each block can obtain its position and posture at the time of shooting through the SfM algorithm (the triangle around the image in the figure represents the position and posture of the acquired image, and different positions and orientations represent different image postures).
[0087] Step 4: Block merging:
[0088] As shown in the attached figure of the specification Figure 6 As shown in the figure, after obtaining the image pose block by block through SfM, each sub-block needs to be merged to finally obtain the pose of all images. Due to certain errors in the calculation within the block, the image pose of the overlapping area is different in each block, and the merged pose needs to be optimized through the global bundle adjustment. The pose of the image in the non-overlapping area is unique and there is no difference between each block. Therefore, it can be fixed or constrained during the optimization process, thereby reducing the optimization variables and achieving the purpose of saving memory usage.
[0089] Second, see Figure 7 As shown, an embodiment of the present application provides a disordered image posture recovery device, the device comprising:
[0090] A similarity map acquisition module is used to sort multiple disordered images according to similarity to obtain a similarity map arranged based on similarity;
[0091] An image segmentation module, used for segmenting the similarity graph into at least two image groups based on a normalized graph cut algorithm;
[0092] An image posture calculation module, which is used to calculate and obtain the image posture corresponding to each of the image groups based on an image posture recovery algorithm;
[0093] The image pose merging module is used to merge the image poses corresponding to each of the image groups to obtain a final image pose.
[0094] In the embodiment of the present application, disordered effects are sorted and segmented based on similarity, and image poses are calculated and merged separately, thereby ensuring efficient image processing efficiency while reducing memory usage and meeting actual needs.
[0095] Furthermore, each vertex in the similarity graph corresponds to one of the disordered images;
[0096] Two adjacent vertices in the similarity graph constitute an edge in the similarity graph, and an edge weight is configured based on the similarity between the disordered images corresponding to the two vertices.
[0097] Furthermore, the number of disordered images corresponding to each of the image groups is the same and is less than the total number of disordered effects;
[0098] The disordered image compositions between the image groups overlap.
[0099] Furthermore, the image segmentation module is further used to segment the similarity graph into at least two initial image groups based on a normalized graph cut algorithm;
[0100] The image segmentation module is also used to integrate and obtain the corresponding image groups based on the similarity of each of the initial image groups; wherein,
[0101] One of the initial image groups corresponds to one of the image groups;
[0102] There is no overlapping relationship between the disordered image components of each of the initial image groups
[0103] The disordered image compositions between the image groups overlap.
[0104] Furthermore, the image pose calculation module is also used to perform parallel calculations on each of the image groups based on an image pose recovery algorithm to obtain the image pose corresponding to each of the image groups.
[0105] It should be noted that the technical problems, technical solutions and technical effects of the disordered image posture restoration device mentioned in the embodiment of the present application are similar in technical principles to those of the disordered image posture restoration method mentioned in the first aspect, so they will not be elaborated here.
[0106] In the description of the present application, it should be noted that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.
[0107] It should be noted that, in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0108] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A method for restoring the posture of disordered images, characterized in that: The method comprises the following steps: Arrange multiple disordered images according to similarity to obtain a similarity graph arranged based on similarity; Based on a normalized graph cut algorithm, segmenting the similarity graph into at least two image groups; Based on the image pose recovery algorithm, the image pose corresponding to each of the image groups is calculated; The image poses corresponding to each of the image groups are combined to obtain a final image pose.
2. The method for restoring the disordered image posture according to claim 1, characterized in that: Each vertex in the similarity graph corresponds to one of the disordered images; Two adjacent vertices in the similarity graph constitute an edge in the similarity graph, and an edge weight is configured based on the similarity between the disordered images corresponding to the two vertices.
3. The method for restoring the disordered image posture according to claim 1, characterized in that: The number of the disordered images corresponding to each of the image groups is the same and is less than the total number of disordered effects; The disordered image compositions between the image groups overlap.
4. The method for restoring the disordered image posture according to claim 1, characterized in that: The method of dividing the similarity graph into at least two image groups based on a normalized graph cut algorithm comprises the following steps: Based on a normalized graph cut algorithm, segmenting the similarity graph into at least two initial image groups; Based on each of the initial image groups, based on the similarity, the corresponding image group is integrated to obtain; wherein, One of the initial image groups corresponds to one of the image groups; There is no overlapping relationship between the disordered image components of each of the initial image groups The disordered image compositions between the image groups overlap.
5. The method for restoring the disordered image posture according to claim 1, characterized in that: The method further comprises the following steps: Based on the image pose recovery algorithm, parallel calculations are performed on each of the image groups to obtain the image pose corresponding to each of the image groups.
6. A disordered image posture recovery device, characterized in that: The device comprises: A similarity map acquisition module is used to sort multiple disordered images according to similarity to obtain a similarity map arranged based on similarity; An image segmentation module, used for segmenting the similarity graph into at least two image groups based on a normalized graph cut algorithm; An image posture calculation module, which is used to calculate and obtain the image posture corresponding to each of the image groups based on an image posture recovery algorithm; The image pose merging module is used to merge the image poses corresponding to each of the image groups to obtain a final image pose.
7. The disordered image posture recovery device according to claim 6, characterized in that: Each vertex in the similarity graph corresponds to one of the disordered images; Two adjacent vertices in the similarity graph constitute an edge in the similarity graph, and an edge weight is configured based on the similarity between the disordered images corresponding to the two vertices.
8. The disordered image posture recovery device according to claim 6, characterized in that: The number of the disordered images corresponding to each of the image groups is the same and is less than the total number of disordered effects; The disordered image compositions between the image groups overlap.
9. The disordered image posture recovery device according to claim 6, characterized in that: The image segmentation module is also used to segment the similarity graph into at least two initial image groups based on a normalized graph cut algorithm; The image segmentation module is also used to integrate and obtain the corresponding image groups based on the similarity of each of the initial image groups; wherein, One of the initial image groups corresponds to one of the image groups; There is no overlapping relationship between the disordered image components of each of the initial image groups The disordered image compositions between the image groups overlap.
10. The disordered image posture recovery device according to claim 6, characterized in that: The image pose calculation module is also used to perform parallel calculations on each of the image groups based on an image pose recovery algorithm to obtain the image pose corresponding to each of the image groups.