A method for batch generation of mask images applied to rapid three-dimensional reconstruction

Abstract

This invention provides a method for batch-generating mask images for rapid 3D reconstruction, relating to the field of image processing technology. The method first constructs a digital scene model with a standardized structure, then projects a standard contour of the model at specific viewpoint coordinates. Next, contour-based graph matching is performed between the projected image and a sampled image to obtain the ROI region in the sampled image. Finally, mask assignment is performed to efficiently and accurately generate a mask image for the sampled image. This invention not only significantly improves processing efficiency but also ensures the accuracy of the mask image, providing strong support for the development and application of 3D reconstruction technology.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for batch generating mask images for rapid three-dimensional reconstruction. Background Art

[0002] When it comes to 3D reconstruction of stereoscopic scenes or structures, the traditional method of manually outlining the target region (ROI) to create a mask image is unable to cope with the processing of massive amounts of sequential image data.

[0003] This method is not only inefficient but also difficult to meet the needs of large-scale data processing. Therefore, in the dense reconstruction calculation (MVS) stage, a large number of unnecessary redundant areas are often involved in the calculation, which greatly affects the computational efficiency. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a method for batch generating mask images for rapid three-dimensional reconstruction, which can greatly improve the processing efficiency and ensure the accuracy of the mask image, providing strong support for the development and application of three-dimensional reconstruction technology.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for batch generating mask images for rapid three-dimensional reconstruction, comprising:

[0007] Collecting a sequence of images of the target structure and scene, and obtaining the spatial coordinates of the image sensor corresponding to each of the sequence images during the collection process;

[0008] Drawing a standardized digital model of the target structure and scene based on the sequence of images; the standardized digital model includes contour information of the target structure and scene;

[0009] In the standardized digital model, the spatial coordinates of the image sensor are used as projection centers, and a projection axis is established based on the posture of the image sensor. A projection surface is established perpendicular to the projection axis and at a preset distance from the projection point. The orthographic projection graphic corresponding to the outer edge contour of the standard digital model is calculated using the projection relationship. The pixel unit size of the image sensor is used to divide the projection surface into grid units. If the grid unit contains projection data, the pixel grayscale value of the grid unit is set to 0, otherwise it is set to 255. A standard sequence projection image that corresponds one-to-one to the sequence image and has the same scale is obtained, and the standard sequence projection image is set as a standard sample image set; each of the standard sequence projection images is a standard image containing contour line data of the target structure;

[0010] Using the Canny operator to perform edge detection on the sequence images in sequence, and obtain an edge line graph corresponding to each of the sequence images;

[0011] Performing shape matching calculation on the edge line graph corresponding to the same coordinates and the sample graph in the standard sample image set using a shape context algorithm to obtain a contour graph of the sample graph corresponding to the edge line graph;

[0012] Calculating the perimeter of all contour lines in the contour map, and setting the grayscale value of all pixels in the area of the contour with the maximum perimeter to 0, and setting the grayscale value of the pixels in the area other than 0 to 255, to obtain a 0-value area and a 255-value area;

[0013] Detecting the 0-value area; if the 0-value area still contains a complete contour, setting the pixel grayscale value of the 0-value area to 255, and repeating the step of "detecting the 0-value area" to perform contour line detection on the 0-value area again; if it still contains a contour, resetting the pixel grayscale value of the 0-value area to 0, so as to detect all contour lines in the 0-value area and implement color assignment;

[0014] The zero-value region is determined as the ROI target region corresponding to the sequence of images, and a corresponding mask image is obtained using the ROI target region.

[0015] Preferably, the preset distance is equal to the sampling focal length when capturing the sequential images of the target structure and the scene.

[0016] Preferably, the same coordinates are projection coordinates and image sensor space coordinates.

[0017] Preferably, the projection axis is horizontal, vertical or any other posture.

[0018] Preferably, the other postures are recorded while obtaining the spatial coordinates of the image sensor corresponding to each of the sequence images during the acquisition process; or

[0019] Obtained through motion recovery calculation.

[0020] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0021] This invention provides a method for batch-generating mask images for rapid 3D reconstruction. First, a structurally standardized digital scene model is constructed. Then, a standard contour projection of the model is performed at specific viewpoint coordinates. Next, contour-based pattern matching is performed between the projected image and a sample image to obtain the ROI region within the sample image. Finally, mask assignment is performed to efficiently and accurately generate a mask image for the sample image. This method not only significantly improves processing efficiency but also ensures the accuracy of the mask image, providing strong support for the development and application of 3D reconstruction technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A flow chart of a method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0025] The purpose of the present invention is to provide a method for batch generating mask images for rapid three-dimensional reconstruction, which can improve processing efficiency while ensuring the accuracy of mask images.

[0026] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Figure 1 A flow chart of the method provided in the embodiment of the present invention is shown in FIG. Figure 1 As shown, the present invention provides a method for batch generating mask images for rapid three-dimensional reconstruction, comprising:

[0028] Step 100: Acquire a sequence of images of the target structure and scene, and obtain the spatial coordinates of the image sensor corresponding to each sequence of images during the acquisition process;

[0029] Step 200: Draw a standardized digital model of the target structure and scene based on the sequence of images; the standardized digital model includes contour information of the target structure and scene;

[0030] Step 300: In the standardized digital model, the spatial coordinates of the image sensor are used as the projection center, and a projection axis is established based on the image sensor posture. A projection surface is established perpendicular to the projection axis and at a preset distance from the projection point. The orthographic projection corresponding to the outer edge contour of the standard digital model is calculated using the projection relationship. The projection surface is divided into grid cells based on the pixel unit size of the image sensor. If the grid cell contains projection data, the pixel grayscale value of the grid cell is set to 0, otherwise it is set to 255. A standard sequence projection image that corresponds one-to-one to the sequence image and has the same scale is obtained, and the standard sequence projection image is set as the standard sample image set; each standard sequence projection image is a standard image containing the contour line data of the target structure;

[0031] Step 400: Use the Canny operator to perform edge detection on the sequence images in sequence to obtain an edge line graph corresponding to each sequence image;

[0032] Step 500: using a shape context algorithm to perform shape matching calculation on the edge line graph corresponding to the same coordinates and the sample image in the standard sample image set to obtain a contour image of the sample image corresponding to the edge line graph;

[0033] Step 600: Calculate the perimeter of all contour lines in the contour map, and set the grayscale value of all pixels in the area of the contour with the maximum perimeter to 0, and the grayscale value of the pixels in the area other than 0 to 255, thereby obtaining a 0-value area and a 255-value area;

[0034] Step 700: Detect the 0-value area. If the 0-value area still contains a complete contour, set the pixel grayscale value of the 0-value area to 255, and repeat the step "Detect the 0-value area" to perform contour line detection on the 0-value area again. If it still contains a contour, reset the pixel grayscale value of the 0-value area to 0, so as to detect all contour lines in the 0-value area and implement color assignment;

[0035] Step 800: Determine the zero-value region as the ROI target region corresponding to the sequence image, and obtain the corresponding mask image based on the ROI target region.

[0036] Specifically, the technical process of this embodiment is as follows:

[0037] Step 1: Collect a sequence of images of the structure and scene. During the acquisition process, the image sensor spatial coordinates (x, y, z) corresponding to each image are obtained through GPS or position measurement device.

[0038] Step 2: Draw a standardized digital model of the structure and scene, which accurately contains the outline information of the structure and scene.

[0039] Specifically, you can use modeling software such as 3DS max, revit, rhino, etc. to directly perform 3D modeling according to the scene or structure, or you can use Python programming and the mayavi library in Python to create and control three-dimensional coordinate points and draw contour lines. If you want detailed steps, you can use Python to draw.

[0040] Step 3: In the standardized digital model, the spatial coordinates of the image sensor in step 1 are used as the projection center O(x0, y0, z0), and the projection axis is established with the same image sensor posture as in step 1 (generally horizontal or vertical. If there is another posture, it can be recorded in step 1 or obtained through motion recovery calculation). A projection surface is established perpendicular to the projection axis and at a distance d from the projection point (d is approximately equal to the sampling focal length f in step 1). The size of the projection surface is consistent with the size of the image sensor. The orthographic projection figure corresponding to the outer edge contour of the standard digital model is calculated using the projection relationship. The pixel unit size in the image sensor is taken to divide the grid unit on the projection surface. If the unit contains projection data, the grayscale value of the pixel of this unit is set to 0, otherwise it is set to 255. That is, a standard sequence of projection images that corresponds one-to-one to the sampled image and has the same scale is obtained, and each projection image is a standard image that only contains the contour line data of the structure. This is set as the standard sample image set C.

[0041] Step 4: Use the Canny operator to perform edge detection on the sequence of sampled images in step 1 in turn, so as to obtain the edge line graph corresponding to each sampled image.

[0042] Specifically, you can import the OpenCV library into Python for processing. The OpenCV library has a special processing function cv2.Canny(). You can input the image and threshold to perform edge calculation on the image.

[0043] Step 5: Use the shape context algorithm to perform shape matching calculation on the edge line graph corresponding to the same coordinates (the projection coordinates are the same as the image sensor space coordinates) and the sample image in step 3 to obtain the contour map of the edge line graph corresponding to the sample image. This step can eliminate the miscellaneous line data in the edge line graph.

[0044] Specifically, since the edge line graph obtained in step 4 is composed of discrete points or discrete lines, although we can see the overall outline of the image, the computer cannot know it. Furthermore, we can use the cv2.findContours function in the OpenCV library in Python to calculate the edge line graph obtained in step 4, thereby calculating the shape outline in the image, and then calculating the descriptor of each point on the outline (the descriptor is obtained by calculating the relative position relationship between this point and other points on the shape, and then performing distribution statistics and normalization). Finally, shape matching can be performed by comparing the similarity of the descriptors in the two images through Euclidean distance.

[0045] Step 6: Based on the contour map obtained in step 5, calculate the perimeter of all contour lines in the map. The grayscale value of all pixels in the area of the contour with the maximum perimeter is set to 0, and the grayscale value of the pixels outside the contour is set to 255.

[0046] Step 7: Detect the 0-value area obtained in step 6. If the area still contains a complete contour, the grayscale value of this part of the pixels is set to 255, and repeat the steps to perform contour line detection on this part of the area again. If it still contains a contour, the grayscale value of the pixels in this area is reset to 0. Based on this rule, all contour lines in the area are calculated for detection and color assignment.

[0047] Step 8: Based on step 7, the area with a grayscale value of 0 is the ROI target area corresponding to the sampling image, thereby obtaining its corresponding mask image.

[0048] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0049] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for batch generation of mask images for rapid three-dimensional reconstruction, characterized in that: include: Acquire a sequence of images of the target structure and scene, and obtain the spatial coordinates of the image sensor corresponding to each of the sequence images during the acquisition process; Drawing a standardized digital model of the target structure and scene based on the sequence of images; the standardized digital model includes contour information of the target structure and scene; In the standardized digital model, the spatial coordinates of the image sensor are used as projection centers, and a projection axis is established based on the posture of the image sensor. A projection surface is established perpendicular to the projection axis and at a preset distance from the projection point. The orthographic projection graphic corresponding to the outer edge contour of the standardized digital model is calculated using the projection relationship. The pixel unit size of the image sensor is used to divide the projection surface into grid units. If the grid unit contains projection data, the pixel grayscale value of the grid unit is set to 0, otherwise it is set to 255. A standard sequence projection image that corresponds one-to-one to the sequence image and has the same scale is obtained, and the standard sequence projection image is set as a standard sample image set; each of the standard sequence projection images is a standard image containing contour line data of the target structure; Using the Canny operator to perform edge detection on the sequence images in sequence, and obtaining an edge line graph corresponding to each of the sequence images; Performing shape matching calculation on the edge line graph corresponding to the same coordinates and the sample graph in the standard sample image set using a shape context algorithm to obtain a contour graph of the sample graph corresponding to the edge line graph; Calculating the perimeter of all contour lines in the contour map, and setting the grayscale value of all pixels in the area of the contour with the maximum perimeter to 0, and setting the grayscale value of the pixels in the area other than 0 to 255, to obtain a 0-value area and a 255-value area; Detecting the zero-value area; if the zero-value area still contains a complete contour, setting the pixel grayscale value of the zero-value area to 255, and repeating the step of "detecting the zero-value area" to perform contour line detection on the zero-value area again; if it still contains a contour, resetting the pixel grayscale value of the zero-value area to 0, so as to detect all contour lines in the zero-value area and implement color assignment; Determine the 0-value region as the ROI target region corresponding to the sequence image, and obtain a corresponding mask image based on the ROI target region; The preset distance is equal to the sampling focal length when capturing the sequential images of the target structure and scene; the same coordinates refer to the projection coordinates being the same as the image sensor space coordinates; the projection axis is horizontal, vertical, or any other posture; The other postures are recorded while acquiring the spatial coordinates of the image sensor corresponding to each of the sequence images during the acquisition process; or are obtained through motion recovery calculation.

Images