Point cloud data acquisition method

Abstract

The invention relates to a point cloud data acquisition method, which comprises the following steps that: a background image without a component and a component image are obtained, and the component image comprises a plurality of view angle images obtained by shooting the component; obtaining image data based on the background image and the component image, wherein the image data comprises a plurality of segmentation masks corresponding to the plurality of view angle images of the component; preprocessing the component image and the image data by adopting a redundant information removal method to obtain a component image with redundant information removed; performing three-dimensional reconstruction on the component image with the redundant information removed by adopting a visual reconstruction method to obtain reconstructed component scale-free point cloud data; and performing scale recovery on the non-scale point cloud data of the component by adopting a scale calibration method to obtain actual point cloud data of the component. The method comprises the following steps: efficiently acquiring a plurality of view angle images and a background image without a component of the component; by removing redundant information of component images and image data, the point cloud data reconstruction time is shortened.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a point cloud data acquisition method. Background Art

[0002] During the construction of steel structures for small and medium-sized projects, the entire structure is split into several components. Each component needs to be geometrically checked, and only components with dimensional deviations within the allowable range are allowed to be transported to the site for assembly, which results in high-frequency and high-failure geometric detection work.

[0003] Existing 3D point cloud data acquisition technologies, such as land-based scanners and handheld scanners, can obtain high-quality 3D point information on the surface of objects. Combined with relevant algorithms, they can realize digital detection of component dimensions, and have been actively explored and widely used in dimension detection and pre-assembly work in various large-scale projects. However, the current point cloud data acquisition equipment is oriented to a small number of objects, and has high professional requirements for surveyors, making it difficult to meet the requirements of component manufacturers for batch point cloud data acquisition of components. In addition, based on the principle of photogrammetry, point cloud data reconstructed from multi-view images have problems of scale loss and excessive redundant information, which is not conducive to batch point cloud data acquisition and storage of small and medium-sized components. Summary of the invention

[0004] The present application provides a point cloud data collection method, which can solve the problems of difficulty and high collection cost in point cloud data collection during digital size detection of small and medium-sized project components in related technologies.

[0005] The present application provides a point cloud data collection method, which includes:

[0006] Acquire a background image without components and a component image, wherein the component image includes multiple viewing angle images taken of the component;

[0007] Obtaining image data based on the background image and the component image, wherein the image data includes a plurality of segmentation masks corresponding to a plurality of viewing angle images of the component;

[0008] Preprocessing the component image and image data by using a redundant information removal method to obtain a component image with redundant information removed;

[0009] The visual reconstruction method is used to perform three-dimensional reconstruction on the component image after removing redundant information, and the reconstructed scale-free point cloud data of the component is obtained;

[0010] The scale calibration method is used to restore the scale of the scale-free point cloud data of the component to obtain the actual point cloud data of the component.

[0011] In one embodiment, the image data is obtained based on the background image and the component image, and the specific steps include:

[0012] Sort the images from each perspective according to the shooting time;

[0013] According to the first rule, obtain the segmentation mask corresponding to the first viewing angle image;

[0014] According to the second rule, a segmentation mask corresponding to the latter of two adjacent viewing angle images is obtained;

[0015] The first rule includes: using a startup prompt optimization generation method, and based on the background image and the first perspective image, obtaining a multi-point prompt corresponding to the first perspective image; using a visual basic model, and based on the first perspective image and its corresponding multi-point prompt, obtaining a segmentation mask corresponding to the first perspective image;

[0016] The second rule includes: using a self-hint optimization generation method, and based on the previous perspective image, the segmentation mask corresponding to the previous perspective image and the next perspective image in two adjacent perspective images, generating a multi-point hint corresponding to the next perspective image; using a visual basic model, and based on the next perspective image and its corresponding multi-point hint, obtaining a segmentation mask corresponding to the next perspective image.

[0017] In one implementation, a startup prompt optimization generation method is adopted, and based on the background image and the first viewing angle image, a multi-point prompt corresponding to the first viewing angle image is obtained, and the specific steps include:

[0018] Perform scale-invariant feature transformation detection and matching between the background image and the first view image to obtain a homography matrix and a matching point set in the background image;

[0019] Create a rectangular bounding box to match the point set;

[0020] Divide the rectangular bounding box into grids to obtain a grid point set of the background image;

[0021] The grid point set of the background image is transformed from the coordinate system of the background image to the coordinate system of the first view image through the homography matrix to obtain a grid point set, and the correspondence between each point in the two sets is retained;

[0022] Extract SIFT features of grid point sets and in the background image and the first view image respectively;

[0023] Calculate the modulus of the vector difference between the SIFT feature of each point and the SIFT feature of the corresponding point in , and retain the points whose modulus of the SIFT feature vector difference in is less than the threshold as the reference point set;

[0024] Use denoising algorithm to remove noise points in point concentration;

[0025] Establish the minimum convex hull of the denoised point set and erode n pixels to obtain the eroded convex hull;

[0026] The points in the convex hull are selected from the point set and used as the multi-point prompt corresponding to the first perspective image.

[0027] In one embodiment, the rectangular bounding box is divided into grids along the direction parallel to the X-axis and the direction parallel to the Y-axis with a spacing of pixels;

[0028] In the background image and the first view image, SIFT features are extracted from the neighborhood pixels of each point in the grid point set and respectively.

[0029] In one implementation, a self-prompt optimization generation method is adopted, and based on two adjacent perspective images, the previous perspective image, the segmentation mask corresponding to the previous perspective image, and the next perspective image, a multi-point prompt corresponding to the current perspective image is generated, and the specific steps include:

[0030] The segmentation mask of the previous view image is divided into several regions in the horizontal and vertical directions;

[0031] Extract the centroid of each divided area as a reference point set;

[0032] Exclude points in the point set that are not within the segmentation mask;

[0033] Extract the SIFT features of the previous view image and the SIFT features of the next view image and perform SIFT feature matching to obtain the homography matrix;

[0034] Multiply the homography matrix by the point set to generate multi-point prompts corresponding to the next perspective image.

[0035] In one embodiment, in preprocessing the component image and image data using a redundant information removal method, preprocessing one of the perspective images and its corresponding image data includes the following steps:

[0036] Expand the segmentation mask corresponding to the current viewing angle image by m pixels;

[0037] Replace the RGB values ​​of the image pixels outside the segmentation mask with the same values;

[0038] Create a rectangular bounding box for the segmentation mask, crop and keep the image within the rectangular box.

[0039] In one implementation, when there are no real geometric features of the camera pose, a scale calibration method is used to restore the scale of the scale-free point cloud data. The specific steps include:

[0040] Extracting the scale-free point cloud size corresponding to the measured size in the scale-free point cloud data;

[0041] Based on the measured size of the component and the size of the scale-free point cloud, the conversion coefficient between the measured size of the component and the size of the scale-free point cloud is obtained;

[0042] Based on the conversion coefficient and the scale-free geometric features of the camera pose, the actual geometric features of the camera pose are obtained to complete the scale restoration of the scale-free point cloud data.

[0043] In one implementation, when there are real geometric features of the camera pose, a scale calibration method is used to restore the scale of the scale-free point cloud data. The specific steps include:

[0044] Based on the real geometric features of the camera pose and the scale-free geometric features of the camera pose, the conversion coefficient between the measured size of the component and the scale-free point cloud size is obtained;

[0045] The coordinate value of the scale-free point cloud data is multiplied by the conversion coefficient to complete the scale restoration of the scale-free point cloud data.

[0046] In one implementation, obtaining a component-free background image and a component image specifically includes:

[0047] A multi-view image acquisition device is built around the component, wherein the multi-view image acquisition device comprises a motion mechanism, a support device and an actuator, wherein the actuator is arranged on the motion mechanism through the support device;

[0048] Conduct track travel calibration;

[0049] The motion mechanism drives the actuator to take pictures of the component and collect a background image without the component at the zero stroke;

[0050] According to the track travel calibration, component images are captured at fixed travel increments.

[0051] In one embodiment, the relationship between the accuracy of the scale-free point cloud data of the component reconstructed by the multi-view image acquisition device and the component image is determined by the field of view of the image acquisition device, the film size, the average distance between the image acquisition device and the component during the entire image acquisition process, the equivalent focal length of the image acquisition device, and the pixel size and coefficient of the image acquired by the image acquisition device.

[0052] The beneficial effects brought by the technical solution provided by the embodiments of the present application include:

[0053] The embodiment of the present application provides a point cloud data acquisition method, which efficiently acquires multiple perspective images of the component and a background image without the component; by removing redundant information of the component image and image data, the storage space of the image and point cloud files can be effectively reduced, the point cloud reconstruction time can be shortened, and the environmental point cloud in the point cloud data can be reduced, which is beneficial to file storage, calculation and point cloud data post-processing; through the scale calibration method, after calibrating a component, the scale of batch component point clouds can be restored without adjusting the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 A schematic diagram of the structure of a multi-view image acquisition device provided in an embodiment of the present application;

[0056] Figure 2 for Figure 1 A schematic diagram of the partially enlarged structure at center A;

[0057] Figure 3 for Figure 1 A schematic diagram of the partially enlarged structure at B in the middle;

[0058] Figure 4 A flow chart of the point cloud data acquisition method provided in an embodiment of the present application.

[0059] In the figure: 1. track; 2. universal joint; 3. camera; 4. stepper motor; 5. track; 6. motor buckle; 7. gear; 8. connector. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] See also Figures 1 to 4 The embodiment of the present application provides a point cloud data collection method, which can solve the problems of difficulty in collecting point cloud data and high collection cost in the digital size detection process of small and medium-sized project components in related technologies.

[0062] The present application provides a point cloud data collection method, which includes:

[0063] 101: Acquire a background image without components and a component image, where the component image includes multiple viewing angle images taken of the component;

[0064] 102: Obtain image data based on the background image and the component image, where the image data includes multiple segmentation masks corresponding to multiple perspective images of the component;

[0065] 103: Preprocessing the component image and the image data using a redundant information removal method to obtain a component image with redundant information removed;

[0066] 104: Using a visual reconstruction method to perform three-dimensional reconstruction on the component image with redundant information removed, and obtaining reconstructed scale-free point cloud data of the component;

[0067] 105: A scale calibration method is used to restore the scale of the component's scale-free point cloud data to obtain the component's actual point cloud data.

[0068] In the present application, multiple perspective images of components and background images without components are efficiently collected for the components; redundant information of component images and image data is removed, which can effectively reduce the storage space of image and point cloud files, shorten the point cloud reconstruction time, and reduce the environmental point clouds in the point cloud data, which is beneficial to file storage, calculation and point cloud data post-processing; after calibrating a component through a scale calibration method, the scale of batch component point clouds can be restored without adjusting the equipment.

[0069] Wherein, in step 101: obtaining a background image and a component image without components, the component image includes a plurality of perspective images taken of the components, and obtaining the background image and the component image without components, specifically including steps 1011 to 1014:

[0070] Step 1011: Build a multi-view image acquisition device around the component.

[0071] Specifically, the multi-view image acquisition device includes a motion mechanism, a support device and an actuator, and the actuator is arranged on the motion mechanism through the support device. Among them, the support device is used to support the entire acquisition device, and can be set as a fixed support or a movable support. In this embodiment, the motion mechanism and the actuator are connected through a universal hinge 2, and a connecting piece 8 is connected between the universal hinge 2 and the actuator. The universal hinge 2 is set to enable the actuator to adjust its posture. The motion mechanism includes a track 1 with a crawler 5 and a programmable stepper motor 4; one end of the stepper motor 4 is fixedly connected to a gear 7 through a motor buckle 6, and a tooth groove is arranged on the crawler 5. Through the meshing transmission of the gear 7 and the tooth groove, the stepper motor 4 can drive the crawler 5 to drive the actuator to move on the track 1; the actuator includes an image acquisition device with a communication function, and the image acquisition device includes a camera 3. The motion mechanism and the actuator communicate through Wifi.

[0072] It should be noted that before collecting the background image and the component image, the travel of the track 1 needs to be calibrated and the number of revolutions of the stepper motor 4 throughout the travel needs to be recorded.

[0073] In this embodiment, the track 1 is set to be circular with a radius of 1.5m. In other embodiments, the track 1 can also be set to be rectangular or other shapes. In addition, the track 1 can also be set to be two rows of parallel straight lines. The camera 3 can be used to collect RGB images. The equivalent focal length of the camera 3 is 27mm, and the average distance between the camera 3 and the component is 1.8m.

[0074] Step 1012: perform track 1 travel calibration.

[0075] Before collecting the background image and the component image, the track 1 stroke calibration needs to be performed. The multi-view image acquisition device collects the background image and the component image with a uniform percentage stroke and is controlled by the stepper motor 4.

[0076] Step 1013: The motion mechanism drives the actuator to photograph the component and collect a background image without the component at the zero stroke.

[0077] Specifically, a background image without components is collected at the zero stroke.

[0078] Step 1014: According to the track 1 travel calibration, component image acquisition is performed with fixed travel increments.

[0079] Specifically, in this embodiment, the track 1 is set to be circular, and multiple perspective images of the component are collected at a stroke increment of 1%, and 101 perspective images are collected.

[0080] It should be noted that, when capturing the component-free background image and the first viewing angle image, the camera 3 is located at the zero travel position of the multi-view image capturing device.

[0081] On the basis of the above embodiment, in this embodiment, the relationship between the multi-view image acquisition device and the component scale-free point cloud data accuracy P reconstructed by the component image is determined by the image acquisition device field of view size F, the film size S, the average distance D between the image acquisition device and the component during the entire image acquisition process, and the equivalent focal length f of the image acquisition device. eq , the pixel size ρ and coefficient λ of the image captured by the image acquisition device are determined.

[0082] Specifically, according to the formula:

[0083] F=S 35 ×D / f eq , Determine, where S 35 The film size is 35 mm, in this embodiment, 36 mm × 24 mm; f eq is the 35 mm equivalent focal length of the image acquisition device; λ can be verified through embodiments and is usually 5 to 10.

[0084] On the basis of the above embodiment, in this embodiment, image data is obtained based on the background image and the component image, and the specific steps include step 1021 to step 1023:

[0085] Step 1021: sort the images of each viewing angle according to the shooting time.

[0086] Specifically, after collecting 1 background image and 101 perspective images, the 101 perspective images are sorted into 1, 2, 3, ..., 101, with the perspective images collected first being sorted first and the perspective images collected later being sorted later.

[0087] Step 1022: According to the first rule, obtain the segmentation mask corresponding to the first viewing angle image.

[0088] Specifically, the first rule includes: adopting the startup prompt optimization generation method, and based on the background image and the first perspective image, obtaining the multi-point prompts corresponding to the first perspective image; adopting the visual basic model, and based on the first perspective image and its corresponding multi-point prompts, obtaining the segmentation mask corresponding to the first perspective image.

[0089] The startup prompt optimization generation method is adopted, and based on the background image and the first viewing angle image, a multi-point prompt corresponding to the first viewing angle image is obtained. The specific steps include:

[0090] First, the scale invariant feature transform (SIFT) of the background image and the first view image is detected and matched to obtain the homography matrix M Sand the matching point set P in the background image M ; Then establish the matching point set P M The rectangular bounding box T B ; Then the rectangular bounding box T B Divide into grids to obtain the grid point set P of the background image B It should be noted that, in this embodiment, the rectangular bounding box T is parallel to the X-axis direction and the Y-axis direction with a spacing of d pixels. B Divide into grids; then use the homography matrix M S , the grid point set P of the background image B Transform the coordinate system of the background image to the coordinate system of the first view image to obtain the grid point set P E , and keep the correspondence between each point in the two sets. The two sets here refer to the grid point set P of the background image. B and grid point set P E ; Then extract the grid point set P from the background image and the first view image respectively B and P F SIFT features, it should be noted that the grid point set P is extracted from the background image and the first view image respectively. B and P F SIFT features in the d / 2 neighborhood pixels of each point in ; then calculate P B SIFT features of each point and P F The modulus length of the vector difference of the SIFT features of the corresponding points in P F The points whose modulus length of SIFT feature vector difference is less than the threshold γ are taken as the reference point set P; and the denoising algorithm is used to remove the noise points in the point set P, establish the minimum convex hull of the denoised point set P and corrode n pixels to obtain the corroded convex hull T C ; Finally, select the points in the convex hull T from the point set P C The point in the image is used as the multi-point prompt P corresponding to the first viewing angle image. S .

[0091] For ease of understanding, the following example is used to illustrate the component as a steel column: First, input the background image and the first view image of the steel column, then perform SIFT feature detection and matching on the background image and the first view image of the steel column to obtain the homography matrix The set of matching points P in the background image M , then establish a matching point set P M The rectangular bounding box T B , and divide the rectangular bounding box into grids along the X-axis and Y-axis with d = 22 pixels, and obtain the grid point set P B ; Then through the homography matrix M S , the grid point set PB Transform to the first perspective image of the steel column and obtain the grid point set P F , retain the correspondence between each point in the two sets; then extract the grid point set P from the background image and the first image of the steel column respectively B and P F SIFT features in the neighborhood pixels of each point d / 2 = 11; then calculate P B SIFT features of each point and F The modulus length of the vector difference of the SIFT features of the corresponding points in P F The points whose modulus of the vector difference of SIFT features is less than the threshold value γ = 450 are taken as the reference point set P, and the noise points in the point set P are removed by using the statistical filtering algorithm; then the minimum convex hull of the denoised point set P is established and eroded by 50 pixels to obtain the eroded convex hull T C , and select the points in the convex hull T from the denoised point set P C The point inside is output as the starting prompt point P of the first perspective image of the steel column. S .

[0092] The first perspective image of the steel column collected and the starting prompt point P of the first perspective image of the steel column S Input to the visual basis model MobileSAM, output the segmentation mask of the first view image.

[0093] Furthermore, the above-mentioned Vision Foundation Model can adopt a series of open source models that support multi-point prompts, such as SegmentAnything Model (SAM), FastSAM, MobileSAM, etc. Among them, the input data in the Vision Foundation Model is the first perspective image and its corresponding multi-point prompts. The output data of the Vision Foundation Model is the segmentation mask of the component in the first perspective image. In this embodiment, preferably, the Vision Foundation Model used is SAM.

[0094] Step 1023: Obtain a segmentation mask corresponding to the latter of two adjacent viewing angle images according to the second rule;

[0095] Specifically, the second rule includes: using a self-hint optimization generation method, and based on the previous perspective image, the segmentation mask corresponding to the previous perspective image and the next perspective image in two adjacent perspective images, generating a multi-point hint corresponding to the next perspective image; using a visual basic model, and based on the next perspective image and its corresponding multi-point hint, obtaining a segmentation mask corresponding to the next perspective image.

[0096] Among them, a self-prompt optimization generation method is adopted, and based on the previous perspective image, the segmentation mask corresponding to the previous perspective image and the next perspective image in the two adjacent perspective images, a multi-point prompt corresponding to the current perspective image is generated. It should be noted that the two adjacent perspective images are input in the order of image acquisition, and the specific steps include: first, the segmentation mask of the previous perspective image is divided into several regions in the horizontal direction and the vertical direction; then the centroid of each divided region is extracted as the reference point set C F ; Then check point set C F Whether each point in is located in the segmentation mask of the previous view image, excluding the point set C F Then extract the SIFT features of the previous view image and the next view image and perform SIFT feature matching to obtain the homography matrix M S ; Finally, the homography matrix M S With point set C F Perform point multiplication to generate the multi-point prompt P corresponding to the next perspective image S .

[0097] It should be noted that the visual basic model can adopt a series of open source models that support multi-point prompts, such as Segment Anything Model (SAM), FastSAM, MobileSAM, etc. The input data of the visual basic model is the next perspective image and its corresponding multi-point prompts; the output data of the visual basic model is the segmentation mask corresponding to the next perspective image.

[0098] Let's take the steel column as an example:

[0099] Select two adjacent perspective images in sorted order, input the previous perspective image of the steel column, the segmentation mask corresponding to the previous perspective image, and the next perspective image of the steel column; then divide the segmentation mask of the previous perspective image of the steel column into 4 regions in the horizontal and vertical directions, and extract the centroid of each divided region as the reference point set C F ; Then check point set C F Each point in the image is located within the segmentation mask of the previous view of the steel column; then the SIFT features of the first view and the second view are extracted and matched to obtain the homography matrix Then according to the homography matrix M S , the point set C in the previous perspective image of the steel column F The coordinates of the points in the image are transformed to the next viewing angle image of the steel column as the self-prompting point P of the next viewing angle image of the steel column. S ; The next perspective image of the steel column and the self-prompting point P of the next perspective image of the steel column SInput to the visual basic model MobileSAM, and output the steel column segmentation mask of the next view image of the steel column. Based on the above steps, the segmentation masks of all the view images of the collected steel column can be obtained.

[0100] Furthermore, the startup prompt optimization generation method is applicable to the first viewing angle image; and the self-prompt optimization generation method is applicable to the latter viewing angle image of two adjacent viewing angle images that does not obtain multi-point prompts.

[0101] On the basis of the above embodiment, in this embodiment, in step 103: using the redundant information removal method to pre-process the component image and the image data, wherein the pre-processing of one of the perspective images and the corresponding image data includes the following steps:

[0102] First, the segmentation mask corresponding to the current view image is expanded by m pixels; then the RGB values ​​of the image pixels outside the segmentation mask are replaced with the same values; finally, a rectangular bounding box of the segmentation mask is established, and the image inside the rectangular box is cropped and retained. It should be noted that the above steps are also referred to when preprocessing the remaining view images and their corresponding image data until the preprocessing of multiple view images and their corresponding image data is completed to obtain the component image with redundant information removed.

[0103] For example: first, the segmentation mask corresponding to the current viewing angle image is expanded by 50 pixels; then the RGB values ​​of the image pixels outside the segmentation mask are changed to [0, 0, 0]; finally, a segmentation mask rectangular bounding box is constructed to crop the image within the bounding box.

[0104] On the basis of the above embodiments, in this embodiment, in step 104: the visual reconstruction method refers to calling an open source algorithm library, and using a Structure from Motion (SfM) algorithm and a Multi-View Stereo (MVS) algorithm to perform three-dimensional reconstruction on the component image with redundant information removed, and obtain reconstructed scale-free point cloud data of the component.

[0105] For example, the open source algorithms OpenMVG and OpenMVS are used to perform 3D reconstruction on the processed multi-view images. The file sizes of the original images and processed multi-view images, the total number of reconstructed point clouds, the number of steel column point clouds, and the reconstruction time are compared, see Table 1:

[0106]

[0107] Table 1

[0108] On the basis of the above embodiment, in this embodiment, in step 105: a scale calibration method is used to restore the scale of the component's scale-free point cloud data to obtain the actual point cloud data of the component:

[0109] When there is no real geometric feature of the camera 3D pose, the scale calibration method is used to restore the scale of the scale-free point cloud data. The specific steps include: firstly, input the measured size of the component and the reconstructed scale-free point cloud data and the camera 3D pose, and extract the scale-free point cloud size d corresponding to the measured size in the scale-free point cloud data. D1 ; Then according to the formula η DP =d P1 / d D1 , based on the measured dimension d of the component P1 , scale-free point cloud size d D1 , obtain the conversion coefficient η between the measured size of the component and the size of the scale-free point cloud DP ;Finally, according to the formula r T =η DP × D1 , based on the conversion factor η DP 、Camera 3 pose scale-free geometric features r D1 , get the actual geometric features r of the camera 3 poses T , complete the scale restoration of scale-free point cloud data.

[0110] For example, when restoring the scale of a steel column point cloud, the true geometric features of the camera's 3D pose are not obtained. The scale calibration method includes the following steps:

[0111] Input the steel column height tape measure measured column height of 997mm and the reconstructed scale-free point cloud and the camera 3 pose output by OpenMVG; then extract the step scale-free steel column point cloud spacing between the upper and lower end plates of 289.5750mm; according to η DP =d P1 / d D1 Calculate the conversion factor η between the measured column height and the distance between the unscaled end plates DP =3.4430, multiply the scale-free steel column point cloud coordinate value by the conversion factor η DP , complete the scale restoration of the steel column point cloud.

[0112] Among them, the random sampling consistency algorithm is used to fit the circle formed by the three camera poses, and the scale-free radius is 405.75mm; according to r T =η DP × D1 Calculate the actual radius r of the camera 3 pose T It is 1397mm.

[0113] When there are real geometric features of the camera's 3D pose, the scale calibration method is used to restore the scale of the scale-free point cloud data. The specific steps include: first, according to the formula η DP =d P1 / d D1 , based on the real geometric features of the camera 3 poses dP1 , camera 3 pose scale-free geometric features d D1 , obtain the conversion coefficient η between the measured size of the component and the size of the scale-free point cloud DP ; Finally, multiply the coordinate value of the scale-free point cloud data by the conversion factor η DP , complete the scale restoration of scale-free point cloud data.

[0114] For example: the component can also be a steel beam.

[0115] The scale calibration method is used to restore the scale of the reconstructed point cloud data. Among them, the actual radius of the camera 3-pose is 1397mm when the steel beam point cloud is restored. The scale calibration method includes the following steps: first, input the actual radius of the known camera 3-pose and the camera 3-pose output by OpenMVG; then, use the random sampling consistency algorithm to calculate the scale-free radius of the steel beam camera 3-pose as 499.65mm; then, according to η DP =d P1 / d D1 The camera 3D pose conversion coefficient of the steel beam component is calculated to be 2.7960; finally, the reconstructed point cloud coordinate value is multiplied by the conversion coefficient to complete the scale restoration of the steel beam point cloud.

[0116] It should be noted that in the above two methods, d D1 and d D1 The corresponding meanings are different.

[0117] In summary, the point cloud data acquisition optimization method constructed in this application can efficiently and automatically collect multiple perspective images of components; by removing redundant image information, it can effectively reduce the storage space of image and point cloud files, shorten the point cloud reconstruction time, and reduce the environmental point cloud in the point cloud data, which is beneficial to file storage, calculation and point cloud post-processing; by starting the prompt optimization generation method and the self-prompt optimization generation method, combined with the visual basic model, component segmentation mask generation of multiple perspective images of different types of components can be achieved without training; through the point cloud scale calibration method, after calibrating a component, the scale of batch component point clouds can be restored without adjusting the equipment.

[0118] In the description of the present application, it should be noted that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0119] It should be noted that, in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0120] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A point cloud data acquisition method, characterized in that: It includes: Acquire a background image without components and a component image, wherein the component image includes multiple viewing angle images taken of the component; Obtaining image data based on the background image and the component image, wherein the image data includes a plurality of segmentation masks corresponding to a plurality of viewing angle images of the component; Preprocessing the component image and image data by using a redundant information removal method to obtain a component image with redundant information removed; The visual reconstruction method is used to perform three-dimensional reconstruction on the component image after removing redundant information, and the reconstructed scale-free point cloud data of the component is obtained; The scale calibration method is used to restore the scale of the scale-free point cloud data of the component to obtain the actual point cloud data of the component.

2. The point cloud data acquisition method according to claim 1, characterized in that: The image data is obtained based on the background image and the component image, and the specific steps include: Sort the images from each perspective according to the shooting time; According to the first rule, obtain the segmentation mask corresponding to the first viewing angle image; According to the second rule, a segmentation mask corresponding to the latter of two adjacent viewing angle images is obtained; The first rule includes: using a startup prompt optimization generation method, and based on the background image and the first perspective image, obtaining a multi-point prompt corresponding to the first perspective image; using a visual basic model, and based on the first perspective image and its corresponding multi-point prompt, obtaining a segmentation mask corresponding to the first perspective image; The second rule includes: using a self-hint optimization generation method, and based on the previous perspective image, the segmentation mask corresponding to the previous perspective image and the next perspective image in two adjacent perspective images, generating a multi-point hint corresponding to the next perspective image; using a visual basic model, and based on the next perspective image and its corresponding multi-point hint, obtaining a segmentation mask corresponding to the next perspective image.

3. The point cloud data acquisition method according to claim 2, characterized in that: The startup prompt optimization generation method is adopted, and based on the background image and the first perspective image, a multi-point prompt corresponding to the first perspective image is obtained. The specific steps include: Perform scale-invariant feature transformation detection and matching between the background image and the first view image to obtain the homography matrix M S and the matching point set P in the background image M ; Establish a matching point set P M The rectangular bounding box T B ; The rectangular bounding box T B Divide into grids to obtain the grid point set P of the background image B ; Through the homography matrix M S , the grid point set P of the background image B Transform the coordinate system of the background image to the coordinate system of the first view image to obtain the grid point set P E , and keep the correspondence between each point in the two sets; Extract the grid point set P from the background image and the first view image respectively B and P F SIFT features; Calculate P B Each point SIFT feature and P F The modulus length of the vector difference of the SIFT features of the corresponding points in P F The points whose modulus of SIFT feature vector difference is less than the threshold γ are taken as the reference point set P; Use denoising algorithm to remove noise points in point set P; Establish the minimum convex hull of the denoised point set P and erode n pixels to obtain the eroded convex hull T C ; Filter out the points in the convex hull T from the point set P C The point in the image is used as the multi-point prompt P corresponding to the first viewing angle image. S .

4. The point cloud data acquisition method according to claim 3, characterized in that: With a spacing of d pixels, the rectangular bounding box T is parallel to the X-axis and the Y-axis. B Divide into grids; Extract the grid point set P from the background image and the first view image respectively B and P F SIFT features within d / 2 neighborhood pixels of each point in .

5. The point cloud data acquisition method according to claim 2, characterized in that: A self-prompt optimization generation method is adopted, and based on the previous perspective image, the segmentation mask corresponding to the previous perspective image and the next perspective image in two adjacent perspective images, a multi-point prompt corresponding to the current perspective image is generated. The specific steps include: The segmentation mask of the previous view image is divided into several regions in the horizontal and vertical directions; Extract the centroid of each divided area as the reference point set C F ; Exclude point set C F Points in that are not within the segmentation mask; Extract the SIFT features of the previous view image and the SIFT features of the next view image and perform SIFT feature matching to obtain the homography matrix M S ; The homography matrix M S With point set C F Perform point multiplication to generate the multi-point prompt P corresponding to the next perspective image S .

6. The point cloud data acquisition method according to claim 1, characterized in that: In preprocessing the component image and image data using the redundant information removal method, preprocessing one of the perspective images and its corresponding image data includes the following steps: Expand the segmentation mask corresponding to the current viewing angle image by m pixels; Replace the RGB values ​​of the image pixels outside the segmentation mask with the same values; Create a rectangular bounding box for the segmentation mask, crop and keep the image within the rectangular box.

7. The point cloud data acquisition method according to claim 1, characterized in that: When there are no real geometric features of the camera pose, the scale calibration method is used to restore the scale of the scale-free point cloud data. The specific steps include: Extract the scale-free point cloud size d corresponding to the measured size in the scale-free point cloud data D1 ; Based on the measured dimension d of the component P1 , scale-free point cloud size d D1 , obtain the conversion coefficient η between the measured size of the component and the size of the scale-free point cloud DP ; Based on the conversion factor η DP , scale-free geometric features of camera pose r D1 , get the actual geometric features r of the camera pose T , complete the scale restoration of scale-free point cloud data.

8. The point cloud data acquisition method according to claim 1, characterized in that: When there are real geometric features of camera pose, the scale calibration method is used to restore the scale of scale-free point cloud data. The specific steps include: Based on the real geometric features of camera pose P1 , scale-free geometric features of camera pose d D1 , obtain the conversion coefficient η between the measured size of the component and the size of the scale-free point cloud DP ; Multiply the coordinate value of the scale-free point cloud data by the conversion factor η DP , complete the scale restoration of scale-free point cloud data.

9. The point cloud data acquisition method according to claim 1, characterized in that: Get the background image and component image without components, including: A multi-view image acquisition device is built around the component, wherein the multi-view image acquisition device comprises a motion mechanism, a support device and an actuator, wherein the actuator is arranged on the motion mechanism through the support device; Conduct track travel calibration; The motion mechanism drives the actuator to take pictures of the component and collect a background image without the component at the zero stroke; According to the track travel calibration, component images are captured at fixed travel increments.

10. The point cloud data acquisition method according to claim 9, characterized in that: The relationship between the multi-view image acquisition device and the component scale-free point cloud data accuracy P reconstructed by the image acquisition device field of view F, the film size S, the average distance D between the image acquisition device and the component during the entire image acquisition process, and the equivalent focal length f of the image acquisition device eq , the pixel size ρ and coefficient λ of the image captured by the image acquisition device are determined.

Images