A point cloud data acquisition method

Abstract

The application relates to a point cloud data acquisition method, which comprises the following steps: acquiring a background image without a component and a component image, wherein the component image comprises multiple perspective images shot on the component; obtaining image data based on the background image and the component image, wherein the image data comprises multiple segmentation masks corresponding to the multiple perspective images of the component; pre-processing the component image and the image data by using a redundant information removal method to obtain a component image with removed redundant information; performing three-dimensional reconstruction on the component image with removed redundant information by using a visual reconstruction method to acquire reconstructed component non-scale point cloud data; and performing scale recovery on the component non-scale point cloud data by using a scale calibration method to acquire actual point cloud data of the component. In the application, multiple perspective images of the component and a background image without the component are efficiently acquired; and the redundant information of the component image and the image data is removed, so that the point cloud data reconstruction time is shortened.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, in particular to a point cloud data acquisition method. BACKGROUND

[0002] In the process of steel structure construction of small and medium-sized projects, the whole structure is divided into several components. Each component needs to be geometrically checked. Only the components with size deviation within the allowable range are allowed to be transported to the site for assembly, which brings high frequency and strong failure of geometric detection work.

[0003] The existing three-dimensional point cloud data acquisition technology such as land-based scanner and handheld scanner can obtain high-quality three-dimensional point information of object surface. Combined with related algorithms, digital detection of component size can be realized, which has been actively explored and widely used in size detection and pre-assembly work of various large projects. However, the current point cloud data acquisition equipment faces a small number of objects, and the professional requirements for measurement personnel are high, which is difficult to meet the batch component point cloud data acquisition requirements of component production plants. In addition, based on the principle of photogrammetry, the point cloud data reconstructed from multiple view images has the problems of scale loss and too much redundant information, which is also not conducive to batch point cloud data acquisition and storage of small and medium-sized components. SUMMARY

[0004] The present application provides a point cloud data acquisition method, which can solve the problems of difficult point cloud data acquisition and high acquisition cost in the process of digital size detection of small and medium-sized project components in related technologies.

[0005] The present application provides a point cloud data acquisition method, which can solve the problems of difficult point cloud data acquisition and high acquisition cost in the process of digital size detection of small and medium-sized project components in related technologies.

[0006] Obtaining a background image without components and a component image, the component image including multiple view images taken of the component;

[0007] Obtaining image data based on the background image and the component image, the image data including multiple segmentation masks corresponding to the multiple view images of the component;

[0008] Pretreating the component image and the image data using a redundant information removal method to obtain a component image with removed redundant information;

[0009] Three-dimensionally reconstructing the component image with removed redundant information using a visual reconstruction method to obtain reconstructed component scale-free point cloud data;

[0010] Restoring the scale of the component scale-free point cloud data using a scale calibration method to obtain actual point cloud data of the component.

[0011] In an embodiment, obtaining the image data based on the background image and the component image includes the following specific steps:

[0012] sort the individual perspective images according to the shooting time;

[0013] According to the first rule, the segmentation mask corresponding to the first perspective image is obtained;

[0014] According to the second rule, the segmentation mask corresponding to the latter one of the two adjacent perspective images is obtained;

[0015] The first rule includes: using a prompt optimization generation method, and based on the background image and the first perspective image, obtaining the multi-point prompt corresponding to the first perspective image; using a visual basic model, and based on the first perspective image and the multi-point prompt corresponding to the first perspective image, obtaining the segmentation mask corresponding to the first perspective image;

[0016] The second rule includes: using a self-prompt optimization generation method, and based on the former one of the two adjacent perspective images, the segmentation mask corresponding to the former one of the two adjacent perspective images, and the latter one of the two adjacent perspective images, generating the multi-point prompt corresponding to the latter one of the two adjacent perspective images; using a visual basic model, and based on the latter one of the two adjacent perspective images and the multi-point prompt corresponding to the latter one of the two adjacent perspective images, obtaining the segmentation mask corresponding to the latter one of the two adjacent perspective images.

[0017] In an embodiment, the multi-point prompt corresponding to the first perspective image is obtained by using a prompt optimization generation method and based on the background image and the first perspective image, and the specific steps include:

[0018] Performing scale-invariant feature transform detection and matching on the background image and the first perspective image to obtain a homography matrix and a matching point set in the background image;

[0019] Establishing a rectangular bounding box based on the matching point set;

[0020] Dividing the rectangular bounding box into a grid to obtain a grid point set of the background image;

[0021] Transforming the grid point set of the background image from the coordinate system of the background image to the coordinate system of the first perspective image by using the homography matrix to obtain a grid point set, and preserving the correspondence of each point in the two sets;

[0022] Extracting SIFT features of the grid point set and from the background image and the first perspective image, respectively;

[0023] Calculating the vector difference of the SIFT features of each point and the SIFT features of the corresponding point, and retaining the points in as reference points whose SIFT feature vector difference is less than a threshold;

[0024] Using a denoising algorithm to remove noise points in the point set;

[0025] Establishing a minimum convex hull of the denoised point set and eroding the convex hull by n pixels to obtain an eroded convex hull;

[0026] Screening the points in the convex hull from the point set, and taking the point as the multi-point prompt corresponding to the first view image.

[0027] In an embodiment, the rectangular bounding box is divided into a grid along the direction parallel to the X axis and the direction parallel to the Y axis at a pixel interval.

[0028] SIFT features in a neighborhood of each point in the grid point set and the first view image are extracted.

[0029] In an embodiment, a self-prompt optimization generation method is used, and based on the previous view image, the segmentation mask corresponding to the previous view image, and the next view image in the adjacent two view images, the multi-point prompt corresponding to the current view image is generated, and the specific steps include:

[0030] The segmentation mask of the previous view image is divided into several regions along the horizontal direction and the vertical direction.

[0031] The centroid of each division region is extracted as a reference point set.

[0032] Points in the point set that are not in the segmentation mask are excluded.

[0033] SIFT features of the previous view image and SIFT features of the next view image are extracted and SIFT feature matching is performed to obtain a homography matrix.

[0034] The homography matrix is point-multiplied with the point set to generate a multi-point prompt corresponding to the next view image.

[0035] In an embodiment, a redundant information removal method is used for preprocessing the component image and the image data, and the preprocessing of one of the view images and the image data corresponding thereto includes the following steps:

[0036] The segmentation mask corresponding to the current view image is dilated by m pixels.

[0037] The image pixel RGB value outside the segmentation mask is replaced with the same value.

[0038] A rectangular bounding box of the segmentation mask is established, and the image within the rectangular box is cropped and retained.

[0039] In an embodiment, when there is no camera pose real geometric feature, a scale calibration method is used to restore the scale of the scaleless point cloud data, and the specific steps include:

[0040] The measured size corresponding to the scaleless point cloud size in the scaleless point cloud data is extracted.

[0041] Based on the measured size of the component, the size of the non-scale point cloud, the conversion coefficient of the measured size of the component and the size of the non-scale point cloud is obtained;

[0042] Based on the conversion coefficient and the non-scale geometric feature of the camera pose, the actual geometric feature of the camera pose is obtained, and the scale recovery of the non-scale point cloud data is completed.

[0043] In an embodiment, when there is a real geometric feature of the camera pose, a scale calibration method is used to recover the scale of the non-scale point cloud data, and the specific steps include:

[0044] Based on the real geometric feature of the camera pose and the non-scale geometric feature of the camera pose, the conversion coefficient of the measured size of the component and the size of the non-scale point cloud is obtained;

[0045] The coordinate value of the non-scale point cloud data is multiplied by the conversion coefficient, and the scale recovery of the non-scale point cloud data is completed.

[0046] In an embodiment, a background image without a component and a component image are obtained, and the specific steps include:

[0047] A multi-view image acquisition device is built around the component, and the multi-view image acquisition device includes a motion mechanism, a support device and an execution mechanism, and the execution mechanism is arranged on the motion mechanism through the support device;

[0048] The track stroke is calibrated;

[0049] The motion mechanism drives the execution mechanism to shoot the component, and a background image without a component is collected at zero stroke;

[0050] According to the track stroke calibration, the component image is collected with a fixed stroke increment.

[0051] In an embodiment, the relationship between the multi-view image acquisition device and the accuracy of the reconstructed non-scale point cloud data of the component image is determined by the field of view size of the image acquisition device, the film size, the average distance between the image acquisition device and the component during the entire image acquisition process, the equivalent focal length of the image acquisition device, the pixel size of the image acquisition device and the coefficient.

[0052] The technical scheme provided by the embodiments of the application has the beneficial effects of:

[0053] The embodiment of the present application provides a point cloud data acquisition method, which efficiently acquires multiple view images of a component and background images without the component; through removal of the component images and image data redundancy information, the image and point cloud file storage space can be effectively reduced, the point cloud reconstruction time is shortened, the environment point cloud in the point cloud data is reduced, and the file storage, calculation and point cloud data post-processing are beneficial; through a scale calibration method, after a component is calibrated, scale recovery of batch component point clouds can be performed without adjusting the equipment. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0055] Figure 1 A multi-view image acquisition device structure schematic diagram provided by the embodiment of the present application is shown in the figure.

[0056] Figure 2 A Figure 1 A local enlarged structure schematic diagram at position A in the figure.

[0057] Figure 3 A Figure 1 A local enlarged structure schematic diagram at position B in the figure.

[0058] Figure 4 A point cloud data acquisition method flowchart provided by the embodiment of the present application is shown in the figure.

[0059] In the figure: 1, track; 2, universal hinge; 3, camera; 4, stepping motor; 5, track; 6, motor buckle; 7, gear; 8, connecting piece. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0061] Referring to Figures 1 to 4 The embodiment of the present application provides a point cloud data acquisition method, which can solve the problems of difficult point cloud data acquisition and high acquisition cost in the related art digital size detection process of small and medium-sized project components.

[0062] The embodiment of the application provides a point cloud data acquisition method, which comprises the following steps:

[0063] 101: acquire a background image without a component and a component image, wherein the component image comprises multiple perspective images of the component;

[0064] 102: acquire image data based on the background image and the component image, wherein the image data comprises multiple segmentation masks corresponding to the multiple perspective images of the component;

[0065] 103: pre-process the component image and the image data by using a redundant information removal method to obtain a component image with removed redundant information;

[0066] 104: perform three-dimensional reconstruction on the component image with removed redundant information by using a visual reconstruction method to acquire reconstructed component scaleless point cloud data;

[0067] 105: perform scale recovery on the component scaleless point cloud data by using a scale calibration method to acquire actual point cloud data of the component.

[0068] In the application, multiple perspective images of a component and a background image without the component are efficiently acquired, redundant information of the component image and the image data is removed, the storage space of the image and the point cloud file can be effectively reduced, the point cloud reconstruction time can be shortened, and the environmental point cloud in the point cloud data can be reduced, which is beneficial to file storage, calculation and point cloud data post-processing; by using the scale calibration method, after a component is calibrated, the scale recovery of batch component point clouds can be performed without adjusting the equipment.

[0069] In step 101, the background image without the component and the component image are acquired, and the component image comprises multiple perspective images of the component, and the acquisition of the background image without the component and the component image comprises the following steps:

[0070] Step 1011: build a multi-perspective image acquisition device around the component.

[0071] Specifically, the multi-view image acquisition device comprises a motion mechanism, a support device and an execution mechanism, the execution mechanism is arranged on the motion mechanism through the support device. The support device is used for supporting the whole acquisition device and can be arranged as a fixed support or a movable support. In this embodiment, the motion mechanism and the execution mechanism are connected through a universal hinge 2, and a connecting piece 8 is connected between the universal hinge 2 and the execution mechanism. The universal hinge 2 is arranged to enable the execution mechanism to adjust the pose. The motion mechanism comprises a track 1 with a track 5 and a programmable stepping motor 4. One end of the stepping motor 4 is fixedly connected with a gear 7 through a motor buckle 6. The track 5 is provided with a gear groove. The gear 7 is in meshing transmission with the gear groove, so that the stepping motor 4 can drive the track 5 to move on the track 1 to drive the execution mechanism to move. The execution mechanism comprises an image acquisition device with communication function, and the image acquisition device comprises a camera 3. The motion mechanism and the execution mechanism are in communication through Wifi.

[0072] It should be noted that the track 1 stroke needs to be calibrated before the background image and component image acquisition, and the number of rotations of the stepping motor 4 in the whole stroke is recorded.

[0073] In this embodiment, the track 1 is arranged in a circular shape, and the radius is 1.5 m. In other embodiments, the track 1 can also be arranged in a rectangular shape or other shapes. In addition, the track 1 can also be arranged as two straight lines arranged in parallel. The camera 3 can be used to acquire RGB images. The equivalent focal length of the camera 3 is 27 mm, and the average distance between the camera 3 and the component is 1.8 m.

[0074] Step 1012: Track 1 stroke calibration.

[0075] The track 1 stroke needs to be calibrated before the background image and component image acquisition. The multi-view image acquisition device acquires the background image and the component image at a uniform percentage stroke, which is controlled by the stepping motor 4.

[0076] Step 1013: The motion mechanism drives the execution mechanism to shoot the component, and acquires the background image without the component at zero stroke.

[0077] Specifically, one background image without the component is acquired at zero stroke.

[0078] Step 1014: According to the track 1 stroke calibration, the component image acquisition is performed at a fixed stroke increment.

[0079] Specifically, in this embodiment, the track 1 is arranged in a circular shape, and the component multi-view image acquisition is performed at a stroke increment of 1%. A total of 101 view images are acquired.

[0080] It should be noted that the camera 3 is located at the zero stroke of the multi-view image acquisition device when the background image without the component and the first view image are acquired.

[0081] On the basis of the above-mentioned embodiments, in the present embodiment, the relationship between the multi-view image acquisition device and the precision P of the component non-scale point cloud data of the reconstructed component image is determined by the field of view size F of the image acquisition device, the film size S, the average distance D between the image acquisition device and the component during the entire image acquisition process, the equivalent focal length f of the image acquisition device eq , the pixel size p of the image acquisition device, and the coefficient λ.

[0082] Specifically, the relationship is determined by the formula:

[0083] F=S 35 ×D / f eq , wherein S 35 is the 35mm film size, which is 36mm x 24mm in the present embodiment; f eq is the 35mm equivalent focal length of the image acquisition device; and λ can be verified by the embodiments and is usually taken as 5-10.

[0084] On the basis of the above-mentioned embodiments, in the present embodiment, the image data is obtained based on the background image and the component image, and the specific steps include steps 1021 to 1023:

[0085] Step 1021: sorting the images according to the shooting time.

[0086] Specifically, after collecting one background image and 101 view images, the 101 view images are sorted in the order of the first collected view image first and the last collected view image last, and the 101 view images are sorted as 1, 2, 3, …, 101.

[0087] Step 1022: obtaining the segmentation mask corresponding to the first view image according to the first rule.

[0088] Specifically, the first rule includes: using a start prompt optimization generation method and obtaining the multi-point prompt corresponding to the first view image based on the background image and the first view image; using a visual basic model and obtaining the segmentation mask corresponding to the first view image based on the first view image and the multi-point prompt corresponding thereto.

[0089] Wherein, using the start prompt optimization generation method and obtaining the multi-point prompt corresponding to the first view image based on the background image and the first view image, the specific steps include:

[0090] First, the Scale Invariant Feature Transform (SIFT) detection and matching of the background image and the first view image are performed to obtain the homography matrix M SMatching point set P in the background image M Then establish a matching point set P. M The rectangular bounding box T B Next, enclose the rectangular box T. B Divide the image into a grid to obtain the grid point set P of the background image. B It should be noted that, in this embodiment, the rectangular bounding box T is defined with a spacing of d pixels, along the direction parallel to the X-axis and the direction parallel to the Y-axis. B Divide into a grid; then use the homography matrix M S The grid set P of the background image B The coordinate system of the background image is transformed to the coordinate system of the first viewpoint image to obtain the grid point set P. E And retain the correspondence between each point in the two sets, where the two sets refer to the grid point set P of the background image. B and grid point set P E Then, grid set P is extracted from the background image and the first viewpoint image, respectively. B and P F The SIFT features are obtained by extracting grid set P from the background image and the first viewpoint image, respectively. B and P F SIFT features within d / 2 neighboring pixels of each point; then calculate P B SIFT features and P at each point F The magnitude of the vector difference of the SIFT features of corresponding points in the data, retaining P F Points whose SIFT feature vector difference magnitude is less than the threshold γ are used as the reference point set P. A denoising algorithm is then used to remove noisy points from point set P. The minimum convex hull of the denoised point set P is then constructed and eroded by n pixels to obtain the eroded convex hull T. C Finally, select points from the point set P that lie within the convex hull T. C The point within is used as the multi-point cue P corresponding to the first viewpoint image. S .

[0091] To facilitate understanding, an example is provided below, using a steel column as an example: First, input the background image and the first-view image of the steel column. Then, perform SIFT feature detection and matching on the background image and the first-view image of the steel column to obtain the homography matrix. Matching point set P in the background image M , and then establish a matching point set P M The rectangular bounding box T B The rectangular bounding box is divided into grids with d = 22 pixels along the X and Y axes to obtain the grid point set P. B Then through the homography matrix M S , grid point set PB Transforming into the first view image of the steel column, the lattice point set P is obtained F , the corresponding relationship of each point in the two sets is retained; and the lattice point sets P B and P F are extracted from the background image and the first image of the steel column respectively B SIFT features in each point d / 2=11 neighborhood pixels; then the vector difference of the SIFT features of each point and the corresponding point in P F is calculated, and the points in P F whose vector difference of the SIFT features is less than the threshold value γ=450 are retained as the reference point set P, and the statistical filtering algorithm is used to remove the noise points in the point set P; then the minimum convex hull of the denoised point set P is established and eroded by 50 pixels to obtain the eroded convex hull T C , and the points located in the convex hull T C are selected from the denoised point set P, and the output is the starting prompt point P S of the first view image of the steel column.

[0092] The first view image of the steel column and the starting prompt point P S of the first view image of the steel column are input into the vision foundation model MobileSAM, and the segmentation mask of the first view image is output.

[0093] Further, the above-mentioned vision foundation model (Vision Foundation Model) can adopt a series of open source models supporting multi-point prompts such as SegmentAnything Model (SAM), FastSAM and MobileSAM. Among them, the input data in the vision foundation model is the first view image and the corresponding multi-point prompt. The output data of the vision foundation model is the segmentation mask of the component in the first view image. In the embodiment, preferably, the vision foundation model adopted is SAM.

[0094] Step 1023: according to the second rule, obtaining the segmentation mask corresponding to the latter view image of the adjacent two view images;

[0095] Specifically, the second rule includes: using a self-prompt optimization generation method, and based on the former view image, the segmentation mask corresponding to the former view image and the latter view image of the adjacent two view images, generating the multi-point prompt corresponding to the latter view image; using a vision foundation model, and based on the latter view image and the corresponding multi-point prompt, obtaining the segmentation mask corresponding to the latter view image.

[0096] Wherein, the self-prompt optimization generation method is adopted, and based on the two adjacent view images, the previous view image, the segmentation mask corresponding to the previous view image and the next view image, the multi-point prompt corresponding to the current view image is generated, it should be noted that the two adjacent view images are input in the image acquisition order, and the specific steps include: first, the segmentation mask of the previous view image is divided into a plurality of regions along the horizontal direction and the vertical direction; then the center of each division region is extracted as a reference point set C F ; then it is verified whether each point in the point set C F is located in the segmentation mask of the previous view image, and the points in the point set C F which are not in the segmentation mask are excluded; then the SIFT features of the previous view image and the SIFT features of the next view image are extracted and matched to obtain a homography matrix M S ; finally, the homography matrix M S is multiplied by the point set C F to generate the multi-point prompt P S corresponding to the next view image.

[0097] It should be noted that the visual base model can adopt a series of open source models supporting multi-point prompts such as Segment Anything Model (SAM), FastSAM, MobileSAM, etc. The input data of the visual base model is the next view image and the corresponding multi-point prompt; the output data of the visual base model is the segmentation mask corresponding to the next view image.

[0098] Still taking the steel column as an example:

[0099] According to the sorting order, the two adjacent view images are selected, the previous view image of the steel column, the segmentation mask corresponding to the previous view image and the next view image of the steel column are input; then the segmentation mask of the previous view image of the steel column is divided into four regions along the horizontal and vertical directions, and the center of each division region is extracted as a reference point set C F ; then it is verified that each point in the point set C F is located in the segmentation mask of the previous view image of the steel column; then the SIFT features of the first view and the second view are extracted and matched to obtain a homography matrix M ; then according to the homography matrix M S , the point coordinates in the point set C F in the previous view image of the steel column are transformed to the next view image of the steel column as the self-prompt point P S of the next view image of the steel column; the next view image of the steel column and the self-prompt point P SThe input is the visual base model MobileSAM, and the output is the steel column segmentation mask of the next view image of the steel column. Based on the above steps, the segmentation mask of all view images of the collected steel column can be obtained.

[0100] Further, the starting prompt optimization generation method is applicable to the first view image; and the self-prompt optimization generation method is applicable to the next view image of the two adjacent view images without obtaining multi-point prompts.

[0101] On the basis of the above embodiment, in this embodiment, in step 103: the redundancy information removal method is used for pre-processing the component image and the image data, wherein the pre-processing of one of the view images and the corresponding image data includes the following steps:

[0102] First, the segmentation mask corresponding to the current view image is expanded by m pixels; then the image pixel RGB value outside the segmentation mask is replaced with the same value; finally, a rectangular bounding box of the segmentation mask is established, and the image within the rectangular box is cropped and retained. It should be noted that the remaining view images and their corresponding image data are also pre-processed according to the above steps until the pre-processing of the multiple view images and their corresponding image data is completed to obtain the component image with removed redundancy information.

[0103] For example: first, the segmentation mask corresponding to the current view image is expanded by 50 pixels; then the image pixel RGB value outside the segmentation mask is changed to [0, 0, 0]; finally, a rectangular bounding box of the segmentation mask is constructed, and the picture within the bounding box is cropped.

[0104] On the basis of the above embodiment, in this embodiment, in step 104: the visual reconstruction method refers to calling an open source algorithm library and using the Structure from Motion (SfM) algorithm and Multi-View Stereo (MVS) algorithm to perform three-dimensional reconstruction on the component image with removed redundancy information, and obtaining the reconstructed component scaleless point cloud data.

[0105] For example: the open source algorithms OpenMVG and OpenMVS are used for three-dimensional reconstruction of the processed multi-view images, and the original image and the processed multi-view image file size, the total number of reconstructed point clouds, the number of steel column point clouds, and the reconstruction time are compared, as shown in Table 1:

[0106]

[0107] Table 1

[0108] On the basis of the above embodiment, in this embodiment, in step 105: the scale calibration method is used to restore the scale of the component scaleless point cloud data, and the actual point cloud data of the component is obtained.

[0109] When the true geometric features of the camera's 3D pose are unavailable, a scale calibration method is used to restore the scale of the scaleless point cloud data. The specific steps include: first, inputting the measured dimensions of the component, the reconstructed scaleless point cloud data, and the camera's 3D pose; then extracting the scaleless point cloud dimension d corresponding to the measured dimensions from the scaleless point cloud data. D1 Then, according to the formula η DP =d P1 / d D1 Based on the measured dimension d of the component P1 Scale-free point cloud size d D1 Obtain the conversion factor η between the measured dimensions of the component and the dimensions of the scale-free point cloud. DP Finally, according to the formula r T =η DP ×r D1 Based on the conversion factor η DP 3D pose of the camera without scale geometric features D1 Obtain the actual geometric features r of the camera's 3D pose. T This completes the scale restoration of scale-free point cloud data.

[0110] For example, when restoring the scale of a steel column point cloud, the true geometric features of the camera's 3D pose are not obtained. The scale calibration method includes the following steps:

[0111] Input the measured height of the steel column (997mm) using a measuring tape, the reconstructed scale-free point cloud, and the camera's 3D pose output from OpenMVG; then extract the distance between the upper and lower end plates of the scale-free steel column point cloud as 289.5750mm; based on η DP =d P1 / d D1 Calculate the conversion factor η between the measured column height and the spacing between the unscaled end plates. DP =3.4430, multiply the coordinate values ​​of the point cloud of the scaleless steel column by the conversion factor η. DP The scale restoration of the point cloud of the steel column was completed.

[0112] Among them, the random sampling consensus algorithm was used to fit the circle formed by the three poses of the camera, and the scale-free radius was obtained as 405.75 mm; according to r T =η DP ×r D1 Calculate the actual radius r of the camera in 3D pose T It is 1397mm.

[0113] When the true geometric features of the camera's 3-position pose are available, a scale calibration method is used to restore the scale of scale-free point cloud data. The specific steps include: first, according to the formula η... DP =d P1 / d D1 Based on the true geometric features of the camera's 3-position pose dP1 , camera 3 pose scale-free geometric feature d D1 , obtain conversion coefficient η of measured size of component and size of scale-free point cloud DP ; finally, multiply the coordinate value of the scale-free point cloud data by the conversion coefficient η DP , complete the scale recovery of the scale-free point cloud data.

[0114] For example, the component can also be a steel beam.

[0115] The scale calibration method is used to recover the scale of the reconstructed point cloud data. Among them, the actual radius of the camera 3 pose of the steel beam point cloud scale recovery is 1397mm, and the scale calibration method includes the following steps: first, input the known camera 3 pose actual radius and the camera 3 pose output by OpenMVG; then, using the random sample consensus algorithm, the scale-free radius of the steel beam camera 3 pose is calculated as 499.65mm; then, according to η DP =d P1 / d D1 The conversion coefficient of the steel beam component camera 3 pose is 2.7960; finally, multiply the coordinate value of the reconstructed point cloud by the conversion coefficient, and complete the scale recovery of the steel beam point cloud.

[0116] It should be noted that in the above two ways, d D1 and d D1 have different meanings.

[0117] In summary, the point cloud data acquisition optimization method built in the present application can efficiently and automatically collect multiple view images of the component; by removing the redundant information of the image, the storage space of the image and the point cloud file can be effectively reduced, the point cloud reconstruction time can be shortened, and the environmental point cloud in the point cloud data can be reduced, which is beneficial to file storage, calculation and point cloud post-processing; by starting the prompt optimization generation method and the self-prompt optimization generation method, combined with the visual basic model, the component segmentation mask generation of multiple view images of different types of components can be realized without training; by using the point cloud scale calibration method, after calibrating a component, the scale of the batch component point cloud can be recovered without adjusting the equipment.

[0118] In the description of the present application, it should be noted that the terms "upper", "lower", and the like are used for indicating the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be interpreted broadly, for example, can be fixed connection, can also be detachable connection, or integral connection; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0119] It should be noted that in the present application, relational terms such as "first" and "second" and the like are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0120] The above is only a specific embodiment of the present application, which enables those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A point cloud data acquisition method, characterized in that, It includes: Acquire a background image without components and an image of the components, wherein the component images include multiple perspective images of the components; Image data is obtained based on background images and component images, wherein the image data includes multiple segmentation masks corresponding to multiple viewpoint images of the component; A redundancy removal method is used to preprocess the component image and image data to obtain a component image with redundant information removed; A visual reconstruction method is used to perform 3D reconstruction on component images with redundant information removed, and scale-free point cloud data of the reconstructed components is obtained. A scale calibration method is used to restore the scale of the scaleless point cloud data of the component and obtain the actual point cloud data of the component.

2. The point cloud data acquisition method as described in claim 1, characterized in that, Image data is obtained based on background and component images. The specific steps include: Images from different perspectives are sorted according to the time of capture. According to the first rule, obtain the segmentation mask corresponding to the first view image; According to the second rule, obtain the segmentation mask corresponding to the latter viewpoint image among two adjacent images; The first rule includes: using a startup prompt optimization generation method, and obtaining multi-point prompts corresponding to the first view image based on the background image and the first view image; using a visual basic model, and obtaining the segmentation mask corresponding to the first view image based on the first view image and its corresponding multi-point prompts; The second rule includes: using a self-prompting optimization generation method, and generating a multi-point cue corresponding to the next view image based on the previous view image, the segmentation mask corresponding to the previous view image, and the next view image in two adjacent view images; using a visual base model, and obtaining the segmentation mask corresponding to the next view image based on the next view image and its corresponding multi-point cue.

3. The point cloud data acquisition method as described in claim 2, characterized in that, An optimized method for generating startup prompts is adopted, and multi-point prompts corresponding to the first viewpoint image are obtained based on the background image and the first viewpoint image. The specific steps include: Performing scale-invariant feature transform detection and matching on the background image and the first view image to obtain a homography matrix M S and a matching point set P in the background image M ; A rectangle enclosing frame T M is established to match the point set P B ; Surrounding rectangle T B Divide into a grid to obtain a grid point set P of the background image B ; Through the homography matrix M S The grid set P of the background image B The coordinate system of the background image is transformed to the coordinate system of the first viewpoint image to obtain the grid point set P. E And retain the correspondence between each point in the two sets; SIFT features of P and P are extracted in the background image and the first view image respectively B and P F 's SIFT features Compute P B SIFT feature of each point S and the vector difference of the SIFT feature of the corresponding point in P F the length of the vector difference of the SIFT feature of each point S and the corresponding point in P, retaining P F the point set P taking the points whose length of the vector difference of the SIFT feature in P is less than the threshold value γ as reference A denoising algorithm is used to remove noise points from the point set P; A minimum convex hull of the denoised point set P is established and eroded by n pixels to obtain an eroded convex hull T C ; Screening points in the convex hull T from the point set P, taking the point as the multi-point prompt P corresponding to the first view image C . S Screening points in the convex hull T from the point set P, taking the point as the multi-point prompt P corresponding to the first view image C . S 4. The point cloud data acquisition method as described in claim 3, characterized in that: The rectangular enclosing frame T is divided into a grid along the direction parallel to the X axis and the direction parallel to the Y axis at intervals of d pixels B divided into a grid; Extract SIFT features in d / 2 neighborhood pixels of each point in P B and P F respectively.

5. The point cloud data acquisition method as described in claim 2, characterized in that, A self-prompting optimization generation method is adopted, and based on the previous viewpoint image, the segmentation mask corresponding to the previous viewpoint image, and the next viewpoint image in two adjacent viewpoint images, multi-point prompts are generated for the current viewpoint image. The specific steps include: The segmentation mask of the previous view image is used to divide the image into several regions along the horizontal and vertical directions. extracting the centroid of each divided region as a reference point set C F ; Exclusion point set C F Points in the middle not in the segmentation mask; extract SIFT features of the previous view image and SIFT features of the next view image and perform SIFT feature matching to obtain a homography matrix M S ; The homography matrix M S is multiplied with the point set C F to generate a multi-point hint P S corresponding to the next view image.

6. The point cloud data acquisition method as described in claim 1, characterized in that: In the preprocessing of component images and image data using a redundancy removal method, the preprocessing of one viewpoint image and its corresponding image data includes the following steps: Dilate the segmentation mask corresponding to the current viewpoint image by m pixels; Replace the RGB values ​​of the image pixels outside the segmentation mask with the same values; Create a rectangular bounding box for the segmentation mask, and crop the image within the rectangle while retaining the image within the bounding box.

7. The point cloud data acquisition method as described in claim 1, characterized in that: When the true geometric features of the camera pose are unavailable, a scale calibration method is used to restore the scale of scale-free point cloud data. The specific steps include: Extract the scale-free point cloud size d corresponding to the measured size from the scale-free point cloud data. D1 ; Based on the measured dimension d of the component P1 Scale-free point cloud size d D1 Obtain the conversion factor η between the measured dimensions of the component and the dimensions of the scale-free point cloud. DP ; Based on the conversion factor η DP Camera pose scale-free geometric features r D1 Obtain the actual geometric features r of the camera pose T This completes the scale restoration of scale-free point cloud data.

8. The point cloud data acquisition method as described in claim 1, characterized in that: When true geometric features of the camera pose are available, a scale calibration method is used to restore the scale of scale-free point cloud data. The specific steps include: Based on the true geometric features of camera pose d P1 , Scale-free geometric features of camera pose D1 Obtain the conversion factor η between the measured dimensions of the component and the dimensions of the scale-free point cloud. DP ; Multiply the coordinate values ​​of the scale-free point cloud data by the conversion factor η. DP This completes the scale restoration of scale-free point cloud data.

9. The point cloud data acquisition method as described in claim 1, characterized in that: Obtain the background image without components and the component images, specifically including: A multi-view image acquisition device is built around the component. The multi-view image acquisition device includes a motion mechanism, a support device, and an execution mechanism. The execution mechanism is mounted on the motion mechanism through the support device. Perform track travel calibration; The motion mechanism drives the actuator to take pictures of the components and captures a background image without components at the zero stroke point; Based on the track travel calibration, component images are acquired using fixed travel increments.

10. The point cloud data acquisition method as described in claim 9, characterized in that: The relationship between the multi-view image acquisition device and the accuracy P of the reconstructed scale-free point cloud data of the component is determined by the field of view F of the image acquisition device, the film size S, the average distance D between the image acquisition device and the component during the entire image acquisition process, and the equivalent focal length f of the image acquisition device. eq The pixel size ρ and coefficient λ of the image acquired by the image acquisition device are determined.

Images