Color point cloud map construction method and device based on optimal image selection and point cloud densification

Abstract

The present invention discloses a method and device for constructing a color point cloud map based on optimal image selection and point cloud thickening. By selecting the optimal image to color the three-dimensional point cloud and upsampling the three-dimensional point cloud based on a dynamically growing Gaussian mixture model, the color error of the three-dimensional color point cloud can be reduced, the uniformity and density of the point cloud can be improved, and the laser radar scanning traces in the three-dimensional point cloud can be effectively eliminated, thereby improving the accuracy of the color point cloud map.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional color point cloud construction, and in particular to a method and device for constructing a color point cloud map based on optimal image selection and point cloud thickening. Background Art

[0002] Traditional 3D color point cloud construction techniques typically use all candidate images to colorize the 3D point cloud within a region. The accuracy of color point cloud maps is significantly limited due to errors in camera intrinsic parameter data, inconsistent exposure parameters between multiple images, and inaccurate image pose estimation. Furthermore, traditional methods directly colorize and accumulate the raw point cloud obtained from LiDAR scanning, resulting in a large number of scanning artifacts in the resulting point cloud, reducing the uniformity and density of the point cloud distribution. Summary of the Invention

[0003] To avoid errors caused by multiple coloring of the same area during point cloud coloring and improve the color accuracy, point cloud distribution uniformity, and density of color point cloud maps, this paper proposes a method and device for constructing color point cloud maps based on optimal image selection and point cloud densification. This method evaluates the coloring accuracy index of all candidate images participating in coloring, selects the optimal image for point cloud coloring, then models the three-dimensional point cloud using a Gaussian mixture model based on dynamic growth, and performs point cloud upsampling within the Gaussian mixture model. This method effectively reduces errors caused by repeated coloring, thereby improving the accuracy of color point cloud map construction.

[0004] The object of the present invention is achieved through the following technical solution: a color point cloud map construction algorithm based on optimal image selection and point cloud thickening, the method comprising the following steps:

[0005] (1) A data acquisition system is formed by rigidly connecting a laser radar and a visible light camera. The system can collect a three-dimensional point cloud of the environment and a visible light camera image, and can calculate the visible light camera pose through the motion trajectory of the data acquisition system and the external parameters of the laser radar and the visible light camera;

[0006] (2) Extract all the plane features in the 3D point cloud to be colored, calculate the parameters of each plane feature, and save each plane feature using an octree data structure;

[0007] (3) All candidate images participating in this colorization are formed into a sliding window, and the colorization accuracy index of each candidate image is calculated according to its corresponding position and posture, and the image with the smallest index in the sliding window is selected as the optimal image;

[0008] (4) Perform Gaussian mixture model fitting based on dynamic growth, specifically:

[0009] Rasterize the 3D point cloud and perform a Gaussian mixture model fitting using the EM algorithm in each grid.

[0010] Analyze the covariance and weight of the sub-Gaussian function of the result model obtained by the first EM algorithm. If the sub-Gaussian function collapses, reduce the number of sub-Gaussian functions. If the sub-Gaussian function does not converge, increase the number of sub-Gaussian functions.

[0011] After dynamically adjusting the number of sub-Gaussian functions, the EM algorithm is used again to fit the Gaussian mixture model to obtain the global Gaussian mixture model;

[0012] (5) Perform upsampling of the 3D point cloud in the global Gaussian mixture model to increase the total number of point clouds. Specifically, starting from the mean position of each sub-Gaussian function, a fixed step size and a fixed number of samples are taken along the direction of the largest and second largest eigenvectors, and new points are generated by weighted combination.

[0013] (6) The point cloud obtained by upsampling in step (5) and the original three-dimensional point cloud are colored according to the optimal image selected in step (3) to obtain a dense colored point cloud with accurate colors.

[0014] Furthermore, the acquisition of the three-dimensional point cloud of the environment, the visible light camera image, and the visible light camera pose is specifically as follows: a data acquisition system equipped with a lidar and a visible light camera is constructed, and the three-dimensional point cloud of the environment is constructed using the simultaneous positioning and mapping algorithm of the lidar to obtain a three-dimensional point cloud map of the environment. The visible light camera pose sequence is calculated through the pose sequence of the system during the data acquisition process and the external parameters between the lidar and the visible light camera.

[0015] Furthermore, the extraction, fitting and preservation of planar features in the three-dimensional point cloud are as follows: after the data acquisition system moves a certain distance, the lidar scanning data is accumulated to obtain the three-dimensional point cloud to be colored, the image sequence taken by the visible light camera and its corresponding pose sequence are obtained, and the RANSAC algorithm is used to extract all planar features in the three-dimensional point cloud to be colored. The extraction is stopped when the number of remaining three-dimensional points in the scanning space is less than a fixed threshold.

[0016] Furthermore, assuming that M plane features are extracted in the space, the 3D point cloud set belonging to the mth plane feature is recorded as There are K points in total, calculate:

[0017]

[0018] in, is the mean position of the mth plane feature, S m is the covariance matrix of the mth plane feature; for S mPerform eigenvalue decomposition to obtain eigenvalues ​​arranged from large to small and its corresponding eigenvector Take the eigenvector corresponding to its minimum eigenvalue To represent the normal vector of the plane feature, denoted as n m ,use n m , and Q m To represent the mth plane feature, all plane features are saved as an octree, with each leaf node saving a plane feature.

[0019] Furthermore, the optimal image corresponding to the current 3D point cloud is selected for accurate coloring. Specifically, assuming that there are a total of I candidate images participating in this coloring, for the i-th image, calculate its coloring accuracy index D i , assuming that the optical axis of the i-th image and the normal vector n of the plane m m The angle is θ im , the distance from the origin of the i-th image to the plane m is h im , then the shading accuracy index of the image is:

[0020]

[0021] Where γ is the field of view of the camera, ω m is the weight coefficient of plane m, the weight coefficient and the number of points on the plane | Q m | is proportional to each image. After obtaining the shading accuracy index of each image, the image with the lowest index is selected as the optimal image for shading the 3D point cloud.

[0022] Furthermore, a Gaussian mixture model based on dynamic growth is fitted and point cloud upsampling is achieved. Specifically, the 3D point cloud is rasterized into a cube, the total number of sub-Gaussian functions in each grid is set to J, and the Gaussian mixture model is fitted once in each grid using the EM algorithm.

[0023] Assume that the mean of a sub-Gaussian function / is μ j , the covariance matrix is ​​∑ j , with weight π j , for ∑ j Perform eigenvalue decomposition to obtain eigenvalues ​​arranged from large to small and its corresponding eigenvector like Smaller and π j If the sub-Gaussian function is smaller, it is considered that the sub-Gaussian function has collapsed and needs to be deleted and J=J-1 is updated. Larger and If it is larger, it is necessary to add a sub-Gaussian function in the grid and update J=J+1;

[0024] After dynamically adjusting the number of sub-Gaussian functions, the EM algorithm is used again to fit the Gaussian mixture model and obtain the final global Gaussian mixture model.

[0025] Furthermore, the dynamic number of sub-Gaussian functions is adjusted as follows: and It is considered that the sub-Gaussian function has collapsed and needs to be deleted. If and It is necessary to add a sub-Gaussian function in the grid.

[0026] Furthermore, for the sub-Gaussian function j, from its mean μ j Start by following the eigenvector corresponding to the largest eigenvalue and the eigenvector corresponding to the second largest eigenvalue The fixed step size and fixed number of samples are taken in the direction of , and the step size is s and the number of samples is N. Then the sampling point set Q for the sub-Gaussian function j is j Expressed as:

[0027]

[0028] Finally, all the sampling points generated by the sub-Gaussian function upsampling are merged to produce a dense sampling point set Q sample , which is used to supplement the original 3D point cloud, that is, to obtain the final 3D point cloud to be colored by union with the original 3D point cloud.

[0029] Furthermore, repeated coloring of the three-dimensional point cloud of the same scanning space is avoided. Specifically, after obtaining the dense sampling point set of the current scanning space, the optimal image is used to color the union of the dense sampling point set and the original three-dimensional point cloud. In addition, an octree of pose trajectories is constructed, whose leaf nodes save the camera position corresponding to the optimal image used for each coloring. The camera position corresponding to the optimal image used for this coloring is placed in the octree. After the data acquisition system obtains a new image, it first determines whether the camera position corresponding to the current image is too close to the camera position stored in the octree. Only when the distance exceeds the threshold is the next coloring performed.

[0030] The present invention also provides a device for constructing a color point cloud map based on optimal image selection and point cloud thickening, comprising a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement the above-mentioned method for constructing a color point cloud map based on optimal image selection and point cloud thickening.

[0031] The present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-mentioned method for constructing a color point cloud map based on optimal image selection and point cloud thickening.

[0032] The present invention has the following beneficial effects: by evaluating the shading accuracy index of all candidate images involved in shading and selecting the optimal image for point cloud shading, errors caused by multiple shading of the same area during the point cloud shading process can be effectively avoided, thereby improving the color accuracy of the colored point cloud map. Furthermore, by modeling and upsampling the 3D point cloud using a Gaussian mixture model based on dynamic growth, scanning artifacts in the original point cloud can be eliminated, improving the uniformity and density of the point cloud distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0034] Figure 1 A flowchart of a method for constructing a color point cloud map based on optimal image selection and point cloud thickening is provided in one embodiment.

[0035] Figure 2 The figure is a comparison diagram showing the effects of constructing a color point cloud map using the existing method and the method of the present invention, as an exemplary embodiment.

[0036] Figure 3 The structure diagram of a device for constructing a color point cloud map based on optimal image selection and point cloud thickening is shown as an exemplary embodiment. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0038] The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the figures is not intended to limit the scope of the claimed invention, but rather merely represents selected embodiments of the present invention. All other embodiments derived by those skilled in the art based on the embodiments of the present invention without inventive effort are intended to be within the scope of protection of the present invention.

[0039] Hereinafter, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather that the present disclosure should be construed to encompass all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of the present disclosure.

[0040] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0041] Furthermore, the terms “first,” “second,” “third,” etc., are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0042] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present invention pertain. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present invention.

[0043] Figure 1 The following is a flow chart of a method for constructing a color point cloud map based on optimal image selection and point cloud thickening, provided in one embodiment. The method includes:

[0044] Step S110: Build a data acquisition system equipped with a laser radar and one or more visible light cameras, use the laser radar's simultaneous positioning and mapping algorithm to construct a three-dimensional point cloud of the environment, obtain a three-dimensional point cloud map of the environment, and calculate the visible light camera pose sequence through the system's pose sequence during the data acquisition process and the external parameters between the laser radar and the visible light camera.

[0045] In step S120, after the data acquisition system has moved a certain distance, it accumulates the LiDAR scan data to obtain a 3D point cloud to be colored. A sequence of visible light camera images and their corresponding pose sequences are acquired. A RANSAC algorithm is then used to extract all planar features from the 3D point cloud to be colored. Extraction stops when the number of remaining 3D points in the scanned space falls below a fixed threshold, W. In this embodiment, the threshold, W, is set to 20. Generally, reliable planar features cannot be extracted if the number of points is less than 20.

[0046] Assume that M plane features are extracted in the space, and the 3D point cloud set belonging to the mth plane feature is recorded as There are K points in total, calculate:

[0047]

[0048] in, is the mean position of the mth plane feature, S m is the covariance matrix of the mth plane feature. m Perform eigenvalue decomposition to obtain eigenvalues ​​arranged from large to small and its corresponding eigenvector Take the eigenvector corresponding to its minimum eigenvalue To represent the normal vector of the plane feature, denoted as n m ,use n m , and Q m To represent the mth plane feature, all plane features are saved as an octree, with each leaf node saving a plane feature.

[0049] Step S130: Select the optimal image corresponding to the current 3D point cloud for accurate coloring. Assume that there are a total of I candidate images participating in this coloring. For the i-th image, calculate its coloring accuracy index Di. Assume that the normal vector n between the optical axis of the i-th image and the plane m is m The angle is θ im , the distance from the origin of the i-th image to the plane m is h im , then the shading accuracy index of the image is:

[0050]

[0051] Where γ is the field of view of the camera, ω m is the weight coefficient of plane m, and the number of points belonging to the plane |Q m | is proportional to each image, and after obtaining the coloring accuracy index of each image, the image with the smallest index is selected as the optimal image for subsequent coloring of the three-dimensional point cloud. In this embodiment, take ω m =β| Q m |, β=0.01.

[0052] Step S140: Perform Gaussian mixture model fitting based on dynamic growth, specifically:

[0053] (1) The 3D point cloud is rasterized into a cube. The total number of sub-Gaussian functions in each grid is set to J. A Gaussian mixture model (GMM) is first fitted in each grid using the expectation-maximization (EM) algorithm. In this embodiment, the grid side length is d = 5 cm and J = 3.

[0054] (2) Assume that the mean of a sub-Gaussian function j is μ j , the covariance matrix is ​​∑ j , with weight π j , for ∑ j Perform eigenvalue decomposition to obtain eigenvalues ​​arranged from large to small and its corresponding eigenvector like Smaller and π j If the sub-Gaussian function is smaller, it is considered that the sub-Gaussian function has collapsed and needs to be deleted, that is, update J = J-1. Larger and If it is large, it is necessary to add a sub-Gaussian function in the grid, that is, update J=J+1; specifically, if and It is considered that the sub-Gaussian function has collapsed and needs to be deleted. If and Then it is necessary to add a sub-Gaussian function in the grid;

[0055] (3) After the dynamic number adjustment of the above sub-Gaussian functions is performed, the EM algorithm is used again to fit the Gaussian mixture model and obtain the final global Gaussian mixture model.

[0056] Step S150: perform upsampling of the 3D point cloud in the global Gaussian mixture model to increase the total number of point clouds, and for the sub-Gaussian function j, select the sub-Gaussian function from its mean μ j Start by following the eigenvector corresponding to the largest eigenvalue and the eigenvector corresponding to the second largest eigenvalue The fixed step size and fixed number of samples are taken in the direction of , and the step size is s and the number of samples is N. Then the sampling point set Q for the sub-Gaussian function j is j It can be expressed as:

[0057]

[0058] Finally, all the sampling points generated by the sub-Gaussian function upsampling are merged to produce a dense sampling point set Q sample , which is used to supplement the original 3D point cloud, that is, to obtain the final 3D point cloud to be colored by union with the original 3D point cloud.

[0059] In step S160, after obtaining a dense set of sampling points in the current scan space, only the optimal image is used to color the final 3D point cloud to be colored. Furthermore, an octree of pose trajectories is constructed, whose leaf nodes store the camera position corresponding to the optimal image used for each coloring. The camera position corresponding to the optimal image used for the current coloring is placed in the octree. After the data acquisition system obtains a new image, it first determines whether the camera position corresponding to the current image is too close to the camera position stored in the octree. The next coloring is performed only when the distance exceeds a threshold D; in this embodiment, the threshold D is set to 1m.

[0060] Depend on Figure 2 As can be seen, (a) shows the color point cloud generated without optimal image selection and point cloud thickening, and (b) shows the color point cloud generated using the method of the present invention with optimal image selection and point cloud thickening. It can be seen that the method of the present invention can significantly improve the accuracy of color point cloud map construction.

[0061] Corresponding to the aforementioned embodiment of the method for constructing a color point cloud map based on optimal image selection and point cloud thickening, the present invention also provides an embodiment of a device for constructing a color point cloud map based on optimal image selection and point cloud thickening.

[0062] See also Figure 3 An embodiment of the present invention provides a device for constructing a color point cloud map based on optimal image selection and point cloud thickening, comprising a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement the method for constructing a color point cloud map based on optimal image selection and point cloud thickening in the above embodiment.

[0063] The embodiment of the color point cloud map construction device based on optimal image selection and point cloud thickening of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 3 As shown, it is a hardware structure diagram of any device with data processing capability where the color point cloud map construction device based on optimal image selection and point cloud thickening of the present invention is located. Figure 3In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, typically based on the actual functions of the device with data processing capabilities, and this will not be described in detail here. The implementation process of the functions and effects of each unit in the above-mentioned apparatus is detailed in the implementation process of the corresponding steps in the above-mentioned method, and will not be described in detail here.

[0064] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for constructing a color point cloud map based on optimal image selection and point cloud thickening in the above embodiment is implemented.

[0065] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0066] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0067] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0068] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

[0069] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A method for constructing a color point cloud map based on optimal image selection and point cloud densification, characterized in that: include: (1) A data acquisition system is formed by rigidly connecting a laser radar and a visible light camera. The system can collect a three-dimensional point cloud of the environment and a visible light camera image, and can calculate the visible light camera pose through the motion trajectory of the data acquisition system and the external parameters of the laser radar and the visible light camera; (2) Extract all the plane features in the 3D point cloud to be colored, calculate the parameters of each plane feature, and save each plane feature using an octree data structure; (3) All candidate images participating in this coloring are formed into a sliding window. The coloring accuracy index of each candidate image is calculated according to its corresponding position and posture, and the image with the smallest index in the sliding window is selected as the optimal image for coloring the 3D point cloud. Specifically, assuming that there are a total of I candidate images participating in this coloring, for the i-th image, calculate its coloring accuracy index D i , assuming that the optical axis of the i-th image and the normal vector n of the plane m m The angle is θ im , the distance from the origin of the i-th image to the plane m is h im , then the shading accuracy index of the image is: Where γ is the field of view of the camera, ω m is the weight coefficient of plane m, the weight coefficient and the number of points on the plane |Q m | proportional to; (4) Perform Gaussian mixture model fitting based on dynamic growth, specifically: Rasterize the 3D point cloud and perform a Gaussian mixture model fitting using the EM algorithm in each grid. Analyze the covariance and weight of the sub-Gaussian function of the result model obtained by the first EM algorithm. If the sub-Gaussian function collapses, reduce the number of sub-Gaussian functions. If the sub-Gaussian function does not converge, increase the number of sub-Gaussian functions. After dynamically adjusting the number of sub-Gaussian functions, the EM algorithm is used again to fit the Gaussian mixture model to obtain the global Gaussian mixture model; (5) Perform upsampling of the 3D point cloud in the global Gaussian mixture model to increase the total number of point clouds. Specifically, starting from the mean position of each sub-Gaussian function, a fixed step size and a fixed number of samples are taken along the direction of the largest and second largest eigenvectors, and new points are generated by weighted combination. (6) The point cloud obtained by upsampling in step (5) and the original three-dimensional point cloud are colored according to the optimal image selected in step (3) to obtain a dense colored point cloud with accurate colors.

2. The method for constructing a color point cloud map based on optimal image selection and point cloud thickening according to claim 1, characterized in that: The acquisition of the three-dimensional point cloud of the environment, the visible light camera image, and the visible light camera pose is specifically as follows: a data acquisition system equipped with a lidar and a visible light camera is constructed, and the three-dimensional point cloud of the environment is constructed using the lidar's simultaneous localization and mapping algorithm to obtain a three-dimensional point cloud map of the environment. The visible light camera pose sequence is calculated through the system's pose sequence during the data acquisition process and the external parameters between the lidar and the visible light camera.

3. The method for constructing a color point cloud map based on optimal image selection and point cloud densification according to claim 1, characterized in that: The extraction, fitting, and preservation of planar features in the three-dimensional point cloud are as follows: after the data acquisition system moves a certain distance, the lidar scanning data is accumulated to obtain the three-dimensional point cloud to be colored, the image sequence taken by the visible light camera and its corresponding pose sequence are obtained, and the RANSAC algorithm is used to extract all planar features in the three-dimensional point cloud to be colored. The extraction is stopped when the number of remaining three-dimensional points in the scan space is less than a fixed threshold.

4. The method for constructing a color point cloud map based on optimal image selection and point cloud thickening according to claim 3, characterized in that: Assume that M plane features are extracted in the space, and the 3D point cloud set belonging to the mth plane feature is recorded as There are K points in total, calculate: in, is the mean position of the mth plane feature, S m is the covariance matrix of the mth plane feature; for S m Perform eigenvalue decomposition to obtain eigenvalues ​​arranged from large to small and its corresponding eigenvector Take the eigenvector corresponding to its minimum eigenvalue To represent the normal vector of the plane feature, denoted as n m ,use n m , and Q m To represent the mth plane feature, all plane features are saved as an octree, with each leaf node saving a plane feature.

5. The method for constructing a color point cloud map based on optimal image selection and point cloud densification according to claim 1, characterized in that: A Gaussian mixture model based on dynamic growth is fitted and point cloud upsampling is achieved. Specifically, the 3D point cloud is rasterized into a cube, the total number of sub-Gaussian functions in each grid is set to J, and the Gaussian mixture model is fitted once in each grid using the EM algorithm. Assume that the mean of a sub-Gaussian function j is μ j , the covariance matrix is ​​∑ j , with weight π j , for ∑ j Perform eigenvalue decomposition to obtain eigenvalues ​​arranged from large to small and its corresponding eigenvector like Smaller and π j If the sub-Gaussian function is smaller, it is considered that the sub-Gaussian function has collapsed and needs to be deleted and J=J-1 is updated. Larger and If it is larger, it is necessary to add a sub-Gaussian function in the grid and update J=J+1; After dynamically adjusting the number of sub-Gaussian functions, the EM algorithm is used again to fit the Gaussian mixture model and obtain the final global Gaussian mixture model.

6. The method for constructing a color point cloud map based on optimal image selection and point cloud thickening according to claim 5, characterized in that: The dynamic number of sub-Gaussian functions is adjusted as follows: and It is considered that the sub-Gaussian function has collapsed and needs to be deleted. If and It is necessary to add a sub-Gaussian function in the grid, where d is the length of the grid side.

7. The method for constructing a color point cloud map based on optimal image selection and point cloud densification according to claim 5, characterized in that: For the Gaussian function j, from its mean μ j Start by following the eigenvector corresponding to the largest eigenvalue and the eigenvector corresponding to the second largest eigenvalue The fixed step size and fixed number of samples are taken in the direction of , and the step size is s and the number of samples is N. Then the sampling point set Q for the sub-Gaussian function j is j Expressed as: Finally, all the sampling points generated by the sub-Gaussian function upsampling are merged to produce a dense sampling point set Q sample , which is used to supplement the original 3D point cloud, that is, to obtain the final 3D point cloud to be colored by union with the original 3D point cloud.

8. The method for constructing a color point cloud map based on optimal image selection and point cloud thickening according to claim 1, characterized in that: Avoid repeated coloring of the three-dimensional point cloud of the same scanning space. Specifically: after obtaining the dense sampling point set of the current scanning space, use the optimal image to color the union of the dense sampling point set and the original three-dimensional point cloud. In addition, construct a pose trajectory octree, whose leaf nodes save the camera position corresponding to the optimal image used for each coloring. The camera position corresponding to the optimal image used for this coloring is placed in the octree. After the data acquisition system obtains a new image, it first determines whether the camera position corresponding to the current image is too close to the camera position saved in the octree. Only when the distance exceeds the threshold will the next coloring be performed.

9. A device for constructing a color point cloud map based on optimal image selection and point cloud densification, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, it is used to implement the color point cloud map construction method based on optimal image selection and point cloud thickening according to any one of claims 1 to 8.

Images