A method and system for acquiring true-color 3D point clouds with depth and color fusion

Abstract

This invention discloses a method and system for acquiring true-color 3D point clouds through depth and color fusion. The method includes: setting up multiple lasers to emit lasers of specific spectral bands; after reflection from the object surface, the lasers are filtered by narrowband filters of the corresponding wavelength band, ensuring that the corresponding sensors only receive a single spectral laser line emitted by the corresponding laser emitter. The laser center point position and intensity response value are extracted using sub-pixel-level laser line thinning technology; a unified measurement coordinate system is calibrated through sensor pose calibration; based on the filter transmittance and the spectral response characteristics of the image sensor, the intensity response value is corrected and compensated, and a compensation factor is calculated to obtain the color value of the measurement point; then, the 3D coordinates are fused with the corrected color information to output true-color point cloud data, simultaneously presenting the object's depth and color information. This achieves the simultaneous acquisition of the object's 3D and color information, resulting in measurement data containing rich spectral features.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional measurement technology, and more specifically, relates to a method and system for acquiring true-color three-dimensional point clouds with minimal depth and color fusion. Background Technology

[0002] In modern industrial production and scientific research, line structured light 3D measurement technology, as a non-contact measurement method, has been widely applied due to its unique advantages. This technology primarily utilizes a specially designed laser light source to generate linear structured light, which is then projected onto the surface of the object being measured. When the structured light is projected onto surfaces of objects with different shapes, the light undergoes corresponding deformation due to the uneven surface features. This deformation directly reflects the geometric information of the object's surface. Subsequently, an image of the object's surface illuminated by the structured light is captured by a camera positioned at a specific angle. A series of image processing algorithms are then used to analyze and process the captured image, thereby calculating the 3D coordinates of the object's surface. With its significant advantages such as high precision, high efficiency, and ease of operation, line structured light 3D measurement technology plays a vital role in numerous fields, including industrial inspection, cultural relic preservation, medical imaging, and virtual reality. Because its measurement process does not require direct contact with the object being measured, it effectively avoids potential damage to fragile or dangerous objects, greatly expanding the application scope of measurement technology.

[0003] However, existing line structured light 3D measurement technology still faces many unresolved issues. Firstly, current structured light sensors are relatively limited in function, only capable of acquiring 3D measurement information of objects, with output typically consisting of geometric point cloud data containing 3D coordinates (X, Y, Z). In actual measurement, if a single-wavelength laser is used as the structured light source, the spectral limitations of this source make it difficult to excite the reflection characteristics of other visible light bands on the object's surface. This results in measurement data containing only geometric information describing the object's 3D dimensions, while a large amount of important information related to the object, such as color, texture, and material, cannot be effectively acquired, making the measurement results unable to intuitively and realistically describe the actual state of the scanned object. In practical applications, measurement data acquired based on single-wavelength structured light can only calculate the object's geometric parameters, failing to analyze the object's material through multispectral reflectance, such as distinguishing between metals and plastics or detecting coating thickness. Furthermore, it struggles to achieve automated classification tasks based on color features, such as identifying the polarity of electronic components. In applications involving joint geometric-optical analysis, it simply cannot meet the actual measurement needs.

[0004] Secondly, traditional color point cloud acquisition methods mostly rely on taking pictures with a camera, then constructing an object model through 3D reconstruction algorithms, and finally colorizing the point cloud in subsequent processing. However, this approach has significant technical flaws. Its spatial resolution and measurement accuracy are relatively low, ultimately generating only coarse model data, making it difficult to accurately colorize individual high-resolution measurement points. The accuracy of this approach is largely limited by the camera's resolution, and the lack of a precise correspondence calibration method in the matching process between camera coordinates and 3D coordinates leads to significant errors in coordinate transformation, resulting in the final color point cloud data failing to accurately reflect the true color and detailed features of the object.

[0005] In summary, existing line structured light 3D measurement technologies and color point cloud acquisition schemes have significant shortcomings in terms of the comprehensiveness, accuracy, and precision of information acquisition. With the increasing demands for the completeness and accuracy of object measurement information in industrial production and scientific research, there is an urgent need for an innovative technical solution to achieve simultaneous high-precision acquisition of both 3D and color information of objects, thereby meeting the measurement needs of increasingly complex application scenarios. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing line structured light 3D measurement technology, such as its inability to acquire object color information and the low accuracy of traditional color point cloud acquisition schemes. Through innovative design, the color intensity response value of the corresponding point is acquired simultaneously when calculating the object's depth information. After compensation and correction, the color and depth information are fused to output a high-precision true-color point cloud in real time, which truly reflects the object's texture, color, and size. This provides reliable data for multiple fields such as industrial inspection and cultural relic protection, expanding the application scope of 3D measurement technology.

[0007] To address the aforementioned deficiencies or improvement needs of existing technologies, as a first aspect of this invention, the present invention provides a method for acquiring true-color 3D point clouds with depth and color fusion, comprising:

[0008] S1. Set up multiple lasers to emit lasers of specific spectral bands. After the laser lines irradiate the surface of the object, they are reflected. Place a narrowband filter corresponding to the wavelength of the laser in front of the corresponding sensor.

[0009] S2. By replacing laser lines with light rays of single-pixel and sub-pixel width;

[0010] S3. Determine the location of the laser center point and the intensity response value at that location;

[0011] S4. By calibrating the sensor pose, the combination of the laser and the imaging system is placed in the same measurement coordinate system, so that the measurement results reflect the same depth information;

[0012] S5. Based on influencing factors including filter transmittance and image sensor spectral response characteristics, the intensity response values ​​obtained by different structured light sensors are corrected and compensated; combined with filter transmittance and quantum efficiency, the channel gain parameters are calibrated, and the compensation factor is calculated to obtain the color value of the measurement point.

[0013] S6. Fuse the 3D coordinate measurement information of each point with the corrected and compensated color information to output true color point cloud data that simultaneously contains depth and color information.

[0014] Furthermore, the specific method for solving the laser center point position and the intensity response value at that position in S3 is as follows:

[0015] Since the cross-sectional light intensity of the laser stripe on the imaging plane approximately follows a Gaussian distribution, the distribution formula is:

[0016] ,

[0017] in, σ is the pixel value of pixel x in the image, I0 is the peak intensity at the center of the stripe, which is the intensity response value at the center of the laser stripe; μ is the ordinate of the actual stripe center point, σ is the stripe width; x is the position coordinate along the cross-sectional direction of the laser stripe, used to describe the light intensity measurement point at different positions on the cross-section;

[0018] Solve using the Gaussian fitting algorithm, taking the natural logarithm of the above formula, and letting... Expanding the squared terms yields a polynomial expression. ,in:

[0019] ,

[0020] ,

[0021] In the formula, They are in different locations. x value; , It is the light intensity at the i-th measurement point;

[0022] By substituting the points near the laser line in the image, the coefficients a, b, and c can be solved using the least squares algorithm. This allows us to calculate the location of the laser center point and the intensity response value at that location.

[0023] ,

[0024] Based on the above formula, the intensity response value I0 corresponding to the coordinates μ of the center point of the laser stripe can be solved, denoted as V(λ). V(λ) is the original intensity response value output by the sensor at a specific wavelength λ in the subsequent calculation; then the coordinates μ of the center of the laser stripe in the image sensor can be calculated.

[0025] Furthermore, the specific quantification method for the factors affecting the transmittance of the filter in S5 is as follows:

[0026] The transmittance characteristic of a filter is expressed by the following formula:

[0027] ,

[0028] Where is the transmittance of T(λ) at a specific wavelength λ. It is the light intensity transmitted through the filter. It is the light intensity incident on the filter; the transmittance of the N filters is expressed as T(λ1), T(λ2) to T(λ3). N ) is used to represent this.

[0029] Furthermore, the specific quantification method for the influencing factors of the image sensor's spectral response characteristics in S5 is as follows:

[0030] The sensitivity of image sensors to different wavelengths of light varies and can be represented by quantum efficiency, QE(λ), which is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np. The specific formula is as follows:

[0031] ,

[0032] Where I is the photocurrent, h is Planck's constant, c is the speed of light, e is the elementary charge, P is the incident light power, and λ is the incident light wavelength.

[0033] Furthermore, the specific method for correcting and compensating the intensity response values ​​obtained by different structured light sensors in step S5 is as follows:

[0034] Different image sensors have their own QE-λ curves, and the quantum efficiency QE(λ) corresponding to a certain wavelength λ is obtained; the quantum efficiencies of N spectra in the image sensor are denoted as QE(λ1), QE(λ2), and so on, to QE(λ). N );

[0035] The formula for the compensated band intensity P(λ) is:

[0036] ,

[0037] Where V(λ) is the original intensity response value output by the sensor at a specific wavelength λ, T(λ) is the transmittance of the filter at a specific wavelength λ, and k(λ) is the channel gain parameter of the image sensor at a specific wavelength λ, which is related to hardware parameters including integration time and area.

[0038] Furthermore, the calculation method for the compensation factor in S5 is as follows:

[0039] ,

[0040] In the formula, S(λ) represents the intensity response value compensation factor.

[0041] Furthermore, the calibration method for the channel gain parameter in S5 is as follows:

[0042] ,

[0043] Under standard white light conditions, assuming the intensities of all bands are the same after compensation, P(λ1) = P(λ2) = P(λ). N This allows for the calibration of the channel gain parameter k(λ).

[0044] Furthermore, the method for calculating the color value of the measurement point in S5 is as follows:

[0045] The compensation factors for the N spectral intensity response values ​​are denoted as S(λ1), S(λ2), and S(λ3). N Finally, the color value of each measurement point is obtained through calculation.

[0046] ,

[0047] Finally, by combining the three-dimensional coordinate measurement information (X,Y,Z) of each point, a true color point cloud (X,Y,Z,C) with depth and color information is output.

[0048] As a second aspect of the present invention, a true-color 3D point cloud acquisition system with depth and color fusion is also provided, comprising:

[0049] A multispectral laser projection and filter receiving unit is used to set up multiple lasers to emit lasers of specific spectral bands. After the laser lines irradiate the surface of the object, they are reflected, and a narrowband filter corresponding to the wavelength of the laser is set in front of the corresponding sensor.

[0050] A laser line subpixel-level refinement unit is used to replace laser lines with light rays of single-pixel and subpixel widths;

[0051] The laser center point and intensity response calculation unit is used to solve for the location of the laser center point and the intensity response value at that location;

[0052] The measurement system pose calibration unit is used to calibrate the sensor pose, ensuring that the combination of the laser and imaging system is in the same measurement coordinate system, so that the measurement results reflect the same depth information.

[0053] The information correction and compensation and color synthesis unit is used to correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics; combined with filter transmittance and quantum efficiency, the channel gain parameters are calibrated and the compensation factor is calculated to obtain the color value of the measurement point.

[0054] The depth and color information fusion output unit is used to fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information, and output true color point cloud data that simultaneously contains depth and color information.

[0055] As a third aspect of the invention, a computer-readable storage medium is also provided, on which a computer program is stored, characterized in that the computer program is executed by a processor of any step of the depth and color fusion true color three-dimensional point cloud acquisition method.

[0056] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0057] 1. The depth and color fusion true-color 3D point cloud acquisition method of the present invention utilizes multiple lasers capable of emitting specific spectral bands. After calibration, the line lasers are projected collinearly onto the object surface. The reflected light is filtered through a narrowband filter corresponding to the laser's wavelength before entering the imaging sensor, achieving precise projection and reception of multispectral lasers. This technical feature ensures that lasers of different wavelengths accurately illuminate the same location on the object, effectively reducing ambient light interference. It allows the imaging sensor to acquire only the laser line image corresponding to the spectrum, laying the foundation for subsequent acquisition of the object's surface contour and color information. Its technical advantage lies in overcoming the limitations of traditional single-wavelength laser measurement, providing the hardware conditions for simultaneously acquiring the object's 3D and color information, and ensuring that the measurement data contains rich spectral features.

[0058] 2. The depth and color fusion-based true-color 3D point cloud acquisition method of this invention employs a laser stripe center point extraction algorithm, replacing laser lines with single-pixel and sub-pixel width rays to achieve laser line thinning. It then combines Gaussian fitting and other algorithms to solve for the laser center point position and intensity response value, achieving high-precision image acquisition and coordinate calculation. This technical feature can accurately extract the center point coordinates from laser stripes with a certain pixel width, achieving sub-pixel accuracy and significantly improving the accuracy of 3D information acquisition. Simultaneously, by calculating the intensity response values ​​of different wavelength lasers at the center point, it provides quantitative data for color information extraction. Its technical effect is reflected in a significant improvement in the spatial resolution and measurement accuracy of 3D point clouds, enabling point cloud data to more accurately reflect the geometric shape and detailed features of the object's surface.

[0059] 3. The depth and color fusion method for acquiring true-color 3D point clouds of this invention comprehensively considers factors such as filter transmittance and image sensor spectral response characteristics to correct and compensate intensity response values. It then fuses the corrected color information with 3D coordinate information to output true-color point cloud data, achieving accurate calculation of color information and fusion of depth information. This technical feature eliminates the influence of differences in filter and sensor characteristics on color acquisition by establishing a color compensation model, ensuring that intensity response values ​​of different wavelength lasers are converted into accurate color values. Its technical effect is that the generated true-color point cloud not only possesses high-precision 3D coordinates but also accurately reproduces the surface color of objects, providing high-quality data with both depth and color for fields such as industrial inspection and cultural relic protection, effectively expanding the application scenarios and practicality of 3D measurement technology. Attached Figure Description

[0060] Figure 1 This is a flowchart of a method for acquiring true-color 3D point clouds with depth and color fusion according to an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram of the measurement principle of the depth color fusion sensor according to an embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of a depth color fusion sensor according to an embodiment of the present invention;

[0063] Figure 4 This is a flowchart illustrating the operation of the depth color fusion sensor according to an embodiment of the present invention.

[0064] Figure 5 This is a system unit diagram of an embodiment of the present invention.

[0065] In all the accompanying drawings, the same reference numerals denote the same technical features, specifically: 1-object under test; 2-laser; 3-laser line; 4-filter; 5-optical lens; 6-sensor. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0067] Example 1

[0068] Please refer to Figure 1 This embodiment 1 provides a method for acquiring true-color 3D point clouds with depth and color fusion, including:

[0069] S1. Set up multiple lasers to emit lasers of specific spectral bands. After the laser lines irradiate the surface of the object, they are reflected. Place a narrowband filter corresponding to the wavelength of the laser in front of the corresponding sensor so that the corresponding sensor can only receive the laser lines of the corresponding spectrum.

[0070] S2. By replacing laser lines with light rays of single-pixel and sub-pixel width, the laser lines are refined to improve the accuracy of 3D information acquisition;

[0071] S3. Determine the location of the laser center point and the intensity response value at that location;

[0072] S4. By calibrating the sensor pose, the combination of the laser and the imaging system is placed in the same measurement coordinate system, so that the measurement results reflect the same depth information;

[0073] S5. Based on influencing factors including filter transmittance and image sensor spectral response characteristics, the intensity response values ​​obtained by different structured light sensors are corrected and compensated; combined with filter transmittance and quantum efficiency, the channel gain parameters are calibrated, and the compensation factor is calculated to obtain the color value of the measurement point.

[0074] S6. Fuse the 3D coordinate measurement information of each point with the corrected and compensated color information to output true color point cloud data that simultaneously contains depth and color information.

[0075] This embodiment 1 further elaborates on the above steps.

[0076] Please refer to Figure 2 In general, this embodiment 1 includes:

[0077] Laser Module: The laser can emit laser light in specific wavelengths as needed, ultimately outputting line lasers of different spectra. Through installation and calibration, the line lasers output by the laser module are made collinear, ensuring the same location of the object being measured.

[0078] Multiple line lasers of different spectral bands illuminate the surface of an object, are reflected, and are collected by an imaging sensor through an optical imaging lens. A filter with a specific bandwidth is installed in front of or behind the lens, and through frequency selection, only laser lines of a specific spectral band can enter the imaging sensor.

[0079] The laser image acquired by the imaging sensor is processed and calculated to extract the laser center, obtain the position of the measurement point in the image coordinate system (image coordinates), perform distance measurement calibration on the laser center coordinates, transform the image coordinates to the object coordinates, and obtain the three-dimensional information of the object to be measured.

[0080] Based on the extraction method of the laser center image coordinates, the intensity response value of the laser wavelength corresponding to the coordinate position is calculated. Multiple measurement modules can obtain multiple sets of channel data containing three-dimensional information and monochrome information for the same position in the overlapping area. After compensation and correction, the three-dimensional depth information and color information of the same point are synthesized to obtain the color information corresponding to the point, thus obtaining the true color point cloud.

[0081] Please refer to Figure 3 as well as Figure 4 In this diagram, 1 represents the object to be measured; 2 represents the laser, with three lasers in the system emitting lasers of specific spectral bands; 3 represents the emitted laser line, which propagates through space and illuminates the surface 1 of the object; 4 represents a filter of a specific wavelength, allowing only the corresponding laser wavelength light to pass through; and 5 represents an optical lens, through which the laser line of a specific wavelength can finally enter the imaging sensor 6 for subsequent data processing and fusion by the line structured light sensor.

[0082] The laser line that strikes the surface of an object is reflected and received by the optical imaging system. In order to selectively filter line lasers of different spectral bands and reduce ambient light interference, narrowband filters with bandwidths corresponding to the laser are installed in front of and behind the optical lens. The filter is an optical option that selectively allows light of a specific wavelength range to pass through, such as the desired red, green, and blue light.

[0083] Then, for different sensor units, since filters of a set wavelength are installed, only laser lines of the corresponding spectrum can be received, which are clearly imaged on the imaging sensor to form a laser stripe with a width of several pixels. The shape of the laser stripe reflects the contour features of the surface of the object to be measured.

[0084] By employing a laser stripe center point extraction algorithm, the imaging coordinates of the laser stripes in the sensor are obtained. Then, through calibration techniques, these imaging coordinates are transformed into the object-space coordinates of the object being measured. Finally, sensor pose calibration ensures that the combination of the three pairs of lasers and the imaging system is placed in the same measurement coordinate system, allowing the measurement results to reflect the same depth information.

[0085] In the processing unit, the influence of the laser line is processed to extract the precise position of the laser center line. Since the laser stripes acquired by the image sensor are usually several to tens of pixels wide, directly using the laser stripe image will inevitably cause a large amount of error and accuracy loss. Therefore, it is necessary to calculate the coordinates of the center point that can accurately represent the contour features of the object from the laser line of a certain width, and use light rays with a single pixel or even sub-pixel width to replace the laser line, thereby achieving "thinning" of the laser line and improving the accuracy of 3D information acquisition.

[0086] Furthermore, since the cross-sectional light intensity of the laser stripes on the imaging plane approximately follows a Gaussian distribution, the distribution formula is:

[0087] ,

[0088] in, is the pixel value of pixel x in the image; I0 is the peak intensity at the center of the fringe, i.e., the intensity response value at the center of the laser fringe; μ is the ordinate of the actual fringe center point; σ is the fringe width; x is the position coordinate along the cross-sectional direction of the laser fringe, used to describe the light intensity measurement point at different positions on that cross-section. The solution is obtained using a Gaussian fitting algorithm. Taking the natural logarithm of the above formula, let... Expanding the squared terms yields a polynomial expression. ,in:

[0089] ,

[0090] ,

[0091] In the formula, They are in different locations. x value; , It is the light intensity at the i-th measurement point;

[0092] By substituting the points near the laser line in the image, the coefficients a, b, and c can be solved using the least squares algorithm. This allows us to calculate the location of the laser center point and the intensity response value at that location.

[0093] ,

[0094] Based on the above formula, the intensity response value I0 corresponding to the coordinates μ of the center point of the laser stripe can be solved, denoted as V(λ). V(λ) is the original intensity response value output by the sensor at a specific wavelength λ in the subsequent calculation. Then, the coordinates μ of the center of the laser stripe in the image sensor can be calculated, with a subdivision coefficient of K. K can be 32, 64, etc. Taking 32 subdivision as an example, the subpixel coordinate accuracy obtained is 1 / 32 pixels.

[0095] The calibration algorithm obtains the object-side 3D coordinates (X, Y, Z) corresponding to the image-side coordinates in the image sensor. Normalization is performed using an M-bit color bit depth; for example, with M=8, the intensity response value ranges from 0 to 255. Before multi-channel color information fusion, factors such as the transmittance of the filter and the spectral response characteristics of the image sensor need to be comprehensively considered to correct and compensate the intensity response values ​​obtained from the N structured light sensors.

[0096] Firstly, regarding the transmittance of a filter, its transmittance characteristics can be approximated by the following formula:

[0097] ,

[0098] Where is the transmittance of T(λ) at a specific wavelength λ. It is the light intensity transmitted through the filter. This represents the light intensity incident on the filter. The transmittance of the N filters is expressed as T(λ1), T(λ2), and so on, up to T(λ...). N ) is used to represent this.

[0099] Furthermore, since image sensors exhibit varying sensitivities to different wavelengths of light, they are typically represented by quantum efficiency (QE). QE(λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np, and the specific formula is as follows:

[0100] ,

[0101] Where I is the photocurrent, h is Planck's constant, c is the speed of light, e is the elementary charge, P is the incident light power, and λ is the incident light wavelength. Different image sensors have their own QE-λ curves, from which the quantum efficiency QE(λ) corresponding to a certain wavelength λ can be obtained. The quantum efficiencies of N spectra in the image sensor are denoted as QE(λ1), QE(λ2), and so on, to QE(λ3). N ).

[0102] The formulas for the compensated band intensity P(λ) and the intensity response compensation factor S(λ) are as follows:

[0103] ,

[0104] ,

[0105] Where V(λ) is the original intensity response value output by the sensor at a specific wavelength λ, T(λ) is the transmittance of the filter at a specific wavelength λ, k(λ) is the channel gain parameter of the image sensor at a specific wavelength λ, which is related to hardware parameters such as integration time and area; QE(λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np.

[0106] In a preferred embodiment, the calibration method for the channel gain parameter is as follows:

[0107] ,

[0108] Under standard white light conditions, assuming the intensities of all bands are the same after compensation, P(λ1) = P(λ2) = P(λ). N This allows for the calibration of the channel gain parameter k(λ).

[0109] The compensation factors for the intensity response values ​​corresponding to the N spectra are denoted as S(λ1), S(λ2), and S(λ3). N Finally, the color value of each measurement point is obtained through calculation.

[0110] ,

[0111] Finally, by combining the three-dimensional coordinate measurement information (X,Y,Z) of each point, a true color point cloud (X,Y,Z,C) with depth and color information is output.

[0112] Taking the most common RGB three-channel color synthesis as an example, for red laser, the intensity response value corresponding to the sub-pixel coordinates is obtained and normalized using an N-bit color depth, denoted as V(R). Similarly, the intensity response values ​​of the other two structured light sensors are V(G) (green laser) and V(B) (blue laser). The transmittance of the filters for red, green, and blue light is denoted as T(R), T(G), and T(B), respectively. The quantum efficiencies of red, green, and blue light on the image sensor are denoted as QE(R), QE(G), and QE(B), respectively. In experiments under standard white light conditions, the channel gain parameters are calibrated to obtain k(R), k(G), and k(B), respectively. Referring to the formula for the compensation factor mentioned above, the compensation factors S(R), S(G), and S(B) for red, green, and blue light are calculated respectively, thus obtaining the color value of the measurement point.

[0113] ,

[0114] Finally, by combining the 3D coordinate measurement information (X, Y, Z) of each point, a true-color point cloud (X, Y, Z, C) synthesized with RGB colors, containing both depth and color information, is output. RGB ).

[0115] Example 2

[0116] Please refer to Figure 5. This embodiment 2 provides a true-color 3D point cloud acquisition system with depth and color fusion, including:

[0117] A multispectral laser projection and filter receiving unit is used to set up multiple lasers to emit lasers of specific spectral bands. After the laser lines irradiate the surface of the object, they are reflected. A narrowband filter corresponding to the wavelength of the laser is set in front of the corresponding sensor so that the corresponding sensor can only receive the laser lines of the corresponding spectrum.

[0118] The laser line subpixel level refinement processing unit is used to improve the accuracy of 3D information acquisition by replacing the laser line with light rays of single pixel and subpixel width.

[0119] The laser center point and intensity response calculation unit is used to solve for the location of the laser center point and the intensity response value at that location;

[0120] The measurement system pose calibration unit is used to calibrate the sensor pose so that the combination of the laser and the imaging system is in the same measurement coordinate system, so that the measurement results reflect the same depth information.

[0121] The information correction and compensation and color synthesis unit is used to correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics; combined with filter transmittance and quantum efficiency, the channel gain parameters are calibrated and the compensation factor is calculated to obtain the color value of the measurement point.

[0122] The depth and color information fusion output unit is used to fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information, and output true color point cloud data that simultaneously contains depth and color information.

[0123] Example 3

[0124] This embodiment 3 also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement any step of a method for acquiring a true-color 3D point cloud with depth and color fusion.

[0125] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0127] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for acquiring true-color 3D point clouds with depth and color fusion, characterized in that, include: S1. Set up multiple lasers to emit lasers of specific spectral bands. After the laser lines irradiate the surface of the object, they are reflected. Place a narrowband filter corresponding to the wavelength of the laser in front of the corresponding sensor. S2. By replacing laser lines with light rays of single-pixel and sub-pixel width; S3. Determine the location of the laser center point and the intensity response value at that location; S4. Through sensor pose calibration, the combination of the laser and imaging system is placed in the same measurement coordinate system, so that the measurement results reflect the same depth information; S5. Based on influencing factors including filter transmittance and image sensor spectral response characteristics, the intensity response values ​​obtained by different structured light sensors are corrected and compensated; combined with filter transmittance and quantum efficiency, the channel gain parameters are calibrated, and the compensation factor is calculated to obtain the color value of the measurement point. S6. Fuse the 3D coordinate measurement information of each point with the corrected and compensated color information to output true color point cloud data that simultaneously contains depth and color information.

2. The method for acquiring true-color 3D point clouds by depth and color fusion according to claim 1, characterized in that, The specific method for solving the laser center point position and the intensity response value at that position in S3 is as follows: Since the cross-sectional light intensity of the laser stripe on the imaging plane approximately follows a Gaussian distribution, the distribution formula is: ， in, σ is the pixel value of pixel x in the image, I0 is the peak intensity at the center of the stripe, which is the intensity response value at the center of the laser stripe; μ is the ordinate of the actual stripe center point, σ is the stripe width; x is the position coordinate along the cross-sectional direction of the laser stripe, used to describe the light intensity measurement point at different positions on the cross-section; Solve using the Gaussian fitting algorithm, taking the natural logarithm of the above formula, and letting... Expanding the squared terms yields a polynomial expression. ,in: ， ， In the formula, They are in different locations. x value; , It is the light intensity at the i-th measurement point; By substituting the points near the laser line in the image, the coefficients a, b, and c can be solved using the least squares algorithm. This allows us to calculate the location of the laser center point and the intensity response value at that location. ， Based on the above formula, the intensity response value corresponding to the coordinate μ of the center point of the laser stripe can be solved. Let V(λ) be the original intensity response value of the sensor output at a specific wavelength λ in the subsequent calculation; then the coordinates μ of the center of the laser stripe in the image sensor can be calculated.

3. The method for acquiring true-color 3D point clouds by depth and color fusion according to claim 1, characterized in that, The specific quantification method for the factors affecting the transmittance of the filter in S5 is as follows: The transmittance characteristic of a filter is expressed by the following formula: ， Where is the transmittance of T(λ) at a specific wavelength λ. It is the light intensity transmitted through the filter. It is the light intensity incident on the filter; the transmittance of the N filters is expressed as T(λ1), T(λ2) to T(λ3). N ) is used to represent this.

4. The method for acquiring true-color 3D point clouds by depth and color fusion according to claim 1, characterized in that, The specific quantification method for the influencing factors of the spectral response characteristics of the image sensor in S5 is as follows: The sensitivity of image sensors to different wavelengths of light varies and can be represented by quantum efficiency, QE(λ), which is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np. The specific formula is as follows: ， Where I is the photocurrent, h is Planck's constant, c is the speed of light, e is the elementary charge, P is the incident light power, and λ is the incident light wavelength.

5. The method for acquiring true-color 3D point clouds with depth and color fusion according to claim 3 or 4, characterized in that, The specific method for correcting and compensating the intensity response values ​​obtained by different structured light sensors in S5 is as follows: Different image sensors have their own QE-λ curves, and the quantum efficiency QE(λ) corresponding to a certain wavelength λ is obtained; the quantum efficiencies of N spectra in the image sensor are denoted as QE(λ1), QE(λ2), and so on, to QE(λ). N ); The formula for the compensated band intensity P(λ) is: ， Where V(λ) is the original intensity response value output by the sensor at a specific wavelength λ, T(λ) is the transmittance of the filter at a specific wavelength λ, k(λ) is the channel gain parameter of the image sensor at a specific wavelength λ, which is related to hardware parameters including integration time and area, and QE(λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np.

6. The method for acquiring true-color 3D point clouds with depth and color fusion according to claim 3 or 4, characterized in that, The calculation method for the compensation factor in S5 is as follows: ， In the formula, T(λ) is the transmittance of the filter at a specific wavelength λ, k(λ) is the channel gain parameter of the image sensor at a specific wavelength λ; QE(λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np, and S(λ) represents the intensity response value compensation factor.

7. The method for acquiring true-color 3D point clouds by depth and color fusion according to claim 5, characterized in that, The calibration method for the channel gain parameter in S5 is as follows: ， Under standard white light conditions, assuming the intensities of all bands are the same after compensation, P(λ1) = P(λ2) = P(λ). N This allows for the calibration of the channel gain parameter k(λ).

8. The method for acquiring true-color 3D point clouds by depth and color fusion according to claim 6, characterized in that, The method for calculating the color values ​​of the measurement points in S5 is as follows: The compensation factors for the intensity response values ​​corresponding to the N spectra are denoted as S(λ1), S(λ2), and S(λ3). N Finally, the color value of each measurement point is obtained through calculation. ， Finally, by combining the three-dimensional coordinate measurement information (X,Y,Z) of each point, a true color point cloud (X,Y,Z,C) with depth and color information is output.

9. A true-color 3D point cloud acquisition system with depth and color fusion, characterized in that, include: A multispectral laser projection and filter receiving unit is used to set up multiple lasers to emit lasers of specific spectral bands. After the laser lines irradiate the surface of the object, they are reflected, and a narrowband filter corresponding to the wavelength of the laser is set in front of the corresponding sensor. A laser line subpixel-level refinement unit is used to replace laser lines with light rays of single-pixel and subpixel widths; The laser center point and intensity response calculation unit is used to solve for the location of the laser center point and the intensity response value at that location; The measurement system pose calibration unit is used to calibrate the sensor pose so that the combination of the laser and the imaging system is in the same measurement coordinate system, so that the measurement results reflect the same depth information. The information correction and compensation and color synthesis unit is used to correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics; combined with filter transmittance and quantum efficiency, the channel gain parameters are calibrated and the compensation factor is calculated to obtain the color value of the measurement point. The depth and color information fusion output unit is used to fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information, and output true color point cloud data that simultaneously contains depth and color information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor using the method for acquiring true-color 3D point clouds with depth and color fusion as described in any one of claims 1-8.

Images