A high-precision point cloud registration method and system fusing multi-modal polarization features

Abstract

The present disclosure relates to a high-precision point cloud registration method and system fusing multi-modal polarization features, comprising: acquiring geometric point cloud data and multi-channel polarization original images of an object to be measured under multiple views; calculating linear polarization degree and polarization angle of each three-dimensional point according to the multi-channel polarization original images, obtaining polarization degree mask through polarization degree threshold binaryzation, and extracting pixel-level polarization degree gradient vector in the effective area; calculating a preliminary polarization normal according to the polarization angle, combining the initial geometric normal obtained from the geometric point cloud data as a macroscopic geometric topological prior direction, disambiguating the azimuth ambiguity of the preliminary polarization normal, and obtaining the polarization normal; taking the point clouds of different views as source point cloud and target point cloud respectively, constructing a joint energy function based on the geometric point cloud data, the pixel-level polarization degree gradient vector and the polarization normal, calculating a rigid transformation matrix by minimizing the function, and completing the point cloud registration. The present disclosure can significantly improve the accuracy and robustness of point cloud registration.

Description

Technical Field

[0001] This disclosure relates to the fields of optical three-dimensional sensing and computer vision technology, and in particular to a high-precision point cloud registration method and system that integrates multimodal polarization features. Background Technology

[0002] Point cloud registration technology is a core step in multi-view 3D measurement systems. Its purpose is to determine the rigid body transformation relationship between the coordinate systems of point clouds acquired from different viewpoints, so as to achieve seamless stitching and complete synthesis of multi-dimensional geometric information.

[0003] For 3D measurement of highly reflective, smooth metal parts, polarization filtering technology is commonly used in industry to remove high-light interference. However, while this technology improves reconstruction accuracy, it also filters out most of the spatial fluctuations in diffuse reflection intensity on the metal surface, resulting in weakened photometric information in the reconstructed point cloud and a degraded surface exhibiting weak geometric texture. The traditional Iterative Closest Point (ICP) algorithm heavily relies on local geometric residual gradients. When dealing with point clouds on smooth metal surfaces, because the geometric residual gradients approach zero tangentially, the algorithm lacks sufficient tangential constraints, easily leading to severe "tangential slip," which limits stitching accuracy and measurement reliability.

[0004] To address the aforementioned issues, existing technologies typically employ physical markers or high-precision auxiliary equipment. The former is invasive and prone to contaminating the workpiece, while the latter is costly and requires stringent environmental conditions. Furthermore, methods relying solely on geometric features or ordinary photometric images exhibit poor stability in high-gloss metallic scenes lacking surface texture. Therefore, achieving high-precision point cloud registration for highly reflective, poorly textured metallic workpieces without the need for external markers or auxiliary equipment has become a pressing technical problem in this field. Summary of the Invention

[0005] To address the challenges of traditional structured light 3D measurement on smooth metal surfaces, such as weakened geometric features due to high-light interference filtering and tangential slippage caused by the high dependence of the traditional Iterative Closest Point (ICP) algorithm on local geometric residual gradients, this disclosure proposes a high-precision point cloud registration method that integrates multimodal polarization features to solve these problems.

[0006] According to one aspect of this disclosure, a high-precision point cloud registration method integrating multimodal polarization features is provided, comprising: S10. Acquire geometric point cloud data and multi-channel polarization original image of the object under test from multiple perspectives, wherein the multi-channel polarization original image includes the original grayscale images of four polarization channels: 0°, 45°, 90° and 135°. S20. Calculate the linear polarization degree and polarization angle of each three-dimensional point based on the multi-channel polarization original image, perform binarization processing on the linear polarization degree according to the preset polarization degree threshold to obtain a polarization degree mask, and extract the pixel-level polarization degree gradient vector from the effective area of ​​the polarization degree mask. S30. Calculate the preliminary polarization normal based on the polarization angle, and calculate the initial geometric normal of each three-dimensional point based on the geometric point cloud data. Use the initial geometric normal as the macroscopic geometric topological prior direction to disambiguate the azimuth ambiguity of the preliminary polarization normal and obtain the polarization normal. S40. Take any two point clouds from different perspectives in the geometric point cloud data under the multi-view perspective as the source point cloud and the target point cloud, respectively. Construct a joint energy function based on the geometric point cloud data, the pixel-level polarization gradient vector and the polarization normal. Calculate the rigid body transformation matrix between the source point cloud and the target point cloud by minimizing the joint energy function to complete the point cloud registration.

[0007] Preferably, the polarization mask is represented as: , In the formula, For polarization mask, For linear polarization degree, This is the set effective threshold for polarization degree.

[0008] Preferably, disambiguation of the azimuth angle ambiguity of the preliminary polarization normal is performed to obtain the polarization normal, including: calculating the dot product of the preliminary polarization normal and the initial geometric normal; if the dot product is greater than or equal to zero, the preliminary polarization normal is taken as the final polarization normal; if the dot product is less than zero, the direction of the preliminary polarization normal is reversed to take the final polarization normal, expressed as: , In the formula, This is the preliminary polarization normal vector calculated based on the polarization angle. This is the vector after reversing the direction of the initial polarization normal. Let be the initial geometric normal vector. This is the final polarization normal after disambiguation.

[0009] Preferably, the joint energy function is expressed as: , In the formula, Let be the rigid body transformation matrix to be solved. For geometric point-surface constraints, The polarization gradient tangential constraint term is constructed using a first-order Taylor expansion. This is a constraint term for the polarization normal direction. and These are the weighting coefficients.

[0010] Preferably, the geometric point-surface constraint term is expressed as: , In the formula, Let be a rotation matrix. It is a translation vector. For points in the source point cloud, For the points in the target point cloud, for The initial geometric normal vector.

[0011] Preferably, the polarization gradient tangential constraint term is expressed as: , In the formula, Source point cloud midpoint The linear polarization degree value, For the target point in the cloud The linear polarization degree value, For the corresponding point in the target point cloud The transpose of the pixel-level polarization gradient vector at that location. It is the three-dimensional geometric displacement vector between the source point and the target point after rigid body transformation.

[0012] Preferably, the polarization normal direction constraint term is expressed as: , In the formula, Let be a rotation matrix. Points in the source point cloud Initial geometric normal vector, For points in the target point cloud The final polarization normal vector after disambiguation.

[0013] According to one aspect of this disclosure, a high-precision point cloud registration system integrating multimodal polarization features is provided, comprising: The data acquisition module acquires geometric point cloud data and multi-channel polarized original images of the object under test from multiple perspectives. The multi-channel polarized original images include original grayscale images of four polarization channels: 0°, 45°, 90°, and 135°. The polarization feature calculation module calculates the linear polarization degree and polarization angle of each three-dimensional point based on the multi-channel polarization original image, performs binarization processing on the linear polarization degree according to the preset polarization degree threshold to obtain a polarization degree mask, and extracts the pixel-level polarization degree gradient vector from the effective area of ​​the polarization degree mask. The normal calculation module calculates the preliminary polarization normal based on the polarization angle, and calculates the initial geometric normal of each three-dimensional point based on the geometric point cloud data. Using the initial geometric normal as the macroscopic geometric topological prior direction, it disambiguates the azimuth ambiguity of the preliminary polarization normal to obtain the polarization normal. The joint registration module takes any two point clouds from different perspectives in the geometric point cloud data under the multi-view perspective as the source point cloud and the target point cloud, respectively. Based on the geometric point cloud data, the pixel-level polarization gradient vector and the polarization normal, a joint energy function is constructed. By minimizing the joint energy function, the rigid body transformation matrix between the source point cloud and the target point cloud is calculated to complete the point cloud registration.

[0014] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: execute the above-described high-precision point cloud registration method that integrates multimodal polarization features.

[0015] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program / instructions and a bit stream thereon, wherein the computer program instructions, when executed by a processor, implement the above-described high-precision point cloud registration method that integrates multimodal polarization features to generate the bit stream.

[0016] Compared to the prior art, the beneficial effects of this disclosure are as follows: 1) This disclosure extracts the polarization degree gradient field from the multimodal polarization features and constructs anisotropic tangential constraint gradient on a smooth surface with severely degraded geometric features, thereby elevating the registration lock from the macroscopic geometric layer to the microscopic physical layer.

[0017] 2) This disclosure significantly improves the accuracy and robustness of solving the rotation matrix by introducing a polarization degree mask to isolate the low polarization noise region and combining geometric topological priors to correct the internal and external ambiguities of the zenith angle inversion of the polarization normal.

[0018] 3) This disclosure significantly narrows the optimization solution space by introducing explicit physical constraints, improves the splicing accuracy of smooth, weakly textured metal workpieces, and enables nonlinear optimization calculations to achieve both high accuracy and high efficiency.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0020] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0022] Figure 1 A flowchart of a high-precision point cloud registration method incorporating multimodal polarization features is shown in an embodiment of this disclosure. Figure 2 This diagram illustrates the reprojection error and external parameter positioning pose of the two polarization cameras and the 3LCD projector in an embodiment of this disclosure. Figure 3 The following diagram shows a comparison of the multimodal polarization characteristics of the metal parts in the embodiments of this disclosure from different viewpoints; Figure 4 This invention discloses schematic diagrams of left and right cameras, registration point clouds, and corresponding magnified local views under two measurement scenarios in embodiments of this invention. Figure 5 A comparative experimental result diagram of measurement scenario 1 in this embodiment of the present disclosure is shown; Figure 6 The diagram shows the comparative experimental results of measurement scenario 2 in this embodiment of the present disclosure; Figure 7 A block diagram of a high-precision point cloud registration system that integrates multimodal polarization features is shown in an embodiment of this disclosure. Detailed Implementation

[0023] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0024] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0025] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0026] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this disclosure, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0028] Based on the above ideas, this disclosure proposes a high-precision point cloud registration method that integrates multimodal polarization features. Figure 1 A flowchart illustrating a high-precision point cloud registration method that integrates multimodal polarization features is shown. The method includes: S10. Acquire geometric point cloud data and multi-channel polarization original image of the object under test from multiple perspectives, wherein the multi-channel polarization original image includes the original grayscale images of four polarization channels: 0°, 45°, 90° and 135°. S20. Calculate the linear polarization degree and polarization angle of each three-dimensional point based on the multi-channel polarization original image, perform binarization processing on the linear polarization degree according to the preset polarization degree threshold to obtain a polarization degree mask, and extract the pixel-level polarization degree gradient vector from the effective area of ​​the polarization degree mask. S30. Calculate the preliminary polarization normal based on the polarization angle, and calculate the initial geometric normal of each three-dimensional point based on the geometric point cloud data. Use the initial geometric normal as the macroscopic geometric topological prior direction to disambiguate the azimuth ambiguity of the preliminary polarization normal and obtain the polarization normal. S40. Take any two point clouds from different perspectives in the geometric point cloud data under the multi-view perspective as the source point cloud and the target point cloud, respectively. Construct a joint energy function based on the geometric point cloud data, the pixel-level polarization gradient vector and the polarization normal. Calculate the rigid body transformation matrix between the source point cloud and the target point cloud by minimizing the joint energy function to complete the point cloud registration.

[0029] The high-precision point cloud registration process in this embodiment, which integrates multimodal polarization features, is as follows: First, acquire two polarization cameras and a 3LCD projector (red, green, and blue 3D). The joint calibration parameters of the primary color LCD panel are used to establish the system's geometric relationship through a circular array calibration board. Subsequently, geometric point cloud data and multi-channel polarized original images of the object under test are acquired simultaneously from multiple perspectives. Based on the Stokes vector analysis of the multi-polarized channel images, the linear polarization degree (DoLP) of each three-dimensional point is calculated. A polarization degree mask is constructed to adaptively remove diffuse reflection noise regions, and pixel-level polarization degree gradient vectors are extracted as physical photometric texture features. The initial geometric normal reconstructed by structured light is used as the topological prior direction. The π ambiguity generated when calculating the polarization normal by the polarization angle is eliminated by verifying the model through the dot product sign, and a high-fidelity polarization normal with a unique direction is obtained. A joint energy function is constructed, which includes spatial Euclidean geometric point-surface constraints, polarization gradient tangential constraints to prevent translational slippage, and polarization normal direction constraints to prevent rotational deviation. By iteratively optimizing and minimizing this joint energy function, the optimal rigid body transformation matrix between the source point cloud and the target point cloud is solved, and finally, a seamless and high-precision fused point cloud is output.

[0030] This disclosure further extends the above-described method with detailed possible implementations, specifically including: S10. Acquire geometric point cloud data and multi-channel polarization original images of the object under test from multiple perspectives, wherein the multi-channel polarization original images include original grayscale images of four polarization channels: 0°, 45°, 90° and 135°.

[0031] In one embodiment, a 3D measurement system was constructed, comprising two polarization cameras (forming a binocular stereo vision system) and a 3LCD projector. Before the formal measurement, the joint calibration parameters of the two polarization cameras and the 3LCD projector were first obtained. Specifically, a high-precision circular array calibration board was used, and multiple poses were changed within the measurement space while simultaneously capturing images. By extracting the coordinates of the center pixels and establishing the spatial geometric mapping relationship between the binocular cameras and the projector, the intrinsic parameters and distortion coefficients of each camera, as well as the extrinsic parameters (rotation matrix and translation vector) between system components, were calculated. Figure 2 The reprojection error distribution and extrinsic pose diagram of two polarization cameras and a 3LCD projector are shown. Figure 2 (a) Figure 2 (c) in the diagram is a schematic diagram of the spatial external parameter positioning posture of the two polarization cameras and the 3LCD projector under the calibration conditions. Figure 2 (b) Figure 2(d) in the figure represents the reprojection error statistics under the calibration conditions, which intuitively demonstrates the high precision of the system calibration and verifies the accuracy and reliability of the obtained polarization camera and projector joint calibration parameters. This lays the geometric foundation for the accurate acquisition and registration of geometric point cloud data and multi-channel polarization image data of the object under test.

[0032] After calibration, polarization fringes are projected onto the surface of the metal part under test using the orthogonal polarization characteristics of the color combining prism inside the 3LCD projector. Two polarization cameras simultaneously acquire geometric point cloud data at corresponding viewpoints, as well as the original grayscale images of four polarization channels: 0°, 45°, 90°, and 135°. Let the acquired original grayscale images of the four polarization channels be... , , , ,in These are pixel coordinates.

[0033] S20. Calculate the linear polarization degree and polarization angle of each three-dimensional point based on the multi-channel polarization original image, perform binarization processing on the linear polarization degree according to the preset polarization degree threshold to obtain a polarization degree mask, and extract the pixel-level polarization degree gradient vector from the effective area of ​​the polarization degree mask.

[0034] In this embodiment, after acquiring the multi-polarization channel image, the Stokes vector components are analyzed pixel by pixel. , , According to the measurement principle of the Stokes vector, the polarization state of a light wave can be described by four Stokes components. For a linear polarization measurement system, light intensity is collected at four specific angles by rotating a polarizer. The Stokes vector components are calculated as follows: , , , In the formula, Indicates total light intensity. This represents the difference in linear polarization components in the horizontal / vertical directions. This represents the difference in linear polarization components along the ±45° direction.

[0035] Based on the aforementioned Stokes vector components, the degree of linear polarization of each three-dimensional point... The parsing expression is: , Due to localized oxide layers or scratches on the surface of the metal workpiece, severe depolarization occurs in some areas, preventing the Fresnel reflection law from being satisfied. Therefore, this embodiment sets an effective polarization threshold. A polarization degree mask is constructed to adaptively eliminate low polarization noise regions in diffuse reflection. The polarization degree mask is represented as follows: , In the formula, For polarization mask, For linear polarization degree, This is the set effective threshold for polarization degree.

[0036] For the effective area of ​​the polarization mask ( Extract the pixel-level polarization gradient vector of its local neighborhood, which is expressed as: , Figure 3 This is a cross-viewpoint consistency analysis diagram of the multimodal polarization characteristics of a metal part, in which... Figure 3 Image (a) in the image is the original grayscale image of the metal part captured from the left perspective. Figure 3 (e) in the image is the original grayscale image of the metal part captured from the right perspective. Figure 3 (b) Figure 3 In the image, (f) represents a pseudo-color image of the linear polarization distribution of the metal part from the corresponding viewpoint. Figure 3 (c) Figure 3 In the image, (g) represents the pseudo-color image of the polarization gradient distribution of the metal part from the corresponding viewpoint. Figure 3 (d) in Figure 3 (h) represents the multi-channel polarization fusion image of the corresponding metal part. A comprehensive analysis of the polarization characteristics of the metal part across viewing angles was completed from four dimensions: original image, degree of polarization, polarization gradient and multi-modal fusion. The figure shows that the extracted degree of polarization and its gradient features exhibit extremely high physical consistency under different viewing angles, which provides a stable and reliable anisotropic photometric texture for overcoming the geometric degradation of smooth surfaces.

[0037] S30. Calculate the preliminary polarization normal based on the polarization angle, and calculate the initial geometric normal of each three-dimensional point based on the geometric point cloud data. Use the initial geometric normal as the macroscopic geometric topological prior direction to disambiguate the azimuth ambiguity of the preliminary polarization normal and obtain the polarization normal.

[0038] In this embodiment, based on the analyzed Stokes vector components, the polarization angle (denoted as ) This can be represented as: , According to the inversion of the Fresnel equation, for a metallic surface where specular reflection is dominant (with the refractive index of the metal set as...), n ), degree of polarization and zenith The following relationship exists: , Azimuth The polarization angle obtained by measurement There are specific geometric relationships (for specular reflection) For diffuse reflection Due to the periodicity of the arctangent function, the calculated azimuth angle inevitably has [a certain characteristic]. Ambiguity (i.e., the orientation of the normal has inward / outward ambiguity). Low-frequency geometric normals initially reconstructed using structured light projection. As a priori direction for the macroscopic topological convexity and concavity trend.

[0039] Further, the ambiguity of the azimuth angle of the preliminary polarization normal is disambiguated to obtain the polarization normal, including: calculating the dot product of the preliminary polarization normal and the initial geometric normal; if the dot product is greater than or equal to zero, the preliminary polarization normal is taken as the final polarization normal; if the dot product is less than zero, the direction of the preliminary polarization normal is reversed to take the final polarization normal, expressed as: , In the formula, This is the preliminary polarization normal vector calculated based on the polarization angle. This is the vector after reversing the direction of the initial polarization normal. Let be the initial geometric normal vector. This is the final polarization normal after disambiguation. This mechanism utilizes geometric features to ensure the absolute correctness of the macroscopic normal topology, and also utilizes polarization features to endow the normal with extremely high fidelity and noise resistance.

[0040] S40. Using any two point clouds from different perspectives in the multi-view geometric point cloud data as the source point cloud and the target point cloud respectively, a joint energy function is constructed based on the geometric point cloud data, the pixel-level polarization gradient vector, and the polarization normal. The rigid transformation matrix between the source point cloud and the target point cloud is calculated by minimizing the joint energy function, thus completing point cloud registration. In this embodiment, after obtaining high-fidelity polarization gradients and polarization normals, to improve the accuracy of feature matching, any point in the source point cloud P is defined. The multimodal augmented feature vector is .in It is a three-dimensional coordinate vector. Represents a three-dimensional coordinate vector transpose, and An adaptive weighting coefficient to balance the geometric and physical dimensions.

[0041] For the source point Search for its best corresponding point in the target point cloud Q. When minimizing the mixture distance metric, it is expressed as: , In the formula, For hybrid multimodal distance metrics, In three-dimensional Euclidean coordinates, and To balance the adaptive weighting coefficients of geometric and physical dimensions This represents the pixel-level polarization gradient vector at the corresponding point. Points in the source point cloud Initial geometric normal vector, For points in the target point cloud The final polarization normal vector after disambiguation.

[0042] This metric not only ensures spatial proximity, but also eliminates pseudo-nearest neighbors with inconsistent photometric properties by using polarization gradients and excludes interfering surfaces with opposite orientations by using polarization normals, thus greatly improving the robustness of the initial matching.

[0043] In determining the set of matching point pairs The goal then is to solve for the optimal rigid body transformation matrix. ( It is the mathematical notation for three-dimensional rigid body transformation, including rotation. R Peaceful relocation t This is used to describe the optimal alignment relationship in point cloud registration. A joint energy function is constructed, which includes geometric point-surface constraints, polarization gradient tangential constraints, and polarization normal direction constraints. The joint energy function is expressed as: , In the formula, Let be the rigid body transformation matrix to be solved. For geometric point-surface constraints, The polarization gradient tangential constraint term is constructed using a first-order Taylor expansion. This is a constraint term for the polarization normal direction. and These are the weighting coefficients.

[0044] The geometric point-to-plane constraint adopts a robust point-to-plane model, allowing the point cloud to slide tangentially in the plane. The geometric point-to-plane constraint is expressed as follows: , In the formula, Let be a rotation matrix. It is a translation vector. For points in the source point cloud, For the points in the target point cloud, for The final polarization normal vector after disambiguation.

[0045] The polarization gradient tangential constraint term addresses the tangential slip that easily occurs on smooth metallic surfaces. Since the geometric normal is parallel to the tangential plane, its Jacobian matrix tends to zero in the tangential degrees of freedom (geometric degeneration). In this case, the polarization degree field of the target point cloud is considered as a continuous photometric texture, and a gradient constraint is constructed using a first-order Taylor expansion. The polarization gradient tangential constraint term is expressed as: , In the formula, Source point cloud midpoint The linear polarization degree value, For the target point cloud midpoint The linear polarization degree value, For the corresponding point in the target point cloud The transpose of the pixel-level polarization gradient vector at that location. It is the three-dimensional geometric displacement vector between the source point and the target point after rigid body transformation. It is based on the first-order Taylor expansion. By projecting the geometric displacement vector onto the polarization gradient direction of the target surface, the tangential position is anchored by utilizing the consistency of this physical feature, thereby overcoming the slip problem caused by tangential geometric degradation when relying solely on geometric information.

[0046] The polarization normal direction constraint forces the alignment of polarization normals from two viewpoints. The polarization normal direction constraint term is expressed as: , In the formula, Let be a rotation matrix. Points in the source point cloud Initial geometric normal vector, For points in the target point cloud The final polarization normal vector after disambiguation.

[0047] Since the polarization normal directly reflects the microscopic reflection geometry, this constraint can greatly improve the rotation matrix. The solution accuracy is improved, especially in scenarios where the geometric features degenerate into spheres or planes.

[0048] The extracted multimodal polarization information is fully incorporated into the joint energy function. During the iterative search, the spatial Euclidean distance, polarization gradient residual, and polarization normal angle are minimized simultaneously to perform fine registration of the multi-view point cloud and output the final high-precision registered point cloud.

[0049] Figure 4 The left and right camera views, registration point clouds, and local magnified comparison images are shown in two typical measurement scenarios (Measurement Scenario 1: long and smooth parts; Measurement Scenario 2: complex grooved metal workpieces). Figure 4(a) in the figure is the original physical image of the object in measurement scenario 1. Figure 4 (e) in the figure represents the original physical image of the object in measurement scenario 2. Figure 4 (b) Figure 4 In the diagram, (f) represents the single-view 3D point cloud image acquired by the left camera in the corresponding measurement scene. Figure 4 (c) Figure 4 In the diagram, (g) represents the single-view 3D point cloud image acquired by the right camera in the corresponding measurement scenario. Figure 4 (d) Figure 4 In the diagram, (h) represents the global point cloud map after registration and fusion using the method of this embodiment in the corresponding measurement scenario, verifying the adaptability and registration accuracy of the method for different structural parts.

[0050] Figure 5 and Figure 6 The document details the significant improvements of the proposed method compared to traditional methods in these two measurement scenarios. Figure 5 In the image, (a) shows the registration point cloud results of the parts in measurement scene 1 using traditional algorithm 1 (such as Point-to-Plane ICP). Figure 5 (b) in the figure shows the registration point cloud results of the parts in measurement scenario 1 using traditional algorithm 2 (such as GICP). Figure 5 (c) in the figure represents the registration point cloud results of the parts in measurement scenario 1 using traditional algorithm 3 (such as Robust ICP). Figure 5 In the figure (d), the registration point cloud result of the high-precision point cloud registration method that integrates multi-modal polarization features described in this embodiment is applied to the part in measurement scenario 1. The comparison intuitively demonstrates the advantages of the registration integrity and accuracy of the method in this embodiment on smooth surface parts. Figure 6 (a) in Figure 6 In the figure, (d) represents the registration point cloud results of the corresponding algorithm for the part in measurement scenario 2, which verifies the adaptability and accuracy advantage of the method in this embodiment for complex structural parts.

[0051] Tables 1 and 2 present the comparison results of point cloud registration performance of different algorithms under two measurement scenarios. It can be seen that the method of this embodiment effectively eliminates the tangential slip phenomenon caused by the tangential geometric residual approaching zero in traditional ICP, and the registration accuracy and robustness are greatly improved.

[0052]

[0053] As another aspect of the embodiments of this disclosure, a high-precision point cloud registration system 100 that integrates multimodal polarization features is also provided, such as... Figure 7 As shown, it includes: Data acquisition module 1 acquires geometric point cloud data and multi-channel polarized original images of the object under test from multiple perspectives. The multi-channel polarized original images include original grayscale images of four polarization channels: 0°, 45°, 90°, and 135°. The polarization feature calculation module 2 calculates the linear polarization degree and polarization angle of each three-dimensional point based on the multi-channel polarization original image, performs binarization processing on the linear polarization degree according to the preset polarization degree threshold to obtain a polarization degree mask, and extracts the pixel-level polarization degree gradient vector from the effective area of ​​the polarization degree mask. The normal calculation module 3 calculates the preliminary polarization normal based on the polarization angle, and calculates the initial geometric normal of each three-dimensional point based on the geometric point cloud data. Using the initial geometric normal as the macroscopic geometric topological prior direction, it disambiguates the azimuth ambiguity of the preliminary polarization normal to obtain the polarization normal. The joint registration module 4 takes any two point clouds from different perspectives in the geometric point cloud data under the multi-view perspective as the source point cloud and the target point cloud, respectively. Based on the geometric point cloud data, the pixel-level polarization gradient vector and the polarization normal, it constructs a joint energy function. By minimizing the joint energy function, it calculates the rigid body transformation matrix between the source point cloud and the target point cloud to complete the point cloud registration.

[0054] Without causing contradictions, the above-described modules in the system of the present disclosure embodiments can implement any of the above-described methods.

[0055] Based on the description of the above embodiments, it can be seen that the embodiments of this disclosure can achieve the following technical effects: 1) In this embodiment of the present disclosure, by extracting the polarization degree gradient field in the multimodal polarization features, an anisotropic tangential constraint gradient is constructed on a smooth surface with severely degraded geometric features, thereby elevating the registration lock from the macroscopic geometric layer to the microscopic physical layer.

[0056] 2) The embodiments of this disclosure introduce a polarization degree mask to isolate the low polarization noise region and combine geometric topological priors to correct the internal and external ambiguities of the zenith angle inversion of the polarization normal, which significantly improves the accuracy and robustness of solving the rotation matrix.

[0057] 3) The embodiments of this disclosure significantly narrow the optimization solution space by introducing explicit physical constraints, thereby improving the splicing accuracy of smooth, weakly textured metal workpieces and enabling nonlinear optimization calculations to achieve both high accuracy and high efficiency.

[0058] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured for the aforementioned high-precision point cloud registration method fusing multimodal polarization features. The electronic device can be provided as a terminal, a server, or other form of device.

[0059] This disclosure also proposes a computer-readable storage medium storing a computer program / instructions and a bitstream thereon. When the computer program / instructions are executed by a processor, they generate the bitstream using the aforementioned high-precision point cloud registration method that integrates multimodal polarization features. The computer-readable storage medium can be a non-volatile computer-readable storage medium.

[0060] Those skilled in the art will understand that, in the above-described high-precision point cloud registration method and system that integrates multimodal polarization features in specific embodiments, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0061] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0062] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A high-precision point cloud registration method integrating multimodal polarization features, characterized in that, include: S10. Acquire geometric point cloud data and multi-channel polarization original image of the object under test from multiple perspectives, wherein the multi-channel polarization original image includes the original grayscale images of four polarization channels: 0°, 45°, 90° and 135°. S20. Calculate the linear polarization degree and polarization angle of each three-dimensional point based on the multi-channel polarization original image, perform binarization processing on the linear polarization degree according to the preset polarization degree threshold to obtain a polarization degree mask, and extract the pixel-level polarization degree gradient vector from the effective area of ​​the polarization degree mask. S30. Calculate the preliminary polarization normal based on the polarization angle, and calculate the initial geometric normal for each 3D point based on the geometric point cloud data. Using the initial geometric normal as the macroscopic geometric topological prior direction, disambiguate the azimuth ambiguity of the preliminary polarization normal to obtain the polarization normal. Disambiguating the azimuth ambiguity of the preliminary polarization normal to obtain the polarization normal includes: calculating the dot product of the preliminary polarization normal and the initial geometric normal; if the dot product is greater than or equal to zero, then the preliminary polarization normal is used as the final polarization normal; if the dot product is less than zero, then the direction of the preliminary polarization normal is reversed to use as the final polarization normal, expressed as: ， In the formula, This is the preliminary polarization normal vector calculated based on the polarization angle. This is the vector after reversing the direction of the initial polarization normal. Let be the initial geometric normal vector. This is the final polarization normal after disambiguation; S40. Taking any two point clouds from different perspectives in the multi-view geometric point cloud data as the source point cloud and the target point cloud respectively, a joint energy function is constructed based on the geometric point cloud data, the pixel-level polarization gradient vector, and the polarization normal. The rigid body transformation matrix between the source point cloud and the target point cloud is calculated by minimizing the joint energy function to complete the point cloud registration. The joint energy function is expressed as: ， In the formula, Let be the rigid body transformation matrix to be solved. For geometric point-surface constraints, The polarization gradient tangential constraint term is constructed using a first-order Taylor expansion. This is a constraint term for the polarization normal direction. and These are the weighting coefficients.

2. The method according to claim 1, characterized in that, The polarization mask is represented as follows: ， In the formula, For polarization mask, For linear polarization degree, This is the set effective threshold for polarization degree.

3. The method according to claim 1, characterized in that, The geometric point-surface constraint term is expressed as follows: ， In the formula, For rotation matrix, It is a translation vector. For points in the source point cloud, For the points in the target point cloud, for The initial geometric normal vector.

4. The method according to claim 1, characterized in that, The polarization gradient tangential constraint term is expressed as follows: ， In the formula, Source point cloud midpoint The linear polarization degree value, For the target point in the cloud The linear polarization degree value, For the corresponding point in the target point cloud The transpose of the pixel-level polarization gradient vector at that location. It is the three-dimensional geometric displacement vector between the source point and the target point after rigid body transformation.

5. The method according to claim 1, characterized in that, The polarization normal direction constraint term is expressed as follows: ， In the formula, For rotation matrix, Points in the source point cloud Initial geometric normal vector, For points in the target point cloud The final polarization normal vector after disambiguation.

6. A high-precision point cloud registration system integrating multimodal polarization features, characterized in that, include: The data acquisition module acquires geometric point cloud data and multi-channel polarized original images of the object under test from multiple perspectives. The multi-channel polarized original images include original grayscale images of four polarization channels: 0°, 45°, 90°, and 135°. The polarization feature calculation module calculates the linear polarization degree and polarization angle of each three-dimensional point based on the multi-channel polarization original image, performs binarization processing on the linear polarization degree according to the preset polarization degree threshold to obtain a polarization degree mask, and extracts the pixel-level polarization degree gradient vector from the effective area of ​​the polarization degree mask. The normal calculation module calculates a preliminary polarization normal based on the polarization angle and calculates an initial geometric normal for each 3D point based on the geometric point cloud data. Using the initial geometric normal as a macroscopic geometric topological prior direction, it disambiguates the azimuth ambiguity of the preliminary polarization normal to obtain the polarization normal. Disambiguating the azimuth ambiguity of the preliminary polarization normal to obtain the polarization normal includes: calculating the dot product of the preliminary polarization normal and the initial geometric normal; if the dot product is greater than or equal to zero, the preliminary polarization normal is used as the final polarization normal; if the dot product is less than zero, the direction of the preliminary polarization normal is reversed to obtain the final polarization normal, expressed as: ， In the formula, This is the preliminary polarization normal vector calculated based on the polarization angle. This is the vector after reversing the direction of the initial polarization normal. Let be the initial geometric normal vector. This is the final polarization normal after disambiguation; The joint registration module takes any two point clouds from different perspectives in the multi-view geometric point cloud data as the source point cloud and the target point cloud, respectively. Based on the geometric point cloud data, the pixel-level polarization gradient vector, and the polarization normal, it constructs a joint energy function. By minimizing the joint energy function, it calculates the rigid transformation matrix between the source and target point clouds, thus completing the point cloud registration. The joint energy function is expressed as: ， In the formula, Let be the rigid body transformation matrix to be solved. For geometric point-surface constraints, The polarization gradient tangential constraint term is constructed using a first-order Taylor expansion. This is a constraint term for the polarization normal direction. and These are the weighting coefficients.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the high-precision point cloud registration method that integrates multimodal polarization features as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program / instructions and a bit stream thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the high-precision point cloud registration method according to any one of claims 1-5 to generate the bit stream.

Images