Deformed surface-oriented position sensing mark and reconstruction method thereof

Abstract

The invention provides a position sensing mark for a deformed surface and a reconstruction method thereof, the mark is formed by arranging a plurality of positioning dots in a honeycomb shape, a hexagon is used as a self-recognition unit, and the colors of the positioning dots are selected from a stable color set calibrated by multiple brightness levels. Generating a unique ID according to the color features of the positioning dots and a preset ID reading sequence; the reconstruction method comprises the steps of image acquisition, feature detection, grid reconstruction, ID recognition, three-dimensional reconstruction and experimental verification, and high-precision reconstruction is realized in combination with binocular vision and interpolation correction technologies. According to the method, the dependence of traditional matrix coding on a rigid surface is broken through, the mark field is generated through graph theory modeling, the brightness interference resistance is improved through color calibration, the coding value matching process is not affected by adjacent dots, and the method is suitable for the fields of non-planar object positioning, dynamic garment capturing, AR and the like and has the advantages of being stable in recognition, high in reconstruction precision and high in robustness.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and 3D reconstruction technology, specifically relating to a position-aware marker for deformable surfaces and its reconstruction method. Background Technology

[0002] Deformable surface reconstruction is a core technology in computer vision and robotics, requiring precise tracking of surface points when surfaces undergo deformations such as bending, wrinkling, folding, and stretching. This technology is widely used in scenarios such as non-planar object localization, clothing motion capture, augmented reality, and haptic technology.

[0003] Existing deformable surface reconstruction technologies mainly include three categories: piezoelectric thin films, structured light, and visual positioning markers. Each has significant application limitations: piezoelectric thin films rely on edge-deployed sensors, resulting in insufficient reconstruction accuracy for subtle deformations in the central area, and the sensors are prone to fatigue damage under frequent or extreme deformations; structured light technology is constrained by lighting conditions and scene textures, making it difficult to effectively handle non-rigid deformations; while visual positioning markers have the advantages of low cost and fast response, traditional markers have obvious defects: first, they have weak resistance to deformation, and self-identifying markers and position-aware markers are prone to boundary loss due to physical deformation in practical applications, and factors such as low marker density and excessively large self-identifying unit size will further exacerbate this problem, leading to recognition failure; second, they are prone to color distortion due to changes in brightness, printer characteristics, paper texture, camera sensitivity, and other factors, interfering with the accuracy of color recognition.

[0004] Specifically, the current problems with visual positioning markers are concentrated in two core scenarios: boundary loss and brightness sensitivity. Regarding boundary loss, although position-aware markers can cover several times more feature points than self-identified markers in the same area due to their higher positioning feature density, thus significantly mitigating the impact of boundary loss, deformation of flexible surfaces can easily lead to marker shape distortion and feature point displacement. Complex curved surfaces also have problems such as occlusion and perspective distortion, which can still cause some markers to fail to be detected. Therefore, the design and detection of position-aware markers has become a research focus. Regarding brightness sensitivity, the color distortion caused by brightness sensitivity, together with hardware characteristics and ambient lighting, can seriously interfere with the accurate recognition of marker colors, further limiting the applicability of traditional visual positioning markers.

[0005] In view of this, to address the problems of existing visual markers having weak resistance to deformation and poor sensitivity to brightness, this invention designs a position-aware marker for deformable surfaces. This marker generates triangular markers based on a stable color set calibrated with multiple brightness levels, possessing stronger resistance to brightness sensitivity and deformation, providing robust technical support for tracking deformable surfaces. Summary of the Invention

[0006] To address the problems of boundary loss, brightness sensitivity, and poor coding adaptability of existing visual positioning markers in deformable surface applications, this invention provides a position-aware marker and its reconstruction method for deformable surfaces. By matching the color features of the positioning dots with a stable color set to generate codes, and combining IDs in a specific order, high-precision and robust surface reconstruction can be achieved in complex deformation scenarios.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows: A position-sensing marker for deformable surfaces, the marker being formed by a plurality of positioning dots arranged in a honeycomb pattern; and using a hexagon formed by six positioning dots surrounding a central positioning dot as a self-identifying unit; The color of the positioning dot is selected from a stable color set obtained through color calibration; each reference color in the stable color set has a visual difference with the background color that can be stably distinguished by the machine vision system, and the reference colors satisfy the constraint condition of being farthest apart in the RGB space, thereby ensuring the stability and distinguishability of color recognition. Each self-identifying unit generates a unique self-identifying unit ID by using a preset ID reading order and the color characteristics of each positioning dot within the unit.

[0008] Preferably, the color feature is the inherent encoded identifier of the reference color corresponding to the positioning dot in a stable color set; each reference color in the stable color set corresponds to a unique encoded value. After the positioning dot matches the target reference color, the encoded value of the target reference color is assigned to the positioning dot. Finally, the self-identification unit ID is generated by combining the encoded values ​​of each positioning dot in the self-identification unit with a preset ID reading order. This method of independently encoding based on its own color makes the encoding process unaffected by the colors of adjacent dots, thus improving the robustness of the system.

[0009] Preferably, the stable color set is obtained through the following steps (the core objective of which is to obtain a color set whose color performance remains stable under various different brightness environments): Sample each channel of the RGB color space to generate an initial color palette with the background color as the base color and containing multiple color samples; acquire the actual RGB values ​​of the color samples of the initial color palette at multiple different brightness levels using an image acquisition device (under the condition that the hardware such as printer, camera, and paper are consistent). The actual RGB values ​​of the collected color samples are mapped to a simplified, discrete three-dimensional square reference color grid (such as a 3x3x3 grid). Color samples that can be stably mapped to the same reference point at various brightness levels are selected and marked as the color category represented by the reference point. Color samples that are similar to the background color are removed based on the similarity between the color sample mapping reference point and the background color mapping reference point. Calculate the point cloud density of each color sample in the RGB space, and extract the color sample with the highest density in each category as the color candidate; The optimization objective is to maximize the distance between the selected color combinations in the RGB space and the distance from the background color. Through a multi-objective optimization algorithm, the performance of each candidate color combination is simulated under various brightness levels, and a comprehensive stability score is calculated based on predefined illumination weights. The globally optimal color combination that meets the requirements of color distinguishability and stability is selected from the color candidates to form the stable color set.

[0010] Preferably, the preset ID reading order is as follows: for any self-identifying unit, read the upper left positioning point, upper right positioning point, right positioning point, lower right positioning point, lower left positioning point, and left positioning point in sequence, and finally read the center point; extract the encoding values ​​of each positioning point in sequence and combine them to form the self-identifying unit ID.

[0011] A method for reconstructing deformable surfaces based on any of the above-mentioned position-aware markers, characterized by comprising the following steps: S1: Acquire real-time images of the deformable surface with the position-aware markers attached using a binocular camera; S2: Perform feature detection on the acquired image, identify all positioning points and obtain their centroid coordinates; S3: Based on the centroid coordinates, perform meshing processing, filter and reconstruct a mesh composed of multiple hexagonal self-identifying units; S4: Identify the ID of each hexagonal self-identification unit and obtain the three-dimensional coordinates of each positioning point based on the principle of binocular vision triangulation; S5: Based on the three-dimensional coordinates, the three-dimensional reconstruction of the deformed surface is achieved through surface fitting and interpolation correction.

[0012] Preferably, S2 specifically includes: S21: A low-pass filter is used to blur the image in order to suppress surface texture interference; S22: Convert the blurred image to a grayscale image and obtain a binary image using an adaptive threshold segmentation algorithm; S23: Perform a morphological opening operation on the binary graph and calculate the centroid of the connected region to obtain the centroid coordinates of the positioning point.

[0013] Preferably, S3 specifically includes: S31: Triangulate the centroid coordinates to obtain an initial mesh composed of triangles; S32: Select grid cells that conform to a hexagonal structure from the initial grid; S33: Integrate the filtered hexagonal grid cells to complete grid reconstruction.

[0014] Preferably, the specific steps for identifying the ID of each hexagonal self-identification unit in S4 are as follows: for each hexagonal self-identification unit, the actual RGB value of the positioning point is matched with the RGB value of each reference color in the stable color set to determine the encoding value of the positioning point; the encoding value of each positioning point is extracted based on the preset ID reading order, and the initial ID candidate result is generated by combining them. In this system, each of the positioning dots is simultaneously associated with seven hexagonal self-identification units. The correctness of the dot's encoding value is verified by statistically matching the encoding values ​​of the positioning dot with the seven associated self-identification units. If there is ambiguity in the initial ID candidate results, a voting mechanism is used to determine the correct encoding value of the positioning dot based on the matching encoding values ​​of the seven associated self-identification units, thereby correcting and determining the optimal ID of the hexagonal self-identification unit. This significantly improves the accuracy and fault tolerance of the identification.

[0015] Preferably, the location-aware markers in S1 are prepared based on a marker field generated by a graph theory model, and the generation of the marker field includes the following steps: Based on the 3D model of the target deformable surface, a honeycomb topological network of the target deformable surface is constructed, which is defined as a simple embedding graph F=(V, E, C). The vertex set V contains the theoretical 3D coordinates of the positioning points and the label information to be assigned. The edge set E describes the spatial adjacency relationship between vertices. The face set C defines the local geometric structure through the surrounding order of the vertices. The basic unit of the face set C is a hexagon and constitutes a self-identifying unit. Based on the VF3 graph isomorphism matching principle, subgraphs isomorphic to hexagonal self-identifying units are extracted from graph F as candidates, and non-coplanar candidate subgraphs are filtered based on the consistency of surface normal direction. The label field is generated by assigning color-coded labels to each vertex in the vertex set V using a risk minimization strategy and resolving conflicts using a backtracking mechanism.

[0016] Preferably, the method of assigning color-coded labels to each vertex in the vertex set V using a risk minimization strategy is as follows: The vertex with the most associated candidate subgraphs is assigned a label first, and the optimal label is selected according to the label security function S(Li)=∑logS(Ar), where S(Ar) represents the estimated probability that the r-th hexagonal cell associated with the current vertex can eventually form a globally unique ID under the condition that the label Li is assigned to the current vertex.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention improves the structural stability of markers under deformation scenarios through a honeycomb triangular mesh structure design, breaking through the dependence of traditional matrix coding on rigid surfaces. The stable color set calibrated with multiple brightness levels effectively resists color recognition errors caused by uneven ambient brightness and hardware differences. The ID recognition mechanism, which generates codes independently based on the color characteristics of each dot and combines this with collaborative verification between adjacent units, significantly improves the system's robustness and accuracy. By combining binocular vision with high-precision interpolation correction technology, the reconstruction results achieve sub-pixel accuracy, meeting the application requirements for high-precision reconstruction of deformable surfaces in fields such as industrial inspection, medical navigation, and human-computer interaction. Attached Figure Description

[0018] Figure 1 This is a flowchart of the location-aware marker and its reconstruction method of the present invention; Figure 2 This is a schematic diagram of the position-aware marker of the present invention; Figure 3 This is a schematic diagram of the stable color set of the present invention; Figure 4 This is a schematic diagram of the tag recognition process of the present invention; Figure 5 This is a schematic diagram of the three-dimensional reconstruction effect of the present invention. Detailed Implementation

[0019] The following is in conjunction with the appendix Figure 1-5 The technical solution of the present invention will be described in detail below.

[0020] This invention proposes a position-aware marker and its reconstruction method for deformable surfaces, as detailed below: 1. Obtaining a stable color set Obtaining the stable color set is a systematic color calibration process aimed at selecting color combinations that are insensitive to changes in hardware and lighting. A specific embodiment is as follows: (1) Sample 20 values ​​evenly in each channel of the RGB color space to generate an initial color palette containing 8000 (20^3) color samples.

[0021] (2) Using a fixed combination of printer, camera and paper, the actual RGB values ​​of each color in the color palette were collected under 20 different brightness levels.

[0022] (3) Establish a 3×3×3 three-dimensional reference color grid L, and define its 27 center points as reference colors. For the color collected at each brightness level, find the reference color with the closest Euclidean distance in the RGB space and establish a mapping relationship.

[0023] (4) Filtering and noise reduction: For a color sample in the initial color palette, if it consistently maps to the same reference point across all 20 brightness levels, the color sample is retained, and its type is marked with the reference point number. At the same time, color samples that map to reference points similar to the background color and their related types are removed.

[0024] (5) Density analysis and candidate extraction: For each type of color sample selected in step (4), calculate the point cloud density formed by all RGB values ​​in three-dimensional space at 20 brightness levels, and extract the color sample with the largest point cloud density in that type as the stability representative of that color type and add it to the color candidate set.

[0025] (6) Multi-objective optimization to determine the final set: The optimization objective is to maximize the distance between the finally selected color combinations (e.g., 7 colors) in the RGB space (discrimination) and also maximize the distance from the white background. By enumerating all possible color combinations (e.g., selecting 7 from N candidate colors) and calculating their comprehensive performance at 20 brightness levels (introducing illumination weights), the combination with the highest comprehensive score is selected as the final stable color set K, and its corresponding reference color center point set is denoted as Q.

[0026] 2. Marker Field Generation Position-aware labeling is decomposed into two parts: a back-end labeling field and a front-end representation. The theoretical framework for generating the back-end labeling field is based on graph theory and is a mathematical model. It is defined as a simple embedding graph F=(V, E, C) that satisfies specific conditions. The vertex set V contains coordinate and label information, the edge set E describes spatial adjacency relationships, and the face set C defines the local geometric structure through the vertex wrapping order. The front-end representation transforms the abstract graph structure into a detectable visual pattern through the collaborative design of D-pattern (position features) and L-pattern (label features).

[0027] The core technologies of the generation algorithm include: (1) Isomorphic subgraph extraction: The improved VF3 algorithm is used to extract all subgraphs that are isomorphic to the shape G of the ID label, and non-coplanar isomorphisms are filtered by the consistency of the surface normal direction.

[0028] (2) Constraint label assignment: Assign color labels to vertices (corresponding to the codes in the stable color set). Construct an index matrix D and a label matrix L. Solve the vertex selection and label assignment problem through a risk minimization strategy. Prioritize assigning the vertex with the most associated subgraphs, and select the optimal label based on the label safety S(Li)=∑logS(Ar).

[0029] (3) Flexible backtracking mechanism: Conflict resolution is managed through a state checklist, and a last-in-first-out strategy is adopted to adjust the label allocation to ensure that the algorithm converges within a reasonable time.

[0030] 3. Image Feature Detection The steps for accurately extracting the centroid of a localization point from images acquired by a stereo camera are as follows: Preprocessing: A Gaussian low-pass filter is used to blur the original image to suppress high-frequency noise interference caused by the surface texture of the flexible material.

[0031] Binarization: The blurred image is converted into a grayscale image, and an adaptive threshold segmentation algorithm is used to obtain a binary image, initially separating the foreground (dots) from the background.

[0032] Morphological optimization and centroid extraction: Morphological opening operations are performed on the binary image to eliminate minor noise and smooth the dot boundaries. Subsequently, the regionprops function from the image processing library is used to calculate the centroid coordinates of each connected region (i.e., each dot), forming a centroid coordinate matrix.

[0033] 4. Mesh Reconstruction and ID Recognition Mesh reconstruction: The obtained centroid coordinate matrix is ​​triangulated (e.g., Delaunay triangulation) to form a triangular mesh. Based on the honeycomb structure characteristics, mesh structures in which each internal point connects exactly six adjacent points are selected, thus restoring the hexagonal self-identifying cell mesh, and outliers that do not conform to this topological relationship are removed.

[0034] ID recognition: Encoding value matching: For each identified location point, its actual RGB value is matched with a reference color in the stable color set K (e.g., the minimum Euclidean distance is calculated), and the encoding value corresponding to the matched reference color is assigned to the point.

[0035] Initial ID generation: For each hexagonal self-identification unit, the encoded values ​​of its 7 dots are read sequentially in a preset order (top left → top right → right side → bottom right → bottom left → left side → center), and then concatenated to generate an initial ID candidate.

[0036] Collaborative Verification and Correction: Since each dot simultaneously belongs to a maximum of 7 self-identifying units (centered on itself, and as a vertex of 6 adjacent units), when the initial encoding value of a dot is uncertain, the system checks the consistency of that encoding value across all 7 units to which it belongs. By counting the number of successful matches when it plays different roles in different units, a voting mechanism is used to determine the most likely correct encoding value for that dot, thereby correcting the IDs of the relevant units and ultimately determining the optimal ID.

[0037] 5. Reconstruction and Experimental Verification 3D coordinate acquisition: Based on the identified ID and the corresponding image point pair, the coordinates of each positioning point in 3D space are calculated using the principle of binocular vision triangulation, forming a discrete 3D point cloud.

[0038] Surface fitting and interpolation correction: Initial modeling: Moving least squares method is used to fit the surface of the discrete point cloud. A local neighborhood is defined for each data point, and a local surface patch that fits the local point cloud is constructed by weighted least squares calculation. Then, all surface patches are stitched together to form a continuous initial deformable surface model.

[0039] Refinement and Adaptation: To address potential noise, camera distortion, or feature point localization deviations in the initial model, a spline interpolation algorithm (such as thin-plate splines) is employed for fine-tuning. This algorithm constructs a smooth and continuous surface based on the 3D coordinates of adjacent points, effectively adapting to nonlinear deformations such as bending and wrinkling, filling data gaps, and correcting local deviations.

[0040] Performance control: Through the aforementioned color calibration process, the color of the positioning dots is made consistent across different devices, with a typical target of cross-device color deviation ΔE ≤ 2.5. The final reconstruction system can achieve a reprojection error of no more than 0.5 pixels (sub-pixel accuracy). In scenarios with surface curvature changes ≥ 0.1 / cm, the spatial position error of its 3D reconstruction can be controlled within ≤ 1.2mm, meeting the requirements of high-precision applications.

[0041] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions that fall within the principles of the present invention are within the scope of protection of the present invention. For those skilled in the art, any improvements made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A position-aware marker for a deformable surface, characterized in that, The mark is formed by a number of positioning dots arranged in a honeycomb pattern; and a hexagon formed by six positioning dots surrounding a central positioning dot is used as a self-identifying unit. The color of the positioning dot is selected from a stable color set obtained through color calibration; each reference color in the stable color set has a visual difference with the background color that can be stably distinguished by the machine vision system, and the reference colors satisfy the constraint condition of being farthest apart in the RGB space. Each self-identifying unit generates a unique self-identifying unit ID by using a preset ID reading order and the color characteristics of each positioning dot within the unit.

2. The position-aware marker facing a deformable surface according to claim 1, characterized in that, The color feature is the inherent encoding identifier of the reference color corresponding to the positioning dot in the stable color set; each reference color in the stable color set corresponds to a unique encoding value. After the positioning dot matches the target reference color, the encoding value of the target reference color is assigned to the positioning dot. Finally, the self-identification unit ID is generated by combining the encoding values ​​of each positioning dot in the self-identification unit with the preset ID reading order.

3. The position-aware marker facing a deformable surface according to claim 1, characterized in that, The stable color set is obtained through the following steps: Sample each channel of the RGB color space to generate an initial color palette with the background color as the base color and containing multiple color samples; collect the actual RGB values ​​of the color samples of the initial color palette at multiple different brightness levels using an image acquisition device; The actual RGB values ​​of the collected color samples are mapped to a three-dimensional square reference color grid. Color samples that can be stably mapped to the same reference point under various brightness levels are selected and marked as the color category represented by the reference point. Color samples that are similar to the background color are removed based on the similarity between the color sample mapping reference point and the background color mapping reference point. Calculate the point cloud density of each color sample in the RGB space, and extract the color sample with the highest density in each category as the color candidate; The optimization objective is to maximize the distance between the selected color combinations in the RGB space and the distance from the background color. Through a multi-objective optimization algorithm, the performance of each candidate color combination is simulated under various brightness levels, and a comprehensive stability score is calculated based on predefined illumination weights. The globally optimal color combination that meets the requirements of color distinguishability and stability is selected from the color candidates to form the stable color set.

4. The position-aware marker facing a deformable surface according to claim 1, characterized in that, The preset ID reading order is as follows: for any self-identifying unit, read the upper left positioning circle, upper right positioning circle, right positioning circle, lower right positioning circle, lower left positioning circle, and left positioning circle in sequence, and finally read the center circle; extract the encoding values ​​of each positioning circle in sequence and combine them to form the self-identifying unit ID.

5. A method for reconstructing a deformable surface based on a position-aware marker according to any one of claims 1-4, characterized in that, Includes the following steps: S1: Acquire real-time images of the deformable surface with the position-aware markers attached using a binocular camera; S2: Perform feature detection on the acquired image, identify all positioning points and obtain their centroid coordinates; S3: Based on the centroid coordinates, perform meshing processing, filter and reconstruct a mesh composed of multiple hexagonal self-identifying units; S4: Identify the ID of each hexagonal self-identification unit and obtain the three-dimensional coordinates of each positioning point based on the principle of binocular vision triangulation; S5: Based on the three-dimensional coordinates, the three-dimensional reconstruction of the deformed surface is achieved through surface fitting and interpolation correction.

6. The reconstruction method for deformable surfaces according to claim 5, characterized in that, S2 specifically includes: S21: A low-pass filter is used to blur the image in order to suppress surface texture interference; S22: Convert the blurred image to a grayscale image and obtain a binary image using an adaptive threshold segmentation algorithm; S23: Perform a morphological opening operation on the binary graph and calculate the centroid of the connected region to obtain the centroid coordinates of the positioning point.

7. The reconstruction method for deformable surfaces according to claim 5, characterized in that, S3 specifically includes: S31: Triangulate the centroid coordinates to obtain an initial mesh composed of triangles; S32: Select grid cells that conform to a hexagonal structure from the initial grid; S33: Integrate the filtered hexagonal grid cells to complete grid reconstruction.

8. The reconstruction method for deformable surfaces according to claim 5, characterized in that, The specific steps for identifying the ID of each hexagonal self-identification unit in S4 are as follows: For each hexagonal self-identification unit, the actual RGB value of the positioning point is matched with the RGB value of each reference color in the stable color set to determine the encoding value of the positioning point. Based on the preset ID reading order, the encoding values ​​of each positioning point are extracted and combined to generate the initial ID candidate result. In this process, any one of the positioning points is simultaneously associated with 7 hexagonal self-identification units. The correctness of the point's encoding value is verified by statistically analyzing the matching results of the encoding values ​​of the positioning point in the 7 associated self-identification units. If there is a matching ambiguity in the initial ID candidate results, a voting mechanism is used to determine the correct encoding value of the positioning point based on the matching encoding values ​​of the 7 self-identification units associated with the positioning point, so as to correct and determine the optimal ID of the hexagonal self-identification unit.

9. The reconstruction method for deformable surfaces according to claim 5, characterized in that, The location-aware markers of S1 are prepared based on a marker field generated by a graph theory model. The generation of the marker field includes the following steps: Based on the 3D model of the target deformable surface, a honeycomb topological network of the target deformable surface is constructed, which is defined as a simple embedding graph F=(V, E, C). The vertex set V contains the theoretical 3D coordinates of the positioning points and the label information to be assigned. The edge set E describes the spatial adjacency relationship between vertices. The face set C defines the local geometric structure through the surrounding order of the vertices. The basic unit of the face set C is a hexagon and constitutes a self-identifying unit. Based on the VF3 graph isomorphism matching principle, subgraphs isomorphic to hexagonal self-identifying units are extracted from graph F as candidates, and non-coplanar candidate subgraphs are filtered based on the consistency of surface normal direction. The label field is generated by assigning color-coded labels to each vertex in the vertex set V using a risk minimization strategy and resolving conflicts using a backtracking mechanism.

10. The reconstruction method for deformable surfaces according to claim 9, characterized in that, The risk-minimizing strategy is used to assign color-coded labels to each vertex in the vertex set V, as follows: The vertex with the most associated candidate subgraphs is assigned a label first, and the optimal label is selected according to the label security function S(Li)=∑logS(Ar), where S(Ar) represents the estimated probability that the r-th hexagonal cell associated with the current vertex can eventually form a globally unique ID under the condition that the label Li is assigned to the current vertex.

Images