A semantic segmentation method based on phase profile constraint

Abstract

The application discloses a semantic segmentation method based on phase profile constraint and belongs to the technical field of computer vision. The method uses a calibrated camera-projector system to collect a two-dimensional texture image and a deformed fringe image of a to-be-measured scene under the same field of view; the deformed fringe image is solved through multi-step phase shifting and Gray code technology to obtain an absolute phase distribution and calculate a gradient field; according to a preset mutation condition, a physical edge profile reflecting the depth change of an object is extracted from the gradient field; the physical edge profile is mapped into the two-dimensional texture image based on the corresponding relationship of pixel coordinates to accurately define the semantic object area, the corresponding category label is given, and finally a semantic label mask image is generated. The application uses physical phase information to correct the uncertainty of visual texture edges, effectively solves the edge extraction problem under a complex light scene, realizes the automatic generation of semantic segmentation data, and improves the training precision and robustness of a semantic segmentation model.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a semantic segmentation method based on phase contour constraints. Background Technology

[0002] Semantic segmentation, a core task in computer vision, aims to classify each pixel in an image to achieve pixel-level understanding of the scene. With the rapid development of deep learning technology, convolutional neural networks have achieved significant results in semantic segmentation tasks and are widely used in fields such as autonomous driving, robot navigation, augmented reality, and 3D reconstruction. In these applications, to train a high-precision semantic segmentation model, it is usually necessary to use supervised learning to input massive training datasets containing original images and corresponding semantic label masks into the neural network. Therefore, obtaining high-quality, refined semantic label data is a crucial foundation for building a high-performance semantic segmentation system.

[0003] Current methods for creating and labeling semantic segmentation datasets are mainly divided into two categories: manual labeling and semi-automatic labeling. The most common approach is to rely on manual labeling software to draw object outlines on 2D RGB images by manually plotting points, thereby generating semantic masks. Another approach is to use edge detection algorithms based on visual features, such as the Canny operator, the Sobel operator, or color-based segmentation algorithms, to automatically extract texture boundaries in the image to assist in generating segmented regions. With the development of multimodal technology, some solutions have begun to try to combine depth maps obtained by depth cameras to assist in identifying object edges.

[0004] The aforementioned existing technologies have shortcomings in practical applications. Purely manual annotation is not only time-consuming, labor-intensive, and costly, but also makes it difficult to guarantee pixel-level accuracy of boundaries. The jitter introduced by human operation often results in jagged edges of objects, and the subjective judgment differences among different annotators introduce sample noise. Secondly, traditional automatic extraction methods based on RGB images are highly susceptible to the influence of ambient lighting, shadows, and object surface textures. In particular, when the target object and background colors are similar or the object surface has complex textures, algorithms based on grayscale or color gradients often fail to correctly identify physical edges, resulting in incomplete segmented contours or the inclusion of background areas. Although some solutions introduce depth maps, the depth maps of traditional depth cameras have low resolution and high noise, making it difficult to extract sharp geometric contours and failing to meet the stringent requirements of ground truth data for training high-precision semantic segmentation models. Therefore, how to use physical means to obtain high-precision object edges that are not affected by lighting and texture, in order to automatically generate high-quality semantic segmentation data, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a semantic segmentation method based on phase contour constraints to solve the above-mentioned technical problems, comprising the following steps: S1: Projecting a structured light stripe pattern onto the scene to be tested, acquiring the corresponding deformed stripe image, and obtaining a two-dimensional texture image under the same field of view; S2: Performing phase calculation and unfolding on the deformed stripe image to obtain an absolute phase distribution reflecting the depth information of the scene to be tested; calculating the gradient field of the absolute phase distribution, and identifying the region whose gradient magnitude satisfies the preset abrupt change condition as the physical edge contour of the target object; S3: Based on the pixel coordinate correspondence under the same field of view, mapping the physical edge contour to the two-dimensional texture image, and extracting the semantic object region aligned with the physical edge contour in the two-dimensional texture image; S4: In response to the category assignment operation for the target object, assigning the corresponding semantic category label to the semantic object region, generating a semantic label mask containing physical edge contour and semantic category information, as the semantic segmentation result of the scene to be tested.

[0006] Furthermore, it also includes step S5: obtaining predetermined calibration parameters, constructing a three-dimensional point cloud of the scene to be tested using a camera pinhole imaging model, and mapping the semantic category information in the semantic label mask to the three-dimensional point cloud based on the pixel coordinate correspondence under the same field of view, thereby generating a colored three-dimensional point cloud with semantic annotations.

[0007] Furthermore, in step S1, the projected structured light fringe pattern includes a multi-step phase-shift sinusoidal fringe pattern and a Gray code pattern; correspondingly, in step S2, the wrapped phase is calculated using a deformed fringe image corresponding to the multi-step phase-shift sinusoidal fringe pattern, the fringe order is calculated using a deformed fringe image corresponding to the Gray code pattern, and the wrapped phase is unwrapped according to the fringe order to obtain the absolute phase distribution.

[0008] Furthermore, the absolute phase distribution The calculation formula is:

[0009]

[0010] in, Here are the pixel coordinates of the deformed stripe image. To wrap the phase, The stripe order is the one calculated in step S2.

[0011] Furthermore, in step S2, the physical edge contour of the target object is identified and processed based on the surface depth features of the target object in the scene to be tested: when the region exhibits a depth step feature, the preset abrupt change condition is configured as a gradient magnitude threshold, and the pixel region with a gradient field magnitude greater than the preset threshold is determined as the physical edge contour; when the region exhibits a continuous depth change feature, the preset abrupt change condition is configured as a gradient magnitude peak value, the gradient magnitude peak value of the absolute phase distribution gradient field is detected, and the pixel point where the gradient magnitude peak value is located is determined as the edge point constituting the physical edge contour.

[0012] Furthermore, in step S3, the process of mapping the physical edge contour to the two-dimensional texture image includes: extracting the pixel coordinate sequence of the physical edge contour; converting the pixel coordinate sequence into polygon vertex coordinates and storing it as an intermediate format tag description file; and drawing a closed region in the coordinate system of the two-dimensional texture image using the polygon vertex coordinates to form the semantic object region.

[0013] By directly mapping high-precision physical edges onto texture images, the generated semantic mask edges are smooth, sharp, and objective, which can realistically restore the geometric shape of objects. It effectively corrects the uncertainty of vision-based edges, and the tedious edge drawing work is automatically completed by the algorithm. Humans only need to make simple category assignments, which improves the annotation efficiency and helps to create low-cost, large-scale datasets.

[0014] Furthermore, in step S4, the step of generating a semantic label mask image containing physical edge contours and semantic category information specifically includes: parsing the category ID of the category-specified operation input; looking up a preset color mapping table according to the category ID to determine the corresponding RGB color value or grayscale index value; filling the semantic object region with the RGB color value or grayscale index value to generate a single-channel or three-channel mask image with the same resolution as the two-dimensional texture image.

[0015] The semantic segmentation method disclosed in this invention utilizes the physical properties of absolute phase to directly reflect the geometric depth discontinuity of the object surface. The phase information comes from actively projected coded structured light. Even in environments with weak textures, background interference of the same color, or complex environments, the system can still accurately lock the target contour based on the physical depth signal, effectively overcoming the segmentation failure problem caused by visual camouflage and improving the robustness of the algorithm in complex scenes. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the workflow of the semantic segmentation method based on phase contour constraints of the present invention.

[0017] Figure 2The figure shows the fitting result of the standard sphere diameter of the calibrated camera-projector imaging system in the embodiment.

[0018] Figure 3 The image shows the standard flat-panel plane fitting result of the calibrated camera-projector imaging system in the embodiment.

[0019] Figure 4 The two-dimensional texture image is for test two.

[0020] Figure 5 This is the semantic label mask for test 2.

[0021] Figure 6 This is a color 3D point cloud map with semantic annotations for Test 2.

[0022] Figure 7 The image shows a two-dimensional texture of a white cardboard box against a white background, used for test three.

[0023] Figure 8 This is the absolute phase diagram for test three.

[0024] Figure 9 The image shows the phase diagram of the package for test three.

[0025] Figure 10 The 2D texture image of the sphere is used for test four.

[0026] Figure 11 The phase diagram of the sphere-wrapped structure for test four.

[0027] Figure 12 The image shows the results of the test sphere. The left image is the absolute phase distribution map with the physical edge contour superimposed, and the right image is the mapping result of the physical edge contour of the sphere on the two-dimensional texture image.

[0028] Figure 13 The images shown are two-dimensional texture images of an indoor scene for Test 5. The left image is a test sample from viewpoint 1, and the right image is a test sample from viewpoint 2.

[0029] Figure 14 The image shows the semantic segmentation results obtained using existing technology for the indoor scene of Test 5. The left image shows the segmentation results from viewpoint 1, and the right image shows the segmentation results from viewpoint 2.

[0030] Figure 15 The image shows the outline of the indoor scene in Test 5, with the left image being the outline from view 1 and the right image being the outline from view 2.

[0031] Figure 16 The semantic label mask image obtained by the method of this invention is used for the indoor scene of Test 5. The left image shows the segmentation effect from view 1, and the right image shows the segmentation effect from view 2. Detailed Implementation

[0032] The method of the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0033] The embodiment discloses a semantic segmentation method based on phase contour constraints, referring to... Figure 1 As shown, it includes the following steps:

[0034] Step S1: Project structured light stripe patterns onto the scene to be tested, acquire the corresponding deformed stripe images, and obtain two-dimensional texture images under the same field of view;

[0035] Set up a calibrated camera-projector imaging system, which mainly consists of a digital projector and a high-resolution industrial camera. The camera and projector are fixed together by a rigid bracket to keep their relative position and angle constant.

[0036] Before performing the measurement, the system was calibrated using a planar target-based calibration method. The intrinsic parameters of the camera (focal length, principal point, distortion coefficient) and the projector were calibrated separately, and the extrinsic parameters (rotation matrix) between the two were calibrated. Translation vector This ensures that the system has accurate three-dimensional measurement capabilities, enabling the subsequently generated phase information to accurately reflect the physical depth.

[0037] The projector projects a pre-coded structured light stripe pattern onto the scene to be measured, where the target object is placed. In this embodiment, in order to balance measurement accuracy and the reliability of phase unfolding, the projected pattern sequence includes a multi-step phase shift pattern and a Gray code pattern.

[0038] The camera captures a sequence of images projected onto the surface of an object. Due to the geometric undulations on the object's surface, the projected straight stripes will bend and deform on the object's surface, and the camera captures images of the deformed stripes.

[0039] While keeping the camera position fixed, control the projector to project uniform white light, or turn off the projector and use ambient light to acquire a two-dimensional texture image of the scene under test, such as an RGB color image or a grayscale image, using the same industrial camera.

[0040] Since both the deformed stripe image and the 2D texture image were acquired by the same camera in the same field of view, they are physically aligned, and the coordinates in the texture image are... The pixels and the phase map calculated from the stripe image are in The phase value at that point corresponds to the same physical point in space.

[0041] S2: Perform phase calculation and unfolding on the deformed stripe image to obtain the absolute phase distribution reflecting the depth information of the scene under test; calculate the gradient field of the absolute phase distribution, and identify the region whose gradient amplitude meets the preset abrupt change condition as the physical edge contour of the target object;

[0042] The deformed fringe image acquired in step S1 is processed by calculating the phase pixel by pixel using the least squares method or a multi-step phase-shifting algorithm. For example, the four-step phase-shifting method is used to process the sinusoidal fringe image. , its first Light intensity of amplitude phase-shifted image It can be represented as:

[0043]

[0044] in, Background light intensity, For modulation amplitude, For the desired package phase, The preset phase shift is used. The enclosed phase is calculated by solving a system of simultaneous equations. Due to the periodicity of the sine function, the phase value obtained at this time is folded over. Within the interval, it exhibits a sawtooth-like distribution.

[0045] To obtain a continuous phase distribution, synchronously acquired Gray code images are decoded. First, the Gray code images are binarized and decoded to determine the stripe order of each pixel. According to the formula:

[0046]

[0047] Wrap phase Combined with the fringe order, a continuous absolute phase distribution is synthesized across the entire field. The numerical changes in this absolute phase distribution directly correspond to the changes in the physical depth of the object's surface relative to the camera.

[0048] Since the phase value is directly related to the depth, the absolute phase value will jump at the edge of the object or in the region where the depth jumps. In order to capture this change, the gradient field of the absolute phase distribution is calculated.

[0049] Specifically, edge detection operators (such as the Sobel operator, Canny operator, or simple difference operators) can be used to analyze the absolute phase map. Convolution operations are performed to calculate the gradient components of each pixel in the horizontal and vertical directions, and then the gradient magnitude is synthesized to generate a phase gradient field that reflects the degree of geometric change in the scene.

[0050] Based on the calculated phase gradient field, physical edge contours are identified according to preset abrupt change conditions, such as setting a gradient magnitude threshold. The threshold is set based on the system noise level and the minimum depth jump height of the object under test. When the phase gradient magnitude of a pixel is greater than the threshold, the point is determined to be in the region where the depth changes abruptly, that is, it is identified as the physical edge of the target object.

[0051] At the boundary between the object's outline and the background, the phase gradient will exhibit a large peak, which the system marks as the outer contour. For the internal geometry of the object, as long as there is a depth difference, the phase gradient will also respond, which the system marks as the internal geometric edge.

[0052] To remove phase noise caused by shadows or highlights, morphological processing can be performed on the extracted binarized edge image to obtain a clean physical edge contour map that contains only the geometric information of the object.

[0053] The extraction method in this step does not rely on the color or texture of the object's surface, so even white paper in front of a white wall can have its edges accurately identified.

[0054] Step S3: Based on the pixel coordinate correspondence under the same field of view, map the physical edge contour to the two-dimensional texture image, and extract the semantic object region aligned with the physical edge contour in the two-dimensional texture image;

[0055] Based on the hardware acquisition strategy described in step S1, both the deformed stripe image and the two-dimensional texture image are acquired by the same industrial camera without any pose change, so they are in a completely consistent pixel coordinate system.

[0056] Establish a one-to-one pixel mapping relationship between the physical edge contour and the two-dimensional texture image. For each pixel identified as a "physical edge" in step S2... The system directly maps it to the coordinates in the two-dimensional texture image, which are also... The location.

[0057] This mapping method eliminates the need for complex extrinsic parameter calibration or point cloud projection transformation required for the fusion of traditional heterogeneous sensors (such as LiDAR and cameras), ensuring the absolute positional accuracy of physical edges on the texture image. After superimposing the physical edge contour onto the two-dimensional texture image, the system uses this contour as a rigid constraint to extract the semantic object region of the target object.

[0058] For objects that are complete within the field of view, the extracted physical edge contours usually form a closed loop structure. The system defines the set of all pixels inside the closed contour as the semantic object region. For objects that are truncated or occluded, the physical edge contours and the image boundary together form a closed region.

[0059] In practical engineering implementation, the system can fit discrete physical edge pixels into a vectorized sequence of polygon vertices and record the boundary coordinates of the region in a commonly used annotation format (such as JSON or XML).

[0060] This step utilizes physical information to correct visual misjudgments. In traditional segmentation methods based solely on two-dimensional texture images, when the edge of the target object is similar in color to the background, or when there is strong shadow interference on the object's surface, the visual edge is often blurred or shifted. This embodiment introduces physical edge contours, forcibly defining the boundary between the object and the background in the two-dimensional texture image. No matter how weak the grayscale gradient at the edge of the texture image is, as long as the phase gradient shows a sudden change in depth, it is determined that this is the boundary of the object. In the two-dimensional texture image, a semantic object region that highly matches the real physical shape is "forcibly" cut out, effectively solving the edge blurring problem in complex lighting and weak texture environments.

[0061] Step S4: In response to the category assignment operation for the target object, the corresponding semantic category label is assigned to the semantic object region, and a semantic label mask map containing physical edge contours and semantic category information is generated as the semantic segmentation result of the scene to be tested.

[0062] After extracting the semantic object region, the system receives and responds to a category specification operation to determine the object category to which the region belongs. This operation can be implemented through one of the following two methods or a combination thereof:

[0063] Interactive Labeling: The system provides a human-computer interaction interface that displays a 2D texture image overlaid with physical edge contours. Users can select the corresponding label from a preset category list by clicking or selecting the semantic object area. Since the contours are automatically and accurately generated by the system, users do not need to perform tedious manual outlining; they only need to perform a simple "click-select" operation, greatly improving annotation efficiency.

[0064] Algorithm-assisted specification: By invoking a pre-trained image classification algorithm or utilizing pre-defined scene prior knowledge (e.g., based on the size or height information of an object in three-dimensional space), the algorithm automatically infers candidate categories for the region and generates a category specification signal.

[0065] Upon receiving the category specification signal, the corresponding semantic category label is logically bound to the semantic object region defined in step S3. The binding relationship is stored in a structured intermediate data format, such as a JSON file or an XML file. The file records the sequence of outline coordinate points of the object and the corresponding category name, thus realizing the fusion of geometric information and semantic information.

[0066] To output data in a format that conforms to mainstream computer vision standards, a semantic label mask is generated based on the semantic object region. A blank image with the same resolution as the two-dimensional texture image is initialized, typically a single-channel grayscale image or a three-channel pseudo-color image. All pixels within the semantic object region are traversed, and their pixel values ​​are assigned the corresponding category ID values. For example, background pixels are assigned a value of 0, and target object pixels are assigned the corresponding ID. For visualization requirements, a preset color lookup table can be used to map the category ID to specific RGB colors, such as red for a cup and green for a book, thereby generating a semantic mask with distinct colors.

[0067] The object edges in the generated semantic mask map correspond to the physical phase change locations extracted in step S2. Compared with traditional manually annotated masks, this mask map is smooth and jagged at the edges and does not contain misclassified pixels caused by visual blur.

[0068] S5: Obtain the predetermined calibration parameters, construct the three-dimensional point cloud of the scene to be tested using the camera pinhole imaging model, and map the semantic category information in the semantic label mask to the three-dimensional point cloud based on the pixel coordinate correspondence under the same field of view to generate a color three-dimensional point cloud with semantic annotation.

[0069] To demonstrate the technical effectiveness of the aforementioned semantic segmentation method, tests were conducted in the following different scenarios.

[0070] Test 1: Equipment accuracy verification

[0071] The method of this invention is composed of a series of single-view point clouds stitched together. Hardware performance, calibration accuracy, phase shift algorithm, and shooting environment will all affect the accuracy of the stripe projection profilometry. The resolution of the industrial camera used for testing is 1920*1200.

[0072] The experiment used a standard flat plate and a standard double sphere to verify the accuracy of the equipment, such as... Figure 2 As shown, the standard deviation of the captured standard sphere is 0.06mm, indicating that the overall distortion of the equipment is small, the reconstruction accuracy of the curved surface is of high quality, and the diameter of the sphere fitted by the point cloud is 50.878mm, while the actual diameter of the standard sphere is 50.798mm, with an overall difference of 0.08mm, which meets our expected target for point cloud accuracy. Figure 3 As shown, a standard flat plate was reconstructed in three dimensions. The standard error of the plane was 0.02 mm, indicating good flatness and ensuring that the reconstructed single-view three-dimensional point cloud has ideal quality.

[0073] Test 2:

[0074] like Figures 4 to 6As shown, to verify the effectiveness of the semantic segmentation method described in this embodiment of the invention, tests were conducted in a real-world scenario with complex colors and materials. According to step S1, Figure 4 The image shows the visual texture information of the scene under test, captured by the camera in the calibrated camera-projector system under conditions of projecting uniform white light or natural ambient light. Figure 5 The semantic label mask is the output of step S4. It is generated based on the physical edge contour extracted in step S2 and clearly defines the semantic object region of each target in the scene. Figure 6 The image shown is a color 3D point cloud with semantic annotations, which is the output of step S5. By establishing the pixel coordinate correspondence under the same field of view, the... Figure 5 The semantic category information is mapped to 3D space. As shown in the test results above, this method extracts contours using gradient abrupt changes in absolute phase. This process mainly depends on the geometric shape of the surface of the scene under test, and is minimally affected by the object's surface color, texture, material, and lighting conditions. Compared to manual semantic annotation in existing technologies, the contours generated by this method more realistically fit the physical boundaries of objects, achieving higher precision edge extraction.

[0075] Test 3:

[0076] To address the issue that existing edge detection algorithms based on RGB vision tend to fail in scenes with weak textures or mixed colors, this paper refers to... Figures 7 to 9 As shown, this embodiment underwent targeted robustness testing. The test scenarios selected were a white cardboard box (weak texture scene) with a color extremely close to that of the background wall and a standard sphere with continuously varying surface depth (curved surface scene):

[0077] like Figure 7 As shown, in the test on a white cardboard box, due to the extremely small difference in color gradient between the object and the background, traditional visual algorithms struggle to segment a clear boundary. However, this invention utilizes the absolute phase distribution map obtained through phase calculation, which can keenly capture the depth step signal at the object's edge, thus successfully extracting the complete physical contour based on the abrupt change in phase gradient.

[0078] Figure 9 The image shows the wrapping phase of the scene obtained through a camera sensor. From the wrapping phase, it can be seen that when the stripes on an object's surface are modulated by the object's surface undulations, its phase shifts. When the phase is unfolded, the absolute phase at the connection between the object and the background in the 2D texture image also experiences a corresponding numerical jump due to changes in depth. Figure 8 As shown in the corresponding absolute phase diagram, the smooth absolute phase change of the background itself exhibits an edge-like contour due to the modulation of the object's surface undulation information. This contour can be directly correlated with the object's surface due to the same viewing angle. Figure 7The geometric edge contour of the cardboard box is shown in the two-dimensional texture map. By mapping the edge pixel coordinates of the phase map to the two-dimensional texture map, a relatively accurate edge contour extraction result can be formed.

[0079] Test 4:

[0080] Tests for spheres, such as Figures 10 to 12 As shown,

[0081] This embodiment extracts the corresponding edge contours of spherical objects with continuously changing characteristics. The test results for the sphere are as follows: Figure 12 As shown, the left image is the absolute phase distribution map superimposed with the physical edge contour; the right image is the mapping result of the physical edge contour on the 2D texture image. For the corresponding edge contour extraction of spherical objects with continuously changing features, it can be seen that the interior of the spherical object does not have a large phase jump due to the continuously changing depth features. There is a clear phase change at the boundary between the edge and the background. For spheres with continuously changing depth features, their edges are represented by local peaks in the phase gradient field. By detecting these peaks, the edge can be locked. Moreover, in actual semantic annotation, the accurate extraction of the edge of spherical objects is difficult to achieve manually using commonly used training dataset creation software such as Labelme. However, the phase-based method can efficiently extract the accurate edge contours of various objects in the scene, which also simplifies the semantic dataset creation process.

[0082] The above tests demonstrate that this method, by utilizing physical phase information, can effectively correct the uncertainty of visual texture edges and has strong environmental adaptability.

[0083] Test 5:

[0084] like Figures 13 to 16 As shown, the following is adopted Figure 13 The typical indoor scene shown is used as the experimental dataset for semantic segmentation. Semantic annotations were performed on the blackboard, stool, door, tabletop, sign, and floor in the scene. The dataset consists of 200 training images, 30 validation images, and 10 test images. Samples from two different perspectives were selected for the test. Figure 13 Two-dimensional texture images are shown (left image is view one, right image is view two); Figure 15 The extracted physical edge contour map is shown (the left image corresponds to viewpoint one, and the right image corresponds to viewpoint two). Figure 14 The images show the semantic segmentation results of existing technologies (left image corresponds to viewpoint one, right image corresponds to viewpoint two). Figure 16 The semantic segmentation effect diagrams of the embodiments of the present invention are shown (the left image corresponds to viewpoint one, and the right image corresponds to viewpoint two).

[0085] To maximize the high-precision depth contour constraint capability brought by phase information, this test targets a blackboard target in the scene. First, the phase edge contour of the scene (e.g., ...) is extracted using the method of this invention. Figure 15 As shown in the figure, the pixel positions of the semantic annotations are geometrically constrained by this contour, and then the semantic dataset is subjected to accurate geometric segmentation. The final semantic segmentation result is as follows. Figure 16 As shown in the left and right images; taking the blackboard target in the scene as an example, refer to... Figure 14 Based on the existing segmentation results, it can be seen that existing semantic segmentation methods are affected by factors such as insufficient image resolution, image blurring, and small training sample size. The semantic segmentation results cannot completely cover the full pixel area of ​​the target object, and there are problems such as missed edge detection and boundary offset. Figure 14 As shown in the left and right images; however, this invention uses phase edge contours to geometrically constrain the annotation process, enabling precise positioning of the physical boundaries of the target object and achieving complete and accurate annotation of the target object's pixel region. (Comparison) Figure 14 and Figure 16 It can be seen that after adopting the method of the present invention, the semantic pixel coverage and boundary fit of the blackboard target are significantly improved.

[0086] The visualization results show that when the training set has a small number of samples, the method of this invention can help improve the semantic segmentation accuracy by geometrically segmenting the object. The segmentation results are sharper and more closely aligned at the object edges, effectively suppressing edge blurring and pixel overflow.

[0087] To quantify the metrics, we use the mIoU value to illustrate the proportion of semantically segmented pixels to the total pixel area of ​​the object instance. During validation testing, we calculate the ratio of the overlapping area of ​​the semantically segmented Label_mask and the pre-labeled Pred_mask to the total pixel area of ​​the object, and finally take the average of all test sets to obtain the final mIoU value.

[0088] When pixels in the blackboard region are extracted and calculated separately, the mIoU value of the blackboard in conventional semantic segmentation is 0.81. However, after extracting the edges using the method of this invention, the overall mIoU value is 0.88. This shows that when the training set samples are relatively small, the method provided by this invention can help improve the accuracy of semantic segmentation by geometrically segmenting the object.

[0089] In summary, by introducing high-precision phase physical constraints, this invention not only achieves automated generation of semantic segmentation data but also achieves significant progress in data accuracy and model training performance compared to existing technologies. The above description is merely the best specific embodiment of this invention, but the scope of protection of this invention is not limited thereto. Any variations or substitutions conceived by those skilled in the art within the technical scope disclosed in this invention should be included within the scope of protection of this invention.

Claims

1. A semantic segmentation method based on phase contour constraints, characterized in that, Includes the following steps: S1: Project a structured light stripe pattern onto the scene to be tested, acquire the corresponding deformed stripe image, and obtain a two-dimensional texture image under the same field of view. The structured light stripe pattern includes a multi-step phase-shifting sinusoidal stripe pattern and a Gray code pattern. S2: Perform phase calculation and unwrapping on the deformed fringe image to obtain an absolute phase distribution reflecting the depth information of the scene under test; specifically: calculate the wrapped phase using the deformed fringe image corresponding to the multi-step phase-shift sinusoidal fringe pattern, calculate the fringe order using the deformed fringe image corresponding to the Gray code pattern, and unwrap the wrapped phase according to the fringe order to obtain the absolute phase distribution. The calculation formula is: in, These are the pixel coordinates of the deformed stripe image. To wrap the phase, The stripe order calculated in step S2; The gradient field of the absolute phase distribution is calculated, and regions whose gradient magnitudes satisfy a preset abrupt change condition are identified as the physical edge contours of the target object; specifically, the processing is based on the surface depth features of the target object in the scene under test. When a region exhibits a depth step feature, the preset mutation condition is configured as a gradient magnitude threshold, and pixel regions in the gradient field with magnitudes greater than the preset threshold are determined as physical edge contours. When the region exhibits a continuous change in depth, the preset abrupt change condition is configured as the gradient amplitude peak value. The gradient amplitude peak value of the absolute phase distribution gradient field is detected, and the pixel point where the gradient amplitude peak value is located is determined as the edge point constituting the physical edge contour. S3: Based on the pixel coordinate correspondence under the same field of view, map the physical edge contour to the two-dimensional texture image, and extract the semantic object region aligned with the physical edge contour in the two-dimensional texture image; S4: In response to the category assignment operation for the target object, the corresponding semantic category label is assigned to the semantic object region, and a semantic label mask containing physical edge contours and semantic category information is generated as the semantic segmentation result of the scene to be tested; S5: Obtain the predetermined calibration parameters, construct the three-dimensional point cloud of the scene to be tested using the camera pinhole imaging model, and map the semantic category information in the semantic label mask to the three-dimensional point cloud based on the pixel coordinate correspondence under the same field of view to generate a color three-dimensional point cloud with semantic annotation.

2. The semantic segmentation method based on phase contour constraints according to claim 1, characterized in that, In step S3, the process of mapping the physical edge contour to the two-dimensional texture image includes: extracting the pixel coordinate sequence of the physical edge contour; converting the pixel coordinate sequence into polygon vertex coordinates and storing it as an intermediate format tag description file; and drawing a closed region in the coordinate system of the two-dimensional texture image using the polygon vertex coordinates to form the semantic object region.

3. The semantic segmentation method based on phase contour constraints according to claim 1, characterized in that, In step S4, the step of generating a semantic label mask image containing physical edge contours and semantic category information specifically includes: parsing the category ID of the category-specified operation input; looking up a preset color mapping table according to the category ID to determine the corresponding RGB color value or grayscale index value; filling the semantic object region with the RGB color value or grayscale index value to generate a single-channel or three-channel mask image with the same resolution as the two-dimensional texture image.

Images