Structured light spot local elimination system

Abstract

The invention discloses a structured light spot local elimination system. The system comprises a sample image collection subsystem used for collecting a light spot sample image set and a noisy point sample image set; the measurement subsystem is used for collecting a to-be-processed image containing light spots on the surface of a measured object; the image processing subsystem is used for receiving the light spot sample image set, the noise point sample image set and a to-be-processed image; a training data set is generated based on the light spot sample image set and the noisy point sample image set, a target semantic segmentation model is constructed based on the training data set, and the target semantic segmentation model is used for identifying abnormal light spots in the images; and calling the target semantic segmentation model to process the to-be-processed image to remove the abnormal light spot area in the to-be-processed image so as to obtain the image after the abnormal light spot is removed. According to the method, the high-quality prior data can be obtained, and the semantic segmentation model is constructed based on the prior data, so that the accurate recognition and elimination of the abnormal regions with different shapes in the structured light spots are realized.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically to a structured light spot local removal system. Background Technology

[0002] Laser triangulation is an important non-contact 3D measurement technique. This method projects a laser line onto the surface of the object being measured, and a camera captures an image of the laser spot. Then, based on the relative positions of the laser and camera and system parameters, the three-dimensional information of the laser-irradiated area is calculated. In this measurement system, the image quality of the laser spot is one of the core factors determining the final measurement accuracy.

[0003] However, in practical industrial applications, especially in measurement scenarios involving highly reflective or demanding smooth surfaces, random dust, minor scratches, or inherent defects on the surface of the object being measured can disrupt the morphology of the local structured light spot. This morphological disruption manifests as anomalous noise in the image, with varying shapes and characteristics, constituting a major noise source affecting measurement accuracy. These anomalous areas introduce incorrect positioning information, significantly negatively impacting the final accuracy of 3D data reconstruction.

[0004] Currently, the mainstream solutions to this problem in the industry mostly employ traditional image filtering techniques, aiming to suppress interference caused by changes in the shape of the light spot. However, such methods typically only reduce the intensity of noise, and cannot fundamentally and accurately identify and separate the light spot areas whose shape has been damaged. Therefore, their compensation effect on the final measurement accuracy is very limited.

[0005] Therefore, existing technologies cannot accurately identify and remove abnormal areas of structured light spots in images. Summary of the Invention

[0006] The purpose of this application is to provide a structured light spot local removal system to solve the problem that existing technologies cannot accurately identify and remove abnormal areas of structured light spots in images.

[0007] To achieve the above objectives, the first aspect of this application provides a structured light spot local removal system, which includes: The sample image collection subsystem includes a spot image collection device and a noise image collection device, configured to collect a set of spot sample images and a set of noise sample images; The measurement subsystem is configured to acquire a processing image of the surface of the object being measured, the processing image containing light spots; The image processing subsystem, which communicates with both the sample image collection subsystem and the measurement subsystem, is configured as follows: The receiving sample image collection subsystem collects a set of spot sample images and a set of noise sample images, as well as the image to be processed acquired by the measurement subsystem; A training dataset is generated based on a set of light spot sample images and a set of noise sample images. A target semantic segmentation model is then constructed based on the training dataset. The target semantic segmentation model is used to identify abnormal light spots in images. The target semantic segmentation model is invoked to process the image to be processed, so as to remove the abnormal light spot regions in the image to obtain the image after removing the abnormal light spots.

[0008] In this embodiment of the application, the light spot image collection device includes: Multi-layered reflector; A line structured light laser is positioned above a multi-layered reflector to emit laser light onto the multi-layered reflector, thereby creating multiple light spots at multiple interfaces within the multi-layered reflector. The first image acquisition device is positioned above the multi-layered reflector, which is located within the imaging field of view of the image acquisition device. The first image acquisition device is used to acquire the light spot image on the multi-layered reflector.

[0009] In the embodiments of this application, the multi-layer reflective plate is composed of multiple layers of transparent plates, with an anti-reflective film coated on the upper surface of each transparent plate and an anti-reflective film coated on the lower surface of the bottom transparent plate.

[0010] In this embodiment, the number of layers of the multi-layered reflector is determined according to the depth of field of the image acquisition device.

[0011] In this embodiment of the application, the noise image collection device includes: Transparent panel; A light source, positioned above the transparent panel, is used to emit parallel light onto the transparent panel in order to illuminate the noise on the transparent panel; The second image acquisition device is used to acquire noisy images on the transparent plate.

[0012] In this embodiment, the angle between the parallel light emitted by the light source and the transparent plate is equal to the angle between the emitted laser of the line structured light laser and the plane of the multi-layered reflective plate.

[0013] In this embodiment of the application, the image processing subsystem is further configured to: Semantic segmentation is performed on each spot sample image in the spot sample image set to segment out each spot in each spot sample image, resulting in an independent spot image set. Semantic segmentation is performed on each noise sample image in the noise sample image set to separate each noise point in each noise sample image, resulting in an independent noise image set. Based on the independent light spot image set and the independent noise image set, a training dataset is generated through data augmentation processing. Each group of training data in the training dataset includes augmented training images and corresponding labeled images. Based on a pre-set deep learning training model, a corresponding target semantic segmentation model is generated for each group of training data.

[0014] In this embodiment of the application, the image processing subsystem is further configured to: Multiple independent noise images are randomly extracted from a set of independent noise images. These multiple independent noise images are randomly distributed and fused into a single image to obtain a multi-noise image. A labeled image corresponding to the multi-noise image is then generated, in which the noise regions corresponding to the independent noise in the multi-noise image are marked. The multi-noise image is fused with an independent spot image randomly selected from a set of independent spot images to generate an augmented training image; A set of training data is obtained by associating augmented training images with labeled images; Repeat the above steps to generate multiple sets of training data to obtain the training dataset.

[0015] In this embodiment of the application, the image processing subsystem is further configured to: For each set of training data, from the beginning row to the end row of the augmented training image, an augmented image segment is extracted row by row according to a preset rule to form a subset of augmented image segments; Extract labeled image fragments from the corresponding labeled images that are at the same positions as the augmented training images, forming a subset of labeled image fragments; Based on a pre-defined deep learning training model, a corresponding target semantic segmentation model is generated for each set of training data according to the augmented image fragment subset and the labeled image fragment subset.

[0016] In this embodiment of the application, the image processing subsystem is further configured to: Perform the following operations line by line from the beginning to the end of the image to be processed: Extract the image segment to be processed from the current row according to preset rules; The target semantic segmentation model corresponding to the current row position is invoked to process the image segment to be processed in order to identify abnormal light spot regions; Modify the pixel values ​​of the abnormal light spot areas to the preset background values.

[0017] The above technical solution provides a structured light spot local removal system, including a sample image collection subsystem, a measurement subsystem, and an image processing subsystem. The sample image collection subsystem includes a spot image collection device and a noise image collection device, configured to collect a set of spot sample images and a set of noise sample images. The measurement subsystem is configured to acquire an image to be processed from the surface of the object being measured, the image containing spot light. The image processing subsystem communicates with both the sample image collection subsystem and the measurement subsystem, and is configured to: receive the set of spot sample images and the set of noise sample images collected by the sample image collection subsystem, and the image to be processed acquired by the measurement subsystem; generate a training dataset based on the set of spot sample images and the set of noise sample images, and construct a target semantic segmentation model based on the training dataset, the target semantic segmentation model being used to identify abnormal spot light in the image; and call the target semantic segmentation model to process the image to be processed to remove abnormal spot regions from the image to obtain an image after removing abnormal spot light. This application utilizes a dedicated subsystem for collecting light spot and noise sample images through collaborative design to acquire high-quality prior data. Based on this, a semantic segmentation model is constructed, thereby achieving accurate identification and removal of abnormal regions with different shapes in structured light spots.

[0018] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings: Figure 1 A structural block diagram of a structured light spot local removal system provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of a spot image collection device provided in a specific embodiment of this application; Figure 3 This is a schematic diagram of the structure of a multi-layered reflector provided in a specific embodiment of this application; Figure 4 This is a schematic diagram of the structure of a noise image collection device provided in a specific embodiment of this application; Figure 5 A schematic diagram of the structure of a measurement subsystem provided in a specific embodiment of this application; Figure 6 A schematic diagram of light spot semantic segmentation provided for a specific embodiment of this application; Figure 7 This is a schematic diagram of noise semantic segmentation provided for a specific embodiment of this application.

[0020] Explanation of reference numerals in the attached figures Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0022] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0023] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0024] Figure 1 This is a structural block diagram of a structured light spot local removal system provided in an embodiment of this application. Figure 1 As shown, this application provides a structured light spot local removal system, which may include: The sample image collection subsystem 100 includes a spot image collection device and a noise image collection device, and is configured to collect a set of spot sample images and a set of noise sample images. The measurement subsystem 200 is configured to acquire a processing image of the surface of the object being measured, the processing image containing light spots; Image processing subsystem 300, communicating with sample image collection subsystem 100 and measurement subsystem 200 respectively, is configured to: The receiving sample image collection subsystem 100 collects a set of spot sample images and a set of noise sample images, as well as the image to be processed acquired by the measurement subsystem 200; A training dataset is generated based on a set of light spot sample images and a set of noise sample images. A target semantic segmentation model is then constructed based on the training dataset. The target semantic segmentation model is used to identify abnormal light spots in images. The target semantic segmentation model is invoked to process the image to be processed, so as to remove the abnormal light spot regions in the image to obtain the image after removing the abnormal light spots.

[0025] This application provides a structured light spot local removal system, designed to solve the problem of localized damage to the laser spot morphology caused by defects such as dust and scratches on the surface of the measured object in precision 3D measurements (especially those based on laser triangulation or structured light). The system constructs a complete technical closed loop integrating dedicated hardware data acquisition and advanced artificial intelligence processing, enabling intelligent, precise, and automated removal of abnormal spots.

[0026] Specifically, the structured light spot local removal system includes a sample image collection subsystem 100, a measurement subsystem 200, and an image processing subsystem 300. The sample image collection subsystem 100 is used to acquire high-quality training data, including a spot image collection device and a noise image collection device, used to collect sets of spot sample images and noise sample images, respectively. The spot sample image set represents ideal laser spot images that are close to clean, standard, and free of interference, while the noise sample image set represents images of various typical abnormal spot shapes caused by factors such as dust. The measurement subsystem 200 is used to perform routine 3D measurement tasks. The measurement subsystem 200 is a standard hardware platform for performing 3D measurement tasks, typically including a line structured light laser, an industrial camera, and a lens, used to project laser lines onto the object being measured, and the camera captures images of the surface containing the light spots to be processed.

[0027] The image processing subsystem 300 communicates with both the sample image collection subsystem 100 and the measurement subsystem 200. First, it receives a set of light spot sample images and a set of noise sample images collected by the sample image collection subsystem 100. Then, it generates a training dataset using automated image processing techniques. These automated image processing techniques may include semantic segmentation and data augmentation. Semantic segmentation accurately segments each light spot and each noise point in the original sample images, forming independent image units. Data augmentation synthesizes a massive number of labeled noisy light spot images from the segmented normal light spot images and various noise images using algorithms (such as image fusion and affine transformation). Further, a target semantic segmentation model is trained using the obtained training dataset. Next, the measurement subsystem 200 receives the image to be processed and calls the trained target semantic segmentation model to process it. The target semantic segmentation model performs pixel-level semantic segmentation on the input image to be processed and outputs a segmentation map of the same size as the input image, clearly marking all pixel regions identified as abnormal light spots. Finally, based on the segmentation map, abnormal light spot regions in the image to be processed are removed to obtain the image after removing the abnormal light spots.

[0028] This application embodiment constructs a full-link system integrating dedicated sample acquisition, AI model construction, and online intelligent processing, which enables accurate identification and complete removal of abnormal regions with different shapes in structured light spots. It fundamentally overcomes the technical defect of traditional filtering methods that can only weaken interference but cannot eliminate it, significantly improving the accuracy and reliability of 3D measurement. It is especially suitable for industrial inspection scenarios of smooth surfaces with extremely high measurement accuracy requirements.

[0029] The above technical solution provides a structured light spot local removal system, including a sample image collection subsystem, a measurement subsystem, and an image processing subsystem. The sample image collection subsystem includes a spot image collection device and a noise image collection device, configured to collect a set of spot sample images and a set of noise sample images. The measurement subsystem is configured to acquire an image to be processed from the surface of the object being measured, the image containing spot light. The image processing subsystem communicates with both the sample image collection subsystem and the measurement subsystem, and is configured to: receive the set of spot sample images and the set of noise sample images collected by the sample image collection subsystem, and the image to be processed acquired by the measurement subsystem; generate a training dataset based on the set of spot sample images and the set of noise sample images, and construct a target semantic segmentation model based on the training dataset, the target semantic segmentation model being used to identify abnormal spot light in the image; and call the target semantic segmentation model to process the image to be processed to remove abnormal spot regions from the image to obtain an image after removing abnormal spot light. This application achieves accurate identification and removal of abnormal regions with different shapes in structured light spots by collaboratively designing a dedicated subsystem for collecting light spot and noise sample images.

[0030] In this embodiment of the application, the spot image collection device may include: Multi-layered reflector; A line structured light laser is positioned above a multi-layered reflector to emit laser light onto the multi-layered reflector, thereby creating multiple light spots at multiple interfaces within the multi-layered reflector. The first image acquisition device is positioned above the multi-layered reflector, which is located within the imaging field of view of the image acquisition device. The first image acquisition device is used to acquire the light spot image on the multi-layered reflector.

[0031] In this embodiment, the light spot image collection device is a hardware combination specifically designed to acquire high-quality, highly consistent standard light spot images, including a multi-layered reflector, a line structured light laser, and a first image acquisition device. The multi-layered reflector is a specialized optical element capable of generating multiple reflected light spots simultaneously at different physical depths under a single laser irradiation using its internal structure. The line structured light laser is a device that emits a line-shaped laser beam and is the standard light source for structured light 3D measurement. The first image acquisition device typically consists of an industrial camera and an optical lens, used to capture the light spot image presented on the multi-layered reflector.

[0032] In one example, the spot image collecting device may further include a reflector. During the acquisition of the spot image by the spot image collecting device, the line structured light laser can be placed horizontally. Its emitted laser beam first passes through a 45-degree reflector, then deflects 90 degrees before vertically downwards onto the horizontally placed multi-layered reflector. The first image acquisition device can be placed horizontally alongside the laser, with a 45-degree reflector also installed in front of it, causing its optical path to deflect 90 degrees and then vertically downwards, directly imaging the multi-layered reflector. Using this confocal optical path design allows for a more compact physical layout of the laser and camera, facilitating integration into a miniaturized device housing.

[0033] Preferably, the multi-layered reflector itself can be a sealed unit, filled with inert gas or evacuated to completely eliminate the possibility of dust or fogging inside, ensuring long-term stability. During image acquisition, the first image acquisition device can use a fixed exposure time and gain, supplemented by a uniform surface light source for auxiliary illumination, to simultaneously record the structural position of the multi-layered reflector, facilitating subsequent image alignment and spot extraction. The spot image collection device can be fixed on a vibration-isolated platform and, through program control, can automatically and continuously perform hundreds of image acquisitions, building a massive standard spot database in a short time.

[0034] In this way, the spot image collection device can efficiently acquire multiple standard spot samples in ideal condition in a single imaging process by using multi-layered reflectors, which greatly improves the data acquisition efficiency and internal consistency, and provides a reliable data foundation for training high-precision target semantic segmentation models.

[0035] Figure 2 This is a schematic diagram of the structure of a spot image collection device provided in a specific embodiment of this application. Figure 2 As shown, the specific components of the light spot image collection system include: a line structured light laser 1, a camera 2, a lens 3, and a multi-layered reflector 4, totaling four parts. The laser emitted from the line structured light laser 1 illuminates the multi-layered reflector 4, forming multiple laser spots on the various glass interfaces within it. The camera 2 and lens 3 are combined to form the first image acquisition device, which is installed on one side of the multi-layered reflector 4, allowing the imaging field to observe the multiple laser spots on the multi-layered reflector 4. In the embodiments of this application, the multi-layer reflective plate is composed of multiple layers of transparent plates, with an anti-reflective film coated on the upper surface of each transparent plate and an anti-reflective film coated on the lower surface of the bottom transparent plate.

[0036] In this embodiment, the transparent plate refers to a flat plate made of optical-grade transparent material, such as K9 glass, quartz glass, or optical acrylic, whose surface flatness and internal uniformity must meet imaging requirements. An anti-reflection coating is a multilayer dielectric film deposited on the surface of an optical element, which reduces the reflection loss of specific wavelengths of light at the interface through the interference effect of light. In this embodiment, the multi-layered reflector can be composed of multiple layers of transparent plates, with an anti-reflection coating deposited on the upper surface of each transparent plate and also on the lower surface of the bottommost transparent plate, while the remaining portions are uncoated.

[0037] In one example, the intensity of reflected light is related to the difference in refractive index between adjacent media. To further enhance the contrast of reflected light spots at each interface, the multilayer transparent plate can be made by alternately stacking two types of optical glass with slightly different refractive indices.

[0038] Specifically, during the process of acquiring sample images of light spots using a light spot image collection device, a line structured light laser illuminates a multi-layered reflective plate at a certain tilt angle. After the laser penetrates the upper surface of the first transparent plate, it is reflected on the lower surface of the first transparent plate, forming the first light spot. The transmitted light then undergoes a similar reflection on the upper surface of the second transparent plate, forming the second light spot, and so on. In this way, in a single capture, the first image acquisition device can capture a series of light spot images at different depths but with consistent optical characteristics.

[0039] In one example, the transparent panels can be separated by tiny air gaps instead of physical contact, using precision gaskets to avoid direct contact between the glass that could cause Newton's rings interference fringes. This also makes the upper and lower surfaces of each glass layer independent reflective interfaces, further increasing the number of light spots that can be obtained in a single shot.

[0040] Thus, by optimizing the material combination and precision film system design of the multi-layer transparent plate, it is beneficial to improve the signal quality and consistency of the light spots generated by the multi-layer reflector, while suppressing stray light interference to the greatest extent, providing hardware guarantee for obtaining standard light spot samples with ultra-high signal-to-noise ratio.

[0041] In this embodiment, the number of layers of the multi-layered reflector can be determined based on the depth of field of the image acquisition device.

[0042] In this embodiment, depth of field refers to the depth range along the optical axis of the object space where a clear image can be formed on the image plane. The number of layers in the multi-layered reflector can be determined based on the depth of field of the image acquisition device. Specifically, firstly, the depth of field of the imaging system needs to be calculated using the parameters of the camera and lens and the working distance; then, based on the thickness of a single transparent plate, the maximum number of usable layers is calculated. The number of transparent plate layers is taken as the integer part of the depth of field divided by the thickness of a single transparent plate, to ensure that the light spots formed on all interfaces of the multi-layered reflector fall within the depth of field, thus achieving clear imaging of all layers; at the same time, the depth of field can be fully utilized to ensure the efficiency of data acquisition.

[0043] This application embodiment ensures that the light spots of all layers can be clearly imaged by matching the number of layers of the multi-layered reflector with the imaging depth of field, thereby maximizing the number of samples collected while ensuring the best sample quality.

[0044] Figure 3 This is a schematic diagram of the structure of a multi-layered reflector provided in a specific embodiment of this application. If the interface where the light spot is located can contact the air, the following problems may exist: If the interface is transparent, the laser scattering energy is relatively low, and even tiny dust particles can cause relatively strong scattering, significantly affecting the quality of the light spot; if the interface is opaque, the high-gloss surface will cause severe specular reflection, which will also lead to relatively strong scattering due to tiny dust particles, significantly affecting the quality of the light spot; to obtain high-quality images, such as Figure 3 As shown, a multi-layered reflector can be composed of several layers of high-transmittance flat glass. The upper surface of each glass layer is coated with an anti-reflective film, while the lower surface is uncoated. The bottom glass layer can be coated on both sides. The thickness of each glass layer and the number of glass layers can be determined according to specific circumstances. The glass thickness is preferably approximately three times the width of the laser spot, and the number of glass layers is preferably an integer part of the depth of field divided by the glass thickness.

[0045] In this embodiment of the application, the noise image collection device may include: Transparent panel; A light source, positioned above the transparent panel, is used to emit parallel light onto the transparent panel in order to illuminate the noise on the transparent panel; The second image acquisition device is used to acquire noisy images on the transparent plate.

[0046] In this embodiment, the noise image collection device is a hardware system used to simulate and acquire various typical abnormal light spot morphologies. The transparent plate serves as a carrier to simulate the surface of the actual object being measured, and its surface characteristics can be controlled. The parallel light source is a light source capable of emitting collimated beams; its light rays are almost non-divergent in the propagation direction, forming uniform illumination on a plane to simulate the directional illumination characteristics of a laser. Noise can be contaminated dust, minor scratches, or inherent defects. In this embodiment, naturally settled dust is used as an example; naturally settled dust refers to environmental particles that fall naturally onto the surface under gravity without forced conditions.

[0047] In one example, a transparent panel can be mounted on a programmable translation and rotation stage, allowing it to be exposed to different areas of an environmental chamber. The chamber contains a controllable dust generator that releases standard dust particles with varying particle size distributions at timed and quantitative intervals, simulating surfaces with different levels of contamination and particle characteristics. The parallel light source can be a highly uniform LED collimated light source system with an output light divergence angle of less than 1 degree to ensure parallelism of the illumination. A second image acquisition device is equipped with a high dynamic range camera, capable of simultaneously capturing bright and dark spots caused by dust in a single shot.

[0048] To obtain a richer variety of noise patterns, embodiments of this application may also lightly spray water mist onto the surface of the transparent plate to allow it to become dusty, simulating wet stains; or lightly sand the transparent plate in different directions with fine sandpaper to simulate surface scratches and other defects. All of these operations can be performed under automated control, thereby systematically constructing a large-scale noise image library containing various abnormal patterns.

[0049] In this way, through controlled simulation of pollution type, degree and surface condition, high-quality abnormal spot samples covering various real industrial scenarios can be systematically collected, providing a key negative sample data source for training a target semantic segmentation model with strong generalization ability.

[0050] Figure 4 This is a schematic diagram of a noise image collection device provided in a specific embodiment of this application. Figure 4 As shown, the noise image collection device comprises a camera 2, a lens 3, a parallel light source 5, and a transparent plate 6. The angle between the emitted parallel light from the parallel light source 5 and the plane of the transparent plate 6 is equal to the angle between the emitted laser from the line structured light laser 1 and the plane of the multi-layer reflective plate 4 in the spot image collection system. The emitted parallel light from the parallel light source 5 forms a large-area laser spot on the upper surface of the transparent plate 6. The camera 2 and lens 3 are combined to form a second image acquisition device, which is installed on one side of the transparent plate 6, allowing the spot area on the transparent plate 6 to be observed in the imaging field of view.

[0051] In this embodiment, the angle between the parallel light emitted by the light source and the transparent plate is equal to the angle between the emitted laser of the line structured light laser and the plane of the multi-layered reflective plate.

[0052] It is understandable that the shape, brightness distribution, and imaging characteristics of the laser spot are closely related to the incident angle of the illumination light. To obtain a dusty laser spot image (i.e., a noise sample image) under the same illumination conditions when the laser illuminates a dusty transparent plate surface, the spot image collection device and the noise image collection device must be set to the same illumination angle. Only when the illumination angles are consistent can the optical characteristics of dust and other contaminants imaged in the camera during noise image collection be consistent with the optical characteristics exhibited when the laser illuminates the dust on the surface of the measured object at the same angle during actual measurement. For example, when setting up the spot image collection device, the angle between the line structured laser and the multi-layer reflective plate plane is adjusted to 45 degrees. Therefore, when setting up the noise image collection device, the angle between the parallel light source and the transparent plate plane must also be strictly adjusted to 45 degrees.

[0053] By strictly maintaining the same illumination geometry between the spot image collection device and the noise image collection device, it can be ensured that the optical characteristics of the subsequently generated training data are consistent with those of the real measurement scene, which is beneficial to improving the generalization ability and recognition accuracy of the target semantic segmentation model in practical applications.

[0054] Figure 5 This is a schematic diagram of a measurement subsystem provided in a specific embodiment of this application. Figure 5 As shown, the measurement subsystem comprises a line structured light laser 1, a camera 2, a lens 3, and the object being measured 7. The laser emitted from the line structured light laser 1 illuminates the object being measured 7, forming a large laser spot on its surface. The camera 2 and lens 3 are combined and mounted on one side of the object being measured 7, allowing the imaging field to observe the laser spot formed on its surface.

[0055] In this embodiment of the application, the image processing subsystem can also be configured to: Semantic segmentation is performed on each spot sample image in the spot sample image set to segment out each spot in each spot sample image, resulting in an independent spot image set. Semantic segmentation is performed on each noise sample image in the noise sample image set to separate each noise point in each noise sample image, resulting in an independent noise image set. Based on the independent light spot image set and the independent noise image set, a training dataset is generated through data augmentation processing. Each group of training data in the training dataset includes augmented training images and corresponding labeled images. Based on a pre-set deep learning training model, a corresponding target semantic segmentation model is generated for each group of training data.

[0056] In this embodiment of the application, the independent spot image set or the independent noise image set refers to the set formed by saving each independent spot or noise region segmented from the original sample image as a small sub-image.

[0057] Specifically, in generating the training dataset based on the sets of spot and noise sample images, the image processing subsystem first loads the spot sample image set and calls a preset spot segmentation model to process each spot sample image, obtaining a precise pixel-level mask for each spot. Then, based on these masks, each spot is cropped from the original image and saved as an independent small image with a black background, forming an independent spot image set. Simultaneously, an independent noise image set is obtained using the same method. Next, based on the independent spot and noise image sets, a training dataset is generated through data augmentation. Each training dataset includes augmented training images and corresponding labeled images. Then, the augmented training images from each training dataset are input into a preset deep learning network. The deep learning network predicts the category of each pixel based on image features, then compares the prediction results with the corresponding labeled images, calculates the error, and adjusts the network parameters accordingly. Through repeated learning on a large amount of data, the deep learning network ultimately masters the ability to accurately segment abnormal spots from complex backgrounds, resulting in a target semantic segmentation model that can be applied in practice.

[0058] In this way, by automatically segmenting and augmenting the original samples, massive, diverse and accurately labeled training data are efficiently generated, solving the problems of data scarcity, high labeling costs and insufficient sample diversity faced in deep learning model training.

[0059] In a specific embodiment of this application, existing deep learning annotation software can be used to annotate and train the light spots on a portion of the light spot sample images in the light spot sample image set, generating a light spot semantic segmentation training model. As follows: Figure 6 As shown, Figure 6 This is a schematic diagram of light spot semantic segmentation provided in a specific embodiment of this application. In the figure, gray vertical stripes represent light spots. The left image is a schematic diagram of the original light spot image; the right image is a schematic diagram of the light spot region annotation, in which the light spot is delineated and the light spot region is annotated.

[0060] In a specific embodiment of this application, existing deep learning annotation software can be used to annotate and train noise on a portion of the noise sample images in the noise sample image set, generating a noise semantic segmentation training model. As follows: Figure 7 As shown, Figure 7This is a schematic diagram of noise semantic segmentation provided in a specific embodiment of this application. The left image is a schematic diagram of the original noise image; the right image is a schematic diagram of noise region annotation, in which the noise is delineated and the noise region is annotated.

[0061] In this embodiment of the application, the image processing subsystem can also be configured to: Multiple independent noise images are randomly extracted from a set of independent noise images. These multiple independent noise images are randomly distributed and fused into a single image to obtain a multi-noise image. A labeled image corresponding to the multi-noise image is then generated, in which the noise regions corresponding to the independent noise in the multi-noise image are marked. The multi-noise image is fused with an independent spot image randomly selected from a set of independent spot images to generate an augmented training image; A set of training data is obtained by associating augmented training images with labeled images; Repeat the above steps to generate multiple sets of training data to obtain the training dataset.

[0062] Specifically, multiple noisy images are randomly selected from a set of independent noisy images and randomly pasted onto a blank black background image of the same size as the original image, generating a multi-noisy image containing multiple randomly distributed noise points. Simultaneously, the image processing subsystem automatically generates a corresponding labeled image for this multi-noisy image, marking the precise locations of all noise points. Then, the image processing subsystem randomly selects a spot image from a set of independent spot images and fuses it with the multi-noisy image using image multiplication, thus synthesizing an augmented training image. This augmented training image contains a normal spot with multiple random noise points superimposed on it; the corresponding labeled image for this augmented training image directly inherits the labels from the multi-noisy image. Repeating the above process generates a large-scale training dataset.

[0063] In this embodiment of the application, the image processing subsystem can also be configured to: For each set of training data, from the beginning row to the end row of the augmented training image, an augmented image segment is extracted row by row according to a preset rule to form a subset of augmented image segments; Extract labeled image fragments from the corresponding labeled images that are at the same positions as the augmented training images, forming a subset of labeled image fragments; Based on a pre-defined deep learning training model, a target semantic segmentation model corresponding to each row position is trained and generated according to the augmented image fragment subset and the labeled image fragment subset for each row.

[0064] Specifically, for each set of augmented training images Wn and labeled images Vn in the training data, processing is performed row by row, starting from the m-th row (e.g., skipping the background area without light spots at the top) and ending at the i-th row. The specific processing flow is as follows: For each row j, locate the pixel with the largest grayscale value in that row. This pixel is usually the projection of the center line of the laser spot onto the row. Using this pixel as the center, extract t / 2 pixels to the left and right, forming a horizontal pixel segment of length t, and store it in the image segment subset Mj. Simultaneously, extract a pixel segment of the same length from the exact same position in the labeled image Vn and store it in the labeled segment subset Nj. ​​Here, m, t, and i are all settable parameters, and t is an odd number.

[0065] After repeating the above process to process all rows of all training images, for the j-th row of an image, a training subset specifically for that row position is obtained, consisting of fragments from all training images at that row. Subsequently, a separate target semantic segmentation model Moj is trained based on each row's training subset. In other words, the final result will be the target semantic segmentation model for each row.

[0066] Thus, by adopting a row-by-row differentiation training strategy, a dedicated model is built for each row of the image, which effectively reduces the learning difficulty of a single model and can better capture the local morphological features of the light spot at different row positions, thereby achieving a more refined and accurate anomaly recognition capability than the whole image model.

[0067] In this embodiment of the application, the image processing subsystem is further configured to: Perform the following operations line by line from the beginning to the end of the image to be processed: Extract the image segment to be processed from the current row according to preset rules; The target semantic segmentation model corresponding to the current row position is invoked to process the image segment to be processed in order to identify abnormal light spot regions; Modify the pixel values ​​of the abnormal light spot areas to the preset background values.

[0068] Specifically, when processing a new image P, the image processing subsystem also uses a line-by-line processing method to remove structured light spots. Specifically, for the j-th row of image P, the maximum grayscale value in that row is found, and a pixel segment of length t is extracted centered on this point; this segment is the image segment to be processed, denoted as Poj. Then, the target semantic segmentation model Moj, obtained during the training phase, is called to process Poj. Moj performs semantic segmentation on Poj, outputting a labeled segment of the same length, identifying which pixels are anomalous light spots. Next, based on this segmentation result, all pixel values ​​in the regions marked as anomalous in Poj are modified to a preset background value, such as 0, while the pixel values ​​in normal and background regions remain unchanged. The processed pixel segment is then filled back into the corresponding position of the j-th row of a new, initially empty output image Pe. After all rows have been processed, the output image Pe is the final result of the image P after removing all anomalous light spot regions.

[0069] In this way, by calling the dedicated model for the corresponding row to identify and remove anomalies within a row-by-row framework that is completely consistent with the training phase, the optimality and high precision of the processing flow are ensured, and a high-purity laser spot image that can be used for high-precision 3D reconstruction is finally output.

[0070] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0074] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0075] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0076] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0077] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0078] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A structured light spot local removal system, characterized in that, The structured light spot local elimination system includes: The sample image collection subsystem includes a spot image collection device and a noise image collection device, configured to collect a set of spot sample images and a set of noise sample images; The measurement subsystem is configured to acquire a processing image of the surface of the object being measured, the processing image containing light spots; The image processing subsystem, which communicates with both the sample image collection subsystem and the measurement subsystem, is configured to: The system receives a set of spot sample images and a set of noise sample images collected by the sample image collection subsystem, as well as an image to be processed acquired by the measurement subsystem. A training dataset is generated based on the set of light spot sample images and the set of noise sample images, and a target semantic segmentation model is constructed based on the training dataset. The target semantic segmentation model is used to identify abnormal light spots in the image. The target semantic segmentation model is invoked to process the image to be processed, so as to remove the abnormal light spot regions in the image to be processed, so as to obtain the image after removing the abnormal light spots.

2. The structured light spot local elimination system according to claim 1, characterized in that, The light spot image collection device includes: Multi-layered reflector; A line structured light laser is disposed above the multi-layered reflector and is used to emit laser light onto the multi-layered reflector so that multiple light spots are formed at multiple interfaces inside the multi-layered reflector. A first image acquisition device is disposed above the multi-layered reflector, the multi-layered reflector being located within the imaging field of view of the image acquisition device, and the first image acquisition device is used to acquire light spot images on the multi-layered reflector.

3. The structured light spot local elimination system according to claim 2, characterized in that, The multi-layered reflective plate is composed of multiple layers of transparent plates. The upper surface of each transparent plate is coated with an anti-reflective film, and the lower surface of the bottom transparent plate is coated with an anti-reflective film.

4. The structured light spot local elimination system according to claim 2, characterized in that, The number of layers in the multi-layered reflector is determined according to the depth of field of the image acquisition device.

5. The structured light spot local elimination system according to claim 2, characterized in that, The noise image collection device includes: Transparent panel; A light source is positioned above the transparent plate to emit parallel light onto the transparent plate in order to illuminate the noise points on the transparent plate; The second image acquisition device is used to acquire noise images on the transparent plate.

6. The structured light spot local elimination system according to claim 5, characterized in that, The angle between the parallel light emitted by the light source and the transparent plate is equal to the angle between the laser emitted by the line structured light laser and the plane of the multi-layered reflective plate.

7. The structured light spot local elimination system according to claim 1, characterized in that, The image processing subsystem is also configured to: Semantic segmentation is performed on each spot sample image in the spot sample image set to segment out each spot in each spot sample image, resulting in an independent spot image set. Semantic segmentation is performed on each noise sample image in the noise sample image set to segment out each noise point in each noise sample image, resulting in an independent noise image set. Based on the set of independent spot images and the set of independent noise images, a training dataset is generated through data augmentation processing. Each set of training data in the training dataset includes augmented training images and corresponding labeled images. Based on a preset deep learning training model, a corresponding target semantic segmentation model is generated for each group of training data.

8. The structured light spot local elimination system according to claim 7, characterized in that, The image processing subsystem is also configured to: Multiple independent noise images are randomly extracted from the set of independent noise images, and the multiple independent noise images are randomly distributed and fused into one image to obtain a multi-noise image. A labeled image corresponding to the multi-noise image is generated, and the labeled image marks the noise regions corresponding to the independent noise in the multi-noise image. The multi-noise image is fused with an independent spot image randomly selected from the set of independent spot images to generate the augmented training image; The augmented training images are associated with the labeled images to obtain a set of training data; Repeat the above steps to generate multiple sets of training data to obtain the training dataset.

9. The structured light spot local elimination system according to claim 7, characterized in that, The image processing subsystem is also configured to: For each set of training data, from the beginning row to the end row of the augmented training image, an augmented image segment is extracted row by row according to a preset rule to form a subset of augmented image segments; Extract labeled image segments from the corresponding labeled images that are at the same positions as the augmented training images to form a subset of labeled image segments; Based on the preset deep learning training model, a corresponding target semantic segmentation model is generated for each set of training data according to the augmented image fragment subset and the labeled image fragment subset.

10. The structured light spot local elimination system according to claim 9, characterized in that, The image processing subsystem is also configured to: Perform the following operations line by line from the beginning to the end of the image to be processed: Extract the image segment to be processed in the current row according to the preset rules; The target semantic segmentation model corresponding to the current row position is invoked to process the image segment to be processed in order to identify abnormal light spot regions therein; Modify the pixel values ​​of the abnormal light spot area to the preset background values.

Images