A method for extracting the center of laser structured light stripes

Abstract

The present invention discloses a method for extracting the center of laser structured light stripes, which is applied to the field of machine vision technology. The method comprises the following steps: collecting laser stripe images during the welding process and constructing a laser stripe dataset; preprocessing the laser stripe images and obtaining laser stripe pixels; designing a segmentation network, training and testing it using the laser stripe dataset; and extracting laser stripes using the trained segmentation network. The present invention constructs a laser structured light stripe extraction network with dual encoder fusion to achieve effective encoding of structured light stripe images, enriches the detail feature information of the segmentation network through a detail enhancement unit, and achieves effective fusion of features of different scales through a hierarchical feature fusion unit. The method accurately extracts the stripe center and serves weld tracking based on laser structured light.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and more particularly to a method for extracting the center of laser structured light stripes. Background Art

[0002] Laser structured light vision sensors offer stable illumination and are less susceptible to environmental influences. Furthermore, their specially engineered wavelength and integrated filter significantly reduce the impact of ambient light and arc light during welding on weld images, leading to their widespread application in intelligent welding robots. In weld tracking systems, laser structured light vision sensors extract weld seam information by extracting laser streaks as feedback signals. Their accuracy and reliability are directly linked to weld tracking performance, presenting a pressing challenge for those skilled in the art. Summary of the Invention

[0003] In view of this, the present invention provides a method for extracting the center of laser structured light stripes, constructs a laser structured light stripe center extraction network, and realizes automatic and accurate extraction of laser stripes.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A method for extracting the center of a laser structured light stripe comprises the following steps:

[0006] S1, collect laser stripe images during the welding process and construct a laser stripe dataset;

[0007] S2, preprocessing the laser stripe image to obtain laser stripe pixels;

[0008] S3. Design a segmentation network and train and test it on the laser stripe dataset.

[0009] S4. Use the trained segmentation network to extract laser stripes.

[0010] Optionally, S1 uses a laser structured light vision sensor as a visual perception method and is applied to the weld tracking system to collect laser stripe images under different conditions during the weld tracking process, construct a laser stripe dataset, and divide it into a training set, a validation set, and a test set to serve the training and testing of the model.

[0011] Optionally, the preprocessing in S2 is as follows: preprocessing of the laser stripe area is achieved through image grayscale change and median filtering algorithm, and arc interference in the welding process is removed by differential operation between adjacent images; automatic binary segmentation of the image is achieved through OSTU algorithm to obtain pixel-level laser stripe area.

[0012] Optionally, the design of the segmentation network in S3 includes: designing a semantic feature representation unit and a spatial feature representation unit, constructing a dual encoder structure, and realizing effective encoding of structured light stripe images; constructing an attention-guided interactive fusion unit to realize local feature enhancement; designing a detail enhancement unit to enrich the network's detail feature expression and reduce the impact of the semantic gap problem; constructing a hierarchical feature fusion unit to realize the effective fusion of features of different scales and enhance the network's ability to segment targets of different scales.

[0013] Optionally, the semantic feature representation unit includes three groups of convolutions, the first group of convolutions is one 3*3 convolution, the second group of convolutions is two 3*3 convolutions, and the third group of convolutions is three 3*3 convolutions, for obtaining different receptive fields.

[0014] Optionally, the spatial feature representation unit includes: 2 1*3 convolutions, 2 3*1 convolutions, and 1 3*3 convolution, which are used to enhance spatial detail features in the horizontal and vertical directions.

[0015] Optionally, the interactive fusion unit uses an addition operation to fuse the semantic feature representation unit and the spatial feature representation unit, and sends it to the hybrid attention sub-unit to obtain the attention feature, and multiplies the attention feature with the output feature map of the semantic representation unit and the spatial representation unit respectively to enrich the feature expression of the semantic feature representation unit and the spatial feature representation unit, thereby enhancing the local feature representation capability.

[0016] Optionally, the detail enhancement unit includes: 1 5×5 convolution, 2 1×1 convolutions, and 3 3×3 convolutions; the detail enhancement unit adopts a skip connection method, and through the upsampling-downsampling structure, it increases the receptive field of the network and obtains rich detail information, and through the downsampling-upsampling structure, removes the interference of noise information in the feature map.

[0017] Optionally, the hierarchical feature fusion unit includes a shallow semantic feature fusion subunit and a deep semantic feature fusion subunit. The shallow semantic feature fusion subunit adjusts the size and channel information of the feature map through upsampling operation and 1*1 convolution, and obtains the shallow fusion feature through addition operation; the deep semantic feature fusion subunit obtains the attention weight through addition operation, ReLU function, 1*1 convolution, BN layer and Sigmoid function, multiplies the attention weight with the feature maps of two deep layers to obtain the attention feature, and finally obtains the deep fusion feature through feature concatenation and 3*3 convolution; finally, the deep feature weight and the shallow feature weight are introduced and acted on the deep fusion feature and the shallow fusion feature respectively to obtain the weighted fusion feature.

[0018] It can be seen from the above technical solution that compared with the prior art, the present invention discloses a method for extracting the center of laser structured light stripes, which has the following beneficial effects: the present invention constructs a laser structured light stripe center extraction network to realize automatic and accurate extraction of laser stripes; designs a semantic feature representation unit and a spatial feature representation unit to construct a dual encoder structure to realize effective encoding of structured light stripe images; constructs a detail enhancement unit to enrich the detail feature information of the segmentation network, and at the same time, it can also effectively remove the interference of noise information to ensure the network's pixel-level laser stripe segmentation capability; in addition, in the face of feature information of different scales, a hierarchical feature fusion unit is constructed, and through the shallow semantic feature fusion unit and the deep semantic feature fusion unit, the effective fusion of features of different scales is realized, thereby enhancing the network's segmentation capability for targets of different scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0020] Figure 1 This is a flow chart of the laser structured light stripe center extraction method of the present invention;

[0021] Figure 2 Schematic diagram of the segmentation network of the present invention;

[0022] Figure 3 Schematic diagram of the semantic feature representation unit of the present invention;

[0023] Figure 4 Schematic diagram of a spatial feature characterization unit of the present invention;

[0024] Figure 5 is a schematic diagram of the interactive fusion unit of the present invention;

[0025] Figure 6 is a schematic diagram of a detail enhancement unit of the present invention;

[0026] Figure 7 Schematic diagram of the hierarchical feature fusion unit of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] The embodiment of the present invention discloses a method for extracting the center of laser structured light stripes. Figure 1 As shown, the following steps are included:

[0029] S1, collect laser stripe images during the welding process and construct a laser stripe dataset;

[0030] S2, preprocessing the laser stripe image to obtain laser stripe pixels;

[0031] S3. Design a segmentation network and train and test it on the laser stripe dataset.

[0032] S4. Use the trained segmentation network to extract laser stripes.

[0033] Furthermore, S1 uses a laser structured light vision sensor as a visual perception method and applies it to the weld tracking system. It collects laser stripe images under different conditions during the weld tracking process, constructs a laser stripe dataset, and divides it into training set, validation set and test set to serve the training and testing of the model.

[0034] Furthermore, preprocessing in S2 involves preprocessing the laser streak region through image grayscale changes and a median filter algorithm. Difference calculations between adjacent images are used to remove arc interference during welding. The OSTU algorithm is then used to automatically binarize the image and obtain the laser streak region at the pixel level. To ensure data annotation accuracy, data annotation software can be used to correct imperfect segmentation areas and accurately annotate the laser streak region at the pixel level.

[0035] Further, such as Figure 2 As shown, the segmentation network design in S3 includes: designing semantic feature representation units and spatial feature representation units, constructing a dual encoder structure to achieve effective encoding of structured light streak images; constructing an attention-guided interactive fusion unit to achieve local feature enhancement; designing a detail enhancement unit to enrich the network's detail feature expression and reduce the impact of the semantic gap problem; and constructing a hierarchical feature fusion unit to achieve effective fusion of features at different scales, enhancing the network's ability to segment objects of different scales. In this embodiment, a laser streak extraction network model is built using a deep network programming framework, which adopts an encoder-decoder network architecture as the baseline framework for the segmentation network.

[0036] Further, such as Figure 3 As shown in the figure, the semantic feature representation unit includes three groups of convolutions. The first group of convolutions is one 3*3 convolution, the second group of convolutions is two 3*3 convolutions, and the third group of convolutions is three 3*3 convolutions, which are used to obtain different receptive fields.

[0037] Furthermore, in the semantic feature representation unit, two 3*3 convolutions can obtain the receptive field of a 5*5 convolution, and three 3*3 convolutions can obtain the receptive field of a 7*7 convolution.

[0038] Further, such as Figure 4 As shown in the figure, the spatial feature representation unit includes: 2 1*3 convolutions, 2 3*1 convolutions, and 1 3*3 convolution, which are used to enhance the spatial detail features in the horizontal and vertical directions. It is especially suitable for linear target detection. Based on this feature, the spatial feature representation unit is constructed to effectively enhance the effective feature expression of the linear laser stripe area.

[0039] Further, such as Figure 5 As shown in the figure, the interactive fusion unit uses the addition operation to fuse the semantic feature representation unit and the spatial feature representation unit, and sends it to the hybrid attention sub-unit to obtain the attention feature, and multiplies the attention feature with the output feature map of the semantic representation unit and the spatial representation unit respectively to enrich the feature expression of the semantic feature representation unit and the spatial feature representation unit, thereby enhancing the local feature representation capability.

[0040] Further, such as Figure 6 As shown in the figure, the detail enhancement unit consists of one 5×5 convolution, two 1×1 convolutions, and three 3×3 convolutions. The detail enhancement unit uses skip connections and an upsampling-downsampling structure to increase the network's receptive field and capture rich detail information. The downsampling-upsampling structure removes noise interference from the feature map. The detail enhancement unit effectively enriches the network's detailed feature representation and mitigates the impact of the semantic gap. Furthermore, it effectively reduces the interference of noise in the feature map.

[0041] Further, such as Figure 7As shown in the figure, the hierarchical feature fusion unit includes a shallow semantic feature fusion subunit and a deep semantic feature fusion subunit. The shallow semantic feature fusion subunit adjusts the size and channel information of the feature map through upsampling operation and 1*1 convolution, and obtains the shallow fusion feature through addition operation; the deep semantic feature fusion subunit obtains the attention weight through addition operation, ReLU function, 1*1 convolution, BN layer and Sigmoid function, and multiplies the attention weight with the feature maps of the two deep layers to obtain the attention feature. Finally, the deep fusion feature is obtained through feature concatenation and 3*3 convolution; finally, the deep feature weight and the shallow feature weight are introduced and acted on the deep fusion feature and the shallow fusion feature respectively to obtain the weighted fusion feature.

[0042] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0043] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for extracting the center of laser structured light stripes, characterized in that: The following steps are involved: S1, collect laser stripe images during the welding process and construct a laser stripe dataset; S2, preprocessing the laser stripe image to obtain laser stripe pixels; S3. Design a segmentation network and train and test it on the laser stripe dataset. S4. Extract laser stripes using the trained segmentation network. The segmentation network design in S3 includes: designing semantic feature representation units and spatial feature representation units, constructing a dual encoder structure to achieve effective encoding of structured light stripe images; building an attention-guided interactive fusion unit to achieve local feature enhancement; designing a detail enhancement unit to enrich the network's detail feature expression and reduce the impact of the semantic gap problem; and building a hierarchical feature fusion unit to achieve effective fusion of features at different scales and enhance the network's ability to segment objects at different scales. The interactive fusion unit uses addition operations to fuse the semantic feature representation unit and the spatial feature representation unit, and sends them to the hybrid attention sub-unit to obtain attention features. The attention features are multiplied with the output feature maps of the semantic representation unit and the spatial representation unit respectively to enrich the feature expressions of the semantic feature representation unit and the spatial feature representation unit, thereby enhancing the local feature representation capability. The detail enhancement unit includes: 1 5×5 convolution, 2 1×1 convolutions, and 3 3×3 convolutions. The detail enhancement unit uses a skip connection method and an upsampling-downsampling structure to increase the network's receptive field and obtain rich detail information. The downsampling-upsampling structure removes the interference of noise information in the feature map. The hierarchical feature fusion unit includes a shallow semantic feature fusion subunit and a deep semantic feature fusion subunit. The shallow semantic feature fusion subunit adjusts the size and channel information of the feature map through upsampling operation and 1*1 convolution, and obtains the shallow fusion feature through addition operation; the deep semantic feature fusion subunit obtains the attention weight through addition operation, ReLU function, 1*1 convolution, BN layer and Sigmoid function, and multiplies the attention weight with the feature maps of the two deep layers to obtain the attention feature. Finally, the deep fusion feature is obtained through feature concatenation and 3*3 convolution; finally, the deep feature weight and the shallow feature weight are introduced and applied to the deep fusion feature and the shallow fusion feature respectively to obtain the weighted fusion feature.

2. The method for extracting the center of a laser structured light stripe according to claim 1, characterized in that: S1 uses a laser structured light vision sensor as a visual perception method and is applied to the weld tracking system. It collects laser stripe images under different conditions during the weld tracking process, constructs a laser stripe dataset, and divides it into training set, validation set, and test set to serve the training and testing of the model.

3. The method for extracting the center of a laser structured light stripe according to claim 1, characterized in that: The preprocessing in S2 is as follows: preprocessing of the laser stripe area is achieved through image grayscale change and median filtering algorithm, and the arc interference during the welding process is removed by differential operation between adjacent images; automatic binary segmentation of the image is achieved through the OSTU algorithm to obtain the laser stripe area at the pixel level.

4. The method for extracting the center of a laser structured light stripe according to claim 1, wherein: The semantic feature representation unit includes three groups of convolutions. The first group of convolutions is one 3*3 convolution, the second group of convolutions is two 3*3 convolutions, and the third group of convolutions is three 3*3 convolutions, which are used to obtain different receptive fields.

5. The method for extracting the center of a laser structured light stripe according to claim 1, characterized in that: The spatial feature representation unit includes: 2 1*3 convolutions, 2 3*1 convolutions, and 1 3*3 convolution, which are used to enhance the spatial detail features in the horizontal and vertical directions.

Images