Line structured light image segmentation method based on improved U-Net network

Abstract

The invention discloses a line structure light image segmentation method based on an improved U-Net network, and the method comprises the steps: S1, starting a laser device, projecting line structure laser to the surface of a to-be-measured object, and shooting a complete laser light strip image through an industrial camera; s2, inputting the complete light strip image obtained in the step S1 into an improved U-Net neural network for training to obtain a trained segmentation neural network; wherein the improved U-Net neural network specifically comprises a coding path and a decoding path; and S3, segmenting the line structured light image through the trained segmentation neural network to obtain a segmentation result. According to the invention, noise such as background light and reflected light can be effectively removed in a complex noise environment, and a laser stripe part can be accurately segmented. The improved method has a good removal effect on various noises, the segmentation accuracy reaches 90%, and the segmentation effect is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical measurement, and in particular, relates to a line structured light image segmentation method based on an improved U-Net network. Background Art

[0002] With the continuous development of 3D contour measurement technology, the application of line structured light 3D scanning measurement in various fields is increasing. The main principle is to generate 3D information of the surface of the target object by projecting line structured laser to scan the surface of the target object.

[0003] Laser triangulation is widely used in the field of line structured light 3D scanning measurement due to its simple principle and high measurement accuracy. Traditional scanning measurement methods generally only use a simple threshold segmentation method to pre-process the captured laser images. The processed laser images are easily mixed with noise and cannot meet the requirements of complex working conditions, thus affecting the 3D measurement results.

[0004] Based on the above-mentioned deficiencies in the prior art, the present invention is proposed. Summary of the invention

[0005] In order to solve the problems raised by the background technology, the present invention proposes a line structured light image segmentation method based on an improved U-Net network, which can accurately segment the laser stripe part in a complex noise environment, including the following steps:

[0006] Step S1, turning on the laser to project a line structure laser onto the surface of the object to be measured, and using an industrial camera to capture a complete laser light strip image;

[0007] Step S2, inputting the complete light strip image obtained in step S1 into an improved U-Net neural network for training to obtain a trained segmentation neural network; wherein the improved U-Net neural network specifically includes an encoding path and a decoding path;

[0008] Step S2.1, the encoding path includes an initial convolution module, 4 downsampling convolution modules and 4 SA attention mechanism modules;

[0009] The initial convolution module operates on the input original light stripe image with noise to obtain the initial encoding result;

[0010] Four downsampling convolution modules and four SA attention mechanism modules are processed sequentially to obtain a four-layer encoding result;

[0011] Step S2.2, the decoding path includes 4 AAM attention mechanism modules, 4 upsampling convolution modules and an output convolution module;

[0012] Taking the four-layer encoding results in step S2.1 as input, four upsampling convolution modules and four AAM attention mechanism modules are processed progressively in sequence, and finally the first-layer decoding results are obtained;

[0013] The output convolution module operates the first layer decoding result to obtain the neural network output result, uses the error function to calculate the error between the result and the true label, and then passes the error back to the neural network for cyclic training, and finally obtains a segmentation neural network with a small error value and stability. Step S3, segment the line structured light image through the trained segmentation neural network to obtain the segmentation result.

[0014] Preferably, the step S2.1 is as follows:

[0015] Step S2.1.1, the initial convolution module sequentially performs 1*1Conv, BatchNorm and ReLU operations on the input original light stripe image with noise to obtain the initial encoding result;

[0016] Step S2.1.2, the first downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the initial encoding result in sequence to obtain the downsampling result 1;

[0017] Step S2.1.3, the first SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 1 in sequence to obtain the first layer encoding result;

[0018] Step S2.1.4, the second downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the first layer encoding result in sequence to obtain the downsampling result 2;

[0019] Step S2.1.5, the second SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 2 in sequence to obtain the second layer encoding result;

[0020] Step S2.1.6, the third downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the second layer encoding result in sequence to obtain the downsampling result 3;

[0021] Step S2.1.7, the third SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 3 in sequence to obtain the third layer encoding result;

[0022] Step S2.1.8, the fourth downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the third layer encoding result in sequence to obtain the downsampling result 4;

[0023] In step S2.1.9, the fourth SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 4 in sequence to obtain the fourth layer encoding result.

[0024] Preferably,

[0025] The step S2.2 is specifically as follows:

[0026] Step S2.2.1, the first upsampling convolution module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the fourth layer encoding result in sequence to obtain the upsampling result 1;

[0027] Step S2.2.2, the first AAM attention mechanism module takes the fourth layer encoding result and the upsampling result 1 as input, and performs GAP, 1*1Conv, residual, Softmax and residual operations in sequence to obtain the fourth layer decoding result;

[0028] Step S2.2.3, the second upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the fourth layer decoding result in sequence to obtain the upsampling result 2;

[0029] Step S2.2.4, the second AAM attention mechanism module takes the third layer encoding result and the upsampling result 2 as input, and performs GAP, 1*1Conv, residual, Softmax and residual operations in sequence to obtain the third layer decoding result;

[0030] Step S2.2.5, the third upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the decoding result of the third layer in sequence to obtain the upsampling result 3;

[0031] Step S2.2.6, the third AAM attention mechanism module takes the second layer encoding result and the upsampling result 3 as input, and performs GAP, 1*1Conv, residual, Softmax and residual operations in sequence to obtain the second layer decoding result;

[0032] Step S2.2.7, the fourth upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the second layer decoding result in sequence to obtain an upsampling result 4;

[0033] Step S2.2.8, the fourth AAM attention mechanism module takes the first layer encoding result and the upsampling result 4 as input, and performs GAP (Global Average Pool), 1*1Conv, residual, Softmax and residual operations in sequence to obtain the first layer decoding result;

[0034] Step S2.2.9, the output convolution module performs 1*1Conv, ReLU and 1*1Conv operations on the first layer decoding results in sequence to obtain the line structured light image segmentation results.

[0035] Preferably,

[0036] The training process of the improved U-Net neural network in step S2 uses the Dice indicator to calculate the difference between the output and the true label as the loss function, and its specific formula is:

[0037]

[0038] In the formula, A represents the true label area, B represents the predicted area, and the Dice index represents the degree to which the predicted result is close to the true result.

[0039] The training adopts a batch size of 4, the Adam optimization algorithm, and sets an adaptive learning rate to adjust the parameters of the network model during the training process. A total of 150 rounds of training are performed to obtain a trained segmentation neural network.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] In view of the problem that the traditional line structured light image threshold segmentation method is easily mixed with noise, cannot meet the requirements of complex working conditions, and ultimately affects the three-dimensional reconstruction results, this paper proposes a line structured light image segmentation method based on an improved U-Net network, which can effectively remove background light, reflected light and other noise in a complex noise environment and accurately segment the laser stripe part. The improved method has a good removal effect on various noises, the segmentation accuracy rate reaches 90%, and the segmentation effect is significantly improved.

[0042] Terminology explanation:

[0043] Conv, the abbreviation of Convolution, means convolution, which is one of the core operations of segmentation neural network. It extracts the features of the image by using a set of learnable filters (convolution kernels) and gradually reduces the spatial dimension of the image.

[0044] SA, short for Shuffle Attention, is an attention mechanism used to improve the performance of segmentation neural networks. It introduces more rich context information by randomly disrupting the order of channels, helping the model to better learn and understand the semantic features of images.

[0045] BatchNorm is a technique commonly used in segmentation neural networks. It normalizes the input data in each training batch and adjusts the data to zero mean and unit variance by calculating the mean and standard deviation.

[0046] ReLU, meaning it is a common activation function, widely used in segmentation neural networks;

[0047] SlimConv, which means a lightweight convolutional layer design, is often used in segmentation neural networks. Compared with standard convolution, SlimConv reduces the number of parameters and computational overhead by reducing the number of channels and kernel size, while still maintaining good feature extraction capabilities;

[0048] Residual, which refers to the concept of skip connections in segmentation neural networks, is a special network layer design that allows features to be directly transmitted in the network without being affected by the intermediate layers. This design can effectively alleviate the gradient vanishing problem in deep networks, allowing the model to better learn low-level detail features and high-level semantic features;

[0049] Flip is a data enhancement technique commonly used in segmentation neural networks. It increases the diversity of training data by flipping the input image horizontally or vertically, which helps improve the generalization ability of the model.

[0050] Split, which means a splitting operation in the segmentation neural network, is mainly used to split the input feature map along the channel dimension to obtain multiple sub-feature maps, which can realize parallel processing of different sub-networks and improve computational efficiency;

[0051] Reconstruct refers to the concept of reconstruction operation in segmentation neural network, which restores the feature map or segmentation result finally output by the model to the segmentation result of the original image scale;

[0052] Transform, which means to perform geometric transformation on the input image or feature map to enhance the generalization ability of the model. Common operations include translation, scaling, rotation, etc.

[0053] Fuse means fusing feature information from different sources or at different levels to enhance feature expression capabilities;

[0054] AAM, short for augmented attention module, is an attention mechanism that improves model performance by enhancing contextual semantic information connections;

[0055] GAP, the abbreviation of Global Average Pool, means global pooling operation, which can apply the global average pooling layer on the feature map of the convolutional neural network;

[0056] Softmax, which means an activation function widely used in segmentation neural networks;

[0057] TransConv, which means transposed convolution operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flowchart of the line structured light image segmentation method based on the improved U-Net network of the present invention;

[0059] Figure 2 This is a schematic diagram of the improved U-Net network structure of the present invention;

[0060] Figure 3 It is a schematic diagram of the effect of the line structured light image segmentation method based on the improved U-Net network of the present invention;

[0061] in, Figure 3 a is the line structured light image, Figure 3 b is the segmentation result image;

[0062] Figure 4 This is an experimental comparison chart. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] Figure 1 FIG. 1 is a flowchart of a line structured light image segmentation method based on an improved U-Net network of the present invention; Figure 1 As shown in the figure, the line structured light image segmentation method based on the improved U-Net network of the present invention comprises the following steps:

[0065] Step S1, turning on the laser to project a line structure laser onto the surface of the object to be measured, and using an industrial camera to capture a complete laser light strip image;

[0066] Step S2, inputting the complete light strip image obtained in step 1 into an improved U-Net neural network for training to obtain a trained segmentation neural network;

[0067] Step S2 is based on the improved U-Net neural network structure reference Figure 2 , specifically including: encoding path and decoding path;

[0068] The encoding path described in step S2.1 includes an initial convolution module, four downsampling convolution modules and four SA (ShuffleAttention) attention mechanism modules;

[0069] Step S2.1.1: Further, the initial convolution module sequentially performs 1*1Conv (Convolution), BatchNorm and ReLU operations on the input original light stripe image with noise to obtain an initial encoding result;

[0070] Step S2.1.2: The first downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the initial encoding result in sequence to obtain the downsampling result 1;

[0071] Step S2.1.3: The first SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 1 in sequence to obtain the first layer encoding result;

[0072] Step S2.1.4: The second downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the first layer encoding result in sequence to obtain the downsampling result 2;

[0073] Step S2.1.5: The second SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 2 in sequence to obtain the second layer encoding result;

[0074] Step S2.1.6 The third downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the second layer encoding result in sequence to obtain the downsampling result 3;

[0075] Step S2.1.7 The third SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 3 in sequence to obtain the third layer encoding result;

[0076] Step S2.1.8 The fourth downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the third layer encoding result in sequence to obtain the downsampling result 4;

[0077] Step S2.1.9 The fourth SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 4 in sequence to obtain the fourth layer encoding result;

[0078] The decoding path in step S2.2 includes 4 AAM (augmented attention module) attention mechanism modules, 4 upsampling convolution modules and an output convolution module;

[0079] Step S2.2.1: Further, the first upsampling module sequentially performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the fourth layer encoding result to obtain an upsampling result 1;

[0080] Step S2.2.2: The first AAM attention mechanism module takes the fourth layer encoding result and the upsampling result 1 as input, and performs GAP (Global Average Pool), 1*1Conv, residual, Softmax and residual operations in sequence to obtain the fourth layer decoding result;

[0081] Step S2.2.3: The second upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the fourth layer decoding result in sequence to obtain the upsampling result 2;

[0082] Step S2.2.4 The second AAM attention mechanism module takes the third-layer encoding result and the upsampling result 2 as input, and performs GAP (Global Average Pool), 1*1Conv, residual, Softmax and residual operations in sequence to obtain the third-layer decoding result;

[0083] Step S2.2.5: The third upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the decoding result of the third layer in sequence to obtain the upsampling result 3;

[0084] Step S2.2.6 The third AAM attention mechanism module takes the second layer encoding result and the upsampling result 3 as input, and performs GAP (Global Average Pool), 1*1Conv, residual, Softmax and residual operations in sequence to obtain the second layer decoding result;

[0085] Step S2.2.7 The fourth upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the second layer decoding result in sequence to obtain an upsampling result 4;

[0086] Step S2.2.8 The fourth AAM attention mechanism module takes the first layer encoding result and the upsampling result 4 as input, and performs GAP (Global Average Pool), 1*1Conv, residual, Softmax and residual operations in sequence to obtain the first layer decoding result;

[0087] In step S2.2.9, the output convolution module further performs 1*1Conv, ReLU and 1*1Conv operations on the first layer decoding results in sequence to obtain the line structured light image segmentation results.

[0088] Step S3, segmenting the line structured light image using the trained segmentation neural network to obtain a segmentation result;

[0089] The training process uses the Dice indicator to calculate the difference between the output and the true label as the loss function. The specific formula is:

[0090]

[0091] The training adopts the batch size of 4, the Adam optimization algorithm, and sets the adaptive learning rate to adjust the parameters of the network model during the training process. A total of 150 rounds of training are performed to obtain a trained segmentation neural network. The segmentation effect is as follows: Figure 3 shown.

[0092] In view of the problem that the traditional line structured light image threshold segmentation method is easily mixed with noise, cannot meet the requirements of complex working conditions, and ultimately affects the three-dimensional reconstruction results, this paper proposes a line structured light image segmentation method based on an improved U-Net network, which can effectively remove background light, reflected light and other noise in a complex noise environment and accurately segment the laser stripe part. The method of the present invention is compared with the U-Net segmentation method, and the results are as follows Figure 4 As shown in the figure, the improved method has a good effect in removing various noises, the segmentation accuracy rate reaches 90%, and the segmentation effect is significantly improved.

[0093] Although the functions and working processes of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the above-mentioned specific functions and working processes. The above-mentioned specific implementation methods are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all within the protection of the present invention.

Claims

1. A line structured light image segmentation method based on an improved U-Net network, characterized in that: The following steps are included: Step S1, turning on the laser to project a line structure laser onto the surface of the object to be measured, and using an industrial camera to capture a complete laser light strip image; Step S2, inputting the complete light strip image obtained in step S1 into an improved U-Net neural network for training to obtain a trained segmentation neural network; Wherein, the improved U-Net neural network specifically includes an encoding path and a decoding path; Step S2.1, the encoding path includes an initial convolution module, 4 downsampling convolution modules and 4 SA attention mechanism modules; The initial convolution module operates on the input original light stripe image with noise to obtain the initial encoding result; Four downsampling convolution modules and four SA attention mechanism modules are processed sequentially to obtain a four-layer encoding result; Step S2.2, the decoding path includes 4 AAM attention mechanism modules, 4 upsampling convolution modules and an output convolution module; Taking the four-layer encoding results in step S2.1 as input, four upsampling convolution modules and four AAM attention mechanism modules are processed progressively in sequence, and finally the first-layer decoding results are obtained; The output convolution module operates on the first layer decoding result to obtain the neural network output result, uses the error function to calculate the error between the result and the true label, and then passes the error back to the neural network for cyclic training, and finally obtains a segmentation neural network with a small error value and stability. Step S3, segmenting the line structured light image using the trained segmentation neural network to obtain a segmentation result.

2. According to claim 1, a line structured light image segmentation method based on an improved U-Net network is characterized in that: The step S2.1 is specifically as follows: Step S2.1.1, the initial convolution module sequentially performs 1*1Conv, BatchNorm and ReLU operations on the input original light stripe image with noise to obtain the initial encoding result; Step S2.1.2, the first downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the initial encoding result in sequence to obtain the downsampling result 1; Step S2.1.3, the first SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 1 in sequence to obtain the first layer encoding result; Step S2.1.4, the second downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the first layer encoding result in sequence to obtain the downsampling result 2; Step S2.1.5, the second SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 2 in sequence to obtain the second layer encoding result; Step S2.1.6, the third downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the second layer encoding result in sequence to obtain the downsampling result 3; Step S2.1.7, the third SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 3 in sequence to obtain the third layer encoding result; Step S2.1.8, the fourth downsampling convolution module performs 1*1Conv, BatchNorm, ReLU, 3*3SlimConv, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the third layer encoding result in sequence to obtain the downsampling result 4; In step S2.1.9, the fourth SA attention mechanism module performs Flip, Split, Reconstruct, Transform and Fuse operations on the downsampling result 4 in sequence to obtain the fourth layer encoding result.

3. The line structured light image segmentation method based on the improved U-Net network according to claim 1, characterized in that: The step S2.2 is specifically as follows: Step S2.2.1, the first upsampling convolution module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the fourth layer encoding result in sequence to obtain the upsampling result 1; Step S2.2.2, the first AAM attention mechanism module takes the fourth layer encoding result and the upsampling result 1 as input, and performs GAP, 1*1Conv, residual, Softmax and residual operations in sequence to obtain the fourth layer decoding result; Step S2.2.3, the second upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the fourth layer decoding result in sequence to obtain the upsampling result 2; Step S2.2.4, the second AAM attention mechanism module takes the third layer encoding result and the upsampling result 2 as input, and performs GAP, 1*1Conv, residual, Softmax and residual operations in sequence to obtain the third layer decoding result; Step S2.2.5, the third upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the decoding result of the third layer in sequence to obtain the upsampling result 3; Step S2.2.6, the third AAM attention mechanism module takes the second layer encoding result and the upsampling result 3 as input, and performs GAP, 1*1Conv, residual, Softmax and residual operations in sequence to obtain the second layer decoding result; Step S2.2.7, the fourth upsampling module performs 1*1Conv, BatchNorm, ReLU, TransConv, BatchNorm, ReLU, 1*1Conv, BatchNorm, residual and ReLU operations on the second layer decoding result in sequence to obtain an upsampling result 4; Step S2.2.8, the fourth AAM attention mechanism module takes the first layer encoding result and the upsampling result 4 as input, and performs GAP (Global Average Pool), 1*1Conv, residual, Softmax and residual operations in sequence to obtain the first layer decoding result; Step S2.2.9, the output convolution module performs 1*1Conv, ReLU and 1*1Conv operations on the first layer decoding results in sequence to obtain the line structured light image segmentation results.

4. The method for denoising a three-dimensional point cloud of an optical rotation scanning measuring instrument based on point neighborhood distance clustering according to claim 1, characterized in that: The training process of the improved U-Net neural network in step S2 uses the Dice indicator to calculate the difference between the output and the true label as the loss function, and its specific formula is: In the formula, A represents the true label area, B represents the predicted area, and the Dice index represents the degree to which the predicted result is close to the true result. The training adopts a batch size of 4, the Adam optimization algorithm, and sets an adaptive learning rate to adjust the parameters of the network model during the training process. A total of 150 rounds of training are performed to obtain a trained segmentation neural network.

Images