Laser stripe extraction method, system and device for weld seam image and storage medium

Abstract

The present application belongs to the field of weld image processing, and discloses a laser stripe extraction method, system, device and storage medium for a weld image, comprising: acquiring a weld image, inputting the weld image into a preset laser stripe extraction model to extract laser stripes, and obtaining a laser stripe mask image of the weld image; and performing an AND operation on the weld image and the laser stripe mask image to obtain laser stripes of the weld image. The laser stripe extraction model is obtained by training a lightweight feature extraction model, which can effectively reduce the feature map redundancy and in turn reduce the parameter quantity of the entire model. In addition, a multi-head attention mechanism layer is provided, which can effectively improve the accuracy of the model compared to the simple convolution method. The features of the corresponding encoding layer and decoding layer are spliced to effectively enhance the segmentation effect, while ensuring lightweight and not reducing the extraction accuracy of the model. In addition, the accuracy of the laser stripe extraction result is ensured while greatly reducing the operation time and operation cost.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of graphic data reading, and relates to a laser stripe extraction method, system, device and storage medium for a weld seam image. BACKGROUND

[0002] Welding is an important manufacturing process for connecting multiple workpieces into a whole, and is widely used in the industrial fields of automobile manufacturing, shipbuilding and aerospace. In recent years, with the rapid development of economy, higher requirements are put forward for the quality and efficiency of welding. The traditional manual welding method cannot meet the needs of modern manufacturing due to the limitations of technical level, harsh welding environment and other objective and subjective factors. The application of intelligent welding robots solves the above problems and greatly improves the quality and efficiency of welding.

[0003] The key technology of an intelligent welding robot is to obtain weld seam information to guide the robot to complete the welding work. At present, the laser-based structured light vision weld seam tracking scheme is widely used due to its good anti-interference performance, high measurement accuracy and other characteristics. The laser-based structured light vision scheme projects laser light onto the surface of the workpiece through a laser generator to generate a laser structured light plane, forms a laser stripe, then collects a weld seam image through an industrial camera, reads the image data by a weld seam reading method, segments the weld seam, and finally extracts the weld seam information by a feature extraction method to provide parameters for the weld seam tracking system to guide the robot to complete the welding.

[0004] However, due to the generation of a large amount of arc light, spatter and smoke noise and other noise in the actual welding process, the collected image data is filled with a large amount of image noise, which poses a great challenge to the segmentation of the weld seam. The traditional image processing-based segmentation method is easily affected by strong noise in the image, resulting in reduced segmentation accuracy. Although the deep learning-based method has achieved good accuracy, it is difficult to apply to actual production environment due to the large amount of calculation, slow response speed and high cost. SUMMARY

[0005] The present application aims to overcome the shortcomings of low extraction result accuracy, large amount of calculation and high calculation cost in the prior art when extracting laser stripes from a weld seam image, and provides a laser stripe extraction method, system, device and storage medium for a weld seam image.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a laser stripe extraction method for a weld seam image, comprising: obtaining a weld seam image, and inputting the weld seam image into a preset laser stripe extraction model to perform laser stripe extraction, to obtain a laser stripe mask image of the weld seam image; performing AND operation on the weld seam image and the laser stripe mask image, to obtain a laser stripe of the weld seam image. The laser stripe extraction model is obtained by training a lightweight feature extraction model. The lightweight feature extraction model comprises an encoder, a sparse self-attention layer, a decoder and a segmentation head. The encoder comprises first, second, third and fourth encoding layers connected in sequence. The sparse self-attention layer comprises a first convolution unit and a multi-head attention mechanism layer connected in sequence. The decoder comprises first, second, third and fourth decoding layers connected in sequence. The output of the first encoding layer is connected to the input of the fourth decoding layer. The output of the second encoding layer is connected to the input of the third decoding layer. The output of the third encoding layer is connected to the input of the second decoding layer. The output of the fourth encoding layer is connected to the input of the first decoding layer. The output of the fourth encoding layer is connected to the input of the first convolution unit. The output of the multi-head attention mechanism layer is connected to the input of the first decoding layer. The output of the fourth decoding layer is connected to the input of the segmentation head. The first, second, third and fourth encoding layers and the first decoding layer each comprise a second convolution unit and a dynamic compression multi-layer perception connected in sequence. The dynamic compression multi-layer perception comprises a depth convolution layer, a first normalization layer, a first multi-layer perception, a second multi-layer perception and a first activation layer connected in sequence. The first convolution unit, the second convolution unit, the second, third and fourth decoding layers, the depth convolution layer and the segmentation head are each constructed by a convolution block.

[0008] Optionally, when training the lightweight feature extraction model, the following loss function is used

[0009]

[0010] wherein, is a pixel value matrix of the laser stripe mask image output by the lightweight feature extraction model after inputting the current weld seam image, y is a pixel value matrix of the artificially labeled laser stripe mask image of the current weld seam image; y i is the value of the i th element in y; n is the total number of element values in y, |Q| is the number of non-zero element values in the matrix Q, or y; y i =0 or 1; wherein, y i =1 indicates that the i th pixel point of the laser stripe mask image output by the lightweight feature extraction model is a laser stripe pixel point, y ii-th pixel point of the laser stripe mask image output by the lightweight feature extraction model is a background pixel point; For the i-th element value in the vector, or 1; wherein, indicates that the i-th pixel point of the laser stripe mask image is a laser stripe pixel point, indicates that the i-th pixel point of the laser stripe mask image is a background pixel point.

[0011] Optionally, when training the lightweight feature extraction model, the model parameters of the lightweight feature extraction model are updated by using back propagation.

[0012] Optionally, the first convolution unit, the second convolution unit of the second encoding layer, the second convolution unit of the third encoding layer, the second convolution unit of the fourth encoding layer, and the second convolution unit of the first decoding layer are all 2x2 convolution blocks; the second convolution unit of the first encoding layer is a 4x4 convolution block; the second decoding layer, the third decoding layer, and the fourth decoding layer are all 3x3 convolution blocks, and the segmentation head is a 1x1 convolution block.

[0013] Optionally, the depth convolution layer is a 3x3 convolution block, the first normalization layer is a BN layer, the activation function of the first normalization layer is a SiLu activation function, and the activation function of the first activation layer is a GELU activation function.

[0014] Optionally, the encoder further comprises a first dynamic compression attention layer, and the decoder further comprises a second dynamic compression attention layer; the fourth encoding layer is connected with the first convolution unit and the first decoding layer through the first dynamic compression attention layer; the first decoding layer is connected with the second decoding layer through the second dynamic compression attention layer; the first dynamic compression attention layer and the second dynamic compression attention layer both comprise an adaptive max-pooling layer, a third multi-layer perceptron, a second normalization layer, a fourth multi-layer perceptron, a second activation layer, an up-sampling layer, a third activation layer, and a multiplication operation layer connected in sequence; the inputs of the adaptive pooling layer and the multiplication operation layer of the first dynamic compression attention layer are collectively taken as the input of the first dynamic compression attention layer; the inputs of the adaptive pooling layer and the multiplication operation layer of the second dynamic compression attention layer are collectively taken as the input of the second dynamic compression attention layer; the multiplication operation layer of the first dynamic compression attention layer is used to multiply the input of the first dynamic compression attention layer and the output of the third activation layer of the first dynamic compression attention layer; the multiplication operation layer of the second dynamic compression attention layer is used to multiply the input of the second dynamic compression attention layer and the output of the third activation layer of the second dynamic compression attention layer.

[0015] Optionally, the second normalization layer is an LN layer, the activation function of the second activation layer is a GELU activation function, the upsampling layer is a bilinear interpolation function, and the activation function of the third activation layer is a Sigmoid activation function.

[0016] In a second aspect, the present application provides a laser stripe extraction system for a weld seam image, comprising: a model calling module, configured to obtain a weld seam image, and input the weld seam image into a preset laser stripe extraction model to perform laser stripe extraction, and obtain a laser stripe mask image of the weld seam image; and an image processing module, configured to perform AND operation on the weld seam image and the laser stripe mask image to obtain a laser stripe of the weld seam image; wherein the laser stripe extraction model is obtained by training a lightweight feature extraction model; the lightweight feature extraction model comprises an encoder, a sparse self-attention layer, a decoder and a segmentation head; the encoder comprises first, second, third and fourth encoding layers connected in sequence, the sparse self-attention layer comprises a first convolution unit and a multi-head attention mechanism layer connected in sequence, and the decoder comprises first, second, third and fourth decoding layers connected in sequence; the output of the first encoding layer is connected with the input of the fourth decoding layer; the output of the second encoding layer is connected with the input of the third decoding layer; the output of the third encoding layer is connected with the input of the second decoding layer; the output of the fourth encoding layer is connected with the input of the first decoding layer; the output of the fourth encoding layer is connected with the input of the first convolution unit, the output of the multi-head attention mechanism layer is connected with the input of the first decoding layer; the output of the fourth decoding layer is connected with the input of the segmentation head; the first, second, third and fourth encoding layers and the first decoding layer each comprise a second convolution unit and a dynamic compression multi-layer perception connected in sequence; the dynamic compression multi-layer perception comprises a depth convolution layer, a first normalization layer, a first multi-layer perception, a second multi-layer perception and a first activation layer connected in sequence; the first and second convolution units, the second, third and fourth decoding layers, the depth convolution layer and the segmentation head are each constructed by a convolution block.

[0017] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the laser stripe extraction of the weld seam image.

[0018] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the laser stripe extraction of the weld seam image.

[0019] Compared with the prior art, the present application has the following beneficial effects:

[0020] The laser stripe extraction method of the weld image uses a preset laser stripe extraction model to extract the laser stripe of the obtained weld image, obtains a laser stripe mask image of the weld image, and then performs an AND operation on the weld image and the laser stripe mask image of the weld image to obtain the laser stripe of the weld image. The laser stripe extraction model is trained by a lightweight feature extraction model, and the lightweight feature extraction model is provided with four encoding layers and four decoding layers. Each encoding layer uses a dynamic compression multi-layer perception to find effective semantic information, and different dynamic compression multi-layer perceptions are independent of each other, so that the parameters of each encoding layer can be independently set to adapt to the feature extraction requirements at different stages, thereby effectively reducing the feature map redundancy and reducing the parameter amount of the entire model. In addition, a sparse self-attention layer based on a multi-head attention mechanism layer is set, which effectively improves the accuracy of the model compared with the pure convolution method. Meanwhile, each encoding layer and each decoding layer uses a skip connection mode to splice the features of the corresponding layers, which considers both high-level features and low-level features, effectively enhances the segmentation effect, and finally ensures the lightweight while not reducing the extraction accuracy of the model, thereby greatly reducing the operation time and operation cost while ensuring the accuracy of the laser stripe extraction result.

[0021] Further, the first dynamic compression attention layer and the second dynamic compression attention layer are set, the adaptive max pooling layer is used to select appropriate values, and the multi-layer perception is used to calculate the global attention due to the fixed length of the control value. In the case of linear time complexity, the information in the spatial domain is fused, and a larger receptive field can achieve better results for the segmentation task, which is equivalent to fusing the information of the entire image. Good separation effect is obtained in the experiment, and the increase in the amount of calculation is small. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The laser stripe extraction method of the weld image of the embodiment of the present application is a flow chart.

[0023] Figure 2 The lightweight feature extraction model structure block diagram of the embodiment of the present application is a lightweight feature extraction model structure block diagram.

[0024] Figure 3 The dynamic compression multi-layer perception structure block diagram of the embodiment of the present application is a dynamic compression multi-layer perception structure block diagram.

[0025] Figure 4 The dynamic compression attention layer structure block diagram of the embodiment of the present application is a dynamic compression attention layer structure block diagram.

[0026] Figure 5 The laser stripe extraction system structure block diagram of the weld image of the embodiment of the present application is a laser stripe extraction system structure block diagram. DETAILED DESCRIPTION

[0027] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort should belong to the protection scope of the present application.

[0028] It should be noted that the terms "first", "second", and the like in the specification of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device including a series of steps or units does not necessarily have to be limited to the clearly listed steps or units, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0029] The present application will be described in further detail below with reference to the drawings:

[0030] Referring to Figure 1 In an embodiment of the present application, a laser stripe extraction method for a weld image is provided, which realizes extraction of a laser stripe mask image based on a lightweight laser stripe extraction model, and then realizes acquisition of a laser stripe according to the laser stripe mask image, effectively reducing the calculation amount and calculation time during laser stripe extraction.

[0031] Specifically, the laser stripe extraction method for the weld image includes the following steps:

[0032] S1: A weld image is acquired, and the weld image is input into a preset laser stripe extraction model for laser stripe extraction, to obtain a laser stripe mask image of the weld image.

[0033] S2: The weld image and the laser stripe mask image are subjected to AND operation, to obtain a laser stripe of the weld image.

[0034] The laser stripe extraction model is obtained by training a lightweight feature extraction model.

[0035] The lightweight feature extraction model comprises an encoder, a sparse self-attention layer, a decoder and a segmentation head; the encoder comprises a first encoding layer, a second encoding layer, a third encoding layer and a fourth encoding layer connected in sequence, the sparse self-attention layer comprises a first convolution unit and a multi-head attention mechanism layer connected in sequence, and the decoder comprises a first decoding layer, a second decoding layer, a third decoding layer and a fourth decoding layer connected in sequence; the output of the first encoding layer is connected with the input of the fourth decoding layer; the output of the second encoding layer is connected with the input of the third decoding layer; the output of the third encoding layer is connected with the input of the second decoding layer; the output of the fourth encoding layer is connected with the input of the first decoding layer; the output of the fourth encoding layer is connected with the input of the first convolution unit, the output of the multi-head attention mechanism layer is connected with the input of the first decoding layer; and the output of the fourth decoding layer is connected with the input of the segmentation head.

[0036] The first encoding layer, the second encoding layer, the third encoding layer, the fourth encoding layer and the first decoding layer each comprise a second convolution unit and a dynamic compression multi-layer perception connected in sequence; the dynamic compression multi-layer perception comprises a deep convolution layer, a first normalization layer, a first multi-layer perception, a second multi-layer perception and a first activation layer connected in sequence; the first convolution unit, the second convolution unit, the second decoding layer, the third decoding layer, the fourth decoding layer, the deep convolution layer and the segmentation head are each constructed by a convolution block.

[0037] The weld seam image is specifically a laser stripe image formed by projecting a laser plane to a workpiece surface using a laser.

[0038] The encoder is used to extract sufficient and effective information of the weld seam image, the second convolution unit is used for down-sampling, and the dynamic compression multi-layer perception is used to find effective semantic information, because the semantic information of the weld seam image is not complex, but the fineness of the result has a high requirement, the four encoding layers are set to extract better semantic features, and the outputs of each encoding layer are reserved for subsequent skip connection. The output of the last encoding layer is processed by the first convolution unit and the multi-head attention mechanism layer. The multi-head attention mechanism layer uses the multi-head attention mechanism of the transformer model, has a very small calculation amount under the overall structure, but has a very good extraction effect. For each decoding layer, the output of the previous layer is added to the output of the encoding layer connected in front of the skip connection, and then is up-sampled by the decoding layer to restore the size of the weld seam image, forming a symmetrical U-shaped structure.

[0039] Optionally, the encoder first reduces the image resolution of the weld image by 4 times for downsampling, and then reduces the image resolution of the weld image by 2 times for downsampling, and the corresponding decoder first recovers the image resolution of the weld image by 2 times for upsampling, and then recovers the image resolution of the weld image by 4 times for upsampling, and finally obtains the final segmentation result through the segmentation head.

[0040] Among them, the first and second dynamic compression multi-layer perceptrons of the dynamic compression multi-layer perceptron are respectively used to realize multiple reduction and multiple recovery of the number of feature maps, and the multiple can be set based on training and artificial experience, compared with the static feature map number setting mode, this dynamic mode is more suitable for the feature map number demand at different stages, and effectively reduces the feature map redundancy.

[0041] To sum up, the laser stripe extraction method of the weld image, the laser stripe extraction model is used to extract the laser stripe of the obtained weld image, and the laser stripe mask image of the weld image is obtained, and then the weld image and the laser stripe mask image of the weld image are operated, and the laser stripe of the weld image is obtained. The laser stripe extraction model is obtained by training a lightweight feature extraction model, and the lightweight feature extraction model sets four encoding layers and four decoding layers, each encoding layer uses a dynamic compression multi-layer perceptron to find effective semantic information, and different dynamic compression multi-layers are independent of each other, so the parameters of each layer can be independently set to adapt to the feature extraction demand at different stages, thereby effectively reducing the feature map redundancy, and reducing the parameter amount of the whole model. In addition, the sparse self-attention layer based on the multi-head attention mechanism layer is set, which effectively improves the precision of the model compared with the pure convolution method, and the features of the corresponding layers are spliced by using the skip connection method in each encoding layer and each decoding layer, which considers both high-level features and low-level features, effectively enhances the segmentation effect, and finally ensures the lightweight while not reducing the extraction precision of the model, thereby ensuring the accuracy of the laser stripe extraction result while greatly reducing the operation time and operation cost.

[0042] In one possible implementation, when training the lightweight feature extraction model, the following loss function is used

[0043]

[0044] Among them, is the pixel value matrix of the laser stripe mask image output by the lightweight feature extraction model after inputting the current weld image, y is the pixel value matrix of the artificial labeled laser stripe mask image of the current weld image; y iLet be the value of the i-th element in y; n is . The total number of elements in y; |Q| is the number of non-zero elements in matrix Q. or y; y i = 0 or 1; where y i =1 indicates that the i-th pixel in the laser stripe mask image output by the lightweight feature extraction model is a laser stripe pixel, y i =0 indicates that the i-th pixel in the laser stripe mask image output by the lightweight feature extraction model is a background pixel; for The value of the i-th element in the array. Or 1; where, This indicates that the i-th pixel in the manually annotated laser stripe mask image is the laser stripe pixel. This indicates that the i-th pixel of the manually annotated laser stripe mask image is a background pixel.

[0045] Specifically, when training the lightweight feature extraction model, several weld seam image samples are first manually labeled to obtain manually labeled laser stripe mask images for each weld seam image. Then, the labeled weld seam image samples are divided into training and testing sets, and the lightweight feature extraction model is trained using the training set. Optionally, when training the lightweight feature extraction model using the training set, backpropagation is used to update the model parameters, causing the lightweight feature extraction model to gradually converge, i.e., the loss function curve of the lightweight feature extraction model does not oscillate.

[0046] Finally, the trained lightweight feature extraction model is tested on the test set. When the trained lightweight feature extraction model meets the preset index requirements, the model parameters at this time are retained and applied to the lightweight feature extraction model to obtain the laser stripe extraction model.

[0047] In one possible implementation, the first convolutional unit, the second convolutional unit of the second coding layer, the second convolutional unit of the third coding layer, the second convolutional unit of the fourth coding layer, and the second convolutional unit of the first decoding layer are all 2×2 convolutional blocks; the second convolutional unit of the first coding layer is a 4×4 convolutional block; the second decoding layer, the third decoding layer, and the fourth decoding layer are all 3×3 convolutional blocks, and the segmentation head is a 1×1 convolutional block.

[0048] In one possible implementation, the deep convolutional layer is a 3×3 convolutional block, the first normalization layer is a BN (Batch Normalization) layer, the activation function of the first normalization layer is the SiLu activation function, and the activation function of the first activation layer is the GELU activation function.

[0049] In a possible implementation, the encoder further comprises a first dynamic compression attention layer, and the decoder further comprises a second dynamic compression attention layer; the fourth encoding layer is connected with the first convolution unit and the first decoding layer through the first dynamic compression attention layer; the first decoding layer is connected with the second decoding layer through the second dynamic compression attention layer; the first dynamic compression attention layer and the second dynamic compression attention layer each comprise an adaptive max-pooling layer, a third multi-layer perceptron, a second normalization layer, a fourth multi-layer perceptron, a second activation layer, an up-sampling layer, a third activation layer and a multiplication operation layer connected in sequence; the input of the adaptive max-pooling layer and the multiplication operation layer of the first dynamic compression attention layer are collectively taken as the input of the first dynamic compression attention layer; the input of the adaptive max-pooling layer and the multiplication operation layer of the second dynamic compression attention layer are collectively taken as the input of the second dynamic compression attention layer; the multiplication operation layer of the first dynamic compression attention layer is used for multiplying the input of the first dynamic compression attention layer and the output of the third activation layer of the first dynamic compression attention layer; and the multiplication operation layer of the second dynamic compression attention layer is used for multiplying the input of the second dynamic compression attention layer and the output of the third activation layer of the second dynamic compression attention layer.

[0050] Specifically, the first dynamic compression attention layer with linear complexity is added after the fourth encoding layer to ensure the fineness of the features and control the complexity of the calculation, and the first dynamic compression attention layer is a very light module, and the second dynamic compression attention layer is symmetrically arranged. Due to the high times of down-sampling, the size of the weld image becomes smaller and smaller in the deeper stage, and the semantic degree of the weld image is also higher, and the dynamic compression attention layer can focus on the key part of the weld image and extract the more critical information in the weld image for the current task, thereby assisting the subsequent stage to obtain better results.

[0051] Optionally, the second normalization layer is a layer normalization (LN) layer, the activation function of the second activation layer is a GELU activation function, the up-sampling layer is a bilinear interpolation function, and the activation function of the third activation layer is a Sigmoid activation function.

[0052] In each dynamic compression attention layer, the calculation complexity of the model is controlled through the adaptive max-pooling layer, then the attention matrix is calculated through the two multi-layer perceptron modules and the Sigmoid activation function, and finally the multiplication calculation of the attention matrix and the input is performed to obtain the result.

[0053] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the apparatus embodiment, refer to the method embodiment of the present application.

[0054] Reference is made to Figure 5In still another embodiment of the present application, a laser stripe extraction system for a weld seam image is provided, which can be used to implement the laser stripe extraction method for a weld seam image described above. Specifically, the laser stripe extraction system for a weld seam image comprises a model calling module and an image processing module.

[0055] The model calling module is configured to obtain a weld seam image and input the weld seam image into a preset laser stripe extraction model to perform laser stripe extraction, thereby obtaining a laser stripe mask image of the weld seam image. The image processing module is configured to perform an AND operation on the weld seam image and the laser stripe mask image, thereby obtaining a laser stripe of the weld seam image.

[0056] The laser stripe extraction model is obtained by training a lightweight feature extraction model. The lightweight feature extraction model comprises an encoder, a sparse self-attention layer, a decoder and a segmentation head. The encoder comprises first, second, third and fourth encoding layers connected in sequence. The sparse self-attention layer comprises a first convolution unit and a multi-head attention mechanism layer connected in sequence. The decoder comprises first, second, third and fourth decoding layers connected in sequence. The output of the first encoding layer is connected to the input of the fourth decoding layer. The output of the second encoding layer is connected to the input of the third decoding layer. The output of the third encoding layer is connected to the input of the second decoding layer. The output of the fourth encoding layer is connected to the input of the first decoding layer. The output of the fourth encoding layer is connected to the input of the first convolution unit. The output of the multi-head attention mechanism layer is connected to the input of the first decoding layer. The output of the fourth decoding layer is connected to the input of the segmentation head. The first, second, third and fourth encoding layers and the first decoding layer each comprise a second convolution unit and a dynamic compression multi-layer perception connected in sequence. The dynamic compression multi-layer perception comprises a deep convolution layer, a first normalization layer, a first multi-layer perception, a second multi-layer perception and a first activation layer connected in sequence. The first convolution unit, the second convolution unit, the second decoding layer, the third decoding layer, the fourth decoding layer, the deep convolution layer and the segmentation head are each constructed by a convolution block.

[0057] In a possible implementation, the following loss function is used when training the lightweight feature extraction model

[0058]

[0059] wherein, is a pixel value matrix of the laser stripe mask image output by the lightweight feature extraction model after inputting the current weld seam image, y is a pixel value matrix of the artificially labeled laser stripe mask image of the current weld seam image; y i is the value of the i-th element in y; n is the total number of element values in x and y; |Q| is the number of non-zero element values in matrix Q, or y; y i = 0 or 1; wherein, y i = 1 indicates that the i-th pixel point of the laser stripe mask image output by the lightweight feature extraction model is a laser stripe pixel point, y i = 0 indicates that the i-th pixel point of the laser stripe mask image output by the lightweight feature extraction model is a background pixel point. is the i-th element value in x, or 1; wherein, indicates that the i-th pixel point of the manually annotated laser stripe mask image is a laser stripe pixel point, indicates that the i-th pixel point of the manually annotated laser stripe mask image is a background pixel point.

[0060] In a possible implementation, when training the lightweight feature extraction model, the model parameters of the lightweight feature extraction model are updated by using back propagation.

[0061] In a possible implementation, the first convolution unit, the second convolution unit of the second encoding layer, the second convolution unit of the third encoding layer, the second convolution unit of the fourth encoding layer, and the second convolution unit of the first decoding layer are all 2x2 convolution blocks; the second convolution unit of the first encoding layer is a 4x4 convolution block; the second decoding layer, the third decoding layer, and the fourth decoding layer are all 3x3 convolution blocks, and the segmentation head is a 1x1 convolution block.

[0062] In a possible implementation, the depth convolution layer is a 3x3 convolution block, the first normalization layer is a BN layer, the activation function of the first normalization layer is a SiLu activation function, and the activation function of the first activation layer is a GELU activation function.

[0063] In a possible implementation, the encoder further comprises a first dynamic compression attention layer, and the decoder further comprises a second dynamic compression attention layer; the fourth encoding layer is connected with the first convolution unit and the first decoding layer through the first dynamic compression attention layer; the first decoding layer is connected with the second decoding layer through the second dynamic compression attention layer; the first dynamic compression attention layer and the second dynamic compression attention layer each comprise an adaptive max-pooling layer, a third multi-layer perceptron, a second normalization layer, a fourth multi-layer perceptron, a second activation layer, an up-sampling layer, a third activation layer and a multiplication operation layer connected in sequence; inputs of the adaptive max-pooling layer and the multiplication operation layer of the first dynamic compression attention layer are collectively taken as inputs of the first dynamic compression attention layer; inputs of the adaptive max-pooling layer and the multiplication operation layer of the second dynamic compression attention layer are collectively taken as inputs of the second dynamic compression attention layer; the multiplication operation layer of the first dynamic compression attention layer is configured to multiply the inputs of the first dynamic compression attention layer and an output of the third activation layer of the first dynamic compression attention layer; and the multiplication operation layer of the second dynamic compression attention layer is configured to multiply the inputs of the second dynamic compression attention layer and an output of the third activation layer of the second dynamic compression attention layer.

[0064] In a possible implementation, the second normalization layer is an LN layer, the activation function of the second activation layer is a GELU activation function, the up-sampling layer is a bilinear interpolation function, and the activation function of the third activation layer is a Sigmoid activation function.

[0065] The foregoing embodiment of the laser stripe extraction method of the weld image relates to all relevant contents of each step, which can be referred to the function description of the function module corresponding to the laser stripe extraction system of the weld image in the embodiment of the present application, and will not be described here.

[0066] The division of the modules in the embodiment of the present application is illustrative, and is merely a logical function division. In actual implementation, another division manner can be used. In addition, each function module in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module.

[0067] In another embodiment of the present application, a computer device is provided, which comprises a processor and a memory, the memory is configured to store a computer program, the computer program comprises program instructions, and the processor is configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are particularly suitable for loading and executing one or more instructions in the computer storage medium to implement a corresponding method flow or a corresponding function; the processor in the embodiments of the present application can be used for the operation of the laser stripe extraction method of the weld image.

[0068] In another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in the computer device, and is configured to store programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the computer device, and of course can also include the expansion storage medium supported by the computer device. The computer readable storage medium provides a storage space, and the storage space stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. One or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the laser stripe extraction method of the weld image in the above embodiments.

[0069] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0070] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0071] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0073] Finally, it should be noted that the above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit it. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced, and any modification or replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A method of laser stripe extraction of a weld seam image, characterized in that The method comprises the following steps: obtaining a weld seam image, and inputting the weld seam image into a preset laser stripe extraction model to extract a laser stripe, thereby obtaining a laser stripe mask image of the weld seam image; performing an AND operation on the weld seam image and the laser stripe mask image to obtain a laser stripe of the weld seam image; wherein the laser stripe extraction model is obtained by training a lightweight feature extraction model; the lightweight feature extraction model comprises an encoder, a sparse self-attention layer, a decoder and a segmentation head; the encoder comprises a first encoding layer, a second encoding layer, a third encoding layer and a fourth encoding layer connected in sequence, the sparse self-attention layer comprises a first convolution unit and a multi-head attention mechanism layer connected in sequence, and the decoder comprises a first decoding layer, a second decoding layer, a third decoding layer and a fourth decoding layer connected in sequence; the output of the first encoding layer is connected with the input of the fourth decoding layer; the output of the second encoding layer is connected with the input of the third decoding layer; the output of the third encoding layer is connected with the input of the second decoding layer; the output of the fourth encoding layer is connected with the input of the first decoding layer; the output of the fourth encoding layer is connected with the input of the first convolution unit, the output of the multi-head attention mechanism layer is connected with the input of the first decoding layer; and the output of the fourth decoding layer is connected with the input of the segmentation head; the first encoding layer, the second encoding layer, the third encoding layer, the fourth encoding layer and the first decoding layer each comprise a second convolution unit and a dynamic compression multi-layer perception connected in sequence; the dynamic compression multi-layer perception comprises a depth convolution layer, a first normalization layer, a first multi-layer perception, a second multi-layer perception and a first activation layer connected in sequence; the first convolution unit, the second convolution unit, the second decoding layer, the third decoding layer, the fourth decoding layer, the depth convolution layer and the segmentation head are each constructed by a convolution block.

2. The laser stripe extraction method of a weld seam image according to claim 1, characterized in that, When training the lightweight feature extraction model, the following loss function is adopted wherein, is a pixel value matrix of the laser stripe mask image output by the lightweight feature extraction model after inputting the current weld image, y is a pixel value matrix of the manually labeled laser stripe mask image of the current weld image; y i is the value of the i-th element in y; n is the total number of element values in y and |Q| is the number of non-zero element values in the matrix Q, or y; y i = 0 or 1; where y i =1 indicates that the i-th pixel in the laser stripe mask image output by the lightweight feature extraction model is a laser stripe pixel, y i =0 indicates that the i-th pixel in the laser stripe mask image output by the lightweight feature extraction model is a background pixel; for The value of the i-th element in the array. Or 1; where, This indicates that the i-th pixel in the manually annotated laser stripe mask image is the laser stripe pixel. This indicates that the i-th pixel of the manually annotated laser stripe mask image is a background pixel.

3. The laser stripe extraction method of a weld seam image according to claim 2, characterized in that, When training the lightweight feature extraction model, the model parameters of the lightweight feature extraction model are updated by using back propagation.

4. The method of claim 1, wherein, the first convolution unit, the second convolution unit of the second encoding layer, the second convolution unit of the third encoding layer, the second convolution unit of the fourth encoding layer and the second convolution unit of the first decoding layer are each a 2x2 convolution block; the second convolution unit of the first encoding layer is a 4x4 convolution block; the second decoding layer, the third decoding layer and the fourth decoding layer are each a 3x3 convolution block, and the segmentation head is a 1x1 convolution block.

5. The method of claim 1, wherein, the depth convolution layer is a 3x3 convolution block, the first normalization layer is a BN layer, the activation function of the first normalization layer is a SiLu activation function, and the activation function of the first activation layer is a GELU activation function.

6. The method of laser stripe extraction of a weld seam image according to claim 1, characterized in that, the encoder further comprises a first dynamic compression attention layer, and the decoder further comprises a second dynamic compression attention layer; the fourth encoding layer is connected with the first convolution unit and the first decoding layer through the first dynamic compression attention layer; and the first decoding layer is connected with the second decoding layer through the second dynamic compression attention layer. The first dynamic compression attention layer and the second dynamic compression attention layer each comprise an adaptive max-pooling layer, a third multi-layer perceptron, a second normalization layer, a fourth multi-layer perceptron, a second activation layer, an up-sampling layer, a third activation layer, and a multiplication operation layer connected in sequence; the input of the adaptive pooling layer and the multiplication operation layer of the first dynamic compression attention layer are collectively taken as the input of the first dynamic compression attention layer; The input of the adaptive pooling layer and the multiplication operation layer of the second dynamic compression attention layer are collectively taken as the input of the second dynamic compression attention layer; The multiplication operation layer of the first dynamic compression attention layer is configured to multiply the input of the first dynamic compression attention layer and the output of the third activation layer of the first dynamic compression attention layer; The multiplication operation layer of the second dynamic compression attention layer is configured to multiply the input of the second dynamic compression attention layer and the output of the third activation layer of the second dynamic compression attention layer.

7. The laser stripe extraction method of a weld seam image according to claim 6, characterized in that, The second normalization layer is an LN layer, the activation function of the second activation layer is a GELU activation function, the up-sampling layer is a bilinear interpolation function, and the activation function of the third activation layer is a Sigmoid activation function.

8. A laser stripe extraction system of a weld seam image, characterized in that Comprise: A model calling module configured to obtain a weld image, and input the weld image into a preset laser stripe extraction model to perform laser stripe extraction, and obtain a laser stripe mask image of the weld image; An image processing module configured to perform an AND operation on the weld image and the laser stripe mask image to obtain a laser stripe of the weld image; The laser stripe extraction model is obtained by training a lightweight feature extraction model; The lightweight feature extraction model comprises an encoder, a sparse self-attention layer, a decoder, and a segmentation head; the encoder comprises first, second, third, and fourth encoding layers connected in sequence, the sparse self-attention layer comprises a first convolution unit and a multi-head attention mechanism layer connected in sequence, and the decoder comprises first, second, third, and fourth decoding layers connected in sequence; the output of the first encoding layer is connected to the input of the fourth decoding layer; The output of the second encoding layer is connected to the input of the third decoding layer; the output of the third encoding layer is connected to the input of the second decoding layer; the output of the fourth encoding layer is connected to the input of the first decoding layer; the output of the fourth encoding layer is connected to the input of the first convolution unit; the output of the multi-head attention mechanism layer is connected to the input of the first decoding layer; and the output of the fourth decoding layer is connected to the input of the segmentation head; The first, second, third, and fourth encoding layers and the first decoding layer each comprise a second convolution unit and a dynamic compression multi-layer perceptron connected in sequence; the dynamic compression multi-layer perceptron comprises a deep convolution layer, a first normalization layer, a first multi-layer perceptron, a second multi-layer perceptron, and a first activation layer connected in sequence; and the first convolution unit, the second convolution unit, the second, third, and fourth decoding layers, the deep convolution layer, and the segmentation head are each constructed by a convolution block.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the laser stripe extraction of the weld image according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program, when executed by the processor, implements the steps of the laser stripe extraction of the weld seam image as claimed in any one of claims 1 to 7.

Images