Laser cutting state recognition method based on machine vision

Abstract

The invention relates to a method for recognizing a laser cutting state based on machine vision. The method specifically comprises the steps that a control system obtains a molten pool image in the cutting period of a laser cutting head; the control system identifies the molten pool image by adopting a pre-trained molten pool state classification model to obtain a first identification result; the control system obtains brightness information of the molten pool image; and the control system obtains cutting state information of the molten pool image according to the first recognition result, the brightness information and a preset brightness threshold value in plate attributes. According to the method, the molten pool image can be monitored in real time, the non-thorough cutting condition in the laser cutting process is automatically recognized, manual processing is reduced, and the recognition efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of laser cutting, and particularly to a method for identifying the state of laser cutting based on machine vision. Background Art

[0002] In the field of laser cutting, existing cutting equipment usually operates according to a predetermined path and parameters. When encountering abnormal situations such as incomplete cutting, traditional systems lack an automatic response mechanism, and often require manual intervention by operators to stop cutting, evaluate problems, and restart cutting. This manual intervention method not only increases the workload of operators, but also may lead to low production efficiency and material waste.

[0003] In actual laser cutting state recognition, most are based on the information of the laser cutting head or the information of the sheet to determine whether the cutting is incomplete. Therefore, the accuracy of identifying incomplete cutting is not high, resulting in large errors and prolonged processing time.

[0004] Therefore, there is an urgent need for a method for identifying the state of laser cutting based on machine vision. Summary of the Invention

[0005] (I) Technical Problems to be Solved

[0006] In view of the above-mentioned disadvantages and deficiencies of the prior art, the present invention provides a method for identifying the state of laser cutting based on machine vision, which solves the problem of low accuracy in identifying the state of incomplete cutting in the prior art.

[0007] (II) Technical Solutions

[0008] To achieve the above object, the main technical solutions adopted by the present invention include:

[0009] In a first aspect, an embodiment of the present invention provides a method for identifying the state of laser cutting based on machine vision, including:

[0010] S100. The control system acquires the molten pool image during the cutting of the laser cutting head;

[0011] S200. The control system uses a pre-trained molten pool state classification model to identify the molten pool image and obtains a first recognition result;

[0012] S300. The control system acquires the brightness information of the molten pool image;

[0013] S400. The control system obtains the cutting state information of the molten pool image according to the first recognition result, the brightness information, and the brightness threshold in the pre-given sheet properties.

[0014] Optionally, before S100, the control system obtains the molten pool image during the cutting of the laser cutting head, the method further includes:

[0015] S101. Before the laser cutting head cuts, the control system initializes the camera parameters based on the camera parameter adjustment strategy and obtains the nozzle image information of the laser cutting head based on the initialized camera;

[0016] S102. The control system obtains the nozzle center position and the nozzle area information according to the nozzle image information, and obtains the designated molten pool area of the molten pool image to be captured during the cutting of the laser cutting head according to the nozzle center position and the nozzle area information;

[0017] The molten pool image during the cutting of the laser cutting head is an image including the designated molten pool area;

[0018] S103. When the laser cutting head enters the cutting state, the control system adjusts the camera parameters based on the camera parameter adjustment strategy and captures the molten pool image during the cutting of the laser cutting head based on the adjusted camera parameters.

[0019] Optionally, before the laser cutting head cuts, the control system initializes the camera parameters based on the camera parameter adjustment strategy, including:

[0020] The gain in the initialized camera parameters is 22 - 26 dB, and the exposure time is 550000 - 650000 μs;

[0021] When the laser cutting head enters the cutting state, the control system adjusts the camera parameters based on the camera parameter adjustment strategy, including:

[0022] Adjust the gain in the camera parameters to 0 - 3 dB, and the exposure time to 4500 - 5500 μs.

[0023] Optionally, before S200, the control system uses the pre-trained molten pool state classification model to identify the molten pool image and obtains the first identification result, the method further includes:

[0024] S010. Based on the pre-acquired molten pool images of different sheet metal properties and cutting state information, train the molten pool state classification model;

[0025] The molten pool state classification model includes: a convolutional layer, a batch normalization layer, an activation function layer, a pooling layer, and a fully connected layer connected in sequence;

[0026] The convolutional layer extracts local features from the molten pool image based on the following formula (1);

[0027] Y = σ(W * X + b) (1)

[0028] Among them, X is the input molten pool image, W is the weight matrix, b is the bias vector, * is the convolution operator, representing the process of calculating the dot product; σ is the activation function, and Y is the output of the convolutional layer;

[0029] The batch normalization layer accelerates the training process using the following formula (2);

[0030]

[0031] Among them, Y is the local feature output by the convolutional layer, i.e., the molten pool feature map, and μ B is the mean value of all molten pool images within the current batch, is the variance of all molten pool images within the current batch, ε is a positive number to ensure numerical stability; γ and β are learning parameters, and y is the output of the batch normalization layer;

[0032] The activation function layer learns the mapping relationship according to f(y) = max(0, y), where y is the output of the batch normalization layer and f(y) is the output of the activation function layer.

[0033] Optionally, the max pooling layer in the pooling layer is processed using the following formula (3), and the global average pooling layer is processed using formula (4);

[0034]

[0035] Among them, H in and W in are respectively the height of 112 pixels and the width of 112 pixels of the input molten pool feature map; p is the padding size, k is the pooling window size, the stride s is set to 2, and H out and W out are respectively the height of 56 pixels and the width of 56 pixels of the output feature map

[0036] x i is each element in the feature map, N is the total number of elements in the feature map, and y' is the average value of all elements of the molten pool feature map, serving as the single output value of this feature map;

[0037] The fully connected layer is used to classify the molten pool image according to the following formula (5);

[0038] y1 = Wx1 + b (5)

[0039] Among them, x1 is the result after flattening all elements output by the previous layer; W is the weight matrix; b is the bias vector, and y1 is the output vector after linear transformation;

[0040] The initial learning rate learning_rate of the molten pool state classification model is 0.001, and the batch size batch_size is 32;

[0041] During the training process of the molten pool state classification model, an early stopping mechanism to prevent overfitting is applied to monitor the performance of the validation set, and the optimal model parameters are periodically saved to the best.pth file. The mean squared error (MSE) is used as the evaluation metric.

[0042] where n is the total number of molten pool images in the training dataset, y i is the true value of the i-th molten pool image, and is the predicted value of the i-th molten pool image.

[0043] Optionally, in S400, the control system obtains the cutting state information of the molten pool image according to the first recognition result, the brightness information, and the brightness threshold in the pre-given sheet metal properties, including:

[0044] The first recognition result includes: normal cutting, incomplete cutting, or abnormal cutting;

[0045] If the first recognition result is normal cutting and the brightness information is greater than the brightness threshold corresponding to the currently processed sheet metal, the final cutting state information of the molten pool image is incomplete cutting;

[0046] If the first recognition result is incomplete cutting, there is no need to judge the brightness information, and the final cutting state information of the molten pool image is directly determined as incomplete cutting;

[0047] If the first recognition result is normal cutting and the brightness information is less than or equal to the brightness threshold corresponding to the currently processed sheet metal, the final cutting state information of the molten pool image is normal cutting;

[0048] If the first recognition result is abnormal cutting, the cutting state of the molten pool image is directly returned as abnormal cutting, and there is no need to perform backtracking operations such as the following fallback instructions.

[0049] Optionally, the method further includes:

[0050] S500, when the cutting state information of the molten pool image is incomplete cutting, record the current laser cutting position information, and based on the backtracking strategy, determine that the laser cutting head can return to the position where incomplete cutting first occurred when it needs to fallback;

[0051] Or,

[0052] S500a. During the multi-layer cutting process, when the cutting status information of the molten pool image of the current layer indicates incomplete cutting, record the position information of the current laser cutting. Based on the backtracking strategy, determine that the laser cutting head can return to the position where incomplete cutting first occurred when a retraction is required. When the control system receives an instruction to switch the current layer, check whether there is an incomplete cutting status in the current layer. If there is, the control system issues a retraction instruction so that the laser cutting head re-cuts based on the position where the incomplete cutting occurred. If there is no incomplete cutting status in the current layer, execute the instruction to switch the current layer.

[0053] Optionally, determining that the laser cutting head can return to the position where incomplete cutting first occurred based on the backtracking strategy includes:

[0054] During the cutting process, if the status of incomplete cutting is first detected, the control system records the position information of this incomplete cutting, i.e., the mechanical coordinates (x 0 , y 0 );

[0055] The control system will continuously monitor. Within a preset distance S, if the proportion of images showing incomplete cutting among N1 continuously detected images exceeds half, a first retraction instruction is generated;

[0056] And, the control system records the position information of the laser cutting head so that the laser cutting head returns to the position where incomplete cutting first occurred based on the first retraction instruction for re-cutting;

[0057]

[0058] Where, V represents the current cutting speed, and F represents the frame rate of the camera.

[0059] In a second aspect, an embodiment of the present invention further provides an identification device for the laser cutting state based on machine vision, including:

[0060] An image acquisition module, configured to acquire the molten pool image during the cutting of the laser cutting head;

[0061] An intelligent analysis module, configured to use a pre-trained molten pool state classification model to identify the molten pool image to obtain a first identification result; and obtain the brightness information of the molten pool image; and obtain the cutting state information of the molten pool image according to the first identification result, the brightness information, and the brightness threshold in the pre-given sheet material attributes.

[0062] In a third aspect, an embodiment of the present invention further provides a control system for laser cutting, including: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0063] The memory is used to store computer programs;

[0064] The processor is used to implement the method described in any item of the first aspect when executing the program stored in the memory.

[0065] (III) Beneficial effects

[0066] The method for identifying the laser cutting state based on machine vision in the present invention first makes a judgment by obtaining the molten pool image. Specifically, it makes a judgment on the cutting state based on the intelligent analysis result of the molten pool image, that is, the first recognition result and the brightness information of the molten pool image. It ensures the accuracy of the cutting state judgment through a secondary judgment method, and ensures the safety of the cutting process. The above method can more accurately distinguish between normal cutting and non-penetrating cutting, thereby effectively reducing the possibility of misjudgment.

[0067] In the embodiment of the present invention, in view of the particularity of the molten pool image, when collecting the molten pool image, the parameters of the camera can be adjusted in real time according to the cutting preparation result and the cutting start stage to better ensure that the collected molten pool image is an effective and analyzable molten pool image.

[0068] Furthermore, when the cutting state is identified as non-penetrating cutting, the automatic backtracking mechanism can be triggered according to the position information of the non-penetrating cutting to realize automatic backtracking and re-cutting, thereby reducing the need for manual intervention. Description of the drawings

[0069] Figure 1 It is a schematic diagram of the nozzle calibration step in the method of the embodiment of the present invention;

[0070] Figure 2 It is a schematic diagram of the nozzle calibration result shown in the embodiment of the present invention;

[0071] Figure 3 It is a schematic diagram of the molten pool image in the normal cutting state shown in the embodiment of the present invention;

[0072] Figure 4

[0073] Figure 5 It is a schematic diagram of the molten pool image in the abnormal cutting state (such as perforation) shown in the embodiment of the present invention;

[0074] Figure 6 It is a schematic diagram of the molten pool image in the abnormal cutting state (such as the cutting path turning) shown in the embodiment of the present invention;

[0075] Figure 7 It is a schematic diagram of the molten pool state classification model used in the embodiment of the present invention;

[0076] Figure 8 Schematic diagram of the iterative loss function during the training process of the molten pool state classification model in the embodiments of the present invention;

[0077] Figure 9 Schematic diagram of the method for identifying the laser cutting state based on machine vision provided by the first embodiment of the present invention;

[0078] Figure 10 Schematic diagram of the method for identifying the laser cutting state based on machine vision provided by the second embodiment of the present invention;

[0079] Figure 11 Schematic diagram of the method for identifying the laser cutting state based on machine vision provided by the third embodiment of the present invention. Detailed implementation manners

[0080] To better understand the above technical solutions, the exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more clear and thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0081] The embodiments of the present invention propose a method for identifying the laser cutting state based on machine vision, which captures the image of the molten pool during the cutting process in real time through a camera and analyzes the cutting state in combination with advanced image processing algorithms. This method realizes the intelligent identification of the phenomenon of incomplete cutting by means of the molten pool image; further, when the state of incomplete cutting is identified, the automatic backtracking function can be triggered, thus reducing the need for manual intervention.

[0082] The intelligent backtracking mechanism in this embodiment is an automated solution developed for the possible problem of incomplete cutting during the laser cutting process. This function is based on artificial intelligence algorithms and can monitor the molten pool state in real time to evaluate the cutting effect; once the situation of incomplete cutting is detected, the control system can accurately locate to the specified mechanical coordinates, and the laser head can automatically return to the position of incomplete cutting based on the specified mechanical coordinates for re-cutting.

[0083] The molten pool in this embodiment refers to the liquid metal area formed by locally heating the material to the melting point during metal cutting or welding.

[0084] Embodiment 1

[0085] This embodiment provides a method for identifying the laser cutting state based on machine vision. The execution subject of the method in this embodiment is the control system of the laser cutting head, which is electrically connected to a camera for collecting the molten pool area and controls the laser cutting head to perform intelligent cutting based on the following method, asFigure 11 As shown in the figure, the method of this embodiment includes the following steps:

[0086] S100. The control system acquires the molten pool image during the cutting of the laser cutting head.

[0087] For example, this step S100 may include the following sub-steps:

[0088] Sub-step S101. Before the laser cutting head cuts, the control system initializes the camera parameters based on the camera parameter adjustment strategy and acquires the nozzle image information of the laser cutting head based on the initialized camera.

[0089] For example, after initialization, the gain in the camera parameters is 22 - 26 dB, and the exposure time is 550000 - 650000 μs.

[0090] Sub-step S102. The control system obtains the nozzle center position and nozzle area information (i.e., nozzle radius information) according to the nozzle image information, and obtains the specified molten pool area of the molten pool image to be photographed during the cutting of the laser cutting head or the starting position for determining the shooting according to the nozzle center position and nozzle area information.

[0091] The molten pool image during the cutting of the laser cutting head is an image including the specified molten pool area.

[0092] During the laser cutting process of this embodiment, the plate to be processed does not move, and the laser cutting head moves to achieve cutting. In this embodiment, the camera for photographing the molten pool area is fixed, that is, immovable. The molten pool images obtained by the camera each time may not be fixed or may be different. The state of the molten pool image is judged through the image processing of this embodiment.

[0093] In this embodiment, first, nozzle calibration is performed (obtaining the nozzle center position and nozzle radius information as above). After the calibration is completed, the cutting head is moved above the plate to be processed to start cutting.

[0094] It can be understood that without replacing the nozzle, only one nozzle calibration is performed. What is obtained through the calibration is only the coordinate origin and radius ( Figure 2 the data in the upper left corner), and the shooting area is determined through these data. The obtained shooting area is as shown in Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 shown.

[0095] Sub-step S103. When the laser cutting head enters the cutting state, the control system adjusts the camera parameters based on the camera parameter adjustment strategy and photographs the molten pool image during the cutting of the laser cutting head based on the adjusted camera parameters.

[0096] Adjust the gain in the camera parameters to 0 - 3 dB and the exposure time to 4500 - 5500 μs.

[0097] S200. The control system uses a pre - trained molten pool state classification model to identify the molten pool image and obtains the first recognition result.

[0098] S300. The control system obtains the brightness information of the molten pool image.

[0099] For example, in this embodiment, the molten pool image can be converted from the original color space (such as RGB) to the HSV color space. In the HSV color space, the V (Value / Brightness) component can directly reflect the brightness information of the image.

[0100] During the actual cutting process, the brightness value of each molten pool image is quantified by extracting its V component for comparison and analysis with the above - mentioned reference brightness value (such as). When the brightness of the image is greater than the reference brightness value, it is determined that the molten pool state is incomplete cutting at this time.

[0101] In this embodiment, in order to determine a reference brightness value suitable for evaluating the cutting quality, before executing the method of this embodiment, the average brightness values of images of different thicknesses and plate categories in a specific data set (such as the data set of carbon steel plate cutting) are calculated first, and then the reference brightness values of different thicknesses of various plates are obtained.

[0102] S400. The control system obtains the cutting state information of the molten pool image according to the first recognition result, the brightness information, and the brightness threshold in the pre - given plate properties.

[0103] For example, the first recognition result of this embodiment may include: normal cutting / normal cutting state, incomplete cutting / incomplete cutting state, or abnormal cutting / abnormal cutting state;

[0104] If the first recognition result is normal cutting and the brightness information is greater than the brightness threshold corresponding to the currently processed plate, the final cutting state information of the molten pool image is incomplete cutting;

[0105] If the first recognition result is incomplete cutting / incomplete cutting state, there is no need to judge the brightness information, and it is directly determined that the final cutting state information of the molten pool image is incomplete cutting;

[0106] If the first recognition result is normal cutting / normal cutting state and the brightness information is less than or equal to the brightness threshold corresponding to the currently processed plate, the final cutting state information of the molten pool image is normal cutting;

[0107] If the first recognition result is abnormal cutting / abnormal cutting state, there is no need to perform brightness judgment, directly return the cutting state of the molten pool image as abnormal cutting, and there is no need to execute the backtracking mechanism at the same time.

[0108] The abnormal cutting states in this embodiment include but are not limited to perforation, turning of the cutting path, and the like.

[0109] The method of this embodiment first makes a judgment by obtaining the molten pool image. Specifically, it judges the cutting state based on the intelligent analysis result of the molten pool image, that is, the first recognition result, and the brightness information of the molten pool image. Its secondary judgment method ensures the accuracy of the cutting state judgment and the safety of the cutting process.

[0110] That is, it can more accurately distinguish between normal cutting and incomplete cutting, thereby effectively reducing the possibility of misjudgment.

[0111] Embodiment 2

[0112] This embodiment provides a method for recognizing the laser cutting state based on machine vision. The difference between this method and Embodiment 1 above is that before executing Embodiment 1, the above method also needs to execute the following step S010 to train the molten pool state classification model in advance, and then in S200 of the above Embodiment 1, use the trained molten pool state classification model to recognize the molten pool image.

[0113] S010. Train the molten pool state classification model based on the molten pool images of different sheet material properties and cutting state information obtained in advance;

[0114] The molten pool state classification model can be a Deep Convolutional Network (DCNet), which includes: a convolutional layer, a batch normalization layer, an activation function layer, a pooling layer, and a fully connected layer connected in sequence; as Figure 7 shown;

[0115] The convolutional layer extracts local features from the molten pool image based on the following formula (1);

[0116] Y = σ(W * X + b) (1)

[0117] Among them, X is the input molten pool image. For an RGB image, it is a three-dimensional tensor with a size of 3×112×112. Each channel represents red, green, and blue respectively, and can reflect the temperature distribution or brightness change in different molten pool images; W is the weight matrix with a size of C out ×3×k, where k is the convolution kernel size (3×3). A smaller convolution kernel can capture fine details in the molten pool, and b is the bias vector with a size of C out, that is, each output channel has a corresponding bias value to adjust the activation threshold; * is the convolution operator, representing the process of calculating the dot product; σ is the activation function; the ReLU function is used in this model, which is applied after the convolution result to introduce non-linearity and enhance the expressive ability of the molten pool state classification model, and Y is the output of the convolutional layer.

[0118] The convolutional layer is one of the core components of DCNet. It is responsible for extracting local features from the input molten pool images. By applying a series of learnable filters, the convolution operation can capture the spatial hierarchical structure of the image, such as edges, textures, and shapes. Each filter detects a specific pattern in the image, and multiple filters can capture different types of features. Its operation can be represented by the following formula.

[0119] The batch normalization layer is used to accelerate the training process and improve gradient propagation. It normalizes the data in small batches, making the input of each layer have a similar distribution, thus reducing the problem of "internal covariate shift". This approach helps to stabilize the training, allows the use of higher learning rates, and has a certain degree of regularization effect.

[0120] The batch normalization layer can accelerate the training process using the following formula (2);

[0121]

[0122] Among them, Y is the local feature from the output of the convolutional layer, that is, the molten pool feature map, μ B is the mean of all molten pool images in the current batch, is the variance of all molten pool images in the current batch, ε is a very small positive number to ensure numerical stability, preventing division by zero and ensuring numerical stability; γ and β are two learnable parameters, used to scale and shift the normalized data respectively, so that the model can better adapt to different brightness and contrast conditions of the molten pool images; y is the output after normalization, rescaling, and shifting;

[0123] The activation function layer learns the mapping relationship according to f(y) = max(0, y), where y is the output of the batch normalization layer and f(y) is the output of the activation function layer, that is, the output value after ReLU activation. All negative values are set to zero, and positive values remain unchanged.

[0124] The Rectified Linear Unit (ReLU) is a widely used activation function that introduces non-linearity in the molten pool state classification, enabling the model to learn complex mapping relationships. It can allow active neurons to transmit information while suppressing inactive neurons, thereby improving computational efficiency and reducing the risk of overfitting.

[0125] In the pooling layer, the max - pooling layer is processed using the following formula (3), and the global average - pooling layer is processed using formula (4);

[0126]

[0127] where, H in and W in are the height of 112 pixels and the width of 112 pixels of the input molten - pool feature map respectively; p is the padding size, which is defaulted to 0 without padding; k is the pooling window size, such as 2×2, and the stride s is set to 2, controlling how the pooling window slides; H out and W out are the height of 56 pixels and the width of 56 pixels of the output feature map respectively.

[0128] x i is each element in the feature map, N is the total number of elements in the feature map, and y’ is the average value of all elements of the molten - pool feature map, serving as the single output value of this feature map.

[0129] The role of the max - pooling layer is to reduce the size of the molten - pool feature map. This is very important for reducing computational complexity and preventing overfitting. In addition, max - pooling also provides a form of translational invariance, because even if the molten - pool image has a slight movement in the image, as long as it is still within the pooling window, its feature representation will not change significantly.

[0130] The global average - pooling layer further compresses the feature map to a fixed size, usually 1×1. This operation is very useful for summarizing all the spatial information in the molten - pool image, because it summarizes all the spatial information on each feature map into a single value, representing the overall performance of this feature over the entire image.

[0131] The fully - connected layer is the last layer of DCNet. It makes the final decision on the classification of the molten - pool state based on the features extracted by the previous layers. At this stage, each neuron is connected to all neurons in the previous layer, so it can comprehensively consider all the previously learned features to determine the category to which the image belongs.

[0132] The fully - connected layer is used to classify the molten - pool image according to the following formula (5);

[0133] y1 = W * x1 + b (5)

[0134] where, x1 is the result of flattening all elements output by the previous layer; W is the weight matrix; b is the bias vector, and y1 is the output vector after linear transformation, which usually follows a softmax layer to be converted into a probability distribution;

[0135] The initial learning rate learning_rate of the molten pool state classification model can be 0.001, and the batch size batch_size can be 32; and it can be trained for 250 epochs, using the Adam optimizer combined with weight decay (1e -5 ). In addition, an early stopping mechanism is applied to monitor the performance of the validation set to prevent overfitting, and the optimal model parameters are regularly saved to the best.pth file. The mean-square error (MSE) is used as the evaluation metric, and the calculation formula is as follows,

[0136] where n is the total number of molten pool images in the training dataset, y i is the true value of the i-th molten pool image, is the predicted value of the i-th molten pool image.

[0137] The MSE loss curve generated during the training of the model is as Figure 8 shown. After training, the molten pool state classification model is exported to the ONNX format through the PyTorch library, which is convenient for subsequent direct invocation using the C++ language.

[0138] Deployment of the molten pool state classification model in this embodiment: Through the DNN module in the OpenCV library, the loading of the ONNX file, the forward prediction inference of the image, and the result parsing are implemented using C++, so that the forward inference process is very efficient, reaching 11 - 13 ms.

[0139] During the cutting process, in order to ensure the fast response of the control system and minimize the processing delay, the control system calls the DCNet model once every 50 ms to process and analyze the molten pool image, obtain the cutting state of the laser cutting head, and thus provide immediate feedback, as Figure 9 shown, Figure 9 The molten pool state result in

[0140] can be the cutting state result of the laser cutting head. It should be noted that the camera gain is adjusted to 0 dB and the exposure value is 5000 μs during cutting.

[0141] Example Three

[0142] This embodiment provides a method for identifying the laser cutting state based on machine vision. The difference between this method and the above-mentioned Embodiment 1 is that after obtaining the cutting state information in Embodiment 1, the above method further needs to execute the following step S500 or S500a to effectively implement the intelligent backtracking mechanism of the laser cutting head, that is, it can accurately locate to the first position coordinates where the cutting is not thorough, and make the laser cutting head automatically return to the position where the cutting is not thorough for re-cutting, as Figure 10 shown.

[0143] S500. When the cutting state information of the molten pool image indicates that the cutting is not thorough, record the current laser cutting position information, and based on the backtracking strategy, determine that the laser cutting head can return to the first position where the cutting is not thorough when it needs to retreat;

[0144] Or,

[0145] S500a. During the multi-layer cutting process, when the cutting state information of the molten pool image of the current layer indicates that the cutting is not thorough, record the current laser cutting position information, and based on the backtracking strategy, determine that the laser cutting head can return to the first position where the cutting is not thorough when it needs to retreat. When the control system receives an instruction to switch the current layer, check whether there is a state where the cutting is not thorough in the current layer. If so, the control system issues a backtracking instruction to make the laser cutting head re-cut based on the position where the cutting is not thorough; if there is no state where the cutting is not thorough in the current layer, execute the instruction to switch the current layer.

[0146] Determining that the laser cutting head can return to the first position where the cutting is not thorough when it needs to retreat based on the backtracking strategy in the above S500 or S500a may include:

[0147] During the cutting process, if the state where the cutting is not thorough is detected for the first time, the control system records the position information of this non-through cutting, that is, the mechanical coordinates (x 0 , y 0 );

[0148] The control system will continuously monitor. Within a preset distance S (such as 0.1 m), if the proportion of images showing non-through cutting exceeds half among N1 continuously detected images, a first backtracking instruction is generated;

[0149] In addition, the control system records the position information of the laser cutting head, so that the laser cutting head returns to the position where the non-through cutting first occurred for re-cutting based on the first backtracking instruction;

[0150]

[0151] Among them, V represents the current cutting speed, and F represents the frame rate of the camera. If S is set to 0.1 m, the cutting speed V is 1 m / s, and the frame rate F is 80 fps, it can be calculated that N is 8. If the quantity exceeds 4 in 8 consecutive pictures, it is in a state of incomplete cutting.

[0152] The intelligent backtracking mechanism of this embodiment mainly includes two core elements. One is the specific situation of layer replacement and its corresponding backtracking countermeasures; the other is the backtracking scheme formulated for the possible problem of incomplete cutting during the cutting process, as Figure 10 shown.

[0153] Layer replacement detection: A molten pool state detection mechanism for layer replacement is introduced. When the control system needs to issue an instruction to replace the layer, it will automatically trigger a comprehensive inspection of the previous cutting state. This process is to evaluate whether there is a state of incomplete cutting in the previous layer of material. If the detection result shows that there is indeed an incomplete cutting situation before switching to the new layer, the control system will automatically generate a backward instruction, and the backward coordinates of the initial incomplete cutting state are recorded as (x0, y0).

[0154] The method of this embodiment captures the image of the molten pool during the cutting process in real time through the camera, and combines the secondary judgment to analyze the cutting state. This method can intelligently identify the phenomenon of incomplete cutting and trigger the automatic backtracking mechanism, thus reducing the need for manual intervention.

[0155] Embodiment 4

[0156] This embodiment provides a method that combines the above Embodiment 1 and Embodiment 3. The execution subject of this method is the control system / numerical control system of the laser cutting equipment, and specifically includes the following steps:

[0157] S1. The control system first executes the initialization step of the camera; subsequently, the control system starts nozzle calibration to obtain the nozzle center and radius;

[0158] In this embodiment, in order to ensure the normal invocation of the subsequent intelligent backtracking mechanism, the control system will automatically execute the camera initialization operation, adjust the camera gain and exposure value to ensure the clarity of the collected molten pool image; at the same time, perform the nozzle calibration operation to determine the nozzle center position and its radius to ensure that the image obtained by the camera can clearly present the internal structure of the molten pool and realize the effective monitoring of the molten pool state.

[0159] It can be understood that, as Figure 1 shown, the S1 step specifically includes:

[0160] S1.1 Camera initialization: When calibrating the nozzle, the camera parameters are initialized as follows: the gain is set to 24 dB, and the exposure value is set to 600000 μs. During the cutting operation, since the molten pool generates a relatively high brightness, in order to adapt to this change, the camera parameters need to be adjusted accordingly. At this time, the gain is adjusted to 0 dB, and the exposure time is shortened to 5000 μs, so as to ensure that the captured image is neither overexposed nor can clearly present the internal structure of the molten pool.

[0161] S1.2 Nozzle calibration: The control system controls the laser cutting head to move above the light source (a light source pre-placed for nozzle calibration, the purpose of which is to make the image of the nozzle structure clearer when calibrating the nozzle. In practice, it may not be set separately) to take a picture of the nozzle. At this time, the camera gain is set to 24 dB, and the exposure value is set to 600000 μs. Then, the taken picture is converted into a grayscale image, the contour is extracted, and fitted into a circle. Finally, the center of the nozzle and the radius of the nozzle are obtained and recorded. As Figure 2 shown, Figure 2 in the upper left corner (1360, 1118) is the center point of the nozzle, and the nozzle radius is 149 pixels. By obtaining the center and radius of the nozzle, the nozzle area can be extracted to accurately capture the changes in the morphology of the molten pool inside the nozzle.

[0162] S2. During the cutting process, the control system analyzes the cutting state based on the molten pool images captured by the camera, that is, intelligently analyzes the cutting state to which the molten pool image belongs, and determines the final cutting state according to the analysis results.

[0163] The control system not only uses the molten pool state classification model to evaluate the current cutting condition, but also further determines the specific final cutting state through the threshold set according to different plates. If the judgment result shows that the cutting is incomplete, a retraction signal can be generated to adjust the cutting process; if it is confirmed as normal cutting, the current operation continues.

[0164] It can be understood that the molten pool state classification model in this step can be a pre-trained model. For example, a training dataset is prepared in advance: the camera is used to collect images of the cutting process of various types of plates, and the nozzle calibration data is combined to determine the nozzle area in each image, which only contains the morphology of the molten pool. According to the state of the molten pool, the image categories are pre-classified into three categories: normal cutting state, incomplete cutting state, and abnormal cutting state (including but not limited to perforation and turning of the cutting path, etc.). As Figures 3 to 6 shown, these categories respectively show the morphological characteristics of the molten pool under different conditions. A total of 60000 images are collected in the entire training dataset, aiming to comprehensively cover various possible molten pool states, and then train the molten pool state classification model.

[0165] S3. When the control system receives the retraction signal, the laser cutting head moves to the uncut position according to the recorded coordinate information when the cutting is not thorough; after the laser cutting head reaches the specified position, the cutting operation is restarted; the camera will continue to monitor the new cutting process;

[0166] In this embodiment, once the control system detects the retraction signal of the intelligent backtracking mechanism, it will first lift the laser cutting head and then return it to the initial mechanical coordinate position where the cutting is not thorough to ensure the accuracy of subsequent cutting actions. During the whole process, the camera also needs to monitor the new cutting process in real time to ensure immediate backtracking when the cutting is not thorough again.

[0167] For example, after the control system receives the retraction signal, the laser cutting head immediately lifts, and the lifting position is the mechanical coordinate of the Z axis -10, and then the laser cutting head returns to the initial position (x 0 , y 0 ) where the cutting is not thorough. The backtracking action requires precise control of the movement of the laser cutting head to ensure that it can return to the specified position accurately. After reaching the specified position, the laser head restarts the cutting operation.

[0168] The control system of this embodiment continuously monitors the cutting process and immediately backtracks if it detects that the cutting is not thorough.

[0169] The method of this embodiment has a high degree of intelligence. For example, by using artificial intelligence algorithms, it can automatically identify the situation where the cutting is not thorough and make the optimal decision without manual intervention.

[0170] At the same time, it reduces material waste, and the real-time response mechanism reduces the material waste caused by cutting errors.

[0171] In the above step S2, in order to further ensure the accuracy of the molten pool state category judgment, a brightness threshold is introduced in the method of this embodiment for secondary confirmation. First, when the first recognition result determines normal cutting, the control system will further check the brightness of the current molten pool image in combination with the brightness threshold to which the current sheet belongs.

[0172] The brightness threshold is set based on the average value of the brightness of the cut-through and uncut-through molten pools and adjusted according to different sheet characteristics: for carbon steel, the brightness threshold is set to 80; for stainless steel, the brightness threshold is set to 120. If the brightness of the current molten pool image exceeds the corresponding threshold, even if the first recognition result is initially determined to be normal, the control system will re-determine it as cutting not thorough.

[0173] In this step, by combining the first recognition result of the model and the secondary judgment based on the brightness threshold, this method can more accurately distinguish between normal cutting and cutting not thorough, thereby effectively reducing the possibility of misjudgment.

[0174] The method of this embodiment can more accurately distinguish between normal cutting and incomplete cutting, thereby effectively reducing the possibility of misjudgment.

[0175] In view of the particularity of the molten pool image, when collecting the molten pool image, the parameters of the camera can be adjusted in real time according to the cutting preparation result and the cutting start stage to preferably ensure that the collected molten pool image is an effective and analyzable molten pool image.

[0176] And when the cutting state is identified as incomplete cutting, the automatic backtracking mechanism can be triggered according to the position information of the incomplete cutting to realize automatic backtracking and re-cutting, thereby reducing the need for manual intervention.

[0177] Embodiment 5

[0178] In addition, the embodiment of the present invention further provides an identification device for the laser cutting state based on machine vision, which includes:

[0179] An image acquisition module for acquiring the molten pool image during the cutting of the laser cutting head;

[0180] An intelligent analysis module for identifying the molten pool image by using a pre-trained molten pool state classification model to obtain a first identification result; and obtaining the brightness information of the molten pool image; and obtaining the cutting state information of the molten pool image according to the first identification result, the brightness information, and the brightness threshold in the pre-given sheet material attributes.

[0181] In practical applications, the image acquisition module is further configured to initialize the camera parameters based on the camera parameter adjustment strategy before the laser cutting head cuts, and obtain the nozzle image information of the laser cutting head based on the initialized camera; obtain the nozzle center position and the nozzle area information according to the nozzle image information, and obtain the designated molten pool area of the molten pool to be photographed during the cutting of the laser cutting head according to the nozzle center position and the nozzle area information; when the laser cutting head enters the cutting state, adjust the camera parameters based on the camera parameter adjustment strategy, and photograph the molten pool image during the cutting of the laser cutting head based on the adjusted camera parameters.

[0182] For example, before the laser cutting head cuts, the gain in the initialized camera parameters is 22-26 dB, and the exposure time is 550000-650000 μs; when the laser cutting head enters the cutting state, the gain in the camera parameters is adjusted to 0-3 dB, and the exposure time is 4500-5500 μs.

[0183] In addition, the intelligent analysis module is specifically configured to train the molten pool state classification model based on the molten pool images of different sheet material attributes and cutting state information obtained in advance before obtaining the first identification result; and then use the trained molten pool state classification model to identify the molten pool image to obtain the first identification result.

[0184] In this embodiment, the first recognition result may include: normal cutting, incomplete cutting, or abnormal cutting.

[0185] If the first recognition result is normal cutting and the brightness information is greater than the brightness threshold corresponding to the current processed sheet, the final cutting state information of the molten pool image is incomplete cutting.

[0186] If the first recognition result is incomplete cutting, there is no need to judge the brightness information, and directly determine that the final cutting state information of the molten pool image is incomplete cutting.

[0187] If the first recognition result is normal cutting and the brightness information is less than or equal to the brightness threshold corresponding to the current processed sheet, the final cutting state information of the molten pool image is normal cutting.

[0188] If the first recognition result is abnormal cutting, directly return that the cutting state of the molten pool image is abnormal cutting.

[0189] In the specific implementation process, the recognition device of this embodiment may further include: a backtracking module.

[0190] The backtracking module in this embodiment may be specifically configured to record the current laser cutting position information when the cutting state information of the molten pool image is incomplete cutting, and determine that the laser cutting head can return to the position where incomplete cutting first occurred when a retraction is required based on the backtracking strategy.

[0191] Or, during the multi-layer cutting process, when the cutting state information of the molten pool image of the current layer is incomplete cutting, record the current laser cutting position information, determine that the laser cutting head can return to the position where incomplete cutting first occurred when a retraction is required based on the backtracking strategy, and when receiving an instruction to switch the current layer, check whether there is an incomplete cutting state in the current layer. If there is, issue a retraction instruction so that the laser cutting head re-cuts based on the position where the incomplete cutting belongs; if there is no incomplete cutting state in the current layer, execute the instruction to switch the current layer.

[0192] For example, during the cutting process, if the state of incomplete cutting is first detected, record the position information of this incomplete cutting, that is, the mechanical coordinates (x 0 , y 0 ); and continuously monitor. Within a preset distance S, if the proportion of incomplete cutting shown in N1 consecutive images exceeds half, generate a first retraction instruction; and record the position information of the laser cutting head so that the laser cutting head returns to the position where incomplete cutting first occurred based on the first retraction instruction for re-cutting.

[0193] ; where V represents the current cutting speed and F represents the frame rate of the camera.

[0194] The device of this embodiment can ensure the safety of the cutting process, effectively reduce the possibility of misjudgment, accurately distinguish between normal cutting and incomplete cutting, and improve the intelligence and cutting efficiency of laser cutting.

[0195] The embodiment of the present invention also provides a control system for laser cutting, including: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0196] The memory is used to store computer programs;

[0197] When the processor is used to execute the program stored on the memory, it realizes the method described in any of the above embodiments.

[0198] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0199] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0200] It should be noted that in the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware including several different components and by means of a properly programmed computer. In the claims listing several devices, several of these devices can be embodied by the same hardware. The use of the words first, second, third, etc. is only for convenience of expression and does not indicate any order. These words can be understood as part of the component name.

[0201] In addition, it should be noted that in the description of this specification, the descriptions of terms such as "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0202] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications after learning the basic creative concepts. Therefore, the claims should be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0203] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and its equivalent technologies, the present invention should also include these modifications and variations.

Claims

1. A method for identifying laser cutting status based on machine vision, characterized in that: include: S100, the control system acquires a molten pool image during cutting by the laser cutting head; S200, the control system uses a pre-trained molten pool state classification model to identify the molten pool image and obtain a first identification result; S300, the control system obtains brightness information of the molten pool image; S400: The control system obtains the cutting state information of the molten pool image according to the first recognition result and the brightness information, and a brightness threshold in a predetermined plate attribute.

2. The identification method according to claim 1, characterized in that: S100, before the control system acquires the molten pool image during cutting by the laser cutting head, the method further includes: S101, before the laser cutting head cuts, the control system initializes the camera parameters based on the camera parameter adjustment strategy and obtains the nozzle image information of the laser cutting head based on the initialized camera; S102, the control system obtains the nozzle center position and nozzle area information according to the nozzle image information, and obtains the molten pool designated area of ​​the molten pool image to be captured when the laser cutting head is cutting according to the nozzle center position and nozzle area information; The molten pool image during cutting by the laser cutting head is an image including a designated area of ​​the molten pool; S103: When the laser cutting head enters a cutting state, the control system adjusts the camera parameters based on the camera parameter adjustment strategy, and captures the molten pool image during the cutting process of the laser cutting head based on the adjusted camera parameters.

3. The identification method according to claim 2, characterized in that: The control system initializes the camera parameters based on the camera parameter adjustment strategy before the laser cutting head cuts, including: After initialization, the gain of the camera parameters is 22-26 dB, and the exposure time is 550,000-650,000 μs; When the laser cutting head enters a cutting state, the control system adjusts the camera parameters based on the camera parameter adjustment strategy, including: Adjust the gain in the camera parameters to 0-3dB and the exposure time to 4500-5500μs.

4. The identification method according to claim 1, characterized in that: S200, the control system uses a pre-trained molten pool state classification model to identify the molten pool image. Before obtaining the first recognition result, the method further includes: S010, training the molten pool state classification model based on the molten pool images with different plate properties and cutting state information acquired in advance; The melt pool state classification model includes: a convolutional layer, a batch normalization layer, an activation function layer, a pooling layer, and a fully connected layer connected in sequence; The convolution layer extracts local features from the melt pool image based on the following formula (1); Y=σ(W*X+b) (1) Among them, X is the input melt pool image, W is the weight matrix, b is the bias vector, * is the convolution operator, which indicates the process of calculating the dot product; σ is the activation function, and Y is the output of the convolution layer; The batch normalization layer uses the following formula (2) to accelerate the training process; Among them, Y is the local feature output by the convolution layer, namely the melt pool feature map, μ B is the mean of all melt pool images in the current batch, is the variance of all melt pool images in the current batch, ε is a positive number to ensure numerical stability; γ and β are learning parameters, and y is the output of the batch normalization layer; The activation function layer learns the mapping relationship according to f(y)=max(0,y), where y is the output of the batch normalization layer and f(y) is the output of the activation function layer.

5. The identification method according to claim 4, characterized in that: The maximum pooling layer in the pooling layer is processed using the following formula (3), and the global average pooling layer is processed using formula (4); Among them, H in and W in The height and width of the input melt pool feature map are 112 pixels and 112 pixels respectively; p is the padding size, k is the pooling window size, the step size s is set to 2, and H out and W out The height and width of the output feature map are 56 pixels and 56 pixels respectively; x i is each element in the feature map, N is the total number of elements in the feature map, and y' is the average value of all elements in the melt pool feature map, which is the single output value of the feature map; The fully connected layer is used to classify the melt pool image according to the following formula (5); y1=W*x1+b(5) Among them, x1 is the result of flattening all elements output by the previous layer; W is the weight matrix; b is the bias vector, and y1 is the output vector after linear transformation; The initial learning rate learning_rate of the melt pool state classification model is 0.001, and the batch size batch_size is 32; In the training process of the melt pool state classification model, an early stopping mechanism to prevent overfitting is applied to monitor the performance of the validation set, and the optimal model parameters are periodically saved to the best.pth file, with the mean square error (MSE) as the evaluation indicator. Where n is the total number of melt pool images in the training dataset, and y i is the true value of the i-th melt pool image, is the predicted value of the i-th melt pool image.

6. The identification method according to claim 1, characterized in that: S400, the control system obtains the cutting state information of the molten pool image according to the first recognition result and the brightness information, and the brightness threshold in the pre-given plate attribute, including: The first recognition result includes: normal cutting, incomplete cutting or abnormal cutting; If the first recognition result is normal cutting, and the brightness information is greater than the brightness threshold corresponding to the current processed plate, the final cutting state information of the molten pool image is that the cutting is not through; If the first recognition result is that the cutting is not through, then the brightness information does not need to be judged, and the final cutting state information of the molten pool image is directly determined to be that the cutting is not through; If the first recognition result is normal cutting, and the brightness information is less than or equal to the brightness threshold corresponding to the currently processed plate, the final cutting state information of the molten pool image is normal cutting; If the first recognition result is abnormal cutting, the cutting state of the molten pool image is directly returned as abnormal cutting.

7. The identification method according to claim 1, characterized in that: The method further comprises: S500, when the cutting state information of the molten pool image is that the cutting is not through, the position information of the current laser cutting is recorded, and based on the backtracking strategy, it is determined that the laser cutting head can return to the position where the cutting was not through for the first time when it needs to be retracted; or, S500a. During the multi-layer cutting process, when the cutting status information of the molten pool image of the current layer is that the cutting is not through, the position information of the current laser cutting is recorded, and based on the backtracking strategy, it is determined that the laser cutting head can return to the position where the cutting was not through for the first time when it needs to be retracted. When the control system receives an instruction to switch the current layer, it checks whether there is a state of not cutting through in the current layer. If so, the control system issues a retraction instruction to make the laser cutting head re-cut based on the position where the cutting is not through; if there is no state of not cutting through in the current layer, the instruction to switch the current layer is executed.

8. The identification method according to claim 7, characterized in that: Based on the backtracking strategy, it is determined that the laser cutting head can return to the position where the first cutting was not through when it needs to be retracted, including: During the cutting process, if the state of not cutting through is detected for the first time, the control system records the position information of the not cutting through, that is, the mechanical coordinates (x0, y0); The control system will continue to monitor, and within the preset distance S, if the proportion of incomplete cutting in the N1 images detected continuously exceeds half, a first retraction instruction will be generated; And, the control system records the position information of the laser cutting head, so that the laser cutting head returns to the position where the first incomplete cutting occurs based on the first retraction instruction to perform re-cutting; Among them, V represents the current cutting speed, and F represents the frame rate of the camera.

9. A laser cutting state recognition device based on machine vision, characterized in that: include: An image acquisition module, used to acquire a molten pool image during cutting by the laser cutting head; An intelligent analysis module, used to identify the molten pool image using a pre-trained molten pool state classification model to obtain a first identification result; And obtain brightness information of the molten pool image; and obtain cutting state information of the molten pool image according to the first recognition result and the brightness information, and a brightness threshold in a predetermined plate attribute.

10. A control system for laser cutting, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 8 when executing the program stored in the memory.

Images