Weld joint feature point extraction method and device and storage medium

Abstract

The invention provides a weld joint feature point extraction method and device and a storage medium, and belongs to the technical field of weld joint extraction, and the method comprises the steps: carrying out the image collection of a to-be-extracted light strip through a calibrated gray camera, and obtaining a plurality of original light strip images; preprocessing each original light strip image to obtain a preprocessed light strip image; performing filtering analysis on each preprocessed light strip image through a filtering network to obtain a filtered light strip image; and performing center point extraction and analysis on all the filtered light strip images to obtain a plurality of target center point coordinates. According to the method, the light strip noise can be effectively removed, and the light strip center extraction precision is greatly improved, so that high-precision weld joint feature point extraction is realized, and the problem that the light strip center with strong reflection interference is difficult to extract is solved.

Description

Technical Field

[0001] This invention relates to the field of weld seam extraction technology, specifically to a method, apparatus, and storage medium for extracting weld seam feature points. Background Technology

[0002] Welding is a process that uses heat, pressure, or a combination of both to bond separated metals or other materials atomically together to form a reliable joint. However, in actual industrial production, complex welding conditions are accompanied by intense arc light and dense spatter interference, making it difficult for traditional weld feature point extraction methods to achieve high-precision results. Among traditional weld feature point extraction methods, the gray-scale centroid method is susceptible to noise and lacks stability, while the Steger algorithm, although highly accurate, suffers from complex convolution operations and high computational overhead, making it difficult to meet the stringent real-time requirements of the welding process. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method, apparatus and storage medium for extracting weld feature points, which addresses the shortcomings of the prior art.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A method for extracting weld feature points, comprising the following steps: Multiple original light bar images are obtained by acquiring images of the light bar to be extracted using a calibrated grayscale camera. Each of the original light bar images is preprocessed to obtain a preprocessed light bar image corresponding to each of the original light bar images; Each of the preprocessed light stripe images is filtered and analyzed by a filtering network to obtain a filtered light stripe image corresponding to each of the original light stripe images. Center point extraction and analysis were performed on all the filtered light stripe images to obtain the coordinates of multiple target center points; Weld seam extraction analysis is performed on the coordinates of all the target center points, and the analysis results are used as the weld seam feature point extraction results.

[0005] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: A weld feature point extraction device, comprising: The image acquisition module is used to acquire images of the light stripes to be extracted using a calibrated grayscale camera, thereby obtaining multiple original light stripe images; The preprocessing module is used to preprocess each of the original light bar images to obtain preprocessed light bar images corresponding to each of the original light bar images; The filtering analysis module is used to perform filtering analysis on each of the preprocessed light stripe images through a filtering network to obtain filtered light stripe images corresponding to each of the original light stripe images. The center point extraction and analysis module is used to perform center point extraction and analysis on all the filtered light stripe images to obtain the coordinates of multiple target center points. The extraction result acquisition module is used to perform weld seam extraction analysis on the coordinates of all the target center points, and use the analysis results as the weld seam feature point extraction results.

[0006] Based on the above-mentioned method for extracting weld feature points, the present invention also provides a system for extracting weld feature points.

[0007] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a weld feature point extraction system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the weld feature point extraction method described above is implemented.

[0008] Based on the above-mentioned method for extracting weld feature points, the present invention also provides a computer-readable storage medium.

[0009] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the weld feature point extraction method as described above.

[0010] The beneficial effects of this invention are as follows: An original light stripe image is acquired by a calibrated grayscale camera; a preprocessed light stripe image is obtained by preprocessing the original light stripe image; a filtered light stripe image is obtained by filtering and analyzing the preprocessed light stripe image through a filtering network; the center point of the filtered light stripe image is extracted and analyzed to obtain the coordinates of the target center point; the weld seam is extracted and analyzed based on the coordinates of the target center point, and the analysis results are used as the weld seam feature point extraction results. This effectively removes light stripe noise, significantly improves the accuracy of light stripe center extraction, and thus achieves high-precision weld seam feature point extraction, solving the problem of difficulty in extracting the center of light stripes with strong reflective interference. Attached Figure Description

[0011] Figure 1 A schematic flowchart of the weld feature point extraction method provided in an embodiment of the present invention; Figure 2 A schematic diagram of the original light stripe image for the weld feature point extraction method provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the preprocessed light stripe image of the weld feature point extraction method provided in the embodiments of the present invention; Figure 4 This is a diagram of the filter network structure for the weld feature point extraction method provided in this embodiment of the invention. Figure 5A schematic diagram of the filtered light stripe image of the weld feature point extraction method provided in the embodiments of the present invention; Figure 6 A schematic diagram of the original laser centerline for the weld feature point extraction method provided in an embodiment of the present invention; Figure 7 A schematic diagram of the target center point line of the weld feature point extraction method provided in the embodiment of the present invention; Figure 8 Comparison of weld seam extraction effects of the weld seam feature point extraction method provided in the embodiments of the present invention; Figure 9 This is a block diagram of a weld feature point extraction device provided in an embodiment of the present invention. Detailed Implementation

[0012] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0013] Figure 1 This is a flowchart illustrating a method for extracting weld feature points according to an embodiment of the present invention.

[0014] like Figure 1 and 2 As shown, a method for extracting weld feature points includes the following steps: S1: The calibrated grayscale camera is used to acquire images of the light stripe to be extracted, resulting in multiple original light stripe images; S2: Preprocess each of the original light bar images to obtain preprocessed light bar images corresponding to each of the original light bar images; S3: Each of the preprocessed light stripe images is filtered and analyzed by a filtering network to obtain a filtered light stripe image corresponding to each of the original light stripe images; S4: Perform center point extraction and analysis on all the filtered light stripe images to obtain the coordinates of multiple target center points; S5: Perform weld seam extraction analysis on the coordinates of all the target center points, and use the analysis results as the weld seam feature point extraction results.

[0015] It should be understood that a calibrated grayscale camera is used to acquire laser stripe images of light stripes with strong reflective interference (i.e., the light stripes to be extracted).

[0016] In the above embodiments, the original light stripe image is acquired by a calibrated grayscale camera, the original light stripe image is preprocessed to obtain a preprocessed light stripe image, the preprocessed light stripe image is filtered by a filtering network to obtain a filtered light stripe image, the center point of the filtered light stripe image is extracted and analyzed to obtain the coordinates of the target center point, the weld seam is extracted and analyzed based on the coordinates of the target center point, and the analysis results are used as the weld seam feature point extraction results. This can effectively remove light stripe noise, greatly improve the accuracy of light stripe center extraction, thereby achieving high-precision weld seam feature point extraction and solving the problem of difficulty in extracting the center of light stripes with strong reflective interference.

[0017] Optionally, as an embodiment of the present invention, such as Figure 3 As shown, the process of preprocessing each of the original light bar images to obtain a preprocessed light bar image corresponding to each of the original light bar images includes: Each of the original light bar images is annotated to obtain an annotated light bar image corresponding to each of the original light bar images; Each of the labeled light bar images is converted to a different format to obtain a preprocessed light bar image corresponding to each of the original light bar images.

[0018] It should be understood that the acquired images (i.e., the original light bar images) are then renamed, uploaded to the CVAT platform for dataset label drawing (i.e. annotation processing), and finally, dataset partitioning and format conversion are performed.

[0019] In the above embodiments, each original light bar image is preprocessed to obtain a preprocessed light bar image, which can effectively remove light bar noise and greatly improve the accuracy of light bar center extraction.

[0020] Optionally, as an embodiment of the present invention, such as Figure 4 and 5 As shown, the filtering network includes a YOLOv8 network. The process of performing filtering analysis on each of the preprocessed light bar images through the filtering network to obtain filtered light bar images corresponding to each of the original light bar images includes: The YOLOv8 network is used to predict each of the preprocessed light bar images to obtain prediction boxes corresponding to each of the original light bar images. Import the ground truth bounding boxes corresponding to each of the original light bar images, and perform loss function analysis on each of the predicted bounding boxes and the ground truth bounding boxes corresponding to each of the original light bar images to obtain the loss function corresponding to each of the original light bar images; The YOLOv8 network parameters are updated based on all the loss functions to obtain the updated model; The updated model is used to filter each of the preprocessed light stripe images to obtain filtered light stripe images corresponding to each of the original light stripe images.

[0021] It should be understood that the YOLOv8-seg network (i.e., the YOLOv8 network) is used to train the prepared dataset (i.e., the preprocessed light bar image) to obtain the weight parameter file (i.e. the prediction box) of the trained model, with the suffix .pt.

[0022] Specifically, the size of the network model during training was adjusted, increasing the network size and thus expanding the model's receptive field. A larger receptive field allows the model to capture more global information, helping to improve the accuracy of target localization and segmentation. When the model input is 640x640, the three output feature maps P5, P4, and P3 are 20x20, 40x40, and 80x80, respectively. The 80x80 feature map has the highest resolution, thus enabling it to capture smaller targets. When the model input is changed to 1280x1280, the three output feature maps are 40x40, 80x80, and 160x160, respectively. This is twice the size of the previous model, thereby improving the segmentation accuracy for targets of different sizes. In particular, the problem of insufficient segmentation accuracy for small objects such as weld seam stripes is effectively solved.

[0023] In the above embodiments, the filtered light stripe image is obtained by filtering and analyzing the preprocessed light stripe image through a filtering network, which expands the receptive field of the model and enables the model to capture more global information. This helps to improve the positioning and segmentation accuracy of the target and solves the problem of insufficient segmentation accuracy of weld light stripes.

[0024] Optionally, as an embodiment of the present invention, the process of performing loss function analysis on each of the predicted bounding boxes and the ground truth bounding boxes corresponding to each of the original light bar images to obtain the loss function corresponding to each of the original light bar images includes: The prediction height, prediction width, and center point coordinates of the prediction box corresponding to each of the original light bar images are extracted from each of the prediction boxes respectively. The true height, true width, and center point coordinates of the true frame corresponding to each of the original light bar images are extracted from each of the true frames respectively. The minimum diagonal distance between each predicted bounding box and the ground truth bounding box corresponding to each original light bar image is calculated to obtain the target diagonal distance corresponding to each original light bar image. The loss function corresponding to each original light bar image is obtained by calculating the predicted bounding box, the predicted height corresponding to each original light bar image, the predicted width corresponding to each original light bar image, the center point coordinates of the predicted bounding box corresponding to each original light bar image, the ground truth bounding box corresponding to each original light bar image, the ground truth height corresponding to each original light bar image, the ground truth width corresponding to each original light bar image, the center point coordinates of the ground truth bounding box corresponding to each original light bar image, and the target diagonal distance corresponding to each original light bar image using the first formula. , in, , , in, For loss function, For the prediction box, For the true frame, The vertical weighting coefficient. The coordinates of the center point of the prediction box. The coordinates of the center point of the true bounding box. The diagonal distance to the target. The horizontal weighting coefficient. To predict the width, For the actual width, For the actual height, To predict altitude, This is the scaling factor.

[0025] Specifically, a loss function is used to measure the difference between the model's predicted values ​​and the true values. The smaller the loss function, the better the model's performance. It plays a crucial role in the model training phase, helping to optimize model parameters. Common loss functions include mean squared error (MSE) and cross-entropy loss. YOLOv8 uses the CIOU loss function by default, and its expression is as follows: , in This represents the Euclidean distance between the centers of the predicted bounding box and the ground truth bounding box. This represents the diagonal distance of the smallest closure region containing both the predicted and ground truth boxes.

[0026] , , A good loss function for bounding box regression should consider three important geometric factors: overlapping area, center point distance, and aspect ratio. The native IoU loss considers the overlapping area, while the GIoU loss is largely dependent on the IoU loss. The DIoU loss considers both the overlapping area and the center point distance of the bounding box. The CIoU loss, in addition to DIoU, adds a requirement for consistent aspect ratio. The definition of Shape_iou is shown below: , , Where B and Bgt represent the predicted bounding box and the ground truth bounding box, respectively.

[0027] , in, These are the coordinates of the center point of the ground truth bounding box (i.e., the coordinates of the center point of the actual bounding box). C is the center point coordinate of the predicted bounding box (i.e., the center point coordinate of the predicted bounding box), and C is the diagonal distance of the minimum bounding box between B and Bgt (i.e., the diagonal distance of the target).

[0028] , , Here, scale is a scaling factor that is related to the size of the target in the dataset, and ww and hh represent the weight coefficients in the horizontal and vertical directions, respectively, whose values ​​are related to the shape of the ground truth box.

[0029] .

[0030] It should be understood that, as shown in Table 1, the evaluation parameters were obtained by training the base model (YOLOv8) and the improved model (Shape_iou-YOLOv8) respectively.

[0031] Table 1 As can be seen from the table above, Shape_iou–YOLOv8 performs better than the original YOLOv8 in object detection and instance segmentation tasks, especially in the more challenging metric of mAP@[0.5:0.95], where it shows a significant improvement of 16.4%, indicating that it is stronger in terms of accuracy and generalization ability.

[0032] In the above embodiments, loss function analysis is performed on each predicted box and the ground truth box to obtain the loss function, which enables the model to capture more global information, helps to improve the accuracy of target localization and segmentation, and solves the problem of insufficient segmentation accuracy of weld seam light stripes.

[0033] Optionally, as an embodiment of the present invention, such as Figure 6 and 7 As shown, the process of extracting and analyzing the center points of all the filtered light stripe images to obtain the coordinates of multiple target center points includes: Multiple laser center point regions corresponding to each of the filtered light bar images are extracted from each of the filtered light bar images, and each laser center point region includes multiple original laser center point coordinates; The coordinates of multiple original laser center points corresponding to each laser center point region in each of the filtered light stripe images are fitted to obtain the first straight line equation corresponding to each laser center point region in each of the filtered light stripe images. The slope angles corresponding to the laser center point regions in the filtered light stripe images are extracted from each of the first straight line equations; The least squares algorithm is used to fit the coordinates of multiple original laser center points corresponding to each laser center point region in each of the filtered light stripe images to obtain the first coefficient and the second coefficient corresponding to each laser center point region in each of the filtered light stripe images. The residual value corresponding to the coordinates of each original laser center point, the first coefficient corresponding to each laser center point region in each filtered light stripe image, and the second coefficient corresponding to each laser center point region in each filtered light stripe image are calculated using the second formula. The second formula is: , in, For the first The first filtered light stripe image The laser center point region The residual values ​​corresponding to the original laser center point coordinates For the first The first filtered light stripe image The first laser center point region The original laser center point coordinates, For the first The first filtered light stripe image The first coefficient corresponding to the laser center point region For the first The first filtered light stripe image The second coefficient corresponding to each laser center point region; The error variance corresponding to the coordinates of the original laser center point in each laser center point region in each filtered light stripe image is obtained by calculating each residual value using the third equation. The third equation is: , in, For the first The first filtered light stripe image The laser center point region Error variance corresponding to the coordinates of the original laser center point For the first The first filtered light stripe image The laser center point region The residual values ​​corresponding to the original laser center point coordinates For the first The first filtered light stripe image The laser center point region The residual values ​​corresponding to the original laser center point coordinates For the first The original laser center point coordinates, This represents the number of original laser center point coordinates; Determine whether the residual value meets the first judgment condition. If not, then all the original laser center point coordinates in the laser center point region corresponding to the residual value are taken as the target center point coordinates. If yes, then the original laser center point coordinates corresponding to the residual value, the first coefficient of the original laser center point coordinates corresponding to the residual value, the second coefficient of the original laser center point coordinates corresponding to the residual value, and the slope angle corresponding to the residual value are calculated by the fourth formula to obtain the corrected laser center point coordinates corresponding to the residual value, and the corrected laser center point coordinates are taken as the target center point coordinates, thereby obtaining multiple target center point coordinates. The first judgment condition is: , The fourth formula is: , in, To smooth the weights, For the first The first filtered light stripe image The laser center point region The target center point coordinates corresponding to the original laser center point coordinates For the first The first filtered light stripe image The first laser center point region The original laser center point coordinates, For the first The first filtered light stripe image The first coefficient corresponding to the laser center point region For the first The first filtered light stripe image The second coefficient corresponding to each laser center point region For the first The first filtered light stripe image The slope angle corresponding to the laser center point region.

[0034] Specifically, using existing line laser center point extraction methods, the coordinates of the light stripe's center point can be extracted. Then, a dynamic Region of Interest (ROI) is set with the k-th line laser center point as the starting point and the (k+q)-th line laser center point as the ending point. This ROI contains the coordinates of q line laser center points, and the equation of the straight line fitted to the coordinates of all line laser center points in this ROI region is assumed to be: , A least-squares fitting model is established for the coordinates of all center points within the ROI, thereby calculating... (i.e., the first coefficient) and (i.e., the second coefficient), as shown in the following formula: , in, The coordinates of the center point within the ROI (i.e., the original laser center point coordinates).

[0035] It should be understood that the calculation is... The residual is: , And calculate the error variance within the ROI, as shown in the following formula: .

[0036] Specifically, when discovered When, then point The error is relatively large and further smoothing is required. In this invention... .

[0037] Point Substitute into the equation of the straight line l k have to: , If the calculated points In l k Above, then w i <0, otherwise w i >0. Therefore, the center point of the corrected light stripe can be calculated. As shown in the following formula: , in, , Then update the ROI region, making If in Points within do not satisfy ,but, , This indicates the center point number of the starting point and the center point number of the ending point of the next ROI region. By repeating the above steps, the center point can be smoothed and denoised. The center point extracted by this invention has better smoothness and noise resistance.

[0038] In the above embodiments, center point extraction and analysis are performed on all filtered light stripe images to obtain the coordinates of multiple target center points, thereby achieving center point smoothing and noise reduction, and improving the smoothness and anti-noise effect of the center points.

[0039] Optionally, as an embodiment of the present invention, the process of performing weld seam extraction analysis on the coordinates of all the target center points and using the analysis results as the weld seam feature point extraction results includes: Definition of the first The coordinates of the center point of each target are ; The fifth formula is used to calculate the coordinates of each of the target center points, resulting in a weighted average sum corresponding to the coordinates of each target center point. , in, , in, For the first The weighted average sum of the coordinates of each target center point. For the radius of the domain, For the first Each weighted weight, For the first The curvature corresponding to the coordinates of the target center point For the first Coordinates of the center point of the target. For the first Coordinates of the center point of the target. For the first Coordinates of the center point of each target; When the weighted average sum is greater than a preset threshold, the coordinates of the target center point corresponding to the weighted average sum are used as the rough weld feature points. and the rough weld feature points Corresponding target center point coordinates As the coordinates of the first center point to be processed, and the rough weld feature points are used... Corresponding target center point coordinates As the coordinates of the second center point to be processed; The coordinates of the first center point to be processed and the coordinates of the second center point to be processed are fitted to obtain the equation of the second straight line; A range interval is constructed using the coordinates of the first center point to be processed as the left boundary and the coordinates of the second center point to be processed as the right boundary. All the coordinates of the target center points within the range interval are used as the coordinates of the third center point to be processed, thereby obtaining multiple third center point coordinates to be processed. Calculate the distance between the coordinates of each of the third center points to be processed and the equation of the second straight line to obtain the target distance to the coordinates of each of the third center points to be processed; The maximum value among all the target distances is selected, and the coordinates of the third center point to be processed corresponding to the maximum target distance are used as the result of weld feature point extraction.

[0040] It should be understood that the coordinates of the i-th smoothed point (i.e., the coordinates of the target center point) are already known. Calculate the coordinates of its adjacent neighborhood points. , The curvature is: , Where g is the radius of the domain, in this invention, .

[0041] Then, the curvature within this region is weighted as follows: , in, .

[0042] Specifically, the smoothed coordinate points (i.e., the coordinates of the target center point) are filtered to select those that meet the requirements. The coordinates of the points in this invention .like Not satisfied Then update the filter region, let i = i + 1, and repeat the calculation. This process continues until all smoothed center points (i.e., the coordinates of the target center point) have been traversed. At this point, the rough weld feature points can be obtained. .by (i.e., the coordinates of the first center point to be processed) and (i.e., the coordinates of the second center point to be processed) Establish the equation of the straight line (i.e., the equation of the second straight line), then calculate All points in the interval (referring to smoothed coordinates) (i.e., the coordinates of the target center point) are connected to the line. Distance (i.e., the distance of the second straight line equation) (i.e., target distance). Finally, select the coordinates that satisfy the following conditions: This point is the precise weld feature point. (i.e., the result of extracting weld feature points).

[0043] In the above embodiments, weld seam extraction analysis is performed on the coordinates of all target center points, and the analysis results are used as the weld seam feature point extraction results. This can effectively remove light stripe noise and greatly improve the accuracy of light stripe center extraction, thereby achieving high-precision weld seam feature point extraction and solving the problem of difficulty in extracting the center of light stripes with strong reflective interference.

[0044] Optionally, as another embodiment of the present invention, this invention eliminates the need for additional image processing on the original image. It trains a YOLOv8 model to handle light stripes with strong reflective noise and deploys the model on an RK3588 development board. For images captured by the camera with reflective interference, denoising and segmentation of the light stripes are performed directly, facilitating subsequent high-precision extraction of the light stripe center and weld seam. This significantly reduces the steps involved in light stripe center extraction and improves extraction accuracy. Furthermore, due to the increasing demand for lightweight and convenient devices, deploying deep learning models on embedded platforms will become a future development trend.

[0045] Optionally, as another embodiment of the present invention, in order to address the problem that traditional algorithms have difficulty in solving the problem of accurate extraction of the center of strongly reflective light stripes, the present invention introduces a deep learning model to effectively remove light stripe noise, thereby greatly improving the accuracy of subsequent light stripe center extraction; in view of the future integration of visual welding, considering that embedded systems are more lightweight, visual welding technology is deployed in embedded systems.

[0046] Alternatively, as another embodiment of the present invention, since the model of the present invention needs to be deployed on an embedded device, model conversion is required to adapt it to the development board.

[0047] Alternatively, as another embodiment of the present invention, the process of converting pt format to onnx format is as follows: ONNX (Open Neural Network Exchange) is a format for weight files containing not only weight values ​​but also network flow information, input / output information for each layer, and other auxiliary information. Therefore, ONNX serves as a bridge between deep learning frameworks and inference engines, much like an intermediate language in a compiler. Due to varying framework compatibility, ONNX is typically used only to represent static graphs, which are easier to deploy. Only by converting the trained PT format weight file to ONNX format, and then to RKNN format, can the NPU be used for model inference on the RK3588.

[0048] Alternatively, as another embodiment of the present invention, the process of converting ONNX format to RKNN format is as follows: The Rockchip RK3588 series chips feature a Neural Processing Unit (NPU). RKNN allows users to quickly deploy AI models to Rockchip chips for hardware-accelerated model inference using the NPU. RKNN is the model type used by the Rockchip NPU platform, and model files end with the .rknn extension. Therefore, model files must be converted from ONNX format to RKNN format before they can be used on the RK3588 platform.

[0049] Optionally, as another embodiment of the present invention, the present invention provides a method for using a deep learning model in an embedded system to remove light stripe reflection interference and then accurately extract the center of the light stripe, thereby providing data support for subsequent weld feature point extraction and three-dimensional reconstruction.

[0050] Alternatively, as another embodiment of the present invention, such as Figure 8 As shown, this invention addresses the difficulty of traditional algorithms in removing reflective interference from structured light stripes and improving the accuracy of centerline extraction and fillet weld feature point extraction algorithms. It provides a high-reflectivity light stripe segmentation and denoising method, along with a center point smoothing algorithm and a fillet weld feature point extraction method. This offers a feasible solution for using visual welding technology in embedded systems.

[0051] Optionally, as another embodiment of the present invention, to address the problem of the inaccuracy of extracting the center of strongly reflective light stripes that traditional algorithms struggle with, the present invention introduces a deep learning model to effectively remove light stripe noise, thereby significantly improving the accuracy of subsequent light stripe center extraction and achieving high-precision weld feature point extraction. Considering the future integration of visual welding and the lighter weight of embedded systems, visual welding technology is deployed in embedded systems.

[0052] Alternatively, as another embodiment of the present invention, the specific steps of the present invention are as follows: Step 1: Acquire laser stripe image Operation: Images of reflected light stripes from line structured light are acquired using a camera and laser system and used as training and testing data for the model.

[0053] Purpose: To provide the original images for subsequent dataset creation and model training.

[0054] Step 2: Dataset creation and model training (performed on a PC) Sub-steps: Use the collected images to create datasets (e.g., image annotation).

[0055] The model in .pt format was trained using the modified Rockchip Ultralytics YOLOv8 framework.

[0056] Step 3: Model Format Conversion Sub-steps: Use Ultralytics to convert .pt format to .onnx format.

[0057] Use RKNN-Toolkit2 to convert .onnx models to .rknn models.

[0058] Objective: To enable the model to be deployed on embedded hardware such as RK3588.

[0059] Step 4: Deploy the RKNN model to the embedded system (RK3588) Embedded Systems: Image Reasoning and Center Extraction Step 5: YOLOv8 Light Bar Segmentation Running the RKNN model on RK3588 Output: Laser stripe regions detected by YOLOv8 (mask or bounding box, etc.) Step 6: Extract the center line of the light stripe using the grayscale centroid method. Method: Calculate the centroid of the grayscale distribution of the image pixels to obtain the center line of the light stripe.

[0060] Step 7: Center point extraction method based on local straight line fitting to calculate variance.

[0061] Objective: To remove jitter and spike interference, making the center line smoother and more stable.

[0062] Step 8: A method for extracting fillet welds based on curvature calculation of multi-scale neighborhoods.

[0063] Output: Weld feature points.

[0064] Figure 9This is a block diagram of a weld feature point extraction device provided in an embodiment of the present invention.

[0065] Alternatively, as another embodiment of the present invention, such as Figure 9 As shown, a weld feature point extraction device includes: The image acquisition module is used to acquire images of the light stripes to be extracted using a calibrated grayscale camera, thereby obtaining multiple original light stripe images; The preprocessing module is used to preprocess each of the original light bar images to obtain preprocessed light bar images corresponding to each of the original light bar images; The filtering analysis module is used to perform filtering analysis on each of the preprocessed light stripe images through a filtering network to obtain filtered light stripe images corresponding to each of the original light stripe images. The center point extraction and analysis module is used to perform center point extraction and analysis on all the filtered light stripe images to obtain the coordinates of multiple target center points. The extraction result acquisition module is used to perform weld seam extraction analysis on the coordinates of all the target center points, and use the analysis results as the weld seam feature point extraction results.

[0066] Optionally, as an embodiment of the present invention, the preprocessing module is specifically used for: Each of the original light bar images is annotated to obtain an annotated light bar image corresponding to each of the original light bar images; Each of the labeled light bar images is converted to a different format to obtain a preprocessed light bar image corresponding to each of the original light bar images.

[0067] Optionally, another embodiment of the present invention provides a weld feature point extraction system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the weld feature point extraction method described above. This system can be a computer or similar system.

[0068] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the weld feature point extraction method as described above.

[0069] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0070] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0073] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for extracting feature points of a weld, characterized in that, Includes the following steps: Multiple original light bar images are obtained by acquiring images of the light bar to be extracted using a calibrated grayscale camera. Each of the original light bar images is preprocessed to obtain a preprocessed light bar image corresponding to each of the original light bar images; Each of the preprocessed light stripe images is filtered and analyzed by a filtering network to obtain a filtered light stripe image corresponding to each of the original light stripe images. Center point extraction and analysis were performed on all the filtered light stripe images to obtain the coordinates of multiple target center points; Weld seam extraction analysis is performed on the coordinates of all the target center points, and the analysis results are used as the weld seam feature point extraction results.

2. The method for extracting weld feature points according to claim 1, characterized in that, The process of preprocessing each of the original light bar images to obtain a preprocessed light bar image corresponding to each of the original light bar images includes: Each of the original light bar images is annotated to obtain an annotated light bar image corresponding to each of the original light bar images; Each of the labeled light bar images is converted to a different format to obtain a preprocessed light bar image corresponding to each of the original light bar images.

3. The method for extracting weld feature points according to claim 1, characterized in that, The filtering network includes a YOLOv8 network. The process of performing filtering analysis on each of the preprocessed light bar images through the filtering network to obtain filtered light bar images corresponding to each of the original light bar images includes: The YOLOv8 network is used to predict each of the preprocessed light bar images to obtain prediction boxes corresponding to each of the original light bar images. Import the ground truth bounding boxes corresponding to each of the original light bar images, and perform loss function analysis on each of the predicted bounding boxes and the ground truth bounding boxes corresponding to each of the original light bar images to obtain the loss function corresponding to each of the original light bar images; The YOLOv8 network parameters are updated based on all the loss functions to obtain the updated model; The updated model is used to filter each of the preprocessed light stripe images to obtain filtered light stripe images corresponding to each of the original light stripe images.

4. The method for extracting weld feature points according to claim 3, characterized in that, The process of performing loss function analysis on each predicted bounding box and the ground truth bounding box corresponding to each original light bar image to obtain the loss function corresponding to each original light bar image includes: The prediction height, prediction width, and center point coordinates of the prediction box corresponding to each of the original light bar images are extracted from each of the prediction boxes respectively. The true height, true width, and center point coordinates of the true frame corresponding to each of the original light bar images are extracted from each of the true frames respectively. The minimum diagonal distance between each predicted bounding box and the ground truth bounding box corresponding to each original light bar image is calculated to obtain the target diagonal distance corresponding to each original light bar image. The loss function corresponding to each original light bar image is obtained by calculating the predicted bounding box, the predicted height corresponding to each original light bar image, the predicted width corresponding to each original light bar image, the center point coordinates of the predicted bounding box corresponding to each original light bar image, the ground truth bounding box corresponding to each original light bar image, the ground truth height corresponding to each original light bar image, the ground truth width corresponding to each original light bar image, the center point coordinates of the ground truth bounding box corresponding to each original light bar image, and the target diagonal distance corresponding to each original light bar image using the first formula. ， in, , , in, For loss function, For the prediction box, For the true frame, The vertical weighting coefficient. The coordinates of the center point of the prediction box. The coordinates of the center point of the true bounding box. The diagonal distance to the target. The horizontal weighting coefficient. To predict the width, For the actual width, For the actual height, To predict altitude, This is the scaling factor.

5. The method for extracting weld feature points according to claim 1, characterized in that, The process of extracting and analyzing the center points of all the filtered light stripe images to obtain the coordinates of multiple target center points includes: Multiple laser center point regions corresponding to each of the filtered light bar images are extracted from each of the filtered light bar images, and each laser center point region includes multiple original laser center point coordinates; The coordinates of multiple original laser center points corresponding to each laser center point region in each of the filtered light stripe images are fitted to obtain the first straight line equation corresponding to each laser center point region in each of the filtered light stripe images. The slope angles corresponding to the laser center point regions in the filtered light stripe images are extracted from each of the first straight line equations; The least squares algorithm is used to fit the coordinates of multiple original laser center points corresponding to each laser center point region in each of the filtered light stripe images to obtain the first coefficient and the second coefficient corresponding to each laser center point region in each of the filtered light stripe images. The residual value corresponding to the coordinates of each original laser center point, the first coefficient corresponding to each laser center point region in each filtered light stripe image, and the second coefficient corresponding to each laser center point region in each filtered light stripe image are calculated using the second formula. The second formula is: ， in, For the first The first filtered light stripe image The laser center point region The residual values ​​corresponding to the original laser center point coordinates For the first The first filtered light stripe image The first laser center point region The original laser center point coordinates, For the first The first filtered light stripe image The first coefficient corresponding to the laser center point region For the first The first filtered light stripe image The second coefficient corresponding to each laser center point region; The error variance corresponding to the coordinates of the original laser center point in each laser center point region in each filtered light stripe image is obtained by calculating each residual value using the third equation. The third equation is: ， in, For the first The first filtered light stripe image The laser center point region Error variance corresponding to the coordinates of the original laser center point For the first The first filtered light stripe image The laser center point region The residual values ​​corresponding to the original laser center point coordinates For the first The first filtered light stripe image The laser center point region The residual values ​​corresponding to the original laser center point coordinates For the first The original laser center point coordinates, This represents the number of original laser center point coordinates; Determine whether the residual value meets the first judgment condition. If not, then all the original laser center point coordinates in the laser center point region corresponding to the residual value are taken as the target center point coordinates. If yes, then the original laser center point coordinates corresponding to the residual value, the first coefficient of the original laser center point coordinates corresponding to the residual value, the second coefficient of the original laser center point coordinates corresponding to the residual value, and the slope angle corresponding to the residual value are calculated by the fourth formula to obtain the corrected laser center point coordinates corresponding to the residual value, and the corrected laser center point coordinates are taken as the target center point coordinates, thereby obtaining multiple target center point coordinates. The first judgment condition is: ， The fourth formula is: ， in, To smooth the weights, For the first The first filtered light stripe image The laser center point region The target center point coordinates corresponding to the original laser center point coordinates For the first The first filtered light stripe image The first laser center point region The original laser center point coordinates, For the first The first filtered light stripe image The first coefficient corresponding to the laser center point region For the first The first filtered light stripe image The second coefficient corresponding to the laser center point region For the first The first filtered light stripe image The slope angle corresponding to the laser center point region.

6. The method for extracting weld feature points according to claim 1, characterized in that, The process of performing weld seam extraction analysis on the coordinates of all the target center points and using the analysis results as the weld seam feature point extraction results includes: Definition of the first The coordinates of the center point of each target are ; The fifth formula is used to calculate the coordinates of each of the target center points, resulting in a weighted average sum corresponding to the coordinates of each target center point. ， in, , in, For the first The weighted average sum of the coordinates of each target center point. For the radius of the domain, For the first Each weighted weight, For the first The curvature corresponding to the coordinates of the target center point For the first Coordinates of the center point of each target For the first Coordinates of the center point of each target For the first Coordinates of the center point of each target; When the weighted average sum is greater than a preset threshold, the coordinates of the target center point corresponding to the weighted average sum are used as the rough weld feature points. and the rough weld feature points Corresponding target center point coordinates As the coordinates of the first center point to be processed, and the rough weld feature points are used... Corresponding target center point coordinates As the coordinates of the second center point to be processed; The coordinates of the first center point to be processed and the coordinates of the second center point to be processed are fitted to obtain the equation of the second straight line; A range interval is constructed using the coordinates of the first center point to be processed as the left boundary and the coordinates of the second center point to be processed as the right boundary. All the coordinates of the target center points within the range interval are used as the coordinates of the third center point to be processed, thereby obtaining multiple third center point coordinates to be processed. Calculate the distance between the coordinates of each of the third center points to be processed and the equation of the second straight line to obtain the target distance to the coordinates of each of the third center points to be processed; The maximum value among all the target distances is selected, and the coordinates of the third center point to be processed corresponding to the maximum target distance are used as the result of weld feature point extraction.

7. A device for extracting weld feature points, characterized in that, include: The image acquisition module is used to acquire images of the light stripes to be extracted using a calibrated grayscale camera, thereby obtaining multiple original light stripe images; The preprocessing module is used to preprocess each of the original light bar images to obtain preprocessed light bar images corresponding to each of the original light bar images; The filtering analysis module is used to perform filtering analysis on each of the preprocessed light stripe images through a filtering network to obtain filtered light stripe images corresponding to each of the original light stripe images. The center point extraction and analysis module is used to perform center point extraction and analysis on all the filtered light stripe images to obtain the coordinates of multiple target center points. The extraction result acquisition module is used to perform weld seam extraction analysis on the coordinates of all the target center points, and use the analysis results as the weld seam feature point extraction results.

8. The weld feature point extraction device according to claim 7, characterized in that, The preprocessing module is specifically used for: Each of the original light bar images is annotated to obtain an annotated light bar image corresponding to each of the original light bar images; Each of the labeled light bar images is converted to a different format to obtain a preprocessed light bar image corresponding to each of the original light bar images.

9. A weld feature point extraction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the weld feature point extraction method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the weld feature point extraction method as described in any one of claims 1 to 6.

Images