A method and related apparatus for weld seam grinding and correction based on structured light

Abstract

The embodiment of the application provides a welding seam polishing deviation correction method based on structured light and related equipment, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring a welding image; performing welding spot identification on the welding image through a trained YOLO model to obtain welding point image coordinates; performing coordinate conversion processing on the welding point image coordinates based on the geometric measurement of structured light to obtain real three-dimensional coordinates of the welding point; and performing deviation calculation on a preset theoretical welding seam track and the real three-dimensional coordinates to obtain position deviation, so that the polishing tool is corrected. The embodiment of the application can improve the speed and stability of welding spot identification and the precision of welding seam polishing.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and related equipment for correcting weld grinding based on structured light. Background Technology

[0002] In the field of industrial automation, accurate tracking of the weld seam trajectory is crucial for ensuring processing quality when automating the grinding of welded parts. Currently, vision-based weld seam tracking is the mainstream technology, mainly divided into two categories: one is the traditional structured light method, which directly identifies weld seam features from laser images through image processing algorithms (such as centerline extraction); the other is the pure deep learning method (such as semantic segmentation), which directly identifies the weld seam region from natural light images.

[0003] Traditional structured light methods suffer from poor robustness, heavily relying on clear laser stripes and high-quality images for effectiveness. They are prone to tracking interruptions or positioning errors when encountering strong surface reflections, welding spatter, oil contamination, bevel variations, or low contrast. Deep learning methods struggle to balance accuracy and speed. Models capable of pixel-level segmentation (such as U-Net) have high computational demands, making it difficult to meet the high-frequency requirements of real-time robot correction. While lightweight models are fast, their positioning accuracy often falls short of the sub-pixel accuracy required for grinding processes. Therefore, related weld seam tracking and grinding solutions cannot simultaneously achieve both accuracy and speed, leading to deviations or failures to track weld seams in a timely manner.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose a structured light-based method and related equipment for correcting weld seam grinding, which can improve the speed and stability of weld point identification and the accuracy of weld seam grinding.

[0006] To achieve the above objectives, one aspect of this application proposes a weld grinding and correction method based on structured light, the method comprising: Acquire welding images; The weld point coordinates are obtained by identifying weld points in the welding image using a trained YOLO model. The coordinates of the weld point image are transformed using geometric measurement based on structured light to obtain the true three-dimensional coordinates of the weld point. The deviation between the preset theoretical weld trajectory and the actual three-dimensional coordinates is calculated to obtain the positional deviation, which is then used to correct the grinding tool.

[0007] In some embodiments, the step of identifying weld points in the welding image using a trained YOLO model to obtain weld point image coordinates includes: The welding image is subjected to image recognition using a trained YOLO model to obtain the intersection image of the laser and the weld. Feature extraction is performed on the center line of the laser in the intersecting image to obtain the center line of the light stripe; The centerline of the weld in the intersecting image is obtained by feature extraction. The intersection point of the light stripe centerline and the weld centerline is calculated to obtain the image coordinates of the weld point.

[0008] In some embodiments, the step of performing image recognition on the laser and weld seam of the welding image using a trained YOLO model to obtain an intersecting image of the laser and weld seam includes: The intersection of the laser and the weld in the welding image is identified using a trained YOLO model to obtain the coordinates of the intersection area. The welding image is segmented based on the coordinates of the intersecting region to obtain the intersecting image.

[0009] In some embodiments, the trained YOLO model is obtained through the following steps: Obtain a training dataset, which includes several training images with labeled areas where the laser intersects with the weld. The training image is input into the YOLO model, and the bounding box coordinates are output. With the goal of reducing the deviation between the bounding box coordinates and the true coordinates, the parameters of the YOLO model are adjusted to obtain the trained YOLO model.

[0010] In some embodiments, the structured light-based geometric measurement performs coordinate transformation on the weld point image coordinates to obtain the true three-dimensional coordinates of the weld point, including: Based on the camera's intrinsic parameters and the image coordinates of the weld seam, determine the ray originating from the camera's optical center; The intersection point of the ray and the preset laser plane of the structured light is calculated to obtain the first three-dimensional coordinates in the camera coordinate system; The first three-dimensional coordinates are transformed according to the preset calibration matrix of the structured light to obtain the true three-dimensional coordinates.

[0011] In some embodiments, the step of calculating the deviation between the preset theoretical weld trajectory and the actual three-dimensional coordinates to obtain the positional deviation, in order to correct the grinding tool, includes: The positional deviation is obtained by subtracting the actual three-dimensional coordinates from the coordinates of the theoretical weld trajectory. The movement of the grinding tool is controlled based on the positional deviation in order to track the weld.

[0012] To achieve the above objectives, another aspect of this application provides a weld grinding and correction device, the device comprising: The acquisition module is used to acquire welding images; The recognition module is used to identify weld points in the welding image using a trained YOLO model, and obtain the coordinates of the weld point image. The calculation module is used to perform coordinate transformation processing on the coordinates of the weld point image based on structured light geometric measurement to obtain the true three-dimensional coordinates of the weld point; The deviation correction module is used to calculate the deviation between the preset theoretical weld trajectory and the actual three-dimensional coordinates to obtain the positional deviation, so as to correct the deviation of the grinding tool.

[0013] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0014] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0016] The embodiments of this application include at least the following beneficial effects: This application provides a method, apparatus, electronic device, storage medium, and program product for weld grinding correction based on structured light. This scheme uses a trained YOLO model to identify weld points in the welding image, obtaining the coordinates of the weld point image. Based on structured light geometric measurement, the coordinates of the weld point image are transformed to obtain the true three-dimensional coordinates of the weld point. The deviation between the preset theoretical weld trajectory and the true three-dimensional coordinates is calculated to obtain the positional deviation, which is then used to correct the grinding tool. By using the YOLO model to achieve rapid identification of weld point image coordinates, and then using structured light geometric measurement for coordinate transformation to determine the true three-dimensional coordinates, an organic combination of intelligent identification and geometric measurement is achieved, simultaneously improving the accuracy, stability, and real-time performance of weld grinding tracking. Attached Figure Description

[0017] Figure 1 This is a flowchart of the weld grinding and correction method based on structured light provided in the embodiments of this application; Figure 2This is a complete flowchart of the weld grinding and correction method based on structured light provided in the embodiments of this application; Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] In the field of industrial automation, accurate tracking of the weld seam trajectory is crucial for ensuring processing quality when automating the grinding of welded parts. Currently, vision-based weld seam tracking is the mainstream technology, mainly divided into two categories: one is the traditional structured light method, which directly identifies weld seam features from laser stripe images through image processing algorithms (such as centerline extraction); the other is the pure deep learning method (such as semantic segmentation), which directly identifies the weld seam region from natural light images.

[0021] The aforementioned technologies have the following shortcomings that urgently need to be addressed when applied in complex industrial settings: Traditional structured light methods suffer from poor robustness and rely heavily on manual feature design. Their effectiveness depends heavily on clear laser stripes and high-quality images. When encountering conditions such as strong surface reflections, welding spatter, oil stains, bevel variations, or low contrast, the feature extraction step of traditional algorithms is prone to failure, leading to tracking interruptions or positioning errors.

[0022] Pure deep learning methods struggle to balance accuracy and speed. Models capable of pixel-level segmentation (such as U-Net) have high computational costs, making it difficult to meet the high-frequency requirements of real-time correction for robots. While lightweight models are fast, their positioning accuracy often falls short of the sub-pixel requirements of grinding processes.

[0023] In view of this, this application provides a weld grinding correction method and related equipment based on structured light. This solution realizes the rapid identification of weld point image coordinates through the YOLO model, and then determines the real three-dimensional coordinates through coordinate transformation, realizing the organic combination of intelligent recognition and geometric measurement, while improving the accuracy, stability and real-time performance of weld grinding tracking.

[0024] The weld seam grinding and correction method based on structured light provided in this application relates to the field of computer vision technology. This method can be applied to terminals, servers, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the weld seam grinding and correction method based on structured light, but is not limited to the above forms.

[0025] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0026] Figure 1 This is an optional flowchart of the structured light-based weld grinding and correction method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S104.

[0027] Step S101: Obtain welding images; Step S102: The weld point is identified in the welding image using the trained YOLO model to obtain the coordinates of the weld point image; Step S103: Based on structured light geometric measurement, the coordinates of the weld point image are transformed to obtain the true three-dimensional coordinates of the weld point; Step S104: Calculate the deviation between the preset theoretical weld trajectory and the actual three-dimensional coordinates to obtain the positional deviation, so as to correct the grinding tool.

[0028] Steps S101 to S104 as shown in the embodiments of this application utilize the powerful feature learning and anti-interference capabilities of the YOLO (You Only Look Once, a single-stage target detection algorithm) model to achieve rapid identification of weld point image coordinates through the YOLO model. Then, through coordinate transformation, the true three-dimensional coordinates are determined, realizing the organic combination of intelligent recognition and geometric measurement. This allows the advantages of deep learning and geometric vision to complement each other. The rapid and robust target localization capability of deep learning is combined with the precise three-dimensional geometric measurement capability of structured light vision, thereby improving the accuracy, stability, and real-time performance of weld grinding tracking.

[0029] In some embodiments, the structured light-based weld grinding and correction method can be applied to a weld grinding and correction system. The hardware of this system may include, but is not limited to, a robot, grinding tools, a structured light vision sensor, and an industrial control computer (ICC).

[0030] The industrial robot's end effector is equipped with a grinding tool and a structured light vision sensor (including a line laser and an industrial camera); the line laser projects a clear laser line onto the weld seam of the workpiece on the worktable; the camera captures an image of the weld seam with the laser line; the camera is connected to the industrial control computer via a data cable; the industrial control computer is connected to the robot controller via a control cable; the data flow is: camera → industrial control computer → robot controller → robot.

[0031] The robot is used to perform the polishing task; the robot controller is used to control the robot's movement.

[0032] Grinding tools (such as belt sanders, angle grinders, etc.) are installed at the end of the robot and used to grind weld seams.

[0033] The structured light vision sensor, mounted on the robot's end flange, moves with the robot and includes: A line laser, used to project a laser stripe onto the weld seam; A camera is used to capture images of laser stripes on the weld at a specific angle. The relative positions of the camera and the laser are precisely calibrated.

[0034] The industrial control computer (ICC) is used to process welding images captured by the camera, perform coordinate calculations, and generate correction instructions. It has built-in core algorithm software, which is used to implement a weld grinding and correction method based on structured light.

[0035] The robot, carrying grinding tools and vision sensors, moves to a position above the starting point of the weld. A line laser projects a laser stripe onto the weld; due to grooves or height differences within the weld, the laser stripe is distorted. An industrial camera captures this distorted laser stripe from an oblique angle above. The captured image is transmitted in real-time to an industrial computer for processing. After calculating the precise position of the weld, the industrial computer sends a correction command to the robot controller, thereby driving the robot to adjust its posture so that the grinding tools are always precisely aligned with the weld.

[0036] In some embodiments, step S102 may include, but is not limited to, steps S201 to S205: Step S201: The welding image is recognized by the trained YOLO model to obtain the intersection image of the laser and the weld. Step S202: Extract features from the center line of the laser in the intersecting image to obtain the center line of the light stripe; Step S203: Extract features from the centerline of the weld in the intersecting images to obtain the weld centerline; Step S204: Calculate the intersection point of the light stripe centerline and the weld centerline to obtain the weld point image coordinates.

[0037] In step S201 of some embodiments, the welding image is input into a pre-trained YOLO model. The model's task is not to segment the weld or laser with pixel-level precision, but to quickly locate the "intersection area between the structured light stripes and the weld" and output one or more bounding boxes containing the area, thereby obtaining the intersection image within the bounding boxes.

[0038] In some embodiments, step S201 may include, but is not limited to, steps S211 to S212: Step S211: The intersection of the laser and the weld in the welding image is identified using the trained YOLO model to obtain the coordinates of the intersection area; Step S212: Segment the welding image according to the coordinates of the intersecting region to obtain the intersecting image.

[0039] In steps S211 to S212 of some embodiments, the powerful feature learning and anti-interference capabilities of the YOLO model enable the stable localization of the intersection area between the weld and the laser even in complex environments such as reflections and spatter. Based on the intersection area coordinates output by YOLO, a small region of interest (ROI) is cropped from the original image. This greatly reduces background interference, creating ideal conditions for subsequent precise processing.

[0040] In steps S202 and S203 of some embodiments, high-precision sub-pixel processing is performed within a clean ROI image (intersecting image). Within the ROI, a high-precision algorithm is used to extract the center line of the laser stripes, resulting in a light stripe center line composed of a series of sub-pixel points. Similarly, within the ROI, the weld center line is extracted using an image processing algorithm, taking advantage of the characteristic that welds are typically represented as a dark line or groove.

[0041] In step S204 of some embodiments, the extracted laser centerline is fitted or intersected with the weld centerline to calculate the sub-pixel image coordinates (u, v) of the intersection point, which is the weld point image coordinate.

[0042] This embodiment employs a hybrid architecture of "YOLO coarse localization + ROI fine processing." The high robustness of the YOLO model solves the problem of failure when directly processing images in complex backgrounds. The high precision of ROI fine processing addresses the issue of insufficient accuracy or high computational cost of pure deep learning models in pixel-level localization, achieving sub-pixel accuracy within clean, small areas using traditional geometric vision algorithms. Because YOLO inference is extremely fast, subsequent fine processing is also very fast due to the small data volume (ROI), ensuring the overall system meets real-time requirements. This collaborative approach simultaneously achieves robustness, accuracy, and real-time performance in weld seam tracking.

[0043] In some embodiments, the YOLO model can be trained through steps S221 to S223: Step S221: Obtain the training dataset, which includes several training images with labeled laser-weld intersection areas; Step S222: Input the training image into the YOLO model and output the bounding box coordinates; Step S223: With the goal of reducing the deviation between the bounding box coordinates and the true coordinates, the parameters of the YOLO model are adjusted to obtain the trained YOLO model.

[0044] In steps S221 to S223 of some embodiments, the training dataset includes a large number of images collected under different working conditions (different weld types, different lighting, with reflection / spatter) for training. During annotation, a bounding box is drawn in the area where the laser intersects with the weld and labeled as the "intersection area". This allows the model to learn to ignore complex backgrounds and focus on finding the visual features where the intersection exists. By adjusting the YOLO model parameters, the deviation between the bounding box coordinates output by the YOLO model and the true coordinates of the actual intersection area in the validation dataset is reduced, thereby improving the accuracy of the YOLO model's localization.

[0045] In some embodiments, step S103 may include, but is not limited to, steps S301 to S303: Step S301: Determine the ray originating from the camera optical center based on the camera intrinsic parameters and the coordinates of the weld point image; Step S302: Calculate the intersection point of the ray and the preset laser plane of the structured light to obtain the first three-dimensional coordinates in the camera coordinate system; Step S303: The first three-dimensional coordinates are transformed according to the preset calibration matrix of the structured light to obtain the true three-dimensional coordinates.

[0046] In steps S301 to S303 of some embodiments, based on the principle of triangulation, according to the weld point image coordinates (u, v) obtained in step S204, combined with the camera intrinsic parameters, a ray originating from the camera optical center can be determined. The ray intersects with the preset light plane equation, and the intersection point is the first three-dimensional coordinate P_cam(X, Y, Z) of the intersection point in the camera coordinate system. Using the hand-eye calibration matrix T_cam_tool and the end-effector pose fed back by the robot in real time, P_cam is transformed into coordinates (true three-dimensional coordinates) P_base in the robot base coordinate system.

[0047] Hand-eye calibration: Determines the fixed transformation relationship T_cam_tool between the camera coordinate system and the robot end-effector coordinate system.

[0048] Light plane calibration: Determine the precise mathematical equation of the laser plane projected by the laser in the camera coordinate system: A*X+B*Y+C*Z+D=0 (where (X, Y, Z) are points in the camera coordinate system).

[0049] In some embodiments, step S104 may include, but is not limited to, steps S401 to S402: Step S401: Subtract the coordinates of the actual three-dimensional coordinates from the coordinates of the theoretical weld trajectory to obtain the positional deviation; Step S402: Control the movement of the grinding tool according to the positional deviation in order to track the weld.

[0050] In steps S401 to S402 of some embodiments, the weld point P_base measured in real time is compared with the theoretical weld trajectory P_theoretical preset in the robot program to obtain the position deviation ΔP = P_base - P_theoretical. Based on this deviation ΔP, the robot controller corrects the motion trajectory of the robot end effector in real time through the trajectory planner, driving the grinding tool to accurately track the actual direction of the weld.

[0051] This embodiment provides a hybrid vision measurement architecture that organically combines the intelligent recognition of deep learning with the precise geometric measurement of traditional vision, forming a new paradigm for weld seam tracking that combines "intelligence" and "precision".

[0052] In some embodiments, please refer to Figure 2 The structured light-based weld grinding and correction method may include the following steps: Step 1: System Calibration (Offline Preparation). This is fundamental to improving measurement accuracy. It mainly includes: Hand-eye calibration: Determines the fixed transformation relationship T_cam_tool between the camera coordinate system and the robot end-effector coordinate system.

[0053] Light plane calibration: Determine the precise mathematical equation of the laser plane projected by the laser in the camera coordinate system: A*X+B*Y+C*Z+D=0.

[0054] Step 2: Fast coarse localization of intersection regions based on YOLO (online real-time). For each frame of image captured by the camera: YOLO Inference: The image is input into a pre-trained YOLO model. The model's task is not to segment the weld or laser with pixel-level precision, but to quickly locate the "intersection area between structured light stripes and the weld" and output one or more bounding boxes containing that area. ROI cropping: Based on the bounding box coordinates output by YOLO, a small region of interest is cropped from the original image. This greatly reduces background interference, creating ideal conditions for subsequent accurate processing.

[0055] Step 3: Precise feature extraction and intersection calculation within the ROI (online real-time). High-precision sub-pixel level processing is performed within a clean ROI image: Laser stripe center extraction: Within the ROI, a high-precision algorithm is used to extract the center line of the laser stripe, resulting in a light stripe center line composed of a series of sub-pixel level points; Weld feature center extraction: Also within the ROI, the center line of the weld is extracted by image processing algorithms, taking advantage of the fact that the weld usually appears as a dark line or groove. Intersection calculation: The extracted laser stripe center line is fitted or intersected with the weld center line to calculate the subpixel image coordinates (u, v) of the intersection point.

[0056] Step 4: 3D coordinate calculation and trajectory correction (online real-time); 3D Reconstruction: Based on the intersection point image coordinates (u, v) obtained in step three, and combined with the camera intrinsic parameters, a ray originating from the camera's optical center can be determined. This ray intersects the light plane equation calibrated in step one, and the intersection point is the 3D coordinate P_cam(X, Y, Z) of that intersection point in the camera coordinate system. This is based on the principle of triangulation.

[0057] Coordinate transformation: Using the hand-eye calibration matrix T_cam_tool and the robot's real-time feedback end pose, P_cam is transformed into coordinates P_base in the robot's base coordinate system; Trajectory correction: The position deviation ΔP = P_base - P_theoretical is obtained by comparing the real-time measured weld point P_base with the preset theoretical weld trajectory in the robot program. Based on this deviation ΔP, the robot controller uses a trajectory planner to correct the motion trajectory of the robot's end effector in real time, driving the grinding tool to accurately track the actual direction of the weld.

[0058] This application's embodiments utilize the powerful feature learning and anti-interference capabilities of the YOLO model, enabling stable localization of the intersection area between the weld seam and structured light even in complex environments such as reflection and spatter, overcoming the shortcomings of traditional methods that are prone to failure.

[0059] This application embodiment uses the YOLO model to quickly locate the region of interest and concentrates computing resources on sub-pixel-level precise extraction within a small area, thereby achieving high-precision measurement while meeting the high-frequency response requirements of real-time robot control.

[0060] This embodiment constructs a hybrid vision measurement architecture that organically combines the intelligent recognition of deep learning with the precise geometric measurement of traditional vision, forming a new paradigm for weld seam tracking that combines "intelligence" and "precision".

[0061] The weld seam grinding and correction method based on structured light provided in this application can be applied to the following products: Intelligent weld grinding vision system (core component): A standalone, standard product that can be integrated into existing robots or special-purpose machines. It includes an explosion-proof industrial camera, a line laser, a dedicated light source, and a vision controller with built-in dedicated algorithm software (including a YOLO model). It can act as the machine's "eyes" and "brain," outputting high-precision 3D coordinates of weld seams and correction commands.

[0062] Intelligent weld seam grinding robot workstation (integrated solution): A complete turnkey project. It includes the industrial robot body, grinding tools (belt sander, floating grinding head, etc.), the aforementioned intelligent vision system, safety protection, and central control system, realizing full automation of "material arrival-identification-tracking-grinding-unloading".

[0063] Portable intelligent polishing device: For on-site grinding of large, immovable workpieces (such as ship hulls and large storage tanks), develop portable devices integrated on robotic arms or guide rails. Workers can hold or hoist the device to the vicinity of the weld, and the system will automatically guide it to complete precise grinding.

[0064] The structured light-based weld grinding and correction method in this embodiment can be used to solve the automation challenges of post-welding processes, and is mainly applied in the following scenarios: Scenario 1: Steel structure industry - grinding of weld seams on large components; Grinding long welds in large steel structures such as bridges, building steel structures, and wind turbine towers. Due to the long welds and large workpieces, ensuring accurate positioning is challenging. This system automatically identifies and tracks the winding welds, correcting deviations in real time to prevent grinding tools from hitting the workpiece or causing insufficient grinding, thus replacing manual labor in high-intensity, high-dust environments.

[0065] Scenario 2: Shipbuilding and Marine Engineering - Grinding of Hull Welds; Grinding of welded joints in ship hull plates. The workpieces exhibit significant curvature variations, long welds, and harsh working environments. The YOLO model can adapt to different steel plate reflectivity and weld types, enabling automated robotic grinding and significantly improving the automation level of shipbuilding.

[0066] Scenario 3: Pressure vessel and pipeline welding - circumferential / longitudinal seam grinding; Grinding of the inner and outer circumferential welds on pressure vessels such as liquefied natural gas (LNG) storage tanks and chemical tanks. Extremely high requirements are placed on grinding uniformity and surface quality. High-precision intersection detection enables the robot to move precisely along the weld center, ensuring uniform wall thickness after grinding to meet pressure requirements. This is a crucial step in quality control and safe production.

[0067] Scenario 4: Aerospace - Grinding of weld seams on engine components / aircraft skin; Precision grinding of weld seams for repairing aero-engine blades and splicing weld seams for aircraft skin. The materials are mostly high-temperature alloys, the weld seams are narrow, and the precision requirements are extremely high.

[0068] Subpixel-level intersection positioning and real-time correction can meet the stringent quality requirements of the aerospace industry and avoid the instability of manual operation.

[0069] Scenario 5: Rail Transit - Grinding of weld seams in high-speed rail / subway carriages; Grinding and brushing of weld seams in aluminum alloy or stainless steel car bodies. The surface must be smooth and aesthetically pleasing without damaging the base material. Automated grinding ensures consistent weld seam appearance across all car bodies, improving product quality and production speed.

[0070] In some embodiments, the weld grinding correction method extends the recognition capabilities of the YOLO model, enabling it to simultaneously track the following surface features: weld geometry (groove edges, height differences), material reflectivity (reflective area identification), and surface contamination features (spatters, oil stains). For different surface conditions, the system dynamically adjusts the confidence weights of various features: increasing the weight of geometric features under strong reflective conditions; enhancing the discrimination ability of material reflectivity features when spatter contamination is present; and maintaining balanced fusion of multiple features under normal conditions. Through surface feature tracking, the system implemented using the weld grinding correction method can maintain stable operation under the following extreme conditions: when the weld is partially covered by spatter, the weld direction is inferred from surrounding features; when strong reflectivity causes laser stripe breakage, path prediction is performed using material features; and when oil stains cause a decrease in contrast, auxiliary identification is performed by combining geometric features. The system has online learning capabilities, enabling it to: optimize feature weight allocation based on historical data; automatically identify new surface anomaly patterns; and dynamically adjust processing parameters to adapt to environmental changes.

[0071] The surface feature tracking technology in the weld grinding and correction method provided in this application can be further extended to: online welding quality detection, automatic surface defect identification, three-dimensional morphology reconstruction, and automatic quality inspection of industrial products. The introduction of surface feature tracking technology significantly improves adaptability and reliability in complex industrial environments, providing stronger visual perception capabilities for industrial automation.

[0072] This application embodiment also provides a weld grinding and correction device that can implement the above method. The device includes: The acquisition module is used to acquire welding images; The recognition module is used to identify weld points in welding images using a trained YOLO model, and obtain the coordinates of the weld points in the image. The calculation module is used to perform coordinate transformation processing on the coordinates of the weld point image based on structured light geometric measurement to obtain the true three-dimensional coordinates of the weld point; The deviation correction module is used to calculate the deviation between the preset theoretical weld trajectory and the actual three-dimensional coordinates to obtain the positional deviation, so as to correct the grinding tool.

[0073] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0074] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0075] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0076] Please see Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application. The 903 input / output interface is used to implement information input and output. The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0077] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0078] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0079] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0080] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0081] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0082] This application provides a method, apparatus, electronic device, storage medium, and program product for correcting weld grinding based on structured light, achieving hybrid visual correction based on the intersection detection of YOLO and structured light. This embodiment includes at least the following beneficial effects: 1. Significantly improves robustness and reliability in complex industrial scenarios: Initial localization was performed using the YOLO deep learning model. Trained on a large amount of data, the YOLO model possesses powerful feature learning and generalization capabilities, enabling it to intuitively perceive the semantic concept of "weld seam intersecting with laser light," much like the human eye, rather than relying solely on image grayscale or gradient features. Therefore, even under complex conditions, the model can reliably locate the approximate area of ​​the intersection, improving the system's continuous and reliable operation.

[0083] 2. Successfully achieved a balance between high precision and high real-time performance: It adopts a hybrid architecture of "division of labor and cooperation". Let YOLO do what it does best: "coarse localization"; as a single-stage detector, YOLO has an extremely fast inference speed and can quickly lock key regions from the whole map; This allows traditional algorithms to perform precise measurements on a small scale: within a small region of interest provided by YOLO, which has eliminated most interference, traditional sub-pixel precision algorithms such as the gray-scale centroid method and the Steger algorithm are used for feature extraction. Because the processing area is small, the computational load is greatly reduced, resulting in extremely fast speed, while still leveraging the ultra-high precision advantages of these algorithms in local measurements. This architecture enables the system to achieve a closed-loop control frequency of up to tens of hertz while maintaining subpixel-level measurement accuracy, perfectly balancing the contradictory requirements of accuracy and speed.

[0084] 3. Significantly enhances the system's adaptability and versatility for different weld seams: In this embodiment, adaptability is embedded within the YOLO model. By learning from training data containing various weld types, the YOLO model can learn to recognize different weld morphologies. When it needs to adapt to a new weld type, only incremental learning or retraining of the model is required, without changing the core algorithm flow. This gives the system strong generalization capabilities, enabling rapid deployment to new production lines and reducing debugging costs and barriers to entry.

[0085] 4. Optimize the allocation of computing resources to improve the overall efficiency and economy of the system: Limiting time-consuming, precise calculations to a small region of interest significantly reduces the consumption of computing resources, especially CPUs and GPUs. This means that the entire system can be deployed using lower-cost, lower-power industrial computers or embedded devices, helping to reduce hardware costs and making the system more stable and easier to integrate into existing robot controllers.

[0086] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0087] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0088] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0090] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0091] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0093] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0095] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for correcting weld seam grinding based on structured light, characterized in that, The method includes the following steps: Acquire welding images; The weld point coordinates are obtained by identifying weld points in the welding image using a trained YOLO model. The coordinates of the weld point image are transformed using geometric measurement based on structured light to obtain the true three-dimensional coordinates of the weld point. The deviation between the preset theoretical weld trajectory and the actual three-dimensional coordinates is calculated to obtain the positional deviation, which is then used to correct the grinding tool.

2. The method according to claim 1, characterized in that, The step of identifying weld points in the welding image using a trained YOLO model to obtain weld point image coordinates includes: The welding image is subjected to image recognition using a trained YOLO model to obtain the intersection image of the laser and the weld. Feature extraction is performed on the center line of the laser in the intersecting image to obtain the center line of the light stripe; The centerline of the weld in the intersecting image is obtained by feature extraction. The intersection point of the light stripe centerline and the weld centerline is calculated to obtain the image coordinates of the weld point.

3. The method according to claim 2, characterized in that, The step of performing image recognition on the laser and weld seam of the welding image using a trained YOLO model to obtain the intersection image of the laser and weld seam includes: The intersection of the laser and the weld in the welding image is identified using a trained YOLO model to obtain the coordinates of the intersection area. The welding image is segmented based on the coordinates of the intersecting region to obtain the intersecting image.

4. The method according to claim 1, characterized in that, The trained YOLO model is obtained through the following steps: Obtain a training dataset, which includes several training images with labeled areas where the laser intersects with the weld. The training image is input into the YOLO model, and the bounding box coordinates are output. With the goal of reducing the deviation between the bounding box coordinates and the true coordinates, the parameters of the YOLO model are adjusted to obtain the trained YOLO model.

5. The method according to claim 1, characterized in that, The structured light-based geometric measurement performs coordinate transformation on the weld point image coordinates to obtain the true three-dimensional coordinates of the weld point, including: Based on the camera's intrinsic parameters and the image coordinates of the weld seam, determine the ray originating from the camera's optical center; The intersection point of the ray and the preset laser plane of the structured light is calculated to obtain the first three-dimensional coordinates in the camera coordinate system; The first three-dimensional coordinates are transformed according to the preset calibration matrix of the structured light to obtain the true three-dimensional coordinates.

6. The method according to claim 1, characterized in that, The step of calculating the deviation between the preset theoretical weld trajectory and the actual three-dimensional coordinates to obtain the positional deviation, and then correcting the grinding tool, includes: The positional deviation is obtained by subtracting the actual three-dimensional coordinates from the coordinates of the theoretical weld trajectory. The movement of the grinding tool is controlled based on the positional deviation in order to track the weld.

7. A weld seam grinding and correction device, characterized in that, The device includes: The acquisition module is used to acquire welding images; The recognition module is used to identify weld points in the welding image using a trained YOLO model, and obtain the coordinates of the weld point image. The calculation module is used to perform coordinate transformation processing on the coordinates of the weld point image based on structured light geometric measurement to obtain the true three-dimensional coordinates of the weld point; The deviation correction module is used to calculate the deviation between the preset theoretical weld trajectory and the actual three-dimensional coordinates to obtain the positional deviation, so as to correct the deviation of the grinding tool.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images