A dynamic repair welding method and system based on welding quality evaluation

Abstract

The application provides a dynamic repair welding method and system based on welding quality evaluation, and relates to the technical field of data processing.The method comprises the following steps: obtaining three-dimensional point cloud data and image data of a welded area; determining the welding quality of the welded area by a welding quality evaluation model based on deep learning according to the three-dimensional point cloud data and the image data; judging whether the welding quality is qualified; if yes, repair welding is not needed; otherwise, the next step is entered; updating a three-dimensional topological map; performing global repair welding path planning in the three-dimensional topological map by an RRT* algorithm; performing local optimization on the global repair welding path by a gradient ascent algorithm to generate a final repair welding path; performing repair welding according to the final repair welding path, and returning to continue monitoring.The application can adaptively generate a repair welding path according to the change of welding quality in a real-time welding process, dynamically repair welding defects, and effectively improve the degree of welding automation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a dynamic repair welding method and system based on welding quality evaluation. BACKGROUND

[0002] With the rapid development of automation technology, automated welding processes have been widely applied in various manufacturing industries. Modern automated welding systems are usually equipped with various sensors (such as LiDAR, CCD camera, laser radar, etc.) and use computer vision and deep learning techniques to evaluate welding quality. However, even with these technologies, defects such as missed welding and false welding may still occur during the welding process, affecting the quality of the final product. Therefore, how to timely detect and repair these defects during the welding process has become a key issue to improve welding quality and efficiency.

[0003] Traditional welding path planning methods mainly rely on static path calculation. Although these methods can provide basic path planning, they usually ignore quality problems that may occur during the welding process, such as defects, insufficient weld coverage, etc. For these welding defects, manual inspection is often required before repair welding, which seriously affects the improvement of welding automation. SUMMARY

[0004] In view of the above deficiencies of the prior art, the purpose of the embodiments of the present application is to provide a dynamic repair welding method based on welding quality evaluation, which can solve the technical problem that the prior art requires manual inspection before repair welding for welding defects in the welding process, which seriously affects the improvement of welding automation.

[0005] The first aspect of the embodiments of the present application proposes a dynamic repair welding method based on welding quality evaluation, comprising:

[0006] S1: obtaining three-dimensional point cloud data and image data of the welded area;

[0007] S2: determining the welding quality of the welded area according to the three-dimensional point cloud data and image data through a deep learning-based welding quality evaluation model;

[0008] S3: determining whether the welding quality is qualified; if yes, no repair welding is needed; otherwise, the next step is entered;

[0009] S4: updating the three-dimensional topological map based on the three-dimensional point cloud data and image data;

[0010] S5: performing global repair welding path planning in the three-dimensional topological map through the RRT* algorithm;

[0011] S6: performing local optimization on the global repair welding path through the gradient ascent algorithm to generate the final repair welding path;

[0012] S7: According to the final repair welding path, repair welding is performed, and S1 is returned to continue monitoring.

[0013] The second aspect of the embodiment of the present application provides a dynamic repair welding system based on welding quality evaluation, comprising a processor and a memory.

[0014] The memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the dynamic repair welding method based on welding quality evaluation as described in the first aspect.

[0015] The third aspect of the embodiment of the present application provides a readable storage medium, and the readable storage medium stores programs or instructions, and the programs or instructions are executed by the processor to implement the steps of the dynamic repair welding method based on welding quality evaluation as described in the first aspect.

[0016] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0017] In the embodiment of the present application, the repair welding path can be adaptively generated according to the change of the welding quality in the real-time welding process, the welding defects are dynamically repaired, and the degree of automation of welding is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings are only for the purpose of illustrating specific embodiments and are not considered as limiting the present application. Throughout the drawings, the same reference signs represent the same components. Obviously, the accompanying drawings described below are only some embodiments described in the embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0019] Figure 1 is a flow diagram of a dynamic repair welding method based on welding quality evaluation provided by the embodiment of the present application.

[0020] Figure 2 is a structural diagram of a dynamic repair welding system based on welding quality evaluation provided by the embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the personnel in the art better understand the technical solutions in the embodiments of the present application, the technical solutions of the present application will be described clearly and completely in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the scope of protection of the present application.

[0022] The welding quality evaluation-based dynamic repair welding method provided by the embodiments of the present application will be described in detail below in conjunction with the drawings, specific embodiments and application scenarios.

[0023] Referring to the drawings accompanying the specification Figure 1 , a flowchart of a welding quality evaluation-based dynamic repair welding method provided by the embodiments of the present application is shown.

[0024] The welding quality evaluation-based dynamic repair welding method provided by the embodiments of the present application can include the following steps:

[0025] S1: Obtain three-dimensional point cloud data and image data of a welded area.

[0026] S2: According to the three-dimensional point cloud data and the image data, determine the welding quality of the welded area by a welding quality evaluation model based on deep learning.

[0027] In a possible implementation, S2 specifically includes sub-steps S201 to S204:

[0028] 201: In the point cloud branch of the welding quality evaluation model, extract point cloud features of the three-dimensional point cloud data by PointNet+.

[0029] PointNet+ is a deep learning architecture designed for point cloud data (such as 3D scanning or LiDAR data). It is an improved version of PointNet, mainly used for processing unordered point cloud data. In PointNet+, multi-level local feature learning is introduced to enable the model to better capture the local geometric structure of the point cloud data.

[0030] S202: In the image branch of the welding quality evaluation model, extract image features of the image data by a feature pyramid network.

[0031] The feature pyramid network (FPN) is a deep learning architecture widely used in computer vision tasks. The core idea of FPN is to use a pyramid structure to process features of different scales, and to enhance the network's ability to recognize multi-scale objects through top-down feature fusion.

[0032] S203: By using a cross-modal attention fusion mechanism, feature fusion is performed on point cloud features and image features to obtain fused features.

[0033] Specifically, through a cross-modal attention mechanism, the system can automatically learn how to perform weighted fusion based on the importance of image features and point cloud features. In this mechanism, image features act as queries, and point cloud features act as keys and values. The model dynamically adjusts the fusion degree of each modal feature through the attention mechanism. This approach not only improves the efficiency of multimodal information utilization but also enables the welding quality assessment model to more accurately identify welding defects in complex scenarios.

[0034] S204: Based on the fusion characteristics, determine the welding quality results of the welded area. The welding quality results include multiple sub-task results: quality score, defect area, repairability, and weld centerline.

[0035] In one possible implementation, the loss function used to train the welding quality assessment model is specifically obtained by superimposing the loss functions of each sub-task:

[0036] ;

[0037] in, L Represents the loss function. L k Indicates the first k The loss function for each subtask includes quality score, defect area, repairability, and weld centerline. σ k Indicates the first k The noise scale parameter for each subtask, where log represents the logarithmic function. This represents a regularization term used to control the magnitude of the noise scale parameter and prevent the noise scale from becoming too large.

[0038] In this embodiment of the invention, by superimposing the loss functions of multiple subtasks and introducing a noise scale parameter, the weight of each subtask can be dynamically adjusted during training, thereby achieving adaptive learning. Specifically, the loss of each subtask (such as quality score, defect area, repairability, weld centerline) has a corresponding noise scale, reflecting the stability and uncertainty of the task. During training, subtasks with smaller noise scales are assigned higher weights and prioritized for optimization, while subtasks with larger noise scales have their impact on the overall loss relatively reduced.

[0039] In addition, by introducing a regularization term, it is possible to avoid excessive noise scales, thereby preventing the model from overfitting or instability on certain sub-tasks. The advantage of this method is that it can balance the learning of multiple tasks and ensure reasonable weight distribution among multiple objectives, thereby improving overall performance and robustness.

[0040] S3: Determine whether the welding quality is qualified. If yes, no repair welding is needed. Otherwise, go to the next step.

[0041] S4: Update the three-dimensional topological map based on the three-dimensional point cloud data and image data.

[0042] In one possible implementation, S4 specifically includes sub-steps S401 to S410:

[0043] S401: Calibrate the camera and laser sensor.

[0044] It should be noted that through accurate calibration, the three-dimensional point cloud data of the laser sensor and the two-dimensional data of the camera image can be accurately matched, thereby providing reliable data support for subsequent three-dimensional reconstruction and path planning. Calibration can eliminate errors between sensors and ensure the fusion accuracy of multi-modal data.

[0045] S402: Time synchronization of three-dimensional point cloud data and image data.

[0046] It should be noted that time synchronization ensures that different sensors (such as LiDAR and camera) acquire data at the same time point. This is very important for real-time updating of environmental information, because the time difference between different sensors may cause data inconsistency, which in turn affects subsequent path planning and quality evaluation. Synchronized data can be jointly processed within the same time window, improving the accuracy and consistency of the overall data.

[0047] S403: Motion distortion correction of three-dimensional point cloud data.

[0048] It should be noted that in a robot or sensor moving at high speed, the point cloud data collected by the sensor may be affected by motion distortion due to the high speed of movement, resulting in inaccurate data. Through motion distortion correction, errors caused by motion can be eliminated, ensuring the accuracy and reliability of the point cloud data, making subsequent path planning, collision detection and three-dimensional reconstruction more accurate.

[0049] S404: Use the odometry method to register the three-dimensional point cloud data of adjacent frames.

[0050] It should be noted that the odometry algorithm (such as LOAM or WICP) is used to register consecutive point cloud frames, i.e. to calculate the relative pose (position and attitude) between adjacent frames. The purpose of this step is to align point cloud data from different time points to the same coordinate system, providing accurate spatial information for subsequent three-dimensional mapping and path planning. The registration process helps to generate a seamless and continuous environment map, avoiding inconsistencies and errors in the data.

[0051] S405: Use pose graph optimization algorithm to optimize key frames of image data.

[0052] It should be noted that the pose graph optimization algorithm eliminates the accumulated pose error by optimizing the positional relationship of key frames. This helps to optimize the pose of the camera (i.e. the attitude and position of the camera) in the image data, improving the global consistency and accuracy of the environment map. Through key frame optimization, high-quality and low-error image data can be ensured, thereby improving the accuracy of subsequent path planning and quality assessment.

[0053] S406: Obtain new observations based on the registered three-dimensional point cloud data and the image data optimized by key frames.

[0054] It should be noted that by fusing the optimized data of different sensors, a more accurate environment representation can be obtained, providing more accurate input data for path planning, quality assessment and dynamic repair, improving the reliability and execution effect of the overall system.

[0055] S407: Use clipping function to limit new observations for each voxel:

[0056] ;

[0057] wherein, f m represents the clipped new observation, clip represents the clipping function, z represents the measured depth, z s represents the surface depth, μ represents the truncation distance.

[0058] It should be noted that the clipping function limits the measured values (such as depth values) within a reasonable range, avoiding inaccurate data due to noise or outliers affecting subsequent processing. The clipping function ensures that the distance value of each voxel does not exceed the pre-set maximum or minimum range, thereby improving the stability and consistency of the data, and helping to generate a more accurate environment map.

[0059] S408: Assign distance values to each voxel using TSDF fusion:

[0060] ;

[0061] wherein, represents the fused TSDF value, F represents the pre-fused TSDF value, w represents the weight of the pre-fused TSDF value, w m represents the weight of the new observation.

[0062] It should be noted that the TSDF fusion process weights the data from different time points and different sensors to derive the final distance value for each voxel. Through this weighted fusion, historical data and new observation data can be smoothly combined to obtain a more accurate environment model. This helps to reduce noise interference and improve the accuracy of the environment map, providing reliable distance field data for path planning and collision detection.

[0063] S409: Form a TSDF voxel map according to the distance values of each voxel.

[0064] It should be noted that by collecting the distance values of each voxel into a complete TSDF voxel map, a three-dimensional representation of the global environment can be formed. This map can provide detailed information about obstacles, surfaces, and open areas, providing important data support for tasks such as path planning and collision detection. The TSDF map can also reflect the shape and position of obstacles, ensuring collision avoidance during path planning.

[0065] S410: Use the Victor fast sweeping method to solve the ESDF voxel map from the TSDF voxel map.

[0066] It should be noted that the Victor fast sweeping method converts the TSDF map to an ESDF (Euclidean Signed Distance Field) to provide the Euclidean distance to the nearest obstacle for each voxel. ESDF can provide more accurate obstacle information and can efficiently support collision detection and path optimization. By using ESDF, path planning can be more accurate, ensuring that the robot or robotic arm can avoid obstacles and operate according to the optimal path.

[0067] In embodiments of the present application, by fusing data from different sensors (such as LiDAR and camera), the limitations of a single sensor in different scenarios can be compensated for, thereby improving the accuracy of environmental perception. The updated three-dimensional topological map can reflect changes in the environment in real time, providing more accurate obstacle information, spatial structure, and reachable regions.

[0068] S5: Perform global repair path planning in the three-dimensional topological map using the RRT* algorithm.

[0069] wherein the RRT* algorithm is a path planning algorithm based on random sampling, widely used in complex high-dimensional space path planning, especially suitable for dynamic environment and dense obstacle conditions. The algorithm constructs a tree structure (Random Tree), randomly samples points in the space, and expands the tree structure until it is connected to the target point.

[0070] In one possible implementation, S5 specifically includes sub-steps S501 to S505:

[0071] S501: According to the updated ESDF voxel map, determine the obstacle information, reachable area and topological structure of each welding process section.

[0072] It should be noted that by using the updated ESDF voxel map, the obstacle information and reachable area in the environment can be accurately obtained. The ESDF map provides the distance from each point to the nearest obstacle, so that the path planning can avoid obstacles while ensuring that the path is within the reachable area. The introduction of topological structure can effectively divide the welding area into multiple process sections, providing clear spatial partition and organizational structure for subsequent repair welding path planning, ensuring the accuracy and reliability of the welding path.

[0073] S502: According to the starting point and ending point of the repair welding task, initialize the starting and target positions of the RRT* algorithm, and determine the basic parameters of path planning.

[0074] It should be noted that by initializing the starting and target positions of the RRT* algorithm, the starting point and ending point of the repair welding task are determined, which helps the path planning algorithm to effectively sample and search within a clear target range. At the same time, determining the basic parameters of path planning (such as step size, sampling strategy, etc.) can ensure the efficient operation of the algorithm, avoid redundant calculation and unnecessary path search, and improve the efficiency and accuracy of path planning.

[0075] S503: In the three-dimensional topological map, RRT* sampling is performed, and multiple points are sampled in the map by expanding the tree structure, gradually approaching the target area.

[0076] It should be noted that the RRT* algorithm expands the tree structure to sample in the three-dimensional topological map, gradually approaching the target area, which can quickly explore the entire search space. RRT* finds the optimal path between the starting point and the ending point by gradually sampling and optimizing, with good exploration and global optimization capabilities, and can generate a feasible path in a complex three-dimensional environment and continuously optimize the connectivity and efficiency of the path.

[0077] S504: Evaluate the generated path using the cost function.

[0078] Optionally, the cost function includes a trajectory length term, a safety term, and a smoothness term:

[0079] ;

[0080] wherein, C RRT* represents the cost function of RRT *, l represents the trajectory length, represents the safety penalty function, represents the distance from the current trajectory point to the nearest obstacle, λ c represents the weight coefficient of the safety term, represents the curvature, λ b represents the weight coefficient of the smoothness term, d represents the differential symbol, s varies within the interval [0, 1], representing different positions on the path.

[0081] wherein the size of the weight coefficient of the safety term and the weight coefficient of the smoothness term can be set by a person skilled in the art according to actual conditions, and the present application is not limited.

[0082] It should be noted that by introducing the cost function, the quality of the generated path can be comprehensively evaluated. The path length term ensures that the path is as short as possible, reducing redundant motion. The safety term (through the safety penalty function) ensures that the path does not collide with obstacles, ensuring the safety of the path. The smoothness term (through the curvature) controls the smoothness of the path, avoiding excessive curvature or sharp turns in the path, ensuring smoother movement of the robot arm. Through the evaluation of these comprehensive factors, the generated path will be both efficient and safe, meeting the requirements of the welding process.

[0083] S505: Determine the global repair welding path through multiple iterations of optimization.

[0084] In the embodiments of the present application, RRT* can efficiently explore feasible paths in a complex three-dimensional environment and gradually optimize path quality. Using obstacle information and reachable areas in the three-dimensional topological map, RRT* can generate an optimal path from the starting point to the ending point, while ensuring that the path avoids all obstacles and meets the requirements of the welding process (such as path smoothness, safety, etc.). Through multiple iterations and optimization, the algorithm can continuously improve the path, reduce the path length, and improve the smoothness of the path, ultimately generating an efficient and safe repair welding path that provides precise and reliable path planning support for welding tasks.

[0085] S6: Perform local optimization on the global repair welding path through a gradient ascent algorithm to generate a final repair welding path.

[0086] where the gradient ascent algorithm is an iterative algorithm commonly used in optimization problems, mainly used to find the maximum value of the objective function. The basic idea is to start from the initial point, and gradually update the parameters along the gradient direction of the objective function (i.e. the maximum change direction of the objective function). At each iteration, the gradient ascent algorithm adjusts the position of the current solution according to the gradient information of the objective function, moving in the direction of increasing function value, until reaching the local maximum or global maximum.

[0087] In one possible implementation, S6 specifically includes sub-steps S601-S606:

[0088] S601: Parameterize the global repair welding path as a B-spline spline path:

[0089] ;

[0090] where, C represents the trajectory representation function, s varies within the interval [0, 1], representing different positions on the path, P k represents the k th trajectory control point, B k represents the B-spline basis function of the k th trajectory control point, which determines the influence of each control point on the path, K represents the total number of control points.

[0091] It should be noted that parameterizing the path as a B-spline spline path helps to smoothly represent the path between control points, avoiding discontinuous or abrupt transitions in the path. B-spline can smoothly change the shape of the path by adjusting the control points, so that the path not only meets the welding requirements, but also is easy to optimize and constrain (such as curvature, speed). This smooth path representation can improve the stability and accuracy of the welding robot when performing tasks, ensuring smooth and efficient welding process.

[0092] S602: Calculate the gradient of each control point according to the objective function:

[0093] ;

[0094] where, represents the first derivative of the B-spline basis function of the k th trajectory control point, represents the second derivative of the B-spline basis function of the k th trajectory control point, represents the second derivative of the trajectory representation function.

[0095] It's important to note that calculating the gradient at each control point accurately guides the direction of path optimization. The objective function includes multiple optimization terms (such as quality, trajectory length, smoothness, and safety), and gradient calculation effectively measures the direction of path improvement at each control point. In this way, the optimization algorithm can gradually adjust the control points based on feedback from the objective function, causing the path to converge towards the optimal solution, thus achieving efficient path optimization.

[0096] Optionally, the objective function includes a quality term, a trajectory length term, a smoothing term, and a safety term:

[0097] ;

[0098] in, J Describe the objective function. P Indicates the path to be evaluated. , q The quality assessment score is derived from the welding quality assessment model. The locus represents the first derivative of the function. Indicates curvature. This represents the safety penalty function. This represents the distance from the current trajectory point to the nearest obstacle. d Represents the differential symbol. Indicates the quality item. Indicates the trajectory length term. Indicates the smoothing term. Indicates a safety item. α This represents the weighting coefficient of the trajectory length term. β The weighting coefficients of the smoothing term are represented. γ This represents the weighting coefficient of the safety item.

[0099] Those skilled in the art can set the weight coefficients of the trajectory length term, the smoothness term, and the safety term according to the actual situation; this invention does not impose any limitations.

[0100] It should be noted that by decomposing the objective function into quality, trajectory length, smoothness, and safety terms, and assigning different weight coefficients to each term, multiple optimization objectives of the path can be comprehensively considered. In welding path planning, the quality term ensures that the path covers important repair areas; the trajectory length term minimizes path redundancy, reducing movement time and energy consumption; the smoothness term controls the smoothness of the path, avoiding excessive bending and ensuring the accuracy and stability of the welding process; and the safety term, through a safety penalty function, ensures that the path maintains a safe distance from obstacles to avoid collisions. In this way, the various objectives can be balanced, improving the efficiency, safety, and quality of the path in accordance with the welding process, ultimately optimizing the execution effect of the entire welding process.

[0101] S603: Local optimization is performed using the gradient ascent algorithm, and path constraints are considered during the gradient ascent process:

[0102] ;

[0103] wherein, represents the i-th trajectory control point at the j-th iteration, t represents the i-th trajectory control point at the j-th iteration, k represents the i-th trajectory control point at the j-th iteration, represents the i-th trajectory control point at the j-th iteration, t represents the i-th trajectory control point at the j-th iteration, k represents the i-th trajectory control point at the j-th iteration, represents the constraint projection operation, which is used to ensure that the control points still satisfy the curvature, speed, and boundary conditions of the path after updating, P t represents the i-th trajectory control point at the j-th iteration, t represents the learning rate. η

[0104] It should be noted that local optimization using the gradient ascent algorithm can ensure that the path continuously approaches the optimal solution of the objective function in each iteration. By considering path constraints (such as pose, speed, and acceleration limits of the welding gun), the feasibility and safety of the path can be ensured during optimization. The constraint projection operation ensures that the control points still satisfy the process requirements, such as curvature and speed limits, after updating, thereby avoiding paths that do not meet the actual needs of the welding operation.

[0105] Optionally, the path constraints include pose constraints of the welding gun, speed and acceleration limits of the path.

[0106] S604: Collision detection is performed on the path to ensure that the optimized path does not collide with any obstacles.

[0107] It should be noted that collision detection is a key step to ensure path safety. By performing collision detection on the path after each optimization, collisions between the path and obstacles in the environment can be detected and avoided in a timely manner, ensuring the safety and feasibility of the path. Collision detection can effectively prevent unnecessary risks during welding and ensure the safe operation of the welding robot in complex environments.

[0108] S605: Optimization until convergence, generating the final repair welding path.

[0109] ​In the embodiments of the present application, the global repair welding path is locally optimized by the gradient ascent algorithm, which can finely adjust the control points of the path to further improve the path based on meeting all constraint conditions. The gradient ascent algorithm can gradually optimize the smoothness, length and safety of the path under the guidance of the gradient of the objective function, ensuring that the path is more efficient and accurate in the actual welding process. Through local optimization, the quality of the path can be effectively improved, unnecessary turns and length can be reduced, and collisions with obstacles can be avoided, thereby generating an optimal repair welding path to improve welding quality and ensure safety and efficiency.

[0110] S7: repair welding is performed according to the final repair welding path, and S1 is returned to continue monitoring.

[0111] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0112] In the embodiments of the present application, the repair welding path can be adaptively generated according to the change of welding quality in real-time welding process, the welding defects can be dynamically repaired, and the degree of automation of welding can be effectively improved.

[0113] Reference is made to the accompanying drawings Figure 2 , which shows a structural schematic diagram of a dynamic repair welding system based on welding quality evaluation provided by the embodiments of the present application.

[0114] The embodiments of the present application provide a dynamic repair welding system 20 based on welding quality evaluation, which comprises a processor 201 and a memory 202.

[0115] The memory 202 stores programs or instructions executable on the processor 201, and the programs or instructions are executed by the processor 201 to realize the steps of the above-mentioned dynamic repair welding method based on welding quality evaluation, and achieve the same technical effects. To avoid repetition, the present application will not be described again.

[0116] It should be understood that the processor 201 in the embodiments of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0117] It is also to be understood that the memory 202 in embodiments of the present application can be volatile or nonvolatile memory, or can include both volatile and nonvolatile memory. Nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), used as external cache. By way of example, and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0118] The above-described embodiments can be implemented in whole or in part by software, hardware (e.g., circuitry), firmware, or any combination of the three. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another via wired (e.g., infrared, wireless, microwave, etc.) or wireless means. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0119] It should be understood that the size of the sequence number of each process described above in various embodiments of the present application does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0120] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0121] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0122] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation. For example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0123] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0124] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0125] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0126] The embodiments of the present application provide a readable storage medium, which includes: a program or instructions stored on the readable storage medium, the program or instructions are executed by a processor to implement the steps of the dynamic repair welding method based on welding quality evaluation described above, and the same technical effects can be achieved. To avoid repetition, the present application will not be described again.

[0127] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application.

Claims

1. A method for dynamic repair welding based on welding quality assessment, characterized in that, The method comprises the following steps: S1: obtaining three-dimensional point cloud data and image data of a welded area; S2: determining the welding quality of the welded area by a welding quality evaluation model based on deep learning according to the three-dimensional point cloud data and the image data; S3: judging whether the welding quality is qualified or not; if yes, no repair welding is needed; Otherwise, the next step is entered; S4: updating a three-dimensional topological map based on the three-dimensional point cloud data and the image data; S5: performing global repair welding path planning in the three-dimensional topological map by an RRT* algorithm; S6: locally optimizing the global repair welding path by a gradient ascent algorithm to generate a final repair welding path; S7: performing repair welding according to the final repair welding path and returning to S1 for continuous monitoring; The S4 specifically comprises: S401: calibrating internal and external parameters of a camera and a laser sensor; S402: time synchronizing the three-dimensional point cloud data and the image data; S403: correcting motion distortion of the three-dimensional point cloud data; S404: using a mileage calculation method to register three-dimensional point cloud data of adjacent frames; S405: using a pose graph optimization algorithm to optimize key frames of the image data; S406: obtaining new observation values according to the registered three-dimensional point cloud data and the key frame optimized image data; S407: using a clipping function to limit the new observation values of each voxel; S408: using TSDF fusion to assign distance values to each voxel; S409: forming a TSDF voxel map according to the distance values of each voxel; S410: using a Victor fast sweeping method to solve an ESDF voxel map from the TSDF voxel map.

2. The method for dynamic repair welding based on welding quality evaluation according to claim 1, characterized in that, The S2 specifically comprises: S201: extracting point cloud features of the three-dimensional point cloud data by PointNet+ in a point cloud branch of the welding quality evaluation model; S202: extracting image features of the image data by a feature pyramid network in an image branch of the welding quality evaluation model; S203: performing feature fusion on the point cloud features and the image features by a cross-modal attention fusion mechanism to obtain fusion features; S204: determining a welding quality result of the welded area according to the fusion features, wherein the welding quality result comprises multiple sub-task results: quality score, defect area, repairability and weld center line.

3. The method of claim 2, wherein, The loss function during training of the welding quality evaluation model is obtained by superimposing the loss functions of each sub-task.

4. The method for dynamic repair welding based on welding quality evaluation according to claim 1, characterized in that, The S5 specifically comprises: S501: determining obstacle information, reachable areas and topological structures of each welding process section according to the updated ESDF voxel map; S502: initializing starting and target positions of the RRT* algorithm according to a starting point and an ending point of the repair welding task to determine basic parameters of path planning; S503: performing RRT* sampling in the three-dimensional topological map, sampling multiple points in the map by expanding a tree structure to gradually approach a target area; S504: evaluating the generated path by using a cost function; S505: determining the global repair path by multiple iterations of optimization.

5. The method of claim 4, wherein, The cost function includes a trajectory length term, a safety term, and a smoothness term.

6. The method for dynamic repair welding based on welding quality evaluation according to claim 4, characterized in that, The S6 specifically includes: S601: parameterizing the global repair path as a B-spline spline path; S602: calculating the gradient of each control point according to the objective function; S603: using a gradient ascent algorithm to perform local optimization, and considering path constraints during the gradient ascent process; S604: performing collision detection on the path to ensure that the optimized path does not collide with any obstacles; S605: optimizing until convergence to generate the final repair path.

7. The method of claim 6, wherein, The objective function includes a quality term, a trajectory length term, a smoothness term, and a safety term.

8. The method of claim 6, wherein, The path constraints include pose constraints of the welding gun, speed and acceleration limits of the path.

9. A dynamic repair welding system based on weld quality assessment, characterized by, It includes: a processor and a memory; The memory stores programs or instructions that can be run on the processor, and the programs or instructions are executed by the processor to implement the steps of the dynamic repair welding method based on welding quality evaluation as claimed in any one of claims 1 to 8.

Images