Weld detection method, device, system and storage medium

Abstract

The present invention provides a weld detection method, device, system and storage medium, which relates to the technical field of industrial welding, including: obtaining the linear point cloud data of two adjacent devices to be welded for segmentation processing, and determining a target straight line corresponding to each of the two devices to be welded; determining the point located between the two target straight lines from the linear point cloud data, and determining the point located between the two target straight lines as multiple candidate breakpoints corresponding to the weld; according to the position information of each candidate breakpoint, detecting whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold, and determining multiple target breakpoints corresponding to the weld according to the detection result; performing line fitting processing on each target breakpoint according to the position information of each target breakpoint, and determining the fitting path corresponding to the weld; the fitting path is used to guide the welding robot to weld the weld according to the fitting path. The technical solution of the present invention can improve the welding efficiency of the weld and improve the accuracy of welding.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial welding, and in particular to a weld detection method, device, system and storage medium. Background Art

[0002] With the continuous development of computer technology and artificial intelligence technology, the requirements for welding technology in industrial sites are gradually tending towards automation and intelligence. Based on this, automated welding technology with welding robots as the main body has emerged.

[0003] In the related art, when a welding robot is used to weld equipment, welding is usually performed through a teaching and reproduction method, specifically, a teaching trajectory is manually given to guide the welding gun at the end of the welding robot to complete the expected welding work according to the manually given teaching trajectory.

[0004] However, the above technology has the problem of low welding efficiency and deviation between the actual weld and the teaching trajectory during the welding process. Summary of the invention

[0005] The present invention provides a weld detection method, device, system and storage medium, which are used to solve the defects in the prior art that the welding gun at the end of the robot completes the expected welding work according to the teaching trajectory, resulting in low welding efficiency and deviation between the actual weld and the teaching trajectory during the welding process, so as to achieve the purpose of improving the welding efficiency of the weld and improving the accuracy of welding.

[0006] The present invention provides a weld detection method, comprising the following steps.

[0007] Obtain straight line point cloud data of two adjacent devices to be welded, and segment the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the straight line point cloud data includes a plurality of points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the devices to be welded in the weld area;

[0008] According to the intersection of the two target straight lines or the nearest points adjacent to the two target straight lines, a point located between the two target straight lines is determined from the straight line point cloud data, and the point located between the two target straight lines is determined as a plurality of candidate breakpoints corresponding to the weld;

[0009] According to the position information of each candidate breakpoint, whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold is detected, and multiple target breakpoints corresponding to the weld are determined according to the detection result; the distance between the target breakpoints in each row is less than the first distance threshold;

[0010] Line fitting is performed on each target breakpoint according to its position information to determine a fitting path corresponding to the weld; the fitting path is used to guide the welding robot to weld the weld according to the fitting path.

[0011] According to a weld detection method provided by the present invention, the above-mentioned detecting whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold according to the position information of each candidate breakpoint includes:

[0012] A preset number of sliding windows are used to select each candidate breakpoint row by row, and the coordinate difference between adjacent candidate breakpoints in each sliding window is calculated, and each coordinate difference is summed to obtain the difference sum corresponding to each row of sliding windows;

[0013] The distance difference between the difference values ​​of two adjacent rows of sliding windows is calculated, and it is detected whether the distance difference is less than a first distance threshold.

[0014] According to a weld detection method provided by the present invention, the above-mentioned determining multiple target breakpoints corresponding to the weld according to the detection results includes:

[0015] If the detection result is that the distance difference between two adjacent rows of sliding windows is not less than the first distance threshold, it is determined that there are abnormal breakpoints in the two adjacent rows of sliding windows, and the number of abnormal breakpoints is counted;

[0016] It is determined whether the number of abnormal breakpoints exceeds the preset abnormal data quantity threshold. If not, the abnormal breakpoints are interpolated to determine multiple target breakpoints corresponding to the weld.

[0017] According to a weld detection method provided by the present invention, the method further comprises:

[0018] If the number of abnormal breakpoints exceeds the abnormal data quantity threshold, abnormal information is output; the above abnormal information is used to indicate that there is an abnormality in the weld.

[0019] According to a weld detection method provided by the present invention, the above-mentioned determining multiple target breakpoints corresponding to the weld according to the detection results includes:

[0020] If the detection result is that the distance difference between two adjacent rows of sliding windows is less than the first distance threshold, the candidate breakpoints in the two adjacent rows of sliding windows are determined as multiple target breakpoints corresponding to the weld.

[0021] According to a weld detection method provided by the present invention, the above-mentioned segmentation processing of the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded includes:

[0022] Segment the straight line point cloud data to determine multiple candidate straight lines corresponding to the two devices to be welded; each candidate straight line includes at least one point;

[0023] Calculate the density of each candidate line according to the position information of the points on each candidate line and the number of points on the candidate line;

[0024] At least one candidate straight line with a smaller density among the candidate straight lines is deleted to determine a target straight line corresponding to each of the two devices to be welded.

[0025] According to a weld detection method provided by the present invention, the above-mentioned deleting at least one candidate straight line with a smaller density among the candidate straight lines to determine a target straight line corresponding to each of the two devices to be welded includes:

[0026] At least one candidate straight line with a smaller density is deleted from among the candidate straight lines, and at least two intermediate straight lines are determined; each intermediate straight line includes at least one point;

[0027] Acquire a first point set in the straight line point cloud data; the first point set includes a plurality of first points, and each first point is not a point on each intermediate straight line;

[0028] Traversing each first point in the first point set, selecting the first point belonging to each intermediate straight line from each first point and adding it to the intermediate straight line to obtain each processed intermediate straight line;

[0029] According to each processed intermediate straight line, a target straight line corresponding to each of the two devices to be welded is determined.

[0030] According to a weld detection method provided by the present invention, the above-mentioned step of selecting the first points belonging to each intermediate straight line from each first point includes:

[0031] For a first point in the first point set, calculate a first distance and a second distance between the first point and two endpoints of each intermediate straight line;

[0032] If the first distance and the second distance between the first point and any intermediate straight line are both greater than a preset endpoint distance threshold, then calculating a third distance between the first point and any intermediate straight line;

[0033] If the third distance is less than the preset second distance threshold, the first point is added to any intermediate straight line.

[0034] According to a weld detection method provided by the present invention, the above-mentioned determining a target straight line corresponding to each of the two devices to be welded according to each processed intermediate straight line includes:

[0035] The processed intermediate straight lines are subjected to segment fitting and straight line reprojection processing to determine a target straight line corresponding to each of the two devices to be welded.

[0036] According to a weld detection method provided by the present invention, the above-mentioned acquisition of straight line point cloud data of two adjacent devices to be welded includes:

[0037] According to the spatial distance between the line laser camera and the welding gun in the welding robot, the unit scanning distance of the line laser camera is determined; the above-mentioned unit scanning distance is smaller than the spatial distance;

[0038] From the welding start point to the welding end point of the weld, the line laser camera scans the two adjacent devices to be welded in sequence according to the unit scanning distance to obtain the straight line point cloud data of the two adjacent devices to be welded within the corresponding unit scanning distance.

[0039] According to a weld detection method provided by the present invention, the method further comprises:

[0040] For each unit scanning distance, from the starting point to the end point of the unit scanning distance, weld detection is performed on the straight line point cloud data of two adjacent devices to be welded within the unit scanning distance to determine the fitting path corresponding to the weld within the unit scanning distance;

[0041] According to the fitting path of the weld within the unit scanning distance, the teaching path corresponding to the fitting path within the unit scanning distance is interpolated to determine multiple segmented paths corresponding to the weld within the unit scanning distance; each segmented path is a fitting straight line or a fitting curve, which is used to guide the welding robot to weld the weld within the unit scanning distance in sequence according to each of the above segmented paths.

[0042] According to a weld detection method provided by the present invention, the method further comprises:

[0043] When the welding robot is welding the weld according to the fitting path, detecting whether there is any abnormality in the welding process; the above abnormality includes that the actual running path of the welding robot exceeds the limit area or the communication of the welding robot is abnormal;

[0044] If there is an abnormality, turn off the line laser camera and welding gun in the welding robot.

[0045] The present invention also provides a weld detection device, comprising the following modules:

[0046] An acquisition and segmentation module is used to acquire straight line point cloud data of two adjacent devices to be welded, and segment the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the straight line point cloud data includes a plurality of points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the device to be welded in the weld area;

[0047] A candidate breakpoint determination module is used to determine a point between two target straight lines from the straight line point cloud data according to the intersection of the two target straight lines or the nearest points adjacent to the two target straight lines, and determine the points between the two target straight lines as multiple candidate breakpoints corresponding to the weld;

[0048] A detection module, used to detect whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold according to the position information of each candidate breakpoint, and determine multiple target breakpoints corresponding to the weld according to the detection result; the distance between the target breakpoints in each row is less than the first distance threshold;

[0049] The determination module is used to perform line fitting processing on each target breakpoint according to the position information of each target breakpoint to determine the fitting path corresponding to the weld; the above fitting path is used to guide the welding robot to weld the weld according to the fitting path.

[0050] The present invention also provides an operation control system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a weld detection method as described above is implemented.

[0051] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the weld detection method as described in any one of the above is implemented.

[0052] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the weld detection method as described in any one of the above is implemented.

[0053] The weld detection method, device, system and storage medium provided by the present invention segment the straight line point cloud data of two adjacent devices to be welded with a weld, determine a target straight line corresponding to each device to be welded for characterizing the weld area on the device to be welded, then determine a point between the two target straight lines from the straight line point cloud data based on the intersection of the two target straight lines or the adjacent nearest points, so as to obtain multiple candidate breakpoints corresponding to the weld, and detect whether the distance between two adjacent rows of candidate breakpoints is less than a first distance threshold according to the position information of each candidate breakpoint, and determine multiple target breakpoints whose weld distance is less than the first distance threshold according to the detection result, and then perform line fitting processing on the target breakpoint according to the position information of each target breakpoint to determine a fitting path corresponding to the weld, and the fitting path is used to guide the welding robot to weld the weld according to the fitting path. In the technical solution of the present invention, since the breakpoints belonging to the weld can be determined in real time through the straight line point cloud data of two adjacent devices to be welded with a weld and the fitting path of the weld can be fitted accordingly, so as to guide the welding robot to weld the weld in real time through the fitting path, the welding speed of the method of detecting the welding point and guiding the welding robot to weld is faster than that of manually teaching to guide the welding robot to weld, so the efficiency of welding the weld can be improved; in addition, since the fitting path of the weld, that is, the weld trajectory, is determined through the straight line point cloud data of two adjacent devices to be welded with a weld, and the fitting path is determined by limiting the distance threshold condition of the weld breakpoint in the process, the determined weld trajectory is close to the actual weld trajectory, that is, the obtained weld trajectory is relatively accurate, thereby improving the accuracy of welding the weld. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 It is a schematic diagram of an application scenario of the weld detection method provided by the present invention.

[0056] Figure 2 It is a schematic diagram of the software framework of the weld detection method provided by the present invention.

[0057] Figure 3 It is a schematic diagram of TCP and TCF calibration points provided by the present invention.

[0058] Figure 4 It is a schematic diagram of a welding system of a vision-guided welding robot provided by the present invention.

[0059] Figure 5 It is a schematic diagram of the mathematical model of the standard sphere photographed by the line laser camera provided by the present invention.

[0060] Figure 6 It is a schematic flow chart of the weld detection method provided by the present invention.

[0061] Figure 7 It is a schematic diagram of the process of obtaining a straight line point cloud provided by the present invention.

[0062] Figure 8 It is a schematic diagram of partial straight line point cloud data of two adjacent devices to be welded provided by the present invention.

[0063] Fig. 9 It is a schematic diagram of a flow chart of obtaining a target breakpoint of a weld provided by the present invention.

[0064] Fig.10 It is a schematic diagram of the flow of point correction within a straight line provided by the present invention.

[0065] Fig.11 It is a scanning schematic diagram of the line laser camera provided by the present invention.

[0066] Fig.12 It is a schematic diagram of the segmented interpolation teaching path and the actual weld provided by the present invention.

[0067] Fig.13 It is a schematic diagram of the overall welding process provided by the present invention.

[0068] Fig.14 It is a structural schematic diagram of the weld detection device provided by the present invention.

[0069] Fig.15 It is a structural schematic diagram of the operation control system provided by the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The welding market has seen strong demand in recent years, with a high annual growth rate. Laser welding processing, as a relatively new technology, is widely used in consumer electronics, new energy vehicles and other fields. In addition, with the rapid development of science and technology such as computers and artificial intelligence, the requirements for the automation and intelligence level of welding technology in industrial sites are constantly increasing, and automated welding processing systems with welding robots as the main body have gradually replaced manual welding. At present, the domestically developed welding system is limited by the current level of intelligence of robots. Robot welding mainly adopts two methods: manual "teaching and reproduction" and offline programming. These methods have relatively low welding efficiency, and there is a deviation between the actual weld and the teaching trajectory in the production process. Based on this, the embodiments of the present invention provide a weld detection method, device, system and storage medium, which can solve the above-mentioned technical problems.

[0072] First, the application scenario of the embodiment of the present invention is described.

[0073] The weld detection method of the embodiment of the present invention can be applied to Figure 1In the application scenario shown, the system consists of three parts: management platform / data center, front-end interactive device (i.e., device that provides the interactive interface in the figure), and operation control system. The management platform can communicate with the front-end interactive device, which is mainly responsible for the management, display, and storage of big data. The front-end interactive device can receive the operation instructions or control instructions input by the user and send them to the operation control system or management platform, and can provide an interactive interface, which can be a UI interface (User Interface); the front-end interactive device can communicate with the abstract user interface of the operation control system, and is mainly responsible for the core functions of the operation control system such as start and stop, task distribution, hardware status, data query, system calibration, and other configurations. The operation and control system is responsible for the operation and control of the entire system, coordinating the serial data module, calibration module, vision module, synchronization module and execution module; the data module is equivalent to the RAM (Random Access Memory) of the operation and control system / host, responsible for the temporary storage of data such as task queues, hardware status, configuration information, welding data, calibration data and trajectory data, and interacts with the operation and control brain in real time during the operation and control process; the calibration module is an offline module of the operation and control system, including hand-eye calibration and tool calibration, which is called when the line laser camera and welding gun tool on the robot arm tool are installed for the first time to calibrate the line laser camera and welding gun; the vision module is the core module of the operation and control system, responsible for scanning and imaging of welds, weld detection and generating motion paths / welding trajectories, and sending trajectories / paths to the execution module for execution by the execution module; the synchronization module is responsible for outputting hardware pulse signals, has the functions of synchronizing the operation and control process and distance measurement, and is composed of encoders or (or) pulse generators; the execution module can control the base movement by PLC (Programmable Automation Controller).

[0074] Furthermore, the visual module can obtain the current task information from the system data center, obtain image data from the camera in real time, generate motion trajectory and laser welding gun control commands according to the weld tracking algorithm and actual position information, command the entire system to run and stop, and ensure the end-to-end accuracy and stability of the system. The data interaction of the visual module includes: the visual module obtains the point cloud data of the line laser camera and performs preprocessing, visually analyzes and detects the weld, and performs coordinate transformation according to the calibration data; as the basis for providing detection for the decision center, it provides the data basis for the data center; the data center saves the point cloud data and prediction results obtained by the visual module, and saves the current and historical posture status data of the laser welding gun, the robot arm, and the camera; obtains the point cloud and prediction data of the visual module, and obtains the safety monitoring data of the decision center, and provides calibration data to the visual module; the decision center implements path planning according to the prediction results of the visual module, guides the robot arm to correct the weld, and monitors the posture and status of the hardware, monitors the correctness of the weld path, and alarms for abnormal conditions; and returns the status and posture of the equipment to the data center for data recording.

[0075] The above-mentioned motion control system can move according to the motion trajectory given by the visual module, and perform position compensation according to the actual robot arm coordinates to ensure that the movement is accurate and effective within the compensation range. The above-mentioned welding gun can receive the laser welding gun control command of the visual module to execute the opening / closing of the welding gun laser head; and receive the welding gun parameter control command to adjust the laser welding gun power and focus point and other parameters to achieve functions such as precise melting control of the weld.

[0076] Also, see Figure 2The software framework diagram shown in the figure includes the UI layer, the framework layer and the system layer; in actual use, the various module layers can be effectively connected together to provide basic software facilities such as data center, thread management, memory management, start, stop, etc. The UI layer is the main page QT / machine signal, including welding APP (Application, application software), maintenance APP, machine signal, etc. The framework layer includes thread space and third-party libraries. The thread space includes the task queue / task list TaskList received from the UI layer, and also includes the main thread (used to start and end the visual thread and operation control thread), the visual thread (including the business start interface and the business stop interface, used to monitor the task queue / task list) and the operation control thread (including the business start interface and the business stop interface, used to monitor the task queue / task list). The third-party library may include: MemoryPool (memory pool), log4cplusplus (log library), CppNet (C++ network library), eventpp (C++ event library), sqlite (light database) and thread pool. The system layer is Win10 system, including camera SDK (Software Development Kit), robot SDK, welding gun SDK and Disk. Specifically, the main thread of the operation control system controls the entire process from the successful start to the stop of the visual thread and the operation control thread. The threads communicate with each other, and can specify memory to send to other threads for use. Tasks triggered by the interactive interface UI or machine signals of the UI layer are placed in the task list of the thread space. After obtaining the tasks, the visual thread and the operation control thread can start their own business and pass the corresponding status through messages.

[0077] It is understood that the above motion system can be arranged inside the welding robot, or can also be arranged outside the welding robot and communicate with the computer device inside the welding robot to control the welding robot to perform welding. The welding robot includes a mechanical arm, and the line laser camera and the welding gun are both arranged on the mechanical arm.

[0078] Before inspecting the weld, the tools in the welding robot need to be calibrated, specifically, the robot arm, line laser camera and welding gun are calibrated to the base coordinate system of the welding robot / welding robot base. Here, multiple coordinate systems of the hardware can generally be calibrated, including the robot coordinate system, the line laser camera hand-eye coordinate system, the robot flange coordinate system, the laser coordinate system, the world coordinate system, and the coordinate transformation matrix parameters of multiple coordinate systems. The calibration program can be called and the hardware calibration information can be written into the existing database and configuration file.

[0079] The following is an explanation of the tool coordinate system calibration and line laser hand-eye calibration, see Figure 3The schematic diagram of TCP and TCF calibration points shown (the left side is the schematic diagram of TCP calibration points, and the right side is the schematic diagram of TCF calibration points), where the tool coordinate system calibration includes the tool center point position calibration (TCP) , Tool coordinate system posture calibration (TCF) .

[0080] 1. Tool Center Point Calibration (TCP) The steps are as follows:

[0081] 1. The welding robot tool welding gun tip changes its posture and rotates around a fixed point multiple times, and each rotation is recorded (Represents the transformation matrix between the base coordinate T and the coordinate system E where the end flange surface is located).

[0082] 2. Calculate (represents the positional relationship between the tool coordinate system and the welding robot end coordinate system). For the i-th posture transformation, the following formula can be used:

[0083] ;

[0084] Where i represents the i-th posture change; n represents the total number of posture changes; Represents the transformation matrix between the tool coordinate system and the base coordinate system; It represents the calibration relationship between the base coordinates and the coordinate system E where the end flange surface is located; Indicates the calibration relationship between the base coordinate system and the tool coordinate system; The rotation matrix representing the transformation between the base coordinates and the end flange; and is a known quantity, which can be obtained by the above formula .

[0085] 2. Tool Coordinate System Calibration (TCF) Here are the steps:

[0086] 1. The tool coordinate system (TCP) can be obtained by calibrating the tool center point The position of the TCF is calculated by calibrating in the z / x direction. During this process, the posture of the TCF remains unchanged.

[0087] 2. The TCP calibration of the above tool generally takes 4 postures. When calibrating TCF, the fourth posture calibration point is generally taken as position point 1 ( ); The robot starts from position 1 and moves a certain distance in the +x direction to reach position 2 ( ); The robot starts from position 1 and moves a certain distance in the +z direction to reach position 3 ( ). Then the x-axis axial vector X of the tool coordinate system T can be expressed by the following formula: Similarly, the z-axis axial vector Z of the tool coordinate system T can be expressed by the following formula: Then, according to the right-hand rule, the y-axis axial vector Y of the tool coordinate system T can be expressed by the following formula: Then convert XYZ to unit to get , and then get the posture of the tool coordinate T relative to the base coordinate B : The rotation matrix of the end coordinate system E is ,and: .

[0088] 3. Combining the above formulas, we can get the rotation matrix of the tool coordinate system , which is expressed as follows:

[0089] .

[0090] 3. The steps of line laser hand-eye calibration are as follows:

[0091] 1. See Figure 4 The schematic diagram of the welding system of the vision-guided welding robot is shown. The whole system mainly includes: (1) Robot arm base coordinate system; (2) Coordinate system of flange at the end of the robot arm; (3) is the line laser camera coordinate system. The relative position is fixed, and the change matrix needs to be solved; :The change matrix can be used by the robot system to obtain the flange position and posture in real time and convert it into a rotation and translation matrix. Control the robot to make the line laser camera shoot a fixed point P in space with different postures, and calculate the coordinate system of point P. The position of the robot arm end flange in each posture is recorded at the same time. The point P is fixed relative to the base coordinate system to solve The transformation matrix is ​​used to calibrate the hand-eye system of the line laser camera.

[0092] 2. Hand-eye calibration solution, assuming that the coordinates of the space fixed point P in the online laser coordinate system are , the coordinates of the space fixed point in the base coordinate system are ; The transformation matrix from the line laser coordinate system to the tool coordinate system is , the transformation matrix from the tool coordinate system to the base coordinate system is ; Then the line laser coordinate system to the base coordinate system is as follows:

[0093] .

[0094] in, , which can be given by the robotic arm system, the target is The transformation matrix By taking advantage of the fact that a fixed point in space is photographed in different postures and its coordinates remain unchanged in the base coordinate system, the equation can be established:

[0095] .

[0096] Furthermore, after collecting data from multiple postures, a linear equation is established based on the above equation: , use singular value decomposition SVD to decompose and solve X. Where:

[0097] ;

[0098] ;

[0099] Where n is the total number of times point P is photographed in different postures. To ensure calibration accuracy and reliability, the number of measurements n is usually > 6.

[0100] 3. Solve the coordinates of the fixed point in space, that is, solve the coordinates of the center of the standard sphere. Figure 5 The mathematical model of the standard sphere photographed by the line laser camera is shown in the figure. The standard sphere is photographed by the line laser camera, the laser surface is tangent to the sphere, and the center of the standard sphere is , the center of the section circle is . The center of the standard ball is , we can calculate:

[0101] ;

[0102] ;

[0103] .

[0104] in, The center of the standard sphere section circle is taken n times in the same posture, and the coordinates of the center of the sphere are taken as the average value of n times. R is the radius of the standard sphere, and r is the radius of the section circle.

[0105] After the above-mentioned calibration of the welding robot is completed, the weld can be inspected.

[0106] Combine the following Figure 6-Figure 13 A weld detection method according to an embodiment of the present invention is described.

[0107] It should be noted that the execution subject of the embodiments of the present invention may be a weld detection device, or a motion system or electronic device including a weld detection device, or a welding robot including a weld detection device or an operation control system, etc. The following embodiments will be described using the operation control system as the execution subject.

[0108] Figure 6 It is a schematic diagram of the process of the weld detection method provided by the present invention, such as Figure 6 As shown, the method comprises the following steps:

[0109] S102, obtaining straight line point cloud data of two adjacent devices to be welded, and segmenting the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the straight line point cloud data includes a plurality of points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the devices to be welded in the weld area.

[0110] There is a weld between two adjacent devices to be welded, and the straight line point cloud data of the two adjacent devices to be welded are obtained by scanning the surfaces of the two adjacent devices to be welded using a line laser camera arranged at the end of the welding robot.

[0111] Before welding, the welding robot can drive the mechanical arm to reach the starting position, and the starting position should make the center of the line laser camera located near the starting point of the weld between the two adjacent devices to be welded and outside the weld. Then the mechanical arm can drive the line laser camera to move to scan the surface of the two adjacent devices to be welded in the weld area, and obtain the linear point cloud data of the two adjacent devices to be welded. Since the weld is scanned by a line laser camera, and the line laser camera scans a straight line, the point cloud data obtained can be recorded as linear point cloud data. The linear point cloud data of the two adjacent devices to be welded obtained here can be the linear point cloud data of the entire weld area, or the linear point cloud data of part of the weld area. For example, the linear point cloud data of a part of the weld area can be obtained and detected to obtain the welding trajectory of the part of the weld. When the welding gun welds this section of the weld, the weld of the next part of the area continues to be scanned, and this method is repeated to weld the entire weld area.

[0112] Specifically, you can batch receive the line point cloud frame data collected by the line laser camera, downsample and sort the line point cloud data, and output a single frame of line point cloud data (for example, it can include 1600 points). Then you can use the RANSAC (random sample consensus) algorithm to segment all the lines in the line point cloud data. For specific segmentation, see Figure 7The flowchart of obtaining the straight line point cloud shown in the figure, the straight line point cloud data includes multiple points, firstly, n points (generally n is 7) are selected in the straight line point cloud data for initialization, and the least square method is used for straight line fitting to obtain the fitted straight line line; then the distance dis1 from the n+1th point to the straight line is calculated, and it is determined whether dis1 is greater than the set distance threshold (dis_threshold); if dis1 is less than the set distance threshold, the point is included in the straight line; if dis1 is not less than (greater than or equal to) the set distance threshold, it means that the point may not belong to the straight line, in order to determine the subsequent If there are no more points belonging to the straight line, the distance dis2 from each point of k (for example, 7) points to the straight line can be calculated continuously; and it can be determined whether the dis2 of each point of the k points is greater than the set distance threshold; if the dis2 of a point of the k points is less than the set distance threshold, it means that the point belongs to the point on the straight line, and the point can be included in the straight line; if the dis2 is not less than (greater than or equal to) the set distance threshold, the remaining points in the straight line point cloud data except the points on the straight line are obtained, and the initial n points of the remaining points are continuously selected for judgment until the straight line point cloud data including only the straight line is obtained.

[0113] Then, the points in the straight line point cloud data including only straight lines can be processed by straight line fitting, etc. to obtain a target straight line corresponding to each device to be welded. The target straight line of each device to be welded can represent the outer contour of the corresponding device to be welded in the weld area. For example, see Figure 8 The schematic diagram of partial straight line point cloud data of two adjacent devices to be welded is shown, in which the two green straight lines represent the outer contours of the two devices to be welded, and the area between the two green lines is the weld area. Here, for example, the straight line obtained by fitting can be used as the target straight line, or the straight line obtained by fitting can be further processed (such as correction, abnormal point removal, etc.) to obtain the target straight line. In addition, there can be a certain interval between the endpoints of the two target straight lines close to the weld area, or the two target straight lines can also intersect in the welding area.

[0114] S104, according to the intersection of the two target straight lines or the nearest points adjacent to the two target straight lines, determine the points between the two target straight lines from the straight line point cloud data, and determine the points between the two target straight lines as multiple candidate breakpoints corresponding to the weld.

[0115] In this step, after obtaining the target lines of the two devices to be welded, if the two target lines intersect in the welding area, the intersection of the two target lines can be obtained; if the two target lines do not intersect in the welding area, the endpoints of the two target lines close to the weld area (i.e., the closest points of the two target lines) can be obtained, such as Figure 8The red points at the endpoints of the two green lines.

[0116] After determining the intersection of the two target lines or the nearest points adjacent to the two target lines, the points located between the intersection or the nearest points can be determined in the line point cloud data based on the position information of the intersection or the nearest points, such as Figure 8 The multiple green points located below the red points of the two green lines are all disconnected points and can be recorded as breakpoints. These points can then be used as multiple candidate breakpoints corresponding to the weld. At the same time, the position information of these multiple candidate breakpoints can also be obtained in the straight line point cloud data. The position information can be the coordinates of the candidate breakpoints, such as two-dimensional coordinates or three-dimensional coordinates.

[0117] S106, based on the position information of each candidate breakpoint, detecting whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold, and determining multiple target breakpoints corresponding to the weld according to the detection result; the distance between each row of target breakpoints is less than the first distance threshold.

[0118] In this step, after obtaining multiple candidate breakpoints corresponding to the weld and the position information of each candidate breakpoint, these candidate breakpoints can be arranged in rows, and then the distance between two adjacent rows of candidate breakpoints can be calculated. After obtaining the distance between two adjacent rows of candidate breakpoints, it can be determined whether the distance between the two adjacent rows of candidate breakpoints is less than the first distance threshold. If the distance between the two adjacent rows of candidate breakpoints is less than the first distance threshold, it means that the distance between the two adjacent rows of candidate breakpoints is more appropriate, and the two adjacent rows of candidate breakpoints can be output as target breakpoints, that is, multiple target breakpoints corresponding to the weld are obtained, and these target breakpoints are the breakpoints tracked by the subsequent welding robot when welding the weld. If the distance is not less than the first distance threshold, it means that the distance between the two adjacent rows of candidate breakpoints may be large, and it is necessary to determine the multiple target breakpoints corresponding to the weld by interpolation or other methods. This part will be described in detail in the following embodiments. In addition, the size of the above-mentioned first distance threshold can be set according to actual conditions, and is not specifically limited here.

[0119] S108, performing line fitting processing on each target breakpoint according to the position information of each target breakpoint to determine a fitting path corresponding to the weld; the above fitting path is used to guide the welding robot to weld the weld according to the fitting path.

[0120] In this step, after obtaining multiple target breakpoints corresponding to the weld and their position information, the position information of each target breakpoint can be sorted, and the least square method can be used to fit each target breakpoint to obtain a fitting path corresponding to the weld, which can further improve the accuracy of weld breakpoint detection and recognition. It can be understood that the fitting path here can be a straight line or a curve. At the same time, the fitting path can be composed of one line or a combination of multiple segmented lines.

[0121] In addition, the fitting path can correspond to the welding path of the welding robot. After obtaining the fitting path of the weld, the welding robot can generate a corresponding welding path according to the fitting path, and weld the weld according to the welding path. Of course, the pre-set teaching path can also be modified by the fitting path to obtain a smoother and closer to the actual welding path, so that the welding robot can accurately and efficiently achieve accurate welding of the weld in the future.

[0122] In addition, it should be noted that the above-mentioned fitting path can be directly output, or multiple target breakpoints and their position information included in the fitting path can be output, and weld tracking points can be selected according to the target breakpoints to achieve accurate welding of the weld.

[0123] In this embodiment, by segmenting the straight line point cloud data of two adjacent devices to be welded with a weld, a target straight line corresponding to each device to be welded and used to characterize the weld area on the device to be welded is determined, and then a point located between the two target straight lines is determined from the straight line point cloud data based on the intersection of the two target straight lines or the adjacent nearest points to obtain a plurality of candidate breakpoints corresponding to the weld, and according to the position information of each candidate breakpoint, it is detected whether the distance between two adjacent rows of candidate breakpoints is less than a first distance threshold, and according to the detection result, a plurality of target breakpoints whose weld distance is less than the first distance threshold are determined, and then line fitting processing is performed on the target breakpoints according to the position information of each target breakpoint to determine a fitting path corresponding to the weld, and the fitting path is used to guide the welding robot to weld the weld according to the fitting path. In the technical solution of the present invention, since the breakpoints belonging to the weld can be determined in real time through the straight line point cloud data of two adjacent devices to be welded with a weld and the fitting path of the weld can be fitted accordingly, so as to guide the welding robot to weld the weld in real time through the fitting path, the welding speed of the method of detecting the welding point and guiding the welding robot to weld is faster than that of manually teaching to guide the welding robot to weld, so the efficiency of welding the weld can be improved; in addition, since the fitting path of the weld, that is, the weld trajectory, is determined through the straight line point cloud data of two adjacent devices to be welded with a weld, and the fitting path is determined by limiting the distance threshold condition of the weld breakpoint in the process, the determined weld trajectory is close to the actual weld trajectory, that is, the obtained weld trajectory is relatively accurate, thereby improving the accuracy of welding the weld.

[0124] The following embodiment describes the specific process of how to detect whether two adjacent target straight lines meet the threshold condition and determine the target breakpoint according to the detection result.

[0125] In one embodiment, see Fig. 9 In the flowchart of obtaining the target breakpoint of the weld shown in FIG. 1 , the above-mentioned S106 of “detecting whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold according to the position information of each candidate breakpoint” may include the following steps:

[0126] Step A1, using a preset number of sliding windows to select each candidate breakpoint row by row, and calculating the coordinate difference between adjacent candidate breakpoints in each sliding window, and summing the coordinate differences to obtain the difference sum corresponding to each row of sliding windows.

[0127] In this step, a preset number of sliding windows that slide row by row are first set, for example, 50 sliding windows can be set, each row of sliding windows can basically cover the breakpoints on the row, and the preset number of sliding windows can basically cover the breakpoints of all rows. Then, the difference between the position information (specifically, the x-axis coordinates) of adjacent candidate breakpoints in each row of sliding windows can be calculated and recorded as the coordinate difference. Afterwards, the multiple coordinate differences in each row of sliding windows can be summed, and the obtained sum can be recorded as the difference sum corresponding to the row of sliding windows.

[0128] In this way, when the sliding window slides row by row, the sum of the difference values ​​corresponding to each row of the sliding window can be calculated.

[0129] Step A2, calculating the distance difference between the difference sums of two adjacent rows of sliding windows, and detecting whether the distance difference is less than a first distance threshold.

[0130] In this step, after calculating the difference sum sum(i+1) corresponding to each row of sliding windows, the difference of the row of sliding windows and the difference sum sum(i) of the sliding windows of the previous adjacent row can be calculated, and the obtained difference can be recorded as the distance difference dis(i), where i is greater than or equal to 1. Then, it can be determined whether the distance difference is less than the first distance threshold to obtain a determination result, which is the above-mentioned detection result.

[0131] In a possible implementation, if the detection result is that the distance difference between two adjacent rows of sliding windows is less than the first distance threshold, the candidate breakpoints in the two adjacent rows of sliding windows are determined as multiple target breakpoints corresponding to the weld. That is, if the distance difference between two adjacent rows of sliding windows is less than the first distance threshold, it means that the distance between the candidate breakpoints in the two rows of sliding windows is relatively appropriate, and the candidate breakpoints in the two rows of sliding windows and their position information can be output as the target breakpoints of the weld.

[0132] In another possible implementation, if the detection result is that the distance difference between two adjacent rows of sliding windows is not less than the first distance threshold, then optionally, the “determining multiple target breakpoints corresponding to the weld according to the detection result” in the above S104 may include the following steps:

[0133] If the detection result is that the distance difference between two adjacent rows of sliding windows is not less than the first distance threshold, it is determined that there are abnormal breakpoints in the two adjacent rows of sliding windows, and the number of abnormal breakpoints is counted; it is determined whether the number of abnormal breakpoints exceeds the preset abnormal data number threshold. If not, the abnormal breakpoints are interpolated to determine multiple target breakpoints corresponding to the weld.

[0134] Among them, when the distance difference between two adjacent rows of sliding windows is greater than or equal to the first distance threshold, it is determined that there are abnormal breakpoints in the two adjacent rows of sliding windows, for example, the candidate breakpoints in the current row of sliding windows can be used as abnormal breakpoints. Then the number of abnormal breakpoints num can be counted, and it can be determined whether the number of abnormal breakpoints exceeds the set abnormal data number threshold (for example, 10% of the total number of breakpoints NUM). Optionally, if the number of abnormal breakpoints exceeds (that is, greater than) the abnormal data number threshold, abnormal information is output, and the abnormal information is used to indicate that there is an abnormality in the weld. If the number of abnormal breakpoints does not exceed (that is, less than or equal to) the abnormal data number threshold, the abnormal breakpoints in the current row of sliding windows can be interpolated (for example, interpolation and other processing can be performed based on the position information of the candidate breakpoints in the previous row and the next row of sliding windows to supplement the breakpoints), and the repaired normal breakpoints are obtained, and then these normal breakpoints are used as the target breakpoints of the weld.

[0135] In this embodiment, a preset number of sliding windows are used to calculate the sum of the difference between the coordinates of the breakpoints in two adjacent rows of sliding windows, and then the distance difference between the two rows of sliding windows is calculated accordingly. This process is simple and intuitive, and thus the efficiency and accuracy of detecting and identifying the target breakpoints of the weld can be improved. In addition, when there are abnormalities in the candidate breakpoints in the detected sliding window, the abnormal breakpoints can be interpolated to ensure that the breakpoints in the sliding window are normal breakpoints, further improving the accuracy of detecting and identifying the target breakpoints of the weld.

[0136] The following embodiment describes the specific process of how to segment the straight line point cloud data of two adjacent devices to be welded into two target straight lines.

[0137] In one embodiment, the above-mentioned S102 of “segmenting the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded” may include the following steps:

[0138] Step B1, segmenting the straight line point cloud data to determine a plurality of candidate straight lines corresponding to two devices to be welded; each candidate straight line includes at least one point.

[0139] Step B2, calculating the density of each candidate straight line according to the position information of the points on each candidate straight line and the number of points on the candidate straight line.

[0140] Step B3, deleting at least one candidate straight line with a smaller density among the candidate straight lines, and determining a target straight line corresponding to each of the two devices to be welded.

[0141] Among them, you can continue to refer to the above Fig. 9The flowchart shown in the figure can segment the point cloud data belonging to the same line in the line point cloud data, that is, segment all the lines in the line point cloud data to obtain multiple lines, all of which are recorded as candidate lines. Each candidate line can include one or more points, where the points can be points in the line point cloud data. When the above segmentation obtains multiple points on the candidate line, the position information of these multiple points can also be obtained.

[0142] Then, the length of each candidate line can be calculated based on the position information of each point on each candidate line. Specifically, the x-coordinate value in the position information of the two endpoints (i.e., the points at the endpoints) on each candidate line can be obtained. For each candidate line, the difference between the x-coordinates of its two endpoints can be calculated, and the difference obtained is the length of the candidate line. At the same time, the number of points in the candidate line can also be counted, and the length of the candidate line can be divided by the number of points to obtain the density of the candidate line, that is, density = line point cloud length / line point cloud number. In this way, the density of all candidate lines can be calculated.

[0143] Afterwards, the calculated densities of the candidate lines can be sorted (for example, from small to large), and several candidate lines with smaller density can be deleted. When deleting, the density of each candidate line can be compared with a preset density threshold, and the candidate lines with a density less than the density threshold can be deleted. After deleting the candidate lines with smaller density, a target line corresponding to each of the two devices to be welded can be determined through the remaining candidate lines.

[0144] When determining two target lines through the remaining candidate lines, at least one candidate line with a smaller density can be deleted first to determine at least two intermediate lines; each intermediate line includes at least one point; that is, the candidate lines remaining after deleting the candidate lines with a smaller density are recorded as intermediate lines. Then, the points in the line point cloud data that are not on all the intermediate lines can be corrected. For details, see Fig.10 The flowchart of point correction within a straight line is shown. Optionally, the correction process may include: obtaining a first point set in the straight line point cloud data; the first point set includes multiple first points, and each first point is not a point on the intermediate straight line; traversing each first point in the first point set, filtering out the first point belonging to each intermediate straight line from each first point and adding it to the intermediate straight line to obtain each processed intermediate straight line; according to each processed intermediate straight line, determining a target straight line corresponding to each of the two devices to be welded.

[0145] Specifically, first obtain the points in the straight line point cloud data that are not on the middle straight line, record them as first points, and form each first point into a point set to obtain the first point set; then loop through the first point set, and optionally, for a first point in the first point set, calculate the first distance and the second distance between the first point and the two endpoints of each middle straight line; if the first distance and the second distance between the first point and any middle straight line are both greater than a preset endpoint distance threshold, calculate the third distance between the first point and any middle straight line; if the third distance is less than the preset second distance threshold, add the first point to any middle straight line.

[0146] That is to say, for each first point, the distance between the index (i.e., position information) of the first point and the index (i.e., position information) of the two endpoints of each intermediate straight line can be calculated to obtain two distance values ​​between the first point and each intermediate straight line, which are respectively recorded as the first distance Dis1 and the second distance Dis2. Taking an intermediate straight line as an example, the two distance values ​​between the first point and the intermediate straight line can be compared with the endpoint distance threshold (the specific size can be set according to actual conditions), that is, to determine whether the Dis1 of the first point is greater than the endpoint distance threshold nstep and to determine whether Dis2 is greater than the endpoint distance threshold nstep; if the Dis1 of the first point is not greater than the endpoint distance threshold nstep and / or Dis2 is not greater than the endpoint distance threshold nstep, then continue to traverse the next first point in the first point set; if the Dis1 of the first point is greater than the endpoint distance threshold nstep and Dis2 is greater than the endpoint distance threshold nstep, then calculate the distance between the first point and the intermediate straight line. The distance between the lines is recorded as the third distance Dis, and the third distance Dis of the point is compared with the second distance threshold (dis_threshold, the specific size can be set according to the actual situation), that is, it is determined whether the third distance Dis of the point is greater than the second distance threshold dis_threshold; if the third distance Dis of the first point is greater than the second distance threshold dis_threshold, continue to traverse the next first point in the first point set; if the third distance Dis of the first point is not greater than the second distance threshold dis_threshold, then the first point is added to the straight line point cloud data of the middle straight line for subsequent straight line fitting.

[0147] In the above manner, multiple processed intermediate straight lines can be obtained, and then optionally, each processed intermediate straight line can be subjected to segmented fitting and straight line reprojection processing to determine a target straight line corresponding to each of the two devices to be welded. Specifically, segmented point cloud straight line fitting can be performed according to the point cloud segmentation result obtained in the point cloud straight line segmentation stage, that is, multiple segments of intermediate straight lines belonging to the same device to be welded in each intermediate straight line are linearly fitted in the order of position to obtain a target straight line corresponding to each device to be welded. Here, the least squares straight line fitting can be used, or the RANSAC straight line fitting method can be used, and the accurate straight line equation is obtained by the least squares straight line equation. After that, the point cloud straight line reprojection can be performed, and the original straight line point cloud data can be reprojected onto the straight line equation according to the parameters of the fitted straight line equation, which is equivalent to converting the original straight line point cloud data into straight line point cloud data that only includes straight lines, because the point cloud data at the edge of the straight line point cloud weld is not a straight line, but a curved line. The accuracy of breakpoint acquisition can be improved by reprojection.

[0148] In this embodiment, by calculating the density of each candidate line segmented in the line point cloud data and deleting the candidate lines with lower density to determine multiple target lines corresponding to the weld, the calculation of a portion of invalid data can be reduced, further improving the efficiency and accuracy of weld detection. In addition, by correcting the points within the candidate lines in the line point cloud data, more accurate candidate lines can be obtained, thereby improving the accuracy of the target lines finally determined based on them.

[0149] The following embodiment describes the process of how to obtain the straight line point cloud data of two adjacent devices to be welded in segments.

[0150] In one embodiment, the “obtaining straight line point cloud data of two adjacent devices to be welded” in S102 may include the following steps:

[0151] Step C1, determining the unit scanning distance of the line laser camera according to the spatial distance between the line laser camera and the welding gun in the welding robot; the unit scanning distance is smaller than the spatial distance.

[0152] Step C2, starting from the welding start point to the welding end point of the weld, the line laser camera sequentially scans two adjacent devices to be welded according to the unit scanning distance to obtain the straight line point cloud data of the two adjacent devices to be welded within the corresponding unit scanning distance.

[0153] Among them, since there is a spatial distance of up to 50mm between the line laser camera and the welding gun, the line laser camera scans the weld trajectory in advance and the welding action lags behind, so the operation of the robot arm can be controlled in the form of a section of trajectory points to improve the smoothness of the operation. In this embodiment, according to the spatial distance between the above-mentioned line laser camera and the welding gun, a distance smaller than the spatial distance can be selected as the unit scanning distance when the line laser camera scans the weld. For example, 20mm can be selected as the unit scanning distance. After determining the unit scanning distance, the scanning can be started from the weld starting point of the weld area in the two adjacent devices to be welded. Each time the straight line point cloud data within a unit scanning distance is scanned, a weld detection and subsequent welding of the weld within the unit scanning distance are performed. At the same time, the straight line point cloud data within the next unit scanning distance is scanned until the weld end point is scanned, detected and welded. See. Fig.11 The scanning schematic diagram of the line laser camera is shown, where S corresponds to the starting point of the weld and E corresponds to the end point of the weld. During the scanning process of the line laser camera, when the acquisition of the m-th (greater than or equal to 1) unit of straight line point cloud data is completed, the weld detection process is started and the scanning continues. When the m+1-th unit of point cloud is acquired, the processing of the m-th straight line point cloud data is completed. At this time, the line laser camera is still at a safety distance of 10 mm from the m-th area.

[0154] From the above description, it can be seen that since the processing result of the weld detection within the first unit scanning distance is available only when the second unit scanning distance is completed, the first two unit strokes need to be controlled by teaching to actively move the robot arm. When the robot arm is actively controlled by teaching, in order to accurately control the running path of the robot arm, the preset teaching path / teaching trajectory within the unit scanning distance can be interpolated to make it more consistent with the actual running path of the robot arm.

[0155] Optionally, for each unit scanning distance, from the starting point to the end point of the unit scanning distance, weld detection is performed on the straight line point cloud data of two adjacent devices to be welded within the unit scanning distance to determine the fitting path corresponding to the weld within the unit scanning distance; according to the fitting path of the weld within the unit scanning distance, the teaching path corresponding to the fitting path within the unit scanning distance is interpolated to determine multiple segmented paths corresponding to the weld within the unit scanning distance; each segmented path is a fitting straight line or a fitting curve, which is used to guide the welding robot to weld the welds within the unit scanning distance in sequence according to the above-mentioned segmented paths.

[0156] Among them, the above-mentioned interpolation methods for the teaching path include segmented interpolation and overall interpolation. The overall interpolation method mainly records the welding start and end points of the welding task (i.e., the weld) during the teaching process of the welding robot. In the actual welding task, the overall interpolation calculation is performed between the welding start and end points; the trajectory point data is fitted into different short straight lines, and the interpolation parameters within each short straight line remain unchanged. The length of the short straight line determines the length of the interpolation cycle.

[0157] For the segmented interpolation method, see Fig.12 The schematic diagram of the segmented interpolation teaching path and the actual weld is shown in the figure. Some prerequisites are known in advance, such as the running speed of the welding gun and the line laser camera is calculated according to the maximum running speed v; the minimum processing cycle of the robot arm is T, and the stroke of the robot arm during this period is d; then the time taken for a unit scanning distance of 20mm is t, and the number of cycles included is m (The parameters included here will not be fully used later); when the straight line point cloud data of the 20mm weld is obtained, the processing begins, and the starting point of the distance within this unit scanning distance is set to , the distance to the end point is B ; Find the turning point of the actual weld trajectory ; When the robot moves to point A, it starts to execute a new trajectory correction task. The correction amount in the x direction of each interpolation cycle is ; When the robot arm moves to the back p n , change the trajectory correction task again, the correction amount in the x direction for each interpolation cycle is When the robot arm moves to the end point B, it is also the starting point of a new weld trajectory. The above process is followed in the same way.

[0158] From the above description, it can be seen that the operation control system updates a fitting path / trajectory every 20mm (i.e. unit scanning distance). The fitting output of weld detection is no longer multiple high-precision trajectory points, but a fitting straight line or arc, which gives the starting point and intermediate point required for movl (linear motion) or movc (arc motion). Combined with the trajectory correction function of the robot arm, the position correction of the teaching path / trajectory can be achieved by a specified amount, thereby avoiding the deviation between the teaching trajectory and the actual weld and improving the accuracy of weld welding.

[0159] In this embodiment, a smaller unit scanning distance is determined by the spatial distance between the line laser camera and the welding gun, and the weld is scanned with straight line point cloud data within each unit scanning distance, so that the weld welding can be divided into multiple segments, that is, the welding process of the weld is refined, thereby further improving the accuracy of the weld welding. In addition, during the welding process, the teaching path / track can be interpolated through the actual weld to meet the actual weld situation, so that the welding gun can be guided to weld the weld more accurately, that is, the accuracy of the weld welding can be further improved.

[0160] The following embodiments illustrate how to detect and handle abnormalities that occur during welding.

[0161] In one embodiment, the above method may further include the following steps:

[0162] When the welding robot is welding the weld according to the fitting path, it is detected whether there is any abnormality in the welding process; if there is any abnormality, the line laser camera and welding gun in the welding robot are turned off.

[0163] The above-mentioned abnormalities include the actual running path of the welding robot exceeding the limit area or the communication abnormality of the welding robot. The limit area here refers to the non-welding area, that is, the area where welding cannot be performed. The communication abnormality here can be, for example, communication interruption, communication data not meeting the protocol requirements, etc.

[0164] Specifically, when the welding robot welds the corresponding section of the weld according to the fitting path within each unit scanning distance, the position of the welding gun during the welding process can be compared with the preset limit area in real time to determine whether the position of the welding gun exceeds the limit area; if it exceeds, it means that there is an abnormality in the welding process. At this time, it is necessary to turn off the line laser camera and welding gun in the welding robot to avoid welding errors.

[0165] At the same time, during the above welding process, it is also possible to detect in real time whether the communication between the welding robot and the operation control system / front-end interactive equipment / control center, or to detect in real time whether the communication of the welding robot itself is abnormal; if there is an abnormality in the communication of the welding robot, it is necessary to turn off the line laser camera and welding gun in the welding robot to avoid welding errors.

[0166] In this embodiment, welding errors are avoided by detecting whether there is an abnormality during the welding process of the weld and shutting down the line laser camera and welding gun in the welding robot when an abnormality exists.

[0167] The entire welding process is described in detail below through a specific embodiment. Fig.13 The overall process diagram shown in the figure is combined with Figure 1Schematic diagram of the application scenario, the welding process may include the following steps:

[0168] Step 1: First, wait for the welding equipment (such as steel plate) to be in place, and take out a new welding task from the task queue. According to the welding type, adjust the basic configuration such as welding parameters.

[0169] Step 2: Drive the robotic arm of the welding robot to the starting point of the teaching position. The starting point position should make the center of the line laser camera near the starting point of the weld and outside the weld.

[0170] Step 3: For the first two unit strokes, the robot arm is controlled to move actively through teaching. At the same time, the pulse generator triggers the line laser camera to scan the weld to obtain the straight line point cloud data of the weld. At the same time, the end flange position is obtained for the subsequent vision module to calculate the position of the weld point in the base coordinate system. During the movement, any abnormal situation is detected, such as exceeding the limit area, communication interruption, etc. If any abnormal situation occurs, the system immediately turns off the welding gun laser and shuts down.

[0171] Step 4: When the travel distance reaches the specified value, the straight line point cloud data is packaged and sent to the visual inspection module for processing. It should be noted that when the straight line point cloud data is obtained for the first time, it should be determined whether manual intervention is required to manually select the welding point. If necessary, the line laser scanning is stopped and suspended. After the welding point selection is completed, the robot arm and the line laser camera are restarted, and the visual inspection module is turned on at the same time; the result finally returned by the visual module to the control center can generally include the subsequent path planning of the welding gun (i.e., fitting path), path limit, whether the weld is normal, whether the weld contains the starting point or end point, and other information.

[0172] Step 5. According to the return result of the visual module, the control center determines whether there is any abnormality in the weld; if there is any abnormality, the machine will be shut down immediately and wait for manual processing; if it is the first point cloud, it is determined whether there is a weld starting point, if there is no weld starting point, the machine will be shut down immediately and wait for manual processing; if everything is normal, the path trajectory is converted into a robot arm control code to drive the robot arm to run according to the path; it should be noted that if it is the first point cloud, it should be determined whether the welding gun has reached the weld starting point, and the machine will be shut down after the welding gun reaches the weld starting point, and the welding gun opening operation will be executed to wait for welding gun feedback. If there is no response within a certain period of time, the machine will be shut down according to the abnormal situation. If everything is normal, the robot arm will continue to move and return to step 3 to repeat the processing flow steps 3-5; if the trajectory contains the end point, when the welding gun reaches the weld end point, the entire welding process will be stopped and reset.

[0173] It should be noted that Fig.13 The overall flow chart shown is for example only and does not affect the essential content of the technical solution of the present invention.

[0174] It can be seen from the above description that the welding detection method of this embodiment is mainly aimed at the automatic identification and tracking of welds in the welding process. First, the robot performs teaching rough positioning, and uses the robot to drive the linear laser camera to scan the weld to generate weld point cloud data, so as to achieve accurate positioning of the weld in the robot base coordinate system, solve the position of the welding gun according to the automatically identified weld point cloud tracking point, realize automatic online identification and tracking of the weld, and build a welding robot system based on laser vision, which is mainly composed of calibration module, vision module, operation control module, laser welding gun module, software framework, UI module and other modules. And the weld is accurately detected and identified online and stably tracked through the straight line point cloud data of the weld, and the teaching trajectory deviation can be corrected in real time to plan the motion trajectory of the welding gun, so as to improve the efficiency, quality and accuracy of welding. In addition, the welding detection method of this embodiment has a high degree of intelligence and automation, high recognition sensitivity and accuracy, high automatic tracking stability, and high welding real-time performance. It can meet the production efficiency requirements and can meet the automatic identification and tracking of welds of different weld types and different widths. For example, it can efficiently realize visual inspection and tracking of welds with widths of sub-millimeter and millimeter levels, thereby effectively improving the efficiency and quality of welding workpieces.

[0175] The weld detection device provided by the present invention is described below. The weld detection device described below and the weld detection method described above can be referred to each other.

[0176] Fig.14 is a structural schematic diagram of the weld detection device provided by the present invention, see Fig.14 As shown, the device may include the following modules:

[0177] The acquisition and segmentation module 210 is used to acquire the straight line point cloud data of two adjacent devices to be welded, and segment the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the straight line point cloud data includes a plurality of points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the devices to be welded in the weld area;

[0178] A candidate breakpoint determination module 220 is used to determine a point between two target straight lines from the straight line point cloud data according to the intersection of the two target straight lines or the nearest points adjacent to the two target straight lines, and determine the points between the two target straight lines as multiple candidate breakpoints corresponding to the weld;

[0179] The detection module 230 is used to detect whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold according to the position information of each candidate breakpoint, and determine multiple target breakpoints corresponding to the weld according to the detection result; the distance between each row of target breakpoints is less than the first distance threshold;

[0180] The determination module 240 is used to perform line fitting processing on each target breakpoint according to the position information of each target breakpoint to determine the fitting path corresponding to the weld; the above fitting path is used to guide the welding robot to weld the weld according to the fitting path.

[0181] In one embodiment, the detection module 230 is specifically used to

[0182] For each row of target straight lines, a preset number of sliding windows are used to select the breakpoints on the target straight line, and the coordinate differences of adjacent breakpoints in each sliding window are calculated, and the coordinate differences are summed to obtain the difference sum corresponding to the target straight line; the distance difference between the difference sums of two adjacent rows of target straight lines is calculated, and it is detected whether the distance difference is less than the first distance threshold.

[0183] Optionally, the detection module 230 is specifically used to

[0184] If the detection result is that the distance difference between two adjacent rows of sliding windows is not less than the first distance threshold, it is determined that there are abnormal breakpoints in the two adjacent rows of sliding windows, and the number of abnormal breakpoints is counted; it is determined whether the number of abnormal breakpoints exceeds the preset abnormal data number threshold. If not, the abnormal breakpoints are interpolated to determine multiple target breakpoints corresponding to the weld.

[0185] Optionally, the detection module 230 is specifically configured to output abnormal information if the number of abnormal breakpoints exceeds a threshold value of the number of abnormal data; the abnormal information is used to indicate that an abnormality exists in the weld.

[0186] Optionally, the detection module 230 is specifically used to determine the candidate breakpoints in two adjacent rows of sliding windows as multiple target breakpoints corresponding to the weld if the detection result is that the distance difference between two adjacent rows of sliding windows is less than a first distance threshold.

[0187] In one embodiment, the acquisition and segmentation module 210 is specifically used to

[0188] The straight line point cloud data is segmented and processed to determine multiple candidate straight lines corresponding to the two devices to be welded; each candidate straight line includes at least one point; the density of each candidate straight line is calculated according to the position information of the points on each candidate straight line and the number of points on the candidate straight line; at least one candidate straight line with a smaller density among the candidate straight lines is deleted to determine a target straight line corresponding to each of the two devices to be welded.

[0189] Optionally, the acquisition and segmentation module 210 is specifically used to delete at least one candidate straight line with a smaller density among the candidate straight lines, and determine at least two intermediate straight lines; each intermediate straight line includes at least one point; a first point set is acquired in the straight line point cloud data; the first point set includes multiple first points, and each first point is not a point on each intermediate straight line; each first point in the first point set is traversed, and the first point belonging to each intermediate straight line is selected from each first point and added to the intermediate straight line to obtain each processed intermediate straight line; based on each processed intermediate straight line, a target straight line corresponding to each of the two devices to be welded is determined.

[0190] Optionally, the above-mentioned acquisition and segmentation module 210 is specifically used to calculate, for a first point in the first point set, a first distance and a second distance between the first point and the two endpoints of each intermediate straight line; if the first distance and the second distance between the first point and any intermediate straight line are both greater than a preset endpoint distance threshold, then calculate the third distance between the first point and any intermediate straight line; if the third distance is less than the preset second distance threshold, then add the first point to any intermediate straight line.

[0191] Optionally, the acquisition and segmentation module 210 is specifically used to perform segmented fitting and straight line reprojection processing on each processed intermediate straight line to determine a target straight line corresponding to each of the two devices to be welded.

[0192] In one embodiment, the acquisition and segmentation module 210 is specifically used to

[0193] The unit scanning distance of the line laser camera is determined according to the spatial distance between the line laser camera and the welding gun in the welding robot; the above-mentioned unit scanning distance is smaller than the spatial distance; from the welding starting point to the welding end point of the weld, the line laser camera is used to scan two adjacent devices to be welded in sequence according to the unit scanning distance to obtain the straight line point cloud data of the two adjacent devices to be welded within the corresponding unit scanning distance.

[0194] Optionally, the above device further comprises an interpolation module, the interpolation module is used to

[0195] For each unit scanning distance, from the starting point to the end point of the unit scanning distance, weld detection is performed on the straight line point cloud data of two adjacent devices to be welded within the unit scanning distance to determine the fitting path corresponding to the weld within the unit scanning distance; according to the fitting path of the weld within the unit scanning distance, the teaching path corresponding to the fitting path within the unit scanning distance is interpolated to determine multiple segmented paths corresponding to the weld within the unit scanning distance; each segmented path is a fitting straight line or a fitting curve, which is used to guide the welding robot to weld the welds within the unit scanning distance in sequence according to the above segmented paths.

[0196] In one embodiment, the above device may further include an anomaly detection module, the anomaly detection module is used to

[0197] When the welding robot is welding the weld according to the fitting path, detect whether there are any abnormalities in the welding process; the above abnormalities include that the actual running path of the welding robot exceeds the limit area or the communication abnormalities of the welding robot; if there are any abnormalities, turn off the line laser camera and welding gun in the welding robot.

[0198] It should be noted that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0199] Fig.15 An example of a physical structure diagram of an operation and control system is shown below. Fig.15 As shown, the operation control system may include: a processor 310 , a communications interface 320 , a memory 330 and a communication bus 340 , wherein the processor 310 , the communications interface 320 , and the memory 330 communicate with each other via the communication bus 340 . The processor 310 can call the logic instructions in the memory 330 to execute the weld detection method, which includes: obtaining the straight line point cloud data of two adjacent devices to be welded, and segmenting the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the straight line point cloud data includes multiple points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the device to be welded located in the weld area; according to the intersection of the two target straight lines or the nearest point adjacent to the two target straight lines, the point between the two target straight lines is determined from the straight line point cloud data, and the point between the two target straight lines is determined as a plurality of candidate breakpoints corresponding to the weld; according to the position information of each candidate breakpoint, whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold, and according to the detection result, the plurality of target breakpoints corresponding to the weld are determined; the distance between the target breakpoints in each row is less than the first distance threshold; according to the position information of each target breakpoint, each target breakpoint is subjected to line fitting processing to determine the fitting path corresponding to the weld; the fitting path is used to guide the welding robot to weld the weld according to the fitting path.

[0200] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0201] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the weld detection method provided by the above methods, the method comprising: obtaining straight line point cloud data of two adjacent devices to be welded, and segmenting the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the straight line point cloud data includes a plurality of points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the device to be welded located in the weld area; according to the two target straight lines, The intersection of the straight lines or the nearest point adjacent to two target straight lines, the point between the two target straight lines is determined from the straight line point cloud data, and the point between the two target straight lines is determined as a plurality of candidate breakpoints corresponding to the weld; according to the position information of each candidate breakpoint, whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold is detected, and a plurality of target breakpoints corresponding to the weld are determined according to the detection result; the distance between each row of target breakpoints is less than the first distance threshold; line fitting processing is performed on each target breakpoint according to the position information of each target breakpoint to determine the fitting path corresponding to the weld; the above-mentioned fitting path is used to guide the welding robot to weld the weld according to the fitting path.

[0202] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to execute the weld detection method provided by the above methods, the method comprising: obtaining straight line point cloud data of two adjacent devices to be welded, and segmenting the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the above straight line point cloud data includes multiple points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the device to be welded located in the weld area; according to the intersection of the two target straight lines or the nearest points adjacent to the two target straight lines, the points located between the two target straight lines are determined from the straight line point cloud data, and the points located between the two target straight lines are determined as multiple candidate breakpoints corresponding to the weld; according to the position information of each candidate breakpoint, whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold, and according to the detection result, multiple target breakpoints corresponding to the weld are determined; the distance between each row of target breakpoints is less than the first distance threshold; according to the position information of each target breakpoint, each target breakpoint is subjected to line fitting processing to determine the fitting path corresponding to the weld; the above fitting path is used to guide the welding robot to weld the weld according to the fitting path.

[0203] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0204] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A weld detection method, characterized in that: include: Obtain straight line point cloud data of two adjacent devices to be welded, and segment the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the straight line point cloud data includes a plurality of points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the devices to be welded in the weld area; According to the intersection of the two target straight lines or the nearest points adjacent to the two target straight lines, a point between the two target straight lines is determined from the straight line point cloud data, and the point between the two target straight lines is determined as a plurality of candidate breakpoints corresponding to the weld; According to the position information of each of the candidate breakpoints, detecting whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold, and determining a plurality of target breakpoints corresponding to the weld according to the detection result; the distance between the target breakpoints in each row is less than the first distance threshold; Performing line fitting processing on each target breakpoint according to the position information of each target breakpoint to determine a fitting path corresponding to the weld; the fitting path is used to guide the welding robot to weld the weld according to the fitting path; During the process of the welding robot welding the weld according to the fitting path, detecting whether there is an abnormality in the welding process; the abnormality includes that the actual running path of the welding robot exceeds the limit area or the communication of the welding robot is abnormal; If there is an abnormality, the line laser camera and the welding gun in the welding robot are turned off; The welding robot comprises a mechanical arm, a line laser camera and a welding gun, wherein the line laser camera and the welding gun are both arranged on the mechanical arm, and the method further comprises: Calibrate the tool coordinate system posture of the welding robot; The hand-eye calibration of the line laser camera of the welding robot is performed; specifically, the following steps are performed: controlling the robotic arm so that the line laser camera shoots a fixed point P in space in different postures, calculating the position of point P in the coordinate system of the line laser camera, and recording the position and posture of the flange at the end of the robotic arm in each posture. By utilizing the characteristic that the position of point P relative to the base coordinate system remains unchanged, the transformation matrix from the coordinate system of the line laser camera to the coordinate system of the flange at the end of the robotic arm is calculated, so as to realize the calibration of the hand-eye system of the line laser camera; The overall process of the welding robot welding the weld includes: Step 1: First, wait for the two welding devices to be in place, take out a new welding task from the task queue, and adjust the basic configuration according to the welding type; the basic configuration includes welding parameters; Step 2, driving the mechanical arm of the welding robot to reach the starting point of the teaching position, and the starting point position should make the center of the line laser camera located near the starting point of the weld and outside the weld; Step 3: For the first two unit strokes, the robot arm is controlled to move actively by teaching, and the pulse generator triggers the line laser camera to scan the weld to obtain the weld straight line point cloud data, and the end flange position is obtained for the subsequent visual module to calculate the position of the weld point in the base coordinate system; Step 4. When the travel distance reaches the specified value, the straight line point cloud data is packaged and sent to the visual inspection module for processing; when the straight line point cloud data is obtained for the first time, it should be determined whether manual intervention is required to manually select the welding point. If necessary, the machine is stopped and the line laser scanning is suspended. After the welding point selection is completed, the robot arm and the line laser camera are restarted, and the visual inspection module is turned on at the same time; the result finally returned to the control center by the visual module generally includes the subsequent fitting path of the welding gun, the path limit, whether the weld is normal, and whether the weld contains the starting point or end point information; Step 5. According to the return result of the visual module, the control center determines whether there is an abnormality in the weld; if there is an abnormality, the machine will be shut down immediately and wait for manual processing; if it is the first point cloud, it is determined whether there is a weld starting point, and if there is no weld starting point, the machine will be shut down immediately and wait for manual processing; if everything is normal, the trajectory of the fitting path is converted into a robot arm control code to drive the robot arm to run according to the path; if it is the first point cloud, it should be determined whether the welding gun has reached the weld starting point, and the machine will be shut down after the welding gun reaches the weld starting point, and the welding gun opening operation will be executed to wait for welding gun feedback. If there is no response within a certain period of time, the machine will be shut down according to the abnormality. If everything is normal, the robot arm will continue to move and return to step 3, and repeat the processing flow steps 3-5; if the trajectory of the fitting path contains the end point, when the welding gun reaches the weld end point, the entire welding process will be stopped and reset; The calibration of the tool coordinate system posture of the welding robot includes tool center point position calibration (TCP) And tool coordinate system posture calibration (TCF) ; The tool center point calibration (TCP) The steps are as follows: (1) The welding robot tool welding gun tip changes its posture and rotates around a fixed point multiple times, and each rotation is recorded ; Represents the transformation matrix between the base coordinate B and the coordinate system E where the end flange surface is located; (2) Calculate ; Represents the positional relationship between the tool coordinate system and the welding robot end coordinate system. For the i-th posture transformation, refer to the following formula: ； Where i represents the i-th posture change; n represents the total number of posture changes; Represents the transformation matrix between the tool coordinate system and the base coordinate system; It represents the calibration relationship between the base coordinates and the coordinate system E where the end flange surface is located; Indicates the calibration relationship between the base coordinate system and the tool coordinate system; The rotation matrix representing the transformation between the base coordinates and the end flange; and is a known quantity, which can be obtained by the above formula ; The tool coordinate system posture calibration (TCF) Here are the steps: (1) Obtain the tool coordinate system (TCP) by calibrating the tool center point position The position of the TCF is calculated by calibrating the z / x direction. During this process, the TCF posture remains unchanged. (2) The tool coordinate system TCP calibration generally takes four postures. When calibrating TCF, the fourth posture calibration point is generally taken as position point 1. The position point 1 is expressed as ; The robot starts from position 1 and moves a certain distance in the +x direction to position 2, which is represented by ; The robot starts from position 1 and moves a certain distance in the +z direction to position 3, which is represented by ; Then the x-axis axial vector X of the tool coordinate system T is expressed by the following formula: ; Similarly, the z-axis axial vector Z of the tool coordinate system T can be obtained, which is expressed by the following formula: ; Then, the y-axis axial vector Y of the tool coordinate system T is obtained by the right-hand rule, which is expressed by the following formula: Y=Z×X; then XYZ is normalized to obtain X'Y'Z', and then the posture of the tool coordinate T relative to the base coordinate B is obtained : ; The rotation matrix of the end coordinate system E is ,and: ; (3) Combining the above formulas, we can get the rotation matrix of the tool coordinate system: , which is expressed as follows: 。 2. The weld detection method according to claim 1, characterized in that: The detecting, based on the position information of each of the candidate breakpoints, whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold comprises: Using a preset number of sliding windows to select each of the candidate breakpoints in a row, calculating the coordinate difference between adjacent candidate breakpoints in each sliding window, and summing the coordinate differences to obtain the sum of the differences corresponding to each row of the sliding windows; The distance difference between the difference values ​​and the distance difference between two adjacent rows of the sliding windows is calculated, and it is detected whether the distance difference is less than the first distance threshold.

3. The weld detection method according to claim 2, characterized in that: The step of determining a plurality of target breakpoints corresponding to the weld according to the detection results includes: If the detection result is that the distance difference between the two adjacent rows of sliding windows is not less than the first distance threshold, it is determined that there are abnormal breakpoints in the two adjacent rows of sliding windows, and the number of the abnormal breakpoints is counted; It is determined whether the number of the abnormal breakpoints exceeds a preset abnormal data quantity threshold; if not, the abnormal breakpoints are interpolated to determine a plurality of target breakpoints corresponding to the weld.

4. The weld detection method according to claim 3, characterized in that: The method further comprises: If the number of the abnormal breakpoints exceeds the abnormal data quantity threshold, abnormal information is output; the abnormal information is used to indicate that the weld is abnormal.

5. The weld detection method according to any one of claims 2 to 4, characterized in that: The step of determining a plurality of target breakpoints corresponding to the weld according to the detection results includes: If the detection result is that the distance difference between two adjacent rows of the sliding windows is less than the first distance threshold, the candidate breakpoints in the two adjacent rows of the sliding windows are determined as multiple target breakpoints corresponding to the weld.

6. The weld detection method according to any one of claims 1 to 4, characterized in that: The segmentation process of the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded includes: Segmenting the straight line point cloud data to determine a plurality of candidate straight lines corresponding to the two devices to be welded; each of the candidate straight lines includes at least one point; Calculating the density of each candidate straight line according to the position information of the points on each candidate straight line and the number of the points on the candidate straight line; At least one candidate straight line with a smaller density among the candidate straight lines is deleted to determine a target straight line corresponding to each of the two devices to be welded.

7. The weld detection method according to claim 6, characterized in that: The step of deleting at least one candidate straight line with a smaller density among the candidate straight lines to determine a target straight line corresponding to each of the two devices to be welded comprises: At least one candidate straight line with a smaller density is deleted from the candidate straight lines to determine at least two intermediate straight lines; each of the intermediate straight lines includes at least one point; Acquire a first point set in the straight line point cloud data; the first point set includes a plurality of first points, and each of the first points is not a point on each of the intermediate straight lines; Traversing each of the first points in the first point set, selecting first points belonging to each of the intermediate straight lines from each of the first points and adding the first points to the intermediate straight lines, so as to obtain each processed intermediate straight line; According to each of the processed intermediate straight lines, a target straight line corresponding to each of the two devices to be welded is determined.

8. The weld detection method according to claim 7, characterized in that: The step of selecting the first points belonging to the intermediate straight lines from the first points includes: For a first point in the first point set, calculating a first distance and a second distance between the first point and two end points of each of the intermediate straight lines; If the first distance and the second distance between the first point and any intermediate straight line are both greater than a preset endpoint distance threshold, calculating a third distance between the first point and any intermediate straight line; If the third distance is less than a preset second distance threshold, the first point is added to any one of the intermediate straight lines.

9. The weld detection method according to claim 7, characterized in that: Determining a target straight line corresponding to each of the two devices to be welded according to each of the processed intermediate straight lines comprises: Each processed intermediate straight line is subjected to segmented fitting and straight line reprojection processing to determine a target straight line corresponding to each of the two devices to be welded.

10. The weld detection method according to any one of claims 1 to 4, characterized in that: The step of obtaining straight line point cloud data of two adjacent devices to be welded includes: Determine a unit scanning distance of the line laser camera according to the spatial distance between the line laser camera and the welding gun in the welding robot; the unit scanning distance is smaller than the spatial distance; From the welding start point to the welding end point of the weld, the line laser camera sequentially scans the two adjacent devices to be welded according to the unit scanning distance to obtain straight line point cloud data of the two adjacent devices to be welded within the corresponding unit scanning distance.

11. The weld detection method according to claim 10, characterized in that: The method further comprises: For each of the unit scanning distances, from the distance starting point to the distance ending point of the unit scanning distance, weld detection is performed on the straight line point cloud data of the two adjacent devices to be welded within the unit scanning distance to determine a fitting path corresponding to the weld within the unit scanning distance; According to the fitting path of the weld within the unit scanning distance, the teaching path corresponding to the fitting path within the unit scanning distance is interpolated to determine a plurality of segmented paths corresponding to the weld within the unit scanning distance; each segmented path is a fitting straight line or a fitting curve, which is used to guide the welding robot to weld the weld within the unit scanning distance in sequence according to each segmented path.

12. A weld detection device, characterized in that: include: An acquisition and segmentation module is used to acquire straight line point cloud data of two adjacent devices to be welded, and segment the straight line point cloud data to determine a target straight line corresponding to each of the two devices to be welded; the straight line point cloud data includes a plurality of points, there is a weld between the two adjacent devices to be welded, and the target straight line is used to characterize the outer contour of the devices to be welded in the weld area; A candidate breakpoint determination module, used to determine a point between two target straight lines from the straight line point cloud data according to an intersection of the two target straight lines or the nearest points adjacent to the two target straight lines, and determine the points between the two target straight lines as a plurality of candidate breakpoints corresponding to the weld; A detection module, configured to detect whether the distance between two adjacent rows of candidate breakpoints is less than a preset first distance threshold according to the position information of each of the candidate breakpoints, and determine a plurality of target breakpoints corresponding to the weld according to the detection result; The distance between the target breakpoints in each row is less than the first distance threshold; A determination module, used to perform line fitting processing on each target breakpoint according to the position information of each target breakpoint, and determine a fitting path corresponding to the weld; the fitting path is used to guide the welding robot to weld the weld according to the fitting path; The device further comprises an abnormality detection module, wherein the abnormality detection module is used to detect whether there is an abnormality in the welding process when the welding robot welds the weld according to the fitting path; If there is an abnormality, the line laser camera and welding gun in the welding robot are turned off; the abnormality includes that the actual running path of the welding robot exceeds the limit area or the communication of the welding robot is abnormal; The welding robot comprises a mechanical arm, a line laser camera and a welding gun, wherein the line laser camera and the welding gun are both arranged on the mechanical arm, and the device further comprises: A calibration module is used to calibrate the tool coordinate system posture of the welding robot; and to perform hand-eye calibration on the line laser camera of the welding robot; specifically, the module includes: controlling the robotic arm so that the line laser camera shoots a fixed point P in space in different postures, calculating the position of point P in the line laser camera coordinate system, and recording the position and posture of the flange at the end of the robotic arm in each posture. By using the characteristic that the position of point P relative to the base coordinate system remains unchanged, the module solves the transformation matrix from the line laser camera coordinate system to the flange coordinate system at the end of the robotic arm, and realizes the calibration of the hand-eye system of the line laser camera; The overall process of the welding robot welding the weld includes: Step 1: First, wait for the two welding devices to be in place, take out a new welding task from the task queue, and adjust the basic configuration according to the welding type; the basic configuration includes welding parameters; Step 2, driving the mechanical arm of the welding robot to reach the starting point of the teaching position, and the starting point position should make the center of the line laser camera located near the starting point of the weld and outside the weld; Step 3: For the first two unit strokes, the robot arm is controlled to move actively by teaching, and the pulse generator triggers the line laser camera to scan the weld to obtain the weld straight line point cloud data, and the end flange position is obtained for the subsequent visual module to calculate the position of the weld point in the base coordinate system; Step 4. When the travel distance reaches the specified value, the straight line point cloud data is packaged and sent to the visual inspection module for processing; when the straight line point cloud data is obtained for the first time, it should be determined whether manual intervention is required to manually select the welding point. If necessary, the machine is stopped and the line laser scanning is suspended. After the welding point selection is completed, the robot arm and the line laser camera are restarted, and the visual inspection module is turned on at the same time; the result finally returned to the control center by the visual module generally includes the subsequent fitting path of the welding gun, the path limit, whether the weld is normal, and whether the weld contains the starting point or end point information; Step 5. According to the return result of the visual module, the control center determines whether there is an abnormality in the weld; if there is an abnormality, the machine will be shut down immediately and wait for manual processing; if it is the first point cloud, it is determined whether there is a weld starting point, and if there is no weld starting point, the machine will be shut down immediately and wait for manual processing; if everything is normal, the trajectory of the fitting path is converted into a robot arm control code to drive the robot arm to run according to the path; if it is the first point cloud, it should be determined whether the welding gun has reached the weld starting point, and the machine will be shut down after the welding gun reaches the weld starting point, and the welding gun opening operation will be executed to wait for welding gun feedback. If there is no response within a certain period of time, the machine will be shut down according to the abnormality. If everything is normal, the robot arm will continue to move and return to step 3, and repeat the processing flow steps 3-5; if the trajectory of the fitting path contains the end point, when the welding gun reaches the weld end point, the entire welding process will be stopped and reset; The calibration of the tool coordinate system posture of the welding robot includes tool center point position calibration (TCP) And tool coordinate system posture calibration (TCF) ; The tool center point calibration (TCP) The steps are as follows: (1) The welding robot tool welding gun tip changes its posture and rotates around a fixed point multiple times, and each rotation is recorded ; Represents the transformation matrix between the base coordinate B and the coordinate system E where the end flange surface is located; (2) Calculate ; Represents the positional relationship between the tool coordinate system and the welding robot end coordinate system. For the i-th posture transformation, refer to the following formula: ； Where i represents the i-th posture change; n represents the total number of posture changes; Represents the transformation matrix between the tool coordinate system and the base coordinate system; It represents the calibration relationship between the base coordinates and the coordinate system E where the end flange surface is located; Indicates the calibration relationship between the base coordinate system and the tool coordinate system; The rotation matrix representing the transformation between the base coordinates and the end flange; and is a known quantity, which can be obtained by the above formula ; The tool coordinate system posture calibration (TCF) Here are the steps: (1) Obtain the tool coordinate system (TCP) by calibrating the tool center point position The position of the TCF is calculated by calibrating the z / x direction. During this process, the TCF posture remains unchanged. (2) The tool coordinate system TCP calibration generally takes four postures. When calibrating TCF, the fourth posture calibration point is generally taken as position point 1. The position point 1 is expressed as ; The robot starts from position 1 and moves a certain distance in the +x direction to position 2, which is represented by ; The robot starts from position 1 and moves a certain distance in the +z direction to position 3, which is represented by ; Then the x-axis axial vector X of the tool coordinate system T is expressed by the following formula: ; Similarly, the z-axis axial vector Z of the tool coordinate system T can be obtained, which is expressed by the following formula: ; Then, the y-axis axial vector Y of the tool coordinate system T is obtained by the right-hand rule, which is expressed by the following formula: Y=Z×X; then XYZ is normalized to obtain X'Y'Z', and then the posture of the tool coordinate T relative to the base coordinate B is obtained : ; The rotation matrix of the end coordinate system E is ,and: ; (3) Combining the above formulas, we can get the rotation matrix of the tool coordinate system: , which is expressed as follows: 。 13. A transport control system, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the weld detection method according to any one of claims 1 to 11 is implemented.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the weld detection method according to any one of claims 1 to 11 is implemented.

Images