Robot spot welding splash detection method based on visual identification, terminal and storage medium

Abstract

The invention belongs to the technical field of vehicle production detection, and particularly provides a robot spot welding splash detection method based on visual identification, a terminal and a storage medium, and the method comprises the steps that a visual controller recommends a key monitoring area according to received spot welding position data of each robot, and sends the key monitoring area to a visual sensor; the vision sensor monitors the key monitoring area according to the received position information of the key monitoring area and a preset detection period until a spot welding ending signal of the robot is received; the visual sensor transmits monitoring image data back to the visual controller after receiving a spot welding ending signal; the visual controller analyzes the received monitoring image data and judges whether the spot welding splashing phenomenon occurs in the monitoring image data or not; the automatic detection process reduces the workload of manual statistics and improves the detection efficiency; a visual sensor and an intelligent analysis algorithm are used for accurately recognizing the splashing phenomenon, and the welding quality is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle production detection, and in particular relates to a robot spot welding spatter detection method based on visual recognition, a terminal and a storage medium. Background Art

[0002] In the automotive manufacturing industry, the manufacture of body-in-white (BIW) is an extremely critical link. Currently, the manufacture of most automotive BIW is based on steel stampings. Various stampings are precisely combined and connected through welding and other methods to ultimately form a complete body structure. In this process, spot welding is widely used in the BIW welding process due to its advantages such as simple operation, low welding cost, high production efficiency and small welding deformation. In particular, resistance spot welding, as a major method of spot welding, achieves welding through the cooperation of a movable arm and a stationary arm, utilizing the resistance heat generated when current passes through the workpiece. It has become the mainstream technology in the automotive BIW welding process.

[0003] However, when robots perform spot welding, spatter often occurs due to various factors, including welding parameters, material properties, and welding conditions. Spatter not only affects weld quality but can also damage welding equipment and the surrounding environment. Therefore, effective detection and control of spot weld spatter is crucial.

[0004] Currently, the commonly used methods for detecting spatter from robotic spot welding in the industry primarily include manual counting and online detection and recognition. While manual counting methods can identify spatter to a certain extent, they cannot provide comprehensive statistical analysis of the entire process. Furthermore, they are labor-intensive and inefficient, making them difficult to meet the demands of large-scale production. While online detection and recognition methods can cover the entire welding process, their accuracy is often low, and they are prone to misjudgments or omissions, leading to a decline in welding quality. Therefore, an intelligent method that can efficiently and accurately detect spatter from robotic spot welding throughout the entire process is urgently needed to improve the welding quality and production efficiency of body-in-white (BIW) welding. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a robot spot welding spatter detection method, terminal and storage medium based on visual recognition to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a robot spot welding spatter detection method based on visual recognition, comprising: After the workstation is started, the corresponding spot welding instructions are sent to all robots under the workstation; The robots move to their respective target welding positions according to the received spot welding instructions; After the robot reaches the target welding position, it sends the spot welding parameters and robot spot welding position data to the vision controller; The visual controller recommends key monitoring areas based on the received spot welding position data of each robot, and sends the position information of the recommended key monitoring areas to the visual sensor; The visual sensor monitors the key monitoring area according to the received position information of the key monitoring area according to the preset detection cycle until it receives the robot spot welding end signal; After receiving the spot welding end signal, the visual sensor transmits the monitoring image data back to the visual controller; The visual controller analyzes the received monitoring image data to determine whether spot welding spatter occurs in the monitoring image data; If yes, the splash indicator light is started to flash, and the starting position coordinates of the splash phenomenon are calculated based on the monitoring image data; Save the splash data including the coordinates of the starting position of the splash phenomenon to the database.

[0007] A further improvement of the technical solution is that the spot welding parameters include spot welding current, spot welding pressure, spot welding time and vehicle body sheet thickness.

[0008] Further improvements to this technical solution include that the spatter data includes the name of the weld point where the spatter phenomenon starts, the thickness of the vehicle body panel, and the robot's spot welding current, spot welding pressure, and spot welding time when the spatter phenomenon occurs. The visual controller counts the spatter data, counts the spatter frequency of each spatter weld point, and saves it in a database.

[0009] A further improvement of this technical solution is that the visual controller recommends key monitoring areas based on the received spot welding position data of each robot, and the method specifically includes: Obtain the corresponding spatter frequencies from the database according to the received spot welding position data of each robot, and sort the obtained spatter frequencies; The vision controller uses the area within a preset range around the solder joint with the highest spatter frequency as the key monitoring area.

[0010] A further improvement of the technical solution is that the visual controller analyzes the received monitoring image data to determine whether spot welding spatter occurs in the monitoring image data. The method specifically includes: Obtain all image frames with highlighted pixels within a preset time in the key monitoring area; Perform grayscale processing on each frame of image to generate the corresponding grayscale image; Traverse each pixel in the grayscale image and determine whether the grayscale value of the pixel exceeds the preset grayscale threshold; If so, the pixels whose grayscale values ​​exceed the preset grayscale threshold are regarded as highlight pixels, and it is determined that spot welding spatter exists in the monitoring image data.

[0011] A further improvement of this technical solution is to calculate the starting position coordinates of the splash phenomenon based on the monitoring image data, and the method specifically includes: Perform cluster analysis on the highlighted pixels in the surveillance image data and divide them into several groups; For each group, a sector-shaped area is fitted based on the position data of the highlighted pixels using the least squares method; The coordinates (Sx, Sy) of the center point S of the sector area are calculated according to the fitted sector area parameters, and the center point S is determined as the starting position of the highlighted pixel points in the corresponding sector area.

[0012] A further improvement of this technical solution is that before performing cluster analysis on the highlighted pixels in the monitoring image data, the following steps are also included: Perform median filtering on the highlighted pixels in the key monitoring area to remove isolated noise points; Using connected domain analysis, the remaining highlighted pixels are grouped into connected regions, and the area corresponding to each connected region is calculated; Remove connected regions whose area is smaller than the preset area threshold.

[0013] Further improvements to this technical solution also include: The vision controller compares the coordinates (Sx, Sy) of the center point S with the robot spot welding position data corresponding to the key monitoring area to determine whether the center point S is within the preset range around the robot spot welding position; If yes, the robot spot welding position data and corresponding spot welding parameters are stored in the database, and the spatter indicator light is controlled to flash red once; If not, the coordinates (Sx, Sy) of the center point S are compared with the spot welding position data of other robots outside the key monitoring area to find the best fitting weld point, and the robot spot welding position data and spot welding parameters of the best fitting weld point found are stored in the database.

[0014] In a second aspect, a terminal is provided, including: processor, memory, wherein The memory is used to store computer programs, The processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned terminal method.

[0015] According to a third aspect, a computer storage medium is provided, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the methods described in the above aspects.

[0016] The beneficial effects of the present invention are: Improved testing efficiency: The automated testing process reduces the workload of manual statistics, significantly improving testing efficiency and meeting the needs of large-scale production.

[0017] Improve detection accuracy: Utilize visual sensors and intelligent analysis algorithms to accurately identify spatter phenomena, reduce misjudgment and missed detection rates, and ensure welding quality.

[0018] Real-time feedback and monitoring: The splash indicator light provides instant feedback of test results, while monitoring image data provides strong support for quality traceability and process optimization.

[0019] Optimize production processes: By saving spatter data to the database, it is easy to analyze and improve later, optimize welding parameters and process flow, and further improve production efficiency and product quality.

[0020] In addition, the present invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0023] Figure 2 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0026] like Figure 1 As shown, the present invention provides a robot spot welding spatter detection method based on visual recognition, comprising: Step 110: After the workstation is started, corresponding spot welding instructions are sent to all robots under the workstation; Step 120 , the robots move to their respective target welding positions according to the received spot welding instructions; Step 130 , after the robot reaches the target welding position, the robot sends the spot welding parameters and the robot spot welding position data to the vision controller; Step 140: The vision controller recommends a key monitoring area based on the received spot welding position data of each robot, and sends the position information of the recommended key monitoring area to the vision sensor; Step 150 , the visual sensor monitors the key monitoring area according to the received position information of the key monitoring area according to a preset detection cycle until a robot spot welding end signal is received; Step 160 , after receiving the spot welding end signal, the visual sensor transmits the monitoring image data back to the visual controller; Step 170: The visual controller analyzes the received monitoring image data to determine whether spot welding spatter occurs in the monitoring image data; if so, the process proceeds to step 180; Step 180: Start the splash indicator light to flash, and calculate the starting position coordinates of the splash phenomenon based on the monitoring image data; Step 190: Save the splash data including the coordinates of the starting position of the splash phenomenon to a database.

[0027] Specifically, the spot welding parameters include spot welding current, spot welding pressure, spot welding time and body panel thickness; the spatter data includes the name of the weld point where the spatter phenomenon starts, the thickness of the body panel and the robot's spot welding current, spot welding pressure and spot welding time when the spatter phenomenon occurs. The visual controller counts the spatter data, counts the spatter frequency of each spatter weld point position, and saves it in the database.

[0028] The visual controller recommends key monitoring areas based on the received spot welding position data of each robot. The method specifically includes: S151. Obtain corresponding spatter frequencies from a database according to the received spot welding position data of each robot, and sort the obtained spatter frequencies; S152. The vision controller uses the area within a preset range around the solder joint with the highest spatter frequency as a key monitoring area.

[0029] In addition, the visual controller analyzes the received monitoring image data to determine whether spot welding spatter occurs in the monitoring image data. The method specifically includes: S171. Acquire all image frames in the key monitoring area where highlighted pixels appear within a preset time period; S172, performing grayscale processing on each frame of image to generate a corresponding grayscale image; S173, traverse each pixel in the grayscale image and determine whether the grayscale value of the pixel exceeds a preset grayscale threshold; if so, go to S174; S174: Pixels whose grayscale values ​​exceed a preset grayscale threshold are considered as highlight pixels, and it is determined that spot welding spatter exists in the monitoring image data.

[0030] In addition, the starting position coordinates of the splash phenomenon are calculated based on the monitoring image data, and the method specifically includes: S181, performing median filtering on the highlighted pixels in the key monitoring area to remove isolated noise points; S182, using connected domain analysis, grouping the remaining highlighted pixels into connected regions, and calculating the area corresponding to each connected region; S183, removing connected regions whose areas are smaller than a preset area threshold; S184, performing cluster analysis on the highlighted pixels in the connected area of ​​the surveillance image data, and dividing them into several groups; S185. For each group, use the least squares method to fit a sector-shaped area based on the position data of the highlighted pixels; S186 , calculating the coordinates (Sx, Sy) of the center point S of the sector-shaped region according to the fitted parameters of the sector-shaped region, and determining the center point S as the starting position of the highlighted pixel point in the corresponding sector-shaped region.

[0031] In order to improve the detection accuracy, the detection method further includes: The vision controller compares the coordinates (Sx, Sy) of the center point S with the robot spot welding position data corresponding to the key monitoring area to determine whether the center point S is within the preset range around the robot spot welding position; If yes, the robot spot welding position data and corresponding spot welding parameters are stored in the database, and the spatter indicator light is controlled to flash red once; If not, the coordinates (Sx, Sy) of the center point S are compared with the spot welding position data of other robots outside the key monitoring area to find the best-fit weld point. The robot spot welding position data and spot welding parameters of the best-fit weld point found are stored in the database. The best-fit weld point is found by comparing all welds in the area with the spatter S (Sx, Sy) within 5 seconds before and after the spatter is emitted. The weld with the smallest distance of less than 50mm is selected as the best-fit point.

[0032] For example, first, based on the welding point position data, welding time and other data provided by one or more robots to the visual controller, the key monitoring area within a period of time is calculated. The specific method is as follows: Assume the welding points are P1, P2, P3...Pn (n is the number of welding points given by the robot to the vision controller in a period of time); Assume that the horizontal and vertical coordinates of the welding points are (P1x, P1y), (P2x, P2y), (P3x, P3y) ... (Pnx, Pny) respectively. Then the calculation formula of the monitoring focus F coordinate (Fx, Fy) is: Fx = (P1x + P2x + P3x + ... + Pnx) / n, Fy = (P1y + P2y + P3y + ... + Pnx) / n; The calculation formula of the distance between the welding point and the monitoring focus F is: ; ; ;

[0033] .

[0034] Assume that Dmax is the maximum value among D1 to Dn, and the adjustment variable of the key monitoring area is V (range 200-1000mm). Then the key monitoring area is a circular area with the monitoring focus F (Fx, Fy) as the center and Dmax+V as the radius.

[0035] During the first 100 detections and monitoring cycles, the algorithm performs a weighted calculation based on splash detection accuracy (75% weight) and recognition efficiency (25% weight). It then randomly adjusts the value of variable V within a specified range according to a pre-set rule. The V value corresponding to the highest weighted value is selected for use in calculating the key monitoring areas. After 100 cumulative calculations, the V value is no longer adjusted. After every 50 cycles, the V value is randomly adjusted within a specified range according to a pre-set rule, and the V value corresponding to the highest weighted value is continuously updated.

[0036] Figure 2 This is a structural diagram of a terminal 200 provided in an embodiment of the present invention. The terminal 200 can be used to execute the robot spot welding spatter detection method based on visual recognition provided in an embodiment of the present invention.

[0037] The terminal 200 may include a processor 210, a memory 220, and a communication module 230. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention. The server structure may be a bus structure or a star structure, and may include more or fewer components than shown, or may combine certain components or arrange the components differently.

[0038] The memory 220 can be used to store execution instructions of the processor 210. The memory 220 can be implemented by any type of volatile or non-volatile storage terminal, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 220 are executed by the processor 210, the terminal 200 can perform some or all of the steps in the above-described method embodiments.

[0039] The processor 210 is the control center of the storage terminal. It uses various interfaces and lines to connect various parts of the entire electronic terminal. It executes various functions of the electronic terminal and / or processes data by running or executing software programs and / or modules stored in the memory 220, and calling data stored in the memory. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 210 can only include a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0040] The communication module 230 is used to establish a communication channel so that the storage terminal can communicate with other terminals, receive user data sent by other terminals, or send user data to other terminals.

[0041] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program that, when executed, may include some or all of the steps of each embodiment provided herein. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0042] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media capable of storing program code, and includes instructions for causing a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0043] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0044] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, and can be electrical, mechanical or other forms.

[0045] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0046] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0047] Although the present invention has been described in detail with reference to the accompanying drawings and in conjunction with preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, persons of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and such modifications or substitutions shall be within the scope of the present invention. Any changes or substitutions that can be easily conceived by persons skilled in the art within the technical scope disclosed in the present invention shall be within the scope of protection of the present invention.

Claims

1. A robot spot welding spatter detection method based on visual recognition, characterized in that: include: After the workstation is started, the corresponding spot welding instructions are sent to all robots under the workstation; The robots move to their respective target welding positions according to the received spot welding instructions; After the robot reaches the target welding position, it sends the spot welding parameters and the robot spot welding position data to the visual controller; The visual controller recommends a key monitoring area based on the received spot welding position data of each robot, and sends the position information of the recommended key monitoring area to the visual sensor; The visual sensor monitors the key monitoring area according to the received position information of the key monitoring area according to the preset detection cycle until receiving the robot spot welding end signal; After receiving the spot welding end signal, the visual sensor transmits the monitoring image data back to the visual controller; The visual controller analyzes the received monitoring image data to determine whether spot welding spatter occurs in the monitoring image data; If yes, the splash indicator light is started to flash, and the starting position coordinates of the splash phenomenon are calculated based on the monitoring image data; The splash data including the coordinates of the starting position of the splash phenomenon is saved to the database.

2. The robot spot welding spatter detection method based on visual recognition according to claim 1 is characterized in that: Spot welding parameters include spot welding current, spot welding pressure, spot welding time and body panel thickness.

3. The robot spot welding spatter detection method based on visual recognition according to claim 1 is characterized in that: The spatter data includes the name of the weld point where the spatter phenomenon starts, the thickness of the body panel, and the robot's spot welding current, spot welding pressure, and spot welding time when the spatter phenomenon occurs. The visual controller counts the spatter data, the spatter frequency of each spatter weld point, and saves it in the database.

4. The robot spot welding spatter detection method based on visual recognition according to claim 3 is characterized in that: The visual controller recommends key monitoring areas based on the received spot welding position data of each robot, and the method specifically includes: Acquire the corresponding spatter frequencies from the database according to the received spot welding position data of each robot, and sort the acquired spatter frequencies; The vision controller uses the area within a preset range around the solder joint with the highest spatter frequency as the key monitoring area.

5. The robot spot welding spatter detection method based on visual recognition according to claim 1 is characterized in that: The visual controller analyzes the received monitoring image data to determine whether spot welding spatter occurs in the monitoring image data. The method specifically includes: Obtain all image frames with highlighted pixels within a preset time in the key monitoring area; Perform grayscale processing on each frame of image to generate a corresponding grayscale image; Traverse each pixel in the grayscale image to determine whether the grayscale value of the pixel exceeds a preset grayscale threshold; If so, the pixels whose grayscale values ​​exceed the preset grayscale threshold are regarded as highlight pixels, and it is determined that spot welding spatter exists in the monitoring image data.

6. The robot spot welding spatter detection method based on visual recognition according to claim 5 is characterized in that: The method for calculating the starting position coordinates of the splash phenomenon according to the monitoring image data specifically includes: Perform cluster analysis on the highlighted pixels in the surveillance image data and divide them into several groups; For each group, a sector-shaped area is fitted based on the position data of the highlighted pixels using the least squares method; The coordinates (Sx, Sy) of the center point S of the fan-shaped area are calculated according to the fitted fan-shaped area parameters, and the center point S is determined as the starting position of the highlighted pixel point in the corresponding fan-shaped area.

7. The robot spot welding spatter detection method based on visual recognition according to claim 6 is characterized in that: Before clustering analysis of highlight pixels in surveillance image data, the following steps are also included: Perform median filtering on the highlighted pixels in the key monitoring area to remove isolated noise points; Using connected domain analysis, the remaining highlighted pixels are grouped into connected regions, and the area corresponding to each connected region is calculated; Remove connected regions whose area is smaller than the preset area threshold.

8. The robot spot welding spatter detection method based on visual recognition according to claim 6 is characterized in that: Also includes: The visual controller compares the coordinates (Sx, Sy) of the center point S with the robot spot welding position data corresponding to the key monitoring area to determine whether the center point S is within the preset range around the robot spot welding position; If yes, the robot spot welding position data and corresponding spot welding parameters are stored in the database, and the spatter indicator light is controlled to flash red once; If not, the coordinates (Sx, Sy) of the center point S are compared with the spot welding position data of other robots outside the key monitoring area to find the best fitting weld point, and the robot spot welding position data and spot welding parameters of the best fitting weld point found are stored in the database.

9. A terminal, characterized in that: include: processor; A memory for storing execution instructions of the processor; The processor is configured to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images