Method, device, computer and storage medium for determining weld quality inspection area

Abstract

An embodiment of the present invention provides a method, device, computer and storage medium for determining a weld quality inspection area, the method comprising the following steps: acquiring a two-dimensional welding image and a three-dimensional welding image of a target workpiece; performing welding smoke judgment processing on the two-dimensional welding image to obtain a welding smoke judgment result; performing recognition processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image; performing calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix; mapping the weld edge image onto the three-dimensional welding image according to the mapping relationship matrix to obtain a three-dimensional composite image; performing recognition processing on the three-dimensional composite image to obtain a weld target baseline; performing comparison processing on the weld edge image and the weld target baseline to obtain an offset comparison result; and when the offset comparison result meets a preset threshold range, determining the weld edge image as the weld quality inspection area, which can improve the accuracy and efficiency of the weld quality inspection results.

Description

Technical Field

[0001] The embodiments of the present invention relate to, but are not limited to, the field of image recognition, and in particular to a method, device, computer, and storage medium for determining a weld quality inspection area. Background Art

[0002] During the welding process of a welding robot, the welding torch may deviate from the weld due to environmental factors such as strong arc radiation, high temperature, smoke, spatter, groove conditions, processing errors, fixture clamping accuracy, surface conditions, and thermal deformation of the workpiece. The presence of weld defects will weaken the stress-bearing area of ​​the weld, causing stress concentration at the defect, which will have an adverse effect on the strength, impact toughness, and cold bending performance of the connection, and will seriously affect the quality of the product. Therefore, performing welding quality inspections while welding to ensure construction quality and adjust optimal construction parameters is of great significance for ensuring yield and saving costs. However, the currently commonly used automated inspection methods are difficult to determine the weld area due to the generation of welding smoke from the target workpiece during welding, which leads to prone to recognition errors, resulting in a decrease in the accuracy of the weld quality inspection results and, in turn, a decrease in the yield. Summary of the Invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The main purpose of the embodiments of the present invention is to provide a method, device, computer and storage medium for determining a weld quality inspection area, which can improve the accuracy of weld quality inspection results.

[0005] In a first aspect, an embodiment of the present invention provides a method for determining a weld quality inspection area, the method comprising:

[0006] Acquire a two-dimensional welding image and a three-dimensional welding image of the target workpiece after the welding process is completed;

[0007] performing welding smoke determination processing on the two-dimensional welding image to obtain a welding smoke determination result;

[0008] Performing recognition processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image;

[0009] performing calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix;

[0010] Mapping the weld edge image of the two-dimensional welding image onto the three-dimensional welding image according to the mapping relationship matrix to obtain a three-dimensional composite image;

[0011] Performing recognition processing on the three-dimensional composite image to obtain a weld target reference line;

[0012] Performing offset comparison processing on the weld edge image and the weld target baseline to obtain an offset comparison result;

[0013] When the offset comparison result meets the preset threshold range, the weld edge image is determined as the weld quality inspection area.

[0014] In one embodiment, performing welding smoke determination processing on the two-dimensional welding image to obtain a welding smoke determination result includes:

[0015] Using an edge recognition tool to obtain a minimum circumscribed graph of the weld area in the two-dimensional welding image;

[0016] determining whether welding smoke exists in the two-dimensional welding image according to the edge parameters of the minimum circumscribed figure;

[0017] When the edge parameter is greater than a threshold value, the welding smoke judgment result is determined to be the presence of welding smoke, or when the edge parameter is less than the threshold value, the welding smoke judgment result is determined to be the absence of welding smoke.

[0018] In one embodiment, the identifying and processing the two-dimensional welding image according to the welding smoke judgment result to obtain the weld edge image includes:

[0019] When the welding smoke determination result indicates that welding smoke is present, performing welding smoke removal processing on the two-dimensional welding image to obtain a weld edge image;

[0020] or,

[0021] When the welding smoke judgment result is that there is no welding smoke, the two-dimensional welding image is searched and processed by an edge recognition tool to obtain a weld edge image.

[0022] In one embodiment, removing welding smoke from the two-dimensional welding image to obtain a weld edge image includes:

[0023] Performing histogram equalization on the two-dimensional welding image to obtain a two-dimensional welding image after equalization;

[0024] Quantifying the two-dimensional welding image after the equalization process to obtain a welding smoke area and a weld area;

[0025] The weld area is searched and processed by an edge recognition tool to obtain a weld edge image.

[0026] In one embodiment, the identifying and processing the three-dimensional composite image to obtain the weld target reference line includes:

[0027] Performing recognition processing on the three-dimensional composite image to obtain a first area and a second area welded to the first area;

[0028] Fitting is performed based on the first region and the second region to obtain a weld target reference line.

[0029] In one embodiment, the performing recognition processing on the three-dimensional composite image to obtain the first area and the second area welded to the first area includes:

[0030] Acquiring height data of the three-dimensional composite image;

[0031] The three-dimensional composite image is identified and processed according to the height data to obtain a first area and a second area welded to the first area, wherein a height value of the first area is different from a height value of the second area.

[0032] In one embodiment, performing fitting processing on the first region and the second region to obtain a weld target reference line includes:

[0033] Obtaining a height value of the first area and a height value of the second area;

[0034] Performing first-order derivative processing on the height value of the first area and the height value of the second area to obtain at least two edge points between the first area and the second area;

[0035] The weld target reference line tool is used to perform fitting processing on the at least two edge points to obtain a weld target reference line.

[0036] In one embodiment, performing offset comparison between the weld edge image and the weld target baseline to obtain a detection result includes:

[0037] Calculating an offset of a boundary of the weld edge image relative to the weld target reference line;

[0038] The offset is compared with a preset threshold range to obtain an offset comparison result.

[0039] In one embodiment, the calibrating the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix includes:

[0040] performing calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain coordinates of multiple corner points of the two-dimensional welding image and coordinates of multiple corner points of the three-dimensional welding image;

[0041] A conversion matrix calculation process is performed according to the coordinates of the multiple corner points of the two-dimensional welding image and the coordinates of the multiple corner points of the three-dimensional welding image to obtain a mapping relationship matrix.

[0042] In a second aspect, an embodiment of the present invention provides a device for determining a weld quality inspection area, comprising:

[0043] An acquisition module, used to acquire a two-dimensional welding image and a three-dimensional welding image of a target workpiece that has completed the welding process;

[0044] a judgment module, configured to perform welding smoke judgment processing on the two-dimensional welding image to obtain a welding smoke judgment result;

[0045] an identification module, configured to perform identification processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image;

[0046] a calibration module, configured to perform calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix;

[0047] a mapping module, configured to map the weld edge image of the two-dimensional welding image onto the three-dimensional welding image according to the mapping relationship matrix to obtain a three-dimensional composite image;

[0048] A fitting module is used to perform recognition processing on the three-dimensional composite image to obtain a weld target reference line;

[0049] a comparison module, configured to perform an offset comparison process on the weld edge image and the weld target reference line to obtain an offset comparison result;

[0050] A determination module is used to determine the weld edge image as a weld quality detection area when the offset comparison result meets a preset threshold range.

[0051] In a third aspect, an embodiment of the present invention provides a computer comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining the weld quality inspection area as described in the first aspect is implemented.

[0052] In a fourth aspect, a computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are used to execute the weld quality inspection area determination method described in the first aspect.

[0053] An embodiment of the present invention includes: a method for determining a weld quality inspection area includes the following steps: obtaining a two-dimensional welding image and a three-dimensional welding image of a target workpiece that has completed welding processing; performing welding smoke judgment processing on the two-dimensional welding image to obtain a welding smoke judgment result; performing recognition processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image; performing calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix; mapping the weld edge image of the two-dimensional welding image to the three-dimensional welding image according to the mapping relationship matrix to obtain a three-dimensional composite image; performing recognition processing on the three-dimensional composite image to obtain a weld target baseline; performing offset comparison processing on the weld edge image and the weld target baseline to obtain an offset comparison result; when the offset comparison result meets a preset threshold range, determining the weld edge image as the weld quality inspection area. In the technical solution of this embodiment, by comparing the weld edge image identified from the two-dimensional welding image with the weld target baseline identified from the three-dimensional composite image, when the offset comparison result is within the preset threshold range, the weld edge image is determined as the weld quality detection area. Since the weld edge image can be obtained more accurately by recognizing the two-dimensional welding image than by recognizing the three-dimensional welding image, and the weld target baseline can be obtained more accurately by recognizing the three-dimensional welding image than by recognizing the two-dimensional welding image, the weld quality detection area obtained by the above method is more accurate, and welding images with obvious problems can be eliminated, thereby improving the accuracy and efficiency of the weld quality detection results.

[0054] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a schematic diagram of a system architecture platform for executing a method for determining a weld quality inspection area provided by one embodiment of the present invention;

[0056] Figure 2 This is a flow chart of a method for determining a weld quality inspection area provided by one embodiment of the present invention;

[0057] Figure 3 This is a flow chart for judging welding smoke in a method for determining a weld quality inspection area provided by one embodiment of the present invention;

[0058] Figure 4 Schematic diagram of judging welding smoke in a method for determining a weld quality inspection area provided by an embodiment of the present invention;

[0059] Figure 5This is a flow chart of obtaining a weld edge image in a method for determining a weld quality inspection area provided by one embodiment of the present invention;

[0060] Figure 6 1 is a schematic diagram of obtaining a weld edge image in a method for determining a weld quality inspection area provided by an embodiment of the present invention;

[0061] Figure 7 This is a flow chart of generating a weld target reference line in a method for determining a weld quality inspection area provided by one embodiment of the present invention;

[0062] Figure 8 This is a schematic diagram of generating a weld target reference line in a weld quality inspection area determination method provided by an embodiment of the present invention;

[0063] Figure 9 It is a flow chart of obtaining a mapping relationship matrix in a method for determining a weld quality inspection area provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and the like in the specification, claims, or accompanying drawings are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0066] An embodiment of the present invention provides a method, device, computer and storage medium for determining a weld quality inspection area. The method for determining a weld quality inspection area includes the following steps: the method for determining a weld quality inspection area includes the following steps: obtaining a two-dimensional welding image and a three-dimensional welding image of a target workpiece that has completed welding processing; performing welding smoke judgment processing on the two-dimensional welding image to obtain a welding smoke judgment result; performing recognition processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image; performing calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix; mapping the weld edge image of the two-dimensional welding image to the three-dimensional welding image according to the mapping relationship matrix to obtain a three-dimensional composite image; performing recognition processing on the three-dimensional composite image to obtain a weld target baseline, performing offset comparison processing on the weld edge image and the weld target baseline (that is, comparing the distance between the edge relatively parallel to the weld target baseline and the weld target baseline with a preset distance according to the weld target baseline) to obtain an offset comparison result; when the offset comparison result meets the preset threshold range, determining the weld edge image as the weld quality inspection area. In the technical solution of this embodiment, by comparing the weld edge image identified from the two-dimensional welding image with the weld target baseline identified from the three-dimensional composite image, when the offset comparison result is within the preset threshold range, the weld edge image is determined as the weld quality detection area. Since the weld edge image can be obtained more accurately by recognizing the two-dimensional welding image than by recognizing the three-dimensional welding image, and the weld target baseline can be obtained more accurately by recognizing the three-dimensional welding image than by recognizing the two-dimensional welding image, the weld quality detection area obtained by the above method is more accurate, and welding images with obvious problems can be eliminated, thereby improving the accuracy and efficiency of the weld quality detection results.

[0067] The embodiments of the present invention are further described below with reference to the accompanying drawings.

[0068] like Figure 1 As shown, Figure 1 Schematic diagram of a system architecture platform 100 for a method for determining a weld quality inspection area provided by an embodiment of the present invention.

[0069] exist Figure 1 In the example of FIG, the system architecture platform 100 is provided with a processor 110 and a memory 120, wherein the processor 110 and the memory 120 can be connected via a bus or other means. Figure 1 The bus connection is taken as an example.

[0070] The memory 120 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory 120 may include a high-speed random access memory and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely located relative to the processor 110, and these remote memories may be connected to the processor 110 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0071] Those skilled in the art will appreciate that the system architecture platform can be applied to 5G communication network systems and subsequently evolved mobile communication network systems, and this embodiment does not specifically limit this.

[0072] It will be understood by those skilled in the art that Figure 1 The system architecture platform shown in the figure does not constitute a limitation on the embodiments of the present invention, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0073] The system architecture platform 100 can be an independent computer or a system architecture platform 100 that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0074] The computer may further include: a two-dimensional camera module for acquiring two-dimensional images, a three-dimensional camera module for acquiring three-dimensional welding images, a radio frequency (RF) circuit, an input unit, a display unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a processor, and a power supply. Those skilled in the art will appreciate that this embodiment does not limit the structure of the device to a single one, and the device may include more or fewer components than those in this embodiment, or may combine certain components, or have different component arrangements.

[0075] RF circuits can be used to receive and send signals during information transmission or calls. In particular, after receiving the downlink information from the base station, it is sent to the processor for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0076] The memory 120 can also be used to store software programs and modules. The processor executes various functional applications and data processing of the device by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the device (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory and a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0077] The input unit can be used to receive input digital or character information, and to generate key signal input related to the device settings and function control. Specifically, the input unit may include a touch panel and other input devices. A touch panel, also known as a touch screen, can collect touch operations on or near it (such as operations performed on or near the touch panel using any suitable object or accessory such as a finger, stylus, etc.) and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. The touch detection device detects the touch direction and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor, and can receive and execute commands sent by the processor. In addition, the touch panel can be implemented using a variety of types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel, the input unit may also include other input devices. Specifically, other input devices may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0078] The display unit can be used to display input information or provided information and various menus of the device. The display unit may include a display panel, and optionally, the display panel may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an organic light-emitting diode (Organic Light-Emitting Diode, OLED), etc. Further, the touch panel may cover the display panel, and when the touch panel detects a touch operation on or near it, it is transmitted to the processor to determine the category of the touch event, and then the processor provides a corresponding visual output on the display panel according to the category of the touch event. Although the touch panel and the display panel are two independent components to realize the input and output functions of the device, in some embodiments, the touch panel and the display panel can be integrated to realize the input and output functions of the device.

[0079] The device may also include at least one sensor, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel according to the brightness of the ambient light, and the proximity sensor may turn off the display panel and / or backlight when the device is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the device posture (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured on the device, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0080] Audio circuits, speakers, and microphones provide an audio interface. The audio circuit converts received audio data into electrical signals and transmits them to the speaker, which then converts them into sound signals for output. The microphone, on the other hand, converts collected sound signals into electrical signals, which are then received by the audio circuit and converted into audio data. This audio data is then processed by the processor and then transmitted via the RF circuit to, for example, another device, or stored in a memory for further processing.

[0081] WiFi is a short-range wireless transmission technology. Devices using WiFi modules can send and receive emails, browse the web, and access streaming media, providing wireless broadband internet access. A WiFi module is not a required component of a device and can be omitted as needed without changing the essence of the invention.

[0082] Processor 110 is the device's control center, connecting the various components of the device using various interfaces and circuits. By running or executing software programs and / or modules stored in memory and accessing data stored in memory, it performs various device functions and processes data, thereby providing overall device monitoring. Optionally, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a modem processor, with the application processor primarily processing the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor.

[0083] The device also includes a power source (such as a battery) for supplying power to each component. Preferably, the power source can be logically connected to the processor through a power management system, thereby managing charging, discharging, and power consumption through the power management system.

[0084] Although not shown, the device may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0085] Based on the above system architecture platform, various embodiments of the method for determining the weld quality inspection area of ​​the present invention are proposed below.

[0086] like Figure 2 As shown, Figure 2 This is a flowchart of a method for determining a weld quality inspection area provided by an embodiment of the present invention. The method for determining a weld quality inspection area is applied to the above-mentioned architecture platform, and the method for determining a weld quality inspection area includes but is not limited to step S100, step S200, step S300, step S400, step S500, step S600, step S700 and step S800.

[0087] Step S100 , obtaining a two-dimensional welding image and a three-dimensional welding image of a target workpiece after the welding process has been completed.

[0088] Specifically, when the welding process of the target workpiece is completed, the target workpiece will be sent to the weld inspection process. First, the two-dimensional camera module can be controlled to take a picture of the target workpiece at a fixed position to obtain a two-dimensional welding image. The three-dimensional camera module can also be controlled to take a picture of the target workpiece at a fixed position to obtain a three-dimensional welding image. The computer can store the two-dimensional welding image and the three-dimensional welding image in the hard disk according to the detection situation, or directly call the two-dimensional welding image and the three-dimensional welding image into the cache and directly perform the image recognition and analysis process, that is, start the weld quality detection area determination method process.

[0089] Step S200 , performing welding smoke determination processing on the two-dimensional welding image to obtain a welding smoke determination result.

[0090] Specifically, after obtaining the two-dimensional welding image and the three-dimensional welding image, the two-dimensional welding image can be first identified and judged to determine whether welding smoke is generated on the surface of the target workpiece after the welding process, and a welding smoke judgment result can be generated. Subsequently, different recognition programs can be started for the two-dimensional welding image based on the welding smoke judgment result.

[0091] Step S300 , performing recognition processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image.

[0092] Specifically, different recognition processing methods can be applied to the 2D welding image based on the welding smoke determination result, thereby identifying and obtaining a weld edge image from the 2D welding image. For example, if the welding smoke determination result indicates welding smoke is present, the 2D welding image needs to be processed to remove the welding smoke before the weld edge image can be identified. Alternatively, if the welding smoke determination result indicates no welding smoke is present, the 2D welding image can be searched and processed using an edge recognition tool to obtain a weld edge image.

[0093] Step S400 , calibrating the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix.

[0094] Specifically, after obtaining the weld edge image of the two-dimensional welding image, or after obtaining the two-dimensional welding image and the three-dimensional welding image, the two-dimensional welding image and the three-dimensional welding image can be calibrated to obtain a mapping relationship matrix, which is used to calibrate the feature parts in the two-dimensional welding image with the corresponding feature parts in the three-dimensional welding image.

[0095] Step S500 : mapping the weld edge image of the two-dimensional welding image onto the three-dimensional welding image according to the mapping relationship matrix to obtain a three-dimensional composite image.

[0096] Specifically, the weld edge image of the aforementioned generated two-dimensional welding image is mapped to the three-dimensional welding image according to the calculated mapping relationship matrix to obtain a fused three-dimensional composite image. The weld edge image with high accuracy obtained by recognizing the two-dimensional welding image can be accurately mapped to the three-dimensional welding image to prepare the image for subsequent analysis.

[0097] Step S600: performing recognition processing on the three-dimensional composite image to obtain a weld target reference line.

[0098] Specifically, the three-dimensional composite image is recognized and processed to obtain a weld target reference line corresponding to the weld, and the weld target reference line is used as a reference line for comparison with the weld edge image.

[0099] Step S700 : performing offset comparison processing on the weld edge image and the weld target baseline to obtain an offset comparison result.

[0100] Specifically, the offset of the boundary of the weld edge image relative to the weld target baseline is calculated, and then the offset is compared with a preset threshold range to obtain an offset comparison result.

[0101] Step S800: When the offset comparison result meets the preset threshold range, the weld edge image is determined as the weld quality inspection area.

[0102] Specifically, after the three-dimensional composite image generates a weld target baseline corresponding to the weld, the weld edge image and the weld target baseline are offset and compared to obtain an offset comparison result. Then, if the offset comparison result meets a preset threshold range, the weld edge image is determined as a weld quality inspection area. The weld quality inspection area is then inspected to determine whether the target workpiece welding meets the standard. Since two-dimensional welding image recognition can more accurately obtain a weld edge image than three-dimensional welding image recognition, and three-dimensional welding image recognition can more accurately obtain a weld target baseline than two-dimensional welding image recognition, the weld quality inspection area obtained by fusing the images obtained by the two recognitions through the above method is more accurate and can eliminate welding images with obvious problems, thereby improving the accuracy and efficiency of weld quality inspection results.

[0103] In one embodiment, when inspecting the welding quality of a target workpiece, a two-dimensional (2D) welding image and a three-dimensional (3D) welding image of the target workpiece after welding can be obtained. The two-dimensional (2D) welding image is then subjected to image recognition processing. For example, welding smoke determination processing is first performed on the two-dimensional (2D) welding image to obtain a welding smoke determination result. The two-dimensional (2D) welding image is then subjected to recognition processing based on the welding smoke determination result to obtain a weld edge image. After or during processing of the two-dimensional (2D) welding image, the two-dimensional (2D) welding image and the three-dimensional (3D) welding image can be calibrated to obtain a mapping matrix between the two. The weld edge image of the two-dimensional (2D) welding image is then mapped onto the three-dimensional (3D) welding image based on the mapping matrix to obtain a three-dimensional composite image. The three-dimensional composite image is then subjected to recognition processing to obtain a weld target baseline. The weld edge image and the weld target baseline are then subjected to offset comparison processing to obtain an offset comparison result. If the offset comparison result meets a preset threshold range, the weld edge image is determined as a weld quality inspection area. Finally, the weld quality inspection area is inspected to determine whether the weld of the target workpiece meets the standard. In the technical solution of this embodiment, since two-dimensional welding image recognition can obtain the weld edge image more accurately than three-dimensional welding image recognition, and three-dimensional welding image recognition can obtain the weld target baseline more accurately than two-dimensional welding image recognition, the weld quality inspection area obtained by the method of this embodiment is more accurate, and welding images with obvious problems can be eliminated, thereby improving the accuracy and efficiency of the weld quality inspection results.

[0104] Reference Figure 3 In one embodiment, step S200 includes but is not limited to step S310, step S320 and step S330.

[0105] Step S310, using a spot analysis tool to obtain the minimum circumscribed graph of the weld area in the two-dimensional welding image;

[0106] Step S320, judging whether welding smoke exists in the two-dimensional welding image according to the edge parameters of the minimum circumscribed figure;

[0107] Step S330 : determining that the welding smoke determination result is the presence of welding smoke when the edge parameter is greater than the threshold value, or determining that the welding smoke determination result is the absence of welding smoke when the edge parameter is less than the threshold value.

[0108] In one embodiment, referring to Figure 4 , a two-dimensional welding image is obtained by a 2D area array camera (see P1), and then the edge recognition tool is used to obtain the minimum circumscribed rectangle of the weld area in the two-dimensional welding image (see P2). The length and width of the circumscribed rectangle are used to judge whether there is welding smoke on both sides of the weld. When the edge parameter is greater than the threshold, the welding smoke judgment result is determined to be welding smoke, or when the edge parameter is less than the threshold, the welding smoke judgment result is determined to be no welding smoke.

[0109] It should be noted that the edge recognition tool can be used to obtain the minimum circumscribed rectangle of the weld area in the two-dimensional welding image, or other circumscribed shapes, which are not specifically limited in this embodiment.

[0110] It should be noted that the edge recognition tool may be a spot analysis tool or other edge recognition tools, and this embodiment does not impose any sole limitation thereto.

[0111] Reference Figure 5 In one embodiment, when the welding smoke determination result is that there is welding smoke, step S300 includes but is not limited to step S510, step S520 and step S530.

[0112] Step S510, performing histogram equalization processing on the two-dimensional welding image to obtain a two-dimensional welding image after equalization processing;

[0113] Step S520, quantizing the two-dimensional welding image after equalization to obtain welding smoke area and weld area;

[0114] Step S530: searching and processing the weld area using an edge recognition tool to obtain a weld edge image.

[0115] In one embodiment, referring to Figure 6When the welding smoke judgment result of the two-dimensional welding image is that there is welding smoke, the two-dimensional welding image is subjected to histogram equalization processing, and the pixels of the weld area and the welding smoke area in the two-dimensional welding image are relatively evenly distributed to obtain the two-dimensional welding image after equalization processing (see P3). Then, the two-dimensional welding image after equalization processing is subjected to 128-level quantization processing to separate the weld area and the welding smoke area by interval (see P4). Then, the edge of the weld area is searched by the edge recognition tool to obtain the weld edge image, and the weld edge image is output to the edge recognition tool, and the least squares method is used to find the effective point coordinates of the weld edge image (see P5).

[0116] Reference Figure 7 In one embodiment, step S600 includes but is not limited to step S710 and step S720.

[0117] Step S710 : performing recognition processing on the three-dimensional composite image to obtain a first area and a second area welded to the first area.

[0118] Specifically, the advantage of three-dimensional images over two-dimensional images is that more data reflecting image features can be obtained on the three-dimensional image, such as height data, that is, the height data of the three-dimensional composite image can be obtained, and then the three-dimensional composite image is identified and processed based on the height data to obtain a first area and a second area welded to the first area, where the height value of the first area is different from the height value of the second area.

[0119] In one embodiment, the target workpiece is a workpiece formed by welding the side and end plates. Then, after scanning the target workpiece with a three-dimensional scanner, the height data of the side and end plates in the three-dimensional welding image obtained are inconsistent, or the side and end plates in the three-dimensional welding image can be identified by the height data to obtain the side area and the end plate area welded to the side area.

[0120] Step S720: performing fitting processing based on the first region and the second region to obtain a weld target reference line.

[0121] Specifically, the height value of the first area and the height value of the second area are obtained, and then the first-order derivative processing is performed on the height value of the first area and the height value of the second area to obtain at least two edge points between the first area and the second area, and then the weld target baseline tool is used to fit the at least two edge points to obtain the weld target baseline.

[0122] In one embodiment, referring to Figure 8, the target workpiece is formed by welding the side and end plates, so the height values ​​of the side and the end plate can be obtained respectively, and then the edge point of the end plate is calculated by using the first-order derivative horizontal X-direction gradient: Rx = f(x+1)-f(x), and then the fitting line tool is used to fit the end plate edge line. Usually the fitted end plate edge line is a small section, which can be extended to obtain the weld target baseline.

[0123] Reference Figure 9 In one embodiment, step S400 includes but is not limited to step S910 and step S920.

[0124] Step S910, calibrating the two-dimensional welding image and the three-dimensional welding image to obtain coordinates of multiple corner points of the two-dimensional welding image and multiple corner points of the three-dimensional welding image;

[0125] In one embodiment, the coordinates of the two-dimensional welding image captured by the two-dimensional camera are: (x1, y1), (x2, y2), (x3, y3), (x4.y4), (x5, y5), (x6.y6), (x7.y7), (x8.y8), and (x9.y9); the coordinates of the three-dimensional welding image captured by the three-dimensional camera are: (x1', y1'), (x2', y2'), (x3', y3'), (x4', y4'), (x5', y5'), (x6', y6'), (x7', y7'), (x8', y8'), and (x9', y9').

[0126] It should be noted that this embodiment does not limit the number of coordinates, and can be set according to actual conditions and accuracy requirements.

[0127] Step S920 , performing conversion matrix calculation processing based on the coordinates of multiple corner points of the two-dimensional welding image and the coordinates of multiple corner points of the three-dimensional welding image to obtain a mapping relationship matrix.

[0128] In one embodiment, the image coordinates of the 9 points captured by the 2D camera are directly calculated to correspond to the image coordinates captured by the 3D camera, and a mapping relationship matrix A is obtained. The mathematical expression of step S820 is as follows:

[0129]

[0130]

[0131]

[0132] Based on the above-mentioned method for determining the weld quality inspection area, various embodiments of the automatic material selection device, controller and computer-readable storage medium of the present invention are respectively proposed below.

[0133] One embodiment of the present invention further provides a device for determining a weld quality inspection area, comprising:

[0134] An acquisition module, used to acquire a two-dimensional welding image and a three-dimensional welding image of a target workpiece that has completed the welding process;

[0135] A judgment module is used to perform welding smoke judgment processing on the two-dimensional welding image to obtain a welding smoke judgment result;

[0136] The recognition module is used to identify and process the two-dimensional welding image according to the welding smoke judgment result to obtain the weld edge image;

[0137] A calibration module is used to calibrate the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix;

[0138] A mapping module, configured to map the weld edge image of the two-dimensional welding image onto the three-dimensional welding image according to a mapping relationship matrix to obtain a three-dimensional composite image;

[0139] The fitting module is used to identify and process the three-dimensional composite image to obtain the weld target baseline;

[0140] A comparison module is used to perform offset comparison processing on the weld edge image and the weld target baseline to obtain an offset comparison result;

[0141] The determination module is used to determine the weld edge image as the weld quality detection area when the offset comparison result meets the preset threshold range.

[0142] It should be noted that the various embodiments of the above-mentioned automatic material selection device are consistent with the technical means used, the technical problems solved, and the technical effects achieved in the embodiments of the weld quality detection area determination method. They will not be described in detail here. For details, please refer to the embodiments of the weld quality detection area determination method.

[0143] In addition, one embodiment of the present invention provides a computer, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0144] The processor and the memory may be connected via a bus or other means.

[0145] It should be noted that the computer in this embodiment is set as follows Figure 1 The device fault handling guidance device in the embodiment shown includes a memory and a processor, which can form Figure 1 Part of the system architecture platform in the illustrated embodiment, both belong to the same inventive concept, so both have the same implementation principles and beneficial effects, and will not be described in detail here.

[0146] The non-transient software program and instructions required to implement the computer-side weld quality detection area determination method of the above embodiment are stored in the memory. When executed by the processor, the weld quality detection area determination method of the above embodiment is executed, for example, the above-described Figure 2 Method steps S100 to S800, Figure 3 Steps S310 to S330 of the method, Figure 5 Method steps S510 to S530, Figure 7 Steps S710 to S720 of the method and Figure 9 Method steps S910 to S920.

[0147] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are used to execute the above-mentioned computer weld quality detection area determination method, for example, executing the above-described Figure 2 Method steps S100 to S800, Figure 3 Steps S310 to S330 of the method, Figure 5 Method steps S510 to S530, Figure 7 Steps S710 to S720 of the method and Figure 9 Method steps S910 to S920.

[0148] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0149] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above implementation. Those skilled in the art can also make various equivalent modifications or substitutions under the shared conditions that do not violate the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for determining a weld quality inspection area, characterized in that: The method comprises: Acquire a two-dimensional welding image and a three-dimensional welding image of the target workpiece after the welding process is completed; performing welding smoke determination processing on the two-dimensional welding image to obtain a welding smoke determination result; Performing recognition processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image; performing calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix; Mapping the weld edge image of the two-dimensional welding image onto the three-dimensional welding image according to the mapping relationship matrix to obtain a three-dimensional composite image; Performing recognition processing on the three-dimensional composite image to obtain a weld target reference line; Performing offset comparison processing on the weld edge image and the weld target baseline to obtain an offset comparison result; When the offset comparison result is within a preset threshold range, the weld edge image is determined as a weld quality inspection area; The step of performing welding smoke determination processing on the two-dimensional welding image to obtain a welding smoke determination result includes: Using an edge recognition tool to obtain a minimum circumscribed graph of the weld area in the two-dimensional welding image; determining whether welding smoke exists in the two-dimensional welding image according to the edge parameters of the minimum circumscribed figure; When the edge parameter is greater than a threshold value, the welding smoke judgment result is determined to be the presence of welding smoke, or when the edge parameter is less than the threshold value, the welding smoke judgment result is determined to be the absence of welding smoke.

2. The method for determining the weld quality inspection area according to claim 1, characterized in that: The step of performing recognition processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image includes: When the welding smoke determination result indicates that welding smoke is present, performing welding smoke removal processing on the two-dimensional welding image to obtain a weld edge image; or, When the welding smoke judgment result is that there is no welding smoke, the two-dimensional welding image is searched and processed by an edge recognition tool to obtain a weld edge image.

3. The method for determining the weld quality inspection area according to claim 2, characterized in that: When the welding smoke determination result indicates that welding smoke is present, the step of removing welding smoke from the two-dimensional welding image to obtain a weld edge image includes: Performing histogram equalization on the two-dimensional welding image to obtain a two-dimensional welding image after equalization; Quantifying the two-dimensional welding image after the equalization process to obtain a welding smoke area and a weld area; The weld area is searched and processed by an edge recognition tool to obtain a weld edge image.

4. The method for determining the weld quality inspection area according to claim 1, wherein: The step of performing recognition processing on the three-dimensional composite image to obtain a weld target reference line includes: Performing recognition processing on the three-dimensional composite image to obtain a first area and a second area welded to the first area; Fitting is performed based on the first region and the second region to obtain a weld target reference line.

5. The method for determining the weld quality inspection area according to claim 4, characterized in that: The step of performing recognition processing on the three-dimensional composite image to obtain a first area and a second area welded to the first area includes: Acquiring height data of the three-dimensional composite image; The three-dimensional composite image is identified and processed according to the height data to obtain a first area and a second area welded to the first area, wherein a height value of the first area is different from a height value of the second area.

6. The method for determining the weld quality inspection area according to claim 4, characterized in that: The performing fitting processing on the first region and the second region to obtain a weld target reference line includes: Obtaining a height value of the first area and a height value of the second area; Performing first-order derivative processing on the height value of the first area and the height value of the second area to obtain at least two edge points between the first area and the second area; The weld target reference line tool is used to perform fitting processing on the at least two edge points to obtain a weld target reference line.

7. The method for determining the weld quality inspection area according to claim 1, characterized in that: The offset comparison of the weld edge image with the weld target reference line to obtain a detection result includes: Calculating an offset of a boundary of the weld edge image relative to the weld target reference line; The offset is compared with a preset threshold range to obtain an offset comparison result.

8. The method for determining the weld quality inspection area according to claim 1, wherein: The calibrating the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix includes: performing calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain coordinates of multiple corner points of the two-dimensional welding image and coordinates of multiple corner points of the three-dimensional welding image; A conversion matrix calculation process is performed according to the coordinates of the multiple corner points of the two-dimensional welding image and the coordinates of the multiple corner points of the three-dimensional welding image to obtain a mapping relationship matrix.

9. A device for determining a weld quality inspection area, characterized in that: include: An acquisition module, used to acquire a two-dimensional welding image and a three-dimensional welding image of a target workpiece that has completed the welding process; a judgment module, configured to perform welding smoke judgment processing on the two-dimensional welding image to obtain a welding smoke judgment result; The judgment module is further configured to use an edge recognition tool to obtain a minimum circumscribed figure of a weld area in the two-dimensional welding image, and to judge whether welding smoke exists in the two-dimensional welding image based on edge line parameters of the minimum circumscribed figure; and to determine that welding smoke exists in the two-dimensional welding image when the edge line parameters are greater than a threshold value, or to determine that welding smoke does not exist in the image when the edge line parameters are less than the threshold value. an identification module, configured to perform identification processing on the two-dimensional welding image according to the welding smoke judgment result to obtain a weld edge image; a calibration module, configured to perform calibration processing on the two-dimensional welding image and the three-dimensional welding image to obtain a mapping relationship matrix; a mapping module, configured to map the weld edge image of the two-dimensional welding image onto the three-dimensional welding image according to the mapping relationship matrix to obtain a three-dimensional composite image; A fitting module is used to perform recognition processing on the three-dimensional composite image to obtain a weld target reference line; a comparison module, configured to perform an offset comparison process on the weld edge image and the weld target reference line to obtain an offset comparison result; A determination module is used to determine the weld edge image as a weld quality detection area when the offset comparison result meets a preset threshold range.

10. A computer comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining the weld quality inspection area according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method for determining a weld quality inspection area according to any one of claims 1 to 8.

Images