Abnormal welding detection method and related device
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI LIXIANG AUTOMOBILE CO LTD
- Filing Date
- 2024-11-22
- Publication Date
- 2026-05-22
AI Technical Summary
During the electric drive laser welding process, abnormal phenomena such as weld size abnormality, abnormal position, broken weld and incomplete weld, and metal component spatter cannot be effectively detected, which leads to the failure of electric drive controller and affects the stability of the whole vehicle. Existing detection methods based on welding process parameters are insufficient to reflect the abnormal state of laser welding.
By acquiring target welding images, weld seam segmentation points and metal component segmentation points are extracted using a weld seam and metal component segmentation model. Weld seam size, location, broken welds, incomplete welds, and spatter anomalies are detected. Visual methods are used to avoid using welding process parameters.
It enables effective detection of weld abnormalities and welded metal component abnormalities, improves welding quality control, and ensures the stability of the electric drive controller and the whole vehicle.
Smart Images
Figure CN122072956A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of welding anomaly detection, and in particular to a welding anomaly detection method and related apparatus. Background Technology
[0002] During the electro-laser welding process, factors such as welding heat source and material fluctuations can lead to abnormal weld dimensions, abnormal weld positions, weld breaks or incomplete welds, or spatter on the welded metal components. If these abnormal products leave the production line, they can cause the electro-laser controller to malfunction and even affect the stability of the entire vehicle. Therefore, welding anomaly detection is essential. Currently, most welding anomaly detection methods are based on welding process parameters, such as optical signals and welding gas pressure. These process parameters affect the final welding result but cannot directly reflect the abnormal state of the laser welding. Summary of the Invention
[0003] In view of the above problems, this application provides a welding anomaly detection method and related apparatus to detect weld anomalies and welded metal component anomalies. The specific solution is as follows:
[0004] The first aspect of this application provides a method for detecting welding abnormalities, including:
[0005] Acquire a target welding image; wherein the target welding image is an image of a metal component with a weld seam;
[0006] Based on the target welding image, the weld seam segmentation points and metal component segmentation points are obtained;
[0007] Based on the weld seam segmentation points, at least one of the following is performed: weld seam size anomaly detection, incomplete weld detection, and weld breakage detection.
[0008] Based on the weld seam segmentation point and the metal component segmentation point, abnormal weld seam position detection and / or abnormal spatter detection of metal components are performed.
[0009] In one possible implementation, weld breakage detection based on the weld segmentation point includes:
[0010] The weld area formed by the weld seam segmentation points is divided into multiple segmented areas.
[0011] Calculate the proportion of pixels smaller than a preset pixel value within the segmented area;
[0012] If the proportion is less than the preset proportion, a weld breakage anomaly detection result will be generated.
[0013] In one possible implementation, the segmentation of the weld region formed by the weld segmentation points includes:
[0014] The upper and lower points of the weld region are obtained from the weld seam segmentation points;
[0015] Fit the upper edge points to obtain the upper edge line of the weld area, and fit the lower edge points to obtain the lower edge line of the weld area;
[0016] Determine the upper edge line segmentation point and the lower edge line segmentation point corresponding to the upper edge line segmentation point; wherein, the upper edge line segmentation point is the segmentation point of the upper edge line of the weld area, and the lower edge line segmentation point is the segmentation point of the lower edge line of the weld area;
[0017] Based on the upper edge segmentation point and the lower edge segmentation point, the weld is divided into multiple block areas.
[0018] In one possible implementation, detecting weak welds based on the weld seam segmentation points includes:
[0019] The average value of pixels in the weld area formed by the weld seam segmentation points is calculated to obtain the average pixel value of the weld area.
[0020] If the average pixel value of the weld seam area is greater than the preset average value, a false weld anomaly detection result is generated.
[0021] In one possible implementation, the weld size anomaly detection based on the weld segmentation point includes:
[0022] The maximum abscissa value, minimum abscissa value, maximum ordinate value, and minimum ordinate value of the weld seam segmentation point are obtained from the coordinates of the weld seam segmentation point.
[0023] The difference between the maximum abscissa value and the minimum abscissa value of the weld seam dividing point is taken as the weld width, and the difference between the maximum ordinate value and the minimum ordinate value of the weld seam dividing point is taken as the weld height.
[0024] If the weld width exceeds the width threshold range, and / or the weld height exceeds the height threshold range, a weld size anomaly detection result is generated.
[0025] In one possible implementation, the weld position anomaly detection based on the weld seam segmentation point and the metal component segmentation point includes:
[0026] The maximum and minimum ordinate values of the weld seam segmentation points are obtained from the coordinates of the weld seam segmentation points.
[0027] The maximum and minimum ordinate values of the metal element segmentation points are obtained from the coordinates of the metal element segmentation points.
[0028] Calculate the absolute value of the difference between the maximum ordinate value of the weld seam dividing point and the maximum ordinate value of the metal component dividing point to obtain the first edge distance between the weld seam and the metal component;
[0029] Calculate the absolute value of the difference between the minimum ordinate value of the weld seam dividing point and the minimum ordinate value of the metal element dividing point to obtain the second edge distance between the weld seam and the metal element.
[0030] If the first edge distance exceeds the first edge distance threshold range, and / or the second edge distance exceeds the second edge distance threshold range, a weld position anomaly detection result is generated.
[0031] In one possible implementation, detecting spatter anomalies in metal components based on the weld seam segmentation point and the metal component segmentation point includes:
[0032] Based on the weld seam dividing point and the metal component dividing point, a spatter detection area is obtained; wherein, the spatter detection area is the area formed by removing the weld seam area formed by the weld seam dividing point from the metal component area formed by the metal component dividing point;
[0033] The splash detection area is detected using a splash detection model;
[0034] If splashing is detected, a splashing anomaly detection result is generated for the metal component.
[0035] In one possible implementation, obtaining the weld seam segmentation point and the metal component segmentation point based on the target welding image includes:
[0036] The target welding image is input into the weld and metal component segmentation model to obtain the weld segmentation points and metal component segmentation points.
[0037] The weld and metal component segmentation model is generated by training a solid segmentation model with an labeled welding image. The labeled welding image is obtained by annotating the weld and metal components in the welding image to be trained.
[0038] A second aspect of this application provides a welding anomaly detection system, comprising:
[0039] An image acquisition module is used to acquire welding images of a target; wherein the target welding image is an image of a metal component with a weld seam;
[0040] The solid segmentation module is used to obtain weld segmentation points and metal component segmentation points based on the target welding image;
[0041] The first detection module is used to perform at least one of the following based on the weld seam segmentation point: weld seam size abnormality detection, incomplete weld detection, and weld breakage detection.
[0042] The second detection module is used to detect abnormal weld positions and / or abnormal spattering of metal components based on the weld seam segmentation point and the metal component segmentation point.
[0043] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the welding anomaly detection method of the first aspect or any implementation thereof.
[0044] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:
[0045] The memory is used to store computer programs;
[0046] The processor is used to execute the computer program so that the electronic device can implement the welding anomaly detection method of the first aspect or any implementation thereof.
[0047] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to perform the welding anomaly detection method described in the first aspect or any implementation thereof.
[0048] By means of the above technical solution, the welding anomaly detection method and related device provided in this application can obtain weld seam segmentation points and metal component segmentation points based on the target welding image. Then, through the weld seam segmentation points and metal component segmentation points, abnormal weld size, incomplete weld, weld position, and abnormal metal component spatter can be detected, thus achieving the purpose of detecting weld seam anomalies and welded metal component anomalies. Attached Figure Description
[0049] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0050] Figure 1 This application provides a flowchart of a welding anomaly detection method;
[0051] Figure 2 A schematic diagram of weld seam segmentation points and metal component segmentation points provided in this application;
[0052] Figure 3 This is a block diagram provided for this application;
[0053] Figure 4 This application provides a structural diagram of a welding anomaly detection system;
[0054] Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation
[0055] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.
[0056] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.
[0057] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0058] This application provides a method for detecting welding anomalies. The method for detecting welding anomalies according to this application will be described in detail below with reference to the accompanying drawings.
[0059] This application provides a welding anomaly detection method, including:
[0060] Step 101: Obtain the target welding image; wherein, the target welding image is an image of a metal component with a weld.
[0061] The target welding image is an image of a metal component with a weld seam, which may be a copper sheet. Optionally, the target welding image may be an image of an electrically driven laser welding process.
[0062] Step 102: Based on the target welding image, obtain the weld seam segmentation point and the metal component segmentation point.
[0063] The weld seam segmentation points and metal component segmentation points are obtained from the target welding image by inputting the target welding image into the weld seam and metal component segmentation model. This weld seam and metal component segmentation model is generated by training a solid segmentation model using an annotated welding image. The annotated welding image is obtained by annotating the weld seam and metal components in the welding image to be trained.
[0064] The weld seam and metal component segmentation model is generated through training. Specific training methods include:
[0065] Acquire the welding images to be trained;
[0066] Weld seam annotation and metal component annotation are performed on the welding images to be trained to obtain an annotated welding image;
[0067] The labeled welding images are input into the entity segmentation model for training, generating a segmentation model of weld seams and metal components.
[0068] When annotating weld seams and metal components in the welding images to be trained, the labelme tool can be used to annotate weld seams and label them "weldmark," and to annotate metal components and label them "spatter." The entity segmentation network can be the Mask R-CNN model. The annotated welding images are input into the entity segmentation model for training. After training, the model parameters are saved, and the resulting weld seam and metal component segmentation model is used to segment weld seams and metal components.
[0069] After inputting the target welding image into the trained model, weld seam segmentation points and metal component segmentation points can be obtained. Figure 2 This is a schematic diagram of weld seam splitting points and metal component splitting points. Figure 2 In the above, the area formed by weld seam dividing point 1 is weld seam area 11, and the area formed by metal component dividing point 2 is metal component area 22. Weld seam area 11 is located within metal component area 22.
[0070] Step 103: Perform at least one of the following based on the weld seam segmentation point: weld seam size anomaly detection, incomplete weld detection, and weld breakage detection.
[0071] Various weld anomaly detection methods can be performed based on weld seam segmentation points, such as weld seam size abnormalities, incomplete welds, and broken welds. The following sections explain how to perform these weld anomaly detections using weld seam segmentation points.
[0072] In one possible implementation, weld size anomaly detection based on weld segmentation points includes:
[0073] Obtain the maximum x-coordinate value, minimum x-coordinate value, maximum y-coordinate value, and minimum y-coordinate value of the weld seam segmentation point from the coordinates of the weld seam segmentation point.
[0074] The difference between the maximum and minimum abscissa values of the weld seam dividing points is taken as the weld width, and the difference between the maximum and minimum ordinate values of the weld seam dividing points is taken as the weld height.
[0075] If the weld width exceeds the width threshold range, and / or the weld height exceeds the height threshold range, a weld size anomaly detection result will be generated.
[0076] Establish the coordinate system of the target welding image, such as Figure 2 As shown, the positive x-axis is horizontal to the right, and the positive y-axis is vertically downward. The uppermost point 11 of the weld seam dividing point corresponds to the minimum ordinate value of the weld seam dividing point, the lowermost point 12 of the weld seam dividing point corresponds to the maximum ordinate value of the weld seam dividing point, the leftmost point 13 of the weld seam dividing point corresponds to the minimum abscissa value of the weld seam dividing point, and the rightmost point 14 of the weld seam dividing point corresponds to the maximum abscissa value of the weld seam dividing point.
[0077] The calculated weld width is compared with a preset width threshold, which may include a lower width limit and a higher width limit. The lower width limit and the higher width limit form a width threshold range. If the weld width is less than the lower width limit or greater than the higher width limit, the weld size is determined to be abnormal.
[0078] The calculated weld height is compared with a preset height threshold, which may include a lower height limit and a higher height limit. The lower height limit and the higher height limit form a height threshold range. If the weld height is less than the lower height limit or greater than the higher height limit, the weld size is determined to be abnormal.
[0079] In one possible implementation, detecting incomplete welds based on weld seam segmentation points includes:
[0080] The average value of pixels in the weld area formed by the weld seam segmentation points is calculated to obtain the average pixel value of the weld area.
[0081] If the average pixel value in the weld area is greater than the preset average value, a false weld anomaly detection result will be generated.
[0082] The detection of cold welds can be determined by the average pixel value within the entire weld area. When no cold weld has occurred, the weld color is close to black, and the corresponding pixel value in the weld area is close to 0. When a cold weld has occurred, the weld color is whitish, and the corresponding pixel value in the weld area will increase. When the average pixel value of the weld area is greater than the preset average value, a cold weld anomaly detection result is generated.
[0083] The formula for calculating the average pixel value of the weld area is as follows:
[0084]
[0085] In the formula, The average pixel value of the weld area. For pixels pixel values, For pixels The pixel value, where n is the number of pixels.
[0086] In one possible implementation, weld breakage detection based on weld seam segmentation points includes:
[0087] The weld area formed by the weld seam segmentation points is divided into blocks to obtain multiple block areas;
[0088] Calculate the proportion of pixels smaller than a preset pixel value within the segmented area;
[0089] If the proportion is less than the preset proportion, a weld breakage anomaly detection result will be generated.
[0090] Dividing the weld area into multiple sections allows for weld breakage detection in each section. The smaller the section width, the greater the likelihood of detecting weld breakage anomalies.
[0091] If a solder joint breaks in a block area, the proportion of black pixels in that block area will be low. Therefore, the determination of whether a solder joint breakage anomaly has occurred can be made by calculating the proportion of pixels smaller than a preset pixel value in the block area. The proportion of pixels smaller than a preset pixel value in the block area refers to the ratio of the number of pixels smaller than the preset pixel value to the total number of pixels in the block area. If the number of pixels smaller than the preset pixel value is small, it indicates that there are few black pixels. If the proportion of pixels smaller than the preset pixel value in the block area is less than the preset proportion, a solder joint breakage anomaly detection result is generated.
[0092] Optionally, the weld area formed by the weld seam dividing points is segmented, including:
[0093] Obtain the upper and lower points of the weld area from the weld seam segmentation points;
[0094] Fit the upper edge points to obtain the upper edge line of the weld area, and fit the lower edge points to obtain the lower edge line of the weld area.
[0095] Determine the upper edge segment point and the corresponding lower edge segment point; where the upper edge segment point is the segment point of the upper edge of the weld area, and the lower edge segment point is the segment point of the lower edge of the weld area.
[0096] Based on the upper and lower edge segmentation points, the weld seam is divided into multiple block areas.
[0097] The above-described method for detecting weld breaks based on weld seam segmentation points can detect weld breaks in weld seams with inclined angles.
[0098] Obtain the upper and lower points of the weld region from the weld seam segmentation points, including:
[0099] Remove the dividing points that belong to both ends of the weld from the weld dividing points to obtain the truncated dividing points;
[0100] Calculate the average of the ordinates of the cut-off points to obtain the ordinate of the center point;
[0101] The points whose ordinates are less than the ordinate of the center point are taken as the upper edge of the weld area, and the points whose ordinates are greater than the ordinate of the center point are taken as the lower edge of the weld area.
[0102] like Figure 3 As shown, two cutting lines cut off the dividing points at both ends of the weld, determining the vertical center point of each cutting point. The ordinate of this vertical center point is the average of the ordinates of the cutting points. Then, using this vertical center point as the boundary, the upper and lower edges of the weld region are obtained. Fitting the upper edge point yields the upper edge line of the weld region, and fitting the lower edge point yields the lower edge line. The segmentation points of the upper and lower edges are determined. Based on these segmentation points, the weld is divided into multiple sub-regions. Figure 3 There are 5 blocks between the two cut-off lines, namely block 0, block 1, block 2, block 3 and block 4.
[0103] When fitting the top or bottom point, if the top or bottom point includes m dividing points, and the coordinates of these dividing points are (x0, y0), (x1, y1), ..., (x... m ,y m These points form the curve f_0(x,y). The straight line f_1(x,y) that best approximates these dividing points is found through an optimization method and used as the upper or lower boundary line of the weld area. The error function is... .
[0104]
[0105] in,
[0106]
[0107] When determining the upper edge segmentation point and the corresponding lower edge segmentation point, the diameter of the weld process circle can be used as the segment length. Alternatively, the segment length can be smaller than the diameter of the weld process circle. In practical applications, after obtaining the upper and lower edges of the weld area, the number of segments can be determined, and the ratio of the upper edge length to the number of segments can be used as the segment length, thus obtaining the upper and lower edge segmentation points. The number of upper edge segmentation points can be the same as the number of lower edge segmentation points. Connecting the upper edge segmentation points and their corresponding lower edge segmentation points one by one yields multiple segmented areas.
[0108] Of course, when dividing the weld area formed by the weld seam dividing points, it is also possible not to cut off the dividing points at both ends of the weld, but to divide the entire weld area into blocks.
[0109] Step 104: Detect abnormal weld positions and / or abnormal spattering in metal components based on weld seam split points and metal component split points.
[0110] Based on weld seam segmentation points and metal component segmentation points, abnormal weld seam positions can be detected, as well as abnormal spattering in metal components can be detected.
[0111] In one possible implementation, weld position anomaly detection is performed based on weld seam split points and metal component split points, including:
[0112] Obtain the maximum and minimum ordinate values of the weld seam segmentation points from their coordinates;
[0113] Obtain the maximum and minimum ordinate values of the metal component segmentation points from their coordinates;
[0114] Calculate the absolute value of the difference between the maximum ordinate value of the weld seam split point and the maximum ordinate value of the metal component split point to obtain the first edge distance between the weld seam and the metal component;
[0115] Calculate the absolute value of the difference between the minimum ordinate value of the weld seam split point and the minimum ordinate value of the metal component split point to obtain the second edge distance between the weld seam and the metal component.
[0116] If the first margin exceeds the first margin threshold range, and / or the second margin exceeds the second margin threshold range, a weld position anomaly detection result is generated.
[0117] like Figure 2As shown, the uppermost point 11 of the weld seam dividing point corresponds to the minimum ordinate value of the weld seam dividing point, the lowermost point 12 of the weld seam dividing point corresponds to the maximum ordinate value of the weld seam dividing point, the uppermost point 21 of the metal component dividing point corresponds to the minimum ordinate value of the metal component dividing point, and the lowermost point 22 of the metal component dividing point corresponds to the maximum ordinate value of the metal component dividing point.
[0118] The first distance between the weld and the metal component is the distance between the lower edge of the weld and the lower edge of the metal component. This first distance is obtained by calculating the absolute value of the difference between the maximum ordinate value of the weld segmentation point and the maximum ordinate value of the metal component segmentation point. The second distance between the weld and the metal component is the distance between the upper edge of the weld and the upper edge of the metal component. This second distance is obtained by calculating the absolute value of the difference between the minimum ordinate value of the weld segmentation point and the minimum ordinate value of the metal component segmentation point.
[0119] The calculated first margin is compared with a preset first margin threshold, which may include a lower limit and an upper limit. The lower limit and the upper limit form the range of the first margin threshold. If the first margin is less than the lower limit or greater than the upper limit, the weld position is determined to be abnormal.
[0120] The calculated second margin is compared with a preset second margin threshold, which may include a lower limit and an upper limit. The lower limit and the upper limit form the range of the second margin threshold. If the second margin is less than the lower limit or greater than the upper limit, the weld position is determined to be abnormal.
[0121] Of course, for detecting abnormal weld positions, in addition to checking whether the top and bottom edge distances are abnormal, the left and right edge distances can also be checked. The specific process is as follows:
[0122] Obtain the maximum and minimum abscissa values of the weld seam segmentation points from their coordinates;
[0123] Obtain the maximum and minimum abscissa values of the metal component segmentation points from their coordinates;
[0124] Calculate the absolute value of the difference between the maximum abscissa value of the weld seam split point and the maximum abscissa value of the metal component split point to obtain the third edge distance between the weld seam and the metal component.
[0125] Calculate the absolute value of the difference between the minimum abscissa value of the weld seam split point and the minimum abscissa value of the metal component split point to obtain the fourth edge distance between the weld seam and the metal component.
[0126] If the third margin exceeds the third margin threshold range, and / or the fourth margin exceeds the fourth margin threshold range, a weld position anomaly detection result will be generated.
[0127] like Figure 2 As shown, the leftmost point 13 of the weld seam dividing point corresponds to the minimum abscissa value of the weld seam dividing point, the rightmost point 14 of the weld seam dividing point corresponds to the maximum abscissa value of the weld seam dividing point, the leftmost point 23 of the metal component dividing point corresponds to the minimum abscissa value of the metal component dividing point, and the rightmost point 24 of the metal component dividing point corresponds to the maximum abscissa value of the metal component dividing point.
[0128] The third distance between the weld and the metal component is the distance between the right edge of the weld and the right edge of the metal component. This third distance is obtained by calculating the absolute value of the difference between the maximum abscissa value of the weld segmentation point and the maximum abscissa value of the metal component segmentation point. The fourth distance between the weld and the metal component is the distance between the left edge of the weld and the left edge of the metal component. This fourth distance is obtained by calculating the absolute value of the difference between the minimum abscissa value of the weld segmentation point and the minimum abscissa value of the metal component segmentation point.
[0129] The calculated third margin is compared with a preset third margin threshold, which may include a lower limit and an upper limit. The lower limit and the upper limit form the range of the third margin threshold. If the third margin is less than the lower limit or greater than the upper limit, the weld position is determined to be abnormal.
[0130] The calculated fourth margin is compared with a preset fourth margin threshold, which may include a lower limit and an upper limit. The lower limit and the upper limit form the range of the fourth margin threshold. If the fourth margin is less than the lower limit or greater than the upper limit, the weld position is determined to be abnormal.
[0131] In one possible implementation, abnormal spatter detection of metal components is performed based on weld seam split points and metal component split points, including:
[0132] Based on the weld seam segmentation point and the metal component segmentation point, a spatter detection area is obtained; wherein, the spatter detection area is the area formed by removing the weld seam area formed by the weld seam segmentation point from the metal component area formed by the metal component segmentation point.
[0133] The splash detection model is used to detect the splash detection area;
[0134] If splashing is detected, a splashing anomaly detection result is generated for the metal component.
[0135] This splash detection model can be trained based on a convolutional neural network, or other deep learning models. If splash is detected, a splash anomaly detection result is generated for the metal component. The splash detection model can detect... Figure 2 Splash point 3 within the splash detection area.
[0136] This application employs a visual method, without using welding process parameters (such as light signals and welding gas pressure). By inputting the target welding image into the solid segmentation model of the weld and metal components, weld segmentation points and metal component segmentation points can be obtained. Then, based on the weld segmentation points and metal component segmentation points, abnormal weld size, incomplete weld, abnormal weld position, and abnormal spatter on metal components can be detected, thus achieving the purpose of detecting abnormal welds and abnormal welded metal components.
[0137] The above describes a welding anomaly detection method provided by the embodiments of this application. The following will describe the system for performing the above welding anomaly detection method.
[0138] The welding anomaly detection system provided in this application embodiment, such as Figure 4 As shown, the system includes:
[0139] Image acquisition module 401 is used to acquire welding images of targets; wherein, the target welding image is an image of a metal component with a weld.
[0140] The solid segmentation module 402 is used to obtain weld segmentation points and metal component segmentation points based on the target welding image.
[0141] The first detection module 403 is used to perform at least one of the following based on weld seam segmentation points: weld seam size anomaly detection, incomplete weld detection, and weld breakage detection.
[0142] The second detection module 404 is used to detect abnormal weld positions and / or abnormal spattering of metal components based on weld seam segmentation points and metal component segmentation points.
[0143] Entity segmentation module 402 is specifically used for:
[0144] Input the target welding image into the weld and metal component segmentation model to obtain the weld segmentation points and metal component segmentation points;
[0145] The weld and metal component segmentation model is generated by inputting the labeled welding image into the entity segmentation model for training. The labeled welding image is obtained by labeling the weld and metal components in the welding image to be trained.
[0146] The first detection module 403 includes:
[0147] The weld breakage detection unit is specifically used for:
[0148] The weld area formed by the weld seam segmentation points is divided into blocks to obtain multiple block areas;
[0149] Calculate the proportion of pixels smaller than a preset pixel value within the segmented area;
[0150] If the proportion is less than the preset proportion, a weld breakage anomaly detection result will be generated.
[0151] Optionally, the weld area formed by the weld seam dividing points is segmented, including:
[0152] Obtain the upper and lower points of the weld area from the weld seam segmentation points;
[0153] Fit the upper edge points to obtain the upper edge line of the weld area, and fit the lower edge points to obtain the lower edge line of the weld area.
[0154] Determine the upper edge segment point and the corresponding lower edge segment point; where the upper edge segment point is the segment point of the upper edge of the weld area, and the lower edge segment point is the segment point of the lower edge of the weld area.
[0155] Based on the upper and lower edge segmentation points, the weld seam is divided into multiple block areas.
[0156] The first detection module 403 also includes:
[0157] The cold solder joint detection unit is specifically used for:
[0158] The average value of pixels in the weld area formed by the weld seam segmentation points is calculated to obtain the average pixel value of the weld area.
[0159] If the average pixel value in the weld area is greater than the preset average value, a false weld anomaly detection result will be generated.
[0160] The first detection module 403 also includes:
[0161] The weld dimensional anomaly detection unit is specifically used for:
[0162] Obtain the maximum x-coordinate value, minimum x-coordinate value, maximum y-coordinate value, and minimum y-coordinate value of the weld seam segmentation point from the coordinates of the weld seam segmentation point.
[0163] The difference between the maximum and minimum abscissa values of the weld seam dividing points is taken as the weld width, and the difference between the maximum and minimum ordinate values of the weld seam dividing points is taken as the weld height.
[0164] If the weld width exceeds the width threshold range, and / or the weld height exceeds the height threshold range, a weld size anomaly detection result will be generated.
[0165] The second detection module 404 includes:
[0166] The weld position anomaly detection unit is specifically used for:
[0167] Obtain the maximum and minimum ordinate values of the weld seam segmentation points from their coordinates;
[0168] Obtain the maximum and minimum ordinate values of the metal component segmentation points from their coordinates;
[0169] Calculate the absolute value of the difference between the maximum ordinate value of the weld seam split point and the maximum ordinate value of the metal component split point to obtain the first edge distance between the weld seam and the metal component;
[0170] Calculate the absolute value of the difference between the minimum ordinate value of the weld seam split point and the minimum ordinate value of the metal component split point to obtain the second edge distance between the weld seam and the metal component.
[0171] If the first margin exceeds the first margin threshold range, and / or the second margin exceeds the second margin threshold range, a weld position anomaly detection result is generated.
[0172] The second detection module 404 also includes:
[0173] The splash anomaly detection unit is specifically used for:
[0174] Based on the weld seam segmentation point and the metal component segmentation point, a spatter detection area is obtained; wherein, the spatter detection area is the area formed by removing the weld seam area formed by the weld seam segmentation point from the metal component area formed by the metal component segmentation point.
[0175] The splash detection model is used to detect the splash detection area;
[0176] If splashing is detected, a splashing anomaly detection result is generated for the metal component.
[0177] This application also provides an electronic device in its embodiments. (See reference...) Figure 5 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0178] like Figure 5As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. When the electronic device is powered on, the RAM 503 also stores various programs and data required for the operation of the electronic device. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0179] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, memory cards, hard drives, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0180] This electronic device can implement the above-mentioned welding anomaly detection method.
[0181] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the welding anomaly detection methods provided in this application.
[0182] This application also provides a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the welding anomaly detection methods provided in this application.
[0183] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0184] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0185] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.
[0186] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
Claims
1. A method for detecting welding anomalies, characterized in that, include: Acquire a target welding image; wherein the target welding image is an image of a metal component with a weld seam; Based on the target welding image, the weld seam segmentation points and metal component segmentation points are obtained; Based on the weld seam segmentation points, at least one of the following is performed: weld seam size anomaly detection, incomplete weld detection, and weld breakage detection. Based on the weld seam segmentation point and the metal component segmentation point, abnormal weld seam position detection and / or abnormal spatter detection of metal components are performed.
2. The welding anomaly detection method according to claim 1, characterized in that, Weld breakage detection based on the weld seam segmentation points includes: The weld area formed by the weld seam segmentation points is divided into multiple segmented areas. Calculate the proportion of pixels smaller than a preset pixel value within the segmented area; If the proportion is less than the preset proportion, a weld breakage anomaly detection result will be generated.
3. The welding anomaly detection method according to claim 2, characterized in that, The step of dividing the weld area formed by the weld seam segmentation points into blocks includes: The upper and lower points of the weld region are obtained from the weld seam segmentation points; Fit the upper edge points to obtain the upper edge line of the weld area, and fit the lower edge points to obtain the lower edge line of the weld area; Determine the upper edge line segmentation point and the lower edge line segmentation point corresponding to the upper edge line segmentation point; wherein, the upper edge line segmentation point is the segmentation point of the upper edge line of the weld area, and the lower edge line segmentation point is the segmentation point of the lower edge line of the weld area; Based on the upper edge segmentation point and the lower edge segmentation point, the weld is divided into multiple block areas.
4. The welding anomaly detection method according to claim 1, characterized in that, The detection of incomplete welds based on the weld seam segmentation points includes: The average value of pixels in the weld area formed by the weld seam segmentation points is calculated to obtain the average pixel value of the weld area. If the average pixel value of the weld seam area is greater than the preset average value, a false weld anomaly detection result is generated.
5. The welding anomaly detection method according to any one of claims 1 to 4, characterized in that, The detection of weld size anomalies based on the weld segmentation points includes: The maximum abscissa value, minimum abscissa value, maximum ordinate value, and minimum ordinate value of the weld seam segmentation point are obtained from the coordinates of the weld seam segmentation point. The difference between the maximum abscissa value and the minimum abscissa value of the weld seam dividing point is taken as the weld width, and the difference between the maximum ordinate value and the minimum ordinate value of the weld seam dividing point is taken as the weld height. If the weld width exceeds the width threshold range, and / or the weld height exceeds the height threshold range, a weld size anomaly detection result is generated.
6. The welding anomaly detection method according to any one of claims 1 to 4, characterized in that, The detection of abnormal weld positions based on the weld seam segmentation point and the metal component segmentation point includes: The maximum and minimum ordinate values of the weld seam segmentation points are obtained from the coordinates of the weld seam segmentation points. The maximum and minimum ordinate values of the metal element segmentation points are obtained from the coordinates of the metal element segmentation points. Calculate the absolute value of the difference between the maximum ordinate value of the weld seam dividing point and the maximum ordinate value of the metal component dividing point to obtain the first edge distance between the weld seam and the metal component; Calculate the absolute value of the difference between the minimum ordinate value of the weld seam dividing point and the minimum ordinate value of the metal element dividing point to obtain the second edge distance between the weld seam and the metal element. If the first edge distance exceeds the first edge distance threshold range, and / or the second edge distance exceeds the second edge distance threshold range, a weld position anomaly detection result is generated.
7. The welding anomaly detection method according to any one of claims 1 to 4, characterized in that, Based on the weld seam segmentation point and the metal component segmentation point, abnormal spatter detection of metal components is performed, including: Based on the weld seam dividing point and the metal component dividing point, a spatter detection area is obtained; wherein, the spatter detection area is the area formed by removing the weld seam area formed by the weld seam dividing point from the metal component area formed by the metal component dividing point; The splash detection area is detected using a splash detection model; If splashing is detected, a splashing anomaly detection result is generated for the metal component.
8. The welding anomaly detection method according to any one of claims 1 to 4, characterized in that, The step of obtaining weld seam segmentation points and metal component segmentation points based on the target welding image includes: The target welding image is input into the weld and metal component segmentation model to obtain the weld segmentation points and metal component segmentation points. The weld and metal component segmentation model is generated by training a solid segmentation model with an labeled welding image. The labeled welding image is obtained by annotating the weld and metal components in the welding image to be trained.
9. A welding anomaly detection system, characterized in that, include: An image acquisition module is used to acquire welding images of a target; wherein the target welding image is an image of a metal component with a weld seam; The solid segmentation module is used to obtain weld segmentation points and metal component segmentation points based on the target welding image; The first detection module is used to perform at least one of the following based on the weld seam segmentation point: weld seam size abnormality detection, incomplete weld detection, and weld breakage detection. The second detection module is used to detect abnormal weld positions and / or abnormal spattering of metal components based on the weld seam segmentation point and the metal component segmentation point.
10. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the welding anomaly detection method as described in any one of claims 1 to 8.
11. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the welding anomaly detection method as described in any one of claims 1 to 8.
12. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the welding anomaly detection method as described in any one of claims 1 to 8.