Weld defect repair method and device based on visual image and electronic equipment

Abstract

This application provides a method, apparatus, and electronic device for weld defect repair based on visual images, relating to the field of welding technology. The method includes: extracting features from a target weld defect image to determine the defect features of the weld defects included in the target weld defect image, as well as the structural features of the weld in the target weld defect image. Then, based on the defect features and structural features, determining the spatial relationship features between the weld defect and a preset key structure of the weld. Further, combining the defect features, structural features, and spatial relationship features, querying the defect mechanism of the weld defect based on a preset welding mechanism rule base, and generating a repair scheme for the weld defect based on the defect mechanism. Here, the defect mechanism characterizes the formation cause of the weld defect morphology. The method of this application can achieve an automated closed-loop repair process for weld defects.

Description

Technical Field

[0001] This application relates to the field of welding technology, and in particular to a method, apparatus and electronic device for repairing weld defects based on visual images. Background Technology

[0002] In industries such as industrial manufacturing, infrastructure construction, and large equipment maintenance, there are numerous scenarios involving complex structural components, large welded parts, and high-risk welding. To address these scenarios, the need for automation in weld defect detection and repair decisions is becoming increasingly urgent.

[0003] Existing technologies primarily rely on multimodal visual fusion and sensor network-based multi-signal fusion to detect weld defects. Multimodal visual fusion extracts multi-source data, including visible light, 3D morphology, and thermal distribution, from weld defect images, analyzing the weld from different dimensions to identify defects. Sensor network-based multi-signal fusion combines signals from current, voltage, temperature, and eddy current probes to monitor the welding process and predict weld defects.

[0004] However, existing technologies are mainly used to detect weld defects, resulting in a disconnect between weld defect detection and weld defect mechanism inference and repair scheme generation, thus failing to achieve an automated closed loop for weld defect repair. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for repairing weld defects based on visual images, in order to solve the technical problem that the prior art cannot achieve an automated closed-loop repair of weld defects.

[0006] In a first aspect, this application provides a method for repairing weld defects based on visual images, comprising:

[0007] Feature extraction is performed based on the target weld defect image to determine the defect features of the weld defects included in the target weld defect image, as well as the structural features of the weld in the target weld defect image;

[0008] Based on the defect features and the structural features, the spatial relationship features between the weld defect and the preset key structure of the weld are determined;

[0009] Combining the defect features, structural features, and spatial relationship features, the defect mechanism of the weld defect is queried based on a preset welding mechanism rule base, and a repair scheme for the weld defect is generated based on the defect mechanism; wherein, the defect mechanism characterizes the cause of the formation of the defect morphology of the weld defect.

[0010] In one possible design, feature extraction is performed based on the target weld defect image to determine the defect features of the weld defects included in the target weld defect image, as well as the structural features of the weld in the target weld defect image, including:

[0011] The target weld defect image is processed based on a preset image segmentation model to identify the weld and weld defect included in the target weld defect image, and a first pixel-level mask of the weld defect is generated.

[0012] Based on the appearance characteristics of the first pixel-level mask, the defect type of the weld defect is determined; and in combination with the defect type, a first feature extraction is performed based on the first pixel-level mask and / or the target weld defect image to determine the defect characteristics of the weld defect.

[0013] A second feature extraction is performed based on the target weld defect image to determine the structural features of the weld.

[0014] In one possible design, the defect types include crack defects, incomplete penetration defects, slag inclusion defects, and / or porosity defects;

[0015] Based on the defect type, and using the first pixel-level mask and / or the target weld defect image, a first feature extraction is performed to determine the defect features of the weld defect, including:

[0016] Based on the first pixel-level mask and / or the target weld defect image, a first feature extraction algorithm is used to extract the first feature, and the first feature of the crack defect, the second feature of the incomplete penetration defect, the third feature of the slag inclusion defect, and / or the fourth feature of the porosity defect are determined.

[0017] The defect features of the weld defect include the first feature, the second feature, the third feature, and / or the fourth feature.

[0018] In one possible design, a second feature extraction is performed based on the target weld defect image to determine the structural features of the weld, including:

[0019] Based on the target weld defect image, a second feature extraction algorithm is used to extract the second feature to determine the weld toe and weld root in the weld, as well as the heat-affected zone associated with the weld; the heat-affected zone characterizes the area in the base metal that has not melted but whose microstructure and properties have changed due to heat.

[0020] The structural features of the weld include the weld toe, the weld root, and the heat-affected zone.

[0021] In one possible design, based on the defect characteristics and the structural characteristics, the spatial relationship characteristics between the weld defect and the predetermined key structure of the weld are determined, including:

[0022] Based on the defect characteristics, a first location of the weld defect is determined; and based on the structural characteristics, a second location of a preset key structure of the weld is determined; wherein the preset key structure includes the weld toe, weld root, and heat-affected zone;

[0023] Calculate the minimum Euclidean distance between the first position and the second position, and based on the minimum Euclidean distance, determine the spatial relationship characteristics between the weld defect and the preset key structure of the weld.

[0024] In one possible design, combining the defect characteristics, structural characteristics, and spatial relationship characteristics, the defect mechanism of the weld defect is queried based on a preset welding mechanism rule base, including:

[0025] Based on the defect features, structural features, and spatial relationship features, a structured feature vector is constructed;

[0026] The structured feature vector is matched with preset key features in the preset welding mechanism rule base to determine the preset defect mechanism corresponding to the preset key feature that successfully matches the structured feature vector; wherein, the preset welding mechanism rule base includes multiple preset defect morphologies, as well as preset key features and preset defect mechanisms corresponding to each preset defect morphology;

[0027] The preset defect mechanism is determined as the defect mechanism of the weld defect.

[0028] In one possible design, the method further includes:

[0029] Obtain the second pixel-level mask of the weld seam generated by the preset image segmentation model; based on the second pixel-level mask, extract the axial center pixel set of the weld seam;

[0030] The axial center pixel set is subjected to least squares fitting to generate the axial center line of the weld.

[0031] A local weld coordinate system is constructed based on the axial centerline, and the defect features and structural features are transformed into the local weld coordinate system.

[0032] Secondly, this application provides a weld defect repair device based on visual images, comprising:

[0033] The determination module is used to extract features based on the target weld defect image, determine the defect features of the weld defects included in the target weld defect image, and the structural features of the weld in the target weld defect image;

[0034] The determining module is further configured to determine the spatial relationship characteristics between the weld defect and the preset key structure of the weld based on the defect characteristics and the structural characteristics;

[0035] The query module is used to combine the defect features, the structural features, and the spatial relationship features to query the defect mechanism of the weld defect based on a preset welding mechanism rule base;

[0036] A generation module is used to generate a repair scheme for the weld defect based on the defect mechanism; wherein the defect mechanism characterizes the cause of the formation of the defect morphology of the weld defect.

[0037] In one possible design, the determining module includes:

[0038] The processing submodule is used to process the target weld defect image based on a preset image segmentation model, identify the weld and weld defect included in the target weld defect image, and generate a first pixel-level mask of the weld defect;

[0039] The first determining submodule is used for:

[0040] Based on the appearance characteristics of the first pixel-level mask, the defect type of the weld defect is determined; and in combination with the defect type, a first feature extraction is performed based on the first pixel-level mask and / or the target weld defect image to determine the defect characteristics of the weld defect.

[0041] A second feature extraction is performed based on the target weld defect image to determine the structural features of the weld.

[0042] In one possible design, the defect types include crack defects, incomplete penetration defects, slag inclusion defects, and / or porosity defects;

[0043] The first determining submodule includes: an extraction component, used for:

[0044] Based on the first pixel-level mask and / or the target weld defect image, a first feature extraction algorithm is used to extract the first feature, and the first feature of the crack defect, the second feature of the incomplete penetration defect, the third feature of the slag inclusion defect, and / or the fourth feature of the porosity defect are determined.

[0045] The defect features of the weld defect include the first feature, the second feature, the third feature, and / or the fourth feature.

[0046] In one possible design, the extraction component is further used for:

[0047] Based on the target weld defect image, a second feature extraction algorithm is used to extract the second feature to determine the weld toe and weld root in the weld, as well as the heat-affected zone associated with the weld; the heat-affected zone characterizes the area in the base metal that has not melted but whose microstructure and properties have changed due to heat.

[0048] The structural features of the weld include the weld toe, the weld root, and the heat-affected zone.

[0049] In one possible design, the determining module further includes:

[0050] The second determining submodule is used to determine the first location of the weld defect based on the defect characteristics; and to determine the second location of a preset key structure of the weld based on the structural characteristics; wherein the preset key structure includes the weld toe, weld root, and heat-affected zone;

[0051] A calculation submodule is used to calculate the minimum Euclidean distance between the first position and the second position;

[0052] The third determining submodule is used to determine the spatial relationship characteristics between the weld defect and the preset key structure of the weld based on the minimum Euclidean distance.

[0053] In one possible design, the query module includes:

[0054] A submodule is constructed to build a structured feature vector based on the defect features, the structural features, and the spatial relationship features.

[0055] The matching submodule is used to match the structured feature vector with preset key features in the preset welding mechanism rule library;

[0056] The fourth determination submodule is used for:

[0057] Determine the preset defect mechanism corresponding to the preset key feature that successfully matches the structured feature vector; wherein, the preset welding mechanism rule base includes multiple preset defect forms, as well as preset key features and preset defect mechanisms corresponding to each preset defect form;

[0058] The preset defect mechanism is determined as the defect mechanism of the weld defect.

[0059] In one possible design, the weld defect repair device based on visual images also includes:

[0060] The acquisition module is used to acquire the second pixel-level mask of the weld seam generated by the preset image segmentation model;

[0061] An extraction module is used to extract the axial center pixel set of the weld seam based on the second pixel-level mask;

[0062] Processing module, used for:

[0063] The axial center pixel set is subjected to least squares fitting to generate the axial center line of the weld.

[0064] A local weld coordinate system is constructed based on the axial centerline, and the defect features and structural features are transformed into the local weld coordinate system.

[0065] Thirdly, this application provides an electronic device comprising: at least one processor and a memory; the memory storing computer-executable instructions; the at least one processor executing the computer-executable instructions stored in the memory, causing the at least one processor to perform the method described in the first aspect above and various possible designs.

[0066] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the methods described in the first aspect above and various possible designs.

[0067] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect and various possible designs of the first aspect.

[0068] This application provides a visual image-based weld defect repair method, apparatus, and electronic device. It extracts features from a target weld defect image to determine the defect features of the weld defects included in the image, as well as the structural features of the weld within the image. Then, based on the defect and structural features, it determines the spatial relationship between the weld defect and a preset key structure of the weld. Further, combining the defect, structural, and spatial relationship features, it queries the defect mechanism of the weld defect based on a preset welding mechanism rule base and generates a repair scheme based on the defect mechanism. The defect mechanism characterizes the formation cause of the weld defect morphology. By extracting defect, structural, and spatial relationship features, the weld defect is transformed from pixel-level visual data to engineering semantic parameters, enabling the detected weld defect to directly drive weld defect mechanism inference and repair scheme generation, thus achieving an automated closed-loop weld defect repair process. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0070] Figure 1 A flowchart illustrating the weld defect repair method based on visual images provided in this application embodiment. Figure 1 ;

[0071] Figure 2 A flowchart illustrating the weld defect repair method based on visual images provided in this application embodiment. Figure 2 ;

[0072] Figure 3 A scenario illustration of the weld defect repair method based on visual images provided in the embodiments of this application. Figure 1 ;

[0073] Figure 4 A scenario illustration of the weld defect repair method based on visual images provided in the embodiments of this application. Figure 2 ;

[0074] Figure 5 A scenario illustration of the weld defect repair method based on visual images provided in the embodiments of this application. Figure 3 ;

[0075] Figure 6 A schematic diagram of the structure of the weld defect repair device based on visual images provided in the embodiments of this application;

[0076] Figure 7 This is a hardware structure diagram of the electronic device provided in the embodiments of this application.

[0077] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0078] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0079] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0080] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0081] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0082] In industries such as industrial manufacturing, infrastructure construction, and large equipment maintenance, there are numerous scenarios involving complex structural components, large welded parts, and high-risk welding. For example, in the manufacturing of tunnel boring machines (TBMs), the welds of key components such as the cutterhead and support structures need to withstand cyclic impact loads, posing a high risk of defects such as cracks, incomplete penetration, and slag inclusions. In the field of bridge steel structures, welds of steel box girders and stiffening ribs are subjected to the dual effects of alternating loads and environmental corrosion over long periods, making fatigue cracks and weld toe defects typical failure modes. In shipbuilding, pressure vessel, and pipeline welding, weld quality directly affects the safety and service life of the equipment.

[0083] It is evident that the need for automation in weld defect detection and repair decisions is becoming increasingly urgent in order to address the aforementioned scenarios. Existing technologies mainly rely on multimodal visual fusion and multi-signal fusion based on sensor networks to achieve weld defect detection.

[0084] Multimodal visual fusion extracts multi-source data such as visible light, 3D morphology, and thermal distribution from weld defect images, analyzing the weld from different dimensions to identify weld defects. This method typically requires the deployment of complex sensor systems and is sensitive to environmental conditions such as lighting and surface reflection, making it difficult to adapt to large components or complex working conditions.

[0085] Multi-signal fusion based on sensor networks combines signals such as current, voltage, temperature, and eddy current probes to monitor the welding process and predict weld defects. However, this method involves complex wiring, high hardware costs, and sensor installation is limited by the geometry and spatial layout of the components, making it difficult to deploy in dynamic or non-standard welding scenarios.

[0086] Furthermore, the detection results of weld defects in existing technologies are mostly presented as pixel-level masks or classification tags, lacking engineering semantic expression. They cannot be directly mapped to weld defect mechanism inference or repair scheme generation, resulting in a disconnect between weld defect detection and weld defect mechanism inference and repair scheme generation, making it impossible to achieve an automated closed loop for weld defect repair.

[0087] To address the aforementioned technical problems, the inventors first explored the limitations of multimodal visual fusion and sensor network-based multi-signal fusion, namely the disconnect between weld defect detection and weld defect mechanism inference and repair scheme generation. They also discovered the universality advantage of visual images in large components. Subsequently, to address the issue that weld defect detection results cannot be directly mapped to weld defect mechanism inference or repair scheme generation, the inventors proposed extracting defect features, structural features of the weld, and spatial relationship features between weld defects and key weld structures to achieve an engineering semantic representation of the weld and weld defects. Furthermore, to achieve an automated closed-loop weld defect repair process, the inventors proposed relying on a pre-built welding mechanism rule library to automate the matching of feature vectors with repair schemes.

[0088] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0089] This application provides a method for repairing weld defects based on visual images. Figure 1 A flowchart illustrating the weld defect repair method based on visual images provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the weld defect repair method based on visual images includes:

[0090] S101. Based on the target weld defect image, perform feature extraction to determine the defect features of the weld defects included in the target weld defect image, as well as the structural features of the weld in the target weld defect image.

[0091] In this embodiment, a mobile terminal device with industrial image acquisition capabilities, such as an industrial-grade handheld terminal (Personal Digital Assistant, PDA) or industrial tablet, is used to acquire images of target weld defects. It should be understood that a target weld defect image refers to an image recording abnormalities on the surface or inside the welded area (weld) and its heat-affected zone (HAZ). The HAZ refers to the unmelted area in the base material that has undergone changes in microstructure and properties due to heat; the base material is the metal material being welded.

[0092] It should be noted that, in order to ensure that mobile terminal devices can realistically and accurately reproduce weld seams and their HAZ in the real three-dimensional world, the mobile terminal devices need to be calibrated before capturing images. However, due to the inherent optical distortion in the lenses of mobile terminal devices, the original weld defect images captured will contain optical distortion. Therefore, the mobile terminal devices will automatically perform distortion correction on the original weld defect images, ultimately outputting the corrected target weld defect image.

[0093] Specifically, when the mobile terminal device is deployed for the first time, the intrinsic parameter matrix and distortion parameters are calculated through the calibration board, and then the distortion is corrected on the original weld defect image captured based on the intrinsic parameter matrix and distortion parameters.

[0094] Subsequently, the target weld defect image output from the mobile terminal device is input into a preset image segmentation model for processing. The preset image segmentation model in this embodiment has two functions: detection and recognition, and segmentation and mask generation. Specifically, the preset image segmentation model first identifies the weld and weld defect included in the target weld defect image, and then generates a first pixel-level mask for the weld defect and a second pixel-level mask for the weld.

[0095] Interpretive, pixel-level masks are typically binary masks, meaning they use only black and white. For example, white pixel areas are set as regions of interest, and black pixel areas are set as background areas.

[0096] It is worth noting that, for different target weld defect images, although they all capture the weld and its surrounding area, there will be geometric differences between them due to differences in the shooting angle, distance, and position. Therefore, in order to eliminate the geometric differences between different target weld defect images, it is necessary to construct a unified local weld coordinate system so as to map all target weld defect images into the same reference system based on weld geometry.

[0097] The construction of the local weld coordinate system relies on the second pixel-level mask of the weld. Specifically, based on the second pixel-level mask, the axial center pixel set of the weld is extracted. Then, least squares fitting is performed on the axial center pixel set to obtain a smooth line, which is the axial centerline of the weld. It should be noted that when fitting the axial centerline of the weld using the least squares method, a residual threshold of ≤0.5 pixels is set. This means that if the residual of the axial center pixel set of the weld is greater than 0.5 pixels, the axial center pixel set of the weld is re-extracted.

[0098] Furthermore, at any point on the axial centerline, the tangential direction is taken as the local direction vector of the weld, and is defined as local. Axis, and local Axial normal direction calculation, to obtain local Axis. In this way, the construction of the local weld coordinate system is completed.

[0099] It should be understood that this step also includes extracting the defect features of the weld and the structural features of the weld, and both the defect features and structural features need to be transformed into the local weld coordinate system. The extraction process for the defect features of the weld and the structural features of the weld will be explained in detail below.

[0100] (1) Extract the defect characteristics of weld defects.

[0101] Interpretively, in the process of extracting defect features from weld defects, the defect type of the weld defect is first determined based on the appearance characteristics (shape, edges, etc.) of the first pixel-level mask. Subsequently, in combination with the defect type, a first feature extraction is performed based on the first pixel-level mask and / or the target weld defect image to determine the defect features of the weld defect.

[0102] It should be noted that weld defects include crack defects, incomplete penetration defects, slag inclusion defects, and / or porosity defects. Based on this, this embodiment uses a first feature extraction algorithm to extract first features from a first pixel-level mask and / or the target weld defect image, thereby determining the first feature of crack defects, the second feature of incomplete penetration defects, the third feature of slag inclusion defects, and / or the fourth feature of porosity defects. It should be understood that the first, second, third, and / or fourth features together constitute the defect features of the weld defect.

[0103] Specifically, the primary characteristics of crack defects include the principal direction angle, actual crack length, actual crack width, crack endpoint morphology, and contrast (salience).

[0104] The principal direction angle of a crack defect refers to the angle between the crack direction and the weld direction; the crack direction is the principal direction of the crack defect. The principal direction angle is generally used to distinguish between hot cracks and cold cracks. If the principal direction angle is less than or equal to a certain preset threshold, it is a hot crack; otherwise, it is a cold crack.

[0105] Obviously, if the weld defect is a crack defect, then the first pixel-level mask is the pixel-level mask for the crack defect. In this case, all foreground pixels are extracted from the first pixel-level mask. , ), to obtain the foreground pixel set ={( , (), , ), ..., ( , The foreground pixel set can be understood as the region of interest in the first pixel-level mask.

[0106] Subsequently, Principal Component Analysis (PCA) was used to analyze the foreground pixel set. The analysis was performed to obtain the direction vector of the first principal component. :

[0107]

[0108] Then, the crack direction is opposite to the weld direction (locally). The angle between the axes, i.e., the angle between the principal directions. for:

[0109]

[0110] In the formula, Indicates local Unit vector along the axis, This represents the magnitude of the direction vector of the first principal component.

[0111] Furthermore, the endpoints of the crack defect are located in the first pixel-level mask along the crack direction. and This allows us to obtain the pixel length of the crack defect. = Then, the pixel length of the crack defect is calculated. Convert to actual crack length ,Right now = · .

[0112] Explanatory This represents the pixel-to-millimeter conversion factor. At the acquisition site of the target weld defect image, at least one reference marker with known physical dimensions (standard square target, standard specimen, etc.) is set up. The actual physical length of the reference marker is then detected. and its pixel length in the image. Establish conversion ratio: = It should be noted that in the calculation... At that time, three independent calibrations were performed and the average value was taken to ensure that the conversion error was small enough.

[0113] To determine the actual crack width, several cross-sectional scans are performed along the normal direction of the crack, and the maximum continuous pixel width of each cross-section is calculated. Then, for the maximum consecutive pixel width Take the maximum or average value and convert it to the actual crack width. ,Right now = ,or = .

[0114] The characteristic of the crack endpoint morphology depends on the boundary contour curvature and connected topology of the crack defect. Specifically, sub-pixel contour extraction is performed on the boundary of the first pixel-level mask, and then the boundary contour curvature of the crack defect is calculated using the curvature formula. ,Right now, = .in, and This represents the coordinates of a point located on the boundary profile. Denotes the first derivative. It represents the second derivative.

[0115] Furthermore, the local curvature distribution of the crack endpoint region is analyzed to determine the crack endpoint morphology. Interpretatively, if the curvature of the crack endpoint region converges rapidly and exhibits a sharp change, the crack endpoint morphology is determined to be "sharp"; if the crack endpoint region has multiple boundary branches, the crack endpoint morphology is determined to be "bifurcation"; if the crack endpoint region is wide and blunt, and the curvature change is gradual, the crack endpoint morphology is determined to be "blunt".

[0116] In another possible implementation, methods such as Fourier descriptors, multi-scale shape exponents, geometric moments, and signed distance function (SDF) curvature estimation can be used to determine the crack tip morphology. No specific method for determining the crack tip morphology is limited here.

[0117] The contrast (salience) of crack defects depends on the grayscale contrast index between the crack defect and the surrounding weld base material. Determined. Specifically, based on the average gray level of the crack defect. The average grayness of the base material and surrounding weldment Calculate the grayscale contrast index , = It should be understood that the grayscale contrast index... The larger the value, the stronger the contrast of the crack defect.

[0118] It should be noted that the extraction of the four features—main direction angle, actual crack length, actual crack width, and crack end shape—is achieved using a first-pixel level mask, while the extraction of the contrast feature is achieved using the target weld defect image.

[0119] The second feature of incomplete penetration defects includes the actual length, actual width, and offset of the incomplete penetration. Extraction of this second feature requires the use of a target weld defect image. Specifically, a grayscale profile is scanned along the weld direction, i.e., an axial grayscale profile. Then, a continuous dark groove on the axial grayscale profile is identified; this continuous dark groove represents the incomplete penetration defect. Further, based on multiple scanned axial grayscale profiles, the actual length and width of the incomplete penetration defect, as well as the offset, are statistically determined. The offset refers to the distance from the center point of the incomplete penetration defect to the axial centerline of the weld.

[0120] The third feature of slag inclusion defects includes the slag inclusion area, the aspect ratio of the slag inclusion, and the actual length of the slag inclusion. Extraction of this third feature also requires the use of the target weld defect image. It should be noted that when slag inclusion defects exist on the weld, the brightness at the slag inclusion defect location will be abnormal. Therefore, this embodiment extracts the brightness abnormality region, i.e., the area, aspect ratio, and actual length of the slag inclusion defect, by performing connected component analysis on the brightness abnormality region on the weld in the target weld defect image.

[0121] It should be understood that if multiple areas of abnormal brightness are found on the weld through connected component analysis, it means that there are multiple slag inclusion defects.

[0122] The fourth feature of porosity defects includes the porosity diameter, the number of pores per unit weld length, and the average porosity diameter. Extraction of this fourth feature also requires the use of the target weld defect image. Specifically, a minimum circumcircle is fitted to the near-circular dark spots on the weld in the target weld defect image; this minimum circumcircle can be considered as a porosity defect. Then, the diameter of the minimum circumcircle, i.e., the porosity diameter, is calculated. Simultaneously, the number of porosity defects per unit weld length is counted, and the average diameter of the porosity defects per unit weld length is calculated.

[0123] (2) Extract the structural features of the weld.

[0124] Interpretively, a second feature extraction is performed on the target weld defect image to determine the structural features of the weld. Specifically, a second feature extraction algorithm is used to extract second features from the second target weld defect image to determine the structural features of the weld.

[0125] The structural features of a weld include the weld toe and weld root, as well as the HAZ associated with the weld. The weld toe refers to the boundary line between the weld surface and the base metal, while the weld root refers to the starting point on the back side of the weld.

[0126] For the structural feature of the weld toe, a grayscale profile is scanned along the normal direction of the weld, i.e., the normal grayscale profile. Then, based on the grayscale change gradient of each normal grayscale profile, the grayscale extreme points are determined. The grayscale extreme points of each normal grayscale profile together constitute the weld toe point set. By further fitting the weld toe point set along the weld direction, a continuous weld toe line can be obtained.

[0127] For the structural feature of the weld root, the location of the weld root is determined by calculating the grayscale valley point of the dark band at the bottom of the weld.

[0128] For the HAZ structural feature, its location is determined based on the rate of change of grayscale distribution, texture variation, and feature point density around the weld. Interpretably, the grayscale of the HAZ exhibits a slow, monotonous change, the texture transitions from a weld-like pattern to a base metal-like pattern, and the feature point density is typically quite high.

[0129] In one possible implementation, histogram projection, gradient field analysis, wavelet transform, energy function, or adaptive threshold boundary detection methods can also be used to determine the weld toe, weld root, and HAZ. The specific methods for determining the weld toe, weld root, and HAZ are not limited here.

[0130] Understandably, the extraction of the defect features of the weld and the structural features of the weld are both accomplished using relevant feature extraction algorithms.

[0131] S102. Based on defect features and structural features, determine the spatial relationship features between weld defects and the preset key structures of the weld.

[0132] Clearly, after obtaining the defect features and structural features, the first location of the weld defect can be determined based on the defect features, and the second location of the pre-defined critical structure of the weld can be determined based on the structural features. Then, the minimum Euclidean distance between the first and second locations can be calculated. The pre-defined critical structure includes the weld toe, weld root, and HAZ.

[0133] It should be noted that, typically, the calculation involves the minimum Euclidean distance from the boundary of the weld defect to the boundaries of the weld toe, weld root, and HAZ. Based on this minimum Euclidean distance, the spatial relationship characteristics between the weld defect and the pre-defined critical structures of the weld are then determined.

[0134] For example, if the minimum Euclidean distance from the boundary of a weld defect to the weld toe is greater than 0, it means that the spatial relationship between the weld defect and the weld toe is characterized as "the weld defect does not cross the weld toe". As another example, if the minimum Euclidean distance from the boundary of a weld defect to the boundary of the HAZ is equal to 0, it means that the spatial relationship between the weld defect and the HAZ is characterized as "the weld defect crosses the HAZ".

[0135] S103. Combining defect features, structural features, and spatial relationship features, query the defect mechanism of weld defects based on the preset welding mechanism rule library, and generate a repair plan for weld defects based on the defect mechanism.

[0136] In this step, the quantifiable defect features, structural features, and spatial relationship features obtained above are organized into a unified data structure, namely, a structured feature vector. .

[0137] Specifically, ={type, , , , , , , , , , , , , , , } Where type represents the defect type, Indicates the shape of the crack endpoint. and This indicates the actual length and width of the incomplete weld penetration. and This indicates the number of pores and the diameter of the pores per unit weld length. and This indicates the area of ​​the inclusions and the aspect ratio of the inclusions. Indicates the distance from the weld defect to the weld toe. This indicates the distance from the weld defect to the weld root. Indicates whether the crack defect crosses the weld toe. =1 means that the crack defect crosses the weld toe, if =0 means that the crack defect did not cross the weld toe. Indicates whether the weld defect is located in the HAZ. =1, meaning the weld defect is located in the HAZ. =0 means that the weld defect is not located in the HAZ.

[0138] It should be noted that the above refers to structured feature vectors. The expression only lists some features; in fact, all quantifiable features should be included.

[0139] It should be understood that the design of structured feature vectors is not a simple listing of features, but a structured design based on the inference of welding engineering mechanisms and the actual needs of process generation, with each feature corresponding to a specific engineering judgment step. For example, the included angle of the main direction of the crack is used to determine the formation mechanism of the crack defect; the morphology of the crack endpoint is used to assess the activity level of the crack defect; the spatial relationship features between weld defects and weld toe, weld root, and HAZ are used to determine structural hazard and rework priority; and the porosity diameter and the number of pores per unit weld length can map to industry quality standard thresholds. Thus, structured feature vectors can transform welds and weld defects from pixel-level masks into structured engineering parameters that can be used for process generation.

[0140] It is worth noting that structured feature vectors are feature sets based on the constraints of welding materials science and structural mechanics mechanisms. They possess clear engineering semantics and cannot achieve the same decision-making ability through features of a single category. Arbitrarily deleting any feature, such as crack tip morphology or pore diameter, will lead to a break in the mechanism inference chain, thus making it impossible to distinguish critical structural weld defects. Because the construction process of structured feature vectors does not depend on a specific model, they possess cross-model and cross-device versatility and stability.

[0141] After constructing the structured feature vector, the structured feature vector is matched with preset key features in the preset welding mechanism rule base to determine the preset defect mechanism corresponding to the preset key feature that successfully matches the structured feature vector. This preset defect mechanism is the defect mechanism of the weld defect. Here, the defect mechanism refers to the cause of the formation of the defect morphology of the weld defect. The preset welding mechanism rule base includes multiple preset defect morphologies, as well as the preset key features and preset defect mechanisms corresponding to each preset defect morphology.

[0142] Interpretatively, weld defects can be classified into different types, such as crack defects, incomplete penetration defects, slag inclusion defects, and porosity defects. Each type of weld defect typically includes different defect morphologies; for example, crack defects include surface microcracks, short sharp-end cracks, and bifurcated cracks. Therefore, based on the principles of welding materials science, weld mechanical properties, industry-standard practices, and widely used defect judgment rules in engineering fields, the welding engineering industry has deduced the key characteristics and mechanisms of several common pre-defined defect morphologies and developed a pre-defined welding mechanism rule library.

[0143] For example, (1) during the welding process, cracks of different morphologies have stable typical distribution directions in terms of mechanical, metallurgical and shrinkage behavior, that is, the included angle of the principal direction of the crack defect. This describes the directional characteristics of crack defects relative to the weld. Hot cracks typically propagate along the solidification direction of the weld metal, i.e., the weld direction; cold cracks / stress cracks usually appear in the principal stress direction of the structure, often perpendicular or nearly perpendicular to the weld direction. Therefore, if When the angle is close to 0° or less than 30°, the crack defect tends to be a hot crack; when the angle is close to 90° or greater than 60°, the crack defect tends to be a cold crack / stress crack.

[0144] (2) Crack endpoint morphology It can reflect the crack propagation trend. A bifurcation end indicates multi-directional crack propagation, a sharp end indicates high stress concentration at the crack tip and an active crack state, while a blunt end indicates the crack is tending to stop or is in an inactive region. It's worth noting that in materials mechanics, the crack tip morphology... A sharp crack tip means that the stress intensity factor at the crack tip is relatively large, and the crack is more likely to continue propagating. If the crack tip morphology... The terminus is a bifurcation point, which means that the stress intensity factor at the crack tip is larger and the crack tip is unstable, resulting in branched cracks.

[0145] Therefore, if =The bifurcation point indicates a clear risk of crack propagation. =Sharp end, indicating that the crack may propagate. =The blunt end indicates that the crack has a relatively weak tendency to propagate.

[0146] (3) The weld toe is usually the location where fatigue cracking is most likely to occur in the weld, and it is also the area with the highest stress concentration in the weld. If the crack crosses the weld toe, that is... =1, which leads to a reduction in the effective weld leg size and a sharp increase in stress concentration at the weld toe. The effective weld leg refers to the portion of the weld cross-section that can actually transfer the load. Therefore, when When the value is 1, it is determined to be a structurally dangerous crack and must be repaired.

[0147] (4) The engineering characteristics of the incomplete penetration defect are: the incomplete penetration area is a continuous dark groove, and the actual length of the incomplete penetration is [missing information]. The longer the weld, the smaller the effective cross-sectional area. If the weld is not fully penetrated, the actual length... If the weld exceeds a certain size, such as 4mm, it indicates that the root of the weld is not fused properly, which is usually judged as a defective weld by industry standards.

[0148] (5) Due to its high brittleness, the HAZ (weld toe and weld root) has a relatively high probability of cold cracking. Furthermore, both the weld toe and weld root are weak areas in the welded structure. Therefore, if the weld defect is located in the HAZ... =1, requiring high-risk treatment for cold cracking; if the distance from the weld defect to the weld toe... A relatively small distance indicates that the weld defect is close to the fatigue-sensitive zone; if the distance from the weld defect to the weld root is relatively small... If the weld is too small, it will affect the penetration and strength of the weld.

[0149] As can be seen, the preset welding mechanism rule base provides several common preset defect morphologies, along with corresponding preset key features and preset defect mechanisms for each morphology. After matching the structured feature vectors with the preset key features in the preset welding mechanism rule base and identifying the defect mechanism of the weld defect, a repair scheme for the weld defect is generated based on the defect mechanism. Specifically, the correspondence between the preset welding mechanism rule base and the weld defect repair scheme is shown in Table 1.

[0150] Table 1

[0151]

[0152] It should be noted that the terms "large" and "small" mentioned in Table 1 are compared with relevant preset thresholds. That is, if it is greater than or equal to the relevant preset threshold, it is considered "large," and if it is less than the relevant preset threshold, it is considered "small."

[0153] It should be understood that the generation of repair solutions for weld defects is not based on subjective judgment based on human experience, but rather on rule matching based on engineering quantities in structured feature vectors, using physical quantities such as... , , Based on these factors, the corresponding repair plan will be automatically triggered.

[0154] It is evident that the decision-making chain of the repair scheme in this embodiment is indivisible: without structured feature vectors, it is impossible to obtain the engineering semantics related to welding mechanics. Without a pre-set welding mechanism rule base, it is impossible to determine the defect mechanism of the weld defect based on the engineering semantics. Without process generation, it is impossible to output an executable rework scheme based on the defect mechanism. Therefore, structured feature vectors, a pre-set welding mechanism rule base, and process generation together constitute a necessary closed loop from visual image input to engineering processing output; the absence of any link will cause the decision-making chain of the repair scheme to fail to execute.

[0155] In conclusion, Figure 2 A flowchart illustrating the weld defect repair method based on visual images provided in this application embodiment. Figure 2 ,like Figure 2 As shown, the overall process of the weld defect repair method based on visual images in this embodiment is as follows:

[0156] S201. Use a mobile terminal device to capture original images of weld defects.

[0157] S202. Perform distortion correction on the original weld defect image to obtain the target weld defect image.

[0158] S203. Use a preset image segmentation model to process the target weld defect image to generate a first pixel-level mask for the weld defect and a second pixel-level mask for the weld.

[0159] S204. Construct a local weld coordinate system based on the second pixel-level mask.

[0160] S205. Extract the defect features and structural features of the weld. Defect features include the principal direction angle, actual crack length, actual crack width, crack endpoint morphology, and contrast of crack defects; the actual incomplete penetration length, actual incomplete penetration width, and incomplete penetration offset of incomplete penetration defects; the slag inclusion area, slag inclusion aspect ratio, and actual slag inclusion length of slag inclusion defects; and the porosity diameter, number of pores per unit weld length, and average pore diameter of porosity defects. Structural features include the weld toe and weld root, as well as the HAZ associated with the weld.

[0161] S206. Based on the defect characteristics and structural characteristics, determine the spatial relationship characteristics between weld defects and weld toe, weld root and HAZ.

[0162] S207. Construct a structured feature vector based on defect features, structural features, and spatial relationship features.

[0163] S208. Match the structured feature vector with the preset key features in the preset welding mechanism rule library to find the defect mechanism of weld defects.

[0164] S209. Generate and output a repair scheme for weld defects based on the defect mechanism.

[0165] In a specific demonstration example, the fillet welds between the reinforcing ribs of the cutterhead backplate and the base material of a tunnel boring machine (TBM) were inspected. This component, the reinforcing ribs of the cutterhead backplate, has the following structural engineering characteristics: the cutterhead backplate is 40mm thick, the reinforcing rib is 20-24mm thick, and it employs multi-pass, multi-layer fillet welds. The welds exhibit a curved distribution and localized uneven weld toe transitions, making it a critical weld area subjected to alternating loads. During long-term TBM tunneling, these fillet welds bear cyclic impact loads and are prone to defects such as cracks, incomplete penetration, slag inclusions, and porosity.

[0166] Specifically, a portable industrial PDA (fixed focal length) was used to inspect and capture original weld defect images along the backplate of the cutter head. A standard reference piece was deployed next to the weld on the backplate of the cutter head; the actual length of this reference piece was 20mm, and the length detected in the original weld defect image was 195px. Therefore, there is... =20mm / 195px≈0.1026mm / px.

[0167] Using the intrinsic parameter matrix and distortion parameters of a portable industrial PDA, distortion correction is performed on the original weld defect image to obtain the target weld defect image. Then, a preset image segmentation model is used to process the target weld defect image, generating a first pixel-level mask for the weld defect and a second pixel-level mask for the weld. The preset image segmentation model automatically detects that the weld defect is located near the weld toe.

[0168] Furthermore, a local weld coordinate system is constructed based on the second pixel-level mask, and the defect features of weld defects and the structural features of welds involved thereafter are all transformed into this local weld coordinate system.

[0169] Furthermore, measurements of the first pixel-level mask revealed a length of approximately 110 pixels and a width of approximately 6 pixels, initially identifying the weld defect as a crack. PCA analysis was then performed on the first pixel-level mask to obtain the first principal component direction vector, and the principal direction angle of the crack defect was calculated based on this vector. ,get =82°, which means that the crack is approximately perpendicular to the weld.

[0170] The actual crack length is calculated based on the length of the first pixel-level mask (110 pixels). ,get =0.1026×110≈11.3mm. The actual crack width is calculated based on the width of the first pixel-level mask (6 pixels). ,get =0.1026×6≈0.62mm. Analysis of the local curvature distribution at the crack endpoint region revealed a clear bifurcation at one end and a sharp end at the other, indicating the morphology of the crack endpoint. =Bifurcation end.

[0171] The average grayness of the surrounding weld base material was measured. =148, average gray level of crack defects =108. Therefore, the grayscale contrast index between the crack defect and the surrounding weld base material is... = (148-108) / 148≈0.27, which means that the contrast of the crack defect is strong and it is an active crack characteristic.

[0172] In addition, the distance from the crack defect to the weld toe was measured. =0.35mm, and the crack defect is located in the HAZ, that is =1, and the crack defect crosses the weld toe, i.e. =1.

[0173] Based on the features extracted above, a structured feature vector is constructed. ,have: ={type=crack, =11.3, =0.62, =82°, =Bifurcation end, =0.27, =0.35, =1, =1}

[0174] Based on the pre-defined welding mechanism rule base and welding materials science, it can be deduced that: =82°→Transverse crack→Cold crack / stress crack tendency =Bifurcation point → Crack is in the propagation stage =1→Crack crossing the weld toe→Structurally dangerous crack. =1→Crack located in HAZ→Increased brittleness→High risk. The final conclusion is: There is a structurally dangerous transverse crack in the fillet weld between the reinforcing rib of the shield machine cutterhead backplate and the base material, and the risk level of the crack is relatively high.

[0175] It should be noted that, Figure 3 A scenario illustration of the weld defect repair method based on visual images provided in the embodiments of this application. Figure 1 ,like Figure 3As shown, by matching the structured feature vector with the preset welding mechanism rule library, the crack defect is identified as a crack spanning the weld toe, and its defect mechanism is that structural stress concentration leads to strength weakening. Based on this, the repair scheme for cracks spanning the weld toe is automatically generated as follows: open a V-groove → remove the entire crack → re-weld and grind the weld toe transition zone.

[0176] Understandable. Figure 4 A scenario illustration of the weld defect repair method based on visual images provided in the embodiments of this application. Figure 2 , Figure 5 A scenario illustration of the weld defect repair method based on visual images provided in the embodiments of this application. Figure 3 ,like Figure 4 and Figure 5 As shown, to facilitate subsequent quality traceability, structured feature vectors, defect morphology, defect mechanism, and repair schemes can be output and recorded.

[0177] In another specific example, the fillet welds between the web of the main longitudinal beam and the transverse stiffeners of the bridge steel box girder were inspected. The web of the main longitudinal beam is 12-16 mm thick, and the stiffener plates are 10-12 mm thick. CO2 gas shielded welding and multi-pass fillet welding were used. The welds exhibited a long straight line or a slight curve distribution, with localized uneven transitions at the weld toe, classifying them as typical fatigue-sensitive areas under alternating loads, with significant stress concentration at the weld toe. During bridge operation, the fillet welds bear repeated vehicle loads and temperature cycles, making them a high-risk area for transverse cracks, weld toe fatigue cracks, and lack of fusion.

[0178] Specifically, a specialized industrial terminal for bridge inspection is used to move and capture images along the weld direction, maintaining a scanning distance of approximately 0.5m throughout, to obtain the original weld defect image. The bridge web surface is marked with a 30mm standard etched line; this standard etched line has a length of 210px in the original weld defect image, therefore... =30mm / 210px≈0.1429mm / px.

[0179] Using the intrinsic parameter matrix and distortion parameters of a bridge inspection industrial terminal, distortion correction is performed on the original weld defect image to obtain the target weld defect image. Then, a preset image segmentation model is used to process the target weld defect image, generating a first pixel-level mask for the weld defect and a second pixel-level mask for the weld. The preset image segmentation model automatically detects that the weld defect is located near the weld toe.

[0180] Furthermore, a local weld coordinate system is constructed based on the second pixel-level mask, and the defect features of weld defects and the structural features of welds involved thereafter are all transformed into this local weld coordinate system.

[0181] Furthermore, measurements of the first pixel-level mask revealed a length of approximately 85 pixels and a width of approximately 4 pixels, initially identifying the weld defect as a crack. PCA analysis of the first pixel-level mask yielded the first principal component direction vector, which was then used to calculate the principal direction angle of the crack defect. ,get =74°, which means that the crack has a typical transverse fatigue crack trend.

[0182] The actual crack length is calculated based on the length of the first pixel-level mask (85 pixels). ,get =0.1429×85≈12.15mm. The actual crack width is calculated based on the width of the first pixel-level mask (4 pixels). ,get =0.1429×4≈0.57mm. Analysis of the local curvature distribution at the crack endpoint region revealed that one end of the crack defect was sharp, while the other end was slightly curved and blunt, indicating the morphology of the crack endpoint. = (sharp end + blunt end).

[0183] The average grayness of the surrounding weld base material was measured. =162, average gray level of crack defects =132. Therefore, the grayscale contrast index between the crack defect and the surrounding weld base material is... = (162-132) / 162≈0.19, which means that the grayscale of the crack defect is clear and the contrast is obvious.

[0184] In addition, the distance from the crack defect to the weld toe was measured. =0.42mm, and the crack defect is located in the HAZ, that is =1, and the crack defect crosses the weld toe, i.e. =1.

[0185] Based on the features extracted above, a structured feature vector is constructed. ,have: ={type=crack, =12.15, =0.57, θ=74°, =Sharp end + blunt end =0.19, =0.42, =1, =1}

[0186] Based on the pre-defined welding mechanism rule base and the fatigue mechanism of bridge steel structures, it can be deduced that: =74° → Transverse crack → Typical path of fatigue crack =Sharp end + blunt end → Crack may still propagate =1→Crack crosses the weld toe→Crack in the stress concentration zone of the weld toe =1 → Crack located in HAZ → Brittleness increased. ≈12.15mm → Crack length enters fatigue crack zone II. The final conclusion is that weld toe fatigue cracks (cross-weld toe type) exist in the fillet welds between the web of the main longitudinal beam and the transverse stiffeners of the bridge steel box girder, and the risk level of these cracks is relatively high. If these cracks are not addressed promptly, they will significantly affect the fatigue life of the steel box girder.

[0187] Furthermore, the structured feature vectors are matched with the preset welding mechanism rule library, and the corresponding repair scheme is automatically generated according to the defect mechanism of weld toe fatigue crack. Specifically, the repair scheme for weld toe fatigue crack includes: (1) Weld toe grinding and smoothing. The weld toe is processed into a smooth transition curve with a radius of 2-3mm to improve the fatigue performance of the weld toe. (2) Crack grooving treatment. A narrow V-shaped groove is cut along the crack direction, with a groove depth of 2-3mm, so as to completely expose the crack root. (3) Removal of the entire cracked metal. All metal layers affected by the crack are removed using carbon arc gouging or angle grinder. (4) Repair welding. Multi-layer and multi-pass welding is performed using low-hydrogen welding rods, and the welding heat input is controlled. (5) Post-weld inspection. Magnetic particle or ultrasonic testing is used to confirm that the crack has been completely eliminated. (6) Surface shaping and anti-corrosion treatment. The surface is ground smooth and a new anti-corrosion coating is sprayed.

[0188] Similarly, to facilitate subsequent quality traceability, structured feature vectors, defect morphology, defect mechanism, and repair schemes can be output and recorded.

[0189] The weld defect repair method based on visual images provided in this application processes the target weld defect image based on a preset image segmentation model to identify the weld and weld defects included in the target weld defect image and generate a first pixel-level mask of the weld defects. Then, based on the appearance characteristics of the first pixel-level mask, the defect type of the weld defect is determined. Combining the defect type, a first feature extraction is performed based on the first pixel-level mask and / or the target weld defect image to determine the defect features of the weld defect. Simultaneously, a second feature extraction is performed based on the target weld defect image to determine the structural features of the weld. Further, a first location of the weld defect is determined based on the defect features, and a second location of a preset key structure of the weld is determined based on the structural features. The spatial relationship features between the weld defect and the preset key structure of the weld are determined according to the minimum Euclidean distance between the first and second locations. Next, a structured feature vector is constructed based on the defect features, structural features, and spatial relationship features. This structured feature vector is matched with preset key features in a preset welding mechanism rule base to determine the defect mechanism of the weld defect. Finally, a repair scheme for the weld defect is generated based on the defect mechanism. By extracting defect features, structural features, and spatial relationship features, weld defects are transformed from pixel-level visual data into engineering semantic parameters. This enables the detected weld defects to directly drive the inference of weld defect mechanisms and the generation of repair schemes, thus achieving an automated closed loop for weld defect repair.

[0190] Figure 6 A schematic diagram of the structure of the weld defect repair device based on visual images provided in the embodiments of this application is shown below. Figure 6 As shown, the weld defect repair device 600 based on visual images includes: a determination module 601, a query module 602, and a generation module 603;

[0191] The determining module 601 is used to extract features based on the target weld defect image, determine the defect features of the weld defects included in the target weld defect image, and the structural features of the weld in the target weld defect image.

[0192] The determination module 601 is also used to determine the spatial relationship characteristics between weld defects and preset key structures of the weld based on defect characteristics and structural characteristics.

[0193] The query module 602 is used to query the defect mechanism of weld defects based on a preset welding mechanism rule base by combining defect features, structural features and spatial relationship features;

[0194] The generation module 603 is used to generate a repair scheme for weld defects based on the defect mechanism; wherein, the defect mechanism characterizes the cause of the formation of the defect morphology of the weld defect.

[0195] In one possible design, module 601 is defined as including:

[0196] The processing submodule 6011 is used to process the target weld defect image based on a preset image segmentation model, identify the weld and weld defect included in the target weld defect image, and generate a first pixel-level mask of the weld defect.

[0197] The first determining submodule 6012 is used for:

[0198] Based on the appearance characteristics of the first pixel-level mask, the defect type of the weld defect is determined; and combined with the defect type, a first feature extraction is performed based on the first pixel-level mask and / or the target weld defect image to determine the defect characteristics of the weld defect.

[0199] Second feature extraction is performed based on the target weld defect image to determine the structural features of the weld.

[0200] In one possible design, defect types include crack defects, incomplete penetration defects, slag inclusion defects, and / or porosity defects;

[0201] The first determining submodule 6012 includes: an extraction component, used for:

[0202] Based on the first pixel-level mask and / or the target weld defect image, the first feature extraction algorithm is used to extract the first feature to determine the first feature of crack defects, the second feature of incomplete penetration defects, the third feature of slag inclusion defects, and / or the fourth feature of porosity defects.

[0203] Among them, the defect characteristics of weld defects include first characteristics, second characteristics, third characteristics and / or fourth characteristics.

[0204] In one possible design, the extracted components are also used for:

[0205] Based on the target weld defect image, a second feature extraction algorithm is used to extract the second feature to determine the weld toe and weld root in the weld, as well as the heat-affected zone associated with the weld. The heat-affected zone characterizes the area in the base metal that has not melted but whose microstructure and properties have changed due to heat.

[0206] The structural features of the weld include the weld toe, weld root, and heat-affected zone.

[0207] In one possible design, module 601 further includes:

[0208] The second determining submodule 6013 is used to determine the first location of the weld defect based on defect characteristics; and to determine the second location of the preset key structure of the weld based on structural characteristics; wherein the preset key structure includes the weld toe, weld root and heat-affected zone;

[0209] The calculation submodule 6014 is used to calculate the minimum Euclidean distance between the first position and the second position;

[0210] The third determination submodule 6015 is used to determine the spatial relationship characteristics between weld defects and the preset key structures of the weld based on the minimum Euclidean distance.

[0211] In one possible design, query module 602 includes:

[0212] Submodule 6021 is constructed to build structured feature vectors based on defect features, structural features, and spatial relationship features;

[0213] The matching submodule 6022 is used to match the structured feature vector with the preset key features in the preset welding mechanism rule library;

[0214] The fourth determination submodule 6023 is used for:

[0215] Determine the preset defect mechanism corresponding to the preset key feature that successfully matches the structured feature vector; wherein, the preset welding mechanism rule base includes multiple preset defect morphologies, as well as the preset key features and preset defect mechanisms corresponding to each preset defect morphology;

[0216] The preset defect mechanism is determined as the defect mechanism of weld defects.

[0217] In one possible design, the weld defect repair device 600 based on visual images also includes:

[0218] The acquisition module 604 is used to acquire the second pixel-level mask of the weld seam generated by the preset image segmentation model;

[0219] Extraction module 605 is used to extract the axial center pixel set of the weld seam based on the second pixel-level mask;

[0220] Processing module 606 is used for:

[0221] The axial center pixel set is subjected to least squares fitting to generate the axial centerline of the weld.

[0222] A local weld coordinate system is constructed based on the axial centerline, and defect features and structural features are transformed into the local weld coordinate system.

[0223] The weld defect repair device based on visual images provided in this application can be used to perform the weld defect repair method based on visual images in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0224] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented in software via processing element calls, while others are implemented in hardware. Additionally, these modules can be fully or partially integrated together, or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.

[0225] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device may include: a transceiver 71, a processor 72, and a memory 73.

[0226] Processor 72 executes computer execution instructions stored in memory, causing processor 72 to perform the scheme in the above embodiments. Processor 72 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0227] The memory 73 is connected to the processor 72 via the system bus and completes communication between them. The memory 73 is used to store computer program instructions.

[0228] Transceiver 71 can be used to communicate and interact with other devices.

[0229] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0230] The electronic device provided in this application embodiment can be used to execute the method provided in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0231] This application also provides a computer-readable storage medium storing computer-executable instructions that, when executed on a computer, cause the computer to perform the method provided in any of the above embodiments.

[0232] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the method provided in any of the above embodiments.

[0233] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0234] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0235] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0236] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0237] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0238] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0239] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0240] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0241] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0242] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0243] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for repairing weld defects based on visual images, characterized in that, include: Feature extraction is performed based on the target weld defect image to determine the defect features of the weld defects included in the target weld defect image, as well as the structural features of the weld in the target weld defect image; Based on the defect features and the structural features, the spatial relationship features between the weld defect and the preset key structure of the weld are determined; Combining the defect features, structural features, and spatial relationship features, the defect mechanism of the weld defect is queried based on a preset welding mechanism rule base, and a repair scheme for the weld defect is generated based on the defect mechanism; wherein, the defect mechanism characterizes the cause of the formation of the defect morphology of the weld defect.

2. The method according to claim 1, characterized in that, Feature extraction is performed based on the target weld defect image to determine the defect features of the weld defects included in the target weld defect image, as well as the structural features of the weld in the target weld defect image, including: The target weld defect image is processed based on a preset image segmentation model to identify the weld and weld defect included in the target weld defect image, and a first pixel-level mask of the weld defect is generated. Based on the appearance characteristics of the first pixel-level mask, the defect type of the weld defect is determined; and in combination with the defect type, a first feature extraction is performed based on the first pixel-level mask and / or the target weld defect image to determine the defect characteristics of the weld defect. A second feature extraction is performed based on the target weld defect image to determine the structural features of the weld.

3. The method according to claim 2, characterized in that, The defect types include crack defects, incomplete penetration defects, slag inclusion defects and / or porosity defects; Based on the defect type, and using the first pixel-level mask and / or the target weld defect image, a first feature extraction is performed to determine the defect features of the weld defect, including: Based on the first pixel-level mask and / or the target weld defect image, a first feature extraction algorithm is used to extract the first feature, and the first feature of the crack defect, the second feature of the incomplete penetration defect, the third feature of the slag inclusion defect, and / or the fourth feature of the porosity defect are determined. The defect features of the weld defect include the first feature, the second feature, the third feature, and / or the fourth feature.

4. The method according to claim 2, characterized in that, Based on the target weld defect image, a second feature extraction is performed to determine the structural features of the weld, including: Based on the target weld defect image, a second feature extraction algorithm is used to extract the second feature to determine the weld toe and weld root in the weld, as well as the heat-affected zone associated with the weld; the heat-affected zone characterizes the area in the base metal that has not melted but whose microstructure and properties have changed due to heat. The structural features of the weld include the weld toe, the weld root, and the heat-affected zone.

5. The method according to claim 1, characterized in that, Based on the defect characteristics and the structural characteristics, the spatial relationship characteristics between the weld defect and the preset key structure of the weld are determined, including: Based on the defect characteristics, a first location of the weld defect is determined; and based on the structural characteristics, a second location of a preset key structure of the weld is determined; wherein the preset key structure includes the weld toe, weld root, and heat-affected zone; Calculate the minimum Euclidean distance between the first position and the second position, and based on the minimum Euclidean distance, determine the spatial relationship characteristics between the weld defect and the preset key structure of the weld.

6. The method according to claim 1, characterized in that, Combining the defect features, structural features, and spatial relationship features, the defect mechanism of the weld defect is queried based on a preset welding mechanism rule base, including: Based on the defect features, structural features, and spatial relationship features, a structured feature vector is constructed; The structured feature vector is matched with preset key features in the preset welding mechanism rule base to determine the preset defect mechanism corresponding to the preset key feature that successfully matches the structured feature vector; wherein, the preset welding mechanism rule base includes multiple preset defect morphologies, as well as preset key features and preset defect mechanisms corresponding to each preset defect morphology; The preset defect mechanism is determined as the defect mechanism of the weld defect.

7. The method according to any one of claims 2 to 4, characterized in that, The method further includes: Obtain the second pixel-level mask of the weld seam generated by the preset image segmentation model; based on the second pixel-level mask, extract the axial center pixel set of the weld seam; The axial center pixel set is subjected to least squares fitting to generate the axial center line of the weld. A local weld coordinate system is constructed based on the axial centerline, and the defect features and structural features are transformed into the local weld coordinate system.

8. A weld defect repair device based on visual images, characterized in that, include: The determination module is used to extract features based on the target weld defect image, determine the defect features of the weld defects included in the target weld defect image, and the structural features of the weld in the target weld defect image; The determining module is further configured to determine the spatial relationship characteristics between the weld defect and the preset key structure of the weld based on the defect characteristics and the structural characteristics; The query module is used to combine the defect features, the structural features, and the spatial relationship features to query the defect mechanism of the weld defect based on a preset welding mechanism rule base; A generation module is used to generate a repair scheme for the weld defect based on the defect mechanism; wherein the defect mechanism characterizes the cause of the formation of the defect morphology of the weld defect.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the weld defect repair method based on visual images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the weld defect repair method based on visual images as described in any one of claims 1 to 7.

Images