Weld defect detection method

Abstract

This application provides a method for detecting weld defects. The method includes: acquiring a video frame sequence of a weld region; for each frame in the video frame sequence, determining candidate regions and extracting feature information from the candidate regions; wherein, the candidate region represents a potential area in the image where a weld defect exists, and the feature information includes at least the spatial location, texture direction, and appearance features of the candidate region; constructing a cross-frame trajectory based on the feature information and a preset matching rule, and determining the temporal index information of the cross-frame trajectory; wherein, the cross-frame trajectory is constructed from candidate regions belonging to the same physical defect, and the temporal index information represents the stability of the cross-frame trajectory in the time dimension; and determining the weld detection result based on the temporal index information; wherein, the weld detection result indicates whether the cross-frame trajectory is a real defect or noise. This method aims to improve the stability and accuracy of weld defect detection.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and in particular to a method for detecting weld defects. Background Technology

[0002] In the industrial manufacturing sector, welding technology is widely used in aerospace, shipbuilding, pressure vessels, and steel structures. The quality of the weld directly affects the structural strength and safety reliability of the product. However, various defects such as cracks, porosity, and lack of fusion may occur during the welding process. If these defects are not detected in time, they may lead to serious safety accidents.

[0003] Existing weld defect detection technologies mainly fall into two categories: one is based on single-frame image deep learning methods using fixed or sliding cameras, relying on fixed viewpoints and stable lighting conditions for weld defect detection; the other is based on multimodal data fusion methods, such as simultaneously fusing images with infrared, point cloud, or welding electrical parameters to collaboratively detect weld defects. However, these methods either rely on fixed cameras and multimodal sensors, making them difficult to adapt to handheld mobile scanning and narrow, complex weld path scenarios; or they are based solely on single-frame or static features, making them difficult to adapt to mobile shooting conditions such as camera shake, angle shifts, and lighting changes, and lack the ability to effectively distinguish between real weld defects and noise. Therefore, current weld defect detection suffers from poor stability and low accuracy.

[0004] Therefore, there is an urgent need for a solution that can improve the stability and accuracy of weld defect detection. Summary of the Invention

[0005] The weld defect detection method provided in this application aims to improve the stability and accuracy of weld defect detection.

[0006] In a first aspect, embodiments of this application provide a method for detecting weld defects, including:

[0007] Obtain the video frame sequence of the weld area;

[0008] For each frame in the video frame sequence, candidate regions are determined and feature information of the candidate regions is extracted. The candidate region represents a potential area in the image where weld defects exist, and the feature information includes at least the spatial location, texture direction, and appearance features of the candidate region.

[0009] Based on the feature information and the preset matching rules, cross-frame trajectories are constructed, and the temporal index information of the cross-frame trajectories is determined. The cross-frame trajectories are constructed from candidate regions belonging to the same physical defect, and the temporal index information characterizes the stability of the cross-frame trajectories in the time dimension.

[0010] Based on the time-series index information, the weld inspection results are determined; among them, the weld inspection results characterize whether the cross-frame trajectory is a real defect or noise.

[0011] Optionally, as described above, determining candidate regions of an image includes:

[0012] An image is input into a preset visual detection model to obtain candidate regions; wherein the preset visual detection model is at least one of a convolutional neural network, an object detection network, and a classifier.

[0013] Optionally, as described above, a cross-frame trajectory is constructed based on each feature information and a preset matching rule, including:

[0014] For each pair of adjacent frames, determine the candidate region of the previous image in the pair of adjacent frames as the first candidate region, and determine the candidate region of the next image in the pair of adjacent frames as the second candidate region.

[0015] Based on the feature information of the first candidate region and the feature information of the second candidate region, a matching score is determined; wherein, the matching score represents the degree of similarity between the first candidate region and the second candidate region belonging to the same physical defect;

[0016] If the matching score is greater than the preset matching threshold, then the first candidate region and the second candidate region are determined as the target candidate regions;

[0017] Construct cross-frame trajectories based on each target candidate region.

[0018] Optionally, as described above, the matching score is determined based on the feature information of the first candidate region and the feature information of the second candidate region, including:

[0019] A first score is determined based on a preset first coefficient, the spatial position of the first candidate region, and the spatial position of the second candidate region; wherein, the preset first coefficient is used to adjust the degree of influence of inter-frame spatial displacement on the matching score, and the first score characterizes the spatial proximity between the first candidate region and the second candidate region.

[0020] A second score is determined based on a preset second coefficient, the texture direction of the first candidate region, and the texture direction of the second candidate region. The preset second coefficient is used to adjust the degree of influence of the difference in texture direction between frames on the matching score, and the second score represents the degree of directional consistency between the first candidate region and the second candidate region.

[0021] A third score is determined based on the appearance features of the first candidate region and the second candidate region; the third score represents the degree of appearance similarity between the first candidate region and the second candidate region.

[0022] The first, second, and third scores are weighted and merged to obtain the matching score.

[0023] Optionally, as described above, at least one of the weighting coefficients, preset first coefficients, preset second coefficients, and preset matching thresholds involved in the weighted fusion processing is dynamically adjusted by the acquisition parameters.

[0024] The acquisition parameters include at least one or more of the following: the resolution of the image sensor, the scanning speed of the acquisition device, and the frame rate of the video frame sequence. The image sensor is used to acquire the video frame sequence, the acquisition device is used to move the image sensor relative to the weld, and the scanning speed characterizes the movement speed of the acquisition device relative to the weld.

[0025] Optionally, as described above, a cross-frame trajectory is constructed based on each target candidate region, including:

[0026] The candidate regions of each target are sequentially associated in chronological order to construct cross-frame trajectories;

[0027] If the target candidate region is not determined within a preset number of consecutive frames, or if the inter-frame displacement of the target candidate region between two adjacent frames is greater than the preset maximum displacement, then the construction of the cross-frame trajectory will be terminated.

[0028] Optionally, as described above, if the inter-frame displacement of the target candidate region between two adjacent frames is determined to be greater than a preset maximum displacement, the method further includes:

[0029] Determine the rate of change of the texture direction of the target candidate region between two adjacent frames. If the rate of change of direction is greater than a preset rate of change threshold, increase the preset maximum displacement to obtain the maximum displacement of the target.

[0030] If the inter-frame displacement is greater than the target maximum displacement, the construction of the cross-frame trajectory will be terminated.

[0031] Optionally, as described above, the temporal metrics information for cross-frame trajectories are determined, including:

[0032] The actual number of frames in which the candidate region exists in the cross-frame trajectory and the total number of frames spanned by the cross-frame trajectory are determined, and a first index feature is determined based on the actual number of frames and the total number of frames; wherein, the first index feature characterizes the ability of the candidate region to appear stably in the cross-frame trajectory;

[0033] The inter-frame displacement of the candidate region in the cross-frame trajectory is determined between each adjacent frame, and a second index feature is determined based on the inter-frame displacement and the preset maximum displacement; wherein, the second index feature characterizes the stability of the position migration of the candidate region between consecutive frames.

[0034] The morphological information of candidate regions in the cross-frame trajectory is determined, and a third index feature is determined based on the morphological information. The morphological information includes area information, brightness information and texture information, and the third index feature characterizes the change range of the appearance of the candidate region between different frames.

[0035] The first, second, and third indicator features are weighted and fused to obtain the time-series indicator information of the cross-frame trajectory.

[0036] Optionally, as described above, the weld inspection results are determined based on time-series index information, including:

[0037] If the time-series index information is greater than the preset confidence threshold, then the weld inspection result is determined to be a real defect;

[0038] If the time-series index information is less than or equal to the preset confidence threshold, the weld inspection result is determined to be noise.

[0039] Optionally, if the weld inspection result is a true defect, the method further includes:

[0040] The attribute information of the real defect is determined and displayed. The attribute information includes the defect location, defect type, time sequence index information and key frame image. The defect location represents the spatial coordinates of the real defect in the weld area, the defect type represents the category attribute of the real defect, and the key frame image represents the video frame with the best clarity in the cross-frame trajectory.

[0041] Secondly, embodiments of this application provide a weld defect detection device, comprising:

[0042] The acquisition module is used to acquire video frame sequences of the weld area;

[0043] The extraction module is used to determine the candidate regions of each frame in the video frame sequence and extract the feature information of the candidate regions; wherein, the candidate region represents the potential area in the image where there is a weld defect, and the feature information includes at least the spatial location, texture direction and appearance features of the candidate region;

[0044] The construction module is used to construct cross-frame trajectories based on various feature information and preset matching rules, and to determine the temporal index information of the cross-frame trajectories; wherein, the cross-frame trajectories are constructed from candidate regions belonging to the same physical defect, and the temporal index information characterizes the stability of the cross-frame trajectories in the time dimension.

[0045] The determination module is used to determine the weld inspection result based on the time-series index information; wherein, the weld inspection result represents whether the cross-frame trajectory is a real defect or noise.

[0046] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0047] The memory stores the instructions that the computer executes;

[0048] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0049] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0050] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0051] The weld defect detection method provided in this application obtains a video frame sequence of the weld area. Further, for each frame in the video frame sequence, candidate regions of the image are determined, and feature information of the candidate regions is extracted. Further, based on the feature information and preset matching rules, a cross-frame trajectory is constructed, and the temporal index information of the cross-frame trajectory is determined. Further, based on the temporal index information, the weld detection result is determined. Here, the candidate region represents a potential area in the image where a weld defect exists; the feature information includes at least the spatial location, texture direction, and appearance features of the candidate region; the cross-frame trajectory is constructed from candidate regions belonging to the same physical defect; the temporal index information represents the stability of the cross-frame trajectory in the time dimension; and the weld detection result represents whether the cross-frame trajectory is a real defect or noise. The weld defect detection method provided in this application achieves continuous tracking of the same physical defect across multiple frames by constructing cross-frame trajectories. It enhances the accuracy of cross-frame trajectory construction by extracting the spatial location, texture direction, and appearance features of candidate regions. By determining temporal index information, it quantifies the difference between the cross-frame stability of real defects and the transient and random nature of noise. Defect judgment based on temporal index information effectively distinguishes between real defects and transient noise. The method of this application improves the stability and accuracy of weld defect detection. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] Figure 1 Flowchart of the weld defect detection method provided in this application Figure 1 ;

[0054] Figure 2 Flowchart of the weld defect detection method provided in this application Figure 2 ;

[0055] Figure 3 Flowchart of the weld defect detection method provided in this application Figure 3 ;

[0056] Figure 4 Flowchart of the weld defect detection method provided in this application Figure 4 ;

[0057] Figure 5 Flowchart of the weld defect detection method provided in this application Figure 5 ;

[0058] Figure 6 A schematic diagram of the weld defect detection device provided in this application;

[0059] Figure 7 A schematic diagram of the structure of the electronic device provided in this application.

[0060] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0062] In the industrial manufacturing sector, welding technology is widely used in aerospace, shipbuilding, pressure vessels, and steel structures. The quality of the weld directly affects the structural strength and safety reliability of the product. However, various defects such as cracks, porosity, and lack of fusion may occur during the welding process. If these defects are not detected in time, they may lead to serious safety accidents.

[0063] Existing weld defect detection technologies mainly fall into two categories: one is a single-frame image deep learning method based on a fixed camera or sliding camera, which relies on a fixed viewpoint and stable lighting conditions for weld defect detection; the other is a detection method based on multimodal data fusion, such as simultaneously fusing images with infrared, point cloud, or welding electrical parameters to collaboratively detect weld defects.

[0064] However, the above method has the following shortcomings:

[0065] 1) Inability to maintain stability in handheld mobile scanning scenarios: Single-frame image methods rely on a fixed viewpoint and stable lighting, making it difficult to adapt to shaking, angle shifts, and viewpoint changes during handheld shooting. Multimodal sensor solutions require complex deployment and are not suitable for narrow spaces or complex weld seams.

[0066] 2) Lack of utilization of cross-frame temporal patterns in video: Actual weld defects exhibit cross-frame stability in video sequences, while noise such as reflections and dust flashes only appear in a very small number of frames, with large positional jumps. Existing methods are all based on single-frame or static features, failing to utilize this temporal characteristic that allows for the distinction between stable structures and transient noise.

[0067] 3) General video tracking techniques cannot be directly used for weld inspection: Although there are existing video tracking techniques such as optical flow, KLT tracking algorithm, SORT algorithm, and DeepSort algorithm, welds have the following special characteristics: the grayscale of weld texture is uniform and highly repetitive, which leads to unstable optical flow characteristics; specular reflection and local highlights often cause feature point drift or jump; welds have turns and bends, and conventional tracking algorithms cannot maintain trajectory continuity at geometric abrupt changes; noise and defects may be similar in single-frame features, making it difficult to distinguish them by tracking alone.

[0068] Therefore, the current technical problems of poor stability and low accuracy in weld defect detection exist.

[0069] Therefore, there is an urgent need for a solution that can improve the stability and accuracy of weld defect detection.

[0070] The weld defect detection method provided in this application aims to solve the problem that existing weld defect detection technologies are easily affected by shaking, angle shift and lighting changes under mobile shooting conditions, resulting in unstable detection results. At the same time, it solves the problem that existing methods rely on single-frame features and cannot utilize the key temporal pattern of cross-frame stability of weld defects and transient and random noise, thus resulting in high false detection rate and severe trajectory interruption in strong reflection, texture repetition and noise interference scenarios.

[0071] The weld defect detection method provided in this application achieves continuous tracking of the same physical defect across multiple frames by constructing cross-frame trajectories. It enhances the accuracy of cross-frame trajectory construction by extracting the spatial location, texture direction, and appearance features of candidate regions. By determining temporal index information, it quantifies the difference between the cross-frame stability of real defects and the transient and random nature of noise. Defect judgment based on temporal index information effectively distinguishes between real defects and transient noise. The method of this application improves the stability and accuracy of weld defect detection.

[0072] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0073] Figure 1 Flowchart of the weld defect detection method provided in this application Figure 1 ,like Figure 1 As shown, the execution subject of this method can be a host, server, or other device, and the method includes:

[0074] S101. Obtain the video frame sequence of the weld area.

[0075] S102. For each frame of the video frame sequence, determine the candidate region of the image and extract the feature information of the candidate region; wherein, the candidate region represents the potential area in the image where there is a weld defect, and the feature information includes at least the spatial location, texture direction and appearance features of the candidate region.

[0076] S103. Based on the feature information and the preset matching rules, construct the cross-frame trajectory and determine the temporal index information of the cross-frame trajectory; wherein, the cross-frame trajectory is constructed from candidate regions belonging to the same physical defect, and the temporal index information characterizes the stability of the cross-frame trajectory in the time dimension.

[0077] S104. Determine the weld inspection result based on the time-series index information; wherein, the weld inspection result characterizes the cross-frame trajectory as either a real defect or noise.

[0078] In step S101, scanning can be performed along the weld surface to obtain a continuous sequence of video frames.

[0079] Among them, the weld seam area can refer to the connection area formed by the welding process on the workpiece to be inspected and its adjacent areas, and the video frame sequence can refer to a set of continuous image frames arranged in chronological order, with a certain time interval between each frame.

[0080] For example, an acquisition device (such as a camera, handheld terminal, or mobile camera device) can be used to scan along the weld direction at a fixed frame rate (e.g., 30 or 60 frames per second (fps)) to obtain a continuous video data containing weld surface information, i.e., a video frame sequence of the weld area.

[0081] In one possible implementation, to accommodate factors that may affect image quality, such as lighting fluctuations and slight shaking, during mobile shooting, image preprocessing can be performed on each frame.

[0082] For example, image preprocessing may include, but is not limited to, noise suppression, illumination equalization, and edge enhancement operations. Noise suppression may involve using methods such as Gaussian filtering, median filtering, or bilateral filtering to suppress random noise in the image; illumination equalization may involve using methods such as histogram equalization or adaptive gamma correction to compensate for uneven illumination; and edge enhancement may involve using methods such as the Sobel operator or the Laplacian operator to enhance the contrast of weld edges.

[0083] In one possible implementation, a region of interest (ROI) can be extracted from the weld area based on the weld brightness gradient, texture direction, or geometric contour. This ROI serves as a preprocessed image, narrowing the scope of subsequent processing and improving weld defect detection efficiency. Specifically, the weld can be located by detecting regions in the image that have a high brightness gradient (e.g., a gradient magnitude greater than a preset gradient threshold, which can be set according to the image's grayscale dynamic range, such as 5% to 10% of the maximum grayscale value) and exhibit a linear or curved distribution. A preset pixel width (e.g., 50 to 100 pixels) can then be extended along the weld to both sides to generate a rectangular or polygonal ROI containing complete weld information.

[0084] It should be understood that the above image preprocessing and ROI extraction operations are optional steps, and their specific implementation methods can be flexibly adjusted or combined according to the actual application scenario.

[0085] For example, in scenarios with good lighting conditions and stable acquisition equipment, the original image can be directly processed; in scenarios with complex lighting or strong noise interference, multiple preprocessing operations can be combined to improve image quality.

[0086] In step S102, the candidate region represents a potential area in the image where there is a weld defect. This can be understood as a suspected abnormal area identified in a single frame image through preliminary detection. This area may correspond to a real defect or noise interference such as reflection or stains, and further verification and analysis are required.

[0087] It should be understood that at least one candidate region can be identified in a frame of an image, or there may be no candidate region. When there are multiple candidate regions in a frame of an image, each candidate region can be processed independently. That is, when constructing the same cross-frame trajectory, the candidate regions in the cross-frame trajectory come from different frames.

[0088] In one alternative implementation, determining candidate regions of an image may include:

[0089] An image is input into a preset visual detection model to obtain candidate regions; wherein the preset visual detection model is at least one of a convolutional neural network, an object detection network, and a classifier.

[0090] Among these, the pre-defined visual detection model can refer to a machine learning model that has been trained to identify abnormal features in the weld area. Convolutional neural networks can refer to a deep neural network structure containing convolutional layers, pooling layers, and fully connected layers, used to extract hierarchical features of images; object detection networks can refer to network structures that can simultaneously output the target location and category; classifiers can refer to classification models such as support vector machines and random forests.

[0091] In one possible implementation, the image is input into a pre-defined visual detection model. Alternatively, a pre-processed image (e.g., a Region of Interest) can be input into the pre-defined visual detection model to obtain output candidate regions, which can also be called candidate points. During the use of the pre-defined visual detection model, candidate points can be extracted using features such as brightness anomalies, texture interruptions, and geometric abrupt changes. The aim is to cover as many potential defect locations as possible as a basis for subsequently constructing cross-frame trajectories, rather than directly using them as the final criterion for weld defect determination.

[0092] It is understandable that by introducing a pre-set visual detection model to determine the candidate region, a high recall rate for initial screening of weld defects can be achieved at the single-frame level. This provides sufficient candidate samples for subsequent construction of cross-frame trajectories and analysis of the stability of cross-frame trajectories over time, thus avoiding overall detection failure due to missed detections in a single frame.

[0093] Feature information can refer to a quantized representation that describes the attributes of candidate regions and supports cross-frame association matching. Feature information can include at least the spatial location, texture orientation, and appearance features of the candidate regions. Optionally, feature information can also include, but is not limited to, optical flow vector fields, scale-invariant feature points (SIFT (Scale-invariant feature transform) feature points, SURF (Speeded Up Robust Features) feature points, multi-scale texture features, or depth feature embedding vectors.

[0094] Among them, the spatial location of the candidate region can refer to the coordinates of the center point of the candidate region in the image coordinate system; the texture direction of the candidate region can refer to the main direction angle of the weld texture in the candidate region, which can be obtained through gradient histogram or structural tensor analysis; the appearance features of the candidate region can refer to the feature vector describing the visual characteristics of the candidate region, which can be obtained by deep features extracted by convolutional neural networks or traditionally hand-designed features (such as color histograms).

[0095] It is understandable that by extracting the feature information of candidate regions, a multi-dimensional similarity measurement basis can be provided for cross-frame association matching of candidate regions in subsequent cross-frame trajectory construction, thereby improving the recognition accuracy of the same physical defect in consecutive frames.

[0096] In step S103, it should be understood that the candidate region and the feature information have a one-to-one correspondence.

[0097] Cross-frame trajectory can refer to a continuous trajectory sequence formed by associating candidate regions belonging to the same physical defect in a video frame sequence in chronological order according to a preset matching rule.

[0098] The preset matching rules can refer to the criteria used to determine whether candidate regions in different frames originate from the same physical defect. The preset matching rules are used to establish the correspondence between candidate regions in the time dimension to ensure the continuity and accuracy of the trajectory.

[0099] For example, the preset matching rules may include: for candidate regions between adjacent frames, a matching score is calculated based on the proximity of spatial location, the consistency of texture direction, and the similarity of appearance features; if the matching score is greater than a preset matching threshold, they are determined to be the same defect; otherwise, they are determined to be different defects or newly appearing defects. A matching score greater than the preset matching threshold indicates that the candidate region of the next frame and the candidate region of the previous frame originate from the same physical defect, which can be used to construct or continue cross-frame trajectories.

[0100] Furthermore, after constructing the cross-frame trajectory, the temporal index information of the cross-frame trajectory can be determined.

[0101] Among them, time-series index information can refer to comprehensive characterization parameters used to quantify the stability of cross-frame trajectories in the time dimension.

[0102] In one possible implementation, time-series index information can be quantified by the proportion of consecutive occurrences of target candidate regions, the stability of positional changes, and the magnitude of morphological changes. Alternatively, it can be quantified by introducing statistics such as trajectory length, velocity change, acceleration change, and direction change.

[0103] In step S104, the timing index information can be analyzed to determine whether the weld inspection result is a real defect or noise.

[0104] Among them, real defects can refer to physical defects such as cracks, pores, and slag inclusions that actually exist in the weld, while noise refers to false abnormal areas formed in the image by interference factors such as reflection, dust, and stains.

[0105] It should be understood that real defects appear stably in multiple consecutive frames with gradual feature changes, which conforms to the cross-frame stability law; noise appears randomly in a small number of frames with drastic feature jumps, which conforms to the transient randomness law; by quantitatively analyzing the time series index information, the above two situations can be effectively distinguished.

[0106] In one alternative implementation, step S104 may include:

[0107] S1041. If the time-series index information is greater than the preset confidence threshold, then the weld inspection result is determined to be a real defect.

[0108] S1042. If the time-series index information is less than or equal to the preset confidence threshold, then the weld detection result is determined to be noise.

[0109] The preset confidence threshold can refer to the temporal confidence threshold used to distinguish between real defects and noise, and can be set in the range of 0.5 to 0.7. In one possible implementation, the preset confidence threshold can be adaptively adjusted according to the reflection intensity, weld type (e.g., straight weld, circumferential weld, polygonal weld, or spatial curved surface weld), image jitter, and texture complexity to improve the stability of weld defect detection under complex shooting conditions. For example, in a strongly reflective scene, the preset confidence threshold can be appropriately reduced due to increased noise interference; in a jittery scene, the preset confidence threshold can be appropriately reduced due to decreased reliability of positional stability features.

[0110] If the timing index information is greater than the preset confidence threshold, it indicates that the cross-frame trajectory has significant timing stability and conforms to the presentation pattern of real defects. In this case, the weld detection result can be determined to be a real defect. If the timing index information is less than or equal to the preset confidence threshold, it indicates that the timing stability of the cross-frame trajectory is insufficient and conforms to the presentation pattern of noise. In this case, the weld detection result can be determined to be noise.

[0111] It is understandable that by judging the relationship between time-series index information and preset confidence threshold, and thus determining whether the weld detection result is a real defect or noise, the stability quantification analysis at the time-series level can be transformed into the final weld defect judgment decision, thereby significantly improving the robustness and accuracy of weld defect detection under mobile shooting and complex working conditions.

[0112] The weld defect detection method provided in this application achieves continuous tracking of the same physical defect across multiple frames by constructing cross-frame trajectories. It enhances the accuracy of cross-frame trajectory construction by extracting the spatial location, texture direction, and appearance features of candidate regions. By determining temporal index information, it quantifies the difference between the cross-frame stability of real defects and the transient and random nature of noise. Defect judgment based on temporal index information effectively distinguishes between real defects and transient noise. The method of this application improves the stability and accuracy of weld defect detection.

[0113] Figure 2 Flowchart of the weld defect detection method provided in this application Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1Based on the examples, this paper provides a detailed explanation of how to construct cross-frame trajectories in the weld defect detection method. The method includes:

[0114] S201. For each pair of adjacent frames, determine the candidate region of the previous image in the pair of adjacent frames as the first candidate region, and determine the candidate region of the next image in the pair of adjacent frames as the second candidate region.

[0115] For example, for two adjacent frames, the (t-1)th frame and the tth frame, the candidate region of the (t-1)th frame can be represented as... That is, the j-th first candidate region of the previous image, the candidate region of the t-th frame image can be represented as That is, the i-th second candidate region of the next image.

[0116] S202. Determine the matching score based on the feature information of the first candidate region and the feature information of the second candidate region; wherein, the matching score represents the degree of similarity between the first candidate region and the second candidate region as belonging to the same physical defect.

[0117] In one alternative implementation, step S202 may include:

[0118] S2021. Determine the first score based on the preset first coefficient, the spatial position of the first candidate region, and the spatial position of the second candidate region; wherein, the preset first coefficient is used to adjust the degree of influence of inter-frame spatial displacement on the matching score, and the first score characterizes the spatial proximity between the first candidate region and the second candidate region.

[0119] For example, the first score can satisfy:

[0120] ;

[0121] in, This indicates the first score. This represents the spatial location of the j-th first candidate region in the previous image. This indicates the spatial location of the i-th second candidate region in the subsequent image. This represents the preset first coefficient. This represents an exponential function used to map spatial distances to proximity measures in the range of 0 to 1.

[0122] The preset first coefficient is used to adjust the influence of inter-frame spatial displacement on the matching score. In one possible implementation, the natural displacement of the candidate region between consecutive frames can be determined by the frame rate (i.e., the number of image frames acquired per unit time, which can be frames per second), the scanning speed (i.e., the moving speed of the acquisition device relative to the weld, which can be millimeters per second), and the pixel equivalent (i.e., the conversion relationship between the actual physical distance and image pixels, which can be pixels per millimeter). The preset first coefficient can satisfy the following:

[0123] ;

[0124] in, Indicates the scanning speed. This represents the frame interval, which is the time interval between two adjacent frames as indicated by the frame rate. The pixel equivalent is represented by the focal length, object distance, and pixel size of the image sensor (e.g., a camera). For example, the pixel equivalent... satisfy:

[0125] ;

[0126] in, Represents pixel equivalent. This indicates the object distance (i.e., the distance from the camera lens to the weld surface). Indicates the camera's focal length. This indicates the pixel size of the image sensor.

[0127] For example, the camera has a focal length of 50 mm, an object distance of 100 mm, a sensor pixel size of 2 micrometers, and a pixel equivalent of 0.004 mm per pixel.

[0128] For example, the preset first coefficient Within a range of 5 to 15 pixels, to ensure coverage of more than 95% of the candidate region within the normal inter-frame displacement range between consecutive frames.

[0129] S2022. Determine the second score based on the preset second coefficient, the texture direction of the first candidate region, and the texture direction of the second candidate region; wherein, the preset second coefficient is used to adjust the degree of influence of the difference in texture direction between frames on the matching score, and the second score represents the degree of directional consistency between the first candidate region and the second candidate region.

[0130] For example, the second score can satisfy:

[0131] ;

[0132] in, This indicates the second score. This indicates the texture direction of the j-th first candidate region in the previous image. This indicates the texture orientation of the i-th second candidate region in the subsequent image. This represents the preset second coefficient. This represents an exponential function used to map angular differences to a consistency measure in the range of 0 to 1.

[0133] A preset second coefficient is used to adjust the degree of influence of inter-frame texture direction differences on the matching score. For example, the preset second coefficient... Within a range of 5 to 15 degrees, to cover the reasonable range of changes in weld texture direction during normal scanning.

[0134] S2023. Determine a third score based on the appearance features of the first candidate region and the second candidate region; wherein the third score represents the degree of appearance similarity between the first candidate region and the second candidate region.

[0135] For example, the third score can satisfy:

[0136] ;

[0137] in, This indicates the third score. This represents the appearance features of the j-th first candidate region in the previous image. This represents the appearance features of the i-th second candidate region in the subsequent image. This represents the cosine similarity function, used to calculate the cosine value of the angle between two feature vectors.

[0138] S2024. The first score, the second score, and the third score are weighted and fused to obtain the matching score.

[0139] Weighted fusion processing refers to the calculation process of linearly combining multiple values ​​(here, the first score, the second score, and the third score) according to preset weights. Matching score refers to a quantitative indicator that comprehensively reflects the probability that the first candidate region and the second candidate region belong to the same physical defect.

[0140] For example, the matching score satisfies:

[0141] ;

[0142] in, Indicates the matching score. This indicates the first score. This represents the weighting coefficient corresponding to the first score, and can be called the first weighting coefficient. This indicates the second score. This represents the weighting coefficient corresponding to the second score, and can be called the second weighting coefficient. This indicates the third score. The weight coefficient corresponding to the third score can be called the third weight coefficient.

[0143] In one possible implementation, , , It can be adjusted within the range of 0.2 to 0.5 according to the actual working conditions on site, and meets the requirements. This allows for flexible contributions of different feature dimensions to the matching score.

[0144] It is understandable that by weighting and fusing the first, second, and third scores, a matching score is obtained. This enables a comprehensive evaluation of the similarity of candidate regions from three dimensions: spatial location, texture direction, and appearance features. This can fully characterize the correlation characteristics of the same physical defect in consecutive frames, thereby improving the accuracy and robustness of cross-frame trajectory construction.

[0145] S203. If the matching score is greater than the preset matching threshold, then the first candidate region and the second candidate region are determined as the target candidate regions.

[0146] The preset matching threshold can refer to the similarity boundary value used to determine whether two candidate regions from adjacent frames belong to the same physical defect. For example, the preset matching threshold is in the range of 0.5 to 0.7.

[0147] If the matching score is greater than the preset matching threshold, it indicates that the first candidate region and the second candidate region are highly consistent in terms of spatial location, texture direction and appearance features. Therefore, the first candidate region and the second candidate region belong to the same physical defect and are determined as target candidate regions. Conversely, if the matching score is less than or equal to the preset matching threshold, it indicates that the similarity between the two candidate regions is insufficient. Therefore, they are determined to be different defects or newly appearing defects and are not included in the same cross-frame trajectory.

[0148] S204. Construct cross-frame trajectories based on each target candidate region.

[0149] Among them, cross-frame trajectory can refer to a continuous sequence formed by concatenating target candidate regions belonging to the same physical defect in chronological order; there can be at least one cross-frame trajectory or multiple cross-frame trajectories, and each cross-frame trajectory is independent of the others and corresponds to different physical defects.

[0150] In one alternative implementation, step S204 may include:

[0151] S2041. Associate each target candidate region sequentially according to time order to construct a cross-frame trajectory.

[0152] Specifically, for target candidate regions that meet the matching conditions, they are added to the end of an existing cross-frame trajectory or used as the starting point of a new cross-frame trajectory to form a continuous trajectory sequence arranged in chronological order.

[0153] For example, a cross-frame trajectory can be represented as:

[0154] ,

[0155] in, This represents a cross-frame trajectory, which is composed of target candidate regions from n frames of images. The target candidate regions in the cross-frame trajectory come from different frames. This represents the first target candidate region in the cross-frame trajectory, and n represents the number of target candidate regions in the cross-frame trajectory.

[0156] It should be understood that the starting point of the cross-frame trajectory is the first target candidate region that meets the matching conditions, and the target candidate regions that meet the matching conditions in subsequent frames continue the trajectory in turn.

[0157] S2042. If the target candidate region is not determined within a preset number of consecutive frames, or if the inter-frame displacement of the target candidate region between two adjacent frames is greater than the preset maximum displacement, then the construction of the cross-frame trajectory is terminated.

[0158] The consecutive preset frame number can refer to the maximum number of consecutive frames that allow cross-frame trajectory interruption. For example, it can be set to 3 frames. If the target candidate area is not determined within the consecutive preset frame number, it indicates that the physical defect may have disappeared or the tracking has been lost. Then, the construction of the current cross-frame trajectory is terminated to avoid the continuation of the error.

[0159] The preset maximum displacement can refer to the maximum Euclidean distance between the center positions of candidate regions between adjacent frames. For example, the preset maximum displacement can be preset according to the image resolution, scanning speed and frame rate, for example, set to 10 to 20 pixels.

[0160] If the inter-frame displacement of the target candidate region between two adjacent frames is greater than the preset maximum displacement, it indicates that the spatial position of the candidate region may have changed abnormally, such as due to occlusion, defocusing, or tracking errors. In this case, the construction of the current cross-frame trajectory will be terminated.

[0161] It is understandable that by associating target candidate regions that meet the matching conditions in chronological order and terminating the construction of cross-frame trajectories when there is a long interruption or abnormal displacement, the continuity and rationality of cross-frame trajectories can be ensured, avoiding the continuation of erroneous trajectories due to noise interference or tracking failure, thereby improving the reliability of cross-frame trajectory construction.

[0162] In an optional implementation, if the inter-frame displacement of the target candidate region between two adjacent frames is determined to be greater than a preset maximum displacement, the method may further include:

[0163] Determine the rate of change of the texture direction of the target candidate region between two adjacent frames. If the rate of change of direction is greater than the preset rate of change threshold, increase the preset maximum displacement to obtain the target maximum displacement. If the inter-frame displacement is greater than the target maximum displacement, terminate the construction of the cross-frame trajectory.

[0164] The direction change rate can refer to the absolute value of the difference in texture direction between two adjacent frames of the target candidate region. The preset change rate threshold can refer to the angle threshold used to determine whether the weld has turned, for example, it can be set to 30 degrees.

[0165] If the rate of change of direction is greater than the preset rate of change threshold, it indicates that the weld may have turned or bent, causing the inter-frame position jump to exceed the normal range. In this case, the adaptive adjustment mechanism of the maximum displacement is triggered, that is, the preset maximum displacement is increased to obtain the target maximum displacement.

[0166] The target maximum displacement is greater than the preset maximum displacement. For example, the target maximum displacement can be a preset multiple (greater than 1) of the preset maximum displacement, such as 1.5 times.

[0167] Furthermore, if the rate of change of direction is less than or equal to the preset rate of change threshold, but the inter-frame displacement is still greater than the preset maximum displacement, it is determined to be an abnormal jump, and the construction of the cross-frame trajectory is directly terminated.

[0168] It is understandable that if the rate of change of direction is greater than the preset rate of change threshold, it indicates that the weld may turn or bend. By increasing the preset maximum displacement, it is possible to adapt to the inter-frame position jump caused by the change of weld geometry, thereby maintaining the continuity of cross-frame trajectory under complex path conditions and improving the adaptability of weld defect detection to weld turning scenarios.

[0169] In one optional implementation, at least one of the weighting coefficients, preset first coefficients, preset second coefficients, and preset matching thresholds involved in the weighted fusion process is dynamically adjusted by the acquisition parameters.

[0170] The acquisition parameters include at least one or more of the following: the resolution of the image sensor, the scanning speed of the acquisition device, and the frame rate of the video frame sequence. The image sensor is used to acquire the video frame sequence, the acquisition device is used to move the image sensor relative to the weld, and the scanning speed characterizes the movement speed of the acquisition device relative to the weld.

[0171] It should be understood that the above dynamic adjustments can be achieved through preset mapping rules or lookup table mechanisms to ensure that the adaptation relationship between relevant parameters and collected parameters is clear and reproducible.

[0172] For example, if the image sensor resolution is 1080p (progressive scan), the scanning speed of the acquisition device is 5 mm / s, and the frame rate of the video frame sequence is 60 frames / s, then the first weighting coefficient involved in the weighted fusion processing is... The second weighting coefficient is 0.4. 0.3, third weighting coefficient The preset first coefficient is 0.3. 8 pixels, preset second coefficient Given an 8-degree angle and a preset matching threshold of 0.6; and considering the image sensor resolution of 720p, the scanning speed of the acquisition device of 2 millimeters per second, and the frame rate of the video frame sequence of 30 frames per second, the first weighting coefficient involved in the weighted fusion processing is... The second weighting coefficient is 0.5. 0.2, third weighting coefficient The preset first coefficient is 0.3. 12 pixels, preset second coefficient The degree is 12, and the preset matching threshold is 0.55.

[0173] It is understandable that by dynamically adjusting the weighting coefficients, preset first coefficients, preset second coefficients, and preset matching thresholds involved in the weighted fusion processing according to the acquired parameters, the cross-frame trajectory construction method can be ensured to adapt to different equipment configurations and operating conditions, thereby achieving adaptive optimization of the weld defect detection method and improving the versatility and engineering practicality of the weld defect detection method.

[0174] The weld defect detection method provided in this application improves the accuracy and continuity of constructing cross-frame trajectories by using a multi-dimensional matching mechanism that integrates spatial location, texture direction, and appearance features, as well as a dynamic adaptive adjustment method based on acquisition parameters. This significantly improves the stability and accuracy of weld defect detection under mobile shooting and complex working conditions.

[0175] Figure 3 Flowchart of the weld defect detection method provided in this application Figure 3 ,like Figure 3 As shown, this embodiment, based on the aforementioned embodiments, provides a detailed explanation of how to determine the timing index information of cross-frame trajectories in the weld defect detection method. The method includes:

[0176] S301. Determine the actual number of frames in which the candidate region exists in the cross-frame trajectory and the total number of frames spanned by the cross-frame trajectory, and determine the first index feature based on the actual number of frames and the total number of frames; wherein, the first index feature characterizes the ability of the candidate region to appear stably in the cross-frame trajectory.

[0177] For example, the first indicator feature can satisfy:

[0178] ;

[0179] in, Indicates the first indicator characteristic, This indicates the actual number of frames in which the candidate region exists within the cross-frame trajectory. This indicates the total number of frames spanned by the cross-frame trajectory.

[0180] For example, if a cross-frame trajectory spans 10 frames and the cross-frame trajectory consists of candidate regions from 8 frames, then the first indicator feature is 0.8.

[0181] It should be understood that the closer the first indicator feature is to 1, the higher the proportion of the candidate region's continuous occurrence in the cross-frame trajectory, which conforms to the stable cross-frame presentation of real defects; the lower the first indicator feature is, the more frequently the candidate region is interrupted, which conforms to the transient and random characteristics of noise. For example, a first indicator feature within the threshold range of 0.5 to 0.7 can effectively distinguish stable structures from transient noise. When the first indicator feature is greater than this threshold range, the cross-frame trajectory can be identified as a real defect; when the first indicator feature is less than this threshold range, the cross-frame trajectory can be identified as noise.

[0182] S302. Determine the inter-frame displacement of the candidate region in the cross-frame trajectory between adjacent frames, and determine the second index feature based on the inter-frame displacement and the preset maximum displacement; wherein, the second index feature characterizes the stability of the position migration of the candidate region between consecutive frames.

[0183] For example, if the candidate region between adjacent frames is defined as coming from frame (k-1) and frame k, then the inter-frame displacement between these adjacent frames can be expressed as:

[0184] ;

[0185] in, This represents the inter-frame displacement between frame (k-1) and frame k. This indicates the spatial location of the candidate region in the (k-1)th frame. This represents the spatial location of the candidate region in the k-th frame.

[0186] The second indicator characteristic can satisfy:

[0187] ;

[0188] in, This indicates the characteristics of the second indicator. This represents the inter-frame displacement between frame (k-1) and frame k. This indicates the preset maximum displacement. This indicates the length of the cross-frame trajectory, which is the total number of candidate regions in the cross-frame trajectory.

[0189] It should be understood that the closer the second indicator feature is to 1, the smoother the positional migration of the candidate region, which conforms to the continuous and stable trajectory characteristics of a real defect; the lower the second indicator feature is, the more drastic the positional jump, which conforms to the random drift characteristics of noise. For example, a real defect corresponds to a smooth trajectory, and noise corresponds to a jump trajectory. Using, for example, 0.6 as a judgment threshold, stable structures and instantaneous noise can be effectively distinguished. When the second indicator feature is greater than 0.6, the cross-frame trajectory can be identified as a real defect; when the second indicator feature is less than 0.6, the cross-frame trajectory can be identified as noise.

[0190] S303. Determine the morphological information of candidate regions in the cross-frame trajectory, and determine the third index feature based on each morphological information; wherein, the morphological information includes area information, brightness information and texture information, and the third index feature characterizes the change range of the appearance morphology of the candidate region between different frames.

[0191] Among them, area information can refer to the total number of pixels or the actual physical area occupied by the candidate region in the image, brightness information can refer to the average gray value or average brightness value of all pixels in the candidate region, and texture information can refer to the feature vector describing the surface texture characteristics of the candidate region, or it can be understood as the appearance features in step S102, which are used to characterize the visual appearance characteristics of the candidate region, for example, extracted by a convolutional neural network.

[0192] For example, the candidate region is defined to come from the (k-1)th frame and the kth frame. The normalized difference of the candidate region between the (k-1)th frame and the kth frame can be expressed as: , It can satisfy:

[0193] ;

[0194] in, , , They represent the preset feature weights, satisfying... , This represents the area information of the candidate region from the (k-1)th frame. This represents the area information of the candidate region from the k-th frame. This represents the average area information of the candidate regions in each frame. This represents the brightness information of the candidate region from the (k-1)th frame. This represents the brightness information of the candidate region from the k-th frame. This represents the average brightness information of the candidate regions in each frame. This represents the texture information of the candidate region from the (k-1)th frame. This represents the texture information of the candidate region from the k-th frame.

[0195] The third indicator characteristic can be satisfied:

[0196] ;

[0197] in, This indicates the characteristics of the third indicator. This represents the normalized difference of the candidate region between frame (k-1) and frame k.

[0198] It should be understood that the closer the third indicator feature is to 1, the more stable the appearance of the candidate region, which is consistent with the characteristic that real defects are affected by slow changes in viewing angle; the lower the third indicator feature is, the more drastic the appearance changes, which is consistent with the characteristic that noise is greatly affected by illumination and reflection. For example, the morphological changes of real defects are mainly affected by slow changes in camera viewing angle, and the amount of change is small; noise is greatly affected by illumination and reflection. Therefore, a judgment threshold can be set in the range of 0.5 to 0.7. When the third indicator feature is greater than the judgment threshold, the cross-frame trajectory can be identified as a real defect; when the third indicator feature is less than or equal to the judgment threshold, the cross-frame trajectory can be identified as noise.

[0199] S304. Perform weighted fusion processing on the first indicator feature, the second indicator feature, and the third indicator feature to obtain the time-series indicator information of the cross-frame trajectory.

[0200] The weighted fusion process in this step is similar to the weighted fusion process in step S2024. For example, the weight coefficients involved in the weighted fusion process in this step can be the fourth weight coefficient corresponding to the first indicator feature, the fifth weight coefficient corresponding to the second indicator feature, and the sixth weight coefficient corresponding to the third indicator feature.

[0201] The time series indicator information can satisfy:

[0202] ;

[0203] in, Indicates time series indicator information, Indicates the first indicator characteristic, This indicates the characteristics of the second indicator. This indicates the characteristics of the third indicator. This represents the fourth weighting coefficient. This represents the fifth weighting coefficient. This represents the sixth weighting coefficient. , , satisfy: .

[0204] In one possible implementation, the weighting coefficients (fourth weighting coefficient, fifth weighting coefficient, and sixth weighting coefficient) involved in the weighted fusion processing in this step can be adaptively adjusted according to the on-site environment: fourth weighting coefficient In scenarios with strong light reflection (e.g., where the highlight area accounts for more than or equal to 20% of the image), the value range is [0.4, 0.5] to reduce the impact of morphological feature fluctuations caused by reflective interference; the fifth weighting coefficient In scenarios with significant jitter (e.g., the inter-frame displacement variance of an image sequence is greater than or equal to 10 pixels), the value range is [0.3, 0.4] to compensate for the positional shift caused by jitter; the sixth weighting coefficient The value range is [0.3, 0.4] in complex texture scenarios (e.g., image texture entropy greater than or equal to 5) to make full use of differences in appearance details for defect discrimination and improve the recognition accuracy of defects under complex texture conditions.

[0205] It is understandable that by weighted fusion processing of the first, second, and third indicator features, the temporal indicator information of the cross-frame trajectory is obtained. This enables a comprehensive quantification of the temporal stability of the cross-frame trajectory from three dimensions: continuity, positional stability, and morphological consistency. This can fully characterize the essential differences between real defects and noise in temporal characteristics, thereby providing a reliable quantitative basis for accurately distinguishing between real defects and noise in the future.

[0206] The weld defect detection method provided in this application can improve the accuracy and adaptability of determining the temporal index information of cross-frame trajectory by means of multi-dimensional temporal feature extraction and weighted fusion, thereby significantly improving the stability and accuracy of weld defect detection under complex working conditions.

[0207] Figure 4 Flowchart of the weld defect detection method provided in this application Figure 4 ,like Figure 4 As shown, this embodiment, based on the aforementioned embodiments, provides a detailed explanation of the weld defect detection method when the weld detection result is a real defect. The method includes:

[0208] S401. Determine the attribute information of the real defect and display the attribute information; wherein, the attribute information includes the defect location, defect type, time sequence index information and key frame image, the defect location represents the spatial coordinates of the real defect in the weld area, the defect type represents the category attribute of the real defect, and the key frame image represents the video frame with the best clarity in the cross-frame trajectory.

[0209] In one possible implementation, determining the defect location can be done as follows:

[0210] The average value of the center coordinates of the candidate regions in each frame of the cross-frame trajectory is taken to obtain the final spatial coordinates of the real defect (relative to the video frame coordinate system), i.e., the defect location.

[0211] For example, the defect location is (412, 185), in pixels.

[0212] Determining the defect type can be done as follows:

[0213] By combining the output of the single-frame classification model with time-weighted voting, the category attribute of the true defect, i.e. the defect type, is obtained.

[0214] Specifically, the single-frame classification model takes the candidate regions of each frame as input and outputs the defect type corresponding to each candidate region. Then, it fuses the probability distributions of defect categories in each frame across the cross-frame trajectory and takes the defect category with the highest probability as the true defect type. For example, defect types may include, but are not limited to, cracks, porosity, inclusions, and lack of fusion.

[0215] Determining keyframe images can be done as follows:

[0216] From the images of candidate regions in each frame of the cross-frame trajectory, select the frame with the best image clarity as the keyframe image.

[0217] Specifically, sharpness can be quantified by metrics such as Laplacian operator variance or image entropy, and the frame with the highest sharpness value is selected as the keyframe image.

[0218] For example, the keyframe image is the 37th frame of the video frame sequence, and the storage address of this keyframe image is " / results / defect_frame_0037.jpg".

[0219] In one possible implementation, displaying attribute information may include: highlighting the defect location with a highlighted box on the display interface, overlaying the defect type and time-series index information, and providing thumbnail or full-image viewing functionality for keyframe images. Optionally, the attribute information may also be output in a structured data format (such as JSON or XML) to improve the traceability of seam defect detection results.

[0220] The weld defect detection method provided in this application improves the practicality and traceability of weld defect detection results by determining and displaying multi-dimensional attribute information.

[0221] Figure 5 Flowchart of the weld defect detection method provided in this application Figure 5 ,like Figure 5As shown, this embodiment, based on the aforementioned embodiments, provides another description of the weld defect detection method, which includes:

[0222] S501, Video Acquisition.

[0223] In actual engineering deployments, the relevant parameters for constructing cross-frame trajectories involved in this application (including but not limited to the weighting coefficients for fusion processing, the preset maximum displacement, the preset rate of change threshold, the matching threshold, etc.) can be flexibly configured according to the parameters of different acquisition devices, and no specific limitations are imposed on these relevant parameters here. For example, in a scenario with 1080p resolution, a scanning speed of approximately 5 mm per second, and a frame rate of 60 frames per second, a smaller preset maximum displacement and a more balanced weighting coefficient can be set; in a scenario with 720p resolution, a scanning speed of approximately 2 mm per second, and a frame rate of 30 frames per second, the preset maximum displacement can be appropriately increased and the second weighting coefficient (the weighting coefficient related to the texture direction) can be adjusted.

[0224] For example, during actual post-weld inspection, a handheld terminal with 1080p video capture capability was moved at a constant speed along the weld direction to obtain a video frame sequence of approximately 4 seconds, with a frame rate of approximately 60 frames per second. Slight hand tremors and angular shifts occurred during the scanning process, resulting in slight variations in the weld position in each frame. This video frame sequence was acquired for subsequent analysis and processing.

[0225] S502, Pretreatment and ROI Extraction.

[0226] For example, the following operation is performed on each frame of the image:

[0227] Slight denoising was performed using a 5×5 Gaussian filter; adaptive histogram equalization was used to compensate for the brightness distribution; the weld edges were extracted using the Sobel operator; the weld region was located by principal direction detection (e.g., based on gradient histogram statistics or structural tensor eigenvalue decomposition), and a slightly wider ROI region was generated based on this.

[0228] In one possible implementation, the width and position of the weld ROI can be adaptively adjusted according to changes in weld width, shooting distance, and viewing angle. Alternatively, multi-scale ROI overlay can be used to improve the coverage of defects of different sizes.

[0229] In this video frame sequence, the ROI covers an area approximately 50 to 70 pixels wide around the weld centerline.

[0230] S503, Single-frame candidate region generation.

[0231] For example, a small pre-trained convolutional network is input to each ROI in each frame to detect brightness abrupt changes and texture anomalies, resulting in 2 to 5 candidate points (also known as candidate regions) in each frame.

[0232] For example, the following candidate regions were detected in frame 32: a long strip of low brightness; two small bright spots that may be caused by reflections; and a local break in the texture.

[0233] It should be understood that candidate points are only used for subsequent time-series correlation analysis and are not directly used as the basis for judging weld defects.

[0234] S504. Cross-frame associated candidate regions to construct cross-frame trajectories.

[0235] For example, candidate points are correlated across frames between frames 32 and 40.

[0236] First, the three types of similarity between candidate points are calculated, as shown in the following example:

[0237] ① If the spatial position (i.e., center position) of a candidate point differs by approximately 4 pixels between frame 32 and frame 33, then the first score is... For example, it can be 0.8 or higher;

[0238] ②If the difference between texture directions does not exceed 6 degrees, then the second score is... The value ranges from 0.75 to 0.9.

[0239] ③ The cosine similarity between appearance features is about 0.7, which is the third score, and can be characterized as appearance stability.

[0240] The weighted combination yields a matching score of approximately 0.75, which is higher than the matching threshold (e.g., 0.6), thus forming a continuous trajectory.

[0241] Meanwhile, other candidate regions with low cross-frame similarity, such as those with positional shifts exceeding 20 pixels or completely different appearance features, and matching scores below 0.3, are eliminated, and the construction of the cross-frame trajectory is stopped.

[0242] For example, two valid cross-frame tracks are eventually formed in the video frame sequence, where cross-frame track A spans 9 frames and cross-frame track B spans only 2 frames.

[0243] S505, Temporal Feature Quantization.

[0244] Three types of temporal features (first index feature, second index feature and third index feature) are calculated for the cross-frame trajectory A.

[0245] For example, regarding the first indicator feature, the cross-frame trajectory A appears continuously for 9 frames from frame 32 to 40. This indicates that the cross-frame trajectory has obvious cross-frame persistence.

[0246] For the second metric feature, the spatial position jumps of the candidate regions between frames of the cross-frame trajectory A are all between 3 and 6 pixels, and a preset maximum displacement is defined. 15 pixels, second indicator feature This indicates that the cross-frame trajectory exhibits relatively stable positional migration.

[0247] For the third indicator feature, area information, brightness information, and texture information all show relatively small changes across frames, with normalized change values... Within the range of 0.2 to 0.3, for example, the third indicator characteristic This indicates that the morphological changes of the cross-frame trajectory are stable.

[0248] S506, Calculation of time series confidence and defect determination.

[0249] Define the fourth weighting coefficient Fifth weighting coefficient Sixth weighting coefficient Calculate the temporal confidence (i.e., temporal index information) of the cross-frame trajectory A:

[0250] ;

[0251] When the preset confidence threshold is set to 0.6, the cross-frame trajectory A is identified as a real defect.

[0252] Correspondingly, if the consecutive occurrence of cross-frame trajectory B is insufficient (2 frames), the position jump is large, and the temporal confidence is about 0.3, then cross-frame trajectory B is determined to be noise.

[0253] S507, Output Results.

[0254] The output is the attribute information of the actual defect, including defect location, defect type, timing index information, and keyframe images.

[0255] In addition, the output results may also include defect bounding box information, which may refer to the rectangular or polygonal coordinate range used to highlight the actual defect area in the image. For example, the average value is calculated based on the bounding boxes of each frame of the cross-frame trajectory A (determined by the spatial position of the candidate area in step S503), and a safety boundary of 3 to 5 pixels is added to obtain the final selected area, which is the defect bounding box information.

[0256] For example, the defect bounding box information is (380,160,455,208), and its format is (top left x, top left y, bottom right x, bottom right y), with the unit being pixels.

[0257] For example, the output is displayed in JSON format:

[0258] {

[0259] "defect_type": "crack",

[0260] "center_point": {"x": 412, "y": 185},

[0261] "bbox": {"x1": 380, "y1": 160, "x2": 455, "y2": 208},

[0262] "sequence_confidence": 0.835,

[0263] "key_frame_index": 37,

[0264] "key_frame_image": " / results / defect_frame_0037.jpg"

[0265] }

[0266] Among them, "defect_type" represents the defect type, "center_point" represents the defect location, "bbox" represents the defect bounding box information, "sequence_confidence" represents the timing index information, "key_frame_index" represents the frame number of the keyframe image in the video frame sequence, and "key_frame_image" represents the keyframe image.

[0267] It should be noted that the relevant values, weighting coefficients, thresholds, etc., in this embodiment are all exemplary parameters used to illustrate the feasibility of the method of the present invention and do not constitute a limitation of the present invention. The present invention can be flexibly adjusted according to different camera imaging parameters, scanning speeds, and weld morphologies.

[0268] In addition, the following are examples of interruption and recovery of cross-frame trajectories:

[0269] In another weld seam scanning video, due to strong reflections in local areas, no corresponding candidate regions were detected in frames 45 and 46, resulting in a brief interruption across frame A.

[0270] The preset matching rule of the present invention compensates and associates the candidate region of the 47th frame according to the direction change rate and the spatial position change trend of the candidate region in the previous and next frames, so that the cross-frame trajectory A can be restored to continuity.

[0271] The result of the temporal feature calculation is: the total number of frames spanned by the cross-frame trajectory. The actual number of frames in which candidate regions exist in the cross-frame trajectory Then the first indicator feature This indicates that even with a two-frame interruption, the cross-frame trajectory still exhibits the typical stability of a real defect.

[0272] Final time series indicator information The value is higher than the preset confidence threshold of 0.6, so the cross-frame trajectory A is still identified as a real defect.

[0273] In one possible implementation, when candidate regions are lost in some frames due to occlusion, strong reflection, or temporary defocus, strategies such as trajectory interpolation between consecutive frames, bidirectional correlation, or multi-hypothesis tracking can be used to maintain trajectory continuity within a certain frame interval, thereby reducing misjudgments caused by short-term information loss.

[0274] This embodiment demonstrates that the present invention can maintain trajectory continuity in the event of short-term information loss, thereby enhancing stability and reliability in real-world scenarios.

[0275] The following is an example of noise filtering:

[0276] In the video of the weld seam with strong flashes, several momentary bright noise points appear in each frame.

[0277] Multiple candidate regions lasting only 1 to 2 frames were detected, and attempts were made to establish cross-frame trajectories. Analysis revealed that the spatial location jumps of these candidate regions exceeded 20 pixels, the rate of directional change was greater than 25 degrees, and the cosine similarity of their appearance features was less than 0.3. These conditions resulted in matching scores not exceeding the preset matching threshold, thus failing to constitute valid cross-frame trajectories.

[0278] Even if a shorter cross-frame trajectory of length 2 is formed, such as its first index feature... Second indicator characteristics Third indicator characteristics The obtained time series index information If the match value is below a preset threshold, the cross-frame trajectory can be filtered. This negative example demonstrates the effective rejection capability of the present invention in noisy environments.

[0279] It should be understood that the core of this invention lies in constructing cross-frame trajectories of candidate regions for weld defects based on video frame sequences, and using the temporal stability of the cross-frame trajectories to distinguish between real defects and instantaneous noise.

[0280] The weld defect detection method provided in this application improves the stability and accuracy of weld defect detection. Furthermore, this method achieves highly robust weld defect detection relying solely on a single video stream, eliminating the need for multimodal sensors such as infrared, point cloud, and electrical parameter sensors, as well as fixed supports or sliding rail systems. It can be directly deployed on handheld terminals or ordinary industrial cameras, offering advantages in terms of low cost and high flexibility.

[0281] Figure 6 This is a schematic diagram of the weld defect detection device provided in this application, as shown below. Figure 6 As shown, the weld defect detection device 60 provided in this embodiment includes: an acquisition module 601, an extraction module 602, a construction module 603, and a determination module 604.

[0282] The acquisition module 601 is used to acquire the video frame sequence of the weld area;

[0283] The extraction module 602 is used to determine the candidate regions of each frame in the video frame sequence and extract the feature information of the candidate regions; wherein, the candidate region represents the potential area in the image where there is a weld defect, and the feature information includes at least the spatial location, texture direction and appearance features of the candidate region.

[0284] The construction module 603 is used to construct cross-frame trajectories based on various feature information and preset matching rules, and to determine the temporal index information of the cross-frame trajectories; wherein, the cross-frame trajectories are constructed from candidate regions belonging to the same physical defect, and the temporal index information characterizes the stability of the cross-frame trajectories in the time dimension.

[0285] The determination module 604 is used to determine the weld inspection result based on the timing index information; wherein the weld inspection result represents whether the cross-frame trajectory is a real defect or noise.

[0286] In an optional example, the extraction module 602 is further configured to input the image into a preset visual detection model to obtain the output candidate region; wherein the preset visual detection model is at least one of a convolutional neural network, an object detection network, and a classifier.

[0287] In an optional example, the construction module 603 is further configured to, for each pair of adjacent frames, determine a candidate region of the preceding image in the pair of adjacent frames as a first candidate region, and determine a candidate region of the following image in the pair of adjacent frames as a second candidate region.

[0288] Based on the feature information of the first candidate region and the feature information of the second candidate region, a matching score is determined; wherein, the matching score represents the degree of similarity between the first candidate region and the second candidate region belonging to the same physical defect;

[0289] If the matching score is greater than the preset matching threshold, then the first candidate region and the second candidate region are determined as the target candidate regions;

[0290] Construct cross-frame trajectories based on each target candidate region.

[0291] In an optional example, the construction module 603 is further configured to determine a first score based on a preset first coefficient, the spatial position of the first candidate region, and the spatial position of the second candidate region; wherein the preset first coefficient is used to adjust the degree of influence of inter-frame spatial displacement on the matching score, and the first score characterizes the spatial proximity between the first candidate region and the second candidate region.

[0292] A second score is determined based on a preset second coefficient, the texture direction of the first candidate region, and the texture direction of the second candidate region. The preset second coefficient is used to adjust the degree of influence of the difference in texture direction between frames on the matching score, and the second score represents the degree of directional consistency between the first candidate region and the second candidate region.

[0293] A third score is determined based on the appearance features of the first candidate region and the second candidate region; the third score represents the degree of appearance similarity between the first candidate region and the second candidate region.

[0294] The first, second, and third scores are weighted and merged to obtain the matching score.

[0295] In one optional example, at least one of the weighting coefficients, preset first coefficients, preset second coefficients, and preset matching thresholds involved in the weighted fusion process is dynamically adjusted by the acquisition parameters;

[0296] The acquisition parameters include at least one or more of the following: the resolution of the image sensor, the scanning speed of the acquisition device, and the frame rate of the video frame sequence. The image sensor is used to acquire the video frame sequence, the acquisition device is used to move the image sensor relative to the weld, and the scanning speed characterizes the movement speed of the acquisition device relative to the weld.

[0297] In an optional example, module 603 is also used to associate each target candidate region sequentially in time order to construct a cross-frame trajectory;

[0298] If the target candidate region is not determined within a preset number of consecutive frames, or if the inter-frame displacement of the target candidate region between two adjacent frames is greater than the preset maximum displacement, then the construction of the cross-frame trajectory will be terminated.

[0299] In an optional example, if it is determined that the inter-frame displacement of the target candidate region between two adjacent frames is greater than the preset maximum displacement, the construction module 603 is further used to determine the direction change rate of the texture direction of the target candidate region between two adjacent frames. If the direction change rate is greater than the preset change rate threshold, the preset maximum displacement is increased to obtain the target maximum displacement.

[0300] If the inter-frame displacement is greater than the target maximum displacement, the construction of the cross-frame trajectory will be terminated.

[0301] In an optional example, the construction module 603 is further configured to determine the actual number of frames in which the candidate region exists in the cross-frame trajectory and the total number of frames spanned by the cross-frame trajectory, and to determine a first index feature based on the actual number of frames and the total number of frames; wherein the first index feature characterizes the ability of the candidate region to appear stably in the cross-frame trajectory;

[0302] The inter-frame displacement of the candidate region in the cross-frame trajectory is determined between each adjacent frame, and a second index feature is determined based on the inter-frame displacement and the preset maximum displacement; wherein, the second index feature characterizes the stability of the position migration of the candidate region between consecutive frames.

[0303] The morphological information of candidate regions in the cross-frame trajectory is determined, and a third index feature is determined based on the morphological information. The morphological information includes area information, brightness information and texture information, and the third index feature characterizes the change range of the appearance of the candidate region between different frames.

[0304] The first, second, and third indicator features are weighted and fused to obtain the time-series indicator information of the cross-frame trajectory.

[0305] In an optional example, module 604 is further configured to determine that the weld inspection result is a real defect if the timing index information is greater than a preset confidence threshold.

[0306] If the time-series index information is less than or equal to the preset confidence threshold, the weld inspection result is determined to be noise.

[0307] In an optional example, if the weld inspection result is a real defect, the determination module 604 is also used to determine the attribute information of the real defect and display the attribute information; wherein, the attribute information includes the defect location, defect type, timing index information and key frame image, the defect location represents the spatial coordinates of the real defect in the weld area, the defect type represents the category attribute of the real defect, and the key frame image represents the video frame with the best clarity in the cross-frame trajectory.

[0308] The weld defect detection device provided in this embodiment can perform the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0309] Figure 7 A schematic diagram of the structure of the electronic device provided in this application. Figure 7 As shown, the electronic device 70 provided in this embodiment includes at least one processor 701 and a memory 702. Optionally, the electronic device 70 further includes a communication component 703. The processor 701, memory 702, and communication component 703 are connected via a bus 704.

[0310] In a specific implementation, at least one processor 701 executes computer execution instructions stored in memory 702, causing at least one processor 701 to perform the above-described method.

[0311] The specific implementation process of processor 701 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0312] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0313] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0314] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0315] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0316] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0317] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0318] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0319] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0320] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method of detecting a weld defect, characterized by, include: Obtain the video frame sequence of the weld area; For each frame in the video frame sequence, candidate regions of the image are determined and feature information of the candidate regions is extracted; wherein, the candidate region represents a potential area in the image where weld defects exist, and the feature information includes at least the spatial location, texture direction and appearance features of the candidate region; Based on the aforementioned feature information and preset matching rules, a cross-frame trajectory is constructed, and the temporal index information of the cross-frame trajectory is determined; wherein, the cross-frame trajectory is constructed from candidate regions belonging to the same physical defect, and the temporal index information characterizes the stability of the cross-frame trajectory in the time dimension. Based on the time-series index information, the weld inspection result is determined; wherein, the weld inspection result indicates whether the cross-frame trajectory is a real defect or noise.

2. The method of claim 1, wherein, Determining the candidate regions of the image includes: The image is input into a preset visual detection model to obtain the output candidate region; wherein the preset visual detection model is at least one of a convolutional neural network, an object detection network, and a classifier.

3. The method according to claim 1, characterized in that, The step of constructing a cross-frame trajectory based on the aforementioned feature information and preset matching rules includes: For each pair of adjacent frames, a candidate region of the preceding image in the pair of adjacent frames is determined as a first candidate region, and a candidate region of the following image in the pair of adjacent frames is determined as a second candidate region. A matching score is determined based on the feature information of the first candidate region and the feature information of the second candidate region; wherein the matching score represents the degree of similarity between the first candidate region and the second candidate region as belonging to the same physical defect; If the matching score is greater than the preset matching threshold, then the first candidate region and the second candidate region are determined as target candidate regions; The cross-frame trajectory is constructed based on each of the target candidate regions.

4. The method according to claim 3, characterized in that, The step of determining the matching score based on the feature information of the first candidate region and the feature information of the second candidate region includes: A first score is determined based on a preset first coefficient, the spatial position of the first candidate region, and the spatial position of the second candidate region; wherein, the preset first coefficient is used to adjust the degree of influence of inter-frame spatial displacement on the matching score, and the first score characterizes the spatial proximity between the first candidate region and the second candidate region; A second score is determined based on a preset second coefficient, the texture direction of the first candidate region, and the texture direction of the second candidate region; wherein, the preset second coefficient is used to adjust the degree of influence of inter-frame texture direction differences on the matching score, and the second score represents the degree of directional consistency between the first candidate region and the second candidate region; A third score is determined based on the appearance features of the first candidate region and the second candidate region; wherein the third score represents the degree of appearance similarity between the first candidate region and the second candidate region. The first score, the second score, and the third score are weighted and fused to obtain the matching score.

5. The method according to claim 4, characterized in that, The weighting coefficient, the preset first coefficient, the preset second coefficient, and the preset matching threshold involved in the weighted fusion process are at least one of the following dynamically adjusted by the acquisition parameters; The acquisition parameters include at least one or more of the following: the resolution of the image sensor, the scanning speed of the acquisition device, and the frame rate of the video frame sequence. The image sensor is used to acquire the video frame sequence, and the acquisition device is used to move the image sensor relative to the weld. The scanning speed represents the moving speed of the acquisition device relative to the weld.

6. The method according to claim 3, characterized in that, The step of constructing the cross-frame trajectory based on each of the target candidate regions includes: The target candidate regions are sequentially associated in chronological order to construct the cross-frame trajectory; If the target candidate region is not determined within a preset number of consecutive frames, or if the inter-frame displacement of the target candidate region between two adjacent frames is determined to be greater than the preset maximum displacement, then the construction of the cross-frame trajectory is terminated.

7. The method according to claim 6, characterized in that, If it is determined that the inter-frame displacement of the target candidate region between two adjacent frames is greater than a preset maximum displacement, the method further includes: Determine the rate of change of the texture direction of the target candidate region between two adjacent frames. If the rate of change is greater than the preset rate of change threshold, increase the preset maximum displacement to obtain the target maximum displacement. If the inter-frame displacement is greater than the target maximum displacement, then the construction of the cross-frame trajectory is terminated.

8. The method according to claim 1, characterized in that, The temporal index information for determining the cross-frame trajectory includes: The actual number of frames in which the candidate region exists in the cross-frame trajectory and the total number of frames spanned by the cross-frame trajectory are determined, and a first index feature is determined based on the actual number of frames and the total number of frames; wherein, the first index feature characterizes the ability of the candidate region to appear stably in the cross-frame trajectory; The inter-frame displacement of the candidate region in the cross-frame trajectory is determined between each adjacent frame, and a second index feature is determined based on each inter-frame displacement and a preset maximum displacement; wherein, the second index feature characterizes the stability of the position migration of the candidate region between consecutive frames. The morphological information of candidate regions in the cross-frame trajectory is determined, and a third indicator feature is determined based on each of the morphological information; wherein, the morphological information includes area information, brightness information and texture information, and the third indicator feature characterizes the change range of the appearance morphology of the candidate region between different frames; The first indicator feature, the second indicator feature, and the third indicator feature are weighted and fused to obtain the temporal indicator information of the cross-frame trajectory.

9. The method according to claim 1, characterized in that, The step of determining the weld inspection result based on the time-series index information includes: If the time-series index information is greater than the preset confidence threshold, then the weld detection result is determined to be the real defect; If the time-series index information is less than or equal to the preset confidence threshold, then the weld detection result is determined to be noise.

10. The method according to any one of claims 1-9, characterized in that, If the weld inspection result is the actual defect, the method further includes: The attribute information of the real defect is determined and displayed; wherein the attribute information includes defect location, defect type, the time sequence index information and key frame image, the defect location represents the spatial coordinates of the real defect in the weld area, the defect type represents the category attribute of the real defect, and the key frame image represents the video frame with the best clarity in the cross-frame trajectory.

Images