Welding defect identification method and device based on artificial intelligence

Abstract

The invention discloses a welding defect identification method and device based on artificial intelligence, and relates to the technical field of welding defect identification. Comprising the following steps that S1, continuous frame images are collected to obtain welding appearance images, and accurate time is marked on each frame image; s2, carrying out HDR fusion reconstruction; s3, Fourier transform is carried out on the arc light periodic noise, and an improved Retinex algorithm is combined to eliminate non-uniform illumination influence; s4, deep learning denoising based on physical simulation; s5, multi-modal data fusion and welding defect decision making are carried out; s6, performing real-time processing and incremental learning; according to the invention, application of artificial intelligence in welding defect identification is realized, the detection accuracy and efficiency are improved, the welding defect cost is reduced, and the intelligent development of welding quality detection is promoted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of welding defect recognition, in particular to a welding defect recognition method and device based on artificial intelligence. BACKGROUND

[0002] Welding defects (such as cracks, pores, incomplete penetration, etc.) can significantly reduce the mechanical properties of materials, especially in load-bearing structures (bridges, buildings) or high-pressure containers (pipelines, storage tanks), where small defects can cause catastrophic accidents (such as leaks, fractures); for example, welding defects in aerospace engine blades can cause in-flight failures, directly threatening life safety; welding cracks in oil pipelines can cause leaks or even explosions, threatening the environment and life safety; therefore, welding defect recognition is particularly important; Welding defect recognition is a core step in intelligent manufacturing. During the welding process, there are strong arc light, high temperature radiation and electromagnetic interference, which can easily cause noise and interference in the collected images. How to stably and clearly collect images and effectively preprocess them in a complex environment is still a difficult problem to be solved. SUMMARY

[0003] The main purpose of the present application is to provide a welding defect recognition method and device based on artificial intelligence to overcome the problems mentioned in the background art.

[0004] To achieve the above-mentioned purpose, according to one aspect of the present application, a welding defect recognition method based on artificial intelligence is provided, comprising the following steps: S1: using a high dynamic range camera to collect images of the surface of a welding piece that needs to be recognized for welding defects to obtain continuous frame welding appearance images; at the same time, deploying an infrared thermal imager to monitor the temperature field distribution; S2: performing dynamic range fusion processing on multiple images to compensate for the loss of details in high light areas of a single image; hardware trigger synchronous acquisition is performed, and within a single welding cycle, three groups of different exposure images are obtained by switching the light filter wheel: short exposure to capture the arc bright area, medium exposure to obtain the molten pool details, and long exposure to extract dark defects; an adaptive HDR fusion algorithm is used for fusion reconstruction; S3: Fourier transform is performed on the periodic noise of the arc light to design a notch filter, and the specific algorithm is as follows: to retain the medium and high frequency components corresponding to the defect characteristics, where u0 and v0 represent the center position of the noise component in the frequency domain, i.e. the position of the main frequency of the periodic noise, and the filter forms a notch near (u0, v0) to specifically suppress the noise in this frequency area, thereby retaining other medium and high frequency parts; u and v represent the frequency domain coordinates of the image after Fourier transform, where u is the horizontal frequency variable and v is the vertical frequency variable; 0） ​The improved Retinex algorithm is used to eliminate the influence of non-uniform illumination, and the algorithm formula is: wherein G is a Gaussian kernel, and sigma is a standard deviation of the Gaussian kernel, and controls the blurring degree; in this welding scene, those skilled in the art will set its value to 50.

[0005] S4: deep learning denoising based on physical simulation; S5: multi-modal data fusion and welding defect decision; S6: real-time processing and incremental learning.

[0006] Further, a three-waveband filter wheel is customized: waveband 1: 850nm narrowband, waveband 2: 1050nm, waveband 3: visible light + polarizer, and the filter switching time is <10ms, which is synchronized with the welding current pulse; the camera shell adopts a permalloy double-layer shield, the grounding impedance is <0.1Ω, the transmission line uses a DC-DC isolation power module to improve the common-mode rejection ratio, and an optical fiber + photoelectric conversion module is used instead of a copper cable; a semiconductor refrigeration piece is installed on the camera, and the PID temperature control is used to keep the chip temperature <45℃, and a gas curtain is installed at the front end of the lens to prevent splashes from adhering.

[0007] Further, a dynamic range fusion processing is performed, and the specific mode is as follows: S2-1: a Farneback dense optical flow algorithm is used to solve the displacement field between adjacent frames : , the displacement field is approximated by polynomial expansion and the displacement field between adjacent frames is solved by least squares method ; wherein t is time, and (x, y) is the pixel coordinate of the image; S2-2: adaptive weight mapping: a noise mask is calculated by calculating the difference between the BM3D denoising result and the original image, and the noise mask is mapped to the adaptive weight, and the mapping formula is: , k represents the serial number of the image, is the control exposure preference, and the noise mask is calculated by calculating the difference between the BM3D denoising result and the original image, and the calculation formula is: ; S2-3: Laplacian pyramid fusion, and the specific fusion process and formula are as follows: S2-3-1: a Gaussian pyramid is constructed for each image : ; wherein is the index of the pyramid layer, represents the pixel value of the Gaussian pyramid at the coordinate (x, y) layer. represents a 5×5 Gaussian kernel; ; N is an integer, indicating the number of pyramid layers; S2-3-2: Constructing a Laplace Pyramid : ; S2-3-3: Weighted fusion of pyramid layers: ;in Represents the fused Laplacian pyramid layer, Indicates that the kth image is in the Normalized weights of pyramid layers; S2-3-4: Rebuilding from the top: ; S2-4: Use CUDA to implement optical flow calculation and combine it with the selection of pyramid levels. The principle of selecting the number of pyramid levels is as follows: , where W is the image width and H is the image height.

[0008] Furthermore, deep learning denoising based on physical simulation is carried out as follows: COMSOL Multiphysics is used to establish a welding heat-light-electromagnetic multi-physics field model, generate annotated simulation images, and inject random noise into the collected on-site images to enhance the real data; then it is trained using lightweight networks and knowledge distillation methods.

[0009] Furthermore, multimodal data fusion and welding defect decision making are carried out in the following ways: Establish the mapping relationship between visible light image and infrared thermal image. The specific relationship algorithm is as follows: , α, β, and γ are obtained by least square fitting; Deep learning is used to perform visible light and infrared defect detection on the processed images to obtain the visible light defect probability and infrared defect probability. The ultrasonic data is subjected to signal analysis to obtain the ultrasonic defect probability. The visible light defect probability, infrared defect probability, and ultrasonic defect probability are then weighted and calculated to obtain the decision value Score. When Score>0.9, the system will be shut down immediately and marked as a serious defect; When 0.7<Score≤0.9, the production line speed is reduced and manual re-inspection is triggered; When Score≤0.7, data is recorded without interrupting production.

[0010] Further, real-time processing is carried out in the following ways: The acquisition node calls the Aravis driver to acquire continuous frame images to obtain welding appearance images, marks each frame with the precise time, and transmits it to the next topic; The preprocessing node subscribes to the Topic published by the acquisition node, receives the original image, transmits the image data to the CUDA memory quickly, executes the GPU-accelerated preprocessing algorithm, and transmits to the next Topic; the original image data of any one welding appearance image is packaged into a preprocessing task object, and the preprocessing node performs real-time processing on each preprocessing task; The detection node subscribes to the Topic published by the processing node, receives the image data, and performs welding defect detection according to the order of the acquisition time corresponding to the image data, and outputs the welding detection result.

[0011] Further, each preprocessing task is processed in real time, and the specific steps are as follows: Step one: extract the preprocessing task, and parse the acquisition time, resolution, RIO proportion of the preprocessing task from it; use the BRISQUE algorithm to measure the distortion degree by extracting the local statistical features of the image, and output the image quality score; calculate the acquisition time difference value between the acquisition time and the current time to obtain the acquisition time length, and normalize the acquisition time length (T), resolution (R), RIO proportion and quality score (Q), then calculate the time length correlation value, resolution correlation value, RIO correlation value and quality score correlation value respectively, and then perform weighted calculation to obtain the processing value of the preprocessing task; the larger the processing value, the higher the priority of the preprocessing task; the specific calculation formula is: , wherein T is the acquisition time length, Tmax is the maximum tolerance delay of the scene, is the reference baseline resolution, R is the resolution (total pixels of the current image); RIO proportion is the proportion of the region of interest, Q is the quality score, Qmax is the highest distortion score expected in the scene (i.e. the quality score in the worst case of image quality); arrange the processing values of each preprocessing task in descending order, and select the preprocessing task with the largest processing value as the pending task; Step two: the preprocessing node has several parallel threads, each thread can independently preprocess the welding appearance image and output the processed image data; obtain the historical processing image record of the thread, and extract the time length spent for processing each frame of image from it, then calculate the average delay by averaging the time length spent each time in the historical processing image record; Real-time acquisition of the number of pending queues and load (current thread resource occupation rate) of the thread, and normalization processing of the load, the number of pending queues, the average delay, and the throughput, and they are respectively recorded as F, N, C, Y; the load, the number of pending queues, the average delay, and the throughput are calculated to obtain a load-related value, a queue-related value, a delay-related value, and a throughput-related value, and then the load-related value, the queue-related value, the delay-related value, and the throughput-related value are weighted to obtain the performance index of the thread; the specific calculation formula is Wherein Fmax is the maximum load allowed for the thread in the application scenario, Namx is the maximum queue number allowed for the thread, Cmax is the maximum delay allowed for the thread; Ymin is the minimum throughput of the thread; the threads are sorted in descending order according to their corresponding performance indexes, the thread with the largest performance index is selected as the calibration thread, and the pending task is allocated to the calibration thread, and the number of pending queues of the calibration thread is updated; Step three: repeat the above step one to step two until all the pre-processing tasks are allocated.

[0012] Further, incremental learning, the specific mode is: When a new process is detected: extract the shallow features of the current batch of images; calculate the cosine similarity with the historical feature library, if the difference is greater than the difference threshold, then trigger model update; use EWC algorithm for incremental training to retain the original knowledge.

[0013] In order to achieve the above purpose, according to another aspect of the present application, a welding defect identification device based on artificial intelligence is provided, which comprises: an image pre-storage library, an adaptive fusion module, a defect detection module and a real-time processing module. The image pre-storage library receives the continuous frame welding appearance images collected by the high dynamic range camera and the temperature field distribution data monitored by the infrared thermal imager in real time. The adaptive fusion module quickly transmits the image data to the CUDA memory, executes the GPU accelerated preprocessing algorithm, and transmits it to the defect detection module. The defect detection module identifies defects according to the preprocessed welding appearance images to output welding detection results and make corresponding decisions. The real-time processing module encapsulates the original image data of any one welding appearance image into a preprocessing task object, and performs real-time processing on each preprocessing task to control the preprocessing process of the adaptive fusion module, and controls the defect detection sequence of the defect detection module.

[0014] The beneficial effects of the present application are: The present application can take into account the bright area and dark details of the arc, and effectively expand the dynamic range, by collecting multiple frames of images with different exposure times, and using motion compensation, weight mapping and Laplacian pyramid fusion to realize HDR imaging. The present application can suppress noise from both frequency and spatial domains by designing a notch filter using Fourier transform for periodic arc noise, and eliminating the effect of non-uniform illumination by using an improved Retinex algorithm. The present application can improve the accuracy of defect identification by establishing the mapping relationship between visible light and infrared thermal images, and using a multi-modal fusion strategy based on Transformer to fully utilize different modal information; according to the defect confidence score calculated from each modal data, a hierarchical alarm mechanism is set to realize immediate response to serious defects and effective monitoring of slight defects. The present application can significantly improve the efficiency and accuracy of welding defect detection by integrating real-time processing and adaptive learning. In terms of real-time processing, a multi-thread parallel processing architecture is adopted, and GPU is used to accelerate the preprocessing algorithm to ensure that the preprocessing delay of each frame of image is controlled within a reasonable range. This efficient processing capability meets the strict real-time requirement of welding defect detection. At the same time, according to the parameters such as the acquisition time length, resolution, ROI proportion and quality score of the task, the priority of the task is dynamically adjusted and assigned to the most suitable thread for processing. This adaptive task scheduling mechanism ensures the optimal utilization of system resources and improves the overall processing efficiency. In terms of adaptive learning, the introduction of new processes can be detected, and the cosine similarity between the shallow features of the current batch of images and the historical feature library is calculated. When the difference exceeds the preset threshold, the system triggers model updating, and uses the Elastic Weight Consolidation (EWC) algorithm for incremental training to retain the original knowledge while quickly adapting to changes in new materials or processes. This adaptive learning mechanism ensures the continuous optimization of the model and its rapid adaptability to new processes. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of the present application illustrated in the drawings are presented by way of example or for purposes of explanation only. In the drawings: Figure 1 is a schematic diagram of the method of the present application; Figure 2 is a node flow chart of the welding defect image recognition of the present application; Figure 3 is a schematic diagram of the welding defect recognition device of the present application. DETAILED DESCRIPTION

[0016] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0017] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first," "second," and the like in the specification and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate for the embodiments of the present invention described herein. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0019] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0020] According to an embodiment of the present invention, Figure 1 As shown, a welding defect recognition method based on artificial intelligence is provided, which includes the following steps: S1: A high dynamic range (HDR) camera is used to capture images of the weld surface where weld defects need to be identified, producing continuous frames of weld appearance images. The HDR camera can capture a wider range of brightness, reducing overexposure in bright areas. A FLIR thermal imager is also deployed to monitor temperature distribution. A custom three-band filter wheel is used: Band 1: 850nm narrowband (to suppress the main arc frequency band), Band 2: 1050nm (to penetrate welding smoke), and Band 3: visible light + polarizer (to capture surface texture). The filter switching time is less than 10ms and is triggered synchronously with the welding current pulse. The camera housing uses a double-layer shield made of Permalloy (μ alloy), with a grounding impedance of less than 0.1Ω, effectively suppressing external electromagnetic interference. The transmission line uses a DC-DC isolated power supply module (such as the TI ISO7840) to improve the common-mode rejection ratio (CMRR>100dB@1MHz). Optical fiber + optoelectronic conversion modules (such as the Hirose HFBR series) replace copper cables to further prevent electromagnetic interference. The camera is equipped with a semiconductor cooling chip (TEC1-12706), which is used in conjunction with PID temperature control to maintain the chip temperature below 45°C. An air curtain (nitrogen purge) is installed in front of the lens to prevent splashing objects from adhering. It should be noted that the settings of the above-mentioned band, grounding point impedance, common mode rejection ratio, etc. are adjusted by those skilled in the art based on the specific application scenarios of this application. In actual applications, these parameters can be further adjusted and optimized according to the detection object, process conditions and environmental interference conditions to achieve the best detection effect; S2: Perform dynamic range fusion processing on multiple frames to compensate for the detail loss in the highlight area of ​​a single frame image; specifically: Hardware-triggered synchronous acquisition: Within a single welding cycle (approximately 50ms), three sets of different exposure images are acquired by switching the filter wheel: short exposure (1ms) to capture arc bright areas, medium exposure (5ms) to obtain weld pool details, and long exposure (20ms) to extract dark defects. Adaptive HDR fusion algorithm is used for fusion reconstruction to ensure that the melt pool area is not overexposed (texture details are retained), dark defects are visible (signal-to-noise ratio>30dB), and motion blur is suppressed (displacement compensation error<0.5 pixel). Specifically: S2-1: Use Farneback dense optical flow algorithm to solve the displacement field between adjacent frames : , The displacement field between adjacent frames is solved by polynomial expansion approximation and least squares method. ; Where t is time, (x, y) is the pixel coordinate of the image; For the region of interest (RIO) constraint in the weld pool area of ​​the welding scene, optical flow is calculated only within 20% of the image area, which can significantly reduce the amount of calculation. At the same time, saturated pixels (value > 250) are assigned zero weight to effectively prevent false matching caused by overexposure. S2-2: Adaptive weight mapping: The noise mask is obtained by calculating the difference between the BM3D denoising result and the original image, and is mapped to the adaptive weight, so that areas with high signal-to-noise ratio receive higher weights. This can effectively improve the visibility of dark details and defects. The mapping formula is: , k represents the sequence number of the image, To control the exposure preference (technical personnel in the field set the exposure preference of welding scene to 60 according to actual needs), the noise mask It is calculated by the difference between the BM3D denoising result and the original image, and its calculation formula is: , The closer the value is to 1, the higher the signal-to-noise ratio in the area; S2-3: Laplacian pyramid fusion: Using Gaussian pyramid decomposition to obtain a low-resolution image, and then using Laplacian pyramid for multi-scale fusion, it is possible to retain detail information and edge features at different scales. This is very effective in avoiding edge artifacts and detail loss during the fusion process. It should be noted that the pyramid layer index starts from 0 (0 is the original resolution layer); the specific fusion process and formula are as follows: S2-3-1: Construct a Gaussian pyramid for each image : ;in is the pyramid level index, Represents the Gaussian pyramid The pixel value of the layer at coordinate (x, y), represents a 5×5 Gaussian kernel; ; N is an integer, indicating the number of pyramid layers; S2-3-2: Constructing a Laplace Pyramid : ; S2-3-3: Weighted fusion of pyramid layers: ;in Represents the fused Laplacian pyramid layer, Indicates that the kth image is in the Normalized weights of pyramid layers; S2-3-4: Rebuilding from the top: ; S2-4: Using CUDA to implement optical flow calculations and combining it with the selection of pyramid levels can effectively improve real-time performance and adapt to the processing requirements of large-resolution images. The principles for selecting the number of pyramid levels are as follows: , where W is the image width and H is the image height; example: for a 2448×2048 image, take 5 layers (top layer size 76×64); By capturing multiple frames of images at different exposure times and utilizing motion compensation, weight mapping, and Laplace pyramid fusion to achieve HDR imaging, the system can take into account both bright and dark details of the arc, effectively expanding the dynamic range.

[0021] S3: Fourier transform the periodic arc light noise, design a notch filter in the frequency domain, the specific algorithm formula is , to retain the corresponding mid-high frequency components of the defect features; where u0, v0 represent the center position of the noise component in the frequency domain, that is, the position of the main frequency of the periodic noise, and the filter forms a notch near (u0, v 0） , which is specially used to suppress the noise in this frequency area, thereby retaining other mid-high frequency parts; u, v represent the frequency domain coordinates of the image after Fourier transform, where u is the frequency variable in the horizontal direction, and v is the frequency variable in the vertical direction; Use the improved Retine algorithm to eliminate non-uniform illumination, and the algorithm formula is: ; where G is the Gaussian kernel, and σ is the standard deviation of the Gaussian kernel, which controls the degree of blurring; in this and the welding scene, those skilled in the art will take its value as 50; By using Fourier transform to design a notch filter for periodic arc noise, and combining the improved Retinex algorithm to eliminate the influence of non-uniform illumination, noise suppression is carried out from both the frequency domain and the spatial domain.

[0022] S4: Deep learning denoising based on physical simulation: Use COMSOL Multiphysics to establish a welding thermal-optical-electromagnetic multi-physical field model to generate labeled simulation images (containing arc light, spatter, porosity, etc. noise), and perform real data enhancement on the collected field images by random noise injection (Gaussian + Poisson noise); then train through the lightweight network and knowledge distillation method.

[0023] S5: Multi-modal data fusion and welding defect decision: Establish the mapping relationship between visible light images and infrared thermal images, and the specific relationship algorithm is: , α, β, γ are obtained by least squares fitting; according to the least squares fitting, α, β, γ are obtained, so that the mapping model can unify the data of the two modalities to the same physical scale, ensuring that the information collected by different collectors can reflect the real working conditions during subsequent feature extraction, and there is no deviation due to the range difference of the original data of each other; adopt a multi-modal fusion strategy based on Transformer to fully utilize the information of different modalities and improve the accuracy of defect recognition; Use deep learning to detect visible light and infrared defects on the processed images to obtain visible light defect probability and infrared defect probability, which are respectively denoted as and ; At the same time, the ultrasonic data is analyzed to obtain the ultrasonic defect probability, denoted as ; The welding defect decision algorithm formula is: , where w1, w2, and w3 correspond to the weights of visible light, infrared, and ultrasound, respectively. Specifically, w1=0.6, w2=0.3, and w3=0.1. Their values ​​are dynamically adjusted according to the modal reliability; When Score>0.9, the system will be shut down immediately and marked as a serious defect; When 0.7<Score≤0.9, the production line speed is reduced and manual re-inspection is triggered; When Score≤0.7, data is recorded without interrupting production; By establishing a mapping relationship between visible light and infrared thermal images and adopting a Transformer-based multimodal fusion strategy, we can fully utilize information from different modalities and improve the accuracy of defect identification. We calculate the defect confidence score based on the data from each modality and set up a graded alarm mechanism to achieve immediate response to serious defects and effective monitoring of minor defects.

[0024] S6: As Figure 2 As shown, real-time processing and incremental learning: The acquisition node calls the Aravis driver to acquire continuous frame images to obtain welding appearance images, marks each frame with the precise time, and transmits it to the next topic; The preprocessing node subscribes to the topic published by the acquisition node, receives the original image, quickly transfers the image data to CUDA memory, executes GPU-accelerated preprocessing algorithms (HDR fusion reconstruction, optical flow calculation, denoising, etc.), and transfers it to the next topic; the raw image data of any welding appearance image is encapsulated into a preprocessing task object, and the preprocessing node performs real-time processing on each preprocessing task. The processing steps are as follows: Step 1: Extract the preprocessing task and analyze it to obtain the acquisition time, resolution, and RIO ratio of the preprocessing task (RIO ratio refers to the percentage of RIO area to image area. In this application scenario, the maximum RIO ratio is 20%, which is obtained by technical personnel in the field of ​​interest in the molten pool area). Use the BRISQUE algorithm (based on the natural scene statistical model) to extract local statistical features of the image to measure the degree of distortion and output the image quality score. The greater the degree of image distortion, the higher the quality score, which means that the image needs to undergo more sophisticated preprocessing to make the image clearer to meet the subsequent welding defect identification and detection. Calculate the time difference between the acquisition time and the current time to obtain the acquisition time, and normalize the acquisition time (T), resolution (R), RIO ratio and quality score (Q). Then calculate the time correlation value, resolution correlation value, RIO correlation value and quality score correlation value respectively, and perform weighted calculation to obtain the processing value of the preprocessing task. The larger the processing value, the higher the priority of the preprocessing task. The specific calculation formula is: wherein T is the collection duration, Tmax is the maximum tolerated delay of the current scene, is the reference resolution, R is the resolution (total pixels of the current image); RIO is the proportion of the region of interest, Q is the quality score, Qmax is the highest expected distortion score under the current scene (i.e. the quality score in the worst case of image quality); the processing values of each pre-processing task are arranged in descending order, and the pre-processing task with the largest processing value is selected as the pending task; Step two: there are several parallel threads in the pre-processing node, each thread can independently pre-process the welding appearance image and output the pre-processed image data; the historical pre-processed image records of the thread are obtained, and the time length spent in pre-processing each frame of image (for example, the total time delay from data loading to pre-processing completion) is extracted from the historical pre-processed image records, and then the average delay is calculated by averaging the time length spent each time in the historical pre-processed image records. The lower the average delay, the faster the processing speed of the thread, and the more suitable it is for assigning high-priority tasks; according to the historical pre-processed image records, the number of pre-processing tasks completed by the thread per unit time (for example, the number of image frames processed per second) is counted and recorded as the throughput; the thread with high throughput indicates that the resource utilization is good and can carry more complex image pre-processing; The number of pending queues and the load (resource occupancy rate of the current thread) of the thread are obtained in real time, and the load, the number of pending queues, the average delay, and the throughput are normalized and recorded as F, N, C, and Y respectively; the load, the number of pending queues, the average delay, and the throughput are calculated to obtain the load-related value, the queue-related value, the delay-related value, and the throughput-related value, and then the load-related value, the queue-related value, the delay-related value, and the throughput-related value are weighted to obtain the performance index of the thread; the specific calculation formula is wherein Fmax is the maximum load allowed by the thread in the current application scenario, Namx is the maximum number of queues allowed by the thread, Cmax is the maximum delay allowed by the thread; Ymin is the minimum throughput of the thread; the threads are sorted in descending order of their corresponding performance indexes, and the thread with the largest performance index is selected as the designated thread, and the pending task is assigned to the designated thread, and the number of pending queues of the designated thread is updated; Step three: repeat steps one to two until all pre-processing tasks are assigned; The detection node subscribes to the Topic published by the processing node, receives the image data, and performs welding defect detection according to the order of the collection time, and outputs the welding detection result; Adaptive learning: When a new process is detected (such as switching to aluminum welding), shallow features are extracted from the current batch of images (reduced to 100 dimensions using PCA). Cosine similarity is calculated with the historical feature library. If the difference exceeds the difference threshold, a model update is triggered. Incremental training is performed using the Elastic Weight Consolidation (EWC) algorithm, preserving existing knowledge. By extracting shallow features and using cosine similarity to detect new processes, model updates are triggered. Incremental learning, combined with the EWC algorithm, ensures that the model can quickly adapt to changes in new materials or processes.

[0025] By integrating real-time processing with adaptive learning, the efficiency and accuracy of weld defect detection have been significantly improved. For real-time processing, a multi-threaded parallel processing architecture is employed, utilizing GPU-accelerated preprocessing algorithms to ensure that the preprocessing latency for each image frame is kept within a reasonable range. This efficient processing capability meets the stringent real-time requirements of weld defect detection. Furthermore, the system dynamically adjusts task priorities based on parameters such as acquisition duration, resolution, ROI ratio, and quality score, and assigns them to the most appropriate thread for processing. This adaptive task scheduling mechanism ensures optimal utilization of system resources and improves overall processing efficiency. Furthermore, the system can detect the introduction of new processes and extract shallow features from the current batch of images, calculating cosine similarity with a historical feature library. When the difference exceeds a preset threshold, the system triggers a model update, employing the Elastic Weight Consolidation (EWC) algorithm for incremental training. This method retains existing knowledge while rapidly adapting to changes in new materials or processes. This adaptive learning mechanism ensures continuous model optimization and rapid adaptability to new processes.

[0026] According to an embodiment of the present invention, Figure 3 As shown, a welding defect recognition device based on artificial intelligence is also provided, which includes: an image pre-storage library, an adaptive fusion module, a defect detection module and a real-time processing module; The image pre-storage library receives in real time the continuous frame welding appearance images acquired by the high dynamic range camera and the temperature field distribution data monitored by the infrared thermal imager; The adaptive fusion module quickly transfers image data to CUDA memory, executes GPU-accelerated preprocessing algorithms, and transmits it to the defect detection module; The defect detection module identifies defects based on the pre-processed welding appearance image to output welding detection results and make corresponding decisions; The real-time processing module encapsulates the raw image data of any welding appearance image into a preprocessing task object, and processes each preprocessing task in real time to control the preprocessing process of the adaptive fusion module and the defect detection order of the defect detection module.

[0027] The above merely illustrates the embodiments of the present application, and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. within the spirit and principles of the present application should be included in the scope of the claims of the present application.

Claims

1. A welding defect recognition method based on artificial intelligence, characterized in that: This includes the following methods: S1: Use a high dynamic range camera to capture images of the surface of the welded part where welding defects need to be identified to obtain continuous frames of welding appearance images; at the same time, deploy an infrared thermal imager to monitor the temperature field distribution; S2: Dynamic range fusion processing is performed on multiple frames to compensate for detail loss in the highlight area of ​​a single frame. Hardware triggers synchronous acquisition. Within a single welding cycle, three sets of images with different exposures are obtained by switching the filter wheel: short exposure to capture the arc bright area, medium exposure to obtain weld pool details, and long exposure to extract dark defects. An adaptive HDR fusion algorithm is used for fusion reconstruction. S3: Design a notch filter by Fourier transforming the arc light periodic noise, and use the improved Retinex algorithm to eliminate the influence of non-uniform illumination; S4: Deep learning denoising based on physical simulation; S5: Multimodal data fusion and welding defect decision making; S6: Real-time processing and incremental learning.

2. The method for identifying welding defects based on artificial intelligence according to claim 1, characterized in that: Step S1 also includes: customizing a three-band filter wheel: Band 1: 850nm narrowband, Band 2: 1050nm, Band 3: visible light + polarizer, with a filter switching time of less than 10ms and triggered synchronously with the welding current pulse; the camera housing uses a double-layer shield of Permalloy, the grounding point impedance is less than 0.1Ω, the transmission line uses a DC-DC isolated power supply module to improve the common mode rejection ratio, and the optical fiber + photoelectric conversion module replaces the copper cable; the camera is installed with a semiconductor cooler, which cooperates with PID temperature control to keep the chip temperature less than 45°C, and an air curtain is installed at the front of the lens for isolation.

3. The welding defect identification method based on artificial intelligence according to claim 2 is characterized in that: Dynamic range fusion processing, specifically: S2-1: Using the Farneback dense optical flow algorithm, the algorithm for solving the displacement field between adjacent frames is: ; The displacement field between adjacent frames is solved by polynomial expansion approximation and least squares method. ; Where t is time, (x, y) is the pixel coordinate of the image; S2-2: Adaptive weight mapping: The noise mask is calculated by calculating the difference between the BM3D denoising result and the original image, and is mapped to the adaptive weight. The mapping formula is: ； k represents the sequence number of the image, To control exposure preference, the noise mask It is calculated by the difference between the BM3D denoising result and the original image. The calculation formula is: ; S2-3: Laplace pyramid fusion. The specific fusion process and formula are as follows: S2-3-1: Construct a Gaussian pyramid for each image : ;in is the pyramid level index, Represents the Gaussian pyramid The pixel value of the layer at coordinate (x, y), represents a 5×5 Gaussian kernel; ; N is an integer, indicating the number of pyramid layers; S2-3-2: Constructing a Laplace Pyramid : ; S2-3-3: Weighted fusion of pyramid layers: ;in Represents the fused Laplacian pyramid layer, Indicates that the kth image is in the Normalized weights of pyramid layers; S2-3-4: Rebuilding from the top: ; S2-4: Use CUDA to implement optical flow calculation and combine it with the selection of pyramid levels. The principle of selecting the number of pyramid levels is as follows: , where W is the image width and H is the image height.

4. The welding defect identification method based on artificial intelligence according to claim 3 is characterized in that: Deep learning denoising based on physical simulation, specifically: Use COMSOL Multiphysics to establish a welding heat-light-electromagnetic multi-physics model, generate annotated simulation images, and inject random noise into the collected field images to enhance the real data; Then train it through lightweight network and knowledge distillation method.

5. The method for identifying welding defects based on artificial intelligence according to claim 4, characterized in that: Multimodal data fusion and welding defect decision-making, specifically: Establish the mapping relationship between visible light image and infrared thermal image. The specific relationship algorithm is as follows: , α, β, and γ are obtained by least square fitting; Deep learning is used to perform visible light and infrared defect detection on the processed images to obtain the visible light defect probability and infrared defect probability. The ultrasonic data is subjected to signal analysis to obtain the ultrasonic defect probability. The visible light defect probability, infrared defect probability, and ultrasonic defect probability are then weighted and calculated to obtain the decision value Score. When Score>0.9, the system will be shut down immediately and marked as a serious defect; When 0.7<Score≤0.9, the production line speed is reduced and manual re-inspection is triggered; When Score≤0.7, data is recorded without interrupting production.

6. The welding defect identification method based on artificial intelligence according to claim 5 is characterized in that: Real-time processing, specifically: The acquisition node calls the Aravis driver to acquire continuous frame images to obtain welding appearance images, marks each frame with the precise time, and transmits it to the next topic; The preprocessing node subscribes to the topic published by the acquisition node, receives the original image, quickly transfers the image data to the CUDA memory, executes the GPU-accelerated preprocessing algorithm, and transfers it to the next topic. It encapsulates the original image data of any welding appearance image into a preprocessing task object and processes each preprocessing task in real time. The detection node subscribes to the topic published by the processing node, receives image data, and performs welding defect detection in the order of the corresponding acquisition time, and outputs the welding detection results.

7. The method for identifying welding defects based on artificial intelligence according to claim 6, characterized in that: Perform real-time processing on each preprocessing task. The specific steps are as follows: Step 1: Extract the preprocessing task and parse it to obtain the acquisition time, resolution, and RIO ratio of the preprocessing task; use the BRISQUE algorithm to extract the local statistical features of the image to measure the degree of distortion and output the image quality score; calculate the time difference between the acquisition time and the current time to obtain the acquisition duration, and normalize the acquisition duration, resolution, RIO ratio, and quality score. Then calculate them separately to obtain the duration-related value, resolution-related value, RIO-related value, and quality score-related value, and perform weighted calculation on them to obtain the processing value of the preprocessing task; arrange the processing values ​​of each preprocessing task in descending order, and select the preprocessing task with the largest processing value as the pending task; Step 2: The preprocessing node has several parallel threads, each of which can independently preprocess the welding appearance image and output the processed image data. The thread's historical image processing records are obtained, and the time spent on processing each frame of the image is extracted from them. The average time spent on each frame in the historical image processing records is then calculated to obtain the average delay. Based on the historical image processing records, the number of preprocessing tasks completed by the thread per unit time is counted and recorded as the throughput. The number of queues to be processed and the load of the thread are obtained in real time, and the load, number of queues to be processed, average delay, and throughput are normalized and recorded as F, N, C, and Y respectively; the load, number of queues to be processed, average delay, and throughput are calculated to obtain load-related values, queue-related values, delay-related values, and throughput-related values, and then the load-related values, queue-related values, delay-related values, and throughput-related values ​​are weighted to obtain the performance index of the thread; each thread is sorted in descending order according to its corresponding performance index, and the thread with the largest performance index is selected as the calibration thread, and the pending tasks are assigned to the calibration thread, and the number of queues to be processed of the calibration thread is updated at the same time; Step 3: Repeat steps 1 to 2 above until all pre-processing tasks are assigned.

8. The method for identifying welding defects based on artificial intelligence according to claim 7, characterized in that: Incremental learning, specifically: When a new process is detected: extract shallow features of the current batch of images; Calculate the cosine similarity with the historical feature library. If the difference is greater than the difference threshold, trigger the model update; use the EWC algorithm for incremental training to retain the original knowledge.

9. A welding defect recognition device based on artificial intelligence, characterized in that An artificial intelligence-based welding defect recognition method according to any one of claims 1 to 8, the device comprising: an image pre-storage library, an adaptive fusion module, a defect detection module, and a real-time processing module; The image pre-storage library receives in real time the continuous frame welding appearance images acquired by the high dynamic range camera and the temperature field distribution data monitored by the infrared thermal imager; The adaptive fusion module quickly transfers image data to CUDA memory, executes GPU-accelerated preprocessing algorithms, and transmits it to the defect detection module; The defect detection module identifies defects based on the pre-processed welding appearance image to output welding detection results and make corresponding decisions; The real-time processing module encapsulates the raw image data of any welding appearance image into a preprocessing task object, and processes each preprocessing task in real time to control the preprocessing process of the adaptive fusion module and the defect detection order of the defect detection module.

Images