Auto part weld defect detection method and system based on body perception

Abstract

The invention belongs to the technical field of part defect detection, and discloses an automobile part welding seam defect detection method and system based on body perception, which realize closed-loop control of welding seam defect detection through body perception circulation, improve the perception capability and adaptability of the system and realize high-precision and high-efficiency welding seam defect detection. Firstly, a high-resolution camera and an industrial camera are used for obtaining high-definition images of the automobile parts, and the high-definition images are used for recognizing classification and shapes of the automobile parts; further, comprehensive understanding of the shape of the welding seam is formed through the identified automobile parts and in combination with a localized large model; further, according to comprehensive understanding of the shape of the welding seam, a preset rule and a localization large model are combined, and detection steps and tracks are generated; furthermore, the mechanical arm completes the decomposition and execution of specific detection actions according to the generated detection path, such as moving to a specified position, detecting surface bulges, detecting bubbles and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of component defect detection, and in particular, relates to a method and system for detecting weld defects of automobile components based on embodied perception. Background Art

[0002] As the global manufacturing industry shifts toward intelligence and greening, China's automobile industry is in a critical period of transformation and upgrading. In this context, quality control of automobile parts has become one of the key links to achieve the above goals. In particular, welds are a crucial connection method in the manufacturing process of automobile parts, and their quality directly affects the safety and reliability of the entire vehicle. Therefore, the detection of weld defects has become an indispensable part of the automotive manufacturing industry. Due to the changes in the production site environment (such as fluctuations in temperature, humidity, lighting conditions, and the influence of factors such as oil stains and dust), the detection results are prone to errors. At the same time, traditional manual detection methods are inefficient and cannot meet the modern high-efficiency and high-quality production needs.

[0003] However, existing weld defect detection systems and methods, including X-ray detection, lidar detection, magnetic particle detection, artificial intelligence and machine vision, still face many challenges, such as insufficient adaptability to complex-shaped parts and insufficient detection accuracy, resulting in a low level of intelligence.

[0004] Through the above analysis, the technical difficulties in realizing intelligent detection of weld defects of automotive parts are as follows:

[0005] 1) There are many types of auto parts, and the detection trajectory is difficult to plan automatically; there are many types of auto parts, and some of them even have slight differences in shape. This makes it difficult to import and identify new car parts and plan their weld trajectories.

[0006] 2) Weld defects are tiny, slightly different, and partially invisible, making them difficult to locate and identify. There are many types of defects in welds, such as undercuts, cracks, arc pits, slag inclusions, pores, arc scars, and internal bubbles. These defects are often small in size, with subtle differences in features, and some are even hidden, making them difficult to accurately capture and identify during the detection process. Summary of the invention

[0007] In view of the problems existing in the prior art, the present invention provides a method and system for detecting weld defects of automobile parts based on embodied perception.

[0008] The present invention is implemented as follows: a method for detecting weld defects of automobile parts based on embodied perception comprises:

[0009] Step 1: Online fine-grained identification of automobile parts;

[0010] Step 2, automatic generation of weld detection steps and trajectories;

[0011] Step 3: Multimodal fusion detection of weld defects;

[0012] Step 4: Defect report expert system.

[0013] Furthermore, the online fine-grained identification of automobile parts:

[0014] Through the coordination of the logic control unit, the acquisition control unit controls the camera to efficiently acquire images, which are then processed by the AI ​​computing unit, and the automotive parts are identified using an online weakly supervised feature self-clustering algorithm, and the identification results are stored in the data storage unit.

[0015] Furthermore, the weld detection steps and trajectories are automatically generated:

[0016] By identifying the shape of automotive parts and combining it with localized large models, typical welds and their key features of such parts are mined from historical inspection data, the shape and position information of the welds are identified, and with the help of preset rules and localized large models, a reasonable defect detection process and motion path are automatically designed.

[0017] Furthermore, the multimodal fusion detection of weld defects:

[0018] Under the guidance of the logic control unit, the robotic arm works according to the detection trajectory, and the acquisition control unit collects images, 3D point cloud data and X-ray image data; finally, the data is uploaded to the AI ​​computing unit to complete multimodal fusion detection.

[0019] Firstly, a block detection deep learning algorithm is used to segment the ROI to be inspected. Furthermore, a multi-scale long-range fusion and non-local attention deep learning algorithm are used to detect and locate tiny targets in the ROI to be inspected. Then, a deep template matching algorithm is used to identify the fine-grained differences and defect types of visible targets. Finally, 3D point cloud, RGB image and X-ray image fusion detection are used to detect invisible defects.

[0020] Furthermore, the defect report expert system:

[0021] The system will automatically summarize and display the inspection results online, and feed these results back to the customized local big model to achieve intelligent incremental learning of defects.

[0022] Another object of the present invention is to provide an automobile parts weld defect detection system based on embodied perception, comprising:

[0023] The recognition module is used for online fine-grained recognition of automobile parts. Through the coordination of the logic control unit, the acquisition control unit controls the camera to efficiently collect images, and then the AI ​​computing unit processes these images, uses the online weakly supervised feature self-clustering algorithm to identify automobile parts, and stores the recognition results in the data storage unit;

[0024] The automatic trajectory generation module is used for automatic generation of weld detection steps and trajectories. By identifying the shape of automotive parts and combining it with the localized large model, it mines the typical welds and key features of such parts from historical inspection data, identifies the shape and location information of the welds, and automatically designs a reasonable defect detection process and motion path with the help of preset rules and the localized large model.

[0025] The fusion detection module is used for multimodal fusion detection of weld defects; under the guidance of the logic control unit, the robotic arm works according to the detection trajectory, and the acquisition control unit collects images, 3D point cloud data and X-ray image data; finally, the data is uploaded to the AI ​​computing unit to complete the multimodal fusion detection. First, the block detection algorithm is used to segment the ROI to be inspected; further, the multi-scale long-range fusion and non-local attention deep learning algorithms are used to detect and locate small targets in the ROI to be inspected; then, the deep template matching algorithm is used to identify the fine-grained differences and defect types of visible targets; finally, the 3D point cloud, RGB image and X-ray image fusion detection algorithm is used to detect invisible defects;

[0026] The defect report module is used to automatically summarize and display the detection results online through the defect report expert system, and at the same time feed these results back to the customized local big model to achieve intelligent incremental learning of defects.

[0027] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for detecting weld defects of automotive parts based on embodied perception.

[0028] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the method for detecting weld defects of automotive parts based on embodied perception.

[0029] Another object of the present invention is to provide an information data processing terminal, which is used to implement the automobile parts weld defect detection method system based on embodied perception.

[0030] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0031] First, the present invention realizes closed-loop control of weld defect detection of automotive parts through embodied perception loop, improves the perception ability and adaptability of the system, and realizes high-precision and high-efficiency weld defect detection. The specific process includes:

[0032] (1) Automotive parts perception: Use high-resolution cameras and industrial cameras to obtain high-definition images of automotive parts for identification of their classification and shape.

[0033] (2) Scene understanding: By identifying automotive parts and combining them with the localized large model, a comprehensive understanding of the weld shape is formed.

[0034] (3) Active decision-making: Based on a comprehensive understanding of the weld shape, combined with preset rules and localized large models, the detection steps and trajectories are generated.

[0035] (4) Defect detection execution: The robot arm decomposes and executes specific detection actions according to the generated detection path, such as moving to a specified position, detecting surface protrusions, detecting bubbles, etc. Multimodal fusion technology is used to fuse perception data of different modes to form a more comprehensive and accurate environmental model, and detect visible and invisible defects at the same time, improving the accuracy and robustness of detection.

[0036] (5) Feedback verification phase: During the detection process, the system collects feedback information in real time, such as detection results and sensor data, to verify the effect and execution progress and adjust subsequent detection strategies and paths.

[0037] Second, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0038] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0039] The automobile parts weld defect detection system based on embodied perception proposed in the present invention can efficiently and accurately detect automobile parts weld defects, solve the problems of low efficiency, insufficient detection accuracy and high dependence on operator skills of traditional weld detection methods, improve the quality inspection standards of parts in my country's automobile industry, significantly reduce the rework costs and recall costs caused by weld quality problems, and provide strong support for improving the overall quality of automobile parts.

[0040] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0041] This invention will promote the technical application innovation of embodied perception in the detection of weld defects of automobile parts, such as automobile parts identification, automatic generation of detection trajectories, and multi-modal fusion detection of weld defects, promote the improvement of automobile parts production technology, reduce manual detection links, and improve the level of automation in automobile parts production. This invention is cutting-edge and technologically innovative, filling the technical gap of embodied perception in the application of weld defect detection of automobile parts at home and abroad. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of a method for detecting weld defects of automobile parts based on embodied perception provided in an embodiment of the present invention.

[0043] Figure 2 It is a structural frame of an automobile component weld defect detection system based on embodied perception provided by an embodiment of the present invention.

[0044] Figure 3 It is a system function timing diagram provided by an embodiment of the present invention.

[0045] Figure 4 It is a diagram of an online weakly supervised feature self-clustering algorithm provided by an embodiment of the present invention.

[0046] Figure 5 It is a block detection algorithm diagram provided by an embodiment of the present invention.

[0047] Figure 6 It is a multi-scale long-range fusion algorithm diagram provided by an embodiment of the present invention.

[0048] Figure 7 It is a non-local attention algorithm diagram provided by an embodiment of the present invention.

[0049] Figure 8 It is a diagram of a deep template matching algorithm provided by an embodiment of the present invention.

[0050] Fig. 9 This is a block diagram of multimodal data acquisition provided by an embodiment of the present invention.

[0051] Fig.10 It is a multimodal fusion detection algorithm diagram provided by an embodiment of the present invention.

[0052] Fig.11 It is an experimental effect diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0054] like Figure 1 As shown, a method for detecting weld defects of automobile parts based on embodied perception provided by an embodiment of the present invention comprises the following steps:

[0055] S101, online fine-grained identification of automotive parts;

[0056] S102, automatic generation of weld detection steps and trajectories;

[0057] S103, multi-modal fusion detection of weld defects;

[0058] S104, defect report expert system.

[0059] The embodied perception-based weld defect detection method for automotive parts is divided into four main steps: online fine-grained recognition, automatic trajectory generation, multimodal fusion detection, and defect report output. The following is its detailed working principle:

[0060] Step 1: Online fine-grained identification of automotive parts

[0061] The system first uses a combination of high-definition camera equipment and deep learning models to perform online identification of automotive parts. The model can carefully identify different types of parts and accurately locate various weld areas, ensuring that the subsequent inspection can focus on key weld areas, thereby improving the accuracy and pertinence of the overall inspection.

[0062] Step 2: Automatic generation of weld detection steps and trajectories

[0063] Once the weld area is identified, the system will generate the optimal detection trajectory based on the identified weld shape and position. The trajectory generation module will automatically determine the best path based on the geometric characteristics and structural complexity of the weld, thereby ensuring that the detection equipment can cover the weld area efficiently and without blind spots. This automatic trajectory generation function can not only improve detection efficiency, but also ensure full coverage of detection and reduce the risk of omissions.

[0064] Step 3: Multimodal fusion detection of weld defects

[0065] Based on the generated detection trajectory, the system performs multimodal detection of weld defects. This module integrates multiple detection methods such as visual detection, lidar detection and X-ray, and fuses detection data of different modes to identify various potential weld defects, including common defects such as internal pores. Multimodal detection can make up for the limitations of a single detection method and improve the accuracy of detection and the comprehensiveness of defect identification through data fusion.

[0066] Step 4: Defect Report Expert System

[0067] After the inspection is completed, the system will feed back the inspection results to the defect report module and generate a detailed inspection report through the expert system. The report lists the various inspection data of the weld and the specific defect location, defect type and severity found. At the same time, the expert system can automatically analyze the cause of the defect and put forward maintenance or improvement suggestions to assist engineers to quickly deal with weld problems.

[0068] Step 5: Long-term data analysis and model update

[0069] The system will also continuously collect and store inspection data to form a long-term database of weld defects. Through in-depth analysis of the localized large model, the system can identify common patterns and trends of weld defects, and continuously update and optimize the inspection model to improve the accuracy and response speed of inspection.

[0070] The online fine-grained identification of automobile parts provided by the embodiment of the present invention:

[0071] Through the coordination of the logic control unit, the acquisition control unit controls the camera to efficiently acquire images, which are then processed by the AI ​​computing unit, and the automotive parts are identified using an online weakly supervised feature self-clustering algorithm, and the identification results are stored in the data storage unit.

[0072] The weld detection steps and trajectory automatic generation provided by the embodiment of the present invention are as follows:

[0073] By identifying the shape of automotive parts and combining it with localized large models, typical welds and their key features of such parts are mined from historical inspection data, the shape and position information of the welds are identified, and with the help of preset rules and localized large models, a reasonable defect detection process and motion path are automatically designed.

[0074] The multimodal fusion detection of weld defects provided by the embodiment of the present invention:

[0075] Under the guidance of the logic control unit, the robotic arm works according to the detection trajectory, and the acquisition control unit collects images, 3D point cloud data, X-ray images and other sensor data; finally, the data is uploaded to the AI ​​computing unit to complete multimodal fusion detection.

[0076] Firstly, a block detection deep learning algorithm is used to segment the ROI to be inspected. Furthermore, a multi-scale long-range fusion and non-local attention deep learning algorithm are used to detect and locate tiny targets in the ROI to be inspected. Then, a deep template matching algorithm is used to identify the fine-grained differences and defect types of visible targets. Finally, 3D point cloud, RGB image and X-ray image fusion detection are used to detect invisible defects.

[0077] The defect report expert system provided by the embodiment of the present invention:

[0078] The system will automatically summarize and display the inspection results online, and feed these results back to the customized local big model to achieve intelligent incremental learning of defects.

[0079] like Figure 2 As shown, an embodiment of the present invention provides a method system for detecting weld defects of automobile parts based on embodied perception, comprising:

[0080] The recognition module is used for online fine-grained recognition of automobile parts: through the coordination of the logic control unit, the acquisition control unit controls the camera to efficiently collect images, and then the AI ​​computing unit processes these images, uses the online weakly supervised feature self-clustering algorithm to identify automobile parts, and stores the recognition results in the data storage unit;

[0081] The automatic trajectory generation module is used for automatic generation of weld detection steps and trajectories. By identifying the shape of automotive parts and combining it with the localized large model, it mines the typical welds and key features of such parts from historical inspection data, identifies the shape and location information of the welds, and automatically designs a reasonable defect detection process and motion path with the help of preset rules and the localized large model.

[0082] The fusion detection module is used for multimodal fusion detection of weld defects. Under the guidance of the logic control unit, the robot arm works according to the detection trajectory, and the acquisition control unit collects images, 3D point cloud data, X-ray images and other sensor data. Finally, the data is uploaded to the AI ​​computing unit to complete the multimodal fusion detection. First, the block detection deep learning algorithm is used to segment the ROI to be inspected. Furthermore, the multi-scale long-range fusion and non-local attention deep learning algorithms are used to detect and locate small targets in the ROI to be inspected. Then, the deep template matching algorithm is used to identify the fine-grained differences and defect types of visible targets. Finally, 3D point cloud, RGB image and X-ray image fusion detection are used to detect invisible defects.

[0083] The defect report module is used to automatically summarize and display the detection results online through the defect report expert system, and at the same time feed these results back to the customized local big model to achieve intelligent incremental learning of defects.

[0084] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for detecting weld defects of automotive parts based on embodied perception.

[0085] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the method for detecting weld defects of automotive parts based on embodied perception.

[0086] Another object of the present invention is to provide an information data processing terminal, which is used to implement the automobile parts weld defect detection method system based on embodied perception.

[0087] The present invention is specifically implemented:

[0088] 1. System functions and components

[0089] The present invention innovatively proposes a system and method for detecting weld defects of automobile parts based on embodied perception. The system integrates seven logical parts, including multimodal data acquisition, acquisition control, logic control, data storage, AI calculation, user management platform and robotic arm, and consists of the following four functional modules: online fine-grained recognition of automobile parts, automatic generation of weld detection steps and trajectories, multimodal fusion detection of weld defects and defect report expert system. The system function sequence is as follows: Figure 3 shown.

[0090] 1) Online fine-grained identification of automotive parts

[0091] This system adopts an online weakly supervised feature self-clustering recognition algorithm to achieve online precise positioning and recognition and fine-grained classification of automotive parts welds, as well as self-incremental learning of weld shapes and types.

[0092] The core task of this stage is to coordinate the acquisition control unit to control the camera to efficiently collect images through the coordination of the logic control unit, and then the AI ​​computing unit processes these images, uses online weak supervision and feature self-clustering technology to identify auto parts, and stores the identification results in the data storage unit, so as to continuously enrich the localized large model library. With the accumulation of detection cases, the prediction accuracy of the model will gradually improve.

[0093] No subsequent human intervention is required. At the same time, the algorithm has the characteristics of strong anti-interference ability, high stability, and strong real-time performance, which can accurately solve the problems of automobile parts identification and positioning.

[0094] 2) Automatic generation of weld detection steps and trajectories

[0095] By identifying the shape of automotive parts and combining it with localized large models, typical welds and their key features of such parts are mined from historical inspection data, the shape and position information of the welds are identified, and with the help of preset rules and localized large models, a reasonable defect detection process and motion path are automatically designed.

[0096] The realization of this part of the function depends on the command of the logic control unit. The acquisition control unit is responsible for acquiring real-time images, while the AI ​​computing unit retrieves the data of the localized large model from the data storage unit and generates weld detection steps and trajectories in accordance with preset rules.

[0097] 3) Multimodal fusion detection of weld defects

[0098] The localized large model is used to complete the decomposition of tasks according to the weld type. For example, the robot arm moves to position 1 along the defect detection trajectory, detects surface protrusions, detects bubbles, moves to position 2, etc. In view of the detection difficulties such as the variety of defect types, the difficulty in capturing small targets, the difficulty in identifying fine-grained differences, and the difficulty in detecting invisible defects, this system adopts multimodal fusion detection to ensure its detection accuracy. First, the block detection deep learning algorithm is used to segment the ROI to be inspected; further, the multi-scale long-range fusion and non-local attention deep learning algorithms are used to detect and locate small targets in the ROI to be inspected; then, the deep template matching algorithm is used to identify the fine-grained differences and defect types of visible targets; finally, 3D point cloud, RGB image and X-ray image fusion detection are used to detect invisible defects.

[0099] This step completes the multimodal fusion detection of weld defects. First, the AI ​​computing unit automatically performs the disassembly of the detection task; then, under the guidance of the logic control unit, the robotic arm works according to the detection trajectory, and the acquisition control unit collects images, 3D point cloud data and X-ray data; finally, the data is uploaded to the AI ​​computing unit to complete the multimodal fusion detection.

[0100] 4) Defect Report Expert System

[0101] Finally, the system will automatically summarize and display the inspection results online, and feed these results back to the customized local large model to achieve intelligent incremental learning of defects.

[0102] Under the management of the logic control unit, the specific defect detection results obtained in the previous link are presented in real time through external devices such as displays, which is convenient for on-site monitoring. At the same time, all defect data will be safely stored in the data storage unit to enrich the localized large model library and support the long-term development and optimization of the system. Through continuous self-learning, the system can not only improve the understanding and modeling capabilities of weld detection, but also promote the continuous improvement of detection accuracy.

[0103] 2. Deep learning algorithm solution

[0104] (1) Online weakly supervised feature self-clustering algorithm - automobile parts shape recognition and positioning

[0105] Considering the variety of auto parts and the increase in weld types during the inspection process, as well as the possible changes in ambient light, we use an online weakly supervised feature self-clustering weld recognition algorithm in auto parts identification. The stored feature library can effectively solve the problem of false reports caused by sudden changes in various external environments. For example, the feature library stores the characteristics of product specifications in various environments such as changes in indoor light, lens motion blur, and changes in component color, which can effectively solve the problem of false reports caused by environmental changes. In order to cope with the limitations and uncertainties of human cognitive ability and reduce the workload of supervisors to delete redundant features and images, our feature library automatically deletes highly redundant features and their corresponding images through self-clustering methods, ensuring that there is no need to modify the model when adding more auto parts in the future, and new parts can be quickly imported without manual intervention.

[0106] Figure 4 It is an online weakly supervised feature self-clustering algorithm.

[0107] (2) Block detection, multi-scale long-range fusion and non-local attention algorithm - positioning and identification of tiny weld defects.

[0108] ①Block detection algorithm

[0109] In this system, since the weld defect target scale is small, a block detection algorithm is used to extract block features. Taking into account the memory limitation of the project, no sliding window changes and cross-block fusion are performed in the backbone network and intermediate layer structure stages, and cross-block fusion is only performed in the input layer of the prediction head.

[0110] Figure 5 It is a block detection algorithm.

[0111] ② Non-local attention algorithm

[0112] Targets are usually related to the surrounding environment (for example, defects usually appear in concentrated locations or near certain components, or multiple defects are related to each other). This connection can be used to improve the accuracy of target detection. In the detection of defects in welds of automotive parts, small targets have few input pixels, and the amount of information and effective features contained in a single target are not enough for direct and effective detection. It is necessary to enhance the feature extraction of non-local attention, including local and long-distance correlations.

[0113] Figure 6 It is a non-local attention algorithm.

[0114] ③Multi-scale long-range fusion algorithm

[0115] In the detection of weld defects of automotive parts, due to the small target scale, a multi-scale long-range fusion algorithm is used to capture features of different scales by fusing information from different layers of the network. Generally, deep networks can capture higher-level abstract features, while shallow networks can better preserve the spatial details of the original input. By exchanging and fusing information at multiple scales, multi-scale long-range fusion can effectively combine local details with global structures, thereby improving the representation ability and generalization performance of the model.

[0116] Figure 7 It is a multi-scale long-range fusion algorithm.

[0117] (3) Deep template matching algorithm - visual weld defect detection

[0118] This system uses a deep template matching algorithm. After the defect image and the template image are extracted by the backbone network, the features are subtracted and then sent to the feature pyramid network. This is conducive to the network's learning of defect features, effectively reducing the impact of the difficulty in strictly aligning the defect image and the template image, and solving the problem of the difficulty in detecting the subtle differences between the defect and the normal target image in the visible weld defect.

[0119] Figure 8 It is a deep template matching algorithm.

[0120] (4) Multimodal fusion detection algorithm - invisible weld defect detection

[0121] Although it has been enhanced through the detection of small targets and fine-grained difference recognition, there are still invisible weld defects such as internal pores and deformation that are difficult to distinguish on the image. This system collects visible light images, high-quality 3D point cloud data and X-ray images of welds, and uses multi-modal fusion detection algorithms to more comprehensively detect information such as internal pores and deformations in welds, solving the problem of invisible weld defect detection.

[0122] Fig. 9 This is a block diagram of multimodal data acquisition.

[0123] Fig.10 It is a multimodal fusion detection algorithm.

[0124] 4 Technical Features

[0125] The present invention realizes closed-loop control of weld defect detection through embodied perception cycle, improves the perception ability and adaptability of the system, and realizes high-precision and high-efficiency weld defect detection. The specific process includes:

[0126] (1) Automotive parts perception: Use high-resolution cameras and industrial cameras to obtain high-definition images of automotive parts for identification of their classification and shape.

[0127] (2) Scene understanding: By identifying automotive parts and combining them with the localized large model, a comprehensive understanding of the weld shape is formed.

[0128] (3) Active decision-making: Based on a comprehensive understanding of the weld shape, combined with preset rules and localized large models, the detection steps and trajectories are generated.

[0129] (4) Defect detection execution: The robot arm decomposes and executes specific detection actions according to the generated detection path, such as moving to a specified position, detecting surface protrusions, detecting bubbles, etc. Multimodal fusion technology is used to fuse perception data of different modes to form a more comprehensive and accurate environmental model, and detect visible and invisible defects at the same time, improving the accuracy and robustness of detection.

[0130] (5) Feedback verification phase: During the detection process, the system collects feedback information in real time, such as detection results and sensor data, to verify the effect and execution progress and adjust subsequent detection strategies and paths.

[0131] The following is relevant evidence of the technical effects obtained by the embodiments of the present invention:

[0132] Firstly, a block detection deep learning algorithm is used to segment the ROI to be inspected. Furthermore, a multi-scale long-range fusion and non-local attention deep learning algorithm are used to detect and locate tiny targets in the ROI to be inspected. Then, a deep template matching algorithm is used to identify the fine-grained differences and defect types of visible targets. Finally, 3D point cloud, RGB image and X-ray image fusion detection are used to detect invisible defects, and the positioning and classification of various defects such as undercuts, cracks, craters, slag inclusions, pores, arc scars and internal bubbles are completed.

[0133] Experimental results: Fig.11 shown.

[0134] System accuracy comparison:

[0135]

[0136]

[0137] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0138] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for detecting weld defects of automobile parts based on embodied perception, characterized in that: The following steps are involved: Step 1. Online fine-grained identification of automobile parts: Through the combination of high-definition camera equipment and deep learning models, fine-grained online recognition of automotive parts is carried out, the shape of automotive parts is accurately identified and the weld area is located, so as to realize the detection of key weld areas; Step 2. Automatic generation of weld detection steps and trajectories: Generate the optimal inspection trajectory based on the identified shape of automobile parts and weld positions, so that the inspection equipment can fully cover the weld area and ensure that no part is missed during the inspection; Step 3. Multimodal fusion detection of weld defects: Use multi-modal detection methods such as visual detection, lidar detection, and X-ray detection, as well as deep learning algorithms, to detect defects in the weld area, identify various defects such as cracks, pores, and undercuts in the weld, and improve the accuracy and comprehensiveness of detection by fusing multi-modal detection data; Step 4. Defect Report Expert System: The inspection data is transmitted to the defect report module, and a detailed inspection report including the defect location, type and severity is automatically generated through the expert system, and defect cause analysis and repair improvement suggestions are provided.

2. The method and system for detecting weld defects of automobile parts based on embodied perception as claimed in claim 1, characterized in that: The online fine-grained identification of automobile parts: Through the coordination of the logic control unit, the acquisition control unit controls the camera to efficiently acquire images, which are then processed by the AI ​​computing unit, and the automotive parts are identified using an online weakly supervised feature self-clustering algorithm, and the identification results are stored in the data storage unit.

3. The method and system for detecting weld defects of automobile parts based on embodied perception as claimed in claim 1, characterized in that: The weld detection steps and trajectory are automatically generated: By identifying the shape of automotive parts and combining it with localized large models, typical welds and their key features of such parts are mined from historical inspection data, the shape and position information of the welds are identified, and with the help of preset rules and localized large models, a reasonable defect detection process and motion path are automatically designed.

4. The method and system for detecting weld defects of automobile parts based on embodied perception as claimed in claim 1, characterized in that: Multimodal fusion detection of weld defects: Under the guidance of the logic control unit, the robotic arm works according to the detection trajectory, and the acquisition control unit collects images, 3D point cloud data and X-ray image data; finally, the data is uploaded to the AI ​​computing unit to complete multimodal fusion detection. Firstly, a block detection deep learning algorithm is used to segment the ROI to be inspected. Furthermore, a multi-scale long-range fusion and non-local attention deep learning algorithm are used to detect and locate tiny targets in the ROI to be inspected. Then, a deep template matching algorithm is used to identify the fine-grained differences and defect types of visible targets. Finally, 3D point cloud, RGB image and X-ray image fusion detection are used to detect invisible defects.

5. The method and system for detecting weld defects of automobile parts based on embodied perception as claimed in claim 1, characterized in that: The defect report expert system: The system will automatically summarize and display the inspection results online, and feed these results back to the customized local big model to achieve intelligent incremental learning of defects.

6. An automobile parts weld defect detection system based on embodied perception implementing the detection method according to any one of claims 1 to 5, characterized in that: include: The recognition module is used for online fine-grained recognition of automobile parts. Through the coordination of the logic control unit, the acquisition control unit controls the camera to efficiently collect images. The AI ​​computing unit uses the online weakly supervised feature self-clustering algorithm to process the image, recognize the automobile parts, and store the recognition results in the data storage unit; The automatic trajectory generation module is used for automatic generation of weld detection steps and trajectories. It recognizes the shape and position information of welds by identifying the shape of automobile parts, combining localized large models and historical detection data, and automatically designs a reasonable defect detection process and motion path. Fusion detection module, used for multimodal fusion detection of weld defects. The robot arm works according to the detection trajectory. The acquisition control unit collects images, 3D point cloud data and X-ray image data. The AI ​​computing unit uses a block detection algorithm to segment ROIs. Multi-scale long-range fusion and non-local attention algorithms detect tiny targets. Invisible defect detection is achieved by combining 3D point cloud, RGB image and X-ray image. The defect report module is used to automatically summarize and display the detection results through the defect report expert system, and at the same time feed the results back to the localized large model to update the incremental learning data.

7. A computer device, characterized in that: The computer device comprises: Memory for storing computer programs; The processor is used to execute the computer program stored in the memory to perform the steps of the method for detecting weld defects of automobile parts based on embodied perception, including: The control acquisition control unit collects image data and performs component recognition through the AI ​​computing unit; Generate weld inspection trajectory; Perform fusion inspection, collect 3D point cloud, RGB image and X-ray image, and complete defect detection; Summarize defect data and generate inspection reports.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of a method for detecting weld defects of automobile parts based on embodied perception, including: The control acquisition control unit acquires image data of automobile parts and identifies the shapes and features of the parts; Automatically generate inspection tracks and processes based on the identified component information; Collect multimodal data and complete defect detection through multimodal fusion detection algorithm; Generate a test report and feed back the test results to the big model to update the incremental learning data.

9. An information data processing terminal, characterized in that: The information data processing terminal comprises: An automotive parts weld defect detection system based on embodied perception, which is used to realize online identification of automotive parts, weld detection trajectory generation, fusion detection and defect result display; The computing unit, in conjunction with the logic control unit and the acquisition control unit, performs weld defect analysis based on multi-modal fusion detection; The display unit is used to display the detection track, weld defect analysis results and defect reports in real time.

Images