A metal material detection method and device based on multi-mode excitation source

Abstract

The present invention relates to the technical field of metal material detection, and in particular to a metal material detection method and device based on a multi-mode excitation source. When the present invention uses multi-modal data to perform metal material detection, the defect type of the metal material is first determined, and then different multi-modal data feature fusion methods are adopted. When the metal material to be tested is of a single defect type, a convolutional neural network model based on an attention mechanism is used to perform feature fusion of multi-modal features. When the metal material to be tested is of a multi-defect type, a hierarchical weighted summation method is used to perform feature fusion of multi-modal features. This makes feature fusion more purposeful, improves the accuracy of feature fusion, and thereby improves the accuracy of metal material defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of metal material detection, and in particular to a metal material detection method and device based on a multi-mode excitation source. Background Art

[0002] Metal materials are widely used in modern industry, and their quality and performance are directly related to product safety and reliability. Traditional metal material inspection methods primarily include ultrasonic testing, radiographic testing, and eddy current testing. However, these methods have limitations. For example, ultrasonic testing lacks accuracy in qualitative and quantitative analysis of defects, radiographic testing poses radiation hazards to the environment and personnel, and eddy current testing has limited ability to detect deep defects. Therefore, the development of efficient, accurate, non-destructive inspection methods and devices applicable to a wide range of metal materials is of great significance. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention provides a metal material detection method and device based on a multi-mode excitation source, which are used to solve the problems existing in the prior art.

[0004] The present invention provides a metal material detection method based on a multi-mode excitation source, comprising the following steps:

[0005] S1: Set up a multi-mode excitation source for metal material detection;

[0006] S2: Collect multi-mode excitation signals through the detection unit;

[0007] S3: performing a data preprocessing operation on the multi-mode excitation signal to obtain a preprocessed signal;

[0008] S4: performing a feature fusion operation on the preprocessed signal to obtain a multimodal fusion feature;

[0009] The S4 is specifically as follows: S4.1: performing deep feature extraction on the preprocessed signal to obtain ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features; S4.2: inputting the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features into the metal material defect model classification model, and outputting a classification result of the metal material to be tested; S4.3: selecting a feature fusion method for the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features according to the classification result, and outputting a multimodal fusion feature;

[0010] S5: Input the multimodal fusion features into a defect detection model, and output the defects of the metal material to be tested.

[0011] Preferably, the step S4.1 is as follows: inputting the pre-processed ultrasonic signal, electromagnetic signal, thermal imaging signal and vibration signal into a feature extraction model to obtain the ultrasonic signal feature F u , electromagnetic signal characteristics F e , Thermal imaging signal characteristics F t And the vibration signal feature F v .

[0012] Preferably, in S4.2, the metal material defect classification model is a convolutional neural network model, the input of the convolutional neural network model is the ultrasonic signal characteristics, electromagnetic signal characteristics, thermal imaging signal characteristics and vibration signal characteristics, and the classification result of the metal material to be tested is that the metal material to be tested is of two types: single defect type and multiple defect type.

[0013] Preferably, in S4.3, when the metal material to be tested is of a single defect type, the feature fusion method is to perform feature fusion on the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features and vibration signal features through a convolutional neural network model based on an attention mechanism.

[0014] Preferably, in S4.3, when the metal material to be tested is of a multi-defect type, the feature fusion method is to perform feature fusion on the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features and vibration signal features through a hierarchical weighted summation method.

[0015] Preferably, the attention mechanism is specifically:

[0016] Calculate the attention weight of each modality feature;

[0017] The formula is:

[0018]

[0019] ;

[0020] Where a i The attention score for each modality feature, W a and b a is a learnable parameter, F i is the feature of the i-th mode, is the attention weight for each modality feature;

[0021] Performing feature fusion on the ultrasonic signal feature, the electromagnetic signal feature, the thermal imaging signal feature, and the vibration signal feature according to the attention weight of each modal feature to obtain a multimodal fusion feature;

[0022] ;

[0023] Where, F fused is the fused multimodal feature.

[0024] Preferably, the hierarchical weighted summation method is specifically:

[0025] The ultrasonic signal features, electromagnetic signal features, thermal imaging signal features and vibration signal features are aligned to the same size; the aligned features are primary fused to obtain primary fused features; the primary fused features are secondary fused to obtain final multimodal fused features.

[0026] Preferably, the primary fusion specifically comprises: performing preliminary weighted summation on the features of each modality;

[0027] Assume that the primary fusion weight of each modality is w u , w e , w t , w v , then the primary fusion feature F primary The calculation formula is:

[0028] ;

[0029] The primary fusion weight of each modal feature is determined according to the statistical characteristics of each modal feature; specifically:

[0030] ;

[0031] ;

[0032] ;

[0033] .

[0034] Where, 、 、 、 They are the variances of ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features respectively;

[0035] Further weighted summation of the primary fusion features is performed to obtain the secondary fusion feature F mid , assuming that the secondary fusion weight is w mid :

[0036] ;

[0037] Among them, the calculation formula of the secondary fusion weight is:

[0038] ;

[0039] Where α and β are learnable parameters.

[0040] Preferably, the method for determining α and β is: first, using random numbers to assign initial values ​​to α and β, and using the training set to perform multiple training iterations on the convolutional neural network model. In each iteration, the training set is forward propagated to calculate the output and loss value of the convolutional neural network model, and the gradient of the loss value with respect to each parameter is calculated by the backpropagation algorithm. Then, the Adam optimizer is used to update the α and β values ​​until the loss value meets the requirements, and the α and β values ​​at this time are output as the final values.

[0041] According to another aspect of the present invention, a metal material detection device based on a multi-mode excitation source is provided, wherein the device adopts the above-mentioned metal material detection method based on a multi-mode excitation source, and the device comprises:

[0042] Multi-mode excitation source;

[0043] A detection unit, for collecting multi-mode excitation signals;

[0044] a preprocessing unit, configured to perform a data preprocessing operation on the multi-mode excitation signal to obtain a preprocessed signal;

[0045] A feature fusion unit, configured to perform a feature fusion operation on the preprocessed signal to obtain a multimodal fusion feature;

[0046] The defect diagnosis unit is used to input the multimodal fusion features into the defect detection model and output the defects of the metal material to be tested.

[0047] The embodiments of the present invention have the following technical effects:

[0048] When using multimodal data for metal material detection, the present invention first determines the defect type of the metal material and then adopts different multimodal data feature fusion methods. When the metal material to be tested is of a single defect type, a convolutional neural network model based on an attention mechanism is used to perform feature fusion on the multimodal features. When the metal material to be tested is of multiple defect types, a hierarchical weighted summation method is used to perform feature fusion on the multimodal features. This makes feature fusion more purposeful, improves the accuracy of feature fusion, and thus improves the accuracy of metal material defect detection.

[0049] At the same time, when the metal material to be tested is of multiple defect types, through hierarchical weighted sum fusion and dynamic weight adjustment, the primary fusion weight is determined based on the inherent characteristics of multimodal data when the primary weight is determined, and then secondary feature fusion is performed based on the deep learning method, so that the feature fusion is more accurate when facing complex defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 This is a flow chart of a metal material detection method based on a multi-mode excitation source provided by an embodiment of the present invention;

[0052] Figure 2 This is a flowchart of performing a feature fusion operation on the preprocessed signal to obtain multimodal fusion features, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention are described clearly and completely below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0054] Attachment Figure 1 A flow chart of a metal material detection method based on a multi-mode excitation source is shown in the attached figure. Figure 1 As shown, a metal material detection method based on a multi-mode excitation source includes the following steps:

[0055] S1: Set up a multi-mode excitation source for metal material detection;

[0056] In the metal material detection method based on multi-mode excitation source, the excitation source is one of the core parts of the detection system. Its function is to stimulate the response signal inside the metal material through different physical methods, thereby providing basic data for subsequent detection.

[0057] Wherein, the multi-mode excitation source includes one or more of an ultrasonic source, a mechanical vibration source, a heat source and an electromagnetic field source;

[0058] The ultrasonic source emits high-frequency mechanical waves that propagate and reflect within metal materials. When the ultrasonic waves encounter internal defects (such as cracks and pores) within the material, they generate reflected and transmitted waves. By monitoring the propagation characteristics of these waves, information about the material's internal structure can be obtained. The electromagnetic field source generates an alternating electromagnetic field, which induces eddy currents within the metal material. The distribution and intensity of eddy currents are closely related to the material's electrical conductivity, magnetic permeability, and internal defects. By monitoring changes in eddy currents, the material's electromagnetic properties can be analyzed. The heat source locally heats the metal material, causing it to respond thermally. Internal defects (such as cracks and pores) can affect heat conduction, leading to abnormal local temperature distribution. Detecting these temperature changes using thermal imaging technology can locate the defects. The mechanical vibration source applies external vibrations, causing the metal material to vibrate in response. Internal defects can affect the propagation and attenuation characteristics of the vibration. By monitoring parameters such as the frequency and amplitude of the vibration signal, the material's internal structure can be analyzed.

[0059] Furthermore, the parameters of the excitation source include an ultrasonic frequency range of 1 MHz-20 MHz, and a suitable frequency is selected according to the detection target. For example, when detecting thicker metal materials, a lower frequency (such as 1 MHz-5 MHz) is selected; when detecting thinner metal materials or tiny defects, a higher frequency (such as 10 MHz-20 MHz) is selected; the frequency range of the electromagnetic field source is 10 Hz-1 MHz, and a suitable frequency is selected according to the conductivity of the material and the detection depth. For example, when detecting surface and near-surface defects, a higher frequency (such as 100 kHz-1 MHz) is selected; when detecting internal defects, a lower frequency (such as 10 Hz-10kHz) is selected. The heating temperature range of the heat source is 50°C-300°C, and a suitable temperature is selected according to the thermal conductivity characteristics of the material and the detection requirements. For example, when testing thermal resistance changes, a higher heating temperature (e.g., 200°C - 300°C) is selected; when testing thermal conductivity characteristics, a lower heating temperature (e.g., 50°C - 100°C) is selected. The frequency range of the mechanical vibration source is 10 Hz - 200 Hz; the appropriate frequency is selected based on the material's elastic modulus and the test objective. For example, when testing for internal cracks, a lower frequency (e.g., 10 Hz - 50 Hz) is selected; when testing for high-frequency vibration characteristics, a higher frequency (e.g., 100 Hz - 200 Hz) is selected.

[0060] S2: Collect multi-mode excitation signals through the detection unit;

[0061] The purpose of signal acquisition is to collect the response signals of metal materials under different excitation sources. These signals contain important information such as the internal structure, defect characteristics and composition distribution of the metal materials.

[0062] The multi-mode excitation signal acquisition includes ultrasonic signal acquisition, vibration signal acquisition, thermal imaging signal acquisition and electromagnetic signal acquisition.

[0063] Furthermore, a piezoelectric ultrasonic sensor is used to collect ultrasonic signals, which can convert ultrasonic signals into electrical signals; an acceleration sensor or a displacement sensor is used to collect vibration signals, which can convert vibration signals into electrical signals; an infrared thermal imager is used to collect thermal imaging signals, which can convert temperature distribution into thermal imaging images; an electromagnetic induction sensor, such as a coil sensor, is used to collect electromagnetic signals, which can detect changes in eddy current signals; therefore, the detection unit includes a piezoelectric ultrasonic sensor, an acceleration sensor or a displacement sensor, an infrared thermal imager, and a magnetic induction sensor.

[0064] Among them, when collecting multi-mode excitation signals, in certain complex detection scenarios (such as weak response signals), the signal acquisition parameters are dynamically adjusted according to real-time feedback during the detection process; for example, if the signal-to-noise ratio of the signal collected by the ultrasonic detection unit is low, the intensity of the ultrasonic source can be appropriately increased or the emission frequency can be adjusted; if the temperature distribution image collected by the thermal imaging detection unit is not clear, the heating temperature or heating time of the heat source can be adjusted.

[0065] S3: performing a data preprocessing operation on the multi-mode excitation signal to obtain a preprocessed signal;

[0066] Preprocessing collected signals is a key step in signal processing. Its purpose is to remove noise and interference, extract effective signal features, and provide high-quality data for subsequent analysis and processing. The signal acquisition process is inevitably affected by external noise and internal interference, such as electromagnetic interference, environmental vibration, and sensor noise. Preprocessing can effectively remove this noise and improve the signal-to-noise ratio.

[0067] Wherein, the data preprocessing operation includes filtering processing and digital conversion;

[0068] The filtering process specifically includes: filtering the ultrasonic signal with a bandpass filter, setting the passband frequency to 1 MHz - 5 MHz; filtering the electromagnetic signal with a high-pass filter, performing mean filtering on the thermal imaging signal, and performing median filtering on the vibration signal;

[0069] The digital conversion is to convert the ultrasonic signal, the electromagnetic signal and the vibration signal into digital signals through an analog-to-digital converter (ADC).

[0070] S4: performing a feature fusion operation on the preprocessed signal to obtain a multimodal fusion feature;

[0071] In metal material testing, signals from multiple physical fields are typically collected, such as ultrasonic, electromagnetic, thermal, and vibration signals. Each of these signals contains different information about the material's internal structure and defects. To comprehensively analyze the material's properties, this multimodal data needs to be fused. Traditional feature fusion methods (such as principal component analysis and neural networks) have limitations when processing multimodal data, such as inadequate feature extraction and unsatisfactory fusion results. Therefore, this embodiment proposes a multimodal feature fusion and analysis method based on deep learning to improve the accuracy of feature fusion and the depth of analysis.

[0072] Specifically, as attached Figure 2 As shown, the S4 is specifically:

[0073] S4.1: Performing deep feature extraction on the preprocessed signal to obtain ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features;

[0074] The S4.1 is as follows: inputting the pre-processed ultrasonic signal, electromagnetic signal, thermal imaging signal and vibration signal into the feature extraction model to obtain the ultrasonic signal feature F u , electromagnetic signal characteristics F e , Thermal imaging signal characteristics F t And the vibration signal feature F v .

[0075] In this embodiment, the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features are represented in the form of feature vectors;

[0076] The feature extraction model is a convolutional neural network model (CNN).

[0077] S4.2: Inputting the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features into the metal material defect model classification model, and outputting a classification result of the metal material to be tested;

[0078] The metal material defect classification model is a convolutional neural network model, the input of which is the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features. The classification results of the metal material to be tested are that the metal material to be tested is of a single defect type and a multiple defect type.

[0079] The single defect type refers to the presence of one defect in the inspection part of the metal material to be tested; the multiple defect type refers to the presence of at least two defects in the inspection part of the metal material to be tested, including cracks, holes, corrosion, inclusions, uneven distribution of components, etc.

[0080] S4.3: Selecting a feature fusion method for the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features based on the classification results, and outputting a multimodal fusion feature;

[0081] When the metal material to be tested is of a single defect type, the feature fusion method is to fuse the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features through a convolutional neural network model based on an attention mechanism;

[0082] Specifically, the attention mechanism is as follows:

[0083] Calculate the attention weight of each modality feature;

[0084] The formula is:

[0085]

[0086] ;

[0087] Where a i The attention score for each modality feature, W a and b a is a learnable parameter, F i is the feature of the i-th mode, is the attention weight for each modality feature;

[0088] Performing feature fusion on the ultrasonic signal feature, the electromagnetic signal feature, the thermal imaging signal feature, and the vibration signal feature according to the attention weight of each modal feature to obtain a multimodal fusion feature;

[0089] ;

[0090] Where, F fused is the fused multimodal feature;

[0091] In this step, multimodal data is learned through deep feature extraction and attention mechanism to obtain fused multimodal features, thereby improving the accuracy of multimodal feature fusion.

[0092] When the metal material to be tested is of multiple defect types, the feature fusion method is to fuse the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features and vibration signal features through a hierarchical weighted summation method;

[0093] The hierarchical weighted summation method is specifically as follows:

[0094] aligning the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features to the same size;

[0095] Specifically, the sizes of the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features are aligned by upsampling;

[0096] The specific formula is:

[0097]

[0098] Where, 、 、 、 is the feature after alignment;

[0099] Perform primary fusion on the aligned features to obtain primary fusion features;

[0100] The primary fusion specifically includes: performing preliminary weighted summation on the features of each modality;

[0101] Assume that the primary fusion weight of each modality is w u , w e , w t , w v , then the primary fusion feature F primary The calculation formula is:

[0102] ;

[0103] The primary fusion weight of each modal feature is determined according to the statistical characteristics of each modal feature; specifically:

[0104] ;

[0105] ;

[0106] ;

[0107] ;

[0108] Where, 、 、 、 They are the variances of ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features respectively;

[0109] Further weighted summation of the primary fusion features is performed to obtain the secondary fusion feature F mid , assuming that the secondary fusion weight is w mid :

[0110] ;

[0111] Among them, the calculation formula of the secondary fusion weight is:

[0112] ;

[0113] Where α and β are learnable parameters.

[0114] The determination method is as follows: first, use random numbers to assign initial values ​​to α and β, use the training set to perform multiple training iterations (epochs) on the convolutional neural network model, in each iteration, forward propagate the training set, calculate the output and loss value of the convolutional neural network model, and calculate the gradient of the loss value with respect to each parameter through the backpropagation algorithm, then use the Adam optimizer to update the α and β values ​​until the loss value meets the requirements, and output the α and β values ​​at this time as the final values.

[0115] In this scheme, through hierarchical weighted sum fusion and dynamic weight adjustment, the primary fusion weight is determined based on the inherent characteristics of multimodal data when the primary weight is determined, and then secondary feature fusion is performed based on the deep learning method, so that the feature fusion is more accurate when facing complex defects.

[0116] S5: Inputting the multimodal fusion features into a defect detection model to output defects of the metal material to be tested;

[0117] The defect monitoring model is a neural network model, the input of the neural network model is a multimodal feature fusion feature, and the output of the neural network model is a defect type.

[0118] In Example 2, the present invention further provides a metal material detection device based on a multi-mode excitation source, wherein the device adopts a metal material detection method based on a multi-mode excitation source according to Example 1, and the device comprises:

[0119] Multi-mode excitation source;

[0120] A detection unit, for collecting multi-mode excitation signals;

[0121] a preprocessing unit, configured to perform a data preprocessing operation on the multi-mode excitation signal to obtain a preprocessed signal;

[0122] A feature fusion unit, configured to perform a feature fusion operation on the preprocessed signal to obtain a multimodal fusion feature;

[0123] The defect diagnosis unit is used to input the multimodal fusion features into the defect detection model and output the defects of the metal material to be tested.

[0124] Example 3: The present invention also provides an electronic device, including one or more processors and a memory.

[0125] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0126] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement a metal material detection method based on a multi-mode excitation source and / or other desired functions of any embodiment of the present application described above. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0127] In one example, the electronic device may further include an input device and an output device, these components interconnected via a bus system and / or other connection mechanisms (not shown). The input device may include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including warning information, braking force, etc. The output device may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0128] Of course, for the sake of simplicity, components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0129] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to implement the functions of a metal material detection method based on a multi-mode excitation source provided in any embodiment of the present application.

[0130] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0131] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor enables the processor to implement a metal material detection method based on a multi-mode excitation source provided in any embodiment of the present application.

[0132] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A metal material detection method based on a multi-mode excitation source, characterized in that: The following steps are involved: S1: Set up a multi-mode excitation source for metal material detection; S2: Collect multi-mode excitation signals through the detection unit; S3: performing a data preprocessing operation on the multi-mode excitation signal to obtain a preprocessed signal; S4: performing a feature fusion operation on the preprocessed signal to obtain a multimodal fusion feature; The S4 is specifically as follows: S4.1: performing deep feature extraction on the pre-processed signal to obtain ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features; S4.2: Inputting the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features into the metal material defect model classification model, and outputting a classification result of the metal material to be tested; in S4.2, the classification result of the metal material to be tested is that the metal material to be tested is a single defect type or a multiple defect type; S4.3: Selecting a feature fusion method for the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features based on the classification results, and outputting a multimodal fusion feature; In S4.3, when the metal material to be tested is of a single defect type, the feature fusion method is to perform feature fusion on the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features through a convolutional neural network model based on an attention mechanism; In S4.3, when the metal material to be tested is of multiple defect types, the feature fusion method is to fuse the ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features through a hierarchical weighted summation method; S5: Input the multimodal fusion features into a defect detection model, and output the defects of the metal material to be tested.

2. The metal material detection method based on a multi-mode excitation source according to claim 1, characterized in that: The S4.1 is specifically as follows: inputting the pre-processed ultrasonic signal, electromagnetic signal, thermal imaging signal and vibration signal into the feature extraction model to obtain the ultrasonic signal feature F u 、Electromagnetic signal characteristics F e , Thermal imaging signal characteristics F t And the vibration signal feature F v .

3. The metal material detection method based on a multi-mode excitation source according to claim 2, characterized in that: In S4.2, the metal material defect classification model is a convolutional neural network model, and the input of the convolutional neural network model is the ultrasonic signal characteristics, electromagnetic signal characteristics, thermal imaging signal characteristics and vibration signal characteristics.

4. The metal material detection method based on a multi-mode excitation source according to claim 1, characterized in that: The attention mechanism is specifically: Calculate the attention weight of each modality feature; The formula is: ； ； Where a i is the attention score for each modality feature, W a and b a is a learnable parameter, F i is the feature of the i-th mode, is the attention weight for each modality feature; Performing feature fusion on the ultrasonic signal feature, the electromagnetic signal feature, the thermal imaging signal feature, and the vibration signal feature according to the attention weight of each modal feature to obtain a multimodal fusion feature; ； Where, F fused is the fused multimodal feature.

5. The metal material detection method based on a multi-mode excitation source according to claim 1, characterized in that: The hierarchical weighted summation method is specifically as follows: The ultrasonic signal feature F u 、Electromagnetic signal characteristics F e , Thermal imaging signal characteristics F t And the vibration signal feature F v Align to the same size; perform primary fusion on the aligned features to obtain primary fusion features; The primary fusion features are fused twice to obtain the final multimodal fusion features.

6. The metal material detection method based on a multi-mode excitation source according to claim 5, characterized in that: The primary fusion specifically includes: performing preliminary weighted summation on the features of each modality; Assume that the primary fusion weight of each modality is w u , w e , w t , w v , then the primary fusion feature F primary The calculation formula is: ； The primary fusion weight of each modal feature is determined according to the statistical characteristics of each modal feature; specifically: ； ； ； ； Where, 、 、 、 They are the variances of ultrasonic signal features, electromagnetic signal features, thermal imaging signal features, and vibration signal features respectively; Further weighted summation of the primary fusion features is performed to obtain the secondary fusion feature F mid , assuming that the secondary fusion weight is w mid : ； Among them, the calculation formula of the secondary fusion weight is: ； Where α and β are learnable parameters.

7. The metal material detection method based on a multi-mode excitation source according to claim 6, characterized in that: The method for determining α and β is as follows: first, random numbers are used to assign initial values ​​to α and β, and the convolutional neural network model is trained multiple times using the training set. In each iteration, the training set is forward propagated to calculate the output and loss value of the convolutional neural network model, and the gradient of the loss value with respect to each parameter is calculated by the backpropagation algorithm. Then, the Adam optimizer is used to update the α and β values ​​until the loss value meets the requirements, and the α and β values ​​at this time are output as the final values.

8. A metal material detection device based on a multi-mode excitation source, characterized in that: The device adopts the metal material detection method based on a multi-mode excitation source according to any one of claims 1 to 7, and the device includes: Multi-mode excitation source; A detection unit, for collecting multi-mode excitation signals; a preprocessing unit, configured to perform a data preprocessing operation on the multi-mode excitation signal to obtain a preprocessed signal; A feature fusion unit, configured to perform a feature fusion operation on the preprocessed signal to obtain a multimodal fusion feature; The defect diagnosis unit is used to input the multimodal fusion features into the defect detection model and output the defects of the metal material to be tested.

Images