Ultrasonic imaging model training method, molten pool detection method, device and electronic equipment

Abstract

The embodiment of the present application provides an ultrasonic imaging model training method, a molten pool detection method, a device and an electronic device, which relate to the field of laser ultrasonic detection technology. The ultrasonic imaging model training method includes: decoupling the first sample ultrasonic signal obtained by scanning the sample object with the laser ultrasonic detection equipment to obtain a sample sub-ultrasonic signal; using the sample sub-ultrasonic signal of the same type to generate the first sample ultrasonic image corresponding to the type; inputting the image features of the first sample ultrasonic image corresponding to each two types into the self-attention network corresponding to the two types in the ultrasonic imaging model of the initial structure to obtain the first sample image features; fusing the obtained first sample image features and inputting them into the image reconstruction layer of the ultrasonic imaging model of the initial structure to obtain a predicted ultrasonic image; adjusting the model parameters based on the difference between the predicted ultrasonic image and the real ultrasonic image. In this way, the molten pool area can be effectively detected.

Description

Technical Field

[0001] The present application relates to the field of laser ultrasonic detection technology, and in particular to an ultrasonic imaging model training method, a molten pool detection method, a device and an electronic device. Background Art

[0002] In the process of welding a workpiece, it is often necessary to detect the molten pool area in the welding area on the workpiece to ensure the welding quality. In the related art, a visible light image of the molten pool area is often obtained by an image acquisition device to detect the molten pool area through an image processing algorithm. However, the visible light image is easily affected by environmental factors (such as lighting conditions), and only the surface information of the molten pool area can be extracted based on the visible light image, and the internal information of the molten pool area cannot be obtained. Therefore, it is impossible to effectively detect the molten pool area. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an ultrasonic imaging model training method, a molten pool detection method, device and electronic equipment to effectively detect the molten pool area. The specific technical solution is as follows:

[0004] In a first aspect of an embodiment of the present application, a method for training an ultrasound imaging model is provided, the method comprising:

[0005] Scan each sample detection point on the sample object using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point;

[0006] Decoupling the first sample ultrasonic signal corresponding to each sample detection point respectively to obtain at least two types of sample sub-ultrasonic signals;

[0007] Using sample sub-ultrasound signals of the same type, generating a first sample ultrasonic image corresponding to the type;

[0008] Inputting the first sample ultrasound image corresponding to each type into the feature extraction layer corresponding to the type in the ultrasound imaging model of the initial structure, to obtain the image features of the first sample ultrasound image corresponding to the type;

[0009] Inputting the image features of the first sample ultrasound images corresponding to each two types into the self-attention network corresponding to the two types in the ultrasound imaging model of the initial structure, obtaining the first sample image features;

[0010] The obtained first sample image features are fused to obtain second sample image features, and the second sample image features are input into an image reconstruction layer in an ultrasound imaging model of an initial structure to obtain a predicted ultrasound image;

[0011] Based on the difference between the obtained predicted ultrasound image and the real ultrasound image of the sample object, the model parameters of the ultrasound imaging model of the initial structure are adjusted.

[0012] Optionally, before scanning each sample detection point on the sample object using the laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point, the method further includes:

[0013] The laser transmitter in the laser ultrasonic detection device emits a laser to a sample emission position on the sample object, and the ultrasonic receiver in the laser ultrasonic detection device receives an ultrasonic signal generated by the sample object to obtain a current second sample ultrasonic signal;

[0014] Based on the waveform of the current second sample ultrasonic signal, determining whether the current second sample ultrasonic signal reaches a preset decoupling condition; wherein, when the preset decoupling condition is reached, the waveforms of various types of ultrasonic signals included in the current second sample ultrasonic signal do not overlap;

[0015] In the case where the current second sample ultrasonic signal does not reach the preset decoupling condition, the position of the ultrasonic receiver is adjusted; and the step of returning to execute the step of transmitting a laser to a sample transmitting position on the sample object by the laser transmitter in the laser ultrasonic detection device, and receiving the ultrasonic signal generated by the sample object by the ultrasonic receiver in the laser ultrasonic detection device to obtain the current second sample ultrasonic signal;

[0016] When the current second sample ultrasonic signal reaches the preset decoupling condition, determining the relative position between the current laser transmitter and the ultrasonic receiver as the transmitting and receiving position to be used;

[0017] The method of scanning each sample detection point on the sample object using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point includes:

[0018] According to the transmitting and receiving positions to be used, each sample detection point on the sample object is scanned by using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point.

[0019] Optionally, adjusting the position of the ultrasonic receiver includes:

[0020] The position of the ultrasonic receiver is adjusted within a first specified range; wherein, when the position of the ultrasonic receiver belongs to the first specified range, the time at which each type of ultrasonic signal arrives at the ultrasonic receiver calculated based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current second sample ultrasonic signal, and the thickness of the sample object are different, and the difference between the time at which each two types of ultrasonic signals arrive at the ultrasonic receiver calculated is greater than a first threshold.

[0021] Optionally, adjusting the model parameters of the ultrasound imaging model of the initial structure based on the difference between the obtained predicted ultrasound image and the real ultrasound image of the sample object includes:

[0022] Calculating a first sub-loss value representing a mean square error between the obtained predicted ultrasonic image and the true ultrasonic image of the sample object, and a second sub-loss value representing a structural similarity between the obtained predicted ultrasonic image and the true ultrasonic image of the sample object;

[0023] According to the current weight to be used, a weighted sum of the first sub-loss value and the second sub-loss value is calculated to obtain a loss value;

[0024] The model parameters of the ultrasonic imaging model of the initial structure are adjusted according to the obtained loss value.

[0025] Optionally, the method further includes:

[0026] When the model parameters of the ultrasonic imaging model of the initial structure are adjusted according to the obtained loss value, the current weight to be used is adjusted according to the obtained loss value.

[0027] A second aspect of the embodiment of the present application further provides a molten pool detection method, the method comprising:

[0028] Use laser ultrasonic testing equipment to scan each inspection point to be scanned in the molten pool area to be inspected, and obtain the ultrasonic signal to be processed corresponding to each inspection point to be scanned;

[0029] Decoupling the ultrasonic signal to be processed corresponding to each detection point to be scanned respectively to obtain at least two types of sub-ultrasonic signals to be processed;

[0030] Using the sub-ultrasound signals to be processed of the same type, generating an ultrasonic image to be fused corresponding to the type;

[0031] Based on the self-attention mechanism, the image features of each ultrasonic image to be fused are fused to obtain a fused image feature, and the fused image feature is used to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0032] Optionally, the image features of each ultrasonic image to be fused are fused based on the self-attention mechanism to obtain fused image features, and the fused image features are used to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected, including:

[0033] Based on the self-attention mechanism, image features of each two types of ultrasound images to be fused corresponding to each other are fused to obtain image features to be processed;

[0034] The obtained image features to be processed are fused to obtain fused image features, and the fused image features are used to perform image reconstruction to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0035] Optionally, the image features of each two types of ultrasound images to be fused corresponding to each other are fused based on the self-attention mechanism to obtain the image features to be processed, including:

[0036] Inputting each type of ultrasound image to be fused into the feature extraction layer corresponding to the type in the pre-trained ultrasound imaging model to obtain the image features of the ultrasound image to be fused; wherein the ultrasound imaging model is trained based on any one of the methods described in the first aspect above;

[0037] Inputting the image features of the ultrasonic images to be fused corresponding to each two types into the self-attention network corresponding to the two types in the ultrasonic imaging model to obtain the image features to be processed;

[0038] The obtained image features to be processed are fused to obtain fused image features, and the fused image features are used to reconstruct an image to obtain an ultrasonic image for detecting the molten pool area to be detected, including:

[0039] The obtained image features to be processed are fused to obtain fused image features, and are input into the image reconstruction layer in the ultrasonic imaging model to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0040] Optionally, before scanning each to-be-scanned inspection point in the to-be-scanned molten pool area using the laser ultrasonic detection equipment to obtain an ultrasonic signal to be processed corresponding to each to-be-scanned inspection point, the method further includes:

[0041] The laser transmitter in the laser ultrasonic detection device emits a laser to a designated detection position, and the ultrasonic receiver in the laser ultrasonic detection device receives an ultrasonic signal generated by the molten pool area to be detected, so as to obtain a current ultrasonic signal to be observed;

[0042] Based on the waveform of the current ultrasonic signal to be observed, determining whether the current ultrasonic signal to be observed meets a preset decoupling condition; wherein, when the preset decoupling condition is met, the waveforms of various types of ultrasonic signals included in the current ultrasonic signal to be observed do not overlap;

[0043] When the current ultrasonic signal to be observed does not reach the preset decoupling condition, the position of the ultrasonic receiver is adjusted; and the step of returning to execute the step of emitting a laser to a designated detection position by the laser transmitter in the laser ultrasonic detection device, and receiving the ultrasonic signal generated by the molten pool area to be detected by using the ultrasonic receiver in the laser ultrasonic detection device to obtain the current ultrasonic signal to be observed;

[0044] When the current ultrasonic signal to be observed reaches the preset decoupling condition, determining the current relative position between the laser transmitter and the ultrasonic receiver;

[0045] The laser ultrasonic detection device is used to scan each inspection point to be scanned in the molten pool area to be inspected, and the ultrasonic signal to be processed corresponding to each inspection point to be scanned is obtained, including:

[0046] According to the determined relative position, each inspection point to be scanned in the molten pool area to be inspected is scanned using a laser ultrasonic inspection device to obtain an ultrasonic signal to be processed corresponding to each inspection point to be scanned.

[0047] Optionally, adjusting the position of the ultrasonic receiver includes:

[0048] The position of the ultrasonic receiver is adjusted within a second specified range; wherein, when the position of the ultrasonic receiver belongs to the second specified range, based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current ultrasonic signal to be observed, and the thickness of the object to which the molten pool area to be detected belongs, the calculated arrival time of each type of ultrasonic signal at the ultrasonic receiver is different, and the calculated difference between the arrival time of each two ultrasonic signals at the ultrasonic receiver is greater than a second threshold value.

[0049] Optionally, after performing image reconstruction using the fused image features to obtain an ultrasonic image for detecting the molten pool area to be detected, the method further includes:

[0050] The ultrasonic image used to detect the molten pool area to be detected is input into a pre-trained classification network to obtain a detection result of the molten pool area to be detected; wherein the classification network is trained based on the second sample ultrasonic image and the label of the second sample ultrasonic image; the label of each second sample ultrasonic image is used to represent the defects existing in the second sample ultrasonic image.

[0051] According to a third aspect of the embodiments of the present application, there is also provided an ultrasound imaging model training device, the device comprising:

[0052] A sample object scanning module, used to scan each sample detection point on the sample object using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point;

[0053] A sample ultrasonic signal decoupling module, used to decouple the first sample ultrasonic signal corresponding to each sample detection point to obtain at least two types of sample sub-ultrasonic signals;

[0054] A sample ultrasonic image generating module, used for generating a first sample ultrasonic image corresponding to the same type by using sample sub-ultrasonic signals of the same type;

[0055] A sample feature extraction module, used for inputting the first sample ultrasound image corresponding to each type into the feature extraction layer corresponding to the type in the ultrasound imaging model of the initial structure, to obtain the image features of the first sample ultrasound image corresponding to the type;

[0056] A sample self-attention processing module, used for inputting image features of the first sample ultrasound images corresponding to each two types into the self-attention network corresponding to the two types in the ultrasound imaging model of the initial structure, to obtain first sample image features;

[0057] A sample feature fusion module is used to fuse the obtained first sample image features to obtain second sample image features, and input the second sample image features into an image reconstruction layer in an ultrasound imaging model of an initial structure to obtain a predicted ultrasound image;

[0058] The model parameter adjustment module is used to adjust the model parameters of the ultrasound imaging model of the initial structure based on the difference between the obtained predicted ultrasound image and the real ultrasound image of the sample object.

[0059] Optionally, the device further comprises:

[0060] A second sample ultrasonic signal acquisition module is used to, before scanning each sample detection point on the sample object using the laser ultrasonic detection device to obtain the first sample ultrasonic signal corresponding to each sample detection point, emit a laser to the sample emission position on the sample object through the laser transmitter in the laser ultrasonic detection device, and receive the ultrasonic signal generated by the sample object using the ultrasonic receiver in the laser ultrasonic detection device to obtain the current second sample ultrasonic signal;

[0061] A decoupling condition judgment module, used to determine whether the current second sample ultrasonic signal meets a preset decoupling condition based on the waveform of the current second sample ultrasonic signal; wherein, when the preset decoupling condition is met, the waveforms of various types of ultrasonic signals included in the current second sample ultrasonic signal do not overlap;

[0062] a position adjustment module, configured to adjust the position of the ultrasonic receiver when the current second sample ultrasonic signal does not reach the preset decoupling condition; and return to execute the step of transmitting a laser to a sample transmitting position on a sample object by a laser transmitter in the laser ultrasonic detection device, and receiving the ultrasonic signal generated by the sample object by using the ultrasonic receiver in the laser ultrasonic detection device to obtain the current second sample ultrasonic signal;

[0063] a transceiver position determination module, configured to determine the relative position between the laser transmitter and the ultrasonic receiver as the transceiver position to be used when the current second sample ultrasonic signal reaches the preset decoupling condition;

[0064] The sample object scanning module is specifically used to scan each sample detection point on the sample object using a laser ultrasonic detection device according to the transmitting and receiving position to be used, and obtain a first sample ultrasonic signal corresponding to each sample detection point.

[0065] Optionally, the position adjustment module is specifically used to adjust the position of the ultrasonic receiver within a first specified range; wherein, when the position of the ultrasonic receiver belongs to the first specified range, the time for each type of ultrasonic signal to arrive at the ultrasonic receiver calculated based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current second sample ultrasonic signal, and the thickness of the sample object are different, and the difference between the calculated arrival times of each two ultrasonic signals at the ultrasonic receiver is greater than a first threshold.

[0066] Optionally, the model parameter adjustment module is specifically used to calculate a first sub-loss value representing the mean square error between the obtained predicted ultrasonic image and the actual ultrasonic image of the sample object, and a second sub-loss value representing the structural similarity between the obtained predicted ultrasonic image and the actual ultrasonic image of the sample object; calculate the weighted sum of the first sub-loss value and the second sub-loss value according to the current weights to be used to obtain the loss value; and adjust the model parameters of the ultrasonic imaging model of the initial structure according to the obtained loss values.

[0067] Optionally, the device further comprises:

[0068] The weight adjustment module is used to adjust the current weight to be used according to the obtained loss value when adjusting the model parameters of the ultrasonic imaging model of the initial structure according to the obtained loss value.

[0069] The present application also provides a molten pool detection device, the device comprising:

[0070] The scanning module of the molten pool area to be inspected is used to scan each inspection point to be scanned in the molten pool area to be inspected by using a laser ultrasonic detection device to obtain an ultrasonic signal to be processed corresponding to each inspection point to be scanned;

[0071] The ultrasonic signal decoupling module to be processed is used to decouple the ultrasonic signal to be processed corresponding to each detection point to be scanned and inspected, and obtain at least two types of sub-ultrasonic signals to be processed;

[0072] The module for generating ultrasonic images to be fused is used to generate ultrasonic images to be fused corresponding to the same type by using the sub-ultrasonic signals to be processed of the same type;

[0073] The image feature utilization module is used to fuse the image features of each ultrasonic image to be fused based on the self-attention mechanism to obtain a fused image feature, and use the fused image feature to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0074] Optionally, the image feature utilization module includes:

[0075] An image feature acquisition submodule is used to fuse the image features of each two types of ultrasound images to be fused based on a self-attention mechanism to obtain image features to be processed;

[0076] The image reconstruction submodule is used to fuse the obtained image features to be processed to obtain fused image features, and use the fused image features to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0077] Optionally, the image feature acquisition submodule is specifically used to input each type of ultrasound image to be fused into a feature extraction layer corresponding to the type in a pre-trained ultrasound imaging model to obtain image features of the ultrasound image to be fused; wherein the ultrasound imaging model is trained by any of the methods described in the first aspect above; the image features of every two types of ultrasound images to be fused are input into the self-attention network corresponding to the two types in the ultrasound imaging model to obtain image features to be processed;

[0078] The image reconstruction submodule is specifically used to fuse the obtained image features to be processed to obtain fused image features, and input the fused image features into the image reconstruction layer in the ultrasonic imaging model to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0079] Optionally, the device further comprises:

[0080] The ultrasonic signal acquisition module to be observed is used to, before the laser ultrasonic detection device is used to scan each to-be-scanned detection point in the to-be-scanned molten pool area to obtain the to-be-processed ultrasonic signal corresponding to each to-be-scanned detection point, emit a laser to a designated detection position through a laser transmitter in the laser ultrasonic detection device, and use an ultrasonic receiver in the laser ultrasonic detection device to receive the ultrasonic signal generated by the to-be-scanned molten pool area to obtain the current ultrasonic signal to be observed;

[0081] A decoupling determination module, used to determine whether the current ultrasonic signal to be observed meets a preset decoupling condition based on the waveform of the current ultrasonic signal to be observed; wherein, when the preset decoupling condition is met, the waveforms of various types of ultrasonic signals included in the current ultrasonic signal to be observed do not overlap;

[0082] An ultrasonic receiver adjustment module is used to adjust the position of the ultrasonic receiver when the current ultrasonic signal to be observed does not meet the preset decoupling condition; and return to execute the step of emitting a laser to a designated detection position by the laser transmitter in the laser ultrasonic detection equipment, and receiving the ultrasonic signal generated by the molten pool area to be detected by using the ultrasonic receiver in the laser ultrasonic detection equipment to obtain the current ultrasonic signal to be observed;

[0083] A transceiver position acquisition module, used to determine the relative position between the current laser transmitter and the ultrasonic receiver when the current ultrasonic signal to be observed reaches the preset decoupling condition;

[0084] The scanning module for the molten pool area to be detected is specifically used to scan each inspection point to be scanned in the molten pool area to be detected using a laser ultrasonic detection device according to the determined relative position, and obtain an ultrasonic signal to be processed corresponding to each inspection point to be scanned.

[0085] Optionally, the ultrasonic receiver adjustment module is specifically used to adjust the position of the ultrasonic receiver within a second specified range; wherein, when the position of the ultrasonic receiver belongs to the second specified range, based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current ultrasonic signal to be observed, and the thickness of the object to which the molten pool area to be detected belongs, the calculated arrival time of each type of ultrasonic signal at the ultrasonic receiver is different, and the calculated difference between the arrival time of each two ultrasonic signals at the ultrasonic receiver is greater than a second threshold.

[0086] Optionally, the device further comprises:

[0087] A classification detection module is used to reconstruct an image using the fused image features to obtain an ultrasonic image for detecting the molten pool area to be detected, and then input the ultrasonic image for detecting the molten pool area to be detected into a pre-trained classification network to obtain a detection result of the molten pool area to be detected; wherein the classification network is trained based on the second sample ultrasonic image and the label of the second sample ultrasonic image; and the label of each second sample ultrasonic image is used to represent a defect existing in the second sample ultrasonic image.

[0088] The present application also provides an electronic device, including:

[0089] Memory, used to store computer programs;

[0090] The processor is used to implement any of the above-mentioned ultrasonic imaging model training methods or molten pool detection methods when executing the program stored in the memory.

[0091] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it is any of the ultrasonic imaging model training methods or molten pool detection methods described above.

[0092] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the ultrasonic imaging model training methods or molten pool detection methods described above.

[0093] Beneficial effects of the embodiments of the present application:

[0094] An ultrasonic imaging model training method provided in an embodiment of the present application can use a laser ultrasonic detection device to scan each sample detection point on a sample object to obtain a first sample ultrasonic signal corresponding to each sample detection point; decouple the first sample ultrasonic signal corresponding to each sample detection point to obtain at least two types of sample sub-ultrasonic signals; use the sample sub-ultrasonic signals of the same type to generate a first sample ultrasonic image corresponding to the type; input the first sample ultrasonic image corresponding to each type into the feature extraction layer corresponding to the type in the ultrasonic imaging model of the initial structure to obtain the image features of the first sample ultrasonic image corresponding to the type; input the image features of the first sample ultrasonic images corresponding to each two types into the self-attention network corresponding to the two types in the ultrasonic imaging model of the initial structure to obtain the first sample image features; fuse the obtained first sample image features to obtain second sample image features, and input them into the image reconstruction layer in the ultrasonic imaging model of the initial structure to obtain a predicted ultrasonic image; based on the difference between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object, adjust the model parameters of the ultrasonic imaging model of the initial structure.

[0095] Based on the above processing, the laser ultrasonic detection equipment can be used to scan each sample detection point on the sample object, and the first sample ultrasonic signal corresponding to each sample detection point obtained by the scan can be decoupled to obtain at least two types of ultrasonic signals (i.e., sample sub-ultrasonic signals). Different types of ultrasonic signals can reflect the structural information within different depth ranges inside the sample object. The image features of the ultrasonic image generated based on each type of sample sub-ultrasonic signal are fused to obtain a new image feature (i.e., the second sample image feature). The second sample image feature and the predicted ultrasonic image generated using the second sample image feature can include the structural information within different depth ranges inside the sample object.

[0096] During the training process, the ultrasonic imaging model can learn to extract the structural information within different depth ranges of the sample object and generate an ultrasonic image close to the real ultrasonic image based on the extracted information. In this way, using the trained ultrasonic imaging model, it is also possible to generate an ultrasonic image containing the structural information within the molten pool area within different depth ranges. The generated ultrasonic image can contain relatively rich internal structural information of the molten pool area and can effectively reflect the internal structural information of the molten pool area. Accordingly, based on the generated image, the molten pool area can be effectively detected.

[0097] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0099] Figure 1 A schematic diagram of a first flow chart of the ultrasound imaging model training method provided in an embodiment of the present application;

[0100] Figure 2 A schematic diagram of a waveform of a first sample ultrasound signal in an ultrasound imaging model training method provided in an embodiment of the present application;

[0101] Figure 3 A schematic diagram of a process for obtaining fused image features based on an ultrasound imaging model provided in an embodiment of the present application;

[0102] Figure 4 A second flow chart of the ultrasound imaging model training method provided in an embodiment of the present application;

[0103] Figure 5 A schematic diagram of the relationship between the time for different types of ultrasonic signals to reach an ultrasonic receiver and the transmission and reception distance provided in an embodiment of the present application;

[0104] Figure 6 A schematic diagram of a first process flow of a molten pool detection method provided in an embodiment of the present application;

[0105] Figure 7 A second schematic flow chart of the molten pool detection method provided in an embodiment of the present application;

[0106] Figure 8 A schematic diagram of adjusting the relative position between a laser transmitter and an ultrasonic receiver in a molten pool detection method provided in an embodiment of the present application;

[0107] Fig. 9 A third schematic flow chart of the molten pool detection method provided in an embodiment of the present application;

[0108] Fig.10 A schematic diagram of the logical framework of the molten pool detection method provided in an embodiment of the present application;

[0109] Fig.11 A schematic diagram of the structure of an ultrasound imaging model training device provided in an embodiment of the present application;

[0110] Fig.12 A schematic diagram of the structure of a molten pool detection device provided in an embodiment of the present application;

[0111] Fig.13A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0112] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0113] In the process of welding a workpiece, it is often necessary to detect the molten pool area in the welding area on the workpiece to ensure the welding quality. In the related art, a visible light image of the molten pool area is often obtained by an image acquisition device to detect the molten pool area through an image processing algorithm. However, the visible light image is easily affected by environmental factors (such as lighting conditions), and only the surface information of the molten pool area can be extracted based on the visible light image, and the internal information of the molten pool area cannot be obtained. Therefore, it is impossible to effectively detect the molten pool area.

[0114] In order to effectively detect the molten pool area, the present application embodiment provides an ultrasonic imaging model training method, see Figure 1 , Figure 1 A first flow chart of the ultrasound imaging model training method provided in the embodiment of the present application. The method comprises:

[0115] Step S101: Scan each sample detection point on the sample object using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point.

[0116] Step S102: Decoupling the first sample ultrasonic signal corresponding to each sample detection point respectively to obtain at least two types of sample sub-ultrasonic signals.

[0117] Step S103: using sample sub-ultrasound signals of the same type, generating a first sample ultrasonic image corresponding to the type.

[0118] Step S104: inputting the first sample ultrasound image corresponding to each type into the feature extraction layer corresponding to the type in the ultrasound imaging model of the initial structure to obtain the image features of the first sample ultrasound image corresponding to the type.

[0119] Step S105: inputting the image features of the first sample ultrasound images corresponding to each two types into the self-attention network corresponding to the two types in the ultrasound imaging model of the initial structure to obtain the first sample image features.

[0120] Step S106: Fusing the obtained first sample image features to obtain second sample image features, and inputting the second sample image features into the image reconstruction layer in the ultrasound imaging model of the initial structure to obtain a predicted ultrasound image.

[0121] Step S107: adjusting the model parameters of the ultrasound imaging model of the initial structure based on the difference between the obtained predicted ultrasound image and the real ultrasound image of the sample object.

[0122] Based on the above processing, the laser ultrasonic detection equipment can be used to scan each sample detection point on the sample object, and the first sample ultrasonic signal corresponding to each sample detection point obtained by the scan can be decoupled to obtain at least two types of ultrasonic signals (i.e., sample sub-ultrasonic signals). Different types of ultrasonic signals can reflect the structural information within different depth ranges inside the sample object. The image features of the ultrasonic image generated based on each type of sample sub-ultrasonic signal are fused to obtain a new image feature (i.e., the second sample image feature). The second sample image feature and the predicted ultrasonic image generated using the second sample image feature can include the structural information within different depth ranges inside the sample object.

[0123] During the training process, the ultrasonic imaging model can learn to extract the structural information within different depth ranges of the sample object and generate an ultrasonic image close to the real ultrasonic image based on the extracted information. In this way, using the trained ultrasonic imaging model, it is also possible to generate an ultrasonic image containing the structural information within the molten pool area within different depth ranges. The generated ultrasonic image can contain relatively rich internal structural information of the molten pool area and can effectively reflect the internal structural information of the molten pool area. Accordingly, based on the generated image, the molten pool area can be effectively detected.

[0124] For step S101, the laser ultrasonic detection equipment combines laser technology and ultrasonic technology, and the laser ultrasonic detection equipment may include: a laser transmitter (also referred to as a laser), an ultrasonic receiver (also referred to as a laser vibrometer), a motion controller, a signal acquisition and processing module, and a storage and display module. The laser transmitter can generate a high-energy laser beam and emit the generated laser beam to the surface of the object that currently needs to be scanned. After the object absorbs the laser energy, it will expand and contract instantaneously, thereby generating an ultrasonic signal. The ultrasonic receiver can receive the ultrasonic signal generated by the object and convert the received ultrasonic signal into an electrical signal, and can send the converted electrical signal to the signal acquisition and processing module. The signal acquisition and processing module can amplify, filter, digitize, and process the received electrical signal so as to perform data analysis and processing to obtain the internal structural information of the object. Accordingly, the storage and display module can record and display the internal structural information of the object. The motion controller can be used to scan the object. For example, the motion controller can control the laser ultrasonic detection equipment to scan the object according to a preset scanning trajectory, scanning step, and scanning speed.

[0125] Before using the laser ultrasonic detection equipment for scanning, it is often necessary to configure the relevant parameters of the laser ultrasonic detection equipment. For example, the relevant parameters may include: laser power, transmission repetition rate, transceiver position, scanning trajectory, scanning step and scanning speed, etc. The transceiver position is the position of the laser transmitter and the ultrasonic receiver. The relevant parameters can be configured according to the attribute information of the object to be scanned. For example, the attribute information of the object may include: material composition, mechanical properties, sound velocity, thermal damage threshold, etc. For example, in order to avoid damage to the object by the laser, the power (ie, laser power) of the laser transmitter can be determined according to the material composition and thermal damage threshold of the object; the scanning trajectory can be set according to the area currently required to be scanned on the object; the sound velocity propagated inside the object and the mechanical properties of the object can reflect the frequency of the ultrasonic signal generated by the object. Accordingly, in order to ensure that the ultrasonic receiver can receive the ultrasonic signal generated by the object, the receiving frequency of the ultrasonic receiver can be set according to the sound velocity propagated inside the object and the mechanical properties of the object. Subsequently, the object can be scanned according to the relevant parameters that have been configured.

[0126] The sample object may be a pre-set object with a known internal structure. For example, a plurality of sample objects with different internal structures may be pre-designed. The number of sample objects may be set as required without specific limitation. A plurality of sample detection points may be pre-set on the sample object, and the number of sample detection points may be set as required without specific limitation. For example, 10×10 sample detection points may be evenly set in the area currently to be scanned on the sample object, and the distance between each two adjacent sample detection points may be 1 mm or 2 mm.

[0127] For each sample object, a laser ultrasonic detection device can be used to scan each sample detection point on the sample object to obtain a first sample ultrasonic signal corresponding to each sample detection point. For example, the sample object can be scanned by a laser ultrasonic detection device in an A-scan (point scan) manner, and the ultrasonic signal obtained in the A-scan manner can also be referred to as A-scan ultrasonic data. The first sample ultrasonic signal corresponding to each sample detection point can only reflect the structural information within different depth ranges inside the sample object below the sample detection point, and an effective ultrasonic image cannot be generated based only on the structural information within different depth ranges inside the sample object below one sample detection point. Therefore, in order to ensure that an effective ultrasonic image can be generated based on the ultrasonic signal obtained by the subsequent scan, it is necessary to combine the first sample ultrasonic signals corresponding to multiple sample detection points. For example, a laser ultrasonic detection device can be used to scan a row or a column of sample detection points each time to obtain a first sample ultrasonic signal corresponding to each sample detection point.

[0128] With respect to steps S102 and S103, the ultrasonic signal (also referred to as a broadband ultrasonic signal) generated by the object currently being scanned absorbing the laser energy may include multiple types (also referred to as multi-modal) of ultrasonic signals, such as: surface wave signals, Lamb waves (also referred to as guided waves) signals, longitudinal wave signals, transverse wave signals, surface-grazing longitudinal wave signals, air wave signals, etc. The longitudinal wave signal may be further subdivided into a primary reflected longitudinal wave signal and a secondary reflected longitudinal wave signal. That is, the first sample ultrasonic signal may include multiple types of ultrasonic signals, for example, the first sample ultrasonic signal may include at least two types of ultrasonic signals among the above multiple types of ultrasonic signals.

[0129] Since different types of ultrasonic signals have different propagation paths and propagation speeds, the ultrasonic receiver receives different types of ultrasonic signals at different times. Therefore, different types of ultrasonic signals can be distinguished from the first sample ultrasonic signal. That is, the first sample ultrasonic signal corresponding to each sample detection point can be decoupled to obtain at least two types of sample sub-ultrasonic signals. Figure 2 As shown, Figure 2 A waveform diagram of a first sample ultrasonic signal in an ultrasonic imaging model training method provided in an embodiment of the present application. The first sample ultrasonic signal may include: a surface-grazing longitudinal wave signal, a longitudinal wave signal, a surface wave signal, a shear wave signal, and an air wave signal.

[0130] Before decoupling the first sample ultrasonic signal, the first sample ultrasonic signal may be preprocessed. For example, the preprocessing may include at least one of the following: preamplification, time gain compensation, filtering, Hilbert transform, etc. In this way, the accuracy of the sample sub-ultrasonic signal obtained by decoupling may be further improved.

[0131] However, due to the influence of the material type of the object being scanned and the frequency (also called sampling rate) at which the ultrasonic receiver collects ultrasonic signals, if the distance between the laser transmitter and the ultrasonic receiver is close, different types of ultrasonic signals may overlap in the ultrasonic signal received by the ultrasonic receiver. In this case, it is difficult to distinguish different types of ultrasonic signals from the first sample ultrasonic signal, that is, it is difficult to decouple the first sample ultrasonic signal to obtain different types of ultrasonic signals. Therefore, it is necessary to adjust the relative position of the laser transmitter and the ultrasonic receiver to ensure that different types of ultrasonic signals can be distinguished from the first sample ultrasonic signal. The specific method for adjusting the relative position of the laser transmitter and the ultrasonic receiver can refer to the relevant description of steps S110-step S113 in the subsequent embodiments.

[0132] In one implementation, the sample sub-ultrasonic signal may include at least two types of ultrasonic signals: a surface wave signal, a guided wave signal, a longitudinal wave signal, and a shear wave signal. The type of ultrasonic signal included in the sample sub-ultrasonic signal is related to the attribute information of the sample object. For example, in the case where the thickness of the sample object is large, the sample sub-ultrasonic signal may only include a surface wave signal and a guided wave signal. The types of sample sub-ultrasonic signals of different sample objects may be different. The surface wave signal can effectively reflect the structural information within the depth range close to the upper layer inside the sample object. The guided wave signal can effectively reflect the structural information within the middle depth range inside the sample object, and the longitudinal wave signal or the shear wave signal can effectively reflect the structural information within the depth range close to the bottom layer inside the sample object.

[0133] For each type of sample sub-ultrasonic signal, the sample sub-ultrasonic signal of this type obtained by decoupling the first sample ultrasonic signals corresponding to all sample detection points can be used to generate an ultrasonic image of the sample object, that is, to generate a first sample ultrasonic image corresponding to this type. For example, the first sample ultrasonic image corresponding to this type can be generated according to an ultrasonic signal imaging algorithm, combined with the propagation time and intensity of the sample sub-ultrasonic signal of this type. The pixel value of each pixel position in the ultrasonic image can reflect the strength of the ultrasonic signal generated at the real position represented by the pixel position. In this way, the first sample ultrasonic images corresponding to different types obtained based on different types of sample sub-ultrasonic signals can reflect the structural information within different depth ranges inside the sample object.

[0134] For steps S104 to S106, the ultrasonic imaging model is provided with: a feature extraction layer for extracting features from ultrasonic images obtained using different types of ultrasonic signals, for example, the feature extraction layer may be a convolutional neural network. An ultrasonic image generated using any type of ultrasonic signal may be referred to as an ultrasonic image corresponding to that type. The feature extraction layer corresponding to any type in the ultrasonic imaging model is used to extract features from the ultrasonic image corresponding to that type. Accordingly, for each first sample ultrasonic image corresponding to a type, the first sample ultrasonic image may be input into the feature extraction layer corresponding to that type in the ultrasonic imaging model of the initial structure, so as to extract features from the first sample ultrasonic image corresponding to that type and obtain image features of the first sample ultrasonic image corresponding to that type. For example, the extracted image features may include information such as the structure, edge, and texture of the sample object included in the first sample ultrasonic image corresponding to that type.

[0135] The ultrasonic imaging model may also be provided with: a self-attention network for processing the image features of ultrasonic images corresponding to each two types based on the self-attention mechanism. The self-attention network corresponding to any two types in the ultrasonic imaging model is used to process the image features of the ultrasonic images corresponding to the two types based on the self-attention mechanism. Accordingly, the image features of the first sample ultrasonic images corresponding to each two types can be input into the self-attention network corresponding to the two types in the ultrasonic imaging model of the initial structure to obtain the first sample image features. Based on the attention mechanism, the ultrasonic imaging model can learn the correlation between the feature information in ultrasonic images corresponding to different types and different depth ranges.

[0136] For example, the image features of the first sample ultrasound image can be represented in the form of a feature map, and the feature map of the first sample ultrasound image can be flattened into a two-dimensional matrix. In the self-attention network corresponding to the two types, the dot product results of the two-dimensional matrices of the feature maps of the first sample ultrasound images corresponding to the two types can be calculated, that is, the two-dimensional matrices of the feature maps of the first sample ultrasound images corresponding to the two types are matrix multiplied to obtain an attention score matrix representing the similarity between the first sample ultrasound images corresponding to the two types. The attention score matrix is ​​normalized to generate an attention weight. According to the generated attention weight, the feature maps of the first sample ultrasound images corresponding to the two types are weightedly fused to obtain the first sample image features. For example, the two-dimensional matrices of the feature maps of the first sample ultrasound images corresponding to the two types can be used as Q (Query) and K (Key) of the self-attention network corresponding to the two types, respectively, and the dot product results of Q and K are calculated to obtain the attention score matrix. The attention weight obtained by normalizing the attention score matrix can be used as V (Value) of the self-attention network corresponding to the two types. Q and K are weightedly fused according to V to obtain the first sample image features.

[0137] The obtained first sample image features are fused to obtain the second sample image features. For example, the obtained first sample image features can be spliced ​​to obtain the second sample image features; or the weighted sum of the obtained first sample image features can be calculated to obtain the second sample image features. In this way, the information of different types of ultrasound signals can be fused to generate features with richer information.

[0138] The ultrasound imaging model may also be provided with: an image reconstruction layer for performing image reconstruction based on the input image features to restore the ultrasound image. For example, the image reconstruction layer may be a deconvolution layer or an upsampling layer. The obtained second sample image features are input into the image reconstruction layer in the ultrasound imaging model of the initial structure to obtain a predicted ultrasound image. The predicted ultrasound image may include feature information of different types of ultrasound signals.

[0139] For step S107, there may be multiple sample objects. For each sample object, the predicted ultrasonic image obtained using the sample object corresponds to the real ultrasonic image of the sample object. The real ultrasonic image corresponding to any predicted ultrasonic image is: the real ultrasonic image of the sample object scanned by the laser ultrasonic detection device in the process of obtaining the predicted ultrasonic image. Since the internal structure of the sample object is known, the real ultrasonic image of the sample object can be obtained by drawing the internal structure of the sample object. The model parameters of the ultrasonic imaging model of the initial structure can be adjusted based on the difference between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object. For example, the loss value can be determined based on the difference between the obtained predicted ultrasonic image and the corresponding real ultrasonic image. The model parameters of the ultrasonic imaging model of the initial structure are adjusted according to the obtained loss value. Until the preset convergence condition is reached, the trained ultrasonic imaging model is obtained. The preset convergence condition can be that the calculated loss value is less than the preset loss threshold, or the number of times the model parameters are adjusted reaches the preset number.

[0140] For example, the loss value can be calculated based on the difference between the predicted ultrasonic image obtained by the image reconstruction layer in the ultrasonic imaging model of the initial structure and the real ultrasonic image of the sample object when processing each sample object. In the process of calculating a loss value, the calculation can be performed based on the difference between the predicted ultrasonic image obtained by a sample object and the real ultrasonic image of the sample object. Accordingly, the loss value corresponding to each sample object can be obtained. Furthermore, back propagation can be performed based on the calculated loss value to adjust the model parameters of the ultrasonic imaging model of the initial structure. For example, the model parameters of the ultrasonic imaging model of the initial structure can be adjusted based on the loss values ​​corresponding to a batch of sample objects. For example, a batch can contain a preset number of sample objects, and the preset number can be set as needed without specific limitation.

[0141] In one embodiment, see Figure 3 , Figure 3 A flow chart of obtaining fused image features based on an ultrasonic imaging model provided in an embodiment of the present application. Longitudinal wave signals and transverse wave signals may also be referred to as body wave signals. Depth range 1, depth range 2, and depth range 3 respectively represent the depth ranges to which the structural information that can be effectively reflected by the surface wave signal, the guided wave signal, and the body wave signal belongs. Surface wave imaging means generating an ultrasonic image (which may be referred to as a surface wave image) using a surface wave signal, guided wave imaging means generating an ultrasonic image (which may be referred to as a guided wave image) using a guided wave signal, and body wave imaging means generating an ultrasonic image (which may be referred to as a body wave image) using a body wave signal. Reference may be made to the relevant description of step S103 in the above embodiment.

[0142] After generating an ultrasound image using each type of ultrasound signal, the generated ultrasound image can be input into the feature extraction layer corresponding to the type in the ultrasound imaging model ( Figure 3 (not shown), the image features of the generated ultrasound image are obtained. That is, Figure 3 The characteristic signal 1, characteristic signal 2 and characteristic signal 3 in the embodiment may refer to the relevant description of step S104 in the above embodiment.

[0143] like Figure 3 As shown, the ultrasound imaging model may be provided with: self-attention network 1 and self-attention network 2. Self-attention network 1 is used to: based on the self-attention mechanism, process the image features of the surface wave image (i.e., feature signal 1) and the image features of the guided wave image (i.e., feature signal 2) to obtain a fusion vector 1. Self-attention network 2 is used to: based on the self-attention mechanism, process the image features of the guided wave image (i.e., feature signal 2) and the image features of the body wave image (i.e., feature signal 3) to obtain a fusion vector 2. The ultrasound imaging model may also be provided with a self-attention network 3 ( Figure 3 (not shown), the self-attention network 3 is used to: based on the self-attention mechanism, process the image features of the surface wave image (i.e., feature signal 1) and the image features of the body wave image (i.e., feature signal 3) to obtain a fusion vector 3. Reference may be made to the relevant description of step S105 in the above embodiment.

[0144] The obtained fusion vectors are fused to obtain fused features. Subsequently, the obtained fused features can be input into the image reconstruction layer in the ultrasound imaging model to obtain a new ultrasound image. The obtained new ultrasound image may include feature information of different types of ultrasound signals. For details, please refer to the relevant description of step S106 in the above embodiment.

[0145] In the ultrasound imaging model training method provided in the embodiment of the present application, the loss value can be calculated in the following way:

[0146] In one implementation, the mean-square error (MSE) between the predicted ultrasound image and the real ultrasound image of the sample object may be calculated as the loss value.

[0147] In another implementation, step S107 includes: calculating a first sub-loss value representing the mean square error between the obtained predicted ultrasonic image and the actual ultrasonic image of the sample object, and a second sub-loss value representing the structural similarity between the obtained predicted ultrasonic image and the actual ultrasonic image of the sample object; calculating the weighted sum of the first sub-loss value and the second sub-loss value according to the current weights to be used to obtain a loss value; and adjusting the model parameters of the ultrasonic imaging model of the initial structure according to the obtained loss values.

[0148] In an embodiment of the present application, the mean square error between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object can be calculated to obtain a first sub-loss value. Accordingly, the first sub-loss value can directly reflect the difference between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object. The SSIM (Structural Similarity) between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object is calculated, and based on the obtained structural similarity, the degree of difference between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object is determined to obtain a second sub-loss value. Accordingly, the second sub-loss value can reflect the difference between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object from the size of the structural similarity between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object. According to the current weight to be used, the weighted sum of the obtained first sub-loss value and the second sub-loss value is calculated to obtain the loss value.

[0149] Based on the above processing, the loss value can be determined by combining the mean square error and structural similarity between the predicted ultrasonic image and the real ultrasonic image of the sample object. Compared with the method of determining the loss value based only on the mean square error, the combination of structural similarity can further ensure that the ultrasonic image output by the ultrasonic imaging model is closer to the real ultrasonic image, which can improve the accuracy of ultrasonic imaging of the trained ultrasonic imaging model. At the same time, the efficiency of reaching the preset convergence condition during the training of the ultrasonic imaging model can be improved.

[0150] In one embodiment, the method further comprises: when adjusting the model parameters of the ultrasound imaging model of the initial structure according to the obtained loss value, adjusting the current weight to be used according to the obtained loss value.

[0151] In an embodiment of the present application, at the beginning of training, the initial value of the weight to be used can be set, for example, the initial value of the weight to be used can be randomly generated. During the training process, the current weight to be used can also be adjusted according to the loss value. In this way, the weight used when calculating the loss value with better training effect can be determined, and the accuracy of ultrasonic imaging of the trained ultrasonic imaging model can be further improved. In addition, the current weight to be used can also be adjusted by cross-validation to determine the best weight to be used, that is, to determine the best loss combination.

[0152] In one embodiment, in order to ensure that different types of ultrasonic signals can be distinguished from the first sample ultrasonic signal, the relative positions of the laser transmitter and the ultrasonic receiver can be adjusted. Figure 4 , Figure 4 A second flow chart of the ultrasound imaging model training method provided in the embodiment of the present application. Before step S101, the method further includes:

[0153] Step S108: The laser transmitter in the laser ultrasonic detection device emits a laser to the sample emission position on the sample object, and the ultrasonic receiver in the laser ultrasonic detection device receives the ultrasonic signal generated by the sample object to obtain the current second sample ultrasonic signal.

[0154] Step S109: Based on the waveform of the current second sample ultrasonic signal, determine whether the current second sample ultrasonic signal reaches a preset decoupling condition.

[0155] When the preset decoupling condition is reached, the waveforms of various types of ultrasonic signals included in the current second sample ultrasonic signal do not overlap.

[0156] Step S110: when the current second sample ultrasonic signal does not reach the preset decoupling condition, adjust the position of the ultrasonic receiver; and return to execute step S108.

[0157] Step S111: when the current second sample ultrasonic signal reaches a preset decoupling condition, the relative position between the current laser transmitter and the ultrasonic receiver is determined as the transmitting and receiving position to be used.

[0158] Step S101 includes:

[0159] Step S1011: Scan each sample detection point on the sample object using a laser ultrasonic detection device according to the transmitting and receiving positions to be used, and obtain a first sample ultrasonic signal corresponding to each sample detection point.

[0160] In the embodiment of the present application, any position on the sample object can be selected as the sample emission position. The laser can be emitted to the sample emission position by the laser emitter in the laser ultrasonic detection device, and the ultrasonic receiver in the laser ultrasonic detection device can be used to receive the ultrasonic signal generated by the sample object to obtain the current second sample ultrasonic signal. After obtaining the current second sample ultrasonic signal, it can be determined whether there is a portion with a smaller amplitude fluctuation (which can be called a flat portion) with a length greater than a preset threshold in the current second sample ultrasonic signal. For example, Figure 2 The portion between the surface wave signal and the shear wave signal, and the portion between the shear wave signal and the air wave signal in the waveform shown are both flat portions. The laser ultrasonic detection device can determine whether there is a flat portion with a length greater than a preset threshold in the current second sample ultrasonic signal. Alternatively, the staff responsible for training the ultrasonic imaging model can observe the waveform of the current second sample ultrasonic signal to determine whether there is a flat portion with a length greater than a preset threshold in the current second sample ultrasonic signal.

[0161] If there is a flat portion with a length greater than the preset threshold in the current second sample ultrasonic signal, it means that the time interval between different types of ultrasonic signals on both sides of the flat portion reaching the ultrasonic receiver is relatively large, that is, the two different types of ultrasonic signals do not overlap, and different types of ultrasonic signals can be distinguished from the current second sample ultrasonic signal. Accordingly, it can be determined that the current second sample ultrasonic signal meets the preset decoupling condition. If there is no flat portion with a length greater than the preset threshold in the current second sample ultrasonic signal, it means that there is overlap between different types of ultrasonic signals contained in the current third sample signal, and different types of ultrasonic signals cannot be distinguished from the current second sample ultrasonic signal. Accordingly, it can be determined that the current second sample ultrasonic signal does not meet the preset decoupling condition.

[0162] If the current second sample ultrasonic signal does not reach the preset decoupling condition, it means that the distance between the current laser transmitter and the laser receiver is relatively close, and the different types of ultrasonic signals contained in the current third sample signal will overlap. In this case, different types of ultrasonic signals cannot be distinguished from the current second sample ultrasonic signal. Accordingly, the position of the ultrasonic receiver can be adjusted to adjust the relative position of the laser transmitter and the ultrasonic receiver. For example, the position of the ultrasonic receiver can be adjusted according to a preset distance, such as the preset distance can be 1 mm or 1.5 mm. That is, each time the position of the ultrasonic receiver is adjusted, the preset distance is moved. And the laser is emitted again to the sample emission position on the sample object through the laser transmitter, and the ultrasonic signal generated by the sample object is received by the ultrasonic receiver to obtain a new second sample ultrasonic signal, so as to determine whether the preset decoupling condition is reached according to the new second sample ultrasonic signal.

[0163] If the current second sample ultrasonic signal reaches the preset decoupling condition, it means that the distance between the current laser transmitter and the laser receiver will not cause the different types of ultrasonic signals in the ultrasonic signal received by the ultrasonic receiver to overlap, that is, based on the relative position of the current laser transmitter and the laser receiver, the sample object is scanned, and the different types of ultrasonic signals in the first sample ultrasonic signal obtained can be distinguished. Therefore, the relative position between the current laser transmitter and the ultrasonic receiver can be determined as the corresponding transceiver position of the sample object. Subsequently, the laser ultrasonic detection equipment can be used to scan each sample detection point on the sample object according to the corresponding transceiver position of the sample object to obtain the first sample ultrasonic signal corresponding to each sample detection point. And the obtained first sample ultrasonic signal can be decoupled to obtain a sample sub-ultrasonic signal.

[0164] Based on the above processing, before scanning the sample object, the relative position of the laser transmitter and the ultrasonic receiver can be adjusted, so that each sample detection point on the sample object is scanned according to the adjusted relative position of the laser transmitter and the ultrasonic receiver, and the first sample ultrasonic signal obtained can be decoupled to obtain a sample sub-ultrasonic signal. In this way, it is possible to ensure that different types of ultrasonic signals can be distinguished from the first sample ultrasonic signal, and to ensure that subsequent processing can be performed based on different types of ultrasonic signals. That is, it is guaranteed that the ultrasonic imaging model can be trained.

[0165] In one embodiment, adjusting the position of the ultrasound receiver includes:

[0166] The position of the ultrasonic receiver is adjusted within a first specified range; wherein, when the position of the ultrasonic receiver belongs to the first specified range, the time at which each type of ultrasonic signal arrives at the ultrasonic receiver calculated based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current second sample ultrasonic signal, and the thickness of the sample object are different, and the difference between the time at which each two types of ultrasonic signals arrive at the ultrasonic receiver calculated is greater than a first threshold.

[0167] In the embodiment of the present application, for each type of ultrasonic signal contained in the current second sample ultrasonic signal, the time when the ultrasonic signal of this type reaches the ultrasonic receiver can be calculated based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of the ultrasonic signal of this type, and the thickness of the sample object, that is, the time when the ultrasonic receiver receives the ultrasonic signal of this type. It is also possible to adjust the relative position between the laser transmitter and the ultrasonic receiver (which can be called the transceiver distance) to calculate the time when each type of ultrasonic signal reaches the ultrasonic receiver under different transceiver positions. According to the calculated time, it can be determined that the time when each type of ultrasonic signal reaches the ultrasonic receiver is different, and the difference between the time when each two ultrasonic signals reach the ultrasonic receiver is greater than the first threshold value, for example, the first threshold value can be 0.2us or 0.5us. And the position of the laser transmitter can be fixed, combined with the position of the laser transmitter, the first specified range can also be determined.

[0168] For example, the current second sample ultrasonic signal may include: a surface wave signal, a first reflected longitudinal wave signal, a shear wave signal (which may be a first reflected shear wave signal) and a second reflected longitudinal wave signal. The time it takes for the four types of ultrasonic signals to reach the ultrasonic receiver may be calculated respectively by the following formulas.

[0169] ;

[0170] ;

[0171] ;

[0172] ;

[0173] in, , , , They respectively represent the time when the surface wave signal, the first reflected longitudinal wave signal, the shear wave signal (which can be the first reflected shear wave signal) and the second reflected longitudinal wave signal arrive at the ultrasonic receiver. Indicates the relative position between the laser transmitter and the ultrasonic receiver (i.e., the transmission and reception distance), , , represent the wave velocities of surface wave signals, longitudinal wave signals, and shear wave signals, respectively. Indicates the thickness of the sample object.

[0174] Accordingly, the transceiver distance can be adjusted, and the time for each of the four types of ultrasonic signals to arrive at the ultrasonic receiver under different transceiver distances can be calculated according to the above formula. Furthermore, it can be determined that the time for each type of ultrasonic signal to arrive at the ultrasonic receiver is different, and the transceiver distance when the difference between the time for each two ultrasonic signals to arrive at the ultrasonic receiver is greater than the first threshold value can be determined. Combined with the position of the laser transmitter, the first specified range can also be determined. For example, if the thickness of the sample object is 2mm and the first threshold value is 0.2us, Figure 5 As shown, at different transmitting and receiving distances, the time it takes for the surface wave signal 51, the first reflected longitudinal wave signal 52, the first reflected shear wave signal 53 and the second reflected longitudinal wave signal 54 to arrive at the ultrasonic receiver. Figure 5 When the transmitting-receiving distance is 4.8-6.8 mm, and the difference between the time when each two ultrasonic signals arrive at the ultrasonic receiver is greater than the first threshold, the transmitting-receiving distance is the size of the transmitting-receiving distance. That is, when the position of the ultrasonic receiver belongs to the first specified range, the transmitting-receiving distance is 4.8-6.8 mm.

[0175] When the time for each type of ultrasonic signal to arrive at the ultrasonic receiver is different, the ultrasonic signals of each type in the ultrasonic signal received by the ultrasonic receiver (i.e., the second sample ultrasonic signal) often do not overlap, which ensures that the position of the ultrasonic receiver is adjusted within the specified range, and the probability that the second sample ultrasonic signal reaches the preset decoupling condition is high. In this way, the specified range of position adjustment of the ultrasonic receiver can be obtained through theoretical calculation, and the position of the ultrasonic receiver is adjusted within the specified range, and the probability that the second sample ultrasonic signal reaches the preset decoupling condition is high. It is also possible to improve the efficiency of adjusting the position of the ultrasonic receiver while ensuring that the sample object is scanned according to the adjusted relative position of the laser transmitter and the ultrasonic receiver, and the obtained first sample ultrasonic signal can be decoupled.

[0176] The present application also provides a method for detecting a molten pool. Figure 6 , Figure 6 A first flow chart of a molten pool detection method provided in an embodiment of the present application. The method comprises:

[0177] Step S601: Use laser ultrasonic testing equipment to scan each inspection point in the molten pool area to obtain an ultrasonic signal to be processed corresponding to each inspection point.

[0178] Step S602: Decoupling the ultrasonic signal to be processed corresponding to each detection point to be scanned, respectively, to obtain at least two types of sub-ultrasonic signals to be processed.

[0179] Step S603: using the sub-ultrasound signals to be processed of the same type, generating an ultrasonic image to be fused corresponding to the type.

[0180] Step S604: Based on the self-attention mechanism, the image features of each ultrasonic image to be fused are fused to obtain fused image features, and the fused image features are used to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0181] In the embodiment of the present application, the laser ultrasonic detection equipment can be used to scan each inspection point to be scanned in the molten pool area that needs to be inspected (i.e., the molten pool area to be inspected), and the ultrasonic signal to be processed corresponding to each inspection point to be scanned obtained by the scan can be decoupled to obtain at least two types of ultrasonic signals (i.e., sub-ultrasonic signals to be processed). For each type of sub-ultrasonic signal to be processed, the sub-ultrasonic signal to be processed of the type obtained by decoupling the first sample ultrasonic signal corresponding to all the inspection points to be scanned can be used to generate an ultrasonic image of the molten pool area to be inspected, that is, to generate an ultrasonic image to be fused corresponding to the type.

[0182] Steps S601 to S603 may refer to the relevant descriptions of steps S101 to S103 in the above embodiments, respectively. The type of the sub-ultrasonic signal to be processed obtained by decoupling is related to the attribute information of the object to which the molten pool area to be detected belongs. For example, when the thickness of the object to which the molten pool area to be detected belongs is large, the sub-ultrasonic signal to be processed may only include surface wave signals and guided wave signals. The types of ultrasonic signals included in the obtained sub-ultrasonic signals to be processed may be different depending on the object to which the molten pool area to be detected belongs. The surface wave signal can effectively reflect the structural information within the depth range close to the surface of the molten pool inside the molten pool area. The guided wave signal can effectively reflect the structural information within the middle depth range inside the molten pool area, and the longitudinal wave signal or the transverse wave signal can effectively reflect the structural information within the depth range close to the bottom layer inside the molten pool area.

[0183] Accordingly, the fused image features obtained based on the image features of the ultrasonic image generated by each type of sub-ultrasonic signal to be processed, and the ultrasonic image generated using the fused image features (which can be called the image to be detected), can effectively reflect the structural information within the molten pool area at different depths. That is, the image to be detected can include relatively rich internal structural information of the molten pool area, and can more accurately reflect the information of the molten pool area. In this way, based on the image to be detected, the molten pool area to be detected can be effectively detected.

[0184] In addition, in the related art, the X-ray / industrial CT scanning method, phased array ultrasonic measurement method, etc. can also be used to detect the molten pool area to be inspected. However, the X-ray / industrial CT scanning method requires the use of radiation rays, which is less safe, and the equipment required for this method is expensive, large in size, and has limited usage scenarios. Compared with this method, the molten pool detection method provided in the embodiment of the present application does not require the use of radiation rays, is more safe, and the use cost of the laser ultrasonic detection equipment is not high and the size is also small. The phased array ultrasonic measurement method requires the use of a coupling agent, and the ultrasonic probe in this method is large, and it is impossible to detect the quality of the molten pool on a tiny workpiece. Compared with this method, the molten pool detection method provided in the embodiment of the present application does not require the use of a coupling agent, and the laser ultrasonic detection equipment can scan tiny workpieces.

[0185] It can be seen that the molten pool detection method provided in the embodiment of the present application can utilize the flexibility of the laser ultrasonic measurement equipment and be applied to the detection of the molten pool area under a small field of view (i.e., for tiny workpieces). It solves the problems that the equipment is expensive, there is a risk of radiation exposure, and the inability to place an ultrasonic probe, which causes the inability to measure, etc., which exist in the current detection of the molten pool area. And based on the laser ultrasonic detection equipment, it can stimulate a multi-modal, wide-band ultrasonic signal with quality information of the molten pool area to be detected, decouple the obtained ultrasonic signal, and obtain a comprehensive indicator (i.e., fused image features) after processing for evaluating the quality of the molten pool area to be detected, that is, to detect the quality of the molten pool area to be detected. Compared with the method that only uses a single modality of ultrasonic signals, it can comprehensively, efficiently and accurately evaluate the molten pool area.

[0186] In one embodiment, step S604 includes:

[0187] Step 1: Based on the self-attention mechanism, the image features of each two types of ultrasound images to be fused are fused to obtain the image features to be processed.

[0188] Step 2: Fusing the obtained image features to be processed to obtain fused image features, and using the fused image features to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0189] In an embodiment of the present application, the image features of each two types of ultrasound images to be fused can be fused based on the self-attention mechanism to obtain the image features to be processed. The image features of the ultrasound images to be fused corresponding to different types can reflect the structural information within the different depth ranges within the molten pool area. Based on the attention mechanism, the attention to the depth-related feature information in the image features of each two types of ultrasound images to be fused can be enhanced. And the obtained image features to be processed can be fused to obtain fused image features.

[0190] Based on the above processing, the attention mechanism can be used to enhance the attention to the depth-related feature information in the image features of each two types of ultrasonic images to be fused. Accordingly, the obtained image features to be processed can effectively represent the feature information in different depth ranges. The fused image features obtained by fusing the obtained image features to be processed can also include the feature information in different depth ranges within the molten pool area, that is, it can effectively represent the structural information in different depth ranges within the molten pool area. In this way, the ultrasonic image obtained by subsequent image reconstruction using the fused image features can also effectively reflect the structural information in different depth ranges within the molten pool area. That is, the image to be detected can include relatively rich internal structural information of the molten pool area, which can more accurately reflect the information of the molten pool area. Accordingly, based on the image to be detected, the molten pool area to be detected can be effectively detected.

[0191] In one embodiment, step 1 includes:

[0192] Step 11: Input each type of ultrasound image to be fused corresponding to the type into the feature extraction layer corresponding to the type in the pre-trained ultrasound imaging model to obtain the image features of the ultrasound image to be fused.

[0193] The ultrasound imaging model is obtained by training based on any of the ultrasound imaging model training methods described above.

[0194] Step 12: Input the image features of each two types of ultrasound images to be fused into the self-attention network corresponding to the two types in the ultrasound imaging model to obtain the image features to be processed.

[0195] Step 2 includes:

[0196] Step 21: Fusing the obtained image features to be processed to obtain fused image features, and inputting them into the image reconstruction layer in the ultrasonic imaging model to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0197] In the embodiment of the present application, step 11 may refer to the relevant description of step S104 in the above embodiment, step 12 may refer to the relevant description of step S105 in the above embodiment, and step 21 may refer to the relevant description of step S106 in the above embodiment.

[0198] In one embodiment, see Figure 7 , Figure 7 A second flow chart of the molten pool detection method provided in the embodiment of the present application, before step S601, the method further includes:

[0199] Step S605: The laser transmitter in the laser ultrasonic detection equipment emits a laser to the designated detection position, and the ultrasonic receiver in the laser ultrasonic detection equipment receives the ultrasonic signal generated by the molten pool area to be detected, so as to obtain the current ultrasonic signal to be observed.

[0200] Step S606: Based on the waveform of the current ultrasonic signal to be observed, determine whether the current ultrasonic signal to be observed meets a preset decoupling condition.

[0201] When the preset decoupling condition is reached, the waveforms of various types of ultrasonic signals contained in the current ultrasonic signal to be observed do not overlap.

[0202] Step S607: when the current ultrasonic signal to be observed does not reach the preset decoupling condition, adjust the position of the ultrasonic receiver; and return to execute step S605.

[0203] Step S608: When the current ultrasonic signal to be observed reaches a preset decoupling condition, determine the relative position between the current laser transmitter and the ultrasonic receiver.

[0204] Step S601 includes:

[0205] Step S6011: According to the determined relative position, each inspection point to be scanned in the molten pool area to be inspected is scanned using a laser ultrasonic detection device to obtain an ultrasonic signal to be processed corresponding to each inspection point to be scanned.

[0206] In the embodiment of the present application, step S605 can refer to the relevant description of step S108 in the above embodiment. The designated detection position can be any position on the object to which the molten pool area to be detected belongs. For example, the designated detection position can be located near the molten pool area to be detected. Steps S606-S608 can refer to the relevant description of steps S109-S111 in the above embodiment, respectively. Step S6011 can refer to the relevant description of step S1011 in the above embodiment.

[0207] See also Figure 8 , Figure 8 A schematic diagram of adjusting the relative position between the laser transmitter and the ultrasonic receiver in the molten pool detection method provided in an embodiment of the present application. The laser emission point is the designated detection position, and the laser receiving point in the molten pool area (i.e., the molten pool area to be detected) represents the initial position of the ultrasonic receiver and the projection of the position after each adjustment on the molten pool area. The distance between each laser receiving point can represent the preset distance in the above embodiment.

[0208] In one embodiment, adjusting the position of the ultrasound receiver includes:

[0209] The position of the ultrasonic receiver is adjusted within the second specified range; wherein, when the position of the ultrasonic receiver belongs to the second specified range, based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current ultrasonic signal to be observed, and the thickness of the object to which the molten pool area to be detected belongs, the calculated arrival time of each type of ultrasonic signal at the ultrasonic receiver is different, and the calculated difference between the arrival time of each two ultrasonic signals at the ultrasonic receiver is greater than the second threshold value.

[0210] In the embodiment of the present application, the adjustment of the position of the ultrasonic receiver within the second specified range can refer to the relevant description of the adjustment of the position of the ultrasonic receiver within the first specified range in the above embodiment.

[0211] In one embodiment, after step S604, the method further includes:

[0212] An ultrasonic image used to detect the molten pool area to be detected is input into a pre-trained classification network to obtain a detection result of the molten pool area to be detected; wherein the classification network is trained based on a second sample ultrasonic image and a label of the second sample ultrasonic image; and the label of each second sample ultrasonic image is used to represent a defect existing in the second sample ultrasonic image.

[0213] In an embodiment of the present application, ultrasonic images of multiple molten pool areas can also be acquired in advance as sample ultrasonic images to be processed. After the sample ultrasonic images to be processed are preprocessed by denoising, contrast enhancement, size standardization, etc., the types and locations of defects in the preprocessed ultrasonic images can be annotated to obtain labels for each ultrasonic image. For example, the types of defects may include impurities, cracks, and pores. The preprocessed ultrasonic images can be processed using data enhancement technology to obtain a second sample ultrasonic image. For example, the preprocessed ultrasonic images can be rotated, scaled, flipped, etc. to increase the diversity of the second sample ultrasonic images obtained and improve the generalization ability of the classification network.

[0214] The classification network is trained based on the second sample ultrasound image and the label of the second sample ultrasound image to obtain a trained classification network. For example, the classification network may be CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), Transformer (a deep learning model), etc. The second sample ultrasound image is input into the classification network of the initial structure to obtain the defect location and type predicted by the classification network. Based on the difference between the predicted defect location and type and the label of the second sample ultrasound image, the classification loss value is calculated. According to the obtained classification loss value, the parameters of the classification network of the initial structure are adjusted until the convergence condition is reached to obtain a trained classification network. For example, the classification loss value may be L1 loss or L2 loss.

[0215] The classification network can extract the image features of the input ultrasonic image through a series of convolutional layers, such as the shape, size and defect distribution of the molten pool area. The extracted image features can be reduced in dimension and refined through the pooling layer, and finally classified through the fully connected layer to obtain the type of each defect.

[0216] The trained classification network can be used to detect the ultrasonic image (i.e., the image to be detected) used to detect the molten pool area to be detected, and the detection result of the molten pool area to be detected can be obtained, that is, the position and type of defects contained in the molten pool area to be detected can be obtained. Subsequently, the quality level of the molten pool area to be detected can be determined according to the position and type of defects contained in the molten pool area to be detected. For example, the more defects the molten pool area to be detected contains, the lower the quality level.

[0217] Based on the above processing, the detection of the molten pool area to be detected can be realized through the pre-trained classification network, and the detection result of the molten pool area to be detected can be obtained. And the data enhancement technology can be used to improve the generalization performance of the classification network. That is, the classification network has high robustness, and the parameters of the trained classification network can be migrated to a new model set under different working conditions based on the transfer learning technology to help the training of the new model and improve the efficiency of the new model training.

[0218] In one embodiment, see Fig. 9 , Fig. 9 A third flow chart of the molten pool detection method provided in the embodiment of the present application. The method comprises:

[0219] Step S901: Obtain the molten pool object information, and configure the relevant parameters of the laser ultrasonic detection device according to the molten pool object information. The molten pool object information can also be called the welded object information. That is, before using the laser ultrasonic detection device for scanning, configure the relevant parameters according to the attribute information of the object to be scanned.

[0220] Step S902: According to the set scanning parameters, the initial detection position is obtained, the laser is emitted to the initial detection position of the polymer to be detected by the laser, and the ultrasonic signal with the quality information of the molten pool is collected and stimulated by the laser vibrometer. The polymer to be detected is the object to which the molten pool area to be detected belongs, and the initial detection position is the designated detection position in the above embodiment. That is, step S605 in the above embodiment.

[0221] Step S903: Perform multimodal analysis on the ultrasonic information acquired at the current position and adjust the laser ultrasonic transmitting and receiving action position, that is, step S606 and step S607 in the above embodiment.

[0222] Step S904: Repeat the above step S903 to adjust the laser ultrasound transmitting and receiving action position until the ultrasound information of multiple modes can be decoupled, that is, step S606 to step S608 in the above embodiment.

[0223] Step S905: Scan the entire molten pool area according to the above determined positions to obtain original A-scan ultrasonic data at different positions, that is, step S6011 in the above embodiment.

[0224] Step S906: pre-processing the A-scan ultrasound data obtained by the overall scan, and respectively analyzing the ultrasound information of multiple modes at different positions, that is, step S602 in the above embodiment.

[0225] Step S907: extracting acoustic features of ultrasonic waves of different modes at different positions, that is, step 11 in the above embodiment.

[0226] Step S908: assign different weight coefficients to the acoustic features of different modes according to the detection requirements, and fuse their feature vectors into a comprehensive index for comprehensive evaluation of the molten pool quality. That is, step 12 and step 21 in the above embodiment.

[0227] In one embodiment, see Fig.10 , Fig.10 A schematic diagram of the logical framework of the molten pool detection method provided in an embodiment of the present application. Fig.10 The model in is the ultrasound imaging model in the above embodiment. The method comprises:

[0228] Step S1001: Start.

[0229] Step S1002: feature engineering, that is, labeling the sample ultrasound image.

[0230] Step S1003: Model design, that is, designing the structure of the model.

[0231] Step S1004: loss function. That is, define the loss function used in model training, for example, the loss function may be L1 loss or L2 loss.

[0232] Step S1005: Optimization algorithm. That is, an optimization algorithm used in model training is selected, for example, the optimization algorithm may be Adam (an optimization algorithm with adaptive learning rate) or SGD (Stochastic Gradient Descent) algorithm.

[0233] Step S1006: Model training.

[0234] Step S1007: Model evaluation, that is, using a cross-validation approach to verify the effectiveness of the model.

[0235] Step S1008: Whether the performance requirements are met. That is, determine whether the model meets the performance requirements. If yes, execute step S1009; if no, execute step S1003.

[0236] Step S1009: Model optimization. That is, the trained model is optimized by compression, pruning, etc. to make the model lightweight.

[0237] Step S1010: end-to-end imaging is implemented. That is, in the above embodiment, the trained ultrasonic imaging model is used to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0238] Step S1011: Result analysis. That is, the ultrasonic image used to detect the molten pool area to be detected is input into a pre-trained classification network to obtain the detection result of the molten pool area to be detected.

[0239] Based on the same inventive concept, the present application also provides an ultrasound imaging model training device, see Fig.11 , Fig.11 A schematic diagram of the structure of an ultrasound imaging model training device provided in an embodiment of the present application, the device comprising:

[0240] The sample object scanning module 1101 is used to scan each sample detection point on the sample object using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point;

[0241] A sample ultrasonic signal decoupling module 1102 is used to decouple the first sample ultrasonic signal corresponding to each sample detection point to obtain at least two types of sample sub-ultrasonic signals;

[0242] The sample ultrasonic image generating module 1103 is used to generate a first sample ultrasonic image corresponding to the same type by using sample sub-ultrasonic signals of the same type;

[0243] The sample feature extraction module 1104 is used to input the first sample ultrasound image corresponding to each type into the feature extraction layer corresponding to the type in the ultrasound imaging model of the initial structure, and obtain the image features of the first sample ultrasound image corresponding to the type;

[0244] A sample self-attention processing module 1105 is used to input the image features of the first sample ultrasound images corresponding to each two types into the self-attention network corresponding to the two types in the ultrasound imaging model of the initial structure to obtain the first sample image features;

[0245] The sample feature fusion module 1106 is used to fuse the obtained first sample image features to obtain second sample image features, and input the second sample image features into the image reconstruction layer in the ultrasound imaging model of the initial structure to obtain a predicted ultrasound image;

[0246] The model parameter adjustment module 1107 is used to adjust the model parameters of the ultrasound imaging model of the initial structure based on the difference between the obtained predicted ultrasound image and the real ultrasound image of the sample object.

[0247] Based on the ultrasonic imaging model training device provided in the embodiment of the present application, the laser ultrasonic detection equipment can be used to scan each sample detection point on the sample object, and the first sample ultrasonic signal corresponding to each sample detection point obtained by the scan can be decoupled to obtain at least two types of ultrasonic signals (i.e., sample sub-ultrasonic signals). Different types of ultrasonic signals can reflect the structural information within different depth ranges inside the sample object. The image features of the ultrasonic image generated based on each type of sample sub-ultrasonic signal are fused to obtain a new image feature (i.e., the second sample image feature). The second sample image feature and the predicted ultrasonic image generated using the second sample image feature may include structural information within different depth ranges inside the sample object.

[0248] During the training process, the ultrasonic imaging model can learn to extract the structural information within different depth ranges of the sample object and generate an ultrasonic image close to the real ultrasonic image based on the extracted information. In this way, using the trained ultrasonic imaging model, it is also possible to generate an ultrasonic image containing the structural information within the molten pool area within different depth ranges. The generated ultrasonic image can contain relatively rich internal structural information of the molten pool area and can effectively reflect the internal structural information of the molten pool area. Accordingly, based on the generated image, the molten pool area can be effectively detected.

[0249] Optionally, the device further comprises:

[0250] A second sample ultrasonic signal acquisition module is used to, before scanning each sample detection point on the sample object using the laser ultrasonic detection device to obtain the first sample ultrasonic signal corresponding to each sample detection point, emit a laser to the sample emission position on the sample object through the laser transmitter in the laser ultrasonic detection device, and receive the ultrasonic signal generated by the sample object using the ultrasonic receiver in the laser ultrasonic detection device to obtain the current second sample ultrasonic signal;

[0251] A decoupling condition judgment module, used to determine whether the current second sample ultrasonic signal meets a preset decoupling condition based on the waveform of the current second sample ultrasonic signal; wherein, when the preset decoupling condition is met, the waveforms of various types of ultrasonic signals included in the current second sample ultrasonic signal do not overlap;

[0252] a position adjustment module, configured to adjust the position of the ultrasonic receiver when the current second sample ultrasonic signal does not reach the preset decoupling condition; and return to execute the step of transmitting a laser to a sample transmitting position on a sample object by a laser transmitter in the laser ultrasonic detection device, and receiving the ultrasonic signal generated by the sample object by using the ultrasonic receiver in the laser ultrasonic detection device to obtain the current second sample ultrasonic signal;

[0253] a transceiver position determination module, configured to determine the relative position between the laser transmitter and the ultrasonic receiver as the transceiver position to be used when the current second sample ultrasonic signal reaches the preset decoupling condition;

[0254] The sample object scanning module 1101 is specifically used to scan each sample detection point on the sample object using a laser ultrasonic detection device according to the transmitting and receiving position to be used, and obtain a first sample ultrasonic signal corresponding to each sample detection point.

[0255] Optionally, the position adjustment module is specifically used to adjust the position of the ultrasonic receiver within a first specified range; wherein, when the position of the ultrasonic receiver belongs to the first specified range, the time for each type of ultrasonic signal to arrive at the ultrasonic receiver calculated based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current second sample ultrasonic signal, and the thickness of the sample object are different, and the difference between the calculated arrival times of each two ultrasonic signals at the ultrasonic receiver is greater than a first threshold.

[0256] Optionally, the model parameter adjustment module 1107 is specifically used to calculate a first sub-loss value representing the mean square error between the obtained predicted ultrasonic image and the actual ultrasonic image of the sample object, and a second sub-loss value representing the structural similarity between the obtained predicted ultrasonic image and the actual ultrasonic image of the sample object; calculate the weighted sum of the first sub-loss value and the second sub-loss value according to the current weights to be used to obtain the loss value; and adjust the model parameters of the ultrasonic imaging model of the initial structure according to the obtained loss values.

[0257] Optionally, the device further comprises:

[0258] The weight adjustment module is used to adjust the current weight to be used according to the obtained loss value when adjusting the model parameters of the ultrasonic imaging model of the initial structure according to the obtained loss value.

[0259] The present application also provides a molten pool detection device, see Fig.12 , Fig.12 A schematic diagram of the structure of a molten pool detection device provided in an embodiment of the present application, the device comprising:

[0260] The scanning module 1201 for the molten pool region to be inspected is used to scan each inspection point to be scanned in the molten pool region to be inspected by using a laser ultrasonic inspection device to obtain an ultrasonic signal to be processed corresponding to each inspection point to be scanned;

[0261] The ultrasonic signal decoupling module 1202 is used to decouple the ultrasonic signal to be processed corresponding to each detection point to be scanned, and obtain at least two types of sub-ultrasonic signals to be processed;

[0262] The ultrasonic image generation module 1203 is used to generate the ultrasonic image to be fused corresponding to the same type by using the sub-ultrasonic signals to be processed of the same type;

[0263] The image feature utilization module 1204 is used to fuse the image features of each ultrasonic image to be fused based on the self-attention mechanism to obtain a fused image feature, and use the fused image feature to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0264] Based on the molten pool detection device provided in the embodiment of the present application, the fused image features obtained based on the image features of the ultrasonic image generated by each type of sub-ultrasonic signal to be processed, and the ultrasonic image generated using the fused image features (which can be referred to as the image to be detected), can effectively reflect the structural information within the molten pool area at different depths. That is, the image to be detected can include relatively rich internal structural information of the molten pool area, and can more accurately reflect the information of the molten pool area. In this way, based on the image to be detected, the molten pool area to be detected can be effectively detected.

[0265] Optionally, the image feature utilizing module 1204 includes:

[0266] An image feature acquisition submodule is used to fuse the image features of each two types of ultrasound images to be fused based on a self-attention mechanism to obtain image features to be processed;

[0267] The image reconstruction submodule is used to fuse the obtained image features to be processed to obtain fused image features, and use the fused image features to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0268] Optionally, the image feature acquisition submodule is specifically used to input each type of ultrasound image to be fused into the feature extraction layer corresponding to the type in the pre-trained ultrasound imaging model to obtain the image features of the ultrasound image to be fused; wherein the ultrasound imaging model is trained based on any of the ultrasound imaging model training methods described above; the image features of every two types of ultrasound images to be fused are input into the self-attention network corresponding to the two types in the ultrasound imaging model to obtain the image features to be processed;

[0269] The image reconstruction submodule is specifically used to fuse the obtained image features to be processed to obtain fused image features, and input the fused image features into the image reconstruction layer in the ultrasonic imaging model to obtain an ultrasonic image for detecting the molten pool area to be detected.

[0270] Optionally, the device further comprises:

[0271] The ultrasonic signal acquisition module to be observed is used to, before the laser ultrasonic detection device is used to scan each to-be-scanned detection point in the to-be-scanned molten pool area to obtain the to-be-processed ultrasonic signal corresponding to each to-be-scanned detection point, emit a laser to a designated detection position through a laser transmitter in the laser ultrasonic detection device, and use an ultrasonic receiver in the laser ultrasonic detection device to receive the ultrasonic signal generated by the to-be-scanned molten pool area to obtain the current ultrasonic signal to be observed;

[0272] A decoupling determination module, used to determine whether the current ultrasonic signal to be observed meets a preset decoupling condition based on the waveform of the current ultrasonic signal to be observed; wherein, when the preset decoupling condition is met, the waveforms of various types of ultrasonic signals included in the current ultrasonic signal to be observed do not overlap;

[0273] An ultrasonic receiver adjustment module is used to adjust the position of the ultrasonic receiver when the current ultrasonic signal to be observed does not meet the preset decoupling condition; and return to execute the step of emitting a laser to a designated detection position by the laser transmitter in the laser ultrasonic detection equipment, and receiving the ultrasonic signal generated by the molten pool area to be detected by using the ultrasonic receiver in the laser ultrasonic detection equipment to obtain the current ultrasonic signal to be observed;

[0274] A transceiver position acquisition module, used to determine the relative position between the current laser transmitter and the ultrasonic receiver when the current ultrasonic signal to be observed reaches the preset decoupling condition;

[0275] The module 1201 for scanning the molten pool area to be inspected is specifically used to scan each inspection point to be scanned in the molten pool area to be inspected using a laser ultrasonic inspection device according to the determined relative position, and obtain an ultrasonic signal to be processed corresponding to each inspection point to be scanned.

[0276] Optionally, the ultrasonic receiver adjustment module is specifically used to adjust the position of the ultrasonic receiver within a second specified range; wherein, when the position of the ultrasonic receiver belongs to the second specified range, based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current ultrasonic signal to be observed, and the thickness of the object to which the molten pool area to be detected belongs, the calculated arrival time of each type of ultrasonic signal at the ultrasonic receiver is different, and the calculated difference between the arrival time of each two ultrasonic signals at the ultrasonic receiver is greater than a second threshold.

[0277] Optionally, the device further comprises:

[0278] A classification detection module is used to reconstruct an image using the fused image features to obtain an ultrasonic image for detecting the molten pool area to be detected, and then input the ultrasonic image for detecting the molten pool area to be detected into a pre-trained classification network to obtain a detection result of the molten pool area to be detected; wherein the classification network is trained based on the second sample ultrasonic image and the label of the second sample ultrasonic image; and the label of each second sample ultrasonic image is used to represent a defect existing in the second sample ultrasonic image.

[0279] The present application also provides an electronic device, such as Fig.13 As shown, including:

[0280] Memory 1301, used for storing computer programs;

[0281] The processor 1302 is used to implement any of the above-mentioned ultrasonic imaging model training methods or any of the molten pool detection methods when executing the program stored in the memory 1301.

[0282] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 1302, the communication interface, and the memory 1301 communicate with each other via the communication bus.

[0283] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0284] The communication interface is used for communication between the above electronic device and other devices.

[0285] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0286] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0287] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned ultrasonic imaging model training methods or any of the molten pool detection methods are implemented.

[0288] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any ultrasonic imaging model training method or any molten pool detection method in the above-mentioned embodiments.

[0289] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0290] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0291] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the molten pool detection method, device, electronic device, computer-readable storage medium and computer program product, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0292] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for training an ultrasound imaging model, characterized in that: The method comprises: Scan each sample detection point on the sample object using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point; Decoupling the first sample ultrasonic signal corresponding to each sample detection point respectively to obtain at least two types of sample sub-ultrasonic signals; Using sample sub-ultrasound signals of the same type, generating a first sample ultrasonic image corresponding to the type; Inputting the first sample ultrasound image corresponding to each type into the feature extraction layer corresponding to the type in the ultrasound imaging model of the initial structure, to obtain the image features of the first sample ultrasound image corresponding to the type; Inputting the image features of the first sample ultrasound images corresponding to each two types into the self-attention network corresponding to the two types in the ultrasound imaging model of the initial structure, obtaining the first sample image features; The obtained first sample image features are fused to obtain second sample image features, and the second sample image features are input into an image reconstruction layer in an ultrasound imaging model of an initial structure to obtain a predicted ultrasound image; Based on the difference between the obtained predicted ultrasound image and the real ultrasound image of the sample object, the model parameters of the ultrasound imaging model of the initial structure are adjusted.

2. The method according to claim 1, characterized in that Before scanning each sample detection point on the sample object using the laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point, the method further includes: The laser transmitter in the laser ultrasonic detection device emits a laser to a sample emission position on the sample object, and the ultrasonic receiver in the laser ultrasonic detection device receives an ultrasonic signal generated by the sample object to obtain a current second sample ultrasonic signal; Based on the waveform of the current second sample ultrasonic signal, determining whether the current second sample ultrasonic signal reaches a preset decoupling condition; wherein, when the preset decoupling condition is reached, the waveforms of various types of ultrasonic signals included in the current second sample ultrasonic signal do not overlap; In the case where the current second sample ultrasonic signal does not reach the preset decoupling condition, the position of the ultrasonic receiver is adjusted; and the step of returning to execute the step of transmitting a laser to a sample transmitting position on the sample object by the laser transmitter in the laser ultrasonic detection device, and receiving the ultrasonic signal generated by the sample object by the ultrasonic receiver in the laser ultrasonic detection device to obtain the current second sample ultrasonic signal; When the current second sample ultrasonic signal reaches the preset decoupling condition, determining the relative position between the current laser transmitter and the ultrasonic receiver as the transmitting and receiving position to be used; The method of scanning each sample detection point on the sample object using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point includes: According to the transmitting and receiving positions to be used, each sample detection point on the sample object is scanned by using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point.

3. The method according to claim 2, characterized in that The position of the ultrasonic receiver is adjusted, comprising: The position of the ultrasonic receiver is adjusted within a first specified range; wherein, when the position of the ultrasonic receiver belongs to the first specified range, the time at which each type of ultrasonic signal arrives at the ultrasonic receiver calculated based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current second sample ultrasonic signal, and the thickness of the sample object are different, and the difference between the time at which each two types of ultrasonic signals arrive at the ultrasonic receiver calculated is greater than a first threshold.

4. The method according to claim 1, characterized in that The adjusting of the model parameters of the ultrasound imaging model of the initial structure based on the difference between the obtained predicted ultrasound image and the real ultrasound image of the sample object comprises: Calculating a first sub-loss value representing a mean square error between the obtained predicted ultrasonic image and the true ultrasonic image of the sample object, and a second sub-loss value representing a structural similarity between the obtained predicted ultrasonic image and the true ultrasonic image of the sample object; According to the current weight to be used, a weighted sum of the first sub-loss value and the second sub-loss value is calculated to obtain a loss value; The model parameters of the ultrasonic imaging model of the initial structure are adjusted according to the obtained loss value.

5. The method according to claim 4, characterized in that The method further comprises: When the model parameters of the ultrasonic imaging model of the initial structure are adjusted according to the obtained loss value, the current weight to be used is adjusted according to the obtained loss value.

6. A molten pool detection method, characterized in that: The method comprises: Use laser ultrasonic testing equipment to scan each inspection point to be scanned in the molten pool area to be inspected, and obtain the ultrasonic signal to be processed corresponding to each inspection point to be scanned; Decoupling the ultrasonic signal to be processed corresponding to each detection point to be scanned respectively to obtain at least two types of sub-ultrasonic signals to be processed; Using the sub-ultrasound signals to be processed of the same type, generating an ultrasonic image to be fused corresponding to the type; Based on the self-attention mechanism, the image features of each ultrasonic image to be fused are fused to obtain a fused image feature, and the fused image feature is used to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

7. The method according to claim 6, characterized in that The method of fusing the image features of the ultrasonic images to be fused based on the self-attention mechanism to obtain fused image features, and reconstructing the image using the fused image features to obtain an ultrasonic image for detecting the molten pool area to be detected includes: Based on the self-attention mechanism, image features of each two types of ultrasound images to be fused corresponding to each other are fused to obtain image features to be processed; The obtained image features to be processed are fused to obtain fused image features, and the fused image features are used to perform image reconstruction to obtain an ultrasonic image for detecting the molten pool area to be detected.

8. The method according to claim 7, characterized in that The image features of the to-be-fused ultrasound images corresponding to each two types are fused based on the self-attention mechanism to obtain the image features to be processed, including: Input each type of ultrasound image to be fused into the feature extraction layer corresponding to the type in the pre-trained ultrasound imaging model to obtain the image features of the ultrasound image to be fused; wherein the ultrasound imaging model is trained based on any one of the methods described in claims 1 to 5; Inputting the image features of the ultrasonic images to be fused corresponding to each two types into the self-attention network corresponding to the two types in the ultrasonic imaging model to obtain the image features to be processed; The obtained image features to be processed are fused to obtain fused image features, and the fused image features are used to reconstruct an image to obtain an ultrasonic image for detecting the molten pool area to be detected, including: The obtained image features to be processed are fused to obtain fused image features, and are input into the image reconstruction layer in the ultrasonic imaging model to obtain an ultrasonic image for detecting the molten pool area to be detected.

9. The method according to claim 6, characterized in that Before scanning each to-be-scanned inspection point in the to-be-scanned molten pool area using the laser ultrasonic inspection device to obtain the to-be-processed ultrasonic signal corresponding to each to-be-scanned inspection point, the method further includes: The laser transmitter in the laser ultrasonic detection device emits a laser to a designated detection position, and the ultrasonic receiver in the laser ultrasonic detection device receives an ultrasonic signal generated by the molten pool area to be detected, so as to obtain a current ultrasonic signal to be observed; Based on the waveform of the current ultrasonic signal to be observed, determining whether the current ultrasonic signal to be observed meets a preset decoupling condition; wherein, when the preset decoupling condition is met, the waveforms of various types of ultrasonic signals included in the current ultrasonic signal to be observed do not overlap; When the current ultrasonic signal to be observed does not reach the preset decoupling condition, the position of the ultrasonic receiver is adjusted; and the step of returning to execute the step of emitting a laser to a designated detection position by the laser transmitter in the laser ultrasonic detection device, and receiving the ultrasonic signal generated by the molten pool area to be detected by using the ultrasonic receiver in the laser ultrasonic detection device to obtain the current ultrasonic signal to be observed; When the current ultrasonic signal to be observed reaches the preset decoupling condition, determining the current relative position between the laser transmitter and the ultrasonic receiver; The laser ultrasonic detection device is used to scan each inspection point to be scanned in the molten pool area to be inspected, and the ultrasonic signal to be processed corresponding to each inspection point to be scanned is obtained, including: According to the determined relative position, each inspection point to be scanned in the molten pool area to be inspected is scanned using a laser ultrasonic inspection device to obtain an ultrasonic signal to be processed corresponding to each inspection point to be scanned.

10. The method according to claim 9, characterized in that The position of the ultrasonic receiver is adjusted, comprising: The position of the ultrasonic receiver is adjusted within a second specified range; wherein, when the position of the ultrasonic receiver belongs to the second specified range, based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current ultrasonic signal to be observed, and the thickness of the object to which the molten pool area to be detected belongs, the calculated arrival time of each type of ultrasonic signal at the ultrasonic receiver is different, and the calculated difference between the arrival time of each two ultrasonic signals at the ultrasonic receiver is greater than a second threshold value.

11. The method according to claim 6, characterized in that After performing image reconstruction using the fused image features to obtain an ultrasonic image for detecting the molten pool area to be detected, the method further includes: The ultrasonic image used to detect the molten pool area to be detected is input into a pre-trained classification network to obtain a detection result of the molten pool area to be detected; wherein the classification network is trained based on the second sample ultrasonic image and the label of the second sample ultrasonic image; the label of each second sample ultrasonic image is used to represent the defects existing in the second sample ultrasonic image.

12. An ultrasonic imaging model training device, characterized in that: The device comprises: A sample object scanning module, used to scan each sample detection point on the sample object using a laser ultrasonic detection device to obtain a first sample ultrasonic signal corresponding to each sample detection point; A sample ultrasonic signal decoupling module, used to decouple the first sample ultrasonic signal corresponding to each sample detection point to obtain at least two types of sample sub-ultrasonic signals; A sample ultrasonic image generating module, used for generating a first sample ultrasonic image corresponding to the same type by using sample sub-ultrasonic signals of the same type; A sample feature extraction module, used for inputting the first sample ultrasound image corresponding to each type into the feature extraction layer corresponding to the type in the ultrasound imaging model of the initial structure, to obtain the image features of the first sample ultrasound image corresponding to the type; A sample self-attention processing module, used for inputting image features of the first sample ultrasound images corresponding to each two types into the self-attention network corresponding to the two types in the ultrasound imaging model of the initial structure, to obtain first sample image features; A sample feature fusion module is used to fuse the obtained first sample image features to obtain second sample image features, and input the second sample image features into an image reconstruction layer in an ultrasonic imaging model of an initial structure to obtain a predicted ultrasonic image; The model parameter adjustment module is used to adjust the model parameters of the ultrasound imaging model of the initial structure based on the difference between the obtained predicted ultrasound image and the real ultrasound image of the sample object.

13. The device according to claim 12, characterized in that The device also includes: A second sample ultrasonic signal acquisition module is used to, before scanning each sample detection point on the sample object using the laser ultrasonic detection device to obtain the first sample ultrasonic signal corresponding to each sample detection point, emit a laser to the sample emission position on the sample object through the laser transmitter in the laser ultrasonic detection device, and receive the ultrasonic signal generated by the sample object using the ultrasonic receiver in the laser ultrasonic detection device to obtain the current second sample ultrasonic signal; A decoupling condition judgment module, used to determine whether the current second sample ultrasonic signal meets a preset decoupling condition based on the waveform of the current second sample ultrasonic signal; wherein, when the preset decoupling condition is met, the waveforms of various types of ultrasonic signals included in the current second sample ultrasonic signal do not overlap; a position adjustment module, configured to adjust the position of the ultrasonic receiver when the current second sample ultrasonic signal does not reach the preset decoupling condition; and return to execute the step of transmitting a laser to a sample transmitting position on a sample object by a laser transmitter in the laser ultrasonic detection device, and receiving an ultrasonic signal generated by the sample object by an ultrasonic receiver in the laser ultrasonic detection device to obtain a current second sample ultrasonic signal; a transceiver position determination module, configured to determine the relative position between the laser transmitter and the ultrasonic receiver as the transceiver position to be used when the current second sample ultrasonic signal reaches the preset decoupling condition; The sample object scanning module is specifically used to scan each sample detection point on the sample object using a laser ultrasonic detection device according to the transmitting and receiving position to be used, and obtain a first sample ultrasonic signal corresponding to each sample detection point; and / or, The position adjustment module is specifically used to adjust the position of the ultrasonic receiver within a first specified range; wherein, when the position of the ultrasonic receiver belongs to the first specified range, the time at which each type of ultrasonic signal arrives at the ultrasonic receiver calculated based on the relative position between the laser transmitter and the ultrasonic receiver, the wave speed of each type of ultrasonic signal contained in the current second sample ultrasonic signal, and the thickness of the sample object are different, and the calculated difference between the time at which each two types of ultrasonic signals arrive at the ultrasonic receiver is greater than a first threshold; and / or, The model parameter adjustment module is specifically used to calculate a first sub-loss value representing a mean square error between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object, and a second sub-loss value representing a structural similarity between the obtained predicted ultrasonic image and the real ultrasonic image of the sample object; calculate a weighted sum of the obtained first sub-loss value and the second sub-loss value according to the current weight to be used to obtain a loss value; and adjust the model parameters of the ultrasonic imaging model of the initial structure according to the obtained loss value; and / or, The device also includes: The weight adjustment module is used to adjust the current weight to be used according to the obtained loss value when adjusting the model parameters of the ultrasonic imaging model of the initial structure according to the obtained loss value.

14. A molten pool detection device, characterized in that: The device comprises: The scanning module of the molten pool area to be inspected is used to scan each inspection point to be scanned in the molten pool area to be inspected by using a laser ultrasonic detection device to obtain an ultrasonic signal to be processed corresponding to each inspection point to be scanned; The ultrasonic signal decoupling module to be processed is used to decouple the ultrasonic signal to be processed corresponding to each detection point to be scanned, and obtain at least two types of sub-ultrasonic signals to be processed; The module for generating ultrasonic images to be fused is used to generate ultrasonic images to be fused corresponding to the same type by using the sub-ultrasonic signals to be processed of the same type; The image feature utilization module is used to fuse the image features of each ultrasonic image to be fused based on the self-attention mechanism to obtain a fused image feature, and use the fused image feature to reconstruct the image to obtain an ultrasonic image for detecting the molten pool area to be detected.

15. The device according to claim 14, characterized in that The image feature utilization module comprises: An image feature acquisition submodule is used to fuse the image features of each two types of ultrasound images to be fused based on a self-attention mechanism to obtain image features to be processed; An image reconstruction submodule is used to fuse the obtained image features to be processed to obtain fused image features, and use the fused image features to perform image reconstruction to obtain an ultrasonic image for detecting the molten pool area to be detected; and / or, The image feature acquisition submodule is specifically used to input each type of ultrasound image to be fused into the feature extraction layer corresponding to the type in the pre-trained ultrasound imaging model to obtain the image features of the ultrasound image to be fused; wherein the ultrasound imaging model is trained based on the method described in any one of claims 1 to 5; the image features of each two types of ultrasound images to be fused are input into the self-attention network corresponding to the two types in the ultrasound imaging model to obtain the image features to be processed; The image reconstruction submodule is specifically used to fuse the obtained image features to be processed to obtain fused image features, and input the fused image features into the image reconstruction layer in the ultrasonic imaging model to obtain an ultrasonic image for detecting the molten pool area to be detected; and / or, The device also includes: The ultrasonic signal acquisition module to be observed is used to, before the laser ultrasonic detection device is used to scan each to-be-scanned detection point in the to-be-scanned molten pool area to obtain the to-be-processed ultrasonic signal corresponding to each to-be-scanned detection point, emit a laser to a designated detection position through a laser transmitter in the laser ultrasonic detection device, and use an ultrasonic receiver in the laser ultrasonic detection device to receive the ultrasonic signal generated by the to-be-scanned molten pool area to obtain the current ultrasonic signal to be observed; A decoupling determination module, used to determine whether the current ultrasonic signal to be observed meets a preset decoupling condition based on the waveform of the current ultrasonic signal to be observed; wherein, when the preset decoupling condition is met, the waveforms of various types of ultrasonic signals included in the current ultrasonic signal to be observed do not overlap; An ultrasonic receiver adjustment module is used to adjust the position of the ultrasonic receiver when the current ultrasonic signal to be observed does not meet the preset decoupling condition; and return to execute the step of emitting a laser to a designated detection position by the laser transmitter in the laser ultrasonic detection equipment, and receiving the ultrasonic signal generated by the molten pool area to be detected by using the ultrasonic receiver in the laser ultrasonic detection equipment to obtain the current ultrasonic signal to be observed; A transceiver position acquisition module, used to determine the relative position between the current laser transmitter and the ultrasonic receiver when the current ultrasonic signal to be observed reaches the preset decoupling condition; The scanning module for the molten pool area to be detected is specifically used to scan each detection point to be scanned in the molten pool area to be detected using a laser ultrasonic detection device according to the determined relative position, and obtain the ultrasonic signal to be processed corresponding to each detection point to be scanned; and / or, The ultrasonic receiver adjustment module is specifically used to adjust the position of the ultrasonic receiver within a second specified range; wherein, when the position of the ultrasonic receiver belongs to the second specified range, based on the relative position between the laser transmitter and the ultrasonic receiver, the wave velocity of each type of ultrasonic signal contained in the current ultrasonic signal to be observed, and the thickness of the object to which the molten pool area to be detected belongs, the calculated arrival time of each type of ultrasonic signal at the ultrasonic receiver is different, and the calculated difference between the arrival time of each two ultrasonic signals at the ultrasonic receiver is greater than a second threshold value; and / or, The device also includes: A classification detection module is used to reconstruct an image using the fused image features to obtain an ultrasonic image for detecting the molten pool area to be detected, and then input the ultrasonic image for detecting the molten pool area to be detected into a pre-trained classification network to obtain a detection result of the molten pool area to be detected; wherein the classification network is trained based on the second sample ultrasonic image and the label of the second sample ultrasonic image; and the label of each second sample ultrasonic image is used to represent a defect existing in the second sample ultrasonic image.

16. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method described in any one of claims 1-5 or any one of claims 6-11 when executing a program stored in a memory.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 5 or any one of claims 6 to 11 is implemented.

18. A computer program product, characterized in that When the computer program product runs on a computer, the computer is enabled to execute the method according to any one of claims 1 to 5 or any one of claims 6 to 11.

Images