Ultrasonic data processing method, system and electronic equipment

Abstract

The present invention provides an ultrasonic data processing method, system and electronic equipment, which relate to the field of ultrasonic data processing. The method can construct a corresponding independent encoder for ultrasonic signal data under each mode, accurately obtain the latent space representation result of the ultrasonic data by extracting features of ultrasonic signal data of different modes containing distortion or missing, and use a shared decoder corresponding to the independent encoder to complement the distortion and missing of the ultrasonic signal data, thereby realizing accurate complement processing of ultrasonic data in abnormal state.

Description

Technical Field

[0001] The present invention relates to the field of ultrasonic data processing, and in particular to an ultrasonic data processing method, system and electronic equipment. Background Art

[0002] Ultrasonic technology is widely used in medical diagnosis, nondestructive testing, underwater detection and other fields. In the field of industrial testing, ultrasonic data can be used to measure the surface density of materials without destruction. However, in practical applications, due to the influence of factors such as temperature changes and electromagnetic interference, ultrasonic signals will be disturbed by the above environmental factors during the generation and use process, resulting in inaccurate or even invalid measurement results. In addition, in some scenarios with hardware limitations, sampling loss and other situations may occur, affecting the integrity of the signal. The existing technology mainly uses linear filtering, nonlinear filtering, interpolation and other means to complete ultrasonic signals, but the effect is limited when processing complex and changeable ultrasonic signals. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide an ultrasonic data processing method, system and electronic device, which can construct a corresponding independent encoder for ultrasonic signal data under each mode, accurately obtain the latent space representation result of the ultrasonic data after feature extraction of ultrasonic signal data of different modes containing distortion or missing, and use the shared decoder corresponding to the independent encoder to complement the distortion and missing of the ultrasonic signal data, thereby realizing accurate completion processing of ultrasonic data in abnormal state.

[0004] In a first aspect, an embodiment of the present invention provides an ultrasonic data processing method, the method comprising:

[0005] Acquire pre-processed ultrasonic data; wherein the pre-processed ultrasonic data includes distorted or missing multi-modal ultrasonic signal data;

[0006] Determine an independent encoder corresponding to the ultrasonic signal data under each mode, and use the independent encoder to perform encoding calculation on the pre-processed ultrasonic data to generate feature representation data corresponding to the pre-processed ultrasonic data under different modes;

[0007] Determine probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and use the probability distribution data to determine a latent space representation result corresponding to the feature representation data;

[0008] A shared decoder corresponding to a latent space representation result is determined based on an independent encoder, and the shared decoder is used to convert the latent space representation result into distortion-free ultrasonic data.

[0009] Optionally, obtain pre-processed ultrasonic data, including:

[0010] Obtain distortion-free ultrasonic signals in each mode;

[0011] Generate noise data corresponding to the undistorted ultrasonic signal according to a preset random strategy; wherein the noise data at least includes signal data under the missing mode and the singular peak mode;

[0012] After the noise data is superimposed on the distortion-free ultrasonic signal, the preprocessed ultrasonic data is obtained.

[0013] Optionally, determining an independent encoder corresponding to ultrasonic signal data in each mode includes:

[0014] Acquire first ultrasonic signal data corresponding to the missing mode, and determine a first independent encoder of the LSTM structure based on the first ultrasonic signal data;

[0015] Second ultrasonic signal data corresponding to the singular peak mode is obtained, and a second independent encoder of the Transformer structure is determined based on the second ultrasonic signal data.

[0016] Optionally, after encoding and calculating the pre-processed ultrasonic data using an independent encoder, feature representation data corresponding to the pre-processed ultrasonic data in different modes is generated, including:

[0017] After controlling the first independent encoder to receive the first ultrasonic signal data, the encoding layer of the LSTM structure in the first independent encoder is used to perform encoding calculation on the first ultrasonic signal data to obtain a long-range dependency relationship result corresponding to a missing mode in the first ultrasonic signal data;

[0018] After controlling the second independent encoder to receive the second ultrasonic signal data, the encoding layer of the Transformer structure in the second independent encoder is used to perform multi-head self-attention mechanism encoding calculation on the second ultrasonic signal data to obtain context information results corresponding to the singular peak mode in the second ultrasonic signal data;

[0019] The long-range dependency results and context information results are used to determine the feature representation data.

[0020] Optionally, determining probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and determining a latent space representation result corresponding to the feature representation data using the probability distribution data, includes:

[0021] Determine the latent space shared by the independent encoders and map the feature representation data to the latent space;

[0022] After calculating the mean and variance of the feature representation data of different modes, the probability distribution data corresponding to the feature representation data is determined using the mean and variance;

[0023] The feature representation result of the latent space is determined by the probability distribution data, and the feature representation results under different modes are spliced ​​to obtain the latent space representation result.

[0024] Optionally, the feature representation result of the latent space is determined by the probability distribution data, and the feature representation results under different modes are concatenated to obtain the latent space representation result, including:

[0025] The KL divergence value of the feature representation data is calculated using the probability distribution data, and the KL divergence value is determined as the first feature representation result of the latent space;

[0026] Calculate the regularization result and reconstruction loss result of the feature representation data based on the feature representation data under the missing mode and the singular peak mode, and determine the regularization result and the reconstruction loss result as the second feature representation result and the third feature representation result respectively;

[0027] The first feature representation result, the second feature representation result and the third feature representation result are concatenated to obtain the latent space representation result.

[0028] Optionally, the first feature representation result, the second feature representation result, and the third feature representation result are concatenated to obtain a latent space representation result, including:

[0029] Determine, based on the third feature representation result, a first weight value corresponding to the first feature representation result and a second weight value corresponding to the second feature representation result;

[0030] The first feature representation result is updated by multiplying the KL divergence value by the first weight value, and the second feature representation result is updated by multiplying the regularization result by the second weight value;

[0031] The updated first feature representation result, the updated second feature representation result and the third feature representation result are accumulated to obtain a latent space representation result.

[0032] Optionally, determining a shared decoder corresponding to the latent space representation result based on the independent encoder, and converting the latent space representation result into distortion-free ultrasonic data using the shared decoder, including:

[0033] A shared decoder of LSTM structure and its corresponding reconstruction strategy are constructed based on the latent space shared by independent encoders;

[0034] The shared decoder is used to calculate the temporal feature results of the latent space representation results, and the reconstruction strategy is used to reconstruct the temporal feature results to obtain distortion-free ultrasonic data.

[0035] In a second aspect, the present invention provides an ultrasonic data processing system, the system comprising:

[0036] A pre-processed ultrasonic data acquisition module is used to acquire pre-processed ultrasonic data; wherein the pre-processed ultrasonic data includes distorted or missing multi-modal ultrasonic signal data;

[0037] A feature representation data generation module is used to determine the independent encoder corresponding to the ultrasonic signal data under each mode, and after encoding and calculating the pre-processed ultrasonic data using the independent encoder, generate feature representation data corresponding to the pre-processed ultrasonic data under different modes;

[0038] A latent space representation result determination module is used to determine probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and to determine a latent space representation result corresponding to the feature representation data using the probability distribution data;

[0039] The ultrasonic data conversion module is used to determine the shared decoder corresponding to the latent space representation result based on the independent encoder, and use the shared decoder to convert the latent space representation result into distortion-free ultrasonic data.

[0040] In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the ultrasonic data processing method provided in the first aspect.

[0041] In a fourth aspect, an embodiment of the present invention further provides a storage medium storing computer executable instructions. When the computer executable instructions are called and executed by a processor, the computer executable instructions prompt the processor to implement the steps of the ultrasonic data processing method provided in the first aspect.

[0042] An ultrasonic data processing method, system and electronic device provided by an embodiment of the present invention, in the process of completing ultrasonic data, the method first obtains pre-processed ultrasonic data; wherein the pre-processed ultrasonic data contains distorted or missing multi-modal ultrasonic signal data; then determines the independent encoder corresponding to the ultrasonic signal data in each mode, and uses the independent encoder to encode and calculate the pre-processed ultrasonic data, and generates the feature representation data corresponding to the pre-processed ultrasonic data in different modes; then determines the probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and uses the probability distribution data to determine the latent space representation result corresponding to the feature representation data; finally, determines the shared decoder corresponding to the latent space representation result based on the independent encoder, and uses the shared decoder to convert the latent space representation result into distortion-free ultrasonic data. The method can construct a corresponding independent encoder for ultrasonic signal data in each mode, accurately obtain the latent space representation result of ultrasonic data by extracting features of ultrasonic signal data of different modes containing distortion or missing, and use the shared decoder corresponding to the independent encoder to complete the distortion and missing of ultrasonic signal data, thereby realizing accurate completion processing of ultrasonic data in abnormal state.

[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A flowchart of an ultrasonic data processing method provided by an embodiment of the present invention;

[0047] Figure 2 A flowchart of step S101 in an ultrasonic data processing method provided by an embodiment of the present invention;

[0048] Figure 3A flowchart of determining an independent encoder corresponding to ultrasonic signal data in each mode in an ultrasonic data processing method provided by an embodiment of the present invention;

[0049] Figure 4 In an ultrasonic data processing method provided by an embodiment of the present invention, after encoding and calculating the pre-processed ultrasonic data using an independent encoder, a flow chart of feature representation data corresponding to the pre-processed ultrasonic data in different modes is generated;

[0050] Figure 5 A flowchart of step S103 in an ultrasonic data processing method provided by an embodiment of the present invention;

[0051] Figure 6 A flowchart of step S503 in an ultrasonic data processing method provided by an embodiment of the present invention;

[0052] Figure 7 A flowchart of step S603 in an ultrasonic data processing method provided by an embodiment of the present invention;

[0053] Figure 8 A flowchart of step S104 in an ultrasonic data processing method provided by an embodiment of the present invention;

[0054] Fig. 9 A schematic diagram of an ultrasonic data processing system provided by an embodiment of the present invention;

[0055] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0056] icon:

[0057] 910-preprocessing ultrasonic data acquisition module; 920-feature representation data generation module; 930-latent space representation result determination module; 940-ultrasonic data conversion module;

[0058] 101 - processor; 102 - memory; 103 - bus; 104 - communication interface. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Ultrasonic technology is widely used in medical diagnosis, nondestructive testing, underwater detection and other fields. In the field of industrial testing, ultrasonic data can be used to measure the surface density of materials without destructive means. However, in practical applications, due to the influence of factors such as temperature changes and electromagnetic interference, ultrasonic signals are affected by these environmental factors during generation and use, resulting in inaccurate or even invalid measurement results. In addition, in some scenarios with hardware limitations, sampling loss may occur, affecting the integrity of the signal.

[0061] In the prior art, the strategies for completing ultrasonic signals mainly use linear filtering, nonlinear filtering, interpolation and other means, but the effect is limited when processing complex and changeable ultrasonic signals. For example, the working environment of an ultrasonic thickness gauge is relatively complex, and the reliability and accuracy of the compensation process of its ultrasonic data are relatively poor. Based on this, the present invention implements a method, system and electronic device for ultrasonic data processing, which can construct a corresponding independent encoder for ultrasonic signal data under each mode, accurately obtain the latent space representation result of ultrasonic data after feature extraction of ultrasonic signal data of different modes containing distortion or missing, and use a shared decoder corresponding to the independent encoder to complete the distortion and missing of ultrasonic signal data, thereby realizing accurate completion processing of ultrasonic data in abnormal state.

[0062] To facilitate understanding of this embodiment, firstly, an ultrasonic data processing method disclosed in an embodiment of the present invention is described in detail. Figure 1 As shown, including:

[0063] Step S101, obtaining pre-processed ultrasonic data; wherein the pre-processed ultrasonic data includes distorted or missing multi-modal ultrasonic signal data;

[0064] Step S102, determining an independent encoder corresponding to the ultrasonic signal data in each mode, and using the independent encoder to perform encoding calculation on the pre-processed ultrasonic data to generate feature representation data corresponding to the pre-processed ultrasonic data in different modes;

[0065] Step S103, determining probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and determining a latent space representation result corresponding to the feature representation data using the probability distribution data;

[0066] Step S104: determining a shared decoder corresponding to the latent space representation result based on the independent encoder, and converting the latent space representation result into distortion-free ultrasonic data using the shared decoder.

[0067] Specifically, the preprocessed ultrasonic data in the method contains distorted or missing multi-modal ultrasonic signal data, which needs to be processed accordingly to obtain distortion-free ultrasonic data. The preprocessed ultrasonic data contains multi-modal ultrasonic signal data. By determining the independent encoder corresponding to the ultrasonic signal data in each mode, an independent encoder is designed for each mode of ultrasonic signal, and then these independent encoders are used to encode and calculate the preprocessed ultrasonic data, so as to obtain the feature representation data corresponding to the preprocessed ultrasonic data in different modes.

[0068] After the feature representation data is acquired, the mean and variance of the feature representation data corresponding to the preprocessed ultrasonic data in different modes are used to calculate the mean and variance, and the probability distribution data of the feature representation data are determined according to the calculation results of the mean and variance. Then, the feature representation of the latent space is determined by the probability distribution data, and the feature representation of the latent space under different modes is spliced ​​and fused to generate a shared latent space representation result.

[0069] Then, a shared decoder corresponding to the latent space representation result is set based on the independent encoder, and finally the latent space representation result is converted into normal undistorted ultrasonic data using the shared decoder. Optionally, step S101 of obtaining pre-processed ultrasonic data, such as Figure 2 As shown, including:

[0070] Step S201, obtaining a distortion-free ultrasonic signal in each mode;

[0071] Step S202, generating noise data corresponding to the distortion-free ultrasonic signal according to a preset random strategy; wherein the noise data at least includes signal data under a missing mode and a singular peak mode;

[0072] Step S203, after superimposing the noise data onto the distortion-free ultrasonic signal, obtain pre-processed ultrasonic data.

[0073] The acquisition process of preprocessing ultrasonic data can obtain normal, distortion-free ultrasonic data under ideal conditions, generate multimodal ultrasonic data containing distortion and missing values ​​by randomly adding noise or randomly deleting, and obtain normal, distortion-free ultrasonic data with corresponding labels for both after normalization processing and marking of outliers or missing values.

[0074] Optionally, determine the independent encoder corresponding to the ultrasonic signal data in each mode, such as Figure 3 As shown, including:

[0075] Step S301, obtaining first ultrasonic signal data corresponding to the missing mode, and determining a first independent encoder of the LSTM structure based on the first ultrasonic signal data;

[0076] Step S302: Acquire second ultrasonic signal data corresponding to the singular peak mode, and determine a second independent encoder of the Transformer structure based on the second ultrasonic signal data.

[0077] Specifically, the encoder mainly involves two branches, namely the first independent encoder and the second independent encoder. The first independent encoder corresponds to the ultrasonic signal data of the missing segment mode and adopts the LSTM structure; the second independent encoder corresponds to the second independent encoder and adopts the Transformer structure.

[0078] Optionally, after encoding and calculating the pre-processed ultrasonic data using an independent encoder, feature representation data corresponding to the pre-processed ultrasonic data in different modes is generated, such as Figure 4 As shown, including:

[0079] Step S401, after controlling the first independent encoder to receive the first ultrasonic signal data, use the encoding layer of the LSTM structure in the first independent encoder to perform encoding calculation on the first ultrasonic signal data to obtain the long-range dependency relationship result corresponding to the missing mode in the first ultrasonic signal data;

[0080] Step S402, after controlling the second independent encoder to receive the second ultrasonic signal data, use the encoding layer of the Transformer structure in the second independent encoder to perform multi-head self-attention mechanism encoding calculation on the second ultrasonic signal data to obtain the context information result corresponding to the singular peak mode in the second ultrasonic signal data;

[0081] Step S403: determine feature representation data using the long-range dependency result and the context information result.

[0082] In the process of acquiring feature representation data, the ultrasonic signal data corresponding to the singular peak mode in the preprocessed ultrasonic data is input into the encoder of the Transformer structure, and the ultrasonic signal data corresponding to the missing mode in the preprocessed ultrasonic data is input into the encoder of the LSTM structure. Through the encoding layer of the Transformer structure, the different modes and contextual information of the singular peak modal data are captured based on the multi-head self-attention mechanism, so as to better locate and handle abnormal points. By controlling the encoding layer of the LSTM structure to capture the long-range dependency of the missing modal data, the feature representation data under different modes of the preprocessed ultrasonic data are determined respectively.

[0083] Optionally, step S103 of determining probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and determining a latent space representation result corresponding to the feature representation data using the probability distribution data, such as Figure 5 As shown, including:

[0084] Step S501, determining a latent space shared by independent encoders, and mapping feature representation data to the latent space;

[0085] Step S502, after calculating the mean and variance of the feature representation data of different modes, the probability distribution data corresponding to the feature representation data is determined using the mean and variance;

[0086] Step S503, determining the feature representation result of the latent space through the probability distribution data, and concatenating the feature representation results under different modalities to obtain the latent space representation result.

[0087] The latent space is a shared space for the output data of independent encoders. After designing independent encoders for ultrasonic signals of each modality, the output data of the encoders are mapped to the shared latent space. In the process of obtaining the latent space representation results, the feature representation data corresponding to the preprocessed ultrasonic data in different modalities are first used to calculate the mean and variance respectively, and the probability distribution data of the feature representation data is determined based on the calculation results of the mean and variance. Then, the feature representation of the latent space is determined respectively through the probability distribution data, and the feature representation of the latent space under different modalities is spliced ​​and fused to generate a shared latent space representation result.

[0088] Optionally, step S503 of determining the feature representation result of the latent space by using the probability distribution data and concatenating the feature representation results under different modes to obtain the latent space representation result, such as Figure 6 As shown, including:

[0089] Step S601, using the probability distribution data to calculate the KL divergence value of the feature representation data, and determining the KL divergence value as the first feature representation result of the latent space;

[0090] Step S602, calculating a regularization result and a reconstruction loss result of the feature representation data based on the feature representation data under the missing mode and the singular peak mode, and determining the regularization result and the reconstruction loss result as a second feature representation result and a third feature representation result respectively;

[0091] Step S603: Concatenate the first feature representation result, the second feature representation result, and the third feature representation result to obtain a latent space representation result.

[0092] The KL divergence value ensures that the distribution of latent variables is close to the standard normal distribution, which helps to generate a smooth and interpretable latent space;

[0093] The regularization result can specifically impose additional penalties on outliers and missing values, ensuring that the latent space representation results are more robust in these areas. In actual scenarios, regularization terms can be added to prevent overfitting of outliers and missing values, thereby improving the generalization ability of the overall process.

[0094] Optionally, step S603 of concatenating the first feature representation result, the second feature representation result and the third feature representation result to obtain a latent space representation result is as follows: Figure 7 As shown, including:

[0095] Step S701, determining a first weight value corresponding to the first feature representation result and a second weight value corresponding to the second feature representation result based on the third feature representation result;

[0096] Step S702, using the first weight value to multiply the KL divergence value to update the first feature representation result, and using the second weight value to multiply the regularization result to update the second feature representation result;

[0097] Step S703, accumulating the updated first feature representation result, the updated second feature representation result and the third feature representation result to obtain a latent space representation result.

[0098] For data containing singular peaks, weighted MSE can be used to give singular peaks a higher weight, so that the latent space representation results pay more attention to these outliers and improve the quality of reconstruction. For missing segments, lower weights can be used so that the latent space representation results will not overfit in these areas, while still being able to reasonably fill in missing values.

[0099] Optionally, a shared decoder corresponding to the latent space representation result is determined based on the independent encoder, and the shared decoder is used to convert the latent space representation result into undistorted ultrasonic data in step S104, such as Figure 8 As shown, including:

[0100] Step S801, constructing a shared decoder of LSTM structure and its corresponding reconstruction strategy based on the latent space shared by independent encoders;

[0101] Step S802, using a shared decoder to calculate a time series feature result of a latent space representation result, and using a reconstruction strategy to reconstruct the time series feature result to obtain distortion-free ultrasonic data.

[0102] The output of the shared decoder is single-mode data, that is, normal and undistorted ultrasonic signal data. The structure is designed using the LSTM structure. After the shared latent variables are input to the shared decoder corresponding to the multimodal variational autoencoder, the long-range dependencies in the time series are captured through the LSTM of the shared decoding layer, thereby calculating the time series feature results of the latent variables. Then, the ultrasonic data is reconstructed using the time series feature results output by the shared decoding layer to obtain the single-mode normal and undistorted ultrasonic data corresponding to the preprocessed ultrasonic data.

[0103] The above method can be implemented through model training. The loss function during model training is determined by using the above KL divergence value, regularization result and reconstruction loss result. The relevant model corresponding to the entire multimodal variational autoencoder is jointly trained by using data from all modalities. The model is then used to output a normal, distortion-free ultrasonic signal.

[0104] The encoders in the model include the Transformer encoder and the LSTM encoder. The Transformer encoder processes modal data containing singular peaks. Since the Transformer can capture global dependencies in the sequence through the self-attention mechanism, it is very useful for identifying and processing local singular peaks. And the self-attention mechanism allows the model to take into account the information of the entire sequence when processing each time step, so as to better locate and handle anomalies. The Transformer's self-attention mechanism allows all elements in the sequence to be processed in parallel, which can increase the processing speed, especially when processing long sequences. At the same time, the Transformer can be easily extended to process long sequences, which is particularly useful for long time series data in ultrasonic signals. This makes the Transformer perform well when processing high-frequency and high-resolution ultrasonic signals.

[0105] The LSTM encoder processes modal data containing missing segments. Since LSTM can capture long-range dependencies, it is very useful for processing missing segments in time series. The memory cells of LSTM can store long-term information, and can maintain contextual information even when there are missing segments in the sequence. And the memory cells of LSTM can store long-term information, which helps to process discontinuous data. Even in the case of missing data, LSTM can still maintain previous state information through memory cells, so as to better handle missing segments. At the same time, the gating mechanism of LSTM (input gate, forget gate, output gate) can control the flow of information, which helps to process discontinuous data. The gating mechanism enables LSTM to selectively remember or forget information, making it more flexible in processing missing data.

[0106] The decoder in the model converts the representation of the latent space back to normal, undistorted ultrasonic data of a single modality. The decoder structure adopts an LSTM structure. The shared latent variables are input into the shared decoder of the multimodal variational autoencoder. The shared decoding layer is controlled to capture the long-range dependencies in the time series based on the LSTM structure, thereby calculating the time series feature results of the latent variables. By obtaining the time series feature results output by the shared decoding layer and reconstructing the ultrasonic data according to the time series feature results, the normal, undistorted ultrasonic data of a single modality corresponding to the preprocessed ultrasonic data of different modalities is obtained.

[0107] The comprehensive loss function used in the model includes reconstruction loss, KL divergence, and regularization terms, where the reconstruction loss uses weighted mean square error to handle singular peaks and missing segments. Higher weights are set for singular peaks, while lower weights are set for missing segments. KL divergence ensures that the distribution of latent variables is close to the standard normal distribution. A regularization term is added to handle outliers and missing values, and additional penalties are specifically imposed on these areas to improve the robustness and generalization ability of the model.

[0108] The loss function is specifically:

[0109] ;

[0110] in is the reconstruction loss, is the KL divergence, is the regularization term, and is a hyperparameter used to balance the weights of various losses.

[0111] Reconstruction loss:

[0112] ;

[0113] in is the original data, is to reconstruct the data, is the weight used to handle singular peaks and missing segments. For data with singular peak modes, =10; for missing segment modal data, =0.5; for normal data, =1.

[0114] KL divergence:

[0115] ;

[0116] in and are the mean and standard deviation of the latent variables, respectively.

[0117] Regularization term:

[0118] ;

[0119] in is an indicator function, if If it is a singular peak modal data or missing modal data, it returns 1, otherwise it returns 0. The weight of the regularization term.

[0120] When training the model, all modal data are used to jointly train the entire multimodal variational autoencoder model. Then, using the trained multimodal variational autoencoder model, the collected ultrasonic data is input into the model. The model fuses the extracted latent space vectors through encoder encoding and outputs normal distortion-free ultrasonic data through the decoder.

[0121] From the ultrasonic data processing method mentioned in the above embodiment, it can be seen that this method is able to construct a corresponding independent encoder for the ultrasonic signal data under each mode, and accurately obtain the latent space representation result of the ultrasonic data by extracting features of ultrasonic signal data of different modes containing distortion or missing data, and use the shared decoder corresponding to the independent encoder to complement the distortion and missing of the ultrasonic signal data, thereby realizing accurate completion processing of ultrasonic data in abnormal state.

[0122] Corresponding to the ultrasonic data processing method provided in the above embodiment, the embodiment of the present invention provides an ultrasonic data processing system, such as Fig. 9 As shown, the system includes:

[0123] The pre-processed ultrasonic data acquisition module 910 is used to acquire pre-processed ultrasonic data; wherein the pre-processed ultrasonic data includes distorted or missing multi-modal ultrasonic signal data;

[0124] The feature representation data generating module 920 is used to determine the independent encoder corresponding to the ultrasonic signal data in each mode, and generate the feature representation data corresponding to the pre-processed ultrasonic data in different modes after encoding and calculating the pre-processed ultrasonic data using the independent encoder;

[0125] A latent space representation result determination module 930 is used to determine probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and to determine a latent space representation result corresponding to the feature representation data using the probability distribution data;

[0126] The ultrasonic data conversion module 940 is used to determine the shared decoder corresponding to the latent space representation result based on the independent encoder, and use the shared decoder to convert the latent space representation result into distortion-free ultrasonic data.

[0127] From the ultrasonic data processing system mentioned in the above embodiment, it can be seen that the system is able to construct a corresponding independent encoder for the ultrasonic signal data under each mode, and accurately obtain the latent space representation result of the ultrasonic data by extracting features of ultrasonic signal data of different modes containing distortion or missing data, and use the shared decoder corresponding to the independent encoder to complement the distortion and missing data of the ultrasonic signal data, thereby realizing accurate completion processing of ultrasonic data in abnormal state.

[0128] The ultrasonic data processing system provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned ultrasonic data processing method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned ultrasonic data processing method embodiment.

[0129] This embodiment also provides an electronic device. The structural diagram of the electronic device is as follows: Fig.10 As shown, the device includes a processor 101 and a memory 102; wherein the memory 102 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the steps of the above-mentioned ultrasonic data processing method.

[0130] Fig.10 The electronic device shown further includes a bus 103 and a communication interface 104 , and the processor 101 , the communication interface 104 and the memory 102 are connected via the bus 103 .

[0131] The memory 102 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The bus 103 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0132] The communication interface 104 is used to connect to at least one user terminal and other network units through a network interface, and send the encapsulated IPv4 message or IPv4 message to the user terminal through the network interface.

[0133] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 101. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 102, and the processor 101 reads the information in the memory 102 and completes the steps of the method of the above embodiment in combination with its hardware.

[0134] An embodiment of the present invention further provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the ultrasonic data processing method in the above embodiment are executed.

[0135] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that can be executed by a processor. Based on this understanding, the technical solution of the present invention can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0139] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An ultrasonic data processing method, characterized in that: The method comprises: Acquire pre-processed ultrasonic data; wherein the pre-processed ultrasonic data includes distorted or missing multi-modal ultrasonic signal data; Determine an independent encoder corresponding to the ultrasonic signal data in each mode, and use the independent encoder to perform encoding calculation on the pre-processed ultrasonic data to generate feature representation data corresponding to the pre-processed ultrasonic data in different modes; Determine probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and determine a latent space representation result corresponding to the feature representation data using the probability distribution data; Determining a shared decoder corresponding to the latent space representation result based on the independent encoder, and converting the latent space representation result into distortion-free ultrasonic data using the shared decoder; Acquire pre-processed ultrasonic data, including: Obtain distortion-free ultrasonic signals in each mode; Generate noise data corresponding to the distortion-free ultrasonic signal according to a preset random strategy; wherein the noise data at least includes signal data under a missing mode and a singular peak mode; After superimposing the noise data onto the distortion-free ultrasonic signal, the pre-processed ultrasonic data is obtained; Determining probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and determining a latent space representation result corresponding to the feature representation data using the probability distribution data, including: determining a latent space shared by the independent encoders, and mapping the feature representation data to the latent space; After calculating the mean and variance of the feature representation data of different modes, the probability distribution data corresponding to the feature representation data is determined using the mean and the variance; Determine the feature representation result of the latent space through the probability distribution data, and obtain the latent space representation result by splicing the feature representation results under different modes; Determining the feature representation result of the latent space by using the probability distribution data, and concatenating the feature representation results under different modes to obtain the latent space representation result, including: Calculating a KL divergence value of the feature representation data using the probability distribution data, and determining the KL divergence value as a first feature representation result of the latent space; Calculating a regularization result and a reconstruction loss result of the feature representation data based on the feature representation data under the missing mode and the singular peak mode, and determining the regularization result and the reconstruction loss result as a second feature representation result and a third feature representation result, respectively; The first feature representation result, the second feature representation result and the third feature representation result are concatenated to obtain the latent space representation result; The latent space representation result is obtained by concatenating the first feature representation result, the second feature representation result and the third feature representation result, including: Determine, based on the third feature representation result, a first weight value corresponding to the first feature representation result and a second weight value corresponding to the second feature representation result; Using the first weight value to multiply the KL divergence value to update the first feature representation result, and using the second weight value to multiply the regularization result to update the second feature representation result; The updated first feature representation result, the updated second feature representation result and the third feature representation result are accumulated to obtain the latent space representation result.

2. The ultrasonic data processing method according to claim 1, characterized in that: Determining an independent encoder corresponding to the ultrasonic signal data in each mode includes: Acquire first ultrasonic signal data corresponding to the missing mode, and determine a first independent encoder of the LSTM structure based on the first ultrasonic signal data; Second ultrasonic signal data corresponding to the singular peak mode is acquired, and a second independent encoder of a Transformer structure is determined based on the second ultrasonic signal data.

3. The ultrasonic data processing method according to claim 2, characterized in that: After encoding and calculating the pre-processed ultrasonic data using the independent encoder, feature representation data corresponding to the pre-processed ultrasonic data in different modes is generated, including: After controlling the first independent encoder to receive the first ultrasonic signal data, the encoding layer of the LSTM structure in the first independent encoder is used to perform encoding calculation on the first ultrasonic signal data to obtain a long-range dependency result corresponding to the missing mode in the first ultrasonic signal data; After controlling the second independent encoder to receive the second ultrasonic signal data, use the encoding layer of the Transformer structure in the second independent encoder to perform multi-head self-attention mechanism encoding calculation on the second ultrasonic signal data to obtain the context information result corresponding to the singular peak mode in the second ultrasonic signal data; The feature representation data is determined using the long-range dependency result and the context information result.

4. The ultrasonic data processing method according to claim 1, characterized in that: Determining a shared decoder corresponding to the latent space representation result based on the independent encoder, and converting the latent space representation result into distortion-free ultrasonic data using the shared decoder, comprising: Constructing the shared decoder of LSTM structure and its corresponding reconstruction strategy based on the latent space shared by the independent encoders; The shared decoder is used to calculate the time series feature results of the latent space representation results, and the reconstruction strategy is used to reconstruct the time series feature results to obtain the distortion-free ultrasonic data.

5. An ultrasonic data processing system, characterized in that: The system comprises: A pre-processed ultrasonic data acquisition module, used to acquire pre-processed ultrasonic data; wherein the pre-processed ultrasonic data includes distorted or missing multi-modal ultrasonic signal data; A feature representation data generating module is used to determine an independent encoder corresponding to the ultrasonic signal data in each mode, and after encoding and calculating the pre-processed ultrasonic data using the independent encoder, generate feature representation data corresponding to the pre-processed ultrasonic data in different modes; A latent space representation result determination module, used to determine probability distribution data of the feature representation data based on the mean and variance of the feature representation data, and determine a latent space representation result corresponding to the feature representation data using the probability distribution data; An ultrasonic data conversion module, configured to determine a shared decoder corresponding to the latent space representation result based on the independent encoder, and convert the latent space representation result into distortion-free ultrasonic data using the shared decoder; The pre-processed ultrasonic data acquisition module is also used to: acquire the distortion-free ultrasonic signal under each mode; generate noise data corresponding to the distortion-free ultrasonic signal according to a preset random strategy; wherein the noise data at least includes signal data under the missing mode and the singular peak mode; and obtain the pre-processed ultrasonic data after superimposing the noise data on the distortion-free ultrasonic signal; the latent space representation result determination module is also used to: determine the latent space shared by the independent encoders, and map the feature representation data to the latent space; calculate the mean and variance of the feature representation data of different modes, and use the mean and variance to determine the probability distribution data corresponding to the feature representation data; determine the feature representation result of the latent space through the probability distribution data, and obtain the latent space representation result after splicing the feature representation results under different modes; The latent space representation result determination module is further used, in the process of determining the feature representation result of the latent space through the probability distribution data and obtaining the latent space representation result by splicing the feature representation results under different modes, to: calculate the KL divergence value of the feature representation data using the probability distribution data, and determine the KL divergence value as the first feature representation result of the latent space; calculate the regularization result and the reconstruction loss result of the feature representation data based on the feature representation data under the missing mode and the singular peak mode, and determine the regularization result and the reconstruction loss result as the second feature representation result and the third feature representation result respectively; and obtain the latent space representation result by splicing the first feature representation result, the second feature representation result and the third feature representation result; In the process of obtaining the latent space representation result by concatenating the first feature representation result, the second feature representation result and the third feature representation result, the latent space representation result determination module is also used to: determine the first weight value corresponding to the first feature representation result and the second weight value corresponding to the second feature representation result based on the third feature representation result; update the first feature representation result by multiplying the KL divergence value with the first weight value, and update the second feature representation result by multiplying the regularization result with the second weight value; and obtain the latent space representation result by accumulating the updated first feature representation result, the updated second feature representation result and the third feature representation result.

6. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the ultrasonic data processing method according to any one of claims 1 to 4.

Images