Method for training data generation model, method for generating acoustic parameter field, and device

Abstract

This disclosure relates to a training method for a data generation model, a method for generating an acoustic parameter field, and an apparatus. The method includes: configuring a data generation model according to the imaging modalities corresponding to a training sample set; for each training modal image, performing style encoding through a corresponding style encoder to obtain training style features, and performing content encoding through a shared content encoder to obtain training content features; performing modal decoding through a modal decoder based on the corresponding training content features and training style features to obtain a predicted reconstructed image; performing cross-modal fusion processing on multiple training content features, and inputting them into an acoustic decoder for acoustic decoding to obtain a predicted acoustic parameter field; and completing iterative training of the data generation model based on the differences between each predicted reconstructed image and the corresponding training modal image. This embodiment focuses on the inherent feature information of the acoustic parameter field contained in different modal images, ensuring the accuracy and reliability of the generated acoustic parameter field.

Description

Technical Field

[0001] This disclosure relates to the field of acoustic processing technology, and in particular to a training method for a data generation model, a method for generating an acoustic parameter field, and an apparatus. Background Technology

[0002] Computed tomography (CT) is an important imaging modality for quantitatively characterizing the internal structure of a medium. The tissue density information it provides can provide key acoustic parameter priors for the acoustic simulation of ultrasound.

[0003] Due to limitations in the deployment environment, there may be situations in practical applications where CT modal imaging is not supported. Existing technologies typically use deep learning networks to reconstruct pseudo-CT images from modal images acquired using other imaging modalities, and then derive the acoustic parameter field of the medium based on the pseudo-CT images.

[0004] However, most image generation networks rely on strongly supervised training and are easily constrained by factors such as the number of training samples and image registration errors. More importantly, image generation networks focus on the direct mapping of the absolute numerical level of image pixels. Pseudo-CT images can only simulate the visual presentation of the medium in the CT modality, but the acoustic parameters of the medium they represent are not consistent with the actual acoustic parameters of the medium.

[0005] Because acoustic simulation is extremely sensitive to the acoustic parameters of the medium, even small parameter deviations can be amplified exponentially during the acoustic simulation process. Therefore, the acoustic parameter field obtained based on the above scheme cannot meet the core requirements of high accuracy and high reliability in acoustic simulation scenarios. Summary of the Invention

[0006] This disclosure provides a training method for a data generation model, a method and device for generating acoustic parameter fields, to solve the problem of large deviations in acoustic parameters derived from image reconstruction links, reduce the dependence of deep learning networks on modal image pairing, and provide high-accuracy and high-reliability data support for acoustic simulation scenarios.

[0007] One aspect of this disclosure provides a method for training a data generation model, comprising: Obtain a training sample set, which contains at least one training sample with multimodal image data, wherein the multimodal image data includes training modal images corresponding to at least two imaging modalities respectively. Based on the imaging modalities covered by the training sample set, a data generation model is configured, which includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. For each training modality image, style encoding is performed using the style encoder corresponding to the training modality image to obtain training style features, and content encoding is performed using the shared content encoder to obtain training content features. For each imaging modality, modality decoding is performed using a modality decoder specific to that imaging modality, based on the training content features corresponding to the training sample and the training style features corresponding to the imaging modality, to obtain at least one predicted and reconstructed image of the training sample in that imaging modality. After performing cross-modal fusion processing on multiple training content features corresponding to the training sample, the data is input into the acoustic decoder for acoustic decoding to obtain the predicted acoustic parameter field corresponding to the training sample. Based on the difference between each predicted reconstructed image and the corresponding training modality image, an image reconstruction loss value is determined, and the adjustable network and weight parameters in the data generation model are iteratively trained according to the image reconstruction loss value until the training termination condition is met.

[0008] Another aspect of this disclosure provides a method for generating an acoustic parameter field, comprising: Acquire the test modality images corresponding to the test sample and at least one imaging modality, respectively; Acquire an acoustic student generation network deployed based on a data generation model, wherein the acoustic student generation network includes a shared content encoder and an acoustic decoder; The shared content encoder is used to encode the content of each modal image to be tested to obtain at least one content feature to be tested. After performing cross-modal fusion processing on the features of each of the test contents, the data are input into the acoustic decoder for acoustic decoding to obtain the target acoustic parameter field corresponding to the test sample.

[0009] Another aspect of this disclosure provides a training apparatus for a data generation model, comprising: The training sample set acquisition module is used to acquire a training sample set, which contains at least one training sample with multimodal image data, and the multimodal image data includes training modal images corresponding to at least two imaging modalities respectively. The data generation model configuration module is used to configure a data generation model based on the imaging modalities covered by the training sample set. The data generation model includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. The image encoding processing module is used to perform style encoding on each training modality image to obtain training style features, and to perform content encoding on the shared content encoder to obtain training content features. The image reconstruction module is used to perform modal decoding for each imaging modality, based on the training content features corresponding to the training sample and the training style features corresponding to the imaging modality, to obtain at least one predicted reconstructed image of the training sample in the imaging modality. The predictive acoustic parameter field determination module is used to perform cross-modal fusion processing on multiple training content features corresponding to the training sample, and then input them into the acoustic decoder for acoustic decoding to obtain the predictive acoustic parameter field corresponding to the training sample. The data generation model training module is used to determine the image reconstruction loss value based on the difference between each predicted reconstructed image and the corresponding training modality image, and to iteratively train the adjustable network and weight parameters in the data generation model based on the image reconstruction loss value until the training termination condition is met.

[0010] Another aspect of this disclosure provides an apparatus for generating an acoustic parameter field, the apparatus comprising: The module for acquiring images of the modal under test is used to acquire images of the sample under test and at least one imaging modality, respectively. An acoustic generation network acquisition module is used to acquire an acoustic generation network deployed based on a data generation model, wherein the acoustic generation network includes a shared content encoder and an acoustic decoder. The content feature determination module is used to perform content encoding on each modal image to be tested through the shared content encoder to obtain at least one content feature to be tested; The target acoustic parameter field generation module is used to perform cross-modal fusion processing on the features of each of the test contents, and then input them into the acoustic decoder for acoustic decoding to obtain the target acoustic parameter field corresponding to the test sample.

[0011] Another aspect of this disclosure provides an electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the training method of the data generation model described in any embodiment of the present disclosure and / or the generation method of the acoustic parameter field described in any embodiment of the present disclosure.

[0012] Another aspect of this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the training method for the data generation model described in any embodiment of this disclosure and / or the generation method for the acoustic parameter field described in any embodiment of this disclosure.

[0013] Another aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the training method for the data generation model described in any embodiment of this disclosure and / or the generation method for the acoustic parameter field described in any embodiment of this disclosure.

[0014] The technical solution of this disclosure configures a data generation model according to the imaging modality corresponding to the training sample set. The data generation model includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. Iterative training of the adjustable network and weight parameters in the data generation model is completed based on the difference between each predicted reconstructed image and the corresponding training modality image. The shared content encoder is used to extract cross-modal shared semantic feature information containing acoustic parameter fields from training modality images of different modalities of the same training sample. The imaging modality-specific style encoder is used to extract visual presentation-related style feature information specific to the corresponding imaging modality from the training modality images. The image reconstruction achieved by the imaging modality-specific modality decoder enables the model training process to overcome the defect of pairing dependence on modality images, realize the goal of end-to-end autonomous generation of acoustic parameter fields, and solve the problem of large deviation of acoustic parameters derived by relying on image reconstruction links. Thus, it provides high-accuracy and high-reliability data support for acoustic simulation scenarios.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a training method for a data generation model provided in one embodiment of this disclosure; Figure 2 This is a schematic diagram of the model architecture of a data generation model provided in one embodiment of the present disclosure; Figure 3 This is a schematic diagram illustrating the data flow of a data generation model provided in one embodiment of the present disclosure; Figure 4 A flowchart illustrating a training method for another data generation model provided in one embodiment of this disclosure; Figure 5 A flowchart illustrating a method for generating an acoustic parameter field according to an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of a training device for a data generation model provided in one embodiment of the present disclosure; Figure 7 This is a schematic diagram of the structure of an acoustic parameter field generation device provided in one embodiment of the present disclosure; Figure 8 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0019] It should be noted that the terms "training," "to be tested," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] Figure 1 This is a flowchart illustrating a training method for a data generation model according to an embodiment of this disclosure. This embodiment is applicable to supervised training of cross-modal data generation models. The method can be executed by a training device for the data generation model, which can be implemented in hardware and / or software and can be configured in a terminal device. Figure 1 As shown, the method includes: S110. Obtain the training sample set.

[0021] In this embodiment, the training sample set includes at least one training sample with multimodal image data. The multimodal image data includes training modal images corresponding to at least two imaging modalities. The imaging modalities can characterize the imaging principle and / or imaging parameter configuration of the medical imaging device used to acquire the image data of the training modal images.

[0022] Specifically, the multiple imaging modalities corresponding to different training samples can be the same or different. For example, the imaging modalities can be CT modalities, MRI (Magnetic Resonance Imaging) modalities, MRI T1-weighted modalities, MRI T2-weighted modalities, MRI ZTE (Zero Echo Time) modalities, etc., and are not limited to the examples given above.

[0023] S120. Configure a data generation model based on the imaging modalities covered by the training sample set.

[0024] Specifically, the imaging modalities covered by the training sample set include all imaging modalities used by all training modal images in the training sample set.

[0025] In this embodiment, the data generation model includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. Specifically, in the data generation model, the style encoder corresponds one-to-one with the imaging modalities covered by the training sample set, and the modality decoder corresponds one-to-one with the imaging modalities covered by the training sample set.

[0026] S130. For each training modality image, style encoding is performed using the style encoder corresponding to the training modality image to obtain training style features, and content encoding is performed using the shared content encoder to obtain training content features.

[0027] like Figure 2 As shown, a shared content encoder and n style encoders can form an image coding network, which is deployed at the input of the data generation model to extract features and decouple training modal images of different imaging modalities for the same training sample input.

[0028] Specifically, multiple style encoders are used to extract training style features of training modal images of the same training sample with their own imaging modality. The style encoders are adapted to the training modal images through their own imaging modality. The training style features represent the visual presentation-related style feature information specific to the imaging modality used by the training modal images. The training style features of training modal images of different imaging modalities are significantly different. The training style features corresponding to multiple training modal images of the same imaging modality are the same or tend to be consistent.

[0029] For example, the network architecture used by the style encoder may be a Transformer architecture, a CNN (Convolutional Neural Network) architecture, or a combination thereof. This is only an example of the network architecture of the style encoder and is not limited to the example given above.

[0030] In an optional embodiment, the style encoder incorporates a self-attention mechanism for global context-dependent modeling of multi-scale features extracted from multiple style coding layers within the style encoder. This approach enhances the style encoder's ability to capture style feature information specific to the imaging modality, thereby improving the style feature differences between the specific imaging modality and other imaging modalities.

[0031] Specifically, the shared content encoder is used to extract training content features from training modal images of different imaging modalities of the same training sample. The training content features characterize the cross-modal shared semantic feature information of acoustic parameter fields contained in the training modal images. The training content features of different imaging modalities of the same training sample are the same or tend to be consistent.

[0032] For example, the network architecture used by the shared content encoder may be a U-Net network architecture, a Transformer architecture, a CNN architecture, or a combination thereof. This is only an example of the network architecture used by the shared content encoder and is not limited to the example scenarios given above.

[0033] S140. For each imaging modality, modality decoding is performed using a modality decoder specific to the imaging modality, based on the training content features corresponding to the training sample and the training style features corresponding to the imaging modality, to obtain at least one predicted reconstructed image of the training sample in the imaging modality.

[0034] like Figure 2 As shown, the acoustic decoder and n modal decoders can form a data decoding network, which is deployed at the output of the data generation model to complete image reconstruction and acoustic parameter field generation based on two image features decoupled from the same training modal image.

[0035] The modal decoder is used to reconstruct images based on a combination of sample features consisting of training style features and training content features of the training samples. Specifically, the training content features corresponding to the training samples are combined with the training style features of the training samples to obtain a combination of sample features. The modal decoder performs modal decoding on each combination of sample features to obtain at least one predicted reconstructed image.

[0036] For example, the network architecture used by the modal decoder can be a VAE (Variational Autoencoder) architecture, a GAN (Generative Adversarial Network) architecture, a diffusion model, or a physical model. The diffusion model can be a DDIM (Denoising Diffusion Implicit Model) or a DDPM (Denoising Diffusion Probabilistic Model), etc. This is only an example illustration of the network architecture used by the modal decoder and is not limited to the examples given above.

[0037] S150. After performing cross-modal fusion processing on the multiple training content features corresponding to the training sample, the data is input into the acoustic decoder for acoustic decoding to obtain the predicted acoustic parameter field corresponding to the training sample.

[0038] In an optional embodiment, the cross-modal fusion process includes: transforming the training content features of different imaging modalities of the same training sample into a unified feature space, and then fusing them using at least one of feature stitching, weight-based feature fusion, or attention-based feature fusion.

[0039] For example, feature transformation methods include, but are not limited to, feature mapping or normalization processing, and attention-based fusion processing, which are used to align features of multiple training content features of the same training sample to further extract semantic feature information shared across modalities.

[0040] For example, the network architecture used by the acoustic decoder can be a VAE architecture, a GAN architecture, a physical model, or a diffusion model, but is not limited to the examples given.

[0041] The acoustic parameter field is a spatial distribution set that describes the acoustic properties of a spatial medium. For example, acoustic properties include, but are not limited to, sound velocity, density, attenuation coefficient, and acoustic impedance.

[0042] In one optional embodiment, the predicted acoustic parameter field includes a predicted density field and / or a predicted sound velocity field. The predicted density field represents the density distribution information of the training samples, and the predicted sound velocity field represents the sound velocity distribution information of the training samples.

[0043] S160. Based on the difference between each predicted reconstructed image and the corresponding training modality image, determine the image reconstruction loss value, and based on the image reconstruction loss value, iteratively train the adjustable network and weight parameters in the data generation model until the training termination condition is met.

[0044] Specifically, the predicted and reconstructed image is reconstructed based on the training style features and training content features of the corresponding training modality image, or the predicted and reconstructed image is reconstructed based on the training style features of the corresponding training modality image and the training content features of the training modality images of other imaging modalities of the training samples.

[0045] Figure 3 This is a schematic diagram illustrating the data flow of a data generation model provided in one embodiment of this disclosure. Figure 3 Taking a training sample of multimodal image data as an example, the multimodal image data includes CT images, T1 images, T2 images, and ZTE images. Correspondingly, the data generation model is configured with CT encoders, T1 encoders, T2 encoders, and ZTE encoders, as well as CT decoders, T1 decoders, T2 decoders, and ZTE decoders. Specifically, the CT images, T1 images, T2 images, and ZTE images are input into their respective dedicated style encoders to obtain CT style features, T1 style features, T2 style features, and ZTE style features. The CT images, T1 images, T2 images, and ZTE images are all input into a shared content encoder to obtain CT content features, T1 content features, T2 content features, and ZTE content features. All content features are then transformed into a unified feature space.

[0046] The input data for each modality decoder includes the trained style features output by the style encoder of its specific imaging modality and the trained content features corresponding to at least one imaging modality, used to achieve image reconstruction under its specific imaging modality. Taking the T2 decoder as an example, Figure 3 The “T2 reconstruction” in the text includes a predicted reconstructed image in the T2 modality corresponding to at least one of the CT image, T1 image, T2 image and ZTE image. If one of the predicted reconstructed images is generated based on CT content features, then the corresponding training modality image of the predicted reconstructed image is the CT image.

[0047] In an optional embodiment, determining the image reconstruction loss value based on the difference between each predicted reconstructed image and the corresponding training modality image includes: for each predicted reconstructed image, determining the image reconstruction error based on the image difference between the predicted reconstructed image and the corresponding training modality image; and accumulating the image reconstruction errors to obtain the image reconstruction loss value.

[0048] Image reconstruction errors include self-reconstruction errors and / or cross-reconstruction errors. Specifically, if the imaging modality corresponding to the predicted reconstructed image is the same as the imaging modality used by its corresponding training modality image, then the predicted reconstructed image is a self-reconstructed image of the training modality image, and the image difference between the two images is a self-reconstruction difference. If the imaging modality corresponding to the predicted reconstructed image is different from the imaging modality used by its corresponding training modality image, then the predicted reconstructed image is a cross-modality reconstructed image of the training modality image, and the image difference between the two images is a cross-reconstruction difference. The imaging modality corresponding to the predicted reconstructed image is the specific imaging modality of the modality decoder that outputs the predicted reconstructed image.

[0049] For example, the loss function used for image reconstruction error can be the mean squared error function or the mean absolute error function, but it is not limited to the given example.

[0050] In another optional embodiment, determining the image reconstruction loss value based on the difference between each predicted reconstructed image and the corresponding training modality image includes: performing heterogeneous segmentation on each predicted reconstructed image and each training modality image to obtain a medium reconstruction image set corresponding to each predicted reconstructed image and a medium training image set corresponding to each training modality image; for each sample medium, obtaining multiple medium reconstruction images corresponding to the sample medium from each medium reconstruction image set and obtaining multiple medium training images corresponding to the sample medium from each medium training image set; determining the medium reconstruction error of the sample medium based on the image difference between each medium reconstruction image and the corresponding medium training image; obtaining the medium weights corresponding to each sample medium; and performing a weighted summation of each medium reconstruction error based on each medium weight to obtain the image reconstruction loss value.

[0051] In this embodiment, the medium reconstruction image set includes medium reconstruction images corresponding to at least two sample media in the training samples, and the medium training image set includes medium training images corresponding to each of the sample media. The training samples include at least two sample media with differentiated acoustic characteristic distributions, and the degree of influence of different sample media on acoustic propagation varies significantly.

[0052] In one optional embodiment, if the imaging site corresponding to the training sample is the head, then each of the sample media includes skull structure and soft tissue structure.

[0053] The skull, primarily composed of compact and cancellous bone, exhibits a non-uniform anisotropic acoustic distribution due to its internal pores and bone marrow filling structure. This leads to strong Fresnel reflection and refraction of incident sound waves at the interface between the skull and soft tissue structures, resulting in significant loss of incident sound energy. It also causes severe beam distortion, wavefront phase shift, and multiple scattering effects, thus greatly weakening the effective sound energy penetrating the skull. In contrast, the soft tissue structure, centered on the scalp, cerebrospinal fluid, and brain parenchyma, has a uniform and isotropic acoustic parameter distribution. This results in minimal interface reflection loss and a stable sound beam propagation path, with only slight medium absorption attenuation and weak scattering, maximizing the preservation of sound wave amplitude and phase information.

[0054] In this embodiment, the medium weight characterizes the degree of influence of the sample medium on acoustic propagation. Specifically, in an embodiment where the imaging site corresponding to the training sample is the head, the medium weight of the skull structure is higher than that of the soft tissue structure.

[0055] In this embodiment, the medium reconstruction error includes self-reconstruction error and / or cross-reconstruction error. Specifically, if the imaging modality corresponding to the medium reconstruction image is the same as the imaging modality used in its corresponding medium training image, then the medium reconstruction image is a self-reconstructed image of the medium training image, and the image difference between the two images is a self-reconstruction difference; if the imaging modality corresponding to the medium reconstruction image is different from the imaging modality used in its corresponding medium training image, then the medium reconstruction image is a cross-modal reconstruction image of the medium training image, and the image difference between the two images is a cross-reconstruction difference.

[0056] For example, the loss function used for media reconstruction error can be the Dice loss function or the SSIM (Structural Similarity Index) loss function, but it is not limited to the given example.

[0057] This embodiment strengthens the loss constraint on high-attenuation distortion regions in the training samples by setting a medium weighting mechanism in the image dimension, while weakening the loss constraint on low-attenuation distortion regions in the training samples, thereby further improving the reconstruction accuracy of the acoustic parameter field.

[0058] In one alternative embodiment, the adjustable network and weight parameters in the data generation model include the network and weight parameters of the image encoding network and / or the network and weight parameters of the data decoding network.

[0059] For example, training termination conditions may include at least one of loss constraint convergence, the number of iterations reaching a threshold, and performance verification metrics being met, but are not limited to the given example scenarios.

[0060] The technical solution of this embodiment configures a data generation model according to the imaging modality corresponding to the training sample set. The data generation model includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. Based on the difference between each predicted reconstructed image and the corresponding training modality image, iterative training of the adjustable network and weight parameters in the data generation model is completed. The shared content encoder is used to extract cross-modal shared semantic feature information containing acoustic parameter fields from training modality images of different modalities of the same training sample. The imaging modality-specific style encoder is used to extract visual presentation-related style feature information specific to the corresponding imaging modality from the training modality images. The image reconstruction achieved by the imaging modality-specific modality decoder enables the model training process to overcome the defect of pairing dependence on modality images, realize the goal of end-to-end autonomous generation of acoustic parameter fields, and solve the problem of large deviation of acoustic parameters derived by relying on image reconstruction links. Thus, it provides high-accuracy and high-reliability data support for acoustic simulation scenarios.

[0061] Figure 4 This is a flowchart illustrating another training method for a data generation model provided in one embodiment of this disclosure. This embodiment further refines the step described in the above embodiment of "iteratively training the adjustable network and weight parameters in the data generation model based on the image reconstruction loss value until the training termination condition is met." In this embodiment, the step of iteratively training the adjustable network and weight parameters in the data generation model based on the image reconstruction loss value until the training termination condition is met includes: determining an acoustic loss value based on the difference between the predicted acoustic parameter field and the true acoustic parameter field of each training sample; determining a total loss value based on the acoustic loss value and the image reconstruction loss value; and iteratively training the adjustable network and weight parameters in the data generation model based on the total loss value until the training termination condition is met. Figure 4 As shown, the method includes: S210. Obtain the training sample set.

[0062] S220. Configure a data generation model based on the imaging modalities covered by the training sample set.

[0063] S230. For each training modality image, style encoding is performed using the style encoder corresponding to the training modality image to obtain training style features, and content encoding is performed using the shared content encoder to obtain training content features.

[0064] S210-S230 in this embodiment are the same as those in the above embodiment. Figure 1 The S110-S130 shown are the same or similar, and will not be described again in this embodiment.

[0065] In an optional embodiment, the method further includes: determining the feature distances of multiple training content features corresponding to the same training sample, and determining a content consistency loss value based on at least one feature distance; and iteratively adjusting the network and weight parameters of the shared content encoder based on the content consistency loss value.

[0066] In this context, multiple training content features are transformed into a unified feature space. For example, the loss function used for the content consistency loss value can be the L1 loss function, L1 norm loss function, L2 loss function, L2 norm loss function, MSE loss function, or KL divergence function, etc., but is not limited to the given example.

[0067] By forcing the consistency of training content features of different imaging modalities of the same training sample in a unified feature space, the shared content encoder not only improves the ability of focusing semantic feature information of acoustic parameter fields contained in training modal images from the feature extraction dimension, but also improves the convergence speed and stability of the data generation model.

[0068] S240. For each imaging modality, modality decoding is performed using a modality decoder specific to the imaging modality, based on the training content features corresponding to the training sample and the training style features corresponding to the imaging modality, to obtain at least one predicted reconstructed image of the training sample in the imaging modality.

[0069] S250. After performing cross-modal fusion processing on multiple training content features corresponding to the training sample, the data is input into the acoustic decoder for acoustic decoding to obtain the predicted acoustic parameter field corresponding to the training sample.

[0070] S260. Determine the image reconstruction loss value based on the difference between each predicted reconstructed image and the corresponding training modality image.

[0071] S240-S260 in this embodiment are the same as those in the above embodiments. Figure 1 The S140-S160 shown are the same or similar, and will not be described again in this embodiment.

[0072] S270. Determine the acoustic loss value based on the difference between the predicted acoustic parameter field and the true acoustic parameter field for each training sample.

[0073] In this embodiment, the training sample set includes the true acoustic parameter field corresponding to each training sample. In an optional embodiment, the true acoustic parameter field includes at least one of the measured acoustic parameter field, the inverted acoustic parameter field, and the acoustic parameter field based on an empirical model, wherein the inverted acoustic parameter field characterizes the acoustic parameter field obtained by acoustic parameter analysis based on the CT images of the training samples.

[0074] In an alternative embodiment, before determining the acoustic loss value based on the difference between the predicted acoustic parameter field and the true acoustic parameter field for each training sample, the method further includes post-processing the predicted acoustic parameter field to bring it within an allowable physical range. Exemplarily, the post-processing may include scaling and normalization, where normalization may be max-min normalization, but is not limited to the given examples.

[0075] In one optional embodiment, determining the acoustic loss value based on the difference between the predicted acoustic parameter field and the true acoustic parameter field for each training sample includes: for each training sample, quantizing the predicted acoustic parameter field and the true acoustic parameter field to obtain a predicted acoustic index and a true acoustic index respectively; and determining the acoustic loss value based on the difference between the predicted acoustic index and the true acoustic index for each training sample.

[0076] Specifically, acoustic indices characterize the spatial distribution of acoustic parameter fields. For example, acoustic indices include parameter peak values, acoustic impedance step differences, and cross-correlation coefficients of phase difference distributions, but are not limited to the given examples.

[0077] In another optional embodiment, determining the acoustic loss value based on the difference between the predicted acoustic parameter field and the true acoustic parameter field of each training sample includes: for each training sample, performing feature parsing processing on the predicted acoustic parameter field and the true acoustic parameter field to obtain predicted acoustic features and true acoustic features respectively, and determining the acoustic feature difference of the training sample based on the difference between the predicted acoustic features and the true acoustic features; and determining the acoustic loss value based on each of the acoustic feature differences.

[0078] In one optional embodiment, the predicted acoustic features include a set of predicted simulation parameters and / or a set of predicted medium parameters, wherein the real acoustic features and the predicted acoustic features are a one-to-one correspondence of the same type of parameter set.

[0079] In this embodiment, the step of performing feature analysis processing on the predicted acoustic parameter field and the real acoustic parameter field to obtain predicted acoustic features and real acoustic features respectively includes: when the predicted acoustic features include predicted simulation parameters, performing acoustic simulation on the predicted acoustic parameter field and the real acoustic parameter field according to the configuration parameter data of the ultrasonic equipment to obtain predicted simulation parameters and real simulation parameters respectively; when the predicted acoustic features include a predicted medium parameter field set, performing heterogeneity identification on the predicted acoustic parameter field and the real acoustic parameter field to obtain a predicted medium parameter field set and a real medium parameter field set respectively.

[0080] In the embodiment of predictive simulation parameters, configuration parameter data refers to the core boundary condition parameters of ultrasonic equipment hardware configuration, transmission waveform control and sound field numerical calculation used for acoustic simulation calculation. For example, configuration parameter data includes, but is not limited to, transducer placement position, excitation signal center frequency, transmission array element aperture size, array element arrangement and focusing depth.

[0081] In the embodiment where the imaging site of the training sample is the head, taking the predicted acoustic parameter field as an example, an equivalent skull mesh is constructed based on the predicted acoustic parameter field. A full-link numerical simulation of the transcranial acoustic field is then completed based on the configuration parameter data and the equivalent skull mesh to obtain the predicted simulation parameters. The equivalent skull mesh refers to a voxelized mesh model constructed based on the acoustic parameter distribution of the skull structure in the predicted acoustic parameter field. Taking the predicted simulation parameters as an example, the predicted simulation parameters include the focal contour of the -6dB focal region, the sidelobe ratio and peak sound pressure, and the sound wave penetration efficiency. The -6dB focal region refers to the spatial range enclosed by the sound pressure amplitude in the ultrasonic sound field dropping to 1 / 2 of the peak sound pressure (corresponding to a relative sound intensity level of -6dB). The focal contour of the -6dB focal region can be used to reflect the effective range of the ultrasonic sound field, the focusing shape of the ultrasonic beam and the degree of energy concentration, but it is not limited to the example given above.

[0082] In an optional embodiment, the acoustic feature difference includes acoustic simulation difference, and determining the acoustic feature difference of the training sample based on the difference between the predicted acoustic features and the real acoustic features includes: when the predicted acoustic features include predicted simulation parameters, determining the acoustic simulation difference based on the parameter difference between the predicted simulation parameters and the real simulation parameters.

[0083] In an embodiment of the predicted medium parameter field set, the predicted medium parameter field set includes predicted medium parameter fields corresponding to at least two sample media in the training samples, and the real medium parameter field set includes real medium parameter fields corresponding to each of the sample media.

[0084] In an optional embodiment, determining the acoustic feature difference of the training samples based on the difference between the predicted acoustic features and the real acoustic features includes: when the predicted acoustic features include a predicted medium parameter field set, obtaining the medium weights corresponding to each of the sample mediums; determining the medium feature difference corresponding to each sample medium based on the predicted medium parameter field set and the real medium parameter field set; and performing a weighted summation of each of the medium feature differences based on each of the medium weights to obtain the acoustic feature difference of the training samples.

[0085] Specifically, the difference in medium characteristics characterizes the degree of simulation difference and / or index difference of the sample medium, and the medium weight characterizes the degree of influence of the sample medium on acoustic propagation. In the embodiment where the imaging part corresponding to the training sample is the head, the medium weight of the skull structure is higher than that of the soft tissue structure.

[0086] This embodiment strengthens the loss constraint on high-attenuation distortion regions in the training samples by setting a medium weighting mechanism in the acoustic dimension, while weakening the loss constraint on low-attenuation distortion regions in the training samples, thereby further improving the reconstruction accuracy of the acoustic parameter field.

[0087] S280. Determine the total loss value based on the acoustic loss value and the image reconstruction loss value, and iteratively train the adjustable network and weight parameters in the data generation model based on the total loss value until the training termination condition is met.

[0088] In an optional embodiment, determining the total loss value based on the acoustic loss value and the image reconstruction loss value includes: performing a weighted summation of the acoustic loss value and the image reconstruction loss value to obtain the total loss value.

[0089] In another optional embodiment, determining the total loss value based on the acoustic loss value and the image reconstruction loss value includes: determining the feature distances of multiple training content features corresponding to the same training sample, and determining the content consistency loss value based on at least one feature distance; and performing a weighted summation of the content consistency loss value, the image reconstruction loss value, and the acoustic loss value to obtain the total loss value.

[0090] In the above embodiments, the weight coefficient of the acoustic loss value represents the constraint strength of the generation accuracy of the acoustic parameter field, the weight coefficient of the image reconstruction loss value represents the constraint strength of the reconstruction accuracy of the multimodal image, and the weight coefficient of the content consistency loss value represents the consistency constraint strength of the cross-modal shared features of the same training sample.

[0091] The technical solution of this embodiment constructs an end-to-end supervised training system by combining multi-dimensional physical constraints. Combined with the loss optimization guidance of the acoustic parameter field, it guides the data generation model to accurately focus on key semantic feature information that is shared across modalities and is directly and strongly correlated with the acoustic parameter field. This not only improves the generalization ability of the data generation model in cross-modal scenarios and ensures the high accuracy of the acoustic simulation field, but also avoids feature interference and computational consumption caused by pixel-level reconstruction of redundant feature information, thus achieving synergistic optimization of model performance and training efficiency.

[0092] Figure 5This is a flowchart illustrating a method for generating an acoustic parameter field according to an embodiment of this disclosure. This embodiment is applicable to situations where an acoustic parameter field is generated based on a modal image. The method can be executed by an acoustic parameter field generation device, which can be implemented in hardware and / or software and can be configured in a terminal device. Figure 5 As shown, the method includes: S310. Obtain the test mode images corresponding to the test sample and at least one imaging mode, respectively.

[0093] Specifically, the imaging modality characterizes the imaging principle and / or imaging parameter configuration of the medical imaging device used to acquire image data of the training modality images. For example, the imaging modality may be a CT modality, MRI modality, MRI T1-weighted modality, MRI T2-weighted modality, or MRI ZTE modality, etc., and is not limited to the examples given above.

[0094] S320. Obtain the acoustic generation network deployed based on the data generation model.

[0095] The data generation model is a pre-trained deep learning network capable of cross-modal image feature extraction and acoustic parameter field generation. The supervision constraints of the pre-training of the data generation model include at least one of image domain constraints, acoustic parameter field constraints, and feature domain constraints.

[0096] For example, image domain constraints include, but are not limited to, image reconstruction constraints, image adversarial constraints, image discriminative constraints, and image cross-cyclic consistency constraints, while feature domain constraints include, but are not limited to, content consistency constraints, style prior constraints, feature adversarial constraints, and feature discriminative constraints.

[0097] Specifically, the generation chain of the acoustic parameter field is separated from the data generation model, and the pre-trained network and weight parameters in the generation chain are solidified to form an acoustic generation network for engineering deployment. In this embodiment, the acoustic generation network includes a shared content encoder and an acoustic decoder.

[0098] In one alternative embodiment, the data generation model is obtained using the training method for the data generation model described in any embodiment of this disclosure.

[0099] S330. Through the shared content encoder, each modal image to be tested is content encoded to obtain at least one content feature to be tested.

[0100] Specifically, the shared content encoder is used to extract cross-modal shared semantic feature information, including acoustic parameter fields, from one or more test modal images of the test sample. For example, the network architecture used by the shared content encoder can be a U-Net network architecture, a Transformer architecture, a CNN architecture, or a hybrid network architecture combining CNN and Transformer architectures, etc. This is only an illustrative example of the network architecture used by the shared content encoder and is not limited to the examples given above.

[0101] S340. After performing cross-modal fusion processing on the features of each of the test contents, the data is input into the acoustic decoder for acoustic decoding to obtain the target acoustic parameter field corresponding to the test sample.

[0102] For example, the network architecture used by the acoustic decoder can be a VAE architecture, a GAN architecture, a physical model, or a diffusion model, but is not limited to the examples given.

[0103] The acoustic parameter field is a spatial distribution set that describes the acoustic properties of a spatial medium. For example, acoustic properties include, but are not limited to, sound velocity, density, attenuation coefficient, and acoustic impedance.

[0104] In one optional embodiment, the target acoustic parameter field includes a target density field and / or a target sound velocity field. The target density field represents the density distribution information of the training samples, and the target sound velocity field represents the sound velocity distribution information of the training samples.

[0105] Based on the above embodiments, optionally, the method further includes: when each of the imaging modalities does not include a CT modality, acquiring the style features to be tested for each image of the modality to be tested; performing modal decoding on each style feature to be tested and each content feature to be tested using a modal decoder specific to the CT modality in the data generation model to obtain at least one CT reconstructed image; acquiring the modal weights corresponding to each of the imaging modalities, and determining the target CT image of the sample to be tested based on each modal weight and each CT reconstructed image.

[0106] For example, the style features to be tested can be obtained by generating the corresponding modal encoder in the model by inputting the modal image to be tested, or they can be read from a database that stores the style features of the imaging modality, or they can be obtained by interpolating or sampling the baseline style features of the imaging modality according to the resolution of the modal image to be tested, but are not limited to the examples given.

[0107] In an optional embodiment, modal decoding is performed on each style feature to be tested and each content feature to be tested using a CT modality-specific modal decoder in the data generation model to obtain at least one CT reconstructed image. This includes: performing cross-modal fusion processing on each content feature to be tested to obtain fused content features; combining each style feature to be tested with the fused content features; and performing modal decoding on each combined style feature to be tested and fused content feature using a CT modality-specific modal decoder to obtain at least one CT reconstructed image.

[0108] In another optional embodiment, modal decoding is performed on each style feature to be tested and each content feature to be tested using a CT modality-specific modal decoder in the data generation model to obtain at least one CT reconstructed image. This includes: combining each style feature to be tested and each content feature to be tested in pairs, and performing modal decoding on each combined style feature to be tested and fused content feature using a CT modality-specific modal decoder to obtain at least one CT reconstructed image.

[0109] For example, the target CT image of the test sample is obtained by weighted fusion of each CT reconstructed image according to each modality weight, and pixel-level normalization calibration is performed on the fusion result.

[0110] Based on the above embodiments, optionally, the method further includes: performing acoustic parameter analysis on the target CT image to obtain a reference acoustic parameter field of the sample to be tested; and performing cross-validation based on the reference acoustic parameter field and the target acoustic parameter field to obtain a cross-validation result.

[0111] Cross-validation refers to the process of comparing the target acoustic parameter field with a reference acoustic parameter field in a multi-dimensional consistency quantification, and verifying the degree of acoustic information retention of the target CT image in reverse. The cross-validation result characterizes the reliability of the target CT image.

[0112] For example, cross-validation methods include, but are not limited to, verification of distribution deviations in acoustic parameter fields, verification of simulation deviations, and verification of statistical distribution consistency.

[0113] This embodiment transforms the image reconstruction process from uninterpretable to quantifiable through a cross-validation mechanism of two acoustic parameter fields, breaking down the information barrier of the "black box" of deep learning networks, avoiding the failure of subsequent acoustic simulation caused by CT image reconstruction distortion under abnormal samples, and ensuring the robustness of acoustic simulation in complex application scenarios.

[0114] The technical solution of this embodiment, by configuring the acoustic generation network according to the data generation model, realizes an end-to-end generation paradigm from modal image input to acoustic parameter field output. This solves the problem of large deviations in acoustic parameters derived by relying on image reconstruction links, and improves the generation efficiency of acoustic parameter fields. At the same time, the acoustic generation network is compatible with image inputs of single or multiple imaging modalities through a shared content encoder. Whether it is a known imaging modality or a newly added imaging modality, there is no need to reconstruct the main network structure, which improves the generalization ability of the acoustic generation network across modalities and its multi-scene adaptability.

[0115] The following are embodiments of the training apparatus for the data generation model provided in this disclosure. This apparatus and the training method for the data generation model in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the training apparatus for the data generation model, please refer to the content of the training method for the data generation model in the above embodiments.

[0116] Figure 6 This is a schematic diagram of the structure of a training device for a data generation model provided in one embodiment of this disclosure. Figure 6 As shown, the device includes: a training sample set acquisition module 410, a data generation model configuration module 420, an image encoding processing module 430, an image reconstruction module 440, a predicted acoustic parameter field determination module 450, and a data generation model training module 460.

[0117] The training sample set acquisition module 410 is used to acquire a training sample set, which includes at least one training sample with multimodal image data, and the multimodal image data includes training modal images corresponding to at least two imaging modalities respectively. The data generation model configuration module 420 is used to configure a data generation model according to the imaging modalities covered by the training sample set. The data generation model includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. The image encoding processing module 430 is used to perform style encoding through the style encoder corresponding to the training modality image to obtain training style features, and to perform content encoding through the shared content encoder to obtain training content features for each training modality image. The image reconstruction module 440 is used to perform modal decoding for each imaging modality, based on the training content features corresponding to the training sample and the training style features corresponding to the imaging modality, to obtain at least one predicted reconstructed image of the training sample in the imaging modality. The predictive acoustic parameter field determination module 450 is used to perform cross-modal fusion processing on multiple training content features corresponding to the training sample, and then input them into the acoustic decoder for acoustic decoding to obtain the predictive acoustic parameter field corresponding to the training sample. The data generation model training module 460 is used to determine the image reconstruction loss value based on the difference between each predicted reconstructed image and the corresponding training modality image, and to iteratively train the adjustable network and weight parameters in the data generation model based on the image reconstruction loss value until the training termination condition is met.

[0118] The technical solution of this embodiment configures a data generation model according to the imaging modality corresponding to the training sample set. The data generation model includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. Based on the difference between each predicted reconstructed image and the corresponding training modality image, iterative training of the adjustable network and weight parameters in the data generation model is completed. The shared content encoder is used to extract cross-modal shared semantic feature information containing acoustic parameter fields from training modality images of different modalities of the same training sample. The imaging modality-specific style encoder is used to extract visual presentation-related style feature information specific to the corresponding imaging modality from the training modality images. The image reconstruction achieved by the imaging modality-specific modality decoder enables the model training process to overcome the defect of pairing dependence on modality images, realize the goal of end-to-end autonomous generation of acoustic parameter fields, and solve the problem of large deviation of acoustic parameters derived by relying on image reconstruction links. Thus, it provides high-accuracy and high-reliability data support for acoustic simulation scenarios.

[0119] In an optional embodiment, the training sample set includes the real acoustic parameter field corresponding to each training sample, and the data generation model training module 460 includes: The acoustic loss value determination unit is used to determine the acoustic loss value based on the difference between the predicted acoustic parameter field and the true acoustic parameter field for each training sample. The data generation model training unit is used to determine the total loss value based on the acoustic loss value and the image reconstruction loss value, and to iteratively train the adjustable network and weight parameters in the data generation model based on the total loss value until the training termination condition is met.

[0120] In one optional embodiment, the acoustic loss value determination unit includes: The feature parsing processing subunit is used to perform feature parsing processing on the predicted acoustic parameter field and the real acoustic parameter field for each training sample, to obtain the predicted acoustic features and the real acoustic features respectively, and to determine the acoustic feature difference of the training sample based on the difference between the predicted acoustic features and the real acoustic features. The acoustic loss value determination subunit is used to determine the acoustic loss value based on the differences in the acoustic features described.

[0121] In an optional embodiment, the predicted acoustic features include a predicted simulation parameter and / or a predicted medium parameter field set, and the feature parsing processing subunit is specifically used for: When the predicted acoustic features include predicted simulation parameters, acoustic simulations are performed on the predicted acoustic parameter field and the actual acoustic parameter field based on the configuration parameter data of the ultrasonic equipment to obtain the predicted simulation parameters and the actual simulation parameters, respectively. When the predicted acoustic features include a set of predicted medium parameter fields, heterogeneity identification is performed on the predicted acoustic parameter fields and the real acoustic parameter fields to obtain the set of predicted medium parameter fields and the set of real medium parameter fields, respectively. The predicted medium parameter field set includes predicted medium parameter fields corresponding to at least two sample media in the training samples, and the real medium parameter field set includes real medium parameter fields corresponding to each of the sample media.

[0122] In one optional embodiment, the feature parsing processing subunit is specifically used for: When the predicted acoustic features include a predicted medium parameter field set, the medium weights corresponding to each of the sample media are obtained, and the medium weights characterize the degree of influence of the sample media on acoustic propagation. Based on the predicted medium parameter field set and the actual medium parameter field set, determine the medium characteristic differences corresponding to each sample medium; The acoustic feature differences of the training samples are obtained by weighted summation of the feature differences of each medium according to the weight of each medium.

[0123] In one optional embodiment, the data generation model training unit is specifically used for: Determine the feature distances of multiple training content features corresponding to the same training sample, and determine the content consistency loss value based on at least one feature distance; The total loss value is obtained by weighted summing of the content consistency loss value, image reconstruction loss value, and acoustic loss value.

[0124] In an optional embodiment, the data generation model training module 460 includes: The image reconstruction loss value determination unit is used to perform heterogeneous segmentation on each of the predicted reconstruction images and each of the training modality images to obtain a medium reconstruction image set corresponding to each predicted reconstruction image and a medium training image set corresponding to each training modality image; wherein, the medium reconstruction image set includes medium reconstruction images corresponding to at least two sample media in the training samples, and the medium training image set includes medium training images corresponding to each of the sample media. For each sample medium, multiple reconstructed images corresponding to the sample medium are obtained from each of the medium reconstruction image sets, and multiple training images corresponding to the sample medium are obtained from each of the medium training image sets. The medium reconstruction error of the sample medium is determined based on the image difference between each reconstructed image and the corresponding training image. The medium reconstruction error includes self-reconstruction error and / or cross-reconstruction error. Obtain the medium weight corresponding to each of the sample media, whereby the medium weight characterizes the degree of influence of the sample medium on acoustic propagation; The image reconstruction loss value is obtained by weighting and summing the reconstruction errors of each medium according to the weight of each medium.

[0125] The training apparatus for the data generation model provided in this disclosure can execute the training method for the data generation model provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0126] The following are embodiments of the acoustic parameter field generation apparatus provided in this disclosure. This apparatus and the acoustic parameter field generation method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the acoustic parameter field generation apparatus, please refer to the content of the acoustic parameter field generation method in the above embodiments.

[0127] Figure 7 This is a schematic diagram of the structure of an acoustic parameter field generation device provided in one embodiment of this disclosure. Figure 7 As shown, the device includes: a modality image acquisition module 510, an acoustic generation network acquisition module 520, a content feature determination module 530, and a target acoustic parameter field generation module 540.

[0128] The test modality image acquisition module 510 is used to acquire test modality images corresponding to the test sample and at least one imaging modality, respectively. Acoustic student generation network acquisition module 520 is used to acquire an acoustic student generation network deployed according to a data generation model, wherein the acoustic student generation network includes a shared content encoder and an acoustic decoder. The content feature determination module 530 is used to perform content encoding on each modal image to be tested through the shared content encoder to obtain at least one content feature to be tested; The target acoustic parameter field generation module 540 is used to perform cross-modal fusion processing on the features of each of the test contents, and then input the data into the acoustic decoder for acoustic decoding to obtain the target acoustic parameter field corresponding to the test sample.

[0129] The technical solution of this embodiment, by configuring the acoustic generation network according to the data generation model, realizes an end-to-end generation paradigm from modal image input to acoustic parameter field output. This solves the problem of large deviations in acoustic parameters derived by relying on image reconstruction links, and improves the generation efficiency of acoustic parameter fields. At the same time, the acoustic generation network is compatible with image inputs of single or multiple imaging modalities through a shared content encoder. Whether it is a known imaging modality or a newly added imaging modality, there is no need to reconstruct the main network structure, which improves the generalization ability of the acoustic generation network across modalities and its multi-scene adaptability.

[0130] In an optional embodiment, the apparatus further includes: a training apparatus for the data generation model described in any of the above embodiments of the present disclosure.

[0131] In an optional embodiment, the device further includes: The target CT image determination module is used to acquire the style features to be tested for each image of the target modality when the imaging modality does not include a CT modality. By using the CT modality-specific modal decoder in the data generation model, modal decoding is performed on each style feature to be tested and each content feature to be tested to obtain at least one CT reconstructed image; The modal weights corresponding to each of the imaging modalities are obtained, and the target CT image of the sample to be tested is determined based on the modal weights and the CT reconstructed images.

[0132] In an optional embodiment, the device further includes: The cross-validation module is used to perform acoustic parameter analysis on the target CT image to obtain the reference acoustic parameter field of the sample to be tested; Cross-validation is performed based on the reference acoustic parameter field and the target acoustic parameter field to obtain cross-validation results, which characterize the reliability of the target CT image.

[0133] The acoustic parameter field generation apparatus provided in this disclosure can execute the acoustic parameter field generation method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0134] Figure 8This is a schematic diagram of the structure of an electronic device provided according to one embodiment of the present disclosure. The electronic device 10 is intended to represent various forms of digital computers and various forms of mobile devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0135] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor 11. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14.

[0136] The input / output (I / O) interface 15 is also connected to the bus 14. Multiple components in the electronic device 10 are connected to the I / O interface 15, including: input unit 16, output unit 17, storage unit 18, and communication unit 19. The communication unit 19 allows the electronic device 10 to exchange information or data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0137] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. The processor 11 executes the various methods and processes described above, such as the training method of the data generation model and / or the generation method of the acoustic parameter field provided in the above embodiments.

[0138] In some embodiments, the data generation model training method and / or acoustic parameter field generation method provided in the above embodiments can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data generation model training method and / or acoustic parameter field generation method described above can be performed.

[0139] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of embodiments of this disclosure.

[0140] The computer program may be written in any combination of one or more programming languages. These computer programs may be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the functions / operations specified in the flowchart and / or block diagram are performed. The computer program may execute entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0142] The systems and technologies described herein can be implemented in any combination of backend components, middleware components, or frontend components in a computing system that interconnects its components via digital data communication of any form or medium. The computing system may include clients and servers, which typically interact via a communication network, with the client-server relationship established by computer programs running on the respective computers and having a client-server relationship with each other.

[0143] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0144] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A training method for a data generation model, characterized in that, include: Obtain a training sample set, which contains at least one training sample with multimodal image data, wherein the multimodal image data includes training modal images corresponding to at least two imaging modalities respectively. Based on the imaging modalities covered by the training sample set, a data generation model is configured, which includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. For each training modality image, style encoding is performed using the style encoder corresponding to the training modality image to obtain training style features, and content encoding is performed using the shared content encoder to obtain training content features. For each imaging modality, modality decoding is performed using a modality decoder specific to that imaging modality, based on the training content features corresponding to the training sample and the training style features corresponding to the imaging modality, to obtain at least one predicted and reconstructed image of the training sample in that imaging modality. After performing cross-modal fusion processing on multiple training content features corresponding to the training sample, the data is input into the acoustic decoder for acoustic decoding to obtain the predicted acoustic parameter field corresponding to the training sample. Based on the difference between each predicted reconstructed image and the corresponding training modality image, an image reconstruction loss value is determined, and the adjustable network and weight parameters in the data generation model are iteratively trained according to the image reconstruction loss value until the training termination condition is met.

2. The training method according to claim 1, characterized in that, The training sample set includes the real acoustic parameter field corresponding to each training sample. The step of iteratively training the adjustable network and weight parameters in the data generation model based on the image reconstruction loss value until the training termination condition is met includes: The acoustic loss value is determined based on the difference between the predicted acoustic parameter field and the true acoustic parameter field for each training sample. Based on the acoustic loss value and the image reconstruction loss value, a total loss value is determined, and based on the total loss value, the adjustable network and weight parameters in the data generation model are iteratively trained until the training termination condition is met.

3. The training method according to claim 2, characterized in that, The step of determining the acoustic loss value based on the difference between the predicted acoustic parameter field and the true acoustic parameter field for each training sample includes: For each training sample, feature analysis processing is performed on the predicted acoustic parameter field and the real acoustic parameter field to obtain the predicted acoustic features and the real acoustic features, respectively. Based on the difference between the predicted acoustic features and the real acoustic features, the acoustic feature difference of the training sample is determined. The acoustic loss value is determined based on the differences in the acoustic characteristics described.

4. The training method according to claim 3, characterized in that, The predicted acoustic features include predicted simulation parameters and / or a set of predicted medium parameter fields. The feature analysis processing of the predicted acoustic parameter fields and the actual acoustic parameter fields to obtain the predicted acoustic features and actual acoustic features respectively includes: When the predicted acoustic features include predicted simulation parameters, acoustic simulations are performed on the predicted acoustic parameter field and the actual acoustic parameter field based on the configuration parameter data of the ultrasonic equipment to obtain the predicted simulation parameters and the actual simulation parameters, respectively. When the predicted acoustic features include a set of predicted medium parameter fields, heterogeneity identification is performed on the predicted acoustic parameter fields and the real acoustic parameter fields to obtain the set of predicted medium parameter fields and the set of real medium parameter fields, respectively. The predicted medium parameter field set includes predicted medium parameter fields corresponding to at least two sample media in the training samples, and the real medium parameter field set includes real medium parameter fields corresponding to each of the sample media.

5. The training method according to claim 4, characterized in that, The step of determining the acoustic feature differences of the training samples based on the differences between the predicted acoustic features and the true acoustic features includes: When the predicted acoustic features include a predicted medium parameter field set, the medium weights corresponding to each of the sample media are obtained, and the medium weights characterize the degree of influence of the sample media on acoustic propagation. Based on the predicted medium parameter field set and the actual medium parameter field set, determine the medium characteristic differences corresponding to each sample medium; The acoustic feature differences of the training samples are obtained by weighted summation of the feature differences of each medium according to the weight of each medium.

6. The training method according to claim 2, characterized in that, The step of determining the total loss value based on the acoustic loss value and the image reconstruction loss value includes: Determine the feature distances of multiple training content features corresponding to the same training sample, and determine the content consistency loss value based on at least one feature distance; The total loss value is obtained by weighted summing of the content consistency loss value, image reconstruction loss value, and acoustic loss value.

7. The training method according to any one of claims 1-3 and 5-6, characterized in that, The step of determining the image reconstruction loss value based on the difference between each predicted reconstructed image and the corresponding training modality image includes: Heterogeneous segmentation is performed on each of the predicted reconstructed images and each of the training modal images to obtain a medium reconstruction image set corresponding to each predicted reconstructed image and a medium training image set corresponding to each training modal image; wherein, the medium reconstruction image set contains medium reconstruction images corresponding to at least two sample media in the training samples, and the medium training image set contains medium training images corresponding to each of the sample media. For each sample medium, multiple reconstructed images corresponding to the sample medium are obtained from each of the medium reconstruction image sets, and multiple training images corresponding to the sample medium are obtained from each of the medium training image sets. The medium reconstruction error of the sample medium is determined based on the image difference between each reconstructed image and the corresponding training image. The medium reconstruction error includes self-reconstruction error and / or cross-reconstruction error. Obtain the medium weight corresponding to each of the sample media, whereby the medium weight characterizes the degree of influence of the sample medium on acoustic propagation; The image reconstruction loss value is obtained by weighting and summing the reconstruction errors of each medium according to the weight of each medium.

8. A method for generating an acoustic parameter field, characterized in that, include: Acquire the test modality images corresponding to the test sample and at least one imaging modality, respectively; Acquire an acoustic student generation network deployed based on a data generation model, wherein the acoustic student generation network includes a shared content encoder and an acoustic decoder; The shared content encoder is used to encode the content of each modal image to be tested to obtain at least one content feature to be tested. After performing cross-modal fusion processing on the features of each of the test contents, the data are input into the acoustic decoder for acoustic decoding to obtain the target acoustic parameter field corresponding to the test sample.

9. The generation method according to claim 8, characterized in that, The data generation model is obtained using the training method for the data generation model as described in any one of claims 1-7.

10. The generation method according to claim 8, characterized in that, Also includes: In cases where none of the imaging modalities include a CT modality, the style features to be tested for each modality image are obtained. By using the CT modality-specific modal decoder in the data generation model, modal decoding is performed on each style feature to be tested and each content feature to be tested to obtain at least one CT reconstructed image; The modal weights corresponding to each of the imaging modalities are obtained, and the target CT image of the sample to be tested is determined based on the modal weights and the CT reconstructed images.

11. The generation method according to claim 10, characterized in that, Also includes: The target CT image is analyzed for acoustic parameters to obtain the reference acoustic parameter field of the sample to be tested; Cross-validation is performed based on the reference acoustic parameter field and the target acoustic parameter field to obtain cross-validation results, which characterize the reliability of the target CT image.

12. A training device for a data generation model, characterized in that, include: The training sample set acquisition module is used to acquire a training sample set, which contains at least one training sample with multimodal image data, and the multimodal image data includes training modal images corresponding to at least two imaging modalities respectively. The data generation model configuration module is used to configure a data generation model based on the imaging modalities covered by the training sample set. The data generation model includes a shared content encoder, an acoustic decoder, and a style encoder and modality decoder specific to each imaging modality. The image encoding processing module is used to perform style encoding on each training modality image to obtain training style features, and to perform content encoding on the shared content encoder to obtain training content features. The image reconstruction module is used to perform modal decoding for each imaging modality, based on the training content features corresponding to the training sample and the training style features corresponding to the imaging modality, to obtain at least one predicted reconstructed image of the training sample in the imaging modality. The predictive acoustic parameter field determination module is used to perform cross-modal fusion processing on multiple training content features corresponding to the training sample, and then input them into the acoustic decoder for acoustic decoding to obtain the predictive acoustic parameter field corresponding to the training sample. The data generation model training module is used to determine the image reconstruction loss value based on the difference between each predicted reconstructed image and the corresponding training modality image, and to iteratively train the adjustable network and weight parameters in the data generation model based on the image reconstruction loss value until the training termination condition is met.

13. A device for generating an acoustic parameter field, characterized in that, include: The module for acquiring images of the modal under test is used to acquire images of the sample under test and at least one imaging modality, respectively. An acoustic generation network acquisition module is used to acquire an acoustic generation network deployed based on a data generation model, wherein the acoustic generation network includes a shared content encoder and an acoustic decoder. The content feature determination module is used to perform content encoding on each modal image to be tested through the shared content encoder to obtain at least one content feature to be tested; The target acoustic parameter field generation module is used to perform cross-modal fusion processing on the features of each of the test contents, and then input them into the acoustic decoder for acoustic decoding to obtain the target acoustic parameter field corresponding to the test sample.

14. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the training method of the data generation model according to any one of claims 1-7 and / or the method for generating the acoustic parameter field according to any one of claims 8-11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the training method of the data generation model according to any one of claims 1-7 and / or the method for generating the acoustic parameter field according to any one of claims 8-11.

16. A computer program product comprising a computer program that, when executed by a processor, implements the training method for a data generation model according to any one of claims 1-7 and / or the method for generating an acoustic parameter field according to any one of claims 8-11.

Images