Method, device and medium for generating a labeled ultrasound sample image

Abstract

Embodiments of the present application provide a method, device, equipment and medium for generating a labeled ultrasound sample image. In the method, a diffusion model and an auxiliary segmentation network are obtained by pre-training. A target mask image is obtained, and a guide condition is determined based on the target mask image and the auxiliary segmentation network. A target sample image corresponding to the target mask image is generated by the diffusion model based on the guide condition. The target mask image is used to represent a target object contained in the target sample image. In this way, the auxiliary segmentation network and the target mask image can be used as guide conditions to provide directional guidance for the diffusion model, thereby improving the denoising effect of the diffusion model and improving the sample quality of the target sample image generated by the diffusion model, while reducing the difficulty and cost of obtaining the target sample image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for generating labeled ultrasound sample images. Background Technology

[0002] Ultrasound image segmentation is an important research direction in medical image analysis. Ultrasound images contain rich information about biological tissues and organs; therefore, accurate analysis of ultrasound images is of great significance for medical diagnosis and treatment.

[0003] In related technologies, ultrasound image segmentation is often based on models trained using deep learning. Therefore, a large number of labeled sample ultrasound images are required during model training. Specifically, sample ultrasound images can be generated using Field II or based on Generative Adversarial Networks (GANs). However, obtaining sample ultrasound images using these methods is complex and costly. Summary of the Invention

[0004] To overcome the problems existing in related technologies, the present invention provides a method, apparatus, device and medium for generating labeled ultrasound sample images.

[0005] In a first aspect, the present invention provides a method for generating labeled ultrasound sample images, the method comprising:

[0006] Obtain the pre-trained diffusion model and auxiliary segmentation network;

[0007] Acquire a target mask image, and determine guiding conditions based on the target mask image and the auxiliary segmentation network;

[0008] Based on the guiding conditions, a target sample image corresponding to the target mask image is generated through the diffusion model; the target mask image is used to characterize the target object contained in the target sample image.

[0009] Optionally, the diffusion model is trained based on unlabeled first sample images, and the auxiliary segmentation network is trained based on labeled second sample images.

[0010] Optionally, acquiring the target mask image may include:

[0011] Extraction parameters are randomly generated, wherein the extraction parameters include the edge vertices of the target object, multiple edge lengths, and multiple random vectors;

[0012] Polynomial fitting is performed on the extracted parameters to obtain multi-segment edge data of the target object;

[0013] The target mask image is obtained by combining multiple edge data segments of the target object.

[0014] Optionally, generating a target sample image corresponding to the target mask image using the diffusion model based on the guiding conditions includes:

[0015] Input the random noise image into the diffusion model;

[0016] Based on the guiding conditions, adjust the parameter information used by the diffusion model in the prediction process;

[0017] The random noise image is denoised based on the diffusion model with adjusted parameters to obtain the target sample image.

[0018] Optionally, adjusting the parameter information used by the diffusion model in the prediction process based on the guiding conditions includes:

[0019] Obtain the denoised image generated by the diffusion model during this round of prediction;

[0020] To achieve the desired outcome, the gradient information corresponding to the processing of the denoised image using the auxiliary segmentation network is obtained; the desired outcome is that the segmentation result of the auxiliary segmentation network is the same as the target mask image.

[0021] Based on the gradient information and the parameter information of the diffusion model, the parameters of the diffusion model are updated, and the denoised image generated in the next round of prediction is obtained based on the diffusion model with updated parameters.

[0022] After T rounds of prediction, the denoised image output by the diffusion model is determined as the target sample image.

[0023] Optionally, the method further includes:

[0024] Based on the target sample image, the segmentation model to be trained is trained;

[0025] Obtain the segmentation result of the model to be segmented, and adjust the parameters of the segmentation model to be trained based on the segmentation result and the real label information; the real label information is the target mask image corresponding to the target sample image.

[0026] If the stopping condition is met, the segmentation model to be trained is determined as the target segmentation model.

[0027] Secondly, the present invention provides an apparatus for generating labeled ultrasound sample images, the apparatus comprising:

[0028] The first acquisition module is used to acquire the pre-trained diffusion model and the auxiliary segmentation network;

[0029] The second acquisition module is used to acquire a target mask image and determine guiding conditions based on the target mask image and the auxiliary segmentation network;

[0030] The first generation module is used to generate a target sample image corresponding to the target mask image based on the guiding conditions and through the diffusion model; the target mask image is used to characterize the target object contained in the target sample image.

[0031] Optionally, the second acquisition module includes:

[0032] The first generation submodule is used to randomly generate extraction parameters, wherein the extraction parameters include the edge vertices of the target object, multiple edge lengths, and multiple random vectors.

[0033] The first fitting module is used to perform polynomial fitting on the extracted parameters to obtain multi-segment edge data of the target object;

[0034] The first combination module is used to combine multiple edge data of the target object to obtain the target mask image.

[0035] Optionally, the first generation module includes:

[0036] The first input module is used to input the random noise image into the diffusion model;

[0037] The first adjustment module is used to adjust the parameter information used by the diffusion model in the prediction process based on the guiding conditions.

[0038] The first processing module is used to perform denoising processing on the random noise image based on the diffusion model with adjusted parameters to obtain the target sample image.

[0039] Optionally, the first adjustment module includes:

[0040] The first acquisition submodule is used to acquire the denoised image generated by the diffusion model in this round of prediction.

[0041] The second acquisition submodule is used to acquire gradient information corresponding to the processing of the denoised image by the auxiliary segmentation network, based on the target expectation; the target expectation is that the segmentation result of the auxiliary segmentation network is the same as the target mask image.

[0042] The first update module is used to update the parameters of the diffusion model based on the gradient information and the parameter information of the diffusion model, and to obtain the denoised image generated in the next round of prediction based on the diffusion model with updated parameters.

[0043] The first determining module is used to determine the denoised image output by the diffusion model as the target sample image after performing T rounds of prediction.

[0044] Optionally, the device further includes:

[0045] The first training module is used to train the segmentation model to be trained based on the target sample image;

[0046] The third acquisition module is used to acquire the segmentation result of the model to be segmented, and to adjust the parameters of the segmentation model to be trained based on the segmentation result and the real label information; the real label information is the target mask image corresponding to the target sample image.

[0047] The second determining module is used to determine the segmentation model to be trained as the target segmentation model when the stopping condition is met.

[0048] Thirdly, the present invention provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the method for generating labeled ultrasound sample images as described in any one of the first aspects above.

[0049] Fourthly, the present invention provides a readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the steps in the method for generating labeled ultrasound sample images as described in any of the embodiments of the first aspect above.

[0050] In this embodiment of the invention, by pre-training the diffusion model and the auxiliary segmentation network, good image restoration and segmentation capabilities can be obtained, laying the foundation for subsequent generation of target sample images. Furthermore, the auxiliary segmentation network and the target mask image can be used as guiding conditions to provide directional guidance for the diffusion model, improving its denoising effect. Consequently, the sample quality of the target sample images generated based on the diffusion model is correspondingly improved, while reducing the difficulty and cost of acquiring target sample images. Moreover, the ultrasound image samples corresponding to the target mask images generated in this embodiment of the invention provide reliable and effective training samples for the subsequent target segmentation model, improving the segmentation capability of the target segmentation model to a certain extent. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of the steps in a method for generating a labeled ultrasound sample image provided in an embodiment of the present invention;

[0053] Figure 2 The comparison results of the IoU (intersection over union) index of the target segmentation model obtained by using the target sample image as the augmented dataset and the target segmentation model obtained without using the augmented dataset in the target object segmentation task;

[0054] Figure 3 The results show the comparison of the Dice (dice similarity coefficient) index in the target object segmentation task between the target segmentation model obtained using the target sample image as the augmented dataset and the target segmentation model obtained without using the augmented dataset.

[0055] Figure 4 This is a flowchart illustrating the specific steps of a method for generating labeled ultrasound sample images according to an embodiment of the present invention.

[0056] Figure 5(a) is a schematic diagram of a target mask image provided in an embodiment of the present invention;

[0057] Figure 5(b) is a schematic diagram of a target sample image provided in an embodiment of the present invention;

[0058] Figure 6 This is a schematic diagram of the structure of a device for generating labeled ultrasound sample images provided in an embodiment of the present invention;

[0059] Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Echocardiography is now widely used in clinical medicine. Patients often undergo it during cardiology consultations or physical examinations. It is one of the most commonly used techniques for diagnosing and monitoring cardiovascular diseases. Due to the increasing number of patients with cardiovascular diseases, the amount of ultrasound data requiring analysis is growing exponentially. Doctors use echocardiogram images to make diagnoses for patients.

[0062] In the existing technology, many methods have been developed for the automatic analysis and interpretation of cardiac ultrasound data to be identified. Deep learning, due to its excellent feature learning capabilities, has been widely used in cardiac ultrasound image segmentation. Current methods include generating ultrasound data using publicly available software packages such as Field II, and GAN-based ultrasound data generation methods are currently widely used.

[0063] However, among the existing solutions, Field II suffers from the drawbacks of lacking corresponding labels for samples and incurring high computational costs. GAN-based ultrasound data generation methods cannot generate labeled samples, have complex training steps, and produce numerous artifacts and false features in the synthesized results, impacting the generation of subsequent cardiac ultrasound images. In summary, existing ultrasound data generation methods cannot generate labeled data and require substantial computational resources or involve highly complex training processes, resulting in high costs.

[0064] To address the aforementioned problems, this invention provides a method, apparatus, device, and medium for generating labeled ultrasound sample images. It aims to solve the issues of existing ultrasound data generation methods being unable to generate labeled ultrasound image samples and requiring substantial computational resources or involving complex training processes, resulting in high costs. The technical concept of this invention is as follows: an unconditional diffusion model is trained using a large number of unlabeled sample images; then, an auxiliary segmentation network is trained using a small number of labeled sample images; finally, the auxiliary segmentation network provides conditional information using gradients at each step of the diffusion model's denoising process, guiding the direction of image generation by the diffusion model, thereby generating the corresponding target sample image.

[0065] Figure 1 This is a flowchart illustrating the steps of a method for generating tagged ultrasound sample images according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method may include:

[0066] Step 101: Obtain the pre-trained diffusion model and auxiliary segmentation network.

[0067] In this embodiment of the invention, a pre-trained diffusion model is obtained. A diffusion model is a latent variable model designed based on the principles of non-equilibrium thermodynamics. The purpose of this model is to learn how to generate target data samples from noise (sampled from a simple distribution). A standard diffusion model consists of two parts: a forward noise addition process and a reverse noise reduction process. The purpose of the forward diffusion process is to destroy the structure of the input features. The purpose of the reverse diffusion process is to reconstruct the feature structure destroyed in the forward process. Before the forward process begins, a maximum diffusion step T is sampled as the maximum number of noise additions in the forward process. Then, Gaussian noise is added to the original input feature X0 in each diffusion step t. When the maximum diffusion step T is reached, the original feature will be transformed into pure noise X0. T .

[0068] The process of pre-training the diffusion model can be as follows:

[0069] First sample images are acquired, which can be obtained from existing image datasets or acquired through image sensors. The first sample images may include unlabeled ultrasound sample images, each containing a target object. Based on the first sample images, a first training network incorporating temporal information is trained. This first training network can be a U-Net network or other forms of neural networks; this embodiment of the invention does not impose limitations on this. Specifically, a sample image can be randomly selected from the first sample images, and a time step can be randomly selected from 1 to T time steps as the time step used for this training. The randomly selected sample image is input into the first training network, and the denoising result of the next time step output by the first training network is obtained. The denoising result is compared with the true labels using a loss function to adjust the parameters of the first training network and update it. Based on the updated first training network, the above training steps are repeated. When a stopping condition is met, the first training network is determined as the diffusion model. For example, the training process of the diffusion model can be optimized using the AdamW optimizer, with an initial learning rate of 0.0001. Cosine Annealing LR is used as the adjustment strategy to adjust the parameters of the first network to be trained. The hyperparameter (batch size) used during training is set to 1, and a total of 200 epochs are trained.

[0070] Obtain a pre-trained auxiliary segmentation network. This network is used to segment the image. The type of object to be segmented may differ depending on the different training tasks used to train the auxiliary segmentation network. For example, the segmentation object of the auxiliary segmentation network could be an organ object in an ultrasound image, such as the heart. The purpose of training the auxiliary segmentation network is to enable it to have good image segmentation capabilities.

[0071] The process of pre-training to generate an auxiliary segmentation network can be as follows:

[0072] A second sample image is acquired, which can be obtained from an existing image dataset or acquired through an image sensor. The second sample image may include labeled sample images, each containing the target object. Based on the second sample image, a second training network is trained. This second training network can be a U-Net network or other forms of neural networks; this embodiment of the invention does not impose limitations. Specifically, a sample image can be randomly selected from the second sample images. The predicted segmentation result output by the second training network is compared with the label corresponding to that sample image, and a loss function is calculated to adjust the parameters of the second training network and update it. Based on the updated second training network, the above training steps are repeated. When a stopping condition is met, this second training network is designated as an auxiliary segmentation network. For example, the training process of the segmentation model can use the Adam optimizer to optimize the second network to be trained, and set the initial learning rate to 0.0001. The StepLR (Step-Learning Rate) adjustment strategy is used to adjust the parameters of the second network to be trained. The hyperparameter (batch size) used in the training process is set to 16, and a total of 200 epochs are trained.

[0073] Since obtaining unlabeled sample images is relatively easy, the diffusion model can be trained based on a large number of unlabeled sample images, i.e., the first sample images. Conversely, because the number of labeled sample images is relatively small, obtaining a large number of labeled images is quite difficult. Therefore, the auxiliary segmentation network can be trained based on a small number of labeled sample images, i.e., the second sample images. This reduces the difficulty of obtaining sample images while ensuring the performance of both the diffusion model and the auxiliary segmentation network.

[0074] In this embodiment of the invention, the diffusion model can be used to generate ultrasound images, and the auxiliary segmentation network can be used to segment organ objects (such as the heart) in the ultrasound images. In other words, the diffusion model and the auxiliary segmentation network in this embodiment of the invention can be applied to the field of ultrasound image research.

[0075] Step 102: Obtain the target mask image and determine the guiding conditions based on the target mask image and the auxiliary segmentation network.

[0076] In this embodiment of the invention, the target mask image can be a binary image composed of 0s and 1s. The target mask image can be used to represent an image region of a specific object. In this embodiment, the specific object can be a segmentation object that needs to be segmented, i.e., the target object. The target mask image can be generated based on a polynomial fitting method. Based on the target mask image and the trained auxiliary segmentation network, the conditional information used by the diffusion model in generating sample images is determined to guide the direction of sample image generation by the diffusion model.

[0077] Understandably, in the field of ultrasound image segmentation, target mask images can be used to characterize image regions of organ objects (such as the heart).

[0078] Step 103: Based on the guiding conditions, generate a target sample image corresponding to the target mask image through the diffusion model; the target mask image is used to characterize the target object contained in the target sample image.

[0079] In this embodiment of the invention, based on guiding conditions, the parameter information of the trained diffusion model is adjusted so that the output of the diffusion model at each step is controlled and predicts along the gradient direction of the desired data distribution. Based on these guiding conditions, the target sample image generated by the diffusion model is the original image corresponding to the target mask image. In other words, since the target mask image is used as the conditional information of the diffusion model, the image generation task of the diffusion model can be regarded as a generation task conditioned on a known image (target mask image). This is equivalent to generating a target mask image based on the diffusion model, which can characterize the real mask corresponding to the target object contained in the target sample image. The target mask image can serve as the real annotation label corresponding to the target sample image, that is, a labeled sample image is obtained. Correspondingly, in the field of ultrasound image segmentation, a labeled ultrasound sample image is obtained.

[0080] In summary, in this embodiment of the invention, by pre-training the diffusion model and the auxiliary segmentation network, good image restoration and segmentation capabilities can be achieved, laying the foundation for subsequent generation of target sample images. Furthermore, the auxiliary segmentation network and the target mask image can serve as guiding conditions, providing directional guidance to the diffusion model to improve its denoising effect. Consequently, the sample quality of the target sample images generated based on the diffusion model is correspondingly improved, while reducing the difficulty and cost of acquiring target sample images. Moreover, the ultrasound image samples corresponding to the target mask images generated in this embodiment of the invention provide reliable and effective training samples for the subsequent target segmentation model, improving the segmentation capability of the target segmentation model to a certain extent.

[0081] Optionally, step 102 may include the following steps:

[0082] Step 1021: Randomly generate extraction parameters, wherein the extraction parameters include the edge vertices of the target object, multiple edge lengths, and multiple random vectors.

[0083] In this embodiment of the invention, a random parameter set is generated. This random parameter set may include edge vertices, multiple edge lengths, and multiple random vectors, representing the vertex coordinates, edge lengths, and corresponding random vectors of random distribution of the target object. The target object may be the left ventricle of the heart; accordingly, the randomly generated extraction parameters may include the edge vertex coordinates Ap corresponding to the left ventricle of the heart. y Edge length L i and random vector Y i , i = 1, 2, 3.

[0084] Step 1022: Perform polynomial fitting on the extracted parameters to obtain multi-segment edge data of the target object.

[0085] In this embodiment of the invention, polynomial fitting is performed on the extracted parameters, and the corresponding marginal polynomials can be obtained using the least squares estimation method. in, Let $\mathbf{i}$ be the Moore-Penrose inverse of the Vandermonde matrix corresponding to the $i$-th polynomial. The curves corresponding to the marginal polynomials are defined as the edge data of the target object.

[0086] Step 1023: Combine the multiple edge data of the target object to obtain the target mask image.

[0087] In this embodiment of the invention, by combining multiple edge data segments obtained from polynomial fitting, the overall edge data corresponding to the target object can be obtained. Based on the overall edge data, the target mask image can be determined. For example, if the number of randomly generated random vectors is three, the three smooth polynomial curves E1, E2, and E3 can be used to represent the edge polynomials of the target object in different edge directions (such as the left, right, and bottom edge polynomials), thereby obtaining the region LV where the target object is located. The formula is as follows:

[0088]

[0089] In this embodiment of the invention, target mask images can be conveniently generated based on a polynomial fitting method. Since this generation method is relatively easy, it is also convenient to generate a large number of target mask images, thereby obtaining a large number of target sample images, which reduces the difficulty of generating target sample images to a certain extent.

[0090] Optionally, step 103 may include the following steps:

[0091] Step 1031: Input the random noise image into the diffusion model.

[0092] In this embodiment of the invention, a random noise image is acquired and used as input to a pre-trained diffusion model. The diffusion model performs denoising processing on the random noise image based on time steps.

[0093] Step 1032: Based on the guiding conditions, adjust the parameter information used by the diffusion model in the prediction process.

[0094] Optionally, step 1032 may include the following steps:

[0095] Step 1032a: Obtain the denoised image generated by the diffusion model during this round of prediction.

[0096] In this embodiment of the invention, during the denoising process of the diffusion model based on a random noise image, the diffusion steps used are from large to small. Therefore, assuming the current time step is t and the image corresponding to the current time step is x... t Then the image corresponding to the next time step can be x t-1 .

[0097] Step 1032b: For the target expectation, obtain the gradient information corresponding to the processing of the denoised image using the auxiliary segmentation network; the target expectation is that the segmentation result of the auxiliary segmentation network is the same as the target mask image.

[0098] In this embodiment of the invention, the gradient information of the auxiliary segmentation network is used as a guiding condition to adjust the parameter information used by the diffusion model, so that the image generated by the diffusion model must satisfy the shape features corresponding to the target mask image (the region of the target object represented by the target mask image) while ensuring that it conforms to the shape distribution. The process of obtaining the gradient information of the auxiliary segmentation network can be: for the denoised image x obtained in the current prediction process corresponding to the current time step... t , will x t Input the target mask image into the auxiliary segmentation network and use it as the prediction image for the auxiliary segmentation network to obtain the auxiliary segmentation network's prediction of the denoised image x. t The processing yields the loss function used for the target mask image, and based on this loss function, the gradient information is determined. This gradient information is actually the denoised image x obtained from the current prediction process based on the auxiliary segmentation network. t The gradient information is obtained by calculating the gradient.

[0099] Step 1032c: Based on the gradient information and the parameter information of the diffusion model, update the parameters of the diffusion model, and obtain the denoised image generated in the next round of prediction based on the diffusion model with updated parameters.

[0100] In this embodiment of the invention, the parameter information of the trained diffusion model is obtained, and based on the gradient information and parameter information, the parameter information required for the next round of prediction is readjusted to update the parameters of the diffusion model. During the prediction process of the diffusion model, the next round of prediction is performed based on the adjusted parameter information to obtain the denoised image generated in the next round of prediction.

[0101] Specifically, the original denoising process for the diffusion model can be:

[0102] x t-1 =μ t +σ t z

[0103] Where, μ t Let σ be the mean of the next time step predicted by the diffusion model. t Let be the standard deviation, and z be random noise.

[0104] In this embodiment of the invention, for the prediction process corresponding to time step t, the diffusion model will obtain an image x corresponding to that time step. t Using the image x corresponding to that time step t To perform the next round of prediction, gradient information is introduced as a guiding condition during its denoising process, and x is adjusted through the process described in the following formula. t Denoising is performed on each point in the data, thus obtaining the denoising process for the fusion guiding condition, as shown in the following equation:

[0105]

[0106] Where y represents the target mask image, σ t σ is the standard deviation, s is the hyperparameter (hyperparameters can be used to characterize the degree of guidance provided by the auxiliary segmentation network to the diffusion model), z is random noise, and p is the standard deviation. Φ (x t ) is the guiding function, p Φ (x t ,t)=‖F(y|x t ,t)‖1,F(y|x t ,t) can represent the effect of the auxiliary segmentation network on the denoised image x. t The loss function used when the output of the processed image matches the target mask image. This represents the denoised image x obtained by the auxiliary segmentation network in this round of prediction. t The gradient information is obtained by calculating the gradient.

[0107] Since the offset direction follows the gradient provided by the auxiliary segmentation network, and this gradient precisely reflects how the model's generation direction should move to maximize the probability of the specified class, the offset distribution better reflects the given class conditions. This is the guiding role of the auxiliary segmentation network.

[0108] Step 1032d: After the T-round prediction process, the denoised image output by the diffusion model is determined as the target sample image.

[0109] In this embodiment of the invention, after T rounds of prediction, the diffusion model determines the final denoised image as the target sample image. This target sample image contains the target object, and the mask image corresponding to the target object is the target mask image. Furthermore, the position of the target image relative to the target sample image is consistent with the position of the mask in the target mask image. For example, steps 1032a-1032c can be iterated from time step t=1000 until time step t=0, obtaining the denoised image output at time step t=0 and determining it as the target sample image x0. Accordingly, since the target sample image is the sample image corresponding to the target mask image, a label pair (x0, y) can be determined based on the target sample image and the target mask image.

[0110] In this embodiment of the invention, for a prior distribution, an additional gradient direction of an auxiliary segmentation network is added to the original gradient direction towards the data direction, so that the mean estimation at each step is added with the gradient guided by the auxiliary segmentation network, and thus the target sample image corresponding to the target mask image can be obtained based on the diffusion model.

[0111] Optionally, embodiments of the present invention may further include the following steps:

[0112] Step 201: Train the segmentation model to be trained based on the target sample image.

[0113] In this embodiment of the invention, a large number of target sample images with labeled tags (target mask images) can be generated through the steps in the above examples. The segmentation model to be trained can be a U-Net network or other forms of neural networks; this embodiment of the invention does not limit this. A segmentation task is constructed for the segmentation model to be trained, which is used to segment the target objects in the target sample images. Based on the segmentation task and the target sample images, the segmentation model to be trained is trained to obtain the target segmentation model.

[0114] Step 202: Obtain the segmentation result of the model to be segmented, and adjust the parameters of the model to be trained based on the segmentation result and the real label information; the real label information is the target mask image corresponding to the target sample image.

[0115] In this embodiment of the invention, to maximize the similarity between the output predicted by the segmentation network to be trained and the actual segmentation result, the parameters of the segmentation network to be trained are adjusted based on the segmentation result and the real label information corresponding to the target sample image. By continuously adjusting the parameters of the segmentation network to be trained, the similarity between the segmentation result output by the segmentation network to be trained and the real label information is made greater than a first similarity threshold. For example, optimization algorithms such as stochastic gradient descent (SGD) and batch gradient descent (BGD) can be used to adjust the parameters of the segmentation network to be trained. The real label information refers to the target mask image.

[0116] Step 203: If the stopping condition is met, the segmentation model to be trained is determined as the target segmentation model.

[0117] In this embodiment of the invention, the stopping condition may include conditions such as the loss value of the segmentation network to be trained reaching a preset threshold, or the number of training rounds of the segmentation network to be trained reaching a preset number of rounds threshold.

[0118] In this embodiment of the invention, by training the segmentation network to be trained, the segmentation network to be trained can learn general image segmentation capabilities during the training process, so as to better automatically segment images containing target objects.

[0119] In one possible implementation, labeled sample images can be pre-obtained from an image dataset as the initial sample image set for the segmentation model to be trained, and multiple target sample images can be used as an augmented dataset to train the segmentation model. Model training using the target sample images as an augmented dataset, as described in this embodiment of the invention, and model training without using an augmented dataset are performed separately. Figure 2 This document presents a comparison of the IoU index in the target object (left ventricle of the heart) segmentation task between a target segmentation model obtained using the target sample images from embodiments of the present invention as an augmented dataset and a target segmentation model obtained without using an augmented dataset. Figure 3 The results show a comparison of the Dice index in the target object (left ventricle of the heart) segmentation task between the target segmentation model obtained using the target sample image as the augmented dataset in the embodiments of the present invention and the target segmentation model obtained without using the augmented dataset.

[0120] For example, in this embodiment of the invention, the target object may be the heart, and the target sample image may be an ultrasound image. Figure 4 A flowchart illustrating a method for generating labeled ultrasound sample images is shown, such as... Figure 2 As shown, in step 1, an unconditional diffusion model is trained based on unlabeled sample images, and an auxiliary segmentation network is trained based on labeled sample images. X in step 1...T X0 can be a noisy image generated based on unlabeled sample images, and X0 can be a denoised image predicted by the unconditional diffusion model. In step 2, after the unconditional diffusion model and the auxiliary segmentation network have been trained, the denoised image x0 corresponding to time step t is obtained. t Then, an auxiliary segmentation network is used to process x. t Segmentation is performed, and the gradient information corresponding to the loss function is determined by combining the target mask image y. This gradient information is used as a guiding condition to guide the unconditional diffusion model in generating the denoised image x at the next time step t-1. t-1 After multiple iterations, the target sample image x0 corresponding to the target mask image generated by the unconditional diffusion model is obtained. The generated target sample image can be shown in Figure 5(b), and the target mask image can be shown in Figure 5(a).

[0121] Figure 6 This is a schematic diagram of the structure of a device for generating tagged ultrasound sample images provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the device may specifically include:

[0122] The first acquisition module 301 is used to acquire the pre-trained diffusion model and the auxiliary segmentation network;

[0123] The second acquisition module 302 is used to acquire a target mask image and determine guiding conditions based on the target mask image and the auxiliary segmentation network;

[0124] The first generation module 303 is used to generate a target sample image corresponding to the target mask image based on the guiding conditions and through the diffusion model; the target mask image is used to characterize the target object contained in the target sample image.

[0125] This invention provides a device for generating labeled ultrasound sample images. By pre-training a diffusion model and an auxiliary segmentation network, it achieves good image restoration and segmentation capabilities, laying the foundation for subsequent generation of target sample images. Furthermore, the auxiliary segmentation network and target mask image can serve as guiding conditions, providing directional guidance to the diffusion model to improve its denoising performance. Consequently, the sample quality of the target sample images generated based on the diffusion model is improved, while simultaneously reducing the difficulty and cost of acquiring target sample images. Moreover, the ultrasound image samples corresponding to the target mask images generated in this invention provide reliable and effective training samples for subsequent target segmentation models, enhancing the segmentation capability of the target segmentation model to a certain extent.

[0126] Optionally, the second acquisition module 302 includes:

[0127] The first generation submodule is used to randomly generate extraction parameters, wherein the extraction parameters include the edge vertices of the target object, multiple edge lengths, and multiple random vectors.

[0128] The first fitting module is used to perform polynomial fitting on the extracted parameters to obtain multi-segment edge data of the target object;

[0129] The first combination module is used to combine multiple edge data of the target object to obtain the target mask image.

[0130] Optionally, the first generation module 303 includes:

[0131] The first input module is used to input the random noise image into the diffusion model;

[0132] The first adjustment module is used to adjust the parameter information used by the diffusion model in the prediction process based on the guiding conditions.

[0133] The first processing module is used to perform denoising processing on the random noise image based on the diffusion model with adjusted parameters to obtain the target sample image.

[0134] Optionally, the first adjustment module includes:

[0135] The first acquisition submodule is used to acquire the denoised image generated by the diffusion model in this round of prediction.

[0136] The second acquisition submodule is used to acquire gradient information corresponding to the processing of the denoised image by the auxiliary segmentation network, based on the target expectation; the target expectation is that the segmentation result of the auxiliary segmentation network is the same as the target mask image.

[0137] The first update module is used to update the parameters of the diffusion model based on the gradient information and the parameter information of the diffusion model, and to obtain the denoised image generated in the next round of prediction based on the diffusion model with updated parameters.

[0138] The first determining module is used to determine the denoised image output by the diffusion model as the target sample image after performing T rounds of prediction.

[0139] Optionally, the device further includes:

[0140] The first training module is used to train the segmentation model to be trained based on the target sample image;

[0141] The third acquisition module is used to acquire the segmentation result of the model to be segmented, and to adjust the parameters of the segmentation model to be trained based on the segmentation result and the real label information; the real label information is the target mask image corresponding to the target sample image.

[0142] The second determining module is used to determine the segmentation model to be trained as the target segmentation model when the stopping condition is met.

[0143] The present invention also provides an electronic device, see [link to relevant documentation]. Figure 7 The system includes: a processor 401, a memory 402, and a computer program 4021 stored in the memory and executable on the processor. When the processor executes the program, it implements the method for generating labeled ultrasound sample images according to the foregoing embodiments.

[0144] The present invention also provides a readable storage medium, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is able to perform the method for generating labeled ultrasound sample images of the foregoing embodiments.

[0145] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0146] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0147] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0148] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0149] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0150] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0151] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0152] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0153] It should be noted that all actions involving the acquisition of signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and policies of the country where the invention is located, and with authorization from the owner of the relevant device.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0155] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating labeled ultrasound sample images, characterized in that, The method includes: A pre-trained diffusion model and an auxiliary segmentation network are obtained; the diffusion model is trained from a first sample image, which includes unlabeled ultrasound sample images; the auxiliary segmentation network is trained from a second sample image, which includes labeled sample images; the first sample image and the second sample image each contain a target object. Acquire a target mask image, and determine guiding conditions based on the target mask image and the auxiliary segmentation network; Based on the guiding conditions, a target sample image corresponding to the target mask image is generated through the diffusion model; the target mask image is used to characterize the target object contained in the target sample image; The step of generating a target sample image corresponding to the target mask image through the diffusion model based on the guiding conditions includes: Input the random noise image into the diffusion model; Based on the guiding conditions, adjust the parameter information used by the diffusion model in the prediction process; The random noise image is denoised based on the diffusion model with adjusted parameters to obtain the target sample image; The step of adjusting the parameter information used by the diffusion model in the prediction process based on the guiding conditions includes: Obtain the denoised image generated by the diffusion model during this round of prediction; To achieve the desired outcome, the gradient information corresponding to the processing of the denoised image using the auxiliary segmentation network is obtained; the desired outcome is that the segmentation result of the auxiliary segmentation network is the same as the target mask image. Based on the gradient information and the parameter information of the diffusion model, the parameters of the diffusion model are updated, and the denoised image generated in the next round of prediction is obtained based on the diffusion model with updated parameters. After T rounds of prediction, the denoised image output by the diffusion model is determined as the target sample image.

2. The method according to claim 1, characterized in that, The diffusion model is trained based on the first unlabeled sample image, and the auxiliary segmentation network is trained based on the second labeled sample image.

3. The method according to claim 1, characterized in that, The acquisition of the target mask image may include: Extraction parameters are randomly generated, wherein the extraction parameters include the edge vertices of the target object, multiple edge lengths, and multiple random vectors; Polynomial fitting is performed on the extracted parameters to obtain multi-segment edge data of the target object; The target mask image is obtained by combining multiple edge data segments of the target object.

4. The method according to claim 1, characterized in that, The method further includes: Based on the target sample image, the segmentation model to be trained is trained; Obtain the segmentation result of the segmentation model to be trained, and adjust the parameters of the segmentation model to be trained based on the segmentation result and the ground truth label information corresponding to the target sample image; the ground truth label information is the target mask image corresponding to the target sample image. If the stopping condition is met, the segmentation model to be trained is determined as the target segmentation model that has learned general image segmentation capabilities.

5. A device for generating labeled ultrasound sample images, characterized in that, The device includes: The first acquisition module is used to acquire a pre-trained diffusion model and an auxiliary segmentation network; the diffusion model is trained from a first sample image, which includes unlabeled ultrasound sample images; the auxiliary segmentation network is trained from a second sample image, which includes labeled sample images; the first sample image and the second sample image each contain a target object. The second acquisition module is used to acquire a target mask image and determine guiding conditions based on the target mask image and the auxiliary segmentation network; The first generation module is used to generate a target sample image corresponding to the target mask image based on the guiding conditions and through the diffusion model; the target mask image is used to characterize the target object contained in the target sample image; The first generation module includes: The first input module is used to input the random noise image into the diffusion model; The first adjustment module is used to adjust the parameter information used by the diffusion model in the prediction process based on the guiding conditions. The first processing module is used to perform denoising processing on the random noise image based on the diffusion model with adjusted parameters to obtain the target sample image; The first adjustment module includes: The first acquisition submodule is used to acquire the denoised image generated by the diffusion model in this round of prediction. The second acquisition submodule is used to acquire gradient information corresponding to the processing of the denoised image by the auxiliary segmentation network, based on the target expectation; the target expectation is that the segmentation result of the auxiliary segmentation network is the same as the target mask image. The first update module is used to update the parameters of the diffusion model based on the gradient information and the parameter information of the diffusion model, and to obtain the denoised image generated in the next round of prediction based on the diffusion model with updated parameters. The first determining module is used to determine the denoised image output by the diffusion model as the target sample image after performing T rounds of prediction.

6. The apparatus according to claim 5, characterized in that, The second acquisition module includes: The first generation submodule is used to randomly generate extraction parameters, wherein the extraction parameters include the edge vertices of the target object, multiple edge lengths, and multiple random vectors; The first fitting module is used to perform polynomial fitting on the extracted parameters to obtain multi-segment edge data of the target object; The first acquisition submodule is used to combine multiple edge data of the target object to obtain the target mask image.

7. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method for generating labeled ultrasound sample images as described in any one of claims 1-4.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method for generating labeled ultrasound sample images as described in any one of claims 1-4.

Images