Ultrasonic image simulation model training method, generation method, device and medium

Abstract

The present disclosure provides a simulation model training method, generation method, device and medium for ultrasound images, the training method comprising: obtaining several sample ultrasound images of a sample object; obtaining a sample mask image of each sample ultrasound image; inputting each sample mask image into a preset network model, outputting the corresponding first ultrasound image and second ultrasound image; comparing the first ultrasound image and the second ultrasound image of each group with the corresponding sample ultrasound image respectively to obtain a comparison result; iteratively updating the preset network model based on different comparison results until the preset model training conditions are met to obtain an ultrasound image simulation model. The present disclosure uses sample ultrasound images and sample mask images of a target object to train an ultrasound image simulation model; based on the model and the three-dimensional model of the target object, ultrasound images of various positions and angles in the target object are simulated to improve the matching degree between the ultrasound image and the specific structure of the target object.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a simulation model training method, generation method, device and medium for ultrasonic images. Background Art

[0002] In intracardiac surgery, the correct use of ultrasound catheters and accurate identification of ultrasound images are important guarantees for the success of the operation. Ultrasound catheters are advanced medical tools that introduce high-definition ultrasound probes into the heart through vascular intervention to provide high-resolution, real-time cardiac ultrasound images. This technology plays a key role in surgical navigation, helping doctors to accurately observe heart structures, locate and identify abnormal areas, and guide guidewire operations and interventional device implantation. This technology is widely used in cardiac surgery, coronary intervention, and structural heart disease repair. By using ultrasound catheters and accurately identifying ultrasound images, doctors can perform intracardiac surgery more safely and effectively, thereby improving the quality of surgical outcomes and enhancing the quality of life of patients.

[0003] However, learning intracardiac ultrasound imaging is a challenging task for doctors. As the anatomical structure of the heart is complex and varies between individuals, it becomes difficult to understand and learn ultrasound images. The current ultrasound simulation technology is still imperfect and cannot effectively cultivate doctors' ability to learn intracardiac ultrasound imaging. Existing simulation images have problems such as too clear edges, inaccurate correspondence with the heart's anatomical structure, and inability to simulate some ultrasound angles, which limits the role of ultrasound simulation teaching in accumulating doctors' experience in clinical practice. Summary of the invention

[0004] The technical problem to be solved by the present disclosure is to overcome the defects of the existing simulation images in the prior art, such as the edges are too clear, the cardiac anatomical structure cannot be accurately corresponded, and some ultrasonic angles cannot be simulated, and provide a simulation model training method, generation method, device and medium for ultrasonic images.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] The present disclosure provides a training method for an ultrasound image simulation model, the training method comprising:

[0007] Acquire a plurality of sample ultrasound images of the sample object;

[0008] Acquire a sample mask image for each of the sample ultrasound images;

[0009] Input each of the sample mask images into a preset network model, and output a corresponding first ultrasonic image and a second ultrasonic image;

[0010] The preset network model includes a first preset network and a second preset network, the first preset network is used to learn contour information in an ultrasonic image, and the second preset network is used to learn texture information in an ultrasonic image;

[0011] Comparing the first ultrasonic image and the second ultrasonic image of each group with the corresponding sample ultrasonic image to obtain a comparison result;

[0012] The preset network model is iteratively updated based on different comparison results until preset model training conditions are met to obtain the ultrasound image simulation model.

[0013] Preferably, the first preset network and the second preset network both adopt generative adversarial networks.

[0014] Preferably, the step of inputting each of the sample mask images into a preset network model and outputting the corresponding first ultrasonic image and second ultrasonic image comprises:

[0015] Inputting the sample mask image into the first preset network to output the first ultrasound image;

[0016] Inputting the first ultrasound image into the second preset network to output the second ultrasound image;

[0017] The step of comparing the first ultrasonic image and the second ultrasonic image of each group with the corresponding sample ultrasonic image to obtain a comparison result comprises:

[0018] Calculating a first loss value between the first ultrasonic image and the sample ultrasonic image based on a first loss function;

[0019] Calculating a second loss value between the second ultrasonic image and the sample ultrasonic image based on a second loss function;

[0020] The step of iteratively updating the preset network model based on different comparison results until the preset model training conditions are met to obtain the ultrasound image simulation model includes:

[0021] Iteratively updating network parameters of the first preset network based on the first loss value to reduce the first loss value;

[0022] Iteratively updating the network parameters of the second preset network based on the second loss value to reduce the second loss value;

[0023] When the first loss value is less than a first preset threshold value, and the second loss value is less than a second preset threshold value, it is determined that the preset model training condition is met to obtain the ultrasound image simulation model.

[0024] Preferably, the step of acquiring a sample mask image of each sample ultrasound image comprises:

[0025] Performing denoising processing on the sample ultrasonic image to obtain a third ultrasonic image;

[0026] Perform threshold segmentation processing on the third ultrasonic image to obtain the sample mask image.

[0027] Preferably, the feature is that the sample object includes a preset organ and / or a preset tissue.

[0028] Preferably, the predetermined organ includes a heart.

[0029] The present disclosure also provides a method for simulating and generating an ultrasound image, the method comprising:

[0030] Acquire a target mask image of the target object;

[0031] Inputting the target mask image into an ultrasound image simulation model to simulate and generate a target ultrasound image;

[0032] Wherein, the ultrasound image simulation model is trained using the training method as described above.

[0033] Preferably, the step of acquiring a target mask image of the target object comprises:

[0034] Acquire a three-dimensional model of the target object;

[0035] Taking the target position in the three-dimensional model as the vertex and the preset length as the radius, obtaining a simulated sector corresponding to the target angle;

[0036] Obtaining all intersection points of the simulated sector and the three-dimensional model to obtain an initial point set;

[0037] Using a preset screening algorithm to screen the initial point set to obtain a plurality of target point sets;

[0038] Determine a corresponding target contour according to each point in the target point set and two radii of the simulated sector;

[0039] The target mask image is obtained according to each of the target contours.

[0040] Preferably, the step of using a preset screening algorithm to screen the initial point set to obtain a plurality of target point sets comprises:

[0041] Randomly select a point in the initial point set as a seed point, and add the seed point to the first point set;

[0042] Using a greedy algorithm, a point closest to the seed point is selected as the target point;

[0043] When the distance between the target point and the seed point is less than or equal to a preset distance, adding the target point to the first point set;

[0044] The target point is used as a new seed point, and the greedy algorithm is repeatedly executed to select a point closest to the seed point as the target point; when the distance between the target point and the seed point is less than or equal to a preset distance, the target point is added to the first point set until the distance between the target point and the seed point is greater than the preset distance, and the current first point set is used as the target point set.

[0045] Preferably, the step of determining the corresponding target contour according to each point in the target point set and the two radii of the simulated sector comprises:

[0046] If the distance between the first seed point and the last target point in the target point set is less than or equal to the preset distance, connecting the points in the target point set to obtain the target contour;

[0047] If the distance between the first seed point and the last target point in the target point set is greater than the preset distance, and two points in the target point set fall on the same radius of the simulated sector, then connect the points in the target point set and a radius where the simulated sector and the target point set intersect to obtain the target contour;

[0048] If the distance between the first seed point and the last target point in the target point set is greater than the preset distance, and two points in the target point set fall on two radii of the simulated sector respectively, then the points in the target point set and the two radii of the simulated sector are connected to obtain the target contour.

[0049] The present disclosure also provides a training system for an ultrasound image simulation model, the training system comprising:

[0050] A first sample acquisition module, used to acquire a plurality of sample ultrasound images of a sample object;

[0051] A second sample acquisition module, used to acquire a sample mask image of each sample ultrasound image;

[0052] An ultrasonic image output module, used for inputting each of the sample mask images into a preset network model, and outputting a corresponding first ultrasonic image and a second ultrasonic image;

[0053] The preset network model includes a first preset network and a second preset network, the first preset network is used to learn contour information in an ultrasonic image, and the second preset network is used to learn texture information in an ultrasonic image;

[0054] a comparison module, used for comparing the first ultrasonic image and the second ultrasonic image of each group with the corresponding sample ultrasonic image to obtain a comparison result;

[0055] A training module is used to iteratively update the preset network model based on different comparison results until the preset model training conditions are met to obtain the ultrasound image simulation model.

[0056] Preferably, the first preset network and the second preset network both adopt generative adversarial networks.

[0057] Preferably, the ultrasound image output module further includes:

[0058] a first output unit, configured to input the sample mask image into the first preset network to output the first ultrasound image;

[0059] a second output unit, configured to input the first ultrasound image into the second preset network to output the second ultrasound image;

[0060] Preferably, the comparison module further includes:

[0061] A first loss calculation unit, configured to calculate a first loss value between the first ultrasonic image and the sample ultrasonic image based on a first loss function;

[0062] a second loss calculation unit, configured to calculate a second loss value between the second ultrasonic image and the sample ultrasonic image based on a second loss function;

[0063] Preferably, the training module also includes:

[0064] A first parameter updating unit, configured to iteratively update a network parameter of the first preset network based on the first loss value to reduce the first loss value;

[0065] A second parameter updating unit, configured to iteratively update a network parameter of the second preset network based on the second loss value to reduce the second loss value;

[0066] The training completion determination unit is used to determine that the preset model training conditions are met when the first loss value is less than a first preset threshold and the second loss value is less than a second preset threshold, so as to obtain the ultrasound image simulation model.

[0067] Preferably, the second sample acquisition module further includes:

[0068] a denoising unit, configured to perform denoising processing on the sample ultrasonic image to obtain a third ultrasonic image;

[0069] The segmentation unit is used to perform threshold segmentation processing on the third ultrasonic image to obtain the sample mask image.

[0070] Preferably, the sample object includes a preset organ and / or a preset tissue.

[0071] Preferably, the predetermined organ includes a heart.

[0072] The present disclosure also provides a simulation generation system for ultrasound images, the simulation generation system comprising:

[0073] A target acquisition module, used to acquire a target mask image of a target object;

[0074] An ultrasound simulation module, used for inputting the target mask image into an ultrasound image simulation model to simulate and generate a target ultrasound image;

[0075] Wherein, the ultrasound image simulation model is trained using the training system as mentioned above.

[0076] Preferably, the target acquisition module further includes:

[0077] A three-dimensional model acquisition unit, used to acquire a three-dimensional model of the target object;

[0078] A simulated sector acquisition unit, used to acquire a simulated sector corresponding to a target angle by taking the target position in the three-dimensional model as a vertex and a preset length as a radius;

[0079] An initial point set acquisition unit, used to acquire all intersection points of the simulation sector and the three-dimensional model to obtain an initial point set;

[0080] A target point set acquisition unit, used to use a preset screening algorithm to screen the initial point set to obtain a plurality of target point sets;

[0081] A target contour acquisition unit, used for determining a corresponding target contour according to each point in the target point set and two radii of the simulated sector;

[0082] The target mask acquisition unit is used to obtain the target mask image according to each of the target contours.

[0083] Preferably, the target point set acquisition unit further includes:

[0084] A seed point acquisition subunit, used for randomly selecting a point in the initial point set as a seed point, and adding the seed point to the first point set;

[0085] A target point acquisition subunit is used to select a point closest to the seed point as the target point by using a greedy algorithm;

[0086] a distance judgment subunit, configured to add the target point to the first point set when the distance between the target point and the seed point is less than or equal to a preset distance;

[0087] The target point set acquisition subunit is used to adopt the target point as a new seed point, and repeatedly execute the greedy algorithm to select a point closest to the seed point as the target point; when the distance between the target point and the seed point is less than or equal to the preset distance, the target point is added to the first point set until the distance between the target point and the seed point is greater than the preset distance, and the current first point set is used as the target point set.

[0088] Preferably, the target contour acquisition unit further includes:

[0089] A first contour acquisition subunit is used to connect the points in the target point set to obtain the target contour when the distance between the first seed point and the last target point in the target point set is less than or equal to the preset distance;

[0090] A second contour acquisition subunit is configured to connect the points in the target point set and a radius at which the simulated sector and the target point set intersect when the distance between the first seed point and the last target point in the target point set is greater than the preset distance and two points in the target point set fall on the same radius of the simulated sector to obtain the target contour;

[0091] The third contour acquisition subunit is used to connect the points in the target point set and the two radii of the simulated fan to obtain the target contour when the distance between the first seed point and the last target point in the target point set is greater than the preset distance and two points in the target point set fall on the two radii of the simulated fan respectively.

[0092] The present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and used to run on the processor, wherein the processor implements the above-mentioned training method of the ultrasound image simulation model, or the simulation generation method of the ultrasound image when executing the computer program.

[0093] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer program implements the above-mentioned training method of the ultrasound image simulation model, or the simulation generation method of the ultrasound image.

[0094] The present disclosure also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned training method of the ultrasound image simulation model, or the simulation generation method of the ultrasound image.

[0095] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0096] The positive progressive effect of the present disclosure is that: sample ultrasound images of target objects such as the heart and their sample mask images are used to train a preset network model composed of two layers of networks to obtain an ultrasound image simulation model; based on the ultrasound image simulation model and the three-dimensional model of the target object such as the heart, ultrasound images of various positions and angles inside the target object can be simulated, thereby improving the degree of matching between the ultrasound image and the specific structure of the target object, providing doctors with an interactive, high-precision, and realistic ultrasound simulation system, making the simulated ultrasound images real-time and highly simulated, and improving the efficiency and quality of doctors' learning of ultrasound images. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 This is a flowchart of the training method of the ultrasound image simulation model of Example 1 of the present disclosure.

[0098] Figure 2 An example diagram of an ultrasound image of the present disclosure.

[0099] Figure 3 It is a schematic diagram of the structure of the first preset network disclosed in the present invention.

[0100] Figure 4 It is a schematic diagram of the structure of the second preset network of the present invention.

[0101] Figure 5 This is the first flow chart of the method for simulating and generating an ultrasound image according to the second embodiment of the present disclosure.

[0102] Figure 6 This is a second flow chart of the method for simulating and generating an ultrasound image according to Embodiment 2 of the present disclosure.

[0103] Figure 7 An example diagram for determining the closedness of ultrasound image contours for the present disclosure.

[0104] Figure 8 An example diagram of filling the outline of an ultrasound image for the present disclosure.

[0105] Fig. 9 It is a schematic diagram of the structure of the simulator device disclosed in the present invention.

[0106] Fig.10 It is a schematic diagram of the structure of the manipulator device disclosed in the present invention.

[0107] Fig.11 This is a module diagram of a training system for an ultrasound image simulation model according to Embodiment 3 of the present disclosure.

[0108] Fig.12 This is a module diagram of the simulation generation system of ultrasound images of Example 4 of the present disclosure.

[0109] Fig.13 This is a schematic diagram of the structure of an electronic device according to Embodiment 5 of the present disclosure. DETAILED DESCRIPTION

[0110] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0111] Prefixes such as "first" and "second" are used in the embodiments of the present disclosure only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present disclosure does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitation should be constituted due to the use of such prefixes. In addition, in the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0112] In the embodiments of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0113] Example 1

[0114] The present disclosure provides a training method for an ultrasound image simulation model, such as Figure 1 As shown, the training method includes the following steps:

[0115] S1. Acquire several sample ultrasound images of a sample object.

[0116] The sample objects include preset organs, preset tissues, etc.

[0117] The intended organs include but are not limited to the heart.

[0118] For example, when the sample object is a heart, several intracardiac ultrasound images are acquired as sample ultrasound images.

[0119] S2. Obtain a sample mask image for each sample ultrasound image.

[0120] like Figure 2As shown, the left figure shows the original sample ultrasound image; NL-means (a non-local mean filtering algorithm) can be used to denoise the sample ultrasound image to obtain the denoised image shown in the middle figure; and then the image threshold segmentation method is used to generate a sample mask image of the intracardiac tissue as shown in the right figure.

[0121] S3, inputting each sample mask image into a preset network model, and outputting a corresponding first ultrasonic image and a second ultrasonic image;

[0122] The preset network model includes a first preset network and a second preset network, the first preset network is used to learn contour information in an ultrasonic image, and the second preset network is used to learn texture information in an ultrasonic image.

[0123] The preset network model includes a first preset network and a second preset network. The first preset network is mainly used to learn the surrounding information of the ultrasonic image contour at low resolution to obtain the first ultrasonic image; the second preset network is mainly used to learn the texture information in the ultrasonic image at high resolution to obtain the second ultrasonic image.

[0124] S4. Compare the first ultrasonic image and the second ultrasonic image of each group with the corresponding sample ultrasonic image to obtain a comparison result.

[0125] The sample mask image of each group is input into the preset network model, and the first ultrasonic image output by the first preset network is compared with the sample ultrasonic image to obtain a comparison result; the second ultrasonic image output by the second preset network is compared with the sample ultrasonic image to obtain another comparison result.

[0126] S5. Iteratively update the preset network model based on different comparison results until the preset model training conditions are met to obtain an ultrasound image simulation model.

[0127] The preset network model is iteratively updated based on the comparison results, so that the first ultrasonic image and the second ultrasonic image output by the model are closer and closer to the sample ultrasonic image, that is, until the preset model training conditions are met, so as to obtain an ultrasonic image simulation model. The ultrasonic image simulation model can output a corresponding ultrasonic image according to the input mask image.

[0128] In this solution, a preset network model is trained through sample mask images and sample ultrasound images to obtain an ultrasound image simulation model that can learn the contour information and texture information of the ultrasound image, thereby improving the accuracy of ultrasound image simulation.

[0129] In an implementable solution, both the first preset network and the second preset network adopt generative adversarial networks.

[0130] When both the first preset network and the second preset network adopt a generative adversarial network, the obtained ultrasound image simulation model is a two-stage GAN (generative adversarial network) model. The following takes a specific model as an example to illustrate the training process of the ultrasound image simulation model:

[0131] (1) Build and train a first-stage GAN (i.e., the first preset network);

[0132] The one-stage generative adversarial network (GAN) is mainly used to learn the surrounding information of ultrasound image contours at low resolution. First, the original ultrasound image (i.e., the sample ultrasound image) is cropped and reduced to a size of 64x 64. Next, the generator G in the one-stage GAN is used to I and discriminator D I The original image features are learned. The generator is mainly composed of residual blocks, and the discriminator is mainly composed of convolutional layers and pooling layers. Figure 3 As shown, the left figure shows the structure of a one-stage GAN generator, and the right figure shows the structure of a one-stage GAN discriminator, where Conv is a convolutional layer, Residual Block is a residual network layer, Max Pool is a pooling layer, and Softmax is a transfer function layer.

[0133] In the one-stage GAN, the generator loss function L G and the discriminator loss function L D The formula is:

[0134]

[0135] Where n is the batch size, x is the ultrasound image after the sample ultrasound image is downsampled to 64x 64, and x ij is the pixel on the image x, G I (x) is the generated image (i.e., the first ultrasound image), D I (x) is the discrimination score of the image, λ I is the pixel weight parameter.

[0136] In one-stage GAN training, we can adopt a batch size of 512 and choose Adam (an optimizer) as the optimizer for both the generator and the discriminator with a learning rate of 0.001.

[0137] (2) construct and train a two-stage GAN (i.e., the second preset network);

[0138] The two-stage generative adversarial network (GAN) is mainly used to learn the texture information of ultrasound images and upsample the low-resolution generated images to 512x512. The network first receives the image G generated by the first-stage GAN I(x) as input, and then passes through the generator G II and discriminator D II The original image features are learned. The generator is mainly composed of residual blocks and upsampling layers, and the discriminator is mainly composed of convolutional layers, pooling layers, and fully connected layers. Figure 4 As shown, the left figure shows the structure of the two-stage GAN generator, and the right figure shows the structure of the two-stage GAN discriminator. Among them, Conv is the convolution layer, ResidualBlock is the residual network layer, Upsampling is the upsampling layer, and FC is the fully connected layer.

[0139] Similarly, in the two-stage GAN, the generator loss function L G and the discriminator loss function L D The formula can be expressed as:

[0140]

[0141]

[0142] Among them, G II (G I (x)) is the image generated by the two-stage GAN (i.e., the second ultrasound image), λ II is the pixel weight parameter of the two-stage GAN. In the two-stage GAN training, we can use a batch size of 64, and we can also choose Adam as the optimizer for the generator and discriminator, with a learning rate of 0.0002.

[0143] Of course, the above model structure, parameters, etc. are only examples and can be adjusted according to the needs of model training.

[0144] In this scheme, an ultrasound image simulation model is constructed by a two-layer generative adversarial network, which can improve the performance of the model and thus improve the simulation accuracy of ultrasound images.

[0145] In one feasible solution, step S3 includes:

[0146] Inputting the sample mask image into a first preset network to output a first ultrasound image;

[0147] The first ultrasound image is input into a second preset network to output a second ultrasound image.

[0148] Step S4 includes:

[0149] Based on the first loss function, a first loss value between the first ultrasound image and the sample ultrasound image is calculated.

[0150] That is, the first ultrasonic image output by the first preset network is compared with the sample ultrasonic image.

[0151] Based on the second loss function, a second loss value between the second ultrasound image and the sample ultrasound image is calculated.

[0152] That is, the second ultrasonic image output by the second preset network is compared with the sample ultrasonic image.

[0153] Step S5 includes:

[0154] Iteratively updating network parameters of the first preset network based on the first loss value to reduce the first loss value;

[0155] Iteratively updating network parameters of the second preset network based on the second loss value to reduce the second loss value;

[0156] When the first loss value is less than the first preset threshold value, and the second loss value is less than the second preset threshold value, it is determined that the preset model training condition is met to obtain the ultrasound image simulation model.

[0157] That is, the ultrasound image simulation model is continuously updated iteratively so that the ultrasound image simulated by the model becomes closer and closer to the actual sample ultrasound image.

[0158] In this scheme, the first ultrasonic image and the second ultrasonic image output by the model are compared with the sample ultrasonic image respectively, and the ultrasonic image simulation model is updated based on the two comparison results. This can improve the learning accuracy of the model for the contour information and texture information of the ultrasonic image and improve the accuracy of the ultrasonic image simulation.

[0159] In one feasible solution, step S2 includes:

[0160] The sample ultrasonic image is subjected to denoising processing to obtain a third ultrasonic image.

[0161] NL-means may be used to perform denoising on the sample ultrasonic image to obtain a third ultrasonic image.

[0162] Perform threshold segmentation processing on the third ultrasonic image to obtain a sample mask image.

[0163] The third ultrasonic image is then segmented using an image threshold segmentation method to obtain a sample mask image.

[0164] In this solution, by denoising and segmenting the sample ultrasound image, the accuracy of the obtained sample mask image can be improved, thereby improving the accuracy of the model training.

[0165] The training method of the ultrasound image simulation model provided in this embodiment, by inputting a sample mask image into a preset network model, comparing the first ultrasound image and the second ultrasound image output by the model with the sample ultrasound image respectively, and updating the ultrasound image simulation model based on the two comparison results, can improve the model's learning accuracy of the contour information and texture information of the ultrasound image, improve the accuracy of the ultrasound image simulation model training, and thereby improve the accuracy of the ultrasound image simulation.

[0166] Example 2

[0167] This embodiment provides a method for simulating and generating an ultrasonic image. Figure 5 As shown, the simulation generation method includes the following steps:

[0168] S6. Acquire a target mask image of the target object.

[0169] For example, a mask image at a certain position and angle inside the heart is obtained.

[0170] S7, inputting the target mask image into the ultrasound image simulation model to simulate and generate a target ultrasound image;

[0171] The ultrasound image simulation model is trained using the training method in Example 1.

[0172] By inputting the target mask image into the ultrasound image simulation model, the corresponding target ultrasound image can be simulated.

[0173] In this solution, the ultrasound image is simulated and generated by the ultrasound image simulation model, which can improve the convenience and accuracy of ultrasound image acquisition.

[0174] In one feasible solution, Figure 6 As shown, step S6 includes:

[0175] S601: Acquire a three-dimensional model of a target object.

[0176] The CT (computed tomography) image of the target object can be acquired first, and then the image threshold segmentation and Marching Cubes (voxel-based 3D reconstruction method) can be used to perform 3D reconstruction of different parts of the CT image. For example, the left atrium, left ventricle, right atrium, right ventricle, aorta, pulvinar, superior vena cava and heart shell of the heart can be reconstructed to obtain a 3D model of the heart.

[0177] S602: Taking the target position in the three-dimensional model as the vertex and the preset length as the radius, obtain a simulated sector corresponding to the target angle.

[0178] S603: Obtain all intersection points of the simulation sector and the three-dimensional model to obtain an initial point set.

[0179] The initial point set is connected to form the outline of the three-dimensional heart model on the simulated sector surface.

[0180] S604: Use a preset screening algorithm to screen the initial point set to obtain several target point sets.

[0181] S605: Determine the corresponding target contour according to each point in each target point set and two radii of the simulated sector.

[0182] S606: Obtain a target mask image according to each target contour.

[0183] Steps S604 to S606 further process the contour on the simulated sector, determine the connected domain by comparing the positional relationship between each point on the contour and the boundary of the sector, and distinguish between tissue structures and non-tissue structures, so as to obtain a target mask image.

[0184] In this scheme, based on the three-dimensional model of the target object, the target contour on the simulated sector surface at any position and angle can be obtained, and based on the contour, the tissue structure and non-tissue structure of the target object can be determined to obtain the target mask image, thereby improving the flexibility of ultrasound image simulation.

[0185] In one feasible solution, step S604 includes:

[0186] Randomly select a point in the initial point set as a seed point and add the seed point to the first point set;

[0187] Using the greedy algorithm, a point closest to the seed point is selected as the target point;

[0188] When the distance between the target point and the seed point is less than or equal to the preset distance, the target point is added to the first point set;

[0189] The target point is used as a new seed point, and the greedy algorithm is repeatedly executed to select a point closest to the seed point as the target point; when the distance between the target point and the seed point is less than or equal to the preset distance, the target point is added to the first point set until the distance between the target point and the seed point is greater than the preset distance, and the current first point set is used as the target point set.

[0190] Taking the three-dimensional model of the heart as an example, the computer three-dimensional simulation technology can be used to simulate the movement of the virtual probe between the heart models and obtain the point set P where the models of different parts of the heart intersect with the simulated fan i By applying the inverse matrix transformation of the simulated fan to the point set P i , we can map these points from three-dimensional space to the plane and obtain a two-dimensional point set P i ′, that is, a set of two-dimensional expressions of the structural contours of different parts of the heart are obtained.

[0191] For a two-dimensional point set P i ′ Perform rounding and deduplication operations to obtain the pixel coordinate set P i ″ (i.e. the initial point set). i ″, randomly select a seed point p0 as the initial seed point, and define {p0|p0∈P ij ″}. According to the Manhattan distance formula, calculate other points p i The distance d from the seed point p0 i . Through the greedy algorithm, select the minimum distance p i As a candidate seed point (i.e., target point), if d i ≤α (where α represents the category value range), then p i Add to the point set, so {p i |p i ∈P ij ″}, then let p i As the new seed point, continue to select the next nearest neighbor point until the nearest neighbor distance d is found. k >α, we can transform P ij (i.e., the target point set) is regarded as the outline of an ordered independent point set.

[0192] In one feasible solution, step S605 includes:

[0193] If the distance between the first seed point and the last target point in the target point set is less than or equal to the preset distance, then the points in the target point set are connected to obtain the target contour;

[0194] If the distance between the first seed point and the last target point in the target point set is greater than the preset distance, and two points in the target point set fall on the same radius of the simulated sector, then connect the points in the target point set and a radius where the simulated sector and the target point set intersect to obtain the target contour;

[0195] If the distance between the first seed point and the last target point in the target point set is greater than the preset distance, and two points in the target point set fall on the two radii of the simulated sector respectively, then the points in the target point set and the two radii of the simulated sector are connected to obtain the target contour.

[0196] like Figure 7 As shown, we need to determine whether each point set contour is closed. ij If the first point p0 and the last point p n If the distance d≤α, the contour is considered closed (i.e. Figure 7); otherwise it is not closed. Next, we need to determine the point p0 and the point p n Are they on the same side of the fan. If the two points are not on the same side, we need to add fan vertices to close the contour. We determine whether they are on the same side by comparing the slope between the two points. If |y n -y0 / x n -x0|≥β, it means that the two points are on the same side (i.e. Figure 7 The fan-shaped edge is used to close the contour; otherwise, it is located on a different edge (i.e. Figure 7 ), the contour is closed using the vertex and two edges of the fan.

[0197] Finally, if Figure 8 As shown in the figure, we use the contour filling method to fill the interior of the ordered contour point set to generate a MASK (mask) mask image of the heart structure. For example, we use white to fill the interior of the point set contour of the heart shell to present the myocardial tissue; at the same time, we use black to fill the interior of the point set contour of other structures to present the chamber.

[0198] When the ultrasound simulation model is used for intracardiac ultrasound simulation, the simulation device may include: Fig. 9 The simulator device shown and Fig.10 The manipulator device shown.

[0199] (1) Fig. 9 As shown, the simulator device is a device for acquiring a three-dimensional model of a target object and simulating and generating a target ultrasound image, and mainly includes: a bus 91 , a first processor 92 , a memory 93 , an input and output interface 94 , and a first Bluetooth interface 95 .

[0200] (2) Fig.10 As shown, the manipulator device is a device for simulating a three-dimensional probe and moving in a three-dimensional model of a target object to obtain target contours corresponding to different simulated sectors, which mainly includes: a left bending signal key 101, a right bending signal key 102, a forward bending signal key 103, a backward bending signal key 104, a forward signal key 105, a backward signal key 106, a counterclockwise rotation signal key 107, a clockwise rotation signal key 108, and a second Bluetooth interface 109.

[0201] The ultrasound image simulation generation method provided in this embodiment is based on an ultrasound image simulation model and a three-dimensional model of a target object such as the heart. It can simulate ultrasound images of various positions and angles inside the target object, improve the degree of matching between the ultrasound image and the specific structure of the target object, provide doctors with an interactive, high-precision ultrasound simulation system, and improve the efficiency and quality of doctors' learning of ultrasound images.

[0202] Example 3

[0203] This embodiment provides a training system for an ultrasound image simulation model. Fig.11 As shown, the training system includes:

[0204] A first sample acquisition module 1, used to acquire a plurality of sample ultrasound images of a sample object;

[0205] A second sample acquisition module 2, used to acquire a sample mask image of each sample ultrasound image;

[0206] An ultrasonic image output module 3, used for inputting each sample mask image into a preset network model, and outputting a corresponding first ultrasonic image and a second ultrasonic image;

[0207] The preset network model includes a first preset network and a second preset network, the first preset network is used to learn contour information in the ultrasonic image, and the second preset network is used to learn texture information in the ultrasonic image;

[0208] A comparison module 4, used for comparing the first ultrasonic image and the second ultrasonic image of each group with the corresponding sample ultrasonic image to obtain a comparison result;

[0209] The training module 5 is used to iteratively update the preset network model based on different comparison results until the preset model training conditions are met to obtain an ultrasound image simulation model.

[0210] In an implementable solution, both the first preset network and the second preset network adopt generative adversarial networks.

[0211] In an implementable solution, the ultrasound image output module 3 further includes:

[0212] A first output unit, used for inputting the sample mask image into a first preset network to output a first ultrasound image;

[0213] a second output unit, configured to input the first ultrasound image into a second preset network to output a second ultrasound image;

[0214] Comparison module 4 also includes:

[0215] A first loss calculation unit, configured to calculate a first loss value between the first ultrasonic image and the sample ultrasonic image based on a first loss function;

[0216] A second loss calculation unit, configured to calculate a second loss value between the second ultrasonic image and the sample ultrasonic image based on a second loss function;

[0217] Training Module 5 also includes:

[0218] A first parameter updating unit, configured to iteratively update a network parameter of a first preset network based on a first loss value to reduce the first loss value;

[0219] A second parameter updating unit, configured to iteratively update a network parameter of a second preset network based on a second loss value to reduce the second loss value;

[0220] The training completion determination unit is used to determine that the preset model training conditions are met when the first loss value is less than the first preset threshold and the second loss value is less than the second preset threshold, so as to obtain the ultrasound image simulation model.

[0221] In an implementable solution, the second sample acquisition module 2 further includes:

[0222] A denoising unit, used for performing denoising processing on the sample ultrasonic image to obtain a third ultrasonic image;

[0223] The segmentation unit is used to perform threshold segmentation processing on the third ultrasonic image to obtain a sample mask image.

[0224] In one feasible solution, the sample object includes a predetermined organ and / or a predetermined tissue.

[0225] In one practicable embodiment, the predetermined organ includes a heart.

[0226] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only illustrative, in which the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution.

[0227] The training system for the ultrasound image simulation model provided in this embodiment can improve the learning accuracy of the model for the contour information and texture information of the ultrasound image, improve the accuracy of the ultrasound image simulation model training, and thereby improve the accuracy of the ultrasound image simulation by inputting a sample mask image into a preset network model, comparing the first ultrasound image and the second ultrasound image output by the model with the sample ultrasound image respectively, and updating the ultrasound image simulation model based on the two comparison results.

[0228] Example 4

[0229] This embodiment provides a simulation generation system of ultrasound images, such as Fig.12 As shown, the simulation generation system includes:

[0230] A target acquisition module 6 is used to acquire a target mask image of a target object;

[0231] The ultrasound simulation module 7 is used to input the target mask image into the ultrasound image simulation model to simulate and generate the target ultrasound image;

[0232] The ultrasound image simulation model is trained using the training system described in Example 3.

[0233] In an implementable solution, the target acquisition module 6 includes:

[0234] A three-dimensional model acquisition unit, used to acquire a three-dimensional model of a target object;

[0235] A simulated sector acquisition unit is used to acquire a simulated sector corresponding to a target angle by taking a target position in a three-dimensional model as a vertex and a preset length as a radius;

[0236] An initial point set acquisition unit is used to acquire all intersection points of the simulation sector and the three-dimensional model to obtain an initial point set;

[0237] A target point set acquisition unit is used to screen the initial point set using a preset screening algorithm to obtain a number of target point sets;

[0238] A target contour acquisition unit, used for determining a corresponding target contour according to each point in each target point set and two radii of the simulated sector;

[0239] The target mask acquisition unit is used to obtain a target mask image according to each target contour.

[0240] In an implementable solution, the target point set acquisition unit further includes:

[0241] A seed point acquisition subunit is used to randomly select a point in the initial point set as a seed point and add the seed point to the first point set;

[0242] The target point acquisition subunit is used to select a point closest to the seed point as the target point using a greedy algorithm;

[0243] A distance judgment subunit, used for adding the target point to the first point set when the distance between the target point and the seed point is less than or equal to a preset distance;

[0244] The target point set acquisition subunit is used to adopt the target point as a new seed point, and repeatedly execute the greedy algorithm to select a point closest to the seed point as the target point; when the distance between the target point and the seed point is less than or equal to the preset distance, the target point is added to the first point set until the distance between the target point and the seed point is greater than the preset distance, and the current first point set is used as the target point set.

[0245] In an implementable solution, the target contour acquisition unit further includes:

[0246] A first contour acquisition subunit is used to connect the points in the target point set to obtain a target contour when the distance between the first seed point and the last target point in the target point set is less than or equal to a preset distance;

[0247] The second contour acquisition subunit is used for connecting the points in the target point set and a radius where the simulated sector and the target point set have an intersection to obtain the target contour when the distance between the first seed point and the last target point in the target point set is greater than a preset distance and two points in the target point set fall on the same radius of the simulated sector;

[0248] The third contour acquisition subunit is used to connect the points in the target point set and the two radii of the simulated fan to obtain the target contour when the distance between the first seed point and the last target point in the target point set is greater than the preset distance and there are two points in the target point set that fall on the two radii of the simulated fan respectively.

[0249] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only illustrative, in which the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution.

[0250] The ultrasound image simulation generation system provided in this embodiment is based on the ultrasound image simulation model and the three-dimensional model of the target object such as the heart. It can simulate the ultrasound images of various positions and angles inside the target object, improve the matching degree between the ultrasound image and the specific structure of the target object, provide doctors with an interactive, high-precision ultrasound simulation system, and improve the efficiency and quality of doctors' learning of ultrasound images.

[0251] Example 5

[0252] This embodiment provides an electronic device, Fig.13 The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the training method of the ultrasound image simulation model of embodiment 1 or the simulation generation method of the ultrasound image of embodiment 2 when executing the program. Fig.13 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0253] like Fig.13 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0254] The bus 33 includes a data bus, an address bus, and a control bus.

[0255] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0256] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0257] The processor 31 executes various functional applications and data processing by running the computer program stored in the memory 32, such as the training method of the ultrasound image simulation model of Example 1 of the present disclosure, or the simulation generation method of the ultrasound image of Example 2.

[0258] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Fig.13 As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0259] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0260] Example 6

[0261] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the training method of the ultrasound image simulation model of Example 1, or the simulation generation method of the ultrasound image of Example 2.

[0262] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0263] In a possible implementation, the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the training method of the ultrasound image simulation model of Example 1, or the simulation generation method of the ultrasound image of Example 2.

[0264] Among them, the program code for executing the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as a separate software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0265] Example 7

[0266] The embodiments of the present disclosure also provide a computer program product, including a computer program, which, when executed by a processor, implements the training method of the ultrasound image simulation model of the above-mentioned embodiment 1, or the simulation generation method of the ultrasound image of embodiment 2.

[0267] Among them, the program code for executing the computer program product of the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0268] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.

Claims

1. A method for simulating and generating an ultrasonic image, characterized in that: The simulation generation method comprises: Acquire a target mask image of the target object; Inputting the target mask image into an ultrasound image simulation model to simulate and generate a target ultrasound image; The step of acquiring a target mask image of the target object comprises: Acquire a three-dimensional model of the target object; Taking the target position in the three-dimensional model as the vertex and the preset length as the radius, obtaining a simulated sector corresponding to the target angle; Obtaining all intersection points of the simulated sector and the three-dimensional model to obtain an initial point set; Using a preset screening algorithm to screen the initial point set to obtain a plurality of target point sets; Determine a corresponding target contour according to each point in the target point set and two radii of the simulated sector; According to each of the target contours, obtaining the target mask image; The ultrasound image simulation model is obtained by using a training method for the ultrasound image simulation model, and the training method comprises: Acquire a plurality of sample ultrasound images of the sample object; Acquire a sample mask image for each of the sample ultrasound images; Input each of the sample mask images into a preset network model, and output a corresponding first ultrasonic image and a second ultrasonic image; The preset network model includes a first preset network and a second preset network, the first preset network is used to learn contour information in an ultrasonic image, and the second preset network is used to learn texture information in an ultrasonic image; Comparing the first ultrasonic image and the second ultrasonic image of each group with the corresponding sample ultrasonic image to obtain a comparison result; The preset network model is iteratively updated based on different comparison results until preset model training conditions are met to obtain the ultrasound image simulation model.

2. The method for simulating and generating an ultrasonic image according to claim 1, wherein: The first preset network and the second preset network both adopt generative adversarial networks.

3. The method for simulating and generating an ultrasonic image according to claim 2, wherein: The step of inputting each sample mask image into a preset network model and outputting the corresponding first ultrasonic image and second ultrasonic image comprises: Inputting the sample mask image into the first preset network to output the first ultrasound image; Inputting the first ultrasound image into the second preset network to output the second ultrasound image; The step of comparing the first ultrasonic image and the second ultrasonic image of each group with the corresponding sample ultrasonic image to obtain a comparison result comprises: Calculating a first loss value between the first ultrasonic image and the sample ultrasonic image based on a first loss function; Calculating a second loss value between the second ultrasonic image and the sample ultrasonic image based on a second loss function; The step of iteratively updating the preset network model based on different comparison results until the preset model training conditions are met to obtain the ultrasound image simulation model includes: Iteratively updating network parameters of the first preset network based on the first loss value to reduce the first loss value; Iteratively updating the network parameters of the second preset network based on the second loss value to reduce the second loss value; When the first loss value is less than a first preset threshold value, and the second loss value is less than a second preset threshold value, it is determined that the preset model training condition is met to obtain the ultrasound image simulation model.

4. The method for simulating and generating an ultrasonic image according to claim 3, wherein: The step of acquiring a sample mask image of each sample ultrasound image comprises: Performing denoising processing on the sample ultrasonic image to obtain a third ultrasonic image; Perform threshold segmentation processing on the third ultrasonic image to obtain the sample mask image.

5. The method for simulating and generating an ultrasonic image according to any one of claims 1 to 4, characterized in that: The sample object includes a preset organ and / or a preset tissue.

6. The method for simulating and generating an ultrasonic image according to claim 5, characterized in that: The predetermined organ includes a heart.

7. The method for simulating and generating an ultrasonic image according to claim 1, wherein: The step of using a preset screening algorithm to screen the initial point set to obtain a plurality of target point sets comprises: Randomly select a point in the initial point set as a seed point, and add the seed point to the first point set; Using a greedy algorithm, a point closest to the seed point is selected as the target point; When the distance between the target point and the seed point is less than or equal to a preset distance, adding the target point to the first point set; The target point is used as a new seed point, and the greedy algorithm is repeatedly executed to select a point closest to the seed point as the target point; when the distance between the target point and the seed point is less than or equal to a preset distance, the target point is added to the first point set until the distance between the target point and the seed point is greater than the preset distance, and the current first point set is used as the target point set.

8. The method for simulating and generating an ultrasonic image according to claim 7, wherein: The step of determining the corresponding target contour according to each point in the target point set and the two radii of the simulated sector comprises: If the distance between the first seed point and the last target point in the target point set is less than or equal to the preset distance, connecting the points in the target point set to obtain the target contour; If the distance between the first seed point and the last target point in the target point set is greater than the preset distance, and two points in the target point set fall on the same radius of the simulated sector, then connect the points in the target point set and a radius where the simulated sector and the target point set intersect to obtain the target contour; If the distance between the first seed point and the last target point in the target point set is greater than the preset distance, and two points in the target point set fall on two radii of the simulated sector respectively, then the points in the target point set and the two radii of the simulated sector are connected to obtain the target contour.

9. A system for simulating and generating an ultrasonic image, characterized in that: The simulation generation system comprises: A target acquisition module, used to acquire a target mask image of a target object; An ultrasound simulation module, used for inputting the target mask image into an ultrasound image simulation model to simulate and generate a target ultrasound image; The target acquisition module also includes: A three-dimensional model acquisition unit, used to acquire a three-dimensional model of the target object; A simulated sector acquisition unit, used to acquire a simulated sector corresponding to a target angle by taking the target position in the three-dimensional model as a vertex and a preset length as a radius; An initial point set acquisition unit, used to acquire all intersection points of the simulation sector and the three-dimensional model to obtain an initial point set; A target point set acquisition unit, used to use a preset screening algorithm to screen the initial point set to obtain a plurality of target point sets; A target contour acquisition unit, used for determining a corresponding target contour according to each point in the target point set and two radii of the simulated sector; A target mask acquisition unit, used for obtaining the target mask image according to each of the target contours; The ultrasound image simulation model is obtained by using a training system for the ultrasound image simulation model, and the training system comprises: A first sample acquisition module, used to acquire a plurality of sample ultrasound images of a sample object; A second sample acquisition module, used for acquiring a sample mask image of each sample ultrasound image; An ultrasonic image output module, used for inputting each of the sample mask images into a preset network model, and outputting a corresponding first ultrasonic image and a second ultrasonic image; The preset network model includes a first preset network and a second preset network, the first preset network is used to learn contour information in an ultrasonic image, and the second preset network is used to learn texture information in an ultrasonic image; a comparison module, used for comparing the first ultrasonic image and the second ultrasonic image of each group with the corresponding sample ultrasonic image to obtain a comparison result; A training module is used to iteratively update the preset network model based on different comparison results until the preset model training conditions are met to obtain the ultrasound image simulation model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, the method for simulating and generating an ultrasound image according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for simulating and generating an ultrasound image according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for simulating and generating an ultrasound image according to any one of claims 1 to 8 is implemented.

Images