An augmentation method and device based on a small amount of labeled medical image data

Abstract

The application provides an augmentation method and device based on a small amount of labeled medical image data, which comprises: generating a first global image based on a global image generation model, the global image generation model being trained based on target medical image data; determining a plurality of local images based on the first global image, the local images being generated based on the first global image and image data corresponding to the local images in the first global image; and generating a second global image based on the plurality of local images, the accuracy of the second global image being higher than that of the first global image. Since the first global image is generated by using the global image generation model trained based on the target medical image data, the local images are generated based on the first global image and the image data corresponding to the local images in the first global image, and the plurality of local images have continuity, the application achieves the purpose of augmenting more medical image data samples based on a small amount of labeled medical image data.

Description

Technical Field

[0001] This application relates to the field of image processing, specifically to an amplification method and apparatus based on a small amount of labeled medical image data. Background Technology

[0002] Regions of interest (ROIs) segmentation in medical images is a crucial step in diagnosis, treatment, and clinical research. Typically, ROI segmentation in medical images is performed using segmentation models, such as convolutional neural networks (CNNs).

[0003] To ensure the robustness and accuracy of the segmentation model, a large amount of sample data is needed to train it. However, sample data in the medical field is often difficult to obtain. Therefore, how to obtain a large amount of sample data in the medical field has become an urgent problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide an amplification method and apparatus based on a small amount of labeled medical image data.

[0005] In a first aspect, one embodiment of this application provides an augmentation method based on a small amount of labeled medical image data, comprising: generating a first global image based on a global image generation model, wherein the global image generation model is trained based on target medical image data; generating multiple local images based on the first global image, wherein the local images are generated based on the first global image and image data corresponding to the local images in the first global image; and generating a second global image based on the multiple local images; wherein the accuracy of the second global image is higher than that of the first global image.

[0006] In one possible implementation, generating a first global image based on a global image generation model includes: generating a first noisy image based on a segmentation template, wherein the segmentation template is used to indicate the region of interest in the first global image and the second global image; performing multiple noise addition and deletion processes on the first noisy image based on the global image generation model to determine a second noisy image corresponding to a plurality of first time moments, wherein the multiple noise addition and deletion processes correspond one-to-one with the plurality of first time moments; and determining the second noisy image corresponding to the first first time moment among the plurality of first time moments as the first global image.

[0007] In one possible implementation, generating a first noisy image based on a segmentation template includes: generating first noise data corresponding to each of multiple target regions included in the segmentation template; wherein the multiple target regions include regions of interest in the segmentation template and other regions besides the regions of interest; and generating a first noisy image based on the first noise data corresponding to each of the multiple target regions.

[0008] In one possible implementation, the first noisy image is subjected to multiple noise addition and deletion processes to determine the second noisy images corresponding to each of the multiple first moments, including: for any one of the multiple first moments, generating second noise data corresponding to that first moment based on a segmentation template; determining a first noise prediction result for the second noise image corresponding to that first moment, wherein the second noise image corresponding to the last first moment among the multiple first moments is the first noise image; and generating a second noise image corresponding to the previous first moment based on the second noise data, the second noise image, and the first noise prediction result.

[0009] In one possible implementation, determining the first noise prediction result of the second noise image corresponding to the first time moment includes: determining the first feature image of the second noise image corresponding to the first time moment; converting the segmentation template to determine the multi-channel template corresponding to the segmentation template; extracting the second feature image corresponding to the multi-channel template; fusing the second feature image and the first feature image to determine the third feature image of the second noise image corresponding to the first time moment; and determining the first noise prediction result of the second noise image corresponding to the first time moment based on the third feature image.

[0010] In one possible implementation, the global image generation model is trained based on multiple second-time noise images corresponding to the second time step. Each second-time noise image is determined based on the noise data corresponding to that second time step. The multiple second-time noise data are generated based on a segmentation template, wherein the segmentation template is used to indicate the region of interest in the first global image and the second global image.

[0011] In one possible implementation, based on a first global image, multiple local images are determined, including: for each local image, determining a third noise image corresponding to that local image; based on the local image extraction templates corresponding to the first global image and the local images, performing multiple noise addition and deletion processes on the third noise image corresponding to the local image to determine multiple fourth noise images corresponding to each of the multiple third time moments, wherein the multiple noise addition and deletion processes correspond one-to-one with the multiple third time moments; and determining the fourth noise image corresponding to the first third time moment among the multiple third time moments as the local image.

[0012] In one possible implementation, based on the local image extraction templates corresponding to the first global image and the local image, the third noise image corresponding to the local image is subjected to multiple noise addition and deletion processes to determine the fourth noise image corresponding to each of the multiple third time moments. This includes: generating third noise data corresponding to any third time moment among the multiple third time moments; determining a second noise prediction result for the fourth noise image corresponding to the third time moment based on the first global image and the local image extraction templates corresponding to the local image, wherein the fourth noise image corresponding to the last third time moment among the multiple third time moments is the third noise image; and generating the fourth noise image corresponding to the previous third time moment based on the third noise data corresponding to the third time moment, the fourth noise image, and the second noise prediction result.

[0013] In one possible implementation, determining the second noise prediction result of the fourth noise image corresponding to the third time step based on the first global image and the local image extraction template corresponding to the local image includes: determining the fourth feature image corresponding to the fourth noise image corresponding to the third time step; extracting the global feature image corresponding to the first global image; extracting the local feature image corresponding to the local image from the global feature image based on the local image extraction template corresponding to the local image; and determining the second noise prediction result of the noise image corresponding to the third time step based on the fourth feature image, the global feature image, and the local feature image corresponding to the local image.

[0014] In one possible implementation, generating a second global image based on multiple local images includes: stitching together the multiple local images based on local image extraction templates corresponding to each of the multiple local images to generate the second global image; wherein the local image extraction templates corresponding to each of the multiple local images are used to indicate the positional relationship between the multiple local images and the second global image.

[0015] Secondly, one embodiment of this application provides a training method for a segmentation model, comprising: acquiring a target global image, the target global image being generated based on the augmentation method based on a small amount of labeled medical image data described in the first aspect above; and training a segmentation model to be trained based on the target global image.

[0016] Thirdly, one embodiment of this application provides an amplification device based on a small amount of labeled medical image data, comprising: a first determining module, configured to generate a first global image based on a global image generation model, wherein the global image generation model is trained based on target medical image data; a second determining module, configured to generate multiple local images based on the first global image, wherein the local images are generated based on the first global image and image data corresponding to the local images in the first global image; and a generating module, configured to generate a second global image based on the multiple local images; wherein the accuracy of the second global image is higher than that of the first global image.

[0017] Fourthly, one embodiment of this application provides a training apparatus for a segmentation model, comprising: an acquisition module for acquiring a target global image, the target global image being generated based on the amplification method based on a small amount of labeled medical image data described in the first aspect; and a training module for training a segmentation model to be trained based on the target global image.

[0018] Fifthly, one embodiment of this application provides a computer-readable storage medium storing a computer program for performing the methods of the first and second aspects.

[0019] In a sixth aspect, one embodiment of this application provides an electronic device comprising: a processor; a memory for storing processor-executable instructions; and the processor for performing the methods of the first and second aspects.

[0020] The augmentation method based on a small amount of labeled medical image data provided in this application has the following beneficial effects.

[0021] This application first generates a first global image based on a global image generation model trained on target medical image data. Then, based on the first global image, multiple local images are determined. These local images are generated from the first global image and the corresponding image data of the local images within it. Finally, based on these multiple local images, a second global image is generated, with higher accuracy than the first global image. Since the first global image is generated using a global image generation model trained on target medical image data, and the local images are generated from the first global image and the corresponding image data of the local images within it, and these local images are continuous, this application achieves the goal of expanding the medical image data sample from a small amount of labeled medical image data.

[0022] The funding for this application comes from the National Key Research and Development Program of China, under the special project "Early Identification and Treatment Strategies for Fibrotic Interstitial Lung Disease", project number 2021YFC2500700. Attached Figure Description

[0023] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain the application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 The diagram shown illustrates a scenario applicable to an exemplary embodiment of this application.

[0025] Figure 2 The diagram shown is a flowchart illustrating an exemplary embodiment of the present application of an amplification method based on a small amount of labeled medical image data.

[0026] Figure 3 The diagram shown is a schematic diagram of a global image generation model provided in an exemplary embodiment of this application.

[0027] Figure 4 The diagram shown is a schematic diagram of performing multiple noise addition and deletion processes on a first noisy image according to an exemplary embodiment of this application.

[0028] Figure 5 The diagram shown is a schematic diagram of determining a third feature image according to an exemplary embodiment of this application.

[0029] Figure 6 The diagram shown is a schematic diagram of a training global image generation model provided in an exemplary embodiment of this application.

[0030] Figure 7 The diagram shown is a schematic diagram of a local image generation model provided in an exemplary embodiment of this application.

[0031] Figure 8 The diagram shown is a schematic diagram of a local image extraction template provided in an exemplary embodiment of this application.

[0032] Figure 9 The diagram shown is a schematic diagram of the extraction of global feature images and local feature images provided in an exemplary embodiment of this application.

[0033] Figure 10 The diagram shown is a schematic diagram of a training local image generation model provided in an exemplary embodiment of this application.

[0034] Figure 11The diagram shown is a flowchart illustrating a training method for a segmentation model provided in an exemplary embodiment of this application.

[0035] Figure 12 The diagram shown is a schematic representation of the structure of an amplification device based on a small amount of labeled medical image data provided in an exemplary embodiment of this application.

[0036] Figure 13 The diagram shown is a schematic representation of the structure of a training apparatus for a segmentation model provided in an exemplary embodiment of this application.

[0037] Figure 14 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] Application Overview

[0040] In related technologies, medical sample data can be generated through the following methods.

[0041] Method 1: Based on prior information (such as pixel values, shape, and texture features) from existing medical imaging data, medical sample data is manually designed according to computer graphics principles. Specifically, before creating the medical sample data, the designer needs to observe the morphological structure and distribution patterns of the target area, communicate with the doctor, and finally combine the observation results, communication results, and prior information to design the medical sample data. However, this method is complex and requires manual generation, resulting in low efficiency in generating medical sample data. Furthermore, the manually generated medical sample data, combining prior information and observation results, exhibits specific morphological structures with poor randomness, while the actual location and shape of lesions are not regular. Medical sample data generated through this method is very homogeneous.

[0042] Method 2: Image generation based on Generative Adversarial Networks (GANs). Specifically, a GAN consists of a generator and a discriminator. The generator produces medical sample data, while the discriminator determines whether the data input to the discriminator is real medical image data or medical sample data generated by the generator. By training both the generator and discriminator simultaneously, the generator can produce more realistic medical sample data. However, in practical applications, because GANs require simultaneous training of both the generator and discriminator, training is difficult, the model is not easy to converge, the training effect is unstable, and the diversity of generated medical sample data is poor.

[0043] Method 3: Currently, image generation methods based on diffusion models are becoming increasingly popular. However, diffusion models are not necessarily suitable for generating medical sample data. This is because medical data is mostly three-dimensional (such as CT scans and MRIs) or large-sized two-dimensional data (such as pathological images). Typical hardware devices are simply unable to generate complete medical sample data. If partial regions of medical sample data are generated sequentially and then stitched together to obtain the final medical sample data, discontinuities will occur between the regions since they were generated separately, leading to distortion of the resulting medical sample data.

[0044] In view of this, this application provides a method and apparatus for augmenting medical image data based on a small amount of labeled medical image data. The method includes: generating a first global image based on a global image generation model, wherein the global image generation model is trained on target medical image data; determining multiple local images based on the first global image, wherein the local images are generated based on the first global image and the image data corresponding to the local images in the first global image; and generating a second global image based on the multiple local images, wherein the accuracy of the second global image is higher than that of the first global image. Since the first global image is generated using a global image generation model trained on target medical image data, and the local images are generated based on the first global image and the image data corresponding to the local images in the first global image, and the multiple local images are continuous, this application achieves the goal of augmenting more medical image data samples based on a small amount of labeled medical image data.

[0045] Exemplary scenario

[0046] The augmentation method based on a small amount of labeled medical image data proposed in this application can be executed by an electronic device, which can be a terminal, such as a smartphone, tablet, desktop computer, etc., or the electronic device can also be a server.

[0047] Figure 1The diagram shows an application scenario of an amplification method based on a small amount of labeled medical image data provided in an embodiment of this application. Figure 1 The scenario shown includes a server 110, an image generation device 120, and a model training device 130, wherein the server 110 is communicatively connected to the image generation device 120, and the image generation device 120 is communicatively connected to the model training device 130.

[0048] Specifically, server 110 is used to acquire and store target medical image data, and input the target medical image data into image generation device 120; image generation device 120 is used to generate a first global image based on a global image generation model, wherein the global image generation model is trained based on the target medical image data; generate multiple local images based on the first global image, wherein the local images are generated based on the first global image and the image data corresponding to the local images in the first global image; generate a second global image based on the multiple local images; wherein the accuracy of the second global image is higher than that of the first global image; send the second global image to model training device 130; model training device 130 is used to train the segmentation model to be trained based on the second global image.

[0049] Exemplary methods

[0050] Figure 2 The diagram shown is a flowchart illustrating an exemplary embodiment of the augmentation method based on a small amount of labeled medical image data provided in this application. Figure 2 As shown, the amplification method based on a small amount of labeled medical image data provided in this application includes the following steps 201 to 203.

[0051] Step 201: Generate the first global image based on the global image generation model.

[0052] For example, the global image generation model is trained based on the target medical image data.

[0053] Step 202: Based on the first global image, determine multiple local images.

[0054] For example, the local image is generated based on the first global image and the image data corresponding to the local image in the first global image.

[0055] Step 203: Generate a second global image based on multiple local images. The second global image has higher precision than the first global image.

[0056] The following provides detailed examples of steps 201, 202, and 203.

[0057] For step 201, the following example is given.

[0058] The target medical imaging data can be real medical data, such as the medical imaging data of patient Zhang San.

[0059] For example, when generating the first global image based on the global image generation model, the first global image can be generated based on the global image generation model and the segmentation template.

[0060] A global image generation model can be a diffusion model, such as... Figure 3 The diagram illustrates a global image generation model. Figure 3 As shown, the global image generation model can include a global image decoder, a global image encoder, and a temporal encoder. The global image encoder and global image decoder have a U-shaped structure similar to a U-Net network. The global image decoder includes at least one decoding block, and the global image encoder includes at least one encoding block. The decoding and encoding blocks can be any of the following: convolutional blocks, residual blocks, dense blocks, or transformers. The temporal encoder can include at least one fully connected layer. The temporal encoder is used to encode temporal information, and the encoded temporal information can be embedded (e.g., added, multiplied, or merged) into the global image encoder and global image decoder.

[0061] Specifically, the target medical image data may carry labeled information annotated by doctors. The labeled information is used to mark each region of interest in the target medical image data. For example, the region of interest may be a lesion area, and the labeled information may be "label1", "label2", "label3", etc.

[0062] The segmentation template can be generated based on the labeling information of the target medical image data. Therefore, the segmentation template can be used to indicate various regions of interest (ROIs) in the target medical image data (i.e., it can indicate ROIs in the first global image and the second global image). Since other regions (such as tissue and organ regions) are also generated when generating the first global image, in addition to generating ROIs, the segmentation template can also be used to indicate other regions, such as labeling other regions as "label0". By using the regions indicated by the segmentation template, the content generated for each region in the first global image can be determined.

[0063] The segmentation template may also include a randomly generated template, which may also be used to indicate the region of interest (and other regions besides the region of interest) in the first global image and the second global image.

[0064] More specifically, when generating the first global image based on the global image generation model, steps 301 to 303 can be performed.

[0065] Step 301: Generate the first noisy image based on the segmentation template.

[0066] The segmentation template is used to indicate the region of interest in the first global image and the second global image.

[0067] Before performing step 301, the segmentation template can be downsampled. If downsampling is performed, then in step 301, the first noisy image can be generated based on the downsampled segmentation template.

[0068] It should be noted that the noise data corresponding to each region in the first noisy image generated based on the segmentation template is different, and each region in the first noisy image corresponds to each target region in the segmentation template below.

[0069] In one possible implementation, when generating a first noise image based on a segmentation template, first noise data corresponding to each of the multiple target regions included in the segmentation template can be generated firstly based on the segmentation template; wherein, the multiple target regions include the region of interest in the segmentation template and other regions other than the region of interest; and then, the first noise image is generated based on the first noise data corresponding to each of the multiple target regions.

[0070] For example, a noise sequence can be randomly generated, such as {X}. T0 ,X T1 ...X Tn}, where n is a natural number. Where X Ti ~N(0,I), X Ti Let i represent any first noise data in the noise sequence, whose size is the same as the segmentation template size, i = 0, 1, ..., n, where i represents the number of the first noise data and I represents the identity matrix. Then, multiple first noise data in the noise sequence are used as the first noise data corresponding to multiple target regions to construct a first noise image. For example, the first noise image can be generated based on the following formula 1.

[0071] Formula 1: X T =X T0 ×O[M=0]+X T1 ×O[M=1]+...+X Tn ×O[M=n]

[0072] Where O represents the indicative matrix and M represents the segmentation template, O and M have the same size, and O[M=n] means that the region in the segmentation template with region label n has a value of 1 in the corresponding region in matrix O, and the values ​​in other regions of matrix O are 0. In this way, any first noise data multiplied by its corresponding indicative matrix yields its corresponding first noise region. By adding all the first noise regions together, the first noise image is obtained.

[0073] Step 302: Based on the global image generation model, perform multiple noise addition and deletion processes on the first noisy image to determine the second noisy images corresponding to each of the first time moments.

[0074] Among them, multiple noise addition and deletion processes correspond one-to-one with multiple first moments. The number of multiple first moments can be preset, such as 50 first moments.

[0075] In one possible implementation, when performing multiple noise addition and deletion processes on the first noise image to determine multiple second noise images corresponding to each first time point, steps 401 to 403 can be executed.

[0076] It should be noted that, as Figure 4 The diagram shown illustrates multiple noise addition and deletion processes performed on a first noisy image, where the first noisy image is X. T For any one of multiple first moments (e.g.) Figure 4 At time t in the first time, the following steps 401 to 403 can be performed to obtain the time before the first time (e.g., time t). Figure 4 The second noise image corresponding to time t-1 in the data can be obtained by considering the last first time among multiple first times (e.g., ...). Figure 4 (time T in the middle) to the first of multiple first times (e.g., time T) Figure 4 The second noise image at each first moment is obtained by following the sequence of time 0 in the first moment.

[0077] Step 401: For any one of the multiple first moments, generate the second noise data corresponding to that first moment based on the segmentation template.

[0078] Before executing step 401, the segmentation template can be downsampled. If downsampling is performed, then when executing step 401, the second noise data corresponding to the first moment can be generated based on the downsampled segmentation template.

[0079] Specifically, the method used to generate the second noise data corresponding to the first time step based on the segmentation template can be the same as that used to generate the first noise image. Specifically, multiple second sub-noise data can be randomly generated: Z0~N(0,I), Z1~N(0,I), ..., Z n~N(0,I), where I represents the identity matrix, Z i The size of (i = 0, 1, ..., n) is the same as the size of the segmentation template. Then, the generated multiple second sub-noise data are used as the second sub-noise data corresponding to each of the multiple target regions to constitute the second noise data corresponding to the first time moment, that is, calculated according to the following formula 2:

[0080] Formula 2: Z = Z0 × O [M = 0] + Z1 × O [M = 1] + ... + Z n ×O[M=n]

[0081] Where Z represents the second noise data corresponding to the first moment, Z i Let i = 0, 1, ..., n represent any second sub-noise data point, O represent the indicative matrix, and M represent the segmentation template. O and M have the same size. O[M = n] means that the region in the segmentation template with region label n corresponds to a value of 1 in matrix O, while the values ​​in other regions of matrix O are 0. Thus, multiplying any second sub-noise data point by its corresponding indicative matrix yields its corresponding second noise data region. Adding all the second noise data regions together gives the second noise data point corresponding to that first time step.

[0082] Step 402: Determine the first noise prediction result of the second noise image corresponding to the first time point. The second noise image corresponding to the last of the multiple first time points is the first noise image.

[0083] In one possible implementation, when determining the first noise prediction result of the second noise image corresponding to the first time moment, a first feature image of the second noise image corresponding to the first time moment can be determined; then, the first noise prediction result of the second noise image corresponding to the first time moment can be determined based on the first feature image.

[0084] For example, in a global image generation model, any coded block and any decoder block can extract the first feature image of the second noise image. After extracting the first feature image, normalization operations can be performed on the first feature image, such as batch normalization, group normalization, layer normalization, etc. Finally, the first noise prediction result can be obtained based on the first feature image (after normalization) extracted by at least one coded block and / or decoder block.

[0085] In another possible implementation, when determining the first noise prediction result of the second noise image corresponding to the first time moment, steps 501 to 505 can be performed. The execution process of steps 501 to 504 can be found in... Figure 5 The diagram shown illustrates the determination of the third feature image.

[0086] Step 501: Determine the first feature image of the second noise image corresponding to the first moment.

[0087] After extracting the first feature image, normalization operations can be performed on the first feature image, such as batch normalization, group normalization, and layer normalization.

[0088] Step 502: Convert the segmentation template and determine the multi-channel template corresponding to the segmentation template.

[0089] Before performing step 502, the segmentation template can be downsampled. If downsampling is performed, then when performing step 502, the downsampled segmentation template can be converted to determine the multi-channel template corresponding to the segmentation template.

[0090] For example, the region markers of each target region in the segmentation template can be converted into multi-channel data to obtain a multi-channel template.

[0091] In one possible implementation, the region label i of each target region in the segmentation template can be first converted into a triplet (Ri, Gi, Bi) to convert the region label into a color value with color characteristics, where 0≤Ri≤255, 0≤Gi≤255, 0≤Bi≤255, and Ri, Gi, and Bi are all integers. For example, if there are n+1 target regions in the downsampled segmentation template, and their region labels are 0, 1, ..., n, then the triplets converted from the region labels are (R0, G0, B0), (R1, G1, B1), ..., (Rn, Gn, Bn).

[0092] Furthermore, in one possible implementation, for any one of Ri, Gi, and Bi, the value can be randomly selected between 0 and 255.

[0093] Alternatively, to increase the color differences between the markers in each region, and to avoid generating triplets of (0,0,0) that would affect feature fusion calculations, Ri, Gi, and Bi can be made to have non-overlapping values. Therefore, in another possible implementation, steps 601 to 603 can be performed:

[0094] Step 601: Determine the Ri value corresponding to the region marker of each target region.

[0095] For example, you can first set the value range of the R channel, such as r1 to r2, that is, first determine the value range of Ri, where i = 0, 1, ..., n, that is: {R0, R1, ..., Rn} ∈ [r1, r2]; then determine the value of R0 within the value range of Ri (it can be a random value within the value range of Ri, or it can be manually set; here, the value of R0 can be equal to r1). Afterwards, based on the set value range of Ri, determine the difference between every two adjacent Ri channels, as calculated using the following formula: Where, r inter This represents the difference between two adjacent Ri values. Finally, based on the value of R0 and the difference between two adjacent Ri values, the Ri value corresponding to the region marker of each target region is determined, calculated using the following formula: R i =R0+i×r inter .

[0096] Step 602: Determine the Gi value corresponding to the region marker of each target region.

[0097] Similarly, we can first set the range of values ​​for Gi to be {G0, G1, ..., Gn} ∈ [g1, g2]. To avoid repetition between Gi and Ri, we can set g1 = r1 + u1 and g2 = r2 + u2, where u1 and u2 are both non-zero integers (including positive and negative numbers). Then, we determine the value of G0 within the range of values ​​for Gi (this can be done by randomly selecting a value from the range of values ​​for Gi or by manually setting it; here, the value of G0 can be equal to g1). After that, based on the range of values ​​for Gi, we determine the difference between every two adjacent Gi values, as calculated using the following formula: Among them, g inter This represents the difference between two adjacent Gi values. Finally, based on the value of G0 and the difference between two adjacent Gi values, the Gi value corresponding to the region marker of each target region is determined, calculated using the following formula: G i =G0+i×g inter .

[0098] Step 603: Determine the value of Bi corresponding to the region marker of each target region.

[0099] Similarly, we can first set the range of Bi values ​​to {B0, B1, ..., Bn} ∈ [b1, b2]. To avoid Bi overlapping with Ri and Gi, we can set b1 = r1 + v1 and b2 = r2 + v2, where v1 and v2 are both non-zero integers (including positive and negative numbers), and v1 ≠ u1 and v2 ≠ u2. Then, we determine the value of B0 within the range of Bi values ​​(this can be a random value within the range of Bi values ​​or a manually set value; here, the value of B0 can be equal to b1). Afterward, based on the range of Bi values, we determine the difference between every two adjacent Bi values, calculated using the following formula: Among them, b inter This represents the difference between two adjacent Bi values. Finally, based on the value of B0 and the difference between two adjacent Bi values, the value of Bi corresponding to the region marker of each target region is determined, as calculated using the following formula: B i =B0+i×b inter .

[0100] After obtaining the multi-channel template, you can also perform downsampling operations on the multi-channel template.

[0101] Step 503: Extract the second feature image corresponding to the multi-channel template.

[0102] If a downsampling operation was performed on the multi-channel template in step 502, then the multi-channel template here can be the downsampled multi-channel template. When extracting the second feature image corresponding to the multi-channel template, at least one convolution operation can be performed to obtain the second feature image.

[0103] Step 504: Fuse the second feature image and the first feature image to determine the third feature image of the second noise image corresponding to the first time moment.

[0104] For example, the second feature image and the first feature image can be fused by addition and / or multiplication. If a normalization operation was performed in step 501, the first feature image is the normalized first feature image.

[0105] Step 505: Based on the third feature image, determine the first noise prediction result of the second noise image corresponding to the first time moment.

[0106] In a specific example, such as Figure 5 As shown, in the encoder and decoder of the global image generation model, at least one encoding block and / or decoding block can be an autonomous correction and normalization module. Any autonomous correction and normalization module can execute steps 501 to 504. Finally, in step 505, a first noise prediction result can be determined based on the third feature image output by at least one autonomous correction and normalization module.

[0107] Using this method, since the segmentation template indicates the category of each target region, the segmentation template is transformed into a more information-rich multi-channel template, and the second feature image of the multi-channel template is integrated into the first feature image of the second noise image corresponding to the first time step. This allows for the accurate prediction of the first noise prediction result by combining the information of each target region indicated by the segmentation template.

[0108] Step 403: Based on the second noise data, the second noise image and the first noise prediction result corresponding to the first time moment, generate the second noise image corresponding to the previous first time moment.

[0109] In a specific example, when determining the second noise image corresponding to the previous first time point of the first time point, the calculation can be performed based on the following formulas 3 and 4.

[0110] Formula 3:

[0111] Formula 4:

[0112] Where T is the number of pre-set first moments (i.e., the last first moment among multiple first moments), t represents that first moment, and β t β is a preset parameter. t The value of β ranges from [0,1] and increases with increasing t. t The actual value within the range can be determined based on any algorithm or manually. t and Let X be the two parameters calculated according to Formula 3 above. t-1 X represents the second noise image corresponding to the previous first time step. t e represents the second noisy image corresponding to the first moment. θ This represents the first noise prediction result corresponding to the first time step, M represents the segmentation template, and z represents the second noise data corresponding to the first time step.

[0113] Step 303: Determine the second noise image corresponding to the first first moment among multiple first moments as the first global image.

[0114] In one possible implementation, the global image generation model is trained based on multiple second-time-time-corresponding noisy images. Each second-time-corresponding noisy image is determined based on the noise data corresponding to that second-time-time, and the multiple second-time-corresponding noise data are generated based on a segmentation template. The segmentation template is used to indicate the region of interest in the first and second global images.

[0115] Specifically, when training the global image generation model, see [link to relevant documentation]. Figure 6 The schematic diagram shown illustrates the training of the global image generation model, and steps 701 to 706 can be performed.

[0116] Step 701: Obtain the target medical image data and the segmentation template corresponding to the target medical image data.

[0117] The target medical image data and the segmentation template are the same size.

[0118] Step 702: Downsample the target medical image data to obtain a global training sample image, and then downsample the segmentation template to obtain a downsampled segmentation template. The accuracy (resolution) of the global training sample image is lower than that of the target medical image data.

[0119] For any obtained global training sample image, execute steps 703 to 706.

[0120] Step 703: For any second time point among multiple second time points, generate the noise data corresponding to that second time point based on the downsampled segmentation template.

[0121] Here, for any target medical image data, after downsampling, a global training sample image is obtained. Multiple noise data corresponding to the second time step of this global training sample image can be obtained, and thus multiple noise images corresponding to the second time step can be obtained. Since these multiple second time steps are different times, the global image generation model can be trained using the noise images corresponding to different time steps, thereby improving the training effect of the global image generation model.

[0122] Based on the downsampled segmentation template, the noise data corresponding to the second time step is generated using the same method as when generating the second noise data corresponding to the first time step. Specifically, multiple sub-noise data can be randomly generated: e0~N(0,I), e1~N(0,I), ..., e n ~N(0,I), and then the generated multiple sub-noise data are used as the sub-noise data corresponding to each of the multiple target regions to obtain the noise data corresponding to the second time moment, that is, calculated according to the following formula 5:

[0123] Formula 5: e = e0 × O [M = 0] + e1 × O [M = 1] + ... + e n ×O[M=n]

[0124] Where e represents the noise data corresponding to the second time moment, e iLet i = 0, 1, ..., n represent any sub-noise data, where i represents the sub-noise data number. O represents the indicative matrix, and M represents the downsampled segmentation template. O and M have the same size. O[M=n] means that the region in the segmentation template with region label n corresponds to a value of 1 in matrix O, while the values ​​in other regions of matrix O are 0. Thus, multiplying any sub-noise data by its corresponding indicative matrix yields its corresponding noise region. Adding all noise regions together gives the noise data corresponding to the second time step.

[0125] Step 704: For any second time point among multiple second time points, generate a noise image corresponding to that second time point based on the noise data corresponding to that second time point and the global training sample image.

[0126] For example, the noisy image corresponding to the second time step is generated based on the following formula 6:

[0127] Formula 6:

[0128] in, The parameters are calculated based on Formula 3 above, where t represents the second time point, and x... t Let x0 represent the noisy image corresponding to the second time step, x0 represent the global training sample image, and e represent the noisy data corresponding to the second time step.

[0129] Based on Formula 6, the noise image corresponding to the noise data at the second time moment can be obtained, i.e., the noise image corresponding to the second time moment.

[0130] Step 705: For any second time among multiple second time moments, the global image generation model determines the first training noise prediction result corresponding to that second time moment among multiple second time moments based on the noise image corresponding to that second time moment among multiple second time moments and the downsampled segmentation template.

[0131] Similarly, the same execution process as steps 501 to 505 above can be used here to determine the first training noise prediction result corresponding to the second time moment among multiple second time moments.

[0132] Step 706: For any second time among multiple second time moments, based on the first training noise prediction result corresponding to the second time moment and the noise data corresponding to the second time moment among multiple second time moments, determine the first loss value, and train the global image generation model based on the first loss value.

[0133] For example, the parameters of the global image generation model can be adjusted according to the gradient descent method, as shown in the gradient formula below:

[0134] Formula 7:

[0135] in, Let t represent the gradient, t represent the second time step, and e represent the noise data corresponding to the second time step. θ This represents the first training noise prediction result corresponding to the second time step, x. t This represents the noisy image corresponding to the second time step, and M represents the segmentation template after downsampling.

[0136] By training the global image generation model with noisy data generated by segmentation templates, the global image generation model can better learn the differences between various target regions, thereby improving the training effect of the global image generation model.

[0137] For step 202, the following example is given.

[0138] It is understandable that generating local images based on image data from the first global image and image data from local images can incorporate information from surrounding images during the generation of local images, thereby ensuring continuity between the generated multiple local images.

[0139] In one possible implementation, multiple local images can be generated based on a first global image and a local image generation model, wherein the local image generation model can be, for example, a diffusion model.

[0140] For example, a local image generation model can be as follows: Figure 7 As shown, the network includes a local image encoder, a local image decoder, a global feature extractor, a local feature extractor, and a temporal encoder. The local image encoder and local image decoder have a U-shaped structure similar to a U-Net network. The local image encoder is used to extract image features and includes at least one encoding block. The local image decoder is used to deconstruct the input features and generate an image. The local image decoder includes at least one decoding block. The decoding block and encoding block can be any one of the following: convolutional block, residual block, dense block, and transformer. The temporal encoder can include at least one fully connected layer and is used to encode temporal information. The encoded temporal information can be embedded (e.g., added, multiplied, merged, etc.) into the local image encoder and local image decoder.

[0141] In one possible implementation, when determining multiple local images based on a first global image, steps 801 to 803 may be performed.

[0142] Step 801: For each local image, determine the corresponding third noise image.

[0143] Specifically, noise can be randomly generated to obtain a third noise image corresponding to the local image.

[0144] Alternatively, a third noise image can be generated based on the segmentation template. The method for generating the third noise image based on the segmentation template is the same as the method for generating the first noise image based on the segmentation template described above, and will not be repeated here.

[0145] Step 802: Based on the local image extraction templates corresponding to the first global image and the local image, perform multiple noise addition and deletion processes on the third noise image corresponding to the local image to determine the fourth noise image corresponding to each of the multiple third time moments.

[0146] Among them, multiple noise addition and deletion processes correspond one-to-one with multiple third time moments.

[0147] For example, a local image extraction template can be as follows: Figure 8 As shown, the local image extraction template includes the extraction region corresponding to the local image to be generated (e.g., Figure 8 The white area in the image), extract the value of 1 in the extracted area, and other areas outside the extracted area (such as... Figure 8 The black area in the image shows the value of 0, while the values ​​in other areas are 0.

[0148] In one possible implementation, when extracting templates based on the local images corresponding to the first global image and the local images, performing multiple noise addition and deletion processes on the third noise image corresponding to the local image, and determining the fourth noise image corresponding to each of the multiple third time moments, the following steps 901 to 903 can be executed.

[0149] Step 901: For any one of the multiple third time moments, generate the third noise data corresponding to that third time moment.

[0150] Specifically, the third noise data can be generated based on any noise generation method.

[0151] Alternatively, the third noise data can also be generated based on a segmentation template. The method for generating the third noise data based on a segmentation template is the same as the method for generating the second noise data based on a segmentation template described above, and will not be repeated here.

[0152] Step 902: Based on the first global image and the local image extraction template corresponding to the local image, determine the second noise prediction result of the fourth noise image corresponding to the third time moment.

[0153] Among them, the fourth noise image corresponding to the last third time point among multiple third time points is the third noise image.

[0154] In one possible implementation, when determining the second noise prediction result of the fourth noise image corresponding to the third time moment based on the first global image and the local image extraction template corresponding to the local image, the following steps 1001 to 1004 can be performed.

[0155] Step 1001: Determine the fourth feature image corresponding to the fourth noise image at the third time point.

[0156] Here, the fourth feature image can be extracted based on at least one coding block in the local image encoder and / or at least one decoding block in the local image decoder.

[0157] Step 1002: Extract the global feature image corresponding to the first global image.

[0158] Here, as Figure 9 The diagram shown illustrates the extraction of global and local feature images. The global feature image corresponding to the first global image can be extracted based on at least one coding block in the global feature extractor.

[0159] Step 1003: Based on the local image extraction template corresponding to the local image, extract the local feature image corresponding to the local image from the global feature image.

[0160] Specifically, such as Figure 9 As shown, the local image extraction template can be downsampled first to obtain a local feature extraction template, where the value of the extracted region in the local feature extraction template is 1, and the values ​​of other regions are 0. Then, the size of the local feature extraction template is adjusted so that its size is the same as that of the global feature image. Finally, the local feature image in the global feature image is extracted based on the adjusted local feature extraction template. After the local feature image is extracted, the size of the local feature image can also be adjusted by deconvolution and / or upsampling operations.

[0161] Step 1004: Based on the fourth feature image, the global feature image, and the local feature image corresponding to the local image, determine the second noise prediction result of the noise image corresponding to the third time moment.

[0162] Specifically, the fourth feature image, the global feature image, and the local feature image can be fused using at least one of the following methods: addition, multiplication, or channel merging. Then, based on the fused feature data, the second noise prediction result of the noise image corresponding to the third time point can be determined.

[0163] For example, such as Figure 7As shown, the fourth feature image output by the local image encoder can be channel-merged with the global feature image. Then, the feature image output by a partial decoding block in the local image decoder can be fused with the local feature image. Finally, based on at least one decoding block in the local image decoder, the second noise prediction result of the noise image corresponding to the third time step can be determined.

[0164] Step 903: Based on the third noise data, the fourth noise image, and the second noise prediction result corresponding to the third time point, generate the fourth noise image corresponding to the previous third time point.

[0165] Specifically, the fourth noise image corresponding to the previous third time step can be generated according to the following formula 8:

[0166] Formula 8.

[0167] Where t represents the third time point, β t β is a preset parameter. t The value of β ranges from [0,1] and increases with increasing t. t The actual value within the range can be determined based on any algorithm or manually. t and Let x be the two parameters calculated according to Formula 3 above. t-1 This represents the fourth noise image corresponding to the previous third time step, x. t This represents the fourth noise image corresponding to the third time point, e φ M represents the second noise prediction result corresponding to the third time point. fea This represents the local image extraction template (or local feature extraction template), z represents the third noise data corresponding to the third time step, and PI low This represents the first global image.

[0168] Step 803: Determine the fourth noise image corresponding to the first third time point among multiple third time points as a local image.

[0169] In one possible implementation, such as Figure 10 The schematic diagram shown illustrates the training of a local image generation model. The local image generation model can be trained using the following steps 1101 to 1107:

[0170] Step 1101: Obtain the local image extraction template and the target medical image data.

[0171] Specifically, a local image extraction template can be obtained from a template library that stores multiple templates, and the target medical image data can be obtained from a medical image dataset. The local image extraction template can be a template whose size corresponds to that of the target medical image data.

[0172] Step 1102: Downsample the local image extraction template (according to the first downsampling factor) to obtain the local feature extraction template, and downsample the target medical image data (according to the second downsampling factor) to obtain the downsampled target medical image data.

[0173] Here, the accuracy of the downsampled target medical image data is lower than that of the target medical image data, and the first downsampling factor is the same as the second downsampling factor.

[0174] Step 1103: Extract local image training samples from the downsampled target medical image data based on the local image extraction template.

[0175] For any extracted local image training sample, execute steps 1104 to 1107.

[0176] Step 1104: For any one of the multiple fourth time points, generate the noise data corresponding to that fourth time point.

[0177] Here, for any local image training sample, noise data corresponding to multiple fourth time points can be obtained, and thus multiple noise images corresponding to the fourth time points can be obtained. Since these multiple fourth time points are different times, the local image generation model can be trained using the noise images corresponding to different time points, thereby improving the training performance of the local image generation model.

[0178] Among them, the noise data corresponding to any of the multiple fourth time points can be e ~ N(0, I), and the size of the noise data is the same as that of the local image training samples.

[0179] In another possible implementation, the noise data corresponding to any of the multiple fourth time points can be generated based on a segmentation template.

[0180] Step 1105: For any fourth time point among multiple fourth time points, generate a noise image corresponding to that fourth time point based on the noise data corresponding to that fourth time point and the local training sample image.

[0181] For example, the noise image corresponding to the fourth time step can also be generated based on Formula 6 above. Applying Formula 6 here, the meaning of each parameter in Formula 6 is as follows: The parameters are calculated based on Formula 3 above, where t represents the fourth time point, and x... tLet x0 represent the noisy image corresponding to the fourth time step, x0 represent the local training sample image, and e represent the noisy data corresponding to the fourth time step.

[0182] Based on Formula 6, the noise image corresponding to the noise data at the fourth time point can be obtained, i.e., the noise image corresponding to the fourth time point.

[0183] Step 1106: For any fourth time point among multiple fourth time points, the local image generation model predicts the second training noise prediction result corresponding to the fourth time point among multiple fourth time points based on the noise image corresponding to the fourth time point among multiple fourth time points, the downsampled target medical image data, and the local feature extraction template.

[0184] Similarly, the same execution process as steps 1001 to 1004 above can be used here to determine the first training noise prediction result corresponding to the fourth time point among multiple fourth time points.

[0185] Step 1107: For any fourth time among multiple fourth time points, based on the second training noise prediction result corresponding to the fourth time point and the noise data corresponding to the fourth time point among multiple fourth time points, determine the second loss value, and train the local image generation model based on the second loss value.

[0186] For example, the parameters of the local image generation model can be adjusted according to the gradient descent method, as shown in Equation 9 below:

[0187] Formula 9:

[0188] in, Let t represent the gradient, t represent the fourth time step, and e represent the noise data corresponding to the fourth time step. θ This represents the second training noise prediction result corresponding to the fourth time step, x. t This represents the noisy image corresponding to the fourth time step, TM represents the local feature extraction template, and I... low This represents the target medical image data after downsampling.

[0189] For step 203, the following example is provided.

[0190] Specifically, multiple local images can be stitched together according to their respective positions to obtain a second global image. Since the multiple local images are generated based on the first global image and the image data corresponding to the local images in the first global image, the multiple local images are continuous. When stitching them together into the second global image, the edges of each local image can be smoothly connected, thus obtaining a complete second global image.

[0191] In one possible implementation, when generating a second global image based on multiple local images, the multiple local images can be stitched together based on the local image extraction templates corresponding to each of the multiple local images to generate the second global image; wherein, the local image extraction templates corresponding to each of the multiple local images are used to indicate the positional relationship between the multiple local images and the second global image.

[0192] Specifically, since multiple local images are generated by extracting local image features from global image features based on local image extraction templates at each location, multiple local images can be stitched together based on the location corresponding to the local image extraction template (i.e., the location corresponding to the extraction region).

[0193] The augmentation method based on a small amount of labeled medical image data provided in this application first generates a first global image based on a global image generation model trained on the target medical image data. Then, based on the first global image, multiple local images are determined. These local images are generated from the first global image and the image data corresponding to the local images within it. Finally, based on the multiple local images, a second global image is generated, with higher precision than the first global image. Since the first global image is generated using a global image generation model trained on the target medical image data, and the local images are generated from the first global image and the image data corresponding to the local images within it, and there is continuity between the multiple local images, this application achieves the goal of augmenting more medical image data samples based on a small amount of labeled medical image data.

[0194] Based on the same inventive concept, this application also provides a method for training a segmentation model. Figure 11 The diagram shown is a flowchart illustrating a training method for a segmentation model provided in an exemplary embodiment of this application. Figure 11 As shown, the training method for the segmentation model provided in this application embodiment includes the following steps 1201 to 1202.

[0195] Step 1201: Obtain the target global image, which is generated based on the amplification method based on a small amount of labeled medical image data described in the above embodiment.

[0196] Step 1202: Train the segmentation model to be trained based on the target global image.

[0197] The following provides detailed examples of steps 1201 to 1202.

[0198] In one possible implementation, when training a segmentation model based on a global target image, the global target image can be input into the segmentation model to obtain the target prediction result output by the segmentation model. Then, the parameters of the segmentation model can be adjusted based on the target label information and the target prediction result of the global target image.

[0199] The target label information can be manually labeled, or, in the case where the target global image is generated based on a segmentation template, the target label information is the segmentation template. Typically, medical image data requires a significant amount of time and effort from professional physicians to annotate it. Therefore, it is difficult to obtain large-scale medical image data with target label information. Using this method, the target label information of the target global image amplified based on the segmentation template is the segmentation template itself. This eliminates the need for manual labeling, saving manpower and improving production efficiency.

[0200] Exemplary device

[0201] The above text combined Figures 2 to 11 The method embodiments of this application are described in detail below, in conjunction with... Figure 12 and Figure 13 The present application provides a detailed description of the apparatus embodiments. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be found in the foregoing method embodiments.

[0202] Figure 12 The diagram shown is a schematic representation of an amplification device based on a small amount of labeled medical image data provided in an exemplary embodiment of this application. Figure 12 As shown, the amplification device 1300 based on a small amount of labeled medical image data provided in this application embodiment includes:

[0203] The first determining module 1301 is used to generate a first global image based on a global image generation model, wherein the global image generation model is trained based on the target medical image data;

[0204] The second determining module 1302 is used to generate multiple local images based on the first global image, wherein the local images are generated based on the first global image and the image data corresponding to the local images in the first global image;

[0205] The generation module 1303 is used to generate a second global image based on multiple local images; wherein the precision of the second global image is higher than that of the first global image.

[0206] In one possible implementation, the first determining module 1301, when generating a first global image based on a global image generation model, is configured to: generate a first noisy image based on a segmentation template, wherein the segmentation template is used to indicate the region of interest in the first global image and the second global image; perform multiple noise addition and deletion processes on the first noisy image based on the global image generation model to determine a second noisy image corresponding to a plurality of first time moments, wherein the multiple noise addition and deletion processes correspond one-to-one with the plurality of first time moments; and determine the second noisy image corresponding to the first first time moment among the plurality of first time moments as the first global image.

[0207] In one possible implementation, the first determining module 1301, when generating a first noise image based on a segmentation template, is configured to: generate first noise data corresponding to each of the multiple target regions included in the segmentation template; wherein the multiple target regions include regions of interest in the segmentation template and other regions other than regions of interest; and generate a first noise image based on the first noise data corresponding to each of the multiple target regions.

[0208] In one possible implementation, the first determining module 1301, when performing multiple noise addition and deletion processes on the first noise image to determine the second noise images corresponding to each of the multiple first moments, is configured to: for any one of the multiple first moments, generate second noise data corresponding to that first moment based on a segmentation template; determine a first noise prediction result for the second noise image corresponding to that first moment, wherein the second noise image corresponding to the last first moment among the multiple first moments is the first noise image; and generate a second noise image corresponding to the previous first moment based on the second noise data, the second noise image, and the first noise prediction result for that first moment.

[0209] In one possible implementation, the first determining module 1301, when determining the first noise prediction result of the second noise image corresponding to the first time moment, is configured to: determine the first feature image of the second noise image corresponding to the first time moment; convert the segmentation template to determine the multi-channel template corresponding to the segmentation template; extract the second feature image corresponding to the multi-channel template; fuse the second feature image and the first feature image to determine the third feature image of the second noise image corresponding to the first time moment; and determine the first noise prediction result of the second noise image corresponding to the first time moment based on the third feature image.

[0210] In one possible implementation, the global image generation model is trained based on multiple second-time noise images corresponding to the second time step. Each second-time noise image is determined based on the noise data corresponding to that second time step. The multiple second-time noise data are generated based on a segmentation template, wherein the segmentation template is used to indicate the region of interest in the first global image and the second global image.

[0211] In one possible implementation, the second determining module 1302, when determining multiple local images based on the first global image, is configured to: for each local image, determine a third noise image corresponding to that local image; based on the local image extraction templates corresponding to the first global image and the local images, perform multiple noise addition and deletion processes on the third noise image corresponding to the local image to determine a fourth noise image corresponding to each of the multiple third time moments, wherein the multiple noise addition and deletion processes correspond one-to-one with the multiple third time moments; and determine the fourth noise image corresponding to the first third time moment among the multiple third time moments as the local image.

[0212] In one possible implementation, the second determining module 1302, when performing multiple noise addition and deletion processes on the third noise image corresponding to the local image based on the local image extraction template corresponding to the first global image and the local image to determine the fourth noise image corresponding to each of the multiple third time moments, is configured to: generate third noise data corresponding to any third time moment among the multiple third time moments; determine the second noise prediction result of the fourth noise image corresponding to the third time moment based on the first global image and the local image extraction template corresponding to the local image, wherein the fourth noise image corresponding to the last third time moment among the multiple third time moments is the third noise image; and generate the fourth noise image corresponding to the previous third time moment based on the third noise data corresponding to the third time moment, the fourth noise image, and the second noise prediction result.

[0213] In one possible implementation, the second determining module 1302, when determining the second noise prediction result of the fourth noise image corresponding to the third time moment based on the first global image and the local image extraction template corresponding to the local image, is configured to: determine the fourth feature image corresponding to the fourth noise image corresponding to the third time moment; extract the global feature image corresponding to the first global image; extract the local feature image corresponding to the local image from the global feature image based on the local image extraction template corresponding to the local image; and determine the second noise prediction result of the noise image corresponding to the third time moment based on the fourth feature image, the global feature image, and the local feature image corresponding to the local image.

[0214] In one possible implementation, the generation module 1303, when generating a second global image based on multiple local images, is used to: stitch together the multiple local images based on the local image extraction templates corresponding to each of the multiple local images to generate the second global image; wherein, the local image extraction templates corresponding to each of the multiple local images are used to indicate the positional relationship between the multiple local images and the second global image.

[0215] Figure 13 The diagram shown is a structural schematic of a training apparatus for a segmentation model provided in an exemplary embodiment of this application. Figure 13 As shown, the segmentation model training device 1400 provided in this application embodiment includes:

[0216] The acquisition module 1401 is used to acquire the target global image, which is generated based on the amplification method based on a small amount of labeled medical image data described in the above embodiments.

[0217] Training module 1402 is used to train the segmentation model to be trained based on the target global image.

[0218] Below, for reference Figure 14 This describes an electronic device according to embodiments of the present application. Figure 14 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.

[0219] like Figure 14 As shown, the electronic device 1500 includes one or more processors 1501 and memory 1502.

[0220] The processor 1501 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1500 to perform desired functions.

[0221] The memory 1502 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1501 may execute the program instructions to implement the augmentation method based on a small amount of labeled medical image data, the segmentation model training method, and / or other desired functions described in the various embodiments of this application above. The computer-readable storage medium may also store various contents such as a first global image, a local image, a second global image, and target medical image data.

[0222] In one example, the electronic device 1500 may also include an input device 1503 and an output device 1504, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0223] The input device 1503 may include, for example, a keyboard, a mouse, etc.

[0224] The output device 1504 can output various information to the outside, including a first global image, a local image, a second global image, target medical image data, etc. The output device 1504 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0225] Of course, for the sake of simplicity, Figure 14 Only some of the components of the electronic device 1500 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 1500 may include any other suitable components depending on the specific application.

[0226] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the augmentation method and segmentation model training method based on a small amount of labeled medical image data according to various embodiments of this application described above.

[0227] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0228] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the augmentation method and segmentation model training method based on a small amount of labeled medical image data according to various embodiments of this application described above.

[0229] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0230] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0231] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0232] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0233] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0234] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An amplification method based on a small amount of labeled medical image data, characterized in that, include: A first global image is generated based on a global image generation model, wherein the global image generation model is trained based on the target medical image data; Based on the first global image, multiple local images are generated, wherein the local images are generated based on the first global image and the image data corresponding to the local images in the first global image; A second global image is generated based on the multiple local images; wherein the precision of the second global image is higher than that of the first global image. The generation of the first global image based on the global image generation model includes: A first noisy image is generated based on a segmentation template, wherein the segmentation template is used to indicate the region of interest in the first global image and the second global image; Based on the global image generation model, the first noisy image is subjected to multiple noise addition and deletion processes to determine multiple second noisy images corresponding to each first time moment, wherein the multiple noise addition and deletion processes correspond one-to-one with the multiple first time moments; The second noise image corresponding to the first first moment among the plurality of first moments is determined as the first global image; The step of determining multiple local images based on the first global image includes: For each of the local images, a third noise image corresponding to that local image is determined; Based on the local image extraction templates corresponding to the first global image and the local image, the third noise image corresponding to the local image is subjected to multiple noise addition and deletion processes to determine the fourth noise image corresponding to each of the multiple third time moments, wherein the multiple noise addition and deletion processes correspond one-to-one with the multiple third time moments; The fourth noise image corresponding to the first third time point among the plurality of third time points is determined as the local image; The local image extraction templates corresponding to each of the plurality of local images are used to indicate the positional relationship between the plurality of local images and the second global image.

2. The amplification method based on a small amount of labeled medical image data according to claim 1, characterized in that, The step of generating a first noisy image based on a segmentation template includes: Based on the segmentation template, first noise data corresponding to each of the multiple target regions included in the segmentation template is generated; wherein, the multiple target regions include the region of interest in the segmentation template, and other regions other than the region of interest; The first noise image is generated based on the first noise data corresponding to each of the multiple target regions.

3. The amplification method based on a small amount of labeled medical image data according to claim 1 or 2, characterized in that, The step of performing multiple noise addition and deletion processes on the first noisy image to determine multiple second noisy images corresponding to each first time point includes: For any one of the plurality of first moments, based on the segmentation template, generate second noise data corresponding to that first moment; A first noise prediction result is determined for the second noise image corresponding to the first time point, wherein the second noise image corresponding to the last first time point among the plurality of first time points is the first noise image; Based on the second noise data, the second noise image, and the first noise prediction result corresponding to the first moment, a second noise image corresponding to the previous first moment is generated.

4. The amplification method based on a small amount of labeled medical image data according to claim 3, characterized in that, The first noise prediction result for determining the second noise image corresponding to the first moment includes: Determine the first feature image of the second noise image corresponding to the first time point; The segmentation template is converted to determine the multi-channel template corresponding to the segmentation template; Extract the second feature image corresponding to the multi-channel template; The second feature image and the first feature image are fused to determine the third feature image of the second noise image corresponding to the first time moment; Based on the third feature image, the first noise prediction result of the second noise image corresponding to the first time moment is determined.

5. The amplification method based on a small amount of labeled medical image data according to claim 1, characterized in that, The global image generation model is trained based on multiple second-time noise images corresponding to the second time step. Any second-time noise image corresponding to the second time step is determined based on the noise data corresponding to that second time step. The multiple second-time noise data are generated based on a segmentation template, wherein the segmentation template is used to indicate the region of interest in the first global image and the second global image.

6. The amplification method based on a small amount of labeled medical image data according to claim 1, characterized in that, The step of extracting templates from local images corresponding to the first global image and the local images, and performing multiple noise addition and deletion processes on the third noise image corresponding to the local image to determine multiple fourth noise images corresponding to each third time point includes: For any one of the plurality of third time moments, generate the third noise data corresponding to that third time moment; Based on the first global image and the local image extraction template corresponding to the local image, the second noise prediction result of the fourth noise image corresponding to the third time moment is determined, wherein the fourth noise image corresponding to the last third time moment among the plurality of third time moments is the third noise image; Based on the third noise data corresponding to the third time moment, the fourth noise image, and the second noise prediction result, the fourth noise image corresponding to the previous third time moment is generated.

7. The amplification method based on a small amount of labeled medical image data according to claim 6, characterized in that, The step of determining the second noise prediction result of the fourth noise image corresponding to the third time moment based on the first global image and the local image extraction template corresponding to the local image includes: Determine the fourth feature image corresponding to the fourth noise image at the third time point; Extract the global feature image corresponding to the first global image; Based on the local image extraction template corresponding to the local image, the local feature image corresponding to the local image is extracted from the global feature image; Based on the fourth feature image, the global feature image, and the local feature image corresponding to the local image, the second noise prediction result of the noise image corresponding to the third time moment is determined.

8. The amplification method based on a small amount of labeled medical image data according to claim 1, characterized in that, The step of generating a second global image based on the plurality of local images includes: Based on the local image extraction templates corresponding to each of the multiple local images, the multiple local images are stitched together to generate the second global image.

9. A training method for a segmentation model, characterized in that, include: A target global image is acquired, wherein the target global image is generated based on the amplification method based on a small amount of labeled medical image data as described in any one of claims 1 to 8; Based on the target global image, the segmentation model to be trained is trained.

10. An amplification device based on a small amount of labeled medical image data, characterized in that, include: The first determining module is used to generate a first global image based on a global image generation model, wherein the global image generation model is trained based on target medical image data; The second determining module is used to generate multiple local images based on the first global image, wherein the local images are generated based on the first global image and the image data corresponding to the local images in the first global image; A generation module is used to generate a second global image based on the plurality of local images; wherein the precision of the second global image is higher than that of the first global image; The generation of the first global image based on the global image generation model includes: A first noisy image is generated based on a segmentation template, wherein the segmentation template is used to indicate the region of interest in the first global image and the second global image; Based on the global image generation model, the first noisy image is subjected to multiple noise addition and deletion processes to determine multiple second noisy images corresponding to each first time moment, wherein the multiple noise addition and deletion processes correspond one-to-one with the multiple first time moments; The second noise image corresponding to the first first moment among the plurality of first moments is determined as the first global image; The step of determining multiple local images based on the first global image includes: For each of the local images, a third noise image corresponding to that local image is determined; Based on the local image extraction templates corresponding to the first global image and the local image, the third noise image corresponding to the local image is subjected to multiple noise addition and deletion processes to determine the fourth noise image corresponding to each of the multiple third time moments, wherein the multiple noise addition and deletion processes correspond one-to-one with the multiple third time moments; The fourth noise image corresponding to the first third time point among the plurality of third time points is determined as the local image; The local image extraction templates corresponding to each of the plurality of local images are used to indicate the positional relationship between the plurality of local images and the second global image.

11. A training device for a segmentation model, characterized in that, include: The acquisition module is used to acquire a target global image, which is generated based on the amplification method based on a small amount of labeled medical image data as described in any one of claims 1 to 8. The training module is used to train the segmentation model to be trained based on the target global image.

12. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the amplification method based on a small amount of labeled medical image data as described in any one of claims 1 to 8, or for executing the training method of the segmentation model as described in claim 9.

13. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the amplification method based on a small amount of labeled medical image data as described in any one of claims 1 to 8, or to execute the training method of the segmentation model as described in claim 9.

Images