Segmenter-free training data set generation method for abnormal image segmentation

Abstract

The invention provides a segmentation-free training data set generation method for abnormal image segmentation, and relates to the technical field of computer vision and image processing, and the method comprises the steps: taking a reference true value mask as a condition, inputting the condition into a diffusion model, and generating an image containing an abnormal region through random noise and a step-by-step denoising process; in the sampling process of the diffusion model, an attention difference maximization energy function is introduced, and the attention difference maximization energy function dynamically adjusts the alignment degree of an abnormal region of a generated image and a reference truth value mask by comparing the difference between an attention map generated by the diffusion model and the reference truth value mask; and iteratively optimizing the attention difference maximization energy function, and generating an abnormal image matched with the reference true value mask and a mask label pair corresponding to the abnormal image as a training data set. According to the method, the problem of label drift is effectively relieved under the condition of not depending on an additional divider.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a method for generating a segmenter-free training data set for abnormal image segmentation. Background Art

[0002] In the field of computer vision and image processing, the combination of image generation and segmentation tasks has become a key research direction, especially in scenarios where samples are scarce or difficult to label. With the rapid development of deep learning technology, generative adversarial networks (GANs) and diffusion models are increasingly used in image generation, enhancement, and labeling, especially in the fields of medicine and industry. Generating high-quality images and their corresponding labeled data has become a core technology to solve the problem of data scarcity.

[0003] Generative adversarial networks (GANs) are one of the early image generation methods. They generate realistic images through adversarial training between generators and discriminators. However, GANs are prone to mode collapse, which leads to a lack of diversity in generated samples and difficulty in controlling image quality. Despite this, GANs are still widely used in fields such as medical image generation and image restoration. In recent years, diffusion models (such as DDPM) have gradually become the mainstream method in the field of image generation. Diffusion models generate high-quality images by introducing noise in the generation process and gradually denoising it. It has strong controllability and stability, and can effectively avoid the common mode collapse problem of GANs. Therefore, it is widely used in high-quality image generation and reconstruction tasks.

[0004] Abnormal images cover many types, such as medical images, ground dirt images, etc. The goal of the segmentation training data synthesis task is to simultaneously generate a large number of images and their corresponding mask labels. These image-mask pairs serve as additional training data sets for downstream segmentation tasks, especially for tasks with sparse samples or difficult annotation. The existing technology usually adopts the following process: the ground truth (GT) mask (representing the abnormal area) is input into the pre-trained diffusion model to generate an image containing abnormal content. Among them, the GT mask serves as both a guide for the image generation process and a segmentation label for the generated image. However, this process has the problem of mask label drift, that is, the GT mask does not match the actual abnormal area in the generated image. Due to the lack of clear error feedback, this problem is inevitable in the traditional process.

[0005] To address the label drift problem, a straightforward strategy is to feed the generated images with drift problems into an additional frozen pre-trained segmentation model (segmenter), such as Figure 1As shown. The segmentation model is able to ensure that its mask predictions exactly match the anomaly regions in the generated image, but this approach may cause the mask predictions themselves to be affected by the drift problem. Finally, the diffusion model is optimized by calculating the binary cross entropy (BCE) loss between the GT mask and the drift prediction mask, forcing the generated anomaly regions to be closer to the GT mask, thereby alleviating the mask label drift problem. However, this strategy is highly dependent on the performance of the pre-trained segmentation model, which is difficult to achieve in the real world, especially in areas with sparse data. It is a huge challenge to ensure that the segmentation model performs well on limited training samples. In addition, if the segmentation model performs perfectly, there is no need to generate more training samples, which is contrary to the original intention of generating the dataset. At the same time, the introduction of additional segmentation models will significantly increase the computational overhead and have a negative impact on the efficiency of the entire framework. Summary of the invention

[0006] To this end, an embodiment of the present invention provides a method for generating a segmenter-free training dataset for abnormal image segmentation, which aims to solve the problem of mask label drift that occurs when generating abnormal images, and improves the accuracy of generated images by optimizing the generation process without relying on additional segmenters, so that the abnormal areas of the generated images are better aligned with the reference truth (GT) masks, thereby generating higher quality training data for downstream abnormal image segmentation tasks.

[0007] In order to solve the above problem, an embodiment of the present invention provides a method for generating a segmenter-free training data set for abnormal image segmentation, the method comprising: The ground truth mask is used as a conditional input to the diffusion model, and an image containing abnormal regions is generated through random noise and step-by-step denoising process; In the sampling process of the diffusion model, an attention difference maximization energy function is introduced, and the attention difference maximization energy function dynamically adjusts the alignment between the abnormal area of ​​the generated image and the reference truth mask by comparing the difference between the attention map generated by the diffusion model and the reference truth mask; By iteratively optimizing the attention difference maximization energy function, an abnormal image matching the reference truth mask and its corresponding mask label pair are generated as a training data set.

[0008] Preferably, the attention difference maximization energy function includes a first energy function and the second energy function ,in: The first energy function By maximizing the difference between the mean attention of the target area and the mean attention of the non-target area, the localization accuracy of the abnormal area in the generated image is enhanced; The second energy function By penalizing the lowest K attention values ​​in the target area and the highest K attention values ​​in the non-target area, the regional imbalance problem between the target area and the non-target area is solved.

[0009] Preferably, the first energy function The calculation formula is: ; In the formula, represents the ground truth mask, represents the diffusion model at time step The attention map calculated when predicting noise, is the number of pixels in the target area, is the number of pixels in the non-target area, represents element-wise multiplication, represents the sum of all elements in the matrix, represents the absolute value function.

[0010] Preferably, the second energy function The calculation formula is: ; In the formula, represents the ground truth mask, represents the diffusion model at time step The attention map calculated when predicting noise, and Respectively represent the selection of the K highest values ​​and the K lowest values ​​in the non-target area and the target area, represents element-wise multiplication, represents the sum of all elements in the matrix, represents the absolute value function.

[0011] Preferably, the expression of the attention difference maximization energy function is: ; In the formula, represents the attention difference maximization energy function, represents the ground truth mask, represents the diffusion model at time step The attention map calculated when predicting noise, is a weight factor used to balance the first energy function With the second energy function impact.

[0012] Preferably, the sampling process of the diffusion model includes two stages: a contour generation stage and a detail refinement stage. The attention difference maximization energy function is only applied in the contour generation stage of the diffusion model, and the contour generation stage corresponds to the interval of 80% to 100% of the sampling time step of the diffusion model.

[0013] Preferably, a repetition strategy is adopted in the sampling process of the diffusion model, that is, the currently generated intermediate image is resampled as the output of the previous time step at each time step, and the control strength of the attention difference maximization energy function is enhanced through multiple repeated sampling.

[0014] Preferably, the attention map is generated by the cross attention layer of U-Net in the diffusion model, and the specific calculation method is: ; In the formula, represents the attention mechanism function, represents the activation function, represents the diffusion model at time step The attention map calculated when predicting noise, represents the text information in the latent space, Represents the intermediate space characteristics, is the projection key and the dimension of the query, Represents a matrix transpose operation.

[0015] The embodiment of the present invention further provides a system for generating a segmenter-free training data set for abnormal image segmentation, which is used to implement the above-mentioned method for generating a segmenter-free training data set for abnormal image segmentation, and specifically includes: An image generation module, which is used to input the ground truth mask as a condition into the diffusion model, and generate an image containing the abnormal area through random noise and step-by-step denoising process; An alignment adjustment module is used to introduce an attention difference maximization energy function in the sampling process of the diffusion model, wherein the attention difference maximization energy function dynamically adjusts the alignment between the abnormal area of ​​the generated image and the reference truth mask by comparing the difference between the attention map generated by the diffusion model and the reference truth mask; The data set generation module is used to generate abnormal images and their corresponding mask label pairs matching the reference truth mask as training data sets by iteratively optimizing the attention difference maximization energy function.

[0016] An embodiment of the present invention further provides a computer storage medium storing a computer software product, wherein the computer software product includes a number of instructions for enabling a computer device to execute the above-mentioned method for generating a segmenter-free training data set for abnormal image segmentation.

[0017] It can be seen from the above technical solutions that the present invention has the following beneficial effects: (1) Effectively alleviate the label drift problem by maximizing the energy function using the attention difference based on the attention map and the ground truth mask , adjust the target area of ​​the generated image in each sampling step. This effectively avoids the target area mismatch problem caused by label drift.

[0018] (2) No reliance on external segmenters: Traditional diffusion models rely on external segmenters to predict the mask of the generated image, and then compare it with the reference truth mask for optimization. This method is highly dependent on the segmenter and may affect the quality of the generated image when data is scarce or the segmenter is imperfect. Compared with the existing practice of relying on external models, the present invention directly optimizes the generation process through the embedded attention difference maximization energy function, making the target area generation more accurate, significantly improving the consistency between the generated image and the reference truth mask, and greatly reducing the computational cost, improving the stability of the training process.

[0019] (3) The training process does not require additional training data: The present invention adopts a "no training guidance" method, uses the existing diffusion model as the baseline model, and carefully designs an attention-based energy function for guidance. No additional training process is required. This method is very beneficial in scenarios where data is scarce. In the absence of a large amount of labeled data, it can optimize the quality of the image, thereby better supporting downstream tasks.

[0020] (4) Solving the imbalance problem between the target area and the non-target area: The present invention designs an additional energy function (the second energy function ), which significantly solves the problem of abnormal attention values ​​in the generated image when the target area accounts for a small proportion and the non-target area accounts for a large proportion. By adjusting the first K attention values, the generation quality of the target area is effectively improved and label drift is further reduced.

[0021] (5) Improved the quality of generated data: The present invention significantly improves the quality of generated data. In quantitative comparison, the images generated by the method of the present invention are significantly better than other SOTA models in terms of FID and KID indicators, indicating that the generated images are closer to the real data in terms of visual quality and data distribution. In the LPIPS indicator, the method of the present invention also shows high visual consistency. Qualitative comparison shows that the method of the present invention can generate clearer images with richer details, avoiding the blur and artifact problems in traditional methods. In addition, the t-SNE visualization results show that the images generated by the method of the present invention are more consistent with the real images in terms of data distribution, successfully overcoming the data deviation caused by the additional loss function and auxiliary segmentation. In general, the present invention can generate images with higher quality, richer details and more accurate data distribution, thereby improving the quality of generated data.

[0022] (6) Extensive application potential: The training data generated by the method of the present invention can significantly improve the performance of downstream task models. For example, on the ETIS dataset, the mDice of the SAnet model increased by 4.4% and the mIoU increased by 5.3%. By generating high-quality and diverse synthetic data, the technical solution of the present invention can be applied in multiple fields such as medical impact and environmental detection, demonstrating extensive application potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them: Figure 1 A schematic diagram of a strategy for solving the label drift problem provided in the background technology; Figure 2 A flowchart of a method for generating a segmenter-free training data set for abnormal image segmentation provided in an embodiment; Figure 3 is a flow chart of the method of the present invention in an embodiment; Figure 4 Schematic diagram of the mechanism of action of the ADM energy function in the method of the present invention in the embodiment; Figure 5 ADM energy function structure schematic diagram in the embodiment; Figure 6 It is a schematic diagram of the repetition strategy algorithm in the embodiment; Figure 7 This is a graph showing the comparison results between the method of the present invention and different generation models in the embodiments; Figure 8 is a diagram showing the comparison results of a polyp data set in an embodiment; Fig. 9 This is a diagram showing the comparison of segmentation performance on a floor dirt dataset in the embodiment; Fig.10 It is a block diagram of a system for generating a segmenter-free training data set for abnormal image segmentation provided in an embodiment. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1

[0025] In order to solve the problem of mask label drift when generating abnormal images without relying on additional segmenters. Figure 2 As shown, an embodiment of the present invention proposes a method for generating a segmenter-free training data set for abnormal image segmentation, the method comprising: The ground truth mask is used as a conditional input to the diffusion model, and an image containing abnormal regions is generated through random noise and step-by-step denoising process; In the sampling process of the diffusion model, the attention difference maximization energy function is introduced. The attention difference maximization energy function dynamically adjusts the alignment between the abnormal area of ​​the generated image and the reference truth mask by comparing the difference between the attention map generated by the diffusion model and the reference truth mask. By iteratively optimizing the attention difference maximization energy function, anomaly images matching the benchmark truth masks and their corresponding mask label pairs are generated as training datasets.

[0026] From the above technical solution, it can be seen that the present invention proposes a method for generating a segmenter-free training data set for abnormal image segmentation, which optimizes the sampling process by guiding the generation process and avoids the problems caused by introducing additional segmenters in traditional methods. In the method of the present invention, the attention map in the diffusion model is used to replace the mask prediction of the segmenter. Then, the proposed attention difference maximization loss (function) is calculated using the attention map and the GT mask to optimize the image. This operation enables the diffusion model to focus on the area in the benchmark truth mask when generating abnormal content, thereby alleviating the label drift problem.

[0027] In this embodiment, the ground truth (GT) mask is first input into the diffusion model as a condition, and an image containing abnormal regions is generated through random noise and step-by-step denoising process, such as Figure 3 shown.

[0028] Specifically, the GT mask M is used as a conditional input and passed into the diffusion model, which generates an image containing abnormal areas through random noise and step-by-step denoising process. The GT mask M is used as an annotation for the generated image and guides the diffusion model to generate regions where the abnormal contents are located.

[0029] The above diffusion model includes two processes: a forward process and a reverse process. The forward process converts the clear image into pure Gaussian noise by continuously adding noise, and the time step is from 0 to T. On the contrary, the reverse process reconstructs the clear image from pure Gaussian noise, and the time step is from T to 0. Represents the time step The intermediate state of the data, Indicates a clear image.

[0030] The forward process follows a certain plan Add noise, expressed as: ; In the formula, are predefined parameters, is random noise.

[0031] The reverse process can be expressed as: ; In the formula, yes The distribution of , is obtained by training the diffusion model. In practice, the model is usually used to learn only one of them, and the other can be defined in advance.

[0032] Neural networks are usually used to fit .because can be obtained from the noise, we only need the network to learn the noise . The training loss function is Defined as: ; In the formula, represents the mathematical expectation, yes The square of the norm.

[0033] The basic form of the energy-guided fractional-based diffusion model is: ; In the formula, is a predefined parameter, is the fractional function of the gradient in the diffusion model based on the Tweedie formula, is an energy function introduced to measure the difference between the target and generated images.

[0034] Furthermore, in the sampling process of the diffusion model, the present invention introduces the attention difference maximization (ADM) energy function. The ADM energy function dynamically adjusts the alignment between the abnormal area of ​​the generated image and the GT mask by comparing the difference between the attention map generated by the diffusion model and the GT mask, such as Figure 4 shown.

[0035] Since the proposed method aims to enhance the alignment between the activation area of ​​the attention map and the basic GT mask, Alternative ,in The ADM energy function proposed by the present invention is introduced. By introducing the ADM energy function of the present invention, the image is generated. By updating along a combined direction at each time step to alleviate label drift, the new formula becomes: ; in, represents the diffusion model at time step Attention maps computed when predicting noise, generated by the U-Net in the diffusion model. The calculation is as follows: ; In the formula, represents the attention mechanism function, represents the activation function, represents the diffusion model at time step The attention map calculated when predicting noise, represents the text information in the latent space, Represents the intermediate space characteristics, is the projection key and the dimension of the query, Represents a matrix transpose operation.

[0036] Furthermore, the first energy function The detailed design is as follows: ; In the formula, is the number of pixels in the target area, is the number of pixels in the non-target area, represents element-wise multiplication, represents the sum of all elements in the matrix, represents the absolute value function. Perform maximum and minimum normalization so that each activation value ranges from 0 to 1. Therefore, Figure 5 As shown, Select activation values ​​for the target region and encourage their mean to be close to 1. In non-target areas and push its value close to 0.

[0037] like Figure 5 As shown, in the process of using GT mask, the first energy function When sampling, the image is updated by encouraging large values ​​in the target area of ​​the attention map and small values ​​in the non-target area. However, after statistics of the dataset used, it was found that the regional imbalance ratio was high. This situation leads to the first energy function It is difficult to effectively adjust the abnormal attention value, that is, the minimum value of the target area and the maximum value of the non-target area, and the attention map of the intermediate result is a good example.

[0038] Abnormal images often show the characteristics of small target objects, occupying a small area in the image, and medical images are a typical example. This characteristic makes the problem of regional imbalance more prominent. In this case, abnormal attention values ​​such as large attention values ​​in non-target areas and low attention values ​​in target areas cannot be captured by the first energy function. Precise punishment makes it difficult to achieve the optimization goal. Even if the first energy function It has certain control over the position and shape of the target object, but the regional imbalance problem still limits its optimization effect.

[0039] To solve this problem, the present invention proposes a second energy function , used to penalize the first K abnormal attention values. With the help of the second energy function Further adjust the abnormal attention value, the formula is as follows: ; In the formula, and represents selecting the K highest values ​​and the K lowest values ​​in the non-target area and the target area respectively.

[0040] In summary, the first energy function By maximizing the difference between the mean attention value of the target area and the mean attention value of the non-target area, the localization accuracy of the abnormal area in the generated image is enhanced. The second energy function By penalizing the lowest K attention values ​​in the target area and the highest K attention values ​​in the non-target area, the regional imbalance problem between the target area and the non-target area is solved.

[0041] Final ADM energy function The expression is as follows: ; In the formula, is the weight factor used to balance the first energy function With the second energy function impact.

[0042] The intermediate image updated by the ADM energy function of the present invention can be optimized in the direction of alleviating label drift.

[0043] At the same time, we note that the ADM energy function is used to update the entire sampling process This is not reasonable. The diffusion sampling process includes two stages: the contour generation stage and the detail refinement stage. Since the difference between the predicted intermediate image and the final clear image is not significant in the refinement stage, the method of the present invention may not be able to effectively solve the label drift problem, but it will still affect the reconstruction of the clear image. Therefore, the method of the present invention is only introduced in the contour generation stage, where the contour generation stage corresponds to the 80% to 100% interval of the diffusion model sampling time step.

[0044] In addition, in order to enhance the control strength of the energy function, the present invention adopts a repetitive strategy, that is, each time step On the top, output the current model Resample to the output of the previous time step , and use it as the model input. By repeating the sampling process multiple times, the control strength of the ADM energy function is further enhanced, thereby more effectively guiding the accuracy of the generated image. The specific approach is as follows Figure 6 shown.

[0045] Finally, in terms of training the diffusion model, the pre-trained Stable Diffusion (SD) V2.1 and ControlNet were adopted as baseline models, in which the real baseline truth mask M was introduced as a condition of the diffusion model. In addition, SD was enabled to generate abnormal images by fine-tuning ControlNet. For the inference process of the diffusion model, the denoising diffusion implicit model was followed, and 100 sampling time steps were adopted. The guidance scale λ was set to 9, which is exactly the same as ControlNet. The energy function was introduced between 80 and 100 time steps; the strategy was repeated three times; the weight factor Set to 0.5. For the attention map, this example selects the output of the final cross-attention operation in SD's U-Net, which is 64×64 in size. The reason is that the final attention map captures all the effects of the conditional control on the features, and the features used in this layer are closer to the final model output. The number of abnormal pixels, denoted as K, is set to 20% of the total pixels in the target area.

[0046] The advantages of the method of the present invention are further illustrated below in conjunction with specific experiments.

[0047] 1. Image quality assessment This experiment evaluates the quality of generated images using the following metrics: FID: It is used to measure the difference between the generated image and the real image. The lower the FID value, the more similar the generated image is to the real image.

[0048] KID: Similar to FID, but more stable when dealing with fewer samples and does not assume that the data follows a Gaussian distribution.

[0049] LPIPS: Evaluates the diversity of generated images. The higher the LPIPS value, the richer the diversity of generated images.

[0050] like Figure 7 As shown in Figure 1, compared with different generative models, the images generated by the proposed method perform best in most evaluation indicators. The best scores are bolded and the suboptimal ones are underlined.

[0051] 2. Evaluation of downstream model data validity In this experiment, experiments were conducted on five publicly available polyp segmentation datasets and one floor dirt dataset.

[0052] 2.1 Dataset Preparation a) Polyp segmentation datasets: ETIS, CVC-ClinicDB / CVC612, CVC-ColonDB, EndoScene, Kvasir.

[0053] The image-mask pairs used for training come from Kvasir-SEG (900 training samples) and CVC-ClinicDB (550 training samples), with a total of 1450 image-mask pairs as the “train-real” dataset.

[0054] The test sets include: EndoScene (60 test samples), CVC-ClinicDB (62 test samples), CVC-ColonDB (380 test samples), ETIS-LaribPolypDB (196 test samples), and Kvasir (100 test samples).

[0055] b) Floor Dirt Dataset: Contains two types of anomalies: floor stains (500 images) and pet feces (458 images). 3 / 5 of the anomaly images are used for training, and 2 / 5 are used for validation and testing.

[0056] By copying the 1450 pairs of images and masks in “train-real”, 2900 pairs of data were created to obtain the “train-real + train-real” dataset.

[0057] The “train-real + train-ArSDM” dataset is obtained by combining the real images in “train-real” and the 1450 image-mask pairs generated by the ArSDM method.

[0058] We combine the real images (1450 pairs) with the 1450 pairs of images and masks generated by the proposed method to obtain the "train-ours" dataset.

[0059] 2.2 Performance Evaluation and Analysis The above four datasets were trained using two different segmentation models, SANet and Polyp-PVT. The results are as follows Figure 8 As shown. The two indicators mIoU (%) and mDice (%) are used, where a higher value means that the model's segmentation is more accurate. The result with the highest score is highlighted in bold, and the second highest result is underlined. "+" indicates the combination of the "train-real" dataset with other datasets. "train-ArSDM" refers to the dataset generated by the ArSDM generative model. "AVE" represents the average results of the five test sets.

[0060] For the floor dirt dataset, this experiment used 1000 pure synthetic data of each category to train the SegFormer model. The SegFormer model was trained by using the data generated by the method of the present invention and the data generated by the previous method (DFMGAN, Anomaly Diffusion). The two indicators of IoU (%) and F-measure (%) were used. The results are shown in Fig. 9 shown.

[0061] Obviously, by using the training data generated by the method of the present invention, the performance of downstream segmentation tasks is significantly improved. Embodiment 2

[0062] like Fig.10 As shown, the present invention provides a system for generating a training data set without a segmenter for abnormal image segmentation, which is used to implement the method for generating a training data set without a segmenter for abnormal image segmentation in the first embodiment, and specifically includes: An image generation module, which is used to input the ground truth mask as a condition into the diffusion model, and generate an image containing the abnormal area through random noise and step-by-step denoising process; An alignment adjustment module is used to introduce an attention difference maximization energy function in the sampling process of the diffusion model. The attention difference maximization energy function dynamically adjusts the alignment between the abnormal area of ​​the generated image and the reference truth mask by comparing the difference between the attention map generated by the diffusion model and the reference truth mask; The dataset generation module is used to generate abnormal images and their corresponding mask label pairs matching the benchmark truth masks as training datasets by iteratively optimizing the attention difference maximization energy function.

[0063] A segmenter-free training dataset generation system for abnormal image segmentation in this embodiment is used to implement the aforementioned segmenter-free training dataset generation method for abnormal image segmentation. Therefore, the specific implementation method of the segmenter-free training dataset generation system for abnormal image segmentation can be found in the embodiment section of the segmenter-free training dataset generation method for abnormal image segmentation in the aforementioned text. In order to avoid redundancy, it will not be repeated here. Embodiment 3

[0064] An embodiment of the present invention provides a computer storage medium storing a computer software product. The computer software product includes several instructions for enabling a computer device to execute the above-mentioned method for generating a segmenter-free training data set for abnormal image segmentation.

[0065] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0066] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0067] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0068] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A method for generating a segmenter-free training data set for abnormal image segmentation, characterized in that: include: The ground truth mask is used as a conditional input to the diffusion model, and an image containing abnormal regions is generated through random noise and step-by-step denoising process; In the sampling process of the diffusion model, an attention difference maximization energy function is introduced, and the attention difference maximization energy function dynamically adjusts the alignment between the abnormal area of ​​the generated image and the reference truth mask by comparing the difference between the attention map generated by the diffusion model and the reference truth mask; By iteratively optimizing the attention difference maximization energy function, an abnormal image matching the reference truth mask and its corresponding mask label pair are generated as a training data set.

2. The method for generating a segmenter-free training data set for abnormal image segmentation according to claim 1, characterized in that: The attention difference maximization energy function includes a first energy function and the second energy function ,in: The first energy function By maximizing the difference between the mean attention of the target area and the mean attention of the non-target area, the localization accuracy of the abnormal area in the generated image is enhanced; The second energy function By penalizing the lowest K attention values ​​in the target area and the highest K attention values ​​in the non-target area, the regional imbalance problem between the target area and the non-target area is solved.

3. The method for generating a segmenter-free training data set for abnormal image segmentation according to claim 2, characterized in that: The first energy function The calculation formula is: ； In the formula, represents the ground truth mask, represents the diffusion model at time step The attention map calculated when predicting noise, is the number of pixels in the target area, is the number of pixels in the non-target area, represents element-wise multiplication, represents the sum of all elements in the matrix, represents the absolute value function.

4. The method for generating a segmenter-free training data set for abnormal image segmentation according to claim 2, characterized in that: The second energy function The calculation formula is: ； In the formula, represents the ground truth mask, represents the diffusion model at time step The attention map calculated when predicting noise, and Respectively represent the selection of the K highest values ​​and the K lowest values ​​in the non-target area and the target area, represents element-wise multiplication, represents the sum of all elements in the matrix, represents the absolute value function.

5. The method for generating a segmenter-free training data set for abnormal image segmentation according to claim 2, characterized in that: The expression of the attention difference maximization energy function is: ； In the formula, represents the attention difference maximization energy function, represents the ground truth mask, represents the diffusion model at time step The attention map calculated when predicting noise, is a weight factor used to balance the first energy function With the second energy function impact.

6. The method for generating a segmenter-free training data set for abnormal image segmentation according to claim 1, characterized in that: The sampling process of the diffusion model includes two stages: the contour generation stage and the detail refinement stage. The attention difference maximization energy function is only applied in the contour generation stage of the diffusion model, and the contour generation stage corresponds to the interval of 80% to 100% of the diffusion model sampling time step.

7. The method for generating a segmenter-free training data set for abnormal image segmentation according to claim 1, characterized in that: A repetition strategy is adopted in the sampling process of the diffusion model, that is, the currently generated intermediate image is resampled as the output of the previous time step at each time step, and the control strength of the attention difference maximization energy function is enhanced through multiple repeated sampling.

8. The method for generating a segmenter-free training data set for abnormal image segmentation according to claim 1, characterized in that: The attention map is generated by the cross attention layer of U-Net in the diffusion model, and the specific calculation method is: ； In the formula, represents the attention mechanism function, represents the activation function, represents the diffusion model at time step The attention map calculated when predicting noise, represents the text information in the latent space, Represents the intermediate space characteristics, is the projection key and the dimension of the query, Represents a matrix transpose operation.

9. A system for generating a training data set without a segmenter for abnormal image segmentation, characterized in that: The system is used to implement the method for generating a segmenter-free training data set for abnormal image segmentation according to any one of claims 1 to 8, specifically comprising: An image generation module, which is used to input the ground truth mask as a condition into the diffusion model, and generate an image containing the abnormal area through random noise and step-by-step denoising process; An alignment adjustment module is used to introduce an attention difference maximization energy function in the sampling process of the diffusion model, wherein the attention difference maximization energy function dynamically adjusts the alignment between the abnormal area of ​​the generated image and the reference truth mask by comparing the difference between the attention map generated by the diffusion model and the reference truth mask; The data set generation module is used to generate abnormal images and their corresponding mask label pairs matching the reference truth mask as training data sets by iteratively optimizing the attention difference maximization energy function.

10. A computer storage medium, characterized in that: The computer storage medium stores a computer software product, which includes a number of instructions for enabling a computer device to execute the method for generating a segmenter-free training data set for abnormal image segmentation according to any one of claims 1 to 8.

Images