Fetal NT ultrasound image high-quality data expansion method based on dynamic programming optimization

Abstract

The invention is applied to the technical field of ultrasonic medical image processing and analysis, and particularly discloses a fetal NT ultrasonic image high-quality data expansion method based on dynamic programming optimization, which comprises the following steps: step 1, medical knowledge-oriented image transformation and augmentation; step 2, augmenting an improved generative adversarial network (D-GAN); step 3, dynamic planning combination optimization based on the grouped knapsacks; and 4, model training and feedback optimization. According to the fetal NT ultrasonic image high-quality data expansion method based on dynamic programming optimization, multi-method fusion augmentation is carried out on NT images, medical knowledge-oriented image transformation augmentation and an improved D-GAN generative adversarial network are combined, the stability of traditional transformation is kept, meanwhile, abnormal samples with high authenticity are generated, and the high-quality data expansion of the NT ultrasonic images is realized. Normal and abnormal scenes are covered, the diversity of a data set is remarkably improved, in image transformation, an operation area is limited through a mask, the noise injection intensity and the fuzzy kernel size are controlled, the key structure of an NT transparent layer is prevented from being damaged, and it is ensured that an augmented image meets the medical diagnosis standard.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic medical image processing and analysis, and in particular to a method for high-quality data expansion of fetal NT ultrasonic images based on dynamic programming optimization. Background Art

[0002] Fetal nuchal translucency (NT) ultrasound images are an important basis for prenatal screening of fetal chromosomal abnormalities. However, since their acquisition is limited by factors such as fetal posture, equipment performance, and physician experience, high-quality data is scarce. Therefore, in the field of deep learning, data augmentation technology is widely used to alleviate the problem of insufficient data. The technologies mainly include data augmentation based on image transformation and augmentation based on generative adversarial networks (GANs). Among them, data augmentation based on image transformation is simple and easy, but has problems of information redundancy and insufficient diversity; augmentation based on generative adversarial networks is prone to mode collapse in medical image generation, and has limited effect on generating abnormal samples.

[0003] Most existing technologies use a single augmentation method or random combination, resulting in insufficient quality of generated images and unstable improvement of the model's generalization ability, which in turn causes the augmented samples to fail to meet medical diagnostic standards, affecting the conclusions of prenatal fetal testing. Summary of the Invention

[0004] The purpose of the present invention is to provide a high-quality data expansion method for fetal NT ultrasound images based on dynamic programming optimization, so as to solve the problem raised in the above background technology that the image quality generated is insufficient and affects the diagnosis due to the single augmentation method for NT ultrasound images.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization, comprising the following steps: Step 1: Medical knowledge-guided image transformation and augmentation, transforming NT ultrasound images, and controlling and managing NT ultrasound images to ensure clear and standardized images and accurate and reliable measurement results; Step 2: Improved Generative Adversarial Network (D-GAN) augmentation: Optimize and improve the traditional Generative Adversarial Network (GAN), use the improved D-GAN to generate abnormal samples and improve the diversity of the dataset; Step 3: Based on the dynamic programming combination optimization of grouped backpacks, a grouped backpack model is constructed and the optimal augmented combination is selected through a greedy algorithm; Step 4: Model training and feedback optimization; use multiple models to verify the augmentation effect and output the model accuracy and F1 value; Preferably, the medical knowledge-guided image transformation and augmentation process in step 1 includes geometric change optimization, color gamut transformation adjustment, and pixel transformation constraints. The geometric change optimization ensures the integrity of the NT transparent layer structure through rotation to remove black edges and cropping optimization operations; the color gamut transformation adjustment adaptively adjusts the contrast and saturation according to the brightness distribution of the NT ultrasound image to avoid distortion of medical features; the pixel transformation constraint protects the edge clarity of the NT transparent layer through noise injection and Gaussian blur processing, and limits the operation area through masking to avoid key areas of the NT transparent layer.

[0006] By adopting the above technical solution, image transformation augmentation can enhance the authenticity of the image while protecting the key diagnostic features of the image.

[0007] Preferably, when the geometric change optimization part rotates the NT image, the effective area is dynamically calculated based on the image width and height and the rotation angle, and the rotated NT image is cropped. The calculation formula of the cropping range is as follows: ; ; ; ; The mapping range of contrast adjustment in the color gamut conversion adjustment part is ±15% of the original value, the adjustment range of saturation in the color gamut conversion adjustment part is limited to ≤20%, and the noise injection in the pixel conversion constraint part uses Gaussian noise. , the Gaussian kernel size of the Gaussian blur processing in the pixel transformation constraint part is ≤3×3.

[0008] By adopting the above technical solutions, the quality of image data can be enhanced by utilizing the change optimization, color gamut transformation adjustment and pixel transformation constraint solutions.

[0009] Preferably, the improved generative adversarial network (D-GAN) augmentation process in step 2 includes PU risk optimization and hyperparameter adaptation. The PU risk optimization adopts Divergent-GAN (D-GAN), introduces a biased PU risk term in the discriminator loss function, and uses positive samples and unlabeled mixed samples to generate high-authenticity abnormal samples. The positive samples are normal NT images, and the unlabeled mixed samples are normal NT images and abnormal normal NT images; the hyperparameter adaptation process is based on the Bayesian optimization algorithm, and dynamically adjusts the learning rate. The initial value of the dynamically adjusted learning rate is 0.0002, the range of the dynamically adjusted learning rate is 0.0001-0.001, the batch size range of the dynamically adjusted learning rate is 16~64, the momentum term β1 of the dynamically adjusted learning rate ranges from 0.5~0.9, and the training cycle of the dynamically adjusted learning rate is reduced from 500 rounds to 300 rounds.

[0010] By adopting the above technical solution and utilizing the generative adversarial network augmentation scheme, abnormal samples can be generated to enrich the diversity of the data set.

[0011] Preferably, the dynamic programming combination optimization process based on grouped backpack in step three includes a problem modeling part and a greedy algorithm solving part. The problem modeling part divides the 17 augmentation methods into geometric groups, color gamut groups, and pixel groups, selects only one method from each group, and constructs a grouped backpack model with the weighted F1 score as the value and the computational cost as the weight; the greedy algorithm solving part sorts in descending order based on the "value / weight" ratio, gives priority to methods that significantly improve F1 and have low computational cost, and generates the optimal combination solution.

[0012] By adopting the above technical solution and building a model based on weighted F1 score and computational cost, effective combinations can be screened out from a large number of augmentation method combinations.

[0013] Preferably, the model training and feedback optimization process in step 4 includes a multi-model verification part and a closed-loop optimization part. The multi-model verification part uses ResNet and DenseNet to train the augmented data set to test the accuracy and F1 value; the closed-loop optimization part iteratively adjusts the augmentation combination and GAN hyperparameters according to the multi-model verification results until a preset threshold is reached.

[0014] By adopting the above technical solution, the adaptability of data augmentation and model training can be continuously improved through model training and feedback optimization.

[0015] Preferably, the method is run using a fetal NT ultrasound image data augmentation system, which includes a transformation augmentation module, a generative adversarial augmentation module, a combinatorial optimization module, and an image classification module. The transformation augmentation module performs quality control on NT ultrasound images. The generative adversarial augmentation module uses an improved D-GAN to generate abnormal samples to improve the diversity of the data set. The combinatorial optimization module models the grouped knapsack problem and selects the optimal augmentation combination through a cost optimization algorithm. The image classification module verifies the image augmentation effect and outputs the model accuracy and F1 value.

[0016] By adopting the above technical solution and utilizing the constructed fetal NT ultrasound image data augmentation system, data expansion of NT ultrasound images can be achieved.

[0017] Compared with the prior art, the present invention has the following beneficial effects: the method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization: 1. This invention uses a multi-method fusion augmentation method to augment NT images, combining medical knowledge-based image transformation augmentation with an improved D-GAN generative adversarial network. While retaining the stability of traditional transformations, it generates highly realistic abnormal samples, covering both normal and abnormal scenarios, significantly improving the diversity of the dataset. Furthermore, during image transformation, masks are used to limit the operating area, control the noise injection intensity, and blur kernel size to avoid damaging key structures in the NT transparent layer, ensuring that the augmented images meet medical diagnostic standards. This improves the accuracy and efficiency of fetal NT ultrasound image analysis, providing reliable technical support for intelligent prenatal screening. 2. Based on the grouped knapsack model and greedy algorithm, this paper automatically screens the optimal combination from 17 augmentation methods, and uses the ratio of "prediction performance value / computation time" as the basis to balance the augmentation effect and computational efficiency, avoiding the blindness of random combinations. At the same time, the augmentation effect is verified by ResNet and DenseNet. The augmentation strategy and GAN hyperparameters are iteratively adjusted based on accuracy and F1 value, forming an optimization closed loop, steadily improving the model's generalization ability, and providing more reliable data support for prenatal screening. The combined optimization framework can be migrated to other medical image analysis tasks, increasing the versatility of the technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the process structure of the method of the present invention; Figure 2 This is a schematic diagram of the system architecture of the present invention; Figure 3 This is a schematic diagram of the classification structure of the transformation and augmentation module method of the present invention; Figure 4 This is a schematic diagram of the improved D-GAN network structure of the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] See also Figure 1-Figure 4 The present invention provides a technical solution: a high-quality data expansion method for fetal NT ultrasound images based on dynamic programming optimization.

[0021] Methods: A fetal NT ultrasound image data augmentation system was used. The system includes a transformation augmentation module, a generative adversarial augmentation module, a combinatorial optimization module, and an image classification module. The transformation augmentation module performs quality control on NT ultrasound images. The generative adversarial augmentation module uses an improved D-GAN to generate abnormal samples and improve the diversity of the dataset. The combinatorial optimization module models the grouped knapsack problem and selects the optimal augmentation combination through a cost optimization algorithm. The image classification module verifies the image augmentation effect and outputs the model accuracy and F1 value. like Figure 2 As shown in the figure, before data expansion, the fetal NT ultrasound image data augmentation system is first built. The fetal NT ultrasound image data augmentation system has built-in transformation augmentation module, generative adversarial augmentation module, combination optimization module and image classification module. Each module is used to implement different functions. The transformation augmentation module is used to control the quality of the input NT ultrasound image to solve the problems of insufficient original data and uneven quality. The generative adversarial augmentation module uses an improved D-GAN to generate abnormal samples to improve the diversity of the data set. The combination optimization module selects the optimal augmentation combination through an algorithm, and the image classification module verifies the image augmentation effect to ensure the diagnostic quality of the final NT ultrasound image.

[0022] Step 1: The medical knowledge-based image transformation and augmentation process includes geometric change optimization, color gamut transformation adjustment, and pixel transformation constraints. The geometric change optimization ensures the integrity of the NT transparent layer structure through rotation to remove black edges and cropping optimization operations; the color gamut transformation adjustment adaptively adjusts the contrast and saturation according to the brightness distribution of the NT ultrasound image to avoid distortion of medical features; the pixel transformation constraint protects the edge clarity of the NT transparent layer through noise injection and Gaussian blur processing, and limits the operation area through masking to avoid the key area of ​​the NT transparent layer. When the geometric change optimization part rotates the NT image, the effective area is dynamically calculated based on the image width, height, and rotation angle. The rotated NT image is cropped. The calculation formula for the cropping range is as follows: ; ; ; ; The mapping range of contrast adjustment in the color gamut adjustment part is ±15% of the original value, the adjustment range of saturation in the color gamut adjustment part is limited to ≤20%, and the noise injection in the pixel transformation constraint part uses Gaussian noise. ,The Gaussian kernel size of Gaussian blur processing in the pixel transformation constraint part is ≤3×3; like Figure 1 and Figure 3 As shown in the figure, image transformation augmentation is used to improve the quality control of fetal NT ultrasound images and solve the problems of insufficient original data and uneven quality. First, geometric change optimization is performed: rotation to remove black edges, the image is rotated at a random angle of ±30°, and the black edges generated by the rotation are removed by cropping; flipping and cropping, after flipping the image horizontally or vertically, the key NT area is cropped according to a preset ratio, such as 80%; secondly, a color gamut transformation adjustment process is performed, randomly adjusting the image brightness and contrast to simulate images under different lighting conditions. The adjustment range of the image brightness is ±20%, and the adjustment range of the contrast is ±15%. Then, a color space conversion is performed, and the RGB image is converted to HSV or Lab space and the saturation is randomly adjusted. The saturation adjustment range is ±25%, and then converted back to RGB space; finally, pixel transformation constraints are performed, and Gaussian noise and salt and pepper noise are added to simulate real ultrasound image noise. The Gaussian noise , the density of salt and pepper noise = 0.02; use Gaussian blur and random rectangular occlusion to blur and occlude the image, the kernel size of Gaussian blur = 3×3, and the blur occlusion ratio is 10%-20%.

[0023] In step 2, the improved generative adversarial network (D-GAN) augmentation process includes PU risk optimization and hyperparameter adaptation. PU risk optimization uses Divergent-GAN (D-GAN), introduces a biased PU risk term into the discriminator loss function, and uses positive samples and unlabeled mixed samples to generate high-authenticity abnormal samples. The positive samples are normal NT images, and the unlabeled mixed samples are normal NT images and abnormal normal NT images. The hyperparameter adaptation process is based on the Bayesian optimization algorithm and dynamically adjusts the learning rate. The initial value of the dynamic adjustment learning rate is 0.0002, the range of the dynamic adjustment learning rate is 0.0001-0.001, the batch size range of the dynamic adjustment learning rate is 16-64, the momentum term β1 of the dynamic adjustment learning rate ranges from 0.5-0.9, and the training cycle of the dynamic adjustment learning rate is reduced from 500 rounds to 300 rounds. like Figure 4 As shown in the figure, is a positive sample image, is an unlabeled sample image, Generate a network for images, is a hidden variable, is the image discrimination network, is the image binary classification network, In order to generate error counterexample samples, the improved D-GAN generative adversarial augmentation process includes the counterexample generation stage and the classification stage. By introducing the PU risk term and the hyperparameter adaptive strategy, high-quality fetal NT ultrasound image data augmentation is achieved. Obtain positive sample images from a clinically collected normal fetal NT ultrasound image dataset , represents the normal fetal NT ultrasound image; Use normal NT images and a small number of abnormal NT images to form unlabeled sample images , used to provide more complex sample distribution information for the discriminator; The hidden variable Input into the image generation network G, and use the image generation network G to generate hidden variables Process and generate error counterexample samples ; The generated samples Unlabeled sample images Input to the image discrimination network In the image discrimination network Determine whether the input sample is an unlabeled sample image Or generate samples ; The Bayesian optimization algorithm is used to dynamically adjust the hyperparameters of D-GAN. The initial value of the learning rate is set to 0.0002, and the dynamic adjustment range is 0.0001-0.001. In the early stages of training, a larger learning rate can be used to accelerate convergence. As training progresses, the learning rate is gradually reduced to enable the model to adjust parameters more accurately. The batch size adjustment range is 16-64. Through adaptive adjustment of hyperparameters, the training cycle is reduced from the traditional 500 rounds to 300 rounds, reducing training time and computing resource consumption while ensuring model performance.

[0024] The dynamic programming combinatorial optimization process based on grouped knapsacks in step three includes a problem modeling phase and a greedy algorithm solution phase. The problem modeling phase divides the 17 augmentation methods into geometric, color gamut, and pixel groups, selecting only one method from each group. A grouped knapsack model is constructed, using the weighted F1 score as the value and the computational cost as the weight. The greedy algorithm solution phase sorts the methods in descending order based on the "value / weight" ratio, prioritizing methods that significantly improve F1 and have low computational cost to generate the optimal combination solution. The combinatorial optimization is abstracted into a grouped knapsack problem. Each group represents a type of augmentation method, such as geometry, color gamut, pixel, etc. The methods in each group are mutually exclusive. The mathematical model of the grouped knapsack can be described as follows: Suppose there is a set of n items , these items are divided into m groups, , for any , every item , all valuable and weight , the total capacity of the backpack is , define the decision variables as follows: ; The mathematical model of the group knapsack problem can be expressed as an integer linear programming problem: ; ; in , for any ,if and In the same group, That is, there is only one item in this group. The greedy strategy is sorted by the ratio of "the improvement of the augmentation method on the F1 score / computational cost", and the most cost-effective method is given priority. Based on the feedback from the validation set, the combination scheme is iteratively optimized.

[0025] Step 4: The model training and feedback optimization process includes a multi-model validation phase and a closed-loop optimization phase. The multi-model validation phase uses ResNet and DenseNet to train the augmented dataset and test the accuracy and F1 value. The closed-loop optimization phase iteratively adjusts the augmentation combination and DD-GAN hyperparameters based on the multi-model validation results until the preset threshold is reached. First, a preliminary experiment was conducted to test the F1 improvement of each augmentation method on ResNet and DenseNet separately. Then, a combination was generated using a greedy algorithm. Finally, the F1 score of the combination was tested on the validation set. If the F1 score of the combination did not meet the expectations, the combination was readjusted to avoid the blindness of random combinations. By optimizing and selecting augmentation methods with strong complementarity, the model performance improvement was maximized under limited computing resources. Experimental verification and effect analysis: A dataset of fetal NT ultrasound images was obtained. The dataset includes 776 fetal NT ultrasound images, including 376 normal ultrasound images and 400 abnormal ultrasound images. The training set, validation set, and test set are divided into 8:1:1. Four experimental groups are set: experimental control group, single optimal augmentation group, combination optimization group, and random combination group. The experimental control group does not undergo data augmentation. The single optimal augmentation group uses a single augmentation method, such as blurring and rotation. The combination optimization group generates 10 method optimization combinations using a greedy algorithm. The random combination group randomly selects 10 method combinations. Classification accuracy (Accuracy) and weighted F1 score are used as evaluation indicators. The experimental results are shown in the following table:

[0026] Data analysis of the results in the upper surface shows that the combination optimization group significantly improves the accuracy by 27.3%, indicating that noise simulation and detail retention effectively enhance the robustness of the model. The key area protection strategy ensures that the augmented images still meet medical standards and avoids misdiagnosis due to excessive deformation. The accuracy of the combination optimization group on ResNet and DenseNet is 4.9% and 4.3% higher than that of the random combination group, respectively, verifying the effectiveness of the proposed combination optimization algorithm. The weighted F1 score shows that the combination optimization is more adaptable to the problem of class imbalance.

[0027] Working principle: First, the NT ultrasound image is transformed and augmented to achieve control and management of the NT ultrasound image, ensuring image clarity and accurate and reliable measurement results. The improved D-GAN is used to generate abnormal samples to improve the diversity of the data set. Then, based on the dynamic programming combination optimization of the grouped backpack, a grouped backpack model is constructed. The optimal augmentation combination is selected through the greedy algorithm, and the model is trained and optimized with feedback. The augmentation effect is verified by multiple models, and the model accuracy and F1 value are output. The combination result is selected according to the verification result to avoid the blindness of random combination. By optimizing and selecting the augmentation method with strong complementarity, the model performance is maximized under limited computing resources.

[0028] While the embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that various changes, modifications, substitutions, and alterations can be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. A method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization, characterized by: The following steps are involved: Step 1: Medical knowledge-guided image transformation and augmentation, transforming NT ultrasound images, and controlling and managing NT ultrasound images to ensure clear and standardized images and accurate and reliable measurement results; Step 2: Improved Generative Adversarial Network (D-GAN) augmentation: Optimize and improve the traditional Generative Adversarial Network (GAN), use the improved D-GAN to generate abnormal samples and improve the diversity of the dataset; Step 3: Based on the dynamic programming combination optimization of grouped backpacks, a grouped backpack model is constructed and the optimal augmented combination is selected through a greedy algorithm; Step 4: Model training and feedback optimization; use multiple models to verify the augmentation effect and output the model accuracy and F1 value.

2. The method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization according to claim 1, characterized in that: The medical knowledge-guided image transformation and augmentation process in step 1 includes geometric change optimization, color gamut transformation adjustment, and pixel transformation constraints. The geometric change optimization ensures the integrity of the NT transparent layer structure through rotation to remove black edges and cropping optimization operations; the color gamut transformation adjustment adaptively adjusts the contrast and saturation according to the brightness distribution of the NT ultrasound image to avoid distortion of medical features; The pixel transformation constraint protects the edge clarity of the NT transparent layer through noise injection and Gaussian blur processing, and limits the operation area through a mask to avoid the key area of ​​the NT transparent layer.

3. The method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization according to claim 2, characterized in that: When the geometric change optimization part rotates the NT image, the effective area is dynamically calculated based on the image width and height and the rotation angle. The rotated NT image is cropped. The calculation formula of the cropping range is as follows: ； ； ； ； The mapping range of contrast adjustment in the color gamut conversion adjustment part is ±15% of the original value, the adjustment range of saturation in the color gamut conversion adjustment part is limited to ≤20%, and the noise injection in the pixel conversion constraint part uses Gaussian noise. , the Gaussian kernel size of the Gaussian blur processing in the pixel transformation constraint part is ≤3×3.

4. The method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization according to claim 1, characterized in that: The improved generative adversarial network (D-GAN) augmentation process in step 2 includes PU risk optimization and hyperparameter adaptation. The PU risk optimization adopts Divergent-GAN (D-GAN), introduces a biased PU risk term in the discriminator loss function, and uses positive samples and unlabeled mixed samples to generate high-authenticity abnormal samples. The positive samples are normal NT images, and the unlabeled mixed samples are normal NT images and abnormal normal NT images. The hyperparameter adaptation process is based on the Bayesian optimization algorithm and dynamically adjusts the learning rate. The initial value of the dynamically adjusted learning rate is 0.0002, the range of the dynamically adjusted learning rate is 0.0001-0.001, the batch size range of the dynamically adjusted learning rate is 16~64, the momentum term β1 of the dynamically adjusted learning rate ranges from 0.5~0.9, and the training cycle of the dynamically adjusted learning rate is reduced from 500 rounds to 300 rounds.

5. The method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization according to claim 1, characterized in that: The grouped knapsack-based dynamic programming combinatorial optimization process in step three includes a problem modeling part and a greedy algorithm solution part. The problem modeling part divides the 17 augmentation methods into geometric groups, color gamut groups, and pixel groups, selecting only one method from each group, and constructing a grouped knapsack model with the weighted F1 score as the value and the computational cost as the weight. The greedy algorithm solution part sorts the methods in descending order based on the "value / weight" ratio, giving priority to methods that significantly improve F1 and have low computational cost to generate the optimal combination solution.

6. The method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization according to claim 1, characterized in that: The model training and feedback optimization process in step 4 includes a multi-model verification part and a closed-loop optimization part. The multi-model verification part uses ResNet and DenseNet to train the augmented data set to test the accuracy and F1 value; the closed-loop optimization part iteratively adjusts the augmentation combination and D-GAN hyperparameters according to the multi-model verification results until the preset threshold is reached.

7. The method for high-quality data expansion of fetal NT ultrasound images based on dynamic programming optimization according to claim 1, characterized in that: The method is run using a fetal NT ultrasound image data augmentation system, which includes a transformation augmentation module, a generative adversarial augmentation module, a combinatorial optimization module, and an image classification module. The transformation augmentation module performs quality control on NT ultrasound images. The generative adversarial augmentation module uses an improved D-GAN to generate abnormal samples and improve data set diversity. The combinatorial optimization module models the grouped knapsack problem and selects the optimal augmentation combination through a cost optimization algorithm. The image classification module verifies the image augmentation effect and outputs the model accuracy and F1 value.

Images