Generative adversarial network and optimal transmission-based teenager and child bone data enhancement model construction method

Abstract

The invention belongs to the related technical field of deep learning and medical image processing, and particularly relates to a juvenile and children bone data enhancement model construction method based on a generative adversarial network and optimal transmission, and the method comprises the steps: constructing a CNN-GAN-OTD network architecture, and enabling the feature extraction operation of a GAN network structure to be realized through a CNN; a training sample set is adopted to train the CNN-GAN-OTD network architecture, and in the training process, a loss function of a GAN network generator is a feature distribution distance between a minimum generated value and a true value defined and constructed based on the OT theory. According to the invention, the problems of small and unbalanced data set samples in medical image tasks such as bone age evaluation and the like can be solved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to deep learning and medical image processing, and more specifically, to a method for constructing a bone data enhancement model for adolescents and children based on a generative adversarial network and optimal transmission. Background Art

[0002] Among the many applications of deep learning, medical image analysis is a key area. Bone age assessment, a key task in medical image analysis, is crucial for diagnosing childhood growth and developmental disorders, predicting adult height, selecting athletes, and forensic identification. However, deep learning research in bone age assessment faces numerous challenges.

[0003] One major issue is dataset limitations. Medical image datasets are often difficult to share due to concerns about patient privacy, while health data is relatively more readily available. This often results in datasets with small sample sizes and class imbalances. For example, in bone age datasets, the number of samples varies significantly across age groups, with some age groups being scarce. This severely impacts the training effectiveness and generalization capabilities of deep learning models.

[0004] Traditional data augmentation methods, such as affine transformations (including rotation, flipping, and translation) and color correction, offer advantages such as speed, high repeatability, and ease of use. However, these methods suffer from significant drawbacks. The augmented data they generate is highly correlated with the original data, performing only simple transformations on the original data. Consequently, the amount of information added is extremely limited, making it difficult to meet the diverse data requirements of deep learning models and effectively addressing issues such as insufficient and unbalanced sample sizes.

[0005] In recent years, generative adversarial networks (GANs) have demonstrated promising potential in fields such as image generation. GANs consist of two components: a generator and a discriminator, which optimize the quality of generated data through adversarial learning. However, the original GAN structure has serious shortcomings. During training, when the discriminator reaches its optimal state, optimizing the generator is equivalent to optimizing the Jenson-Shannon divergence (JSD) between the true data distribution and the generated data distribution. However, in actual medical image data, the support sets of the true data distribution and the generated data distribution often have no intersection or a very small intersection. In this case, the definition of JSD becomes unreasonable, leading to frequent pattern collapse. This significantly reduces the stability and reliability of GANs in data augmentation applications, making it unable to effectively provide high-quality augmented data for tasks such as bone age assessment.

[0006] Therefore, there is an urgent need for a new data augmentation method that can overcome the defects of traditional methods and existing GAN structures, effectively increase the diversity and quality of data, and improve the performance of deep learning models in medical image tasks such as bone age assessment. Summary of the Invention

[0007] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides a method for constructing a bone data enhancement model for adolescents and children based on generative adversarial networks and optimal transmission, which aims to solve the problem of small and unbalanced data set samples in medical image tasks such as bone age assessment.

[0008] To achieve the above objectives, according to one aspect of the present invention, a method for constructing a bone data enhancement model for adolescents and children based on a generative adversarial network and optimal transmission is provided, comprising:

[0009] Construct a CNN-GAN-OTD network architecture, where the feature extraction operation of the GAN network structure is implemented through a convolutional neural network (CNN);

[0010] The CNN-GAN-OTD network architecture is trained using a training sample set. During the training process, the loss function of the GAN network generator is defined based on the OT theory to minimize the feature distribution distance between the generated value and the true value.

[0011] Furthermore, the loss function Dist of the GAN network generator CG (p r ,p g )for:

[0012]

[0013] Where p r represents the real data distribution, p g represents the generated data distribution, x r and x g They represent the distribution of real data p r and generate data distribution p g The sample obtained by sampling, x r and x′ r They represent the distribution of real data p r Two different samples obtained by independent sampling, x g and x′ g They represent the data distribution p generated from g Two different samples obtained by independent sampling, d c represents the distance metric function based on cosine similarity, represents the mathematical expectation,

[0014] Furthermore, the activation function layer of the discriminator in the GAN network is the LeakyReLU activation function layer.

[0015] According to another aspect of the present invention, a method for enhancing bone data of adolescents and children is provided, wherein the bone data enhancement model of adolescents and children constructed by the construction method described above is used to perform data enhancement.

[0016] According to another aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, which includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the steps of the above-mentioned method.

[0018] According to another aspect of the present invention, a computer program product is provided, comprising a computer program or instructions, which implement the steps of the method described above when executed by a processor.

[0019] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0020] 1. The present invention proposes a method for constructing a bone data enhancement model for adolescents and children based on a generative adversarial network and optimal transmission. First, a key improvement is made to the original GAN structure, and all the multi-layer perceptrons (MLP) therein are replaced with convolutional neural networks (CNN). CNN has significant advantages in the field of images. It can automatically learn the feature representation of images and efficiently process image data through structures such as convolutional layers and pooling layers, thereby greatly enhancing the performance of GAN in processing image data, enabling it to better capture the complex feature information in medical images. In addition, by converting the loss function of GAN into a form based on OT theory, the transmission of data distribution forms a convex function, ensuring the uniqueness of the optimal solution and effectively avoiding the instability problems that may occur in traditional GAN training. Therefore, the present invention can effectively solve the key problems of insufficient and unbalanced data set sample size faced in medical image tasks such as bone age assessment, significantly improve the quality and diversity of generated data, and thus greatly enhance the performance of deep learning models in these tasks, ensuring the accuracy and reliability of bone age assessment results.

[0021] 2. This paper further proposes using a LeakyReLU activation function layer as the activation function for the discriminator in the GAN network. By introducing a small slope in the negative range, this layer addresses the neuron death problem of ReLU, maintains gradient flow during negative inputs, and improves training stability. This layer is computationally efficient and preserves sparsity, making it suitable for this task. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic flow chart of a method for constructing a bone data enhancement model for adolescents and children based on a generative adversarial network and optimal transmission provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of the basic structure of a generative adversarial network provided by an embodiment of the present invention;

[0024] Figure 3 This is a basic training flow chart provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0026] Example 1

[0027] A method for constructing a bone data enhancement model for adolescents and children based on generative adversarial networks and optimal transmission, such as Figure 1 As shown, including:

[0028] Construct a CNN-GAN-OTD network architecture, where the feature extraction operation of the GAN network structure is implemented through a convolutional neural network (CNN);

[0029] The CNN-GAN-OTD network architecture is trained using a training sample set. During the training process, the loss function of the GAN network generator is defined based on the OT theory to minimize the feature distribution distance between the generated value and the true value.

[0030] This example proposes a unique network architecture (CNN-GAN-OTD, abbreviated as Convolutional Neural Network-Generative Adversarial Network-Optimal Transmission Distance). First, a key improvement is made to the original GAN structure, replacing all multi-layer perceptrons (MLPs) with convolutional neural networks (CNNs). CNNs have significant advantages in the field of imaging. They can automatically learn feature representations of images and efficiently process image data through structures such as convolutional layers and pooling layers. This greatly enhances the performance of GANs in processing image data, enabling them to better capture the complex feature information in medical images.

[0031] Regarding the deep fusion optimal transfer (OT) theory, the method of this embodiment has made an innovative transformation to the loss function of GAN. The core of the OT theory is to find a mapping with the minimum transmission cost between the source distribution and the target distribution. In this embodiment, by converting the loss function of GAN into a form based on OT theory, the transmission of data distribution forms a convex function, which ensures the uniqueness of the optimal solution and effectively avoids the instability problems that may occur in traditional GAN training. In the key link of distance measurement, after in-depth research and experimental comparison, the present invention selects cosine distance as the basic form of transmission distance. Unlike traditional metrics such as Euler distance, cosine distance is highly insensitive to the scale and outliers of the data. It can accurately measure the distribution difference between generated data and real data in a complex medical image data environment, thereby providing a strong guarantee for generating high-quality enhanced data.

[0032] During the specific training process, the generator takes noise samples z drawn from a uniform distribution between [-1, 1] as input, and performs complex transformations and feature extraction through its internal CNN structure to ultimately generate data with a distribution of pg(x). The discriminator receives both the real dataset XR and the generated data xg, according to a carefully designed formula:

[0033]

[0034] (in ) accurately calculates the transmission distance between the distribution of the real dataset and the distribution of synthetic data. Through continuous adversarial training, the generator and discriminator compete with each other to continuously optimize the distance between the generated data and the real data. This enables the generator to gradually generate high-quality data that is highly similar to the real data distribution and rich in diversity, effectively expanding the original dataset and providing sufficient and high-quality training data for deep learning models.

[0035] That is, the loss function Dist of the GAN network generator can be used as an optimization. CG (p r ,p g )for:

[0036]

[0037] Where p r represents the real data distribution, p g represents the generated data distribution, x r and x g They represent the distribution of real data p r and generate data distribution p g The sample obtained by sampling, x r and x′ r They represent the distribution of real data p r Two different samples obtained by independent sampling, x g and x′ g They represent the data distribution p generated from g Two different samples obtained by independent sampling, d c represents the distance metric function based on cosine similarity, represents the mathematical expectation,

[0038] In addition, as a preferred implementation, the activation function layer of the discriminator in the GAN network is a LeakyReLU activation function layer.

[0039] Figure 2 The basic structure of a generative adversarial network is demonstrated. In this embodiment, both the generator and the discriminator are built based on CNN. The generator gradually converts input noise into image data with rich features through multiple convolutional layers and upsampling layers, while the discriminator uses convolutional layers and fully connected layers to determine the authenticity of the input data and calculate the transmission distance from the true data distribution. This fully demonstrates the improvement of the original GAN structure of the present invention. Figure 3 This is a detailed flowchart of CNN-GAN-OTD, which clearly shows the flow path and processing process of data in the network. It starts with the generator receiving noise samples, generates data through a series of convolution and nonlinear transformations, and then the discriminator processes the real data and generated data and calculates the distance. Finally, the parameters of the generator and discriminator are updated through backpropagation. The whole process repeats until the network reaches a balanced state and generates high-quality enhanced data.

[0040] This embodiment aims to solve the problem of small and unbalanced data set samples in medical image tasks such as bone age assessment. By constructing a unique CNN-GAN-OTD network architecture, the multi-layer perceptron in the original GAN structure is replaced by a convolutional neural network, and the optimal transmission theory is integrated to transform the loss function, and the cosine distance is used to measure the data distribution difference. The CNN-GAN-OTD method of this embodiment was rigorously verified on the public data set CIFAR-10. The experimental results show that, whether it is the generated result based on the original data or the generated result after adding different degrees of noise to the original data, this embodiment has achieved extremely excellent results in two important evaluation indicators, the IS coefficient and the FID coefficient. Compared with other existing data enhancement methods, such as traditional data enhancement methods based on image affine transformation, WGAN, etc., the data generated by the CNN-GAN-OTD method is more realistic in visual effects, has richer and more diverse features, and has significantly lower sensitivity to noise. This fully proves that the present embodiment can generate high-quality, high-diversity enhanced data, effectively making up for the shortcomings of the original data set.

[0041] In the key medical image task of bone age classification, CNN-GAN-OTD was used as a data enhancement module for in-depth indirect comparative experiments. The experimental results clearly show that the classification results based on CNN-GAN-OTD data enhancement are significantly better in accuracy than those based on other traditional data enhancement methods. By using the data enhancement method provided in this embodiment, the performance of the deep learning model in the bone age assessment task has been greatly improved, and bone age can be predicted more accurately, providing reliable support for medical diagnosis and research, and has important clinical application value.

[0042] An implementation example is given as follows:

[0043] (1) Network construction

[0044] Generator and Discriminator Structural Design: The generator adopts a multi-layer structure based on CNN. First, the input layer receives noise samples z drawn from a uniform distribution in the range [-1, 1]. Feature extraction and amplification are then performed through a series of convolutional layers, batch normalization layers, and ReLU activation function layers. For example, the initial convolutional layer can use a smaller convolution kernel (such as 3x3) and an appropriate stride (such as 1). As the network progresses, the number of convolution kernels and the size of the feature map are gradually increased. Upsampling operations (such as transposed convolution) gradually convert the low-dimensional noise data into high-dimensional image data. The discriminator is also built on CNN and consists of multiple convolutional layers, batch normalization layers, and LeakyReLU activation function layers. The design of its convolutional layers matches that of the generator. By gradually reducing the size of the feature map and increasing the number of convolution kernels, features are extracted and analyzed for the input real data and generated data. Finally, the judgment result and transmission distance are output through the fully connected layer.

[0045] Determination of network parameters: During the experiment, after multiple rounds of optimization and adjustment, a relatively suitable set of network parameters was determined. The batch size was set to 64. This parameter fully utilizes the GPU's computing resources while ensuring training efficiency and avoiding memory overflow issues. The learning rate was set to 0.0002, and the Adam optimizer was used for parameter updates. The Adam optimizer can adaptively adjust the learning rate to accelerate training convergence. The negative slope of the LeakyReLU activation function was set to 0.2, which helps alleviate the gradient vanishing problem while ensuring the network's nonlinear expression capabilities. However, it should be noted that these parameters are not fixed. In actual applications, they can be appropriately adjusted and optimized based on factors such as the specific characteristics of the dataset and the performance of the hardware device.

[0046] (2) Data preparation

[0047] Data Collection and Preprocessing: For the bone age assessment task, a large dataset of hand X-ray images was collected and comprehensively preprocessed. First, the images were normalized, mapping pixel values to the range [0, 1] or [-1, 1]. This ensured consistent data scale across images, facilitating network training and processing. Furthermore, the images were subjected to operations such as denoising and contrast enhancement to improve image quality and clarity, reducing the impact of noise and blur on model training.

[0048] Dataset Partitioning: The preprocessed dataset is divided into training, validation, and test sets according to a specific ratio. Typically, a ratio of 8:1:1 is used: 80% of the data is used as the training set for model training, 10% of the data is used as the validation set for adjusting model hyperparameters and monitoring performance changes during training, and 10% of the data is used as the test set for final evaluation of model performance and generalization. During the partitioning process, ensure that the data in each set is representative and random to avoid biased data distribution that could affect experimental results.

[0049] (3) Training process

[0050] Generator Training: During training, the generator extracts noise samples z from a uniform distribution in the range [-1, 1] and feeds them into the constructed CNN architecture. After a series of convolution, batch normalization, and activation functions, data xg is generated. The generator's goal is to continuously adjust its parameters so that the generated data xg closely resembles the distribution of real data, thereby deceiving the discriminator. In each training iteration, the generator's parameters are updated using the backpropagation algorithm based on the loss information fed back by the discriminator, gradually improving the quality of the generated data.

[0051] Discriminator training: The discriminator receives both the real dataset XR and the data xg generated by the generator. For the real data, the discriminator must accurately identify it as a real sample, outputting a probability close to 1. For the generated data, the discriminator must accurately identify it as a fake sample, outputting a probability close to 0. The discriminator calculates the transmission distance between the real dataset distribution pr(x) and the synthetic data distribution pg(x) according to the distance formula defined above and uses this as part of the loss function. The discriminator's parameters are updated through the backpropagation algorithm, enabling it to better distinguish between real and generated data while accurately measuring the difference between the data distributions.

[0052] Adversarial training and optimization: Throughout the training process, the generator and the discriminator are trained alternately, forming an adversarial process. In each round of training, the parameters of the discriminator are first fixed, and the generator is trained several times to make the data it generates more realistic; then the parameters of the generator are fixed, and the discriminator is trained several times to improve the discriminant's discrimination ability. By continuously repeating this adversarial training process, the generator and the discriminator promote each other and gradually reach a balanced state. During the training process, the performance indicators of the model on the validation set (such as IS coefficient, FID coefficient, and bone age classification accuracy, etc.) are continuously monitored, and the training parameters and strategies, such as the decay of the learning rate and the method of data enhancement, are adjusted according to the changes in the indicators to ensure that the model can be continuously optimized, and ultimately obtain high-quality generated data, effectively expand the original data set, and improve the performance of deep learning models in tasks such as bone age assessment.

[0053] In actual implementation, it is necessary to closely monitor the network's training status and performance changes, and flexibly adjust the network structure, parameters, and training strategy based on specific circumstances to achieve optimal data augmentation and model performance. Furthermore, further exploration of integration with other advanced technologies, such as more complex CNN architectures and attention mechanisms, can be explored to continuously expand the application potential and performance improvement potential of this invention.

[0054] Example 2

[0055] A method for enhancing bone data of adolescents and children uses the adolescent and child bone data enhancement model constructed by the construction method described above to perform data enhancement.

[0056] The relevant technical solutions are the same as above and will not be repeated here.

[0057] Example 3

[0058] The present application also relates to an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0059] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The processor may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The memory may be used to store computer programs and / or modules, and the processor may perform various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory.

[0060] The relevant technical solutions are the same as above and will not be repeated here.

[0061] Example 4

[0062] The present application also relates to a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0063] Specifically, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0064] The relevant technical solutions are the same as above and will not be repeated here.

[0065] Example 5

[0066] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the method of the above-described embodiment of the present invention.

[0067] The relevant technical solutions are the same as above and will not be repeated here.

[0068] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a bone data enhancement model for adolescents and children based on generative adversarial networks and optimal transmission, characterized by: include: Construct a CNN-GAN-OTD network architecture, where the feature extraction operation of the GAN network structure is implemented through a convolutional neural network (CNN); The CNN-GAN-OTD network architecture is trained using a training sample set. During the training process, the loss function of the GAN network generator is defined based on the OT theory to minimize the feature distribution distance between the generated value and the true value.

2. The method for constructing a bone data enhancement model for adolescents and children according to claim 1, wherein: The loss function Dist of the GAN network generator CG (p r ,p g )for: Where p r represents the real data distribution, p g represents the generated data distribution, x r and x g They represent the distribution of real data p r and generate data distribution p g The sample obtained by sampling, x r and x′ r They represent the distribution of real data p r Two different samples obtained by independent sampling, x g and x′ d They represent the data distribution p generated from g Two different samples obtained by independent sampling, d c represents the distance metric function based on cosine similarity, represents the mathematical expectation, 3. The method for constructing a bone data enhancement model for adolescents and children according to claim 1, wherein: The activation function layer of the discriminator in the GAN network is the LeakyReLU activation function layer.

4. A method for enhancing bone data of adolescents and children, characterized in that: Data enhancement is performed using the adolescent and children bone data enhancement model constructed by the construction method according to any one of claims 1 to 3.

5. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to perform the steps of the method according to any one of claims 1 to 4.

7. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images