A grid deformation data enhancement method based on a WGAN-GP model

Abstract

The application discloses a kind of grid deformation data enhancement methods based on WGAN-GP model, it is related to image processing, data enhancement technical field, including: constructing training dataset and test dataset;WGAN-GP model is constructed;WGAN-GP model is trained based on training dataset;The performance of model is evaluated using test dataset, determine grid deformation data enhancement model;Random noise is respectively used as input with medical endoscope image, output corresponding deformation grid, respectively with medical endoscope image and deformation grid warp operation obtains the medical endoscope image after data enhancement.This application constructs WGAN-GP model and automatically generates deformation grid, by deformation grid to medical endoscope image is carried out deformation enhancement, enhanced medical endoscope image is real and has certain diversity, effectively solve the problem of data sample scarcity and data sample unbalance, to improve the precision and generalization of artificial intelligence algorithm model based on medical endoscope image.

Description

Technical Field

[0001] This invention relates to the fields of image processing and data augmentation technology, and more specifically to a mesh deformation data augmentation method based on the WGAN-GP model. Background Technology

[0002] In recent years, AI technology has been widely applied in medical image analysis and medical decision-making, driving traditional medical research into a new stage. As an important research direction of "AI + Medicine," AI-driven medical image analysis technology is becoming increasingly mature. However, due to the sensitivity and privacy of medical image data, and the extreme difficulty of high-quality data annotation, most medical image datasets are relatively small, which limits the application of AI models in the field of medical image analysis. Data augmentation technology expands existing medical image data (especially those with a limited number of categories), making the data samples richer and more balanced, effectively improving the performance of AI models (especially deep neural network models) in the field of medical image analysis.

[0003] Currently, in the field of medical endoscopic image data augmentation, geometric transformation techniques such as translation, flipping, rotation, scaling, cropping, and affine transformations are commonly used to augment existing medical endoscopic images. These traditional methods alleviate the problem of insufficient data samples to some extent, but they rely on manually preset parameters and cannot be specifically designed for medical endoscopic images, resulting in poor realism of the augmented image data. Consequently, the augmented data offers limited improvement in accuracy for subsequent tasks such as lesion diagnosis and segmentation. In recent years, Generative Adversarial Networks (GANs), due to their powerful image generation capabilities, have been increasingly applied to medical image data augmentation. GANs directly generate realistic medical images by inputting random noise, aiming to expand medical image datasets. However, training such models directly to generate medical images requires a large amount of data. Due to the sensitivity and privacy concerns of medical endoscopic image data, the dataset size is often small, making these methods unsuitable.

[0004] Therefore, proposing a grid deformation data augmentation method that requires a small amount of data sample and can automatically generate deformable grids to deform and enhance medical endoscope images in order to expand the dataset, so that the enhanced medical endoscope images are realistic and have a certain degree of diversity, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a mesh deformation data augmentation method based on the WGAN-GP model. This method, tailored to the characteristics of medical endoscopic images, automatically generates deformable meshes by constructing a WGAN-GP model, and then uses these deformable meshes to augment the medical endoscopic images, thereby expanding the dataset. The augmented medical endoscopic images are realistic and possess a certain degree of diversity. To achieve the above objectives, the present invention adopts the following technical solution:

[0006] This invention directly generates deformable meshes using the WGAN-GP model, and then performs a warp operation on the deformable meshes and medical endoscope images to obtain data-enhanced medical endoscope images. Instead of directly generating images through the model, this reduces the data sample requirements during model training. Furthermore, the loss function for model training incorporates a smooth deformation term and a large deformation term (LA) for the lesion region. The smooth deformation term constrains the model to produce smooth deformations, ensuring the realism and reasonableness of the deformations, while the large deformation term for the lesion region encourages larger deformations in the lesion region, ensuring the significance and diversity of the deformations in the lesion region.

[0007] A method for augmenting mesh deformation data based on the WGAN-GP model includes the following steps:

[0008] S1: Construct the training and testing datasets, dividing the normalized medical endoscopic image data into the training set X and the testing set X in a 7:3 ratio. train and test set X test The training set images and their corresponding lesion regions (LA) are packaged into a training dataset.

[0009] S2: Construct the WGAN-GP model, which includes a generator network Net based on residual structure. G and the discriminator network Net based on residual structure D ;

[0010] S3: Iteratively train the WGAN-GP model based on the training dataset, and configure the discriminator network Net in the WGAN-GP model. D and generator network Net G The loss function is used to iteratively optimize the discriminator network Net by alternating between the loss functions. D and generator network Net G Continue until the model converges, then save the model parameters;

[0011] S4: Test Set X Using Medical Endoscopic Images test Evaluate the generator network Net after training. G Evaluate the performance of the generator network Net that meets the requirements. G This refers to the grid deformation data augmentation model.aug ;

[0012] S5: Sampled random noise {z1, z2, ..., z} that follows a standard normal distribution. k}, respectively, random noise {z1, z2, ..., z k} and medical endoscopic image X as a grid deformation data augmentation model aug The model is input and the corresponding deformed mesh {g1, g2, ..., g} is output. k}, the medical endoscope image X is compared with the deformable mesh {g1, g2, ..., g}, respectively. k After performing the warp operation, the data-augmented medical endoscope image {X} is obtained. aug_1 X aug_2 , ..., X aug_k}

[0013] Optionally, the generator network Net in step S2 G The convolutional neural network based on residual structures includes a first residual structure block, a second residual structure block, an upsampling block, convolutional layers, batch normalization layers, max pooling layers, linear layers, reshape layers, and Tanh activation functions. The first and second residual structure blocks are composed of convolutional layers, batch normalization layers, and LeakyReLU activation functions connected in series, and the lower-level outputs are connected to the higher-level outputs through residual connections. The upsampling block is composed of convolutional layers, batch normalization layers, LeakyReLU activation functions, and upsampling layers connected in series.

[0014] Optional, generator network Net G The input is a training set of medical endoscope images X. train And random noise z, generator network Net G The output is the deformed mesh g.

[0015] Optionally, the discriminator network Net in step S2 D This is a convolutional neural network based on residual structures, which includes a third residual structure block, a fourth residual structure block, convolutional layers, layer normalization layers, max pooling layers, adaptive average pooling layers, reshape layers, and linear layers. The third and fourth residual structure blocks are composed of convolutional layers, layer normalization layers, and ReLU activation functions alternately connected in series, and the low-level outputs are connected to the high-level outputs through residual connections.

[0016] Optional, discriminator network Net D The input is a training set of medical endoscope images X. train and deformed image X train_aug Discriminator Network (Net) D The output is the corresponding value Net.D (X train ) and Net D (X train_aug ).

[0017] Optionally, step S3 specifically includes the following steps:

[0018] S3-1: Initialize the generator network Net G and discriminator network Net D ;

[0019] S3-2: Training the medical endoscope image set X train and random noise z as Net G The input is the deformed mesh g;

[0020] S3-3: Training the medical endoscope image set X train The warp operation is performed on the deformed mesh g to obtain the deformed image X. train_aug The training set of medical endoscope images X train and deformed image X train_aug Input discriminator network Net D Output the corresponding value Net D (X train ) and Net D (X train_aug );

[0021] S3-4: Constructing the discriminator network Net D The loss function is L D =L m +λ1×L Grad_penalty L D1 =-Net D (X train )+Net D (X train_aug ), L Grad_penalty λ1 is the gradient penalty term, and λ1 is the weight parameter;

[0022] S3-5: Constructing the Generator Network Net G The loss function is L G =L G1 +λ2×L smooth +λ3×L LA_deform L G1 =-Net D (X train_aug ), L smooth For smooth deformation term, L LA_deform λ2 and λ3 are the large deformation terms of the lesion region LA, and λ3 are the weighting parameters.

[0023] S3-6: Constructing monitoring indicators L during training monitor =Net D (X train )-Net D (X train_aug This is used to observe the training trend of the WGAN-GP model;

[0024] S3-7: Using the loss function L respectively D and L G Alternating Iterative Optimization Discriminator Network Net D and generator network Net G Observation and monitoring indicator item L monitor The convergence status of the monitoring indicator L monitor Stop model training when the curve converges, and save the trained discriminator network Net. D and generator network Net G Model parameters.

[0025] Optionally, the smooth deformation term L in steps S3-5 smooth For the constraint generator network Net G The loss function term that produces smooth deformation. Where N is the number of pixels within △g, and p represents any point within △g. x and p y Let p be the position of p in the X and Y directions, respectively, and Δg = g - g0, where g0 is the deformed mesh of the original image - the original image.

[0026] Optionally, in steps S3-5, the lesion region LA large deformation term L LA_deform To encourage the lesion region LA to produce a larger deformation loss function term, Where N is the number of pixels in △g, R is any point in △g that belongs to the lesion region LA, R′ is any point in △g that does not belong to the lesion region LA, a is a constant (a>1), △g=g-g0, and g0 is the deformed grid of the original image-original image.

[0027] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a mesh deformation data augmentation method based on the WGAN-GP model, which has the following advantages:

[0028] 1. This invention addresses the characteristic that patients' breathing or movement during the actual acquisition of medical endoscopic images can cause local displacement or deformation of organs and tissues. The method automatically generates a deformation mesh by constructing a WGAN-GP model, and then enhances the medical endoscopic image through deformation mesh. The enhanced medical endoscopic image is realistic and has a certain degree of diversity.

[0029] 2. The method of the present invention directly generates deformable meshes through the WGAN-GP model, and then enhances the medical endoscope image through deformation of the deformable meshes, instead of generating the image directly through the model, which reduces the data sample requirements of the model training process.

[0030] 3. The method of the present invention designs a smooth deformation term and a lesion region LA large deformation term in the loss function of model training. The smooth deformation term constrains the model to produce smooth deformation, ensuring the authenticity and rationality of the deformation. The lesion region LA large deformation term encourages the lesion region to produce large deformation, ensuring the significance and diversity of the deformation of the lesion region.

[0031] 4. The method of this invention can not only perform data augmentation on medical endoscopic images, but also perform synchronous data augmentation on the corresponding lesion area labels, effectively expanding the dataset of medical endoscopic images and their lesion area annotations, thereby solving the problems of scarce and imbalanced data samples; using the expanded dataset to train artificial intelligence algorithm models (especially deep neural network models) for medical image tasks such as lesion diagnosis and lesion segmentation can effectively improve the accuracy and generalization of the final model. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of a mesh deformation data augmentation method based on the WGAN-GP model;

[0034] Figure 2 The generator network Net in this invention G Model structure diagram;

[0035] Figure 3 This is a structural diagram of the first residual structural block;

[0036] Figure 4 This is a structural diagram of the second residual structural block;

[0037] Figure 5 This is a structural diagram of the upsampling block;

[0038] Figure 6 X is the training set of medical endoscope images in this invention. train A schematic diagram of the warp operation performed on the deformed mesh g;

[0039] Figure 7 The discriminator network Net in this invention D Model structure diagram;

[0040] Figure 8 This is a structural diagram of the third residual structural block;

[0041] Figure 9 This is a structural diagram of the fourth residual structural block;

[0042] Figure 10 This is an example of the application of a grid deformation data augmentation method based on the WGAN-GP model on a gastrointestinal endoscopy dataset. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figure 1 The figure shows a mesh deformation data augmentation method based on the WGAN-GP model provided by the present invention. The method specifically includes the following steps:

[0045] S1: Construct the training and testing datasets, including dividing the normalized medical endoscopic image data into the training set X and the testing set X in a 7:3 ratio. train and test set X test The training set images and their corresponding lesion regions (LA) are packaged into a training dataset.

[0046] Medical endoscopic image data is stored in RGB format. The values ​​of each color channel are divided by 255 to normalize them to 0-1. The normalized medical endoscopic image data is then divided into a training set X in a 7:3 ratio. train and test set X test Medical endoscopic image training set X train X sets of medical endoscope images were used to train the WGAN-GP model. test Used to evaluate the WGAN-GP model after training.

[0047] The constructed training dataset also includes a training set X of medical endoscope images. train The corresponding lesion area LA is labeled. The lesion area LA is stored in a binary map. Pixels within the lesion area are represented by 1, and pixels outside the lesion area are represented by 0.

[0048] S2: Construct the WGAN-GP model, which includes a generator network Net based on residual structure. G and the discriminator network Net based on residual structure D .

[0049] Generator Networks (Net) G The model structure is as follows Figure 2 As shown, this network is a convolutional neural network based on residual structures, consisting of a first residual structure block (such as...). Figure 3 As shown), the second residual structure block (as shown) Figure 4 As shown), upsampling block (such as) Figure 5 The input feature map consists of convolutional layers, batch normalization layers, max pooling layers, linear layers, reshape layers, and Tanh activation functions. The first and second residual structure blocks are composed of concatenated convolutional layers, batch normalization layers, and LeakyReLU activation functions, connected via residual connections between lower and higher level outputs. The input feature map passes through the first residual structure block, while the output feature map size remains unchanged. However, after passing through the second residual structure block, the output feature map size is reduced to [a smaller value than the original value]. The upsampling block consists of convolutional layers, batch normalization layers, LeakyReLU activation functions, and upsampling layers connected in alternating sequences.

[0050] Generator Networks (Net) G The input is a training set of medical endoscope images X. train Given random noise z (z follows a standard normal distribution, i.e., a normal distribution N(0,1) with a mean of 0 and a standard deviation of 1), the generator network Net... G The output is a deformable mesh g; the medical endoscope image training set X train (Taking size = 3×512×512 as an example) After passing through the corresponding convolutional layer, batch normalization layer, max pooling layer, first residual structure block and second residual structure block, the output feature map F is generated. X The size is 128×64×64; random noise z (taking dimension = 100 as an example) passes through the corresponding linear layer, reshape layer and first residual structure block to output feature map F. z The size is 128×64×64; the feature map F X and F z The feature map F is stitched together along the channel dimension. X+z The size is 256×64×64; F X+z After passing through 3 upsampling blocks, convolutional layers, and the Tanh activation function, and adjusting the dimensional order, a deformable mesh g with size = 512 × 512 × 2 is output.

[0051] Figure 6X, a training set of medical endoscopic images train The diagram illustrates the warp operation on the deformable mesh g, which is an array of size (H, W, 2), where H and W are the training set of medical endoscope images X, respectively. train The dimensions in the length and width directions; taking H=W=512 as an example, the deformable mesh g is an array of size=(512, 512, 2).

[0052] The value of the deformed mesh g at (h, w) (g) 1(h，w) g 2(h,w) Indicator image X train_aug The pixel value at (h, w) is derived from the medical endoscope image training set X. train In (g 1(h，w) g 2(h,w) The value at position ) determines the pixel value, i.e.

[0053] The deformable mesh g has a numerical range of [-1, 1], corresponding to the medical endoscope image training set X. train The top left, bottom left, top right, and bottom right positions are represented by (-1, -1), (-1, 1), (1, -1), and (1, 1), respectively.

[0054] The coordinates indicated by the deformed mesh g (g 1(h，w) g 2(h,w) The numerical range of ) is between [-1, 1], and it needs to be based on a floating-point coordinate value in the medical endoscope image training set X. train The pixel values ​​are sampled, and this method uses bilinear interpolation for filling, i.e., the medical endoscope image training set X. train In (g 1(h，w) g 2(h,w) The pixel value corresponding to ) By (g) 1(h，w) g 2(h,w) The values ​​of the four surrounding pixels were calculated using bilinear interpolation.

[0055] Discriminator Network (Net) D The model structure is as follows Figure 7 As shown, this network is a convolutional neural network based on residual structures, consisting of a third residual structure block (such as...). Figure 8 As shown), the fourth residual structure block (as shown) Figure 9The input feature map consists of a convolutional layer, a layer normalization layer, a max pooling layer, an adaptive average pooling layer, a reshape layer, and a linear layer. The third and fourth residual structure blocks are composed of convolutional layers, layer normalization layers, and ReLU activation functions, which are alternately connected in series. The lower-level outputs are connected to the higher-level outputs through residual connections. The size of the output feature map remains unchanged after the input feature map passes through the third residual structure block, while the size of the output feature map is reduced to half of the original size after the input feature map passes through the fourth residual structure block.

[0056] Discriminator Network (Net) D The input is a training set of medical endoscope images X. train and deformed image X train_aug The output of the discriminator network NetD is the corresponding value Net. D (X train ) and Net D (X train_aug Medical endoscopic image training set X train and deformed image X train_aug (Taking size = 3×512×512 as an example) Input discriminator network Net D After passing through convolutional layers, layer normalization layers, max pooling layers, a third residual block, and two fourth residual blocks, the feature map size gradually decreases while the number of channels gradually increases, resulting in a feature map size of 256×32×32. Following an adaptive average pooling layer, the feature map size becomes 256×1×1. After dimensionality reduction via a reshape layer and two linear layers, the final output is the corresponding Net value. D (X train ) and Net D (X train_aug ).

[0057] S3: Iteratively train the WGAN-GP model based on the training dataset, and configure the discriminator network Net in the WGAN-GP model. D and generator network Net G The loss function is used to iteratively optimize the discriminator network Net by alternating between the loss functions. D and generator network Net G Continue until the model converges, then save the model parameters.

[0058] S3-1: Initialize the generator network Net using the kaiming_normal parameter initialization method. G and discriminator network Net D The parameters.

[0059] S3-2: Training the medical endoscope image set X train And random noise z as the generator network NetG The input is the deformed mesh g.

[0060] S3-3: Training the medical endoscope image set X train The warp operation is performed on the deformed mesh g to obtain the deformed image X. train_aug The training set of medical endoscope images X train and deformed image X train_aug Input discriminator network Net D Output the corresponding value Net D (X train ) and Net D (X train_aug ).

[0061] S3-4: Constructing the discriminator network Net D The loss function is L D =L D1 +λ1×L Grad_penalty L D1 =-Net D (X train )+Net D (X train_aug ), L Grad_penalty λ1 is the gradient penalty term, and λ1 is the weight parameter.

[0062] The iterative optimization process of the WGAN-GP model is a minimax game problem, namely, finding the generator network Net. G and discriminator network Net D The balance point between them; generator network Net G The goal is to make the output deformable mesh g match the training set X of medical endoscope images. train The deformed image X obtained after performing the warp operation train_aug The distribution is as close as possible to the medical endoscope image training set X. train The distribution of the discriminator network Net; D The goal is to distinguish the medical endoscope images in the training set X as much as possible. train and deformed image X train_aug The generator network Net is trained iteratively and continuously. G and discriminator network Net D Eventually, an equilibrium point is reached.

[0063] During the training of the WGAN-GP model, W=Net is used. D (X train )-Net D (X train_aug To approximate the training set X of medical endoscope images train Distribution and Deformation Image Xtrain_aug The smaller the Wasserstein distance between the distributions, i.e., W, the more deformed the image X. train_aug The closer the distribution is to the medical endoscope image training set X train The distribution of .

[0064] Discriminator Network (Net) D The goal is to distinguish the medical endoscope images in the training set X as much as possible. train and deformed image X train_aug Even if W = Net D (X train )-Net D (X train_aug If the discriminator network Net is as large as possible, then the discriminator network Net will be larger. D The first part of the loss function can be expressed as L D1 =-Net D (X train )+Net D (X train_aug ).

[0065] Using W=Net D (X train )-Net D (X train_aug To approximate the training set X of medical endoscope images train and deformed image X train_aug The Wasserstein distance between distributions needs to satisfy the Lipschitz continuity condition, therefore in the discriminator network Net D Introducing a gradient penalty term into the loss function in Random number β∈[0,1).

[0066] Therefore, the loss function of the discriminator network NetD is defined as: L D =L D1 +λ1×L Grad_penalty , where λ1 is the weight parameter.

[0067] S3-5: Constructing the Generator Network Net G The loss function is L G =L G1 +λ2×L smooth +λ3×L LA_deform L G1 =-Net D (X train_aug ), L smooth For smooth deformation term, L LA_deform λ2 and λ3 are the large deformation terms of the lesion region LA, and the weighting parameters are λ2 and λ3.

[0068] Generator Network (Net) G The goal is to deform the image X train_aug The distribution is as close as possible to the medical endoscope image training set X. train The distribution, even if we get W = Net D (X train )-Net D (X train_aug ) should be as small as possible, because Net D (X train This item is related to the generator network Net. G Irrelevant, therefore the generator network Net G The first part of the loss function can be expressed as L G1 =-Net D (X train_aug ).

[0069] In order to make the generator network Net G The deformation of the output deformable mesh g is more realistic and smoother in the generator network Net. G A smoothing deformation term is introduced into the loss function to constrain the generator network NetG to produce smooth deformation. Where N is the number of pixels within △g, and p represents any point within △g. x and p y Let p be the position of p in the X and Y directions, respectively, and Δg = g - g0, where g0 is the deformed mesh of the original image - the original image.

[0070] In order to make the generator network Net G The deformation corresponding to the output deformable mesh g results in a large deformation at the lesion region LA, in the generator network Net. G The loss function incorporates a large deformation term for the lesion region (LA). In the constructed loss function for the large deformation term of the lesion region (LA), different weights are assigned to the deformation of the lesion region (LA) labeled area and the non-lesion region (LA) labeled area, i.e. Where N is the number of pixels in △g, R is any point in △g that belongs to the lesion region LA, R′ is any point in △g that does not belong to the lesion region LA, a is a constant (a>1), △g=g-g0, and g0 is the deformed grid of the original image-original image.

[0071] Therefore, the generator network Net G The loss function is defined as: L G =L G1 +λ2×L smooth +λ3×L LA_deform, where λ2 and λ3 are weight parameters.

[0072] S3-6: Constructing monitoring indicators L during training monitor =Net D (X train )-Net D (X train_aug (This is used to observe the training trend of the WGAN-GP model.)

[0073] S3-7: Using the loss function L respectively D and L G Alternating Iterative Optimization Discriminator Network Net D and generator network Net G Observation and monitoring indicator item L monitor The convergence status of the monitoring indicator L monitor Stop model training when the curve converges, and save the trained discriminator network Net. D and generator network Net G Model parameters.

[0074] In each training cycle, through the loss function L D Iterative optimization of the discriminator network Net D Through the loss function L G Iterative optimization generator network Net G The relevant training parameters are shown in the table below:

[0075]

[0076] S4: Test Set X Using Medical Endoscopic Images test Evaluate the generator network Net after training. G Evaluate the performance of the generator network Net that meets the requirements. G This refers to the grid deformation data augmentation model. aug .

[0077] S5: Sampled random noise {z1, z2, ..., z} that follows a standard normal distribution. k}, respectively, random noise {z1, z2, ..., z k (and medical endoscopic images X as a grid deformation data augmentation model) aug The model is input and the corresponding deformed mesh {g1, g2, ..., g} is output. k The medical endoscope image X is compared with the deformable mesh {g1, g2, ..., g...}. k After performing the warp operation, the data-augmented medical endoscope image {X} is obtained. aug_1 X aug_2 , ..., X aug_k}

[0078] like Figure 10 The image shows an application example of the grid deformation data augmentation method based on the WGAN-GP model provided by this invention on a gastrointestinal endoscopic polyp segmentation dataset. It can be seen that this method can not only perform data augmentation on endoscopic images, but also perform synchronous data augmentation on the corresponding lesion area labels.

[0079] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0080] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for augmenting grid deformation data based on the WGAN-GP model, characterized in that, Includes the following steps: S1: Construct the training and testing datasets. Divide the normalized medical endoscopic image data into the training set and the testing set in a 7:3 ratio. and test set The training set images and their corresponding lesion regions (LA) are packaged into a training dataset. S2: Construct the WGAN-GP model, which includes a generator network based on residual structure. and discriminator networks based on residual structures ; S3: Iteratively train the WGAN-GP model based on the training dataset, and configure the discriminator network in the WGAN-GP model. and generator networks The loss function is used to iteratively optimize the discriminator network by alternating between the loss functions. and generator networks Continue until the model converges, then save the model parameters; Step S3 specifically includes the following steps: S3-1: Initialize the generator network and discriminator network ; S3-2: Training set of medical endoscope images and random noise As Input, output deformed mesh ; S3-3: Training set of medical endoscope images and deformable mesh Perform a warp operation to obtain the deformed image. Training set of medical endoscope images and deformed images Input discriminator network Output the corresponding value and ; S3-4: Constructing the discriminator network The loss function is ,in , This is a gradient penalty term. These are weight parameters; S3-5: Constructing a Generator Network The loss function is ,in , For smooth deformation term, For the LA large deformation term in the lesion area, and These are weight parameters; S3-6: Constructing monitoring indicators during training This is used to observe the training trend of the WGAN-GP model; S3-7: Using the loss function respectively and Alternating Iterative Optimization of Discriminator Network and generator networks Observation and monitoring indicators The convergence status of the monitoring indicators Stop model training when the curve converges, and save the trained discriminator network. and generator networks Model parameters; S4: Test Set Using Medical Endoscopic Images Evaluate the generator network after training. Evaluate generator networks whose performance meets requirements. This is a grid deformation data augmentation model. ; S5: Random noise sampled according to a standard normal distribution Random noise With medical endoscope images As a data augmentation model for grid deformation The model input, Corresponding deformation mesh ,Will Separate and deformable mesh After performing the warp operation, the data-augmented medical endoscope image is obtained. .

2. The method for enhancing mesh deformation data based on the WGAN-GP model according to claim 1, characterized in that, The generator network described in step S2 The convolutional neural network based on residual structures includes a first residual structure block, a second residual structure block, an upsampling block, convolutional layers, batch normalization layers, max pooling layers, linear layers, reshape layers, and Tanh activation functions. The first and second residual structure blocks are composed of convolutional layers, batch normalization layers, and LeakyReLU activation functions connected in series, and the lower-level outputs are connected to the higher-level outputs through residual connections. The upsampling block is composed of convolutional layers, batch normalization layers, LeakyReLU activation functions, and upsampling layers connected in series.

3. The method for enhancing mesh deformation data based on the WGAN-GP model according to claim 2, characterized in that, The generator network The input is a training set of medical endoscope images. and random noise Generator Network The output is a deformable mesh. .

4. The method for augmenting mesh deformation data based on the WGAN-GP model according to claim 1, characterized in that, The discriminator network described in step S2 This is a convolutional neural network based on residual structures, which includes a third residual structure block, a fourth residual structure block, convolutional layers, layer normalization layers, max pooling layers, adaptive average pooling layers, reshape layers, and linear layers. The third and fourth residual structure blocks are composed of convolutional layers, layer normalization layers, and ReLU activation functions alternately connected in series, and the low-level outputs are connected to the high-level outputs through residual connections.

5. A method for augmenting mesh deformation data based on the WGAN-GP model according to claim 4, characterized in that, The discriminator network The input is a training set of medical endoscope images. and deformed images Discriminator Network The output is the corresponding value. and .

6. The method for augmenting mesh deformation data based on the WGAN-GP model according to claim 1, characterized in that, The smooth deformation term described in step S3-5 For constraint generator networks The loss function term that produces smooth deformation. ,in for Number of inner pixels express Any point inside, and They are respectively Positions in the X and Y directions , This is the original image - the deformed mesh of the original image.

7. The method for augmenting mesh deformation data based on the WGAN-GP model according to claim 1, characterized in that, The LA large deformation term in the lesion area described in steps S3-5 To encourage the lesion region LA to produce a larger deformation loss function term, ,in for Number of inner pixels for Any point within the LA region of the lesion. for Any point within that area that is not part of the LA region of the lesion. It is a constant. , , This is the original image - the deformed mesh of the original image.

Images