Method for medical image enhancement based on stream matching

Abstract

The application provides a medical image enhancement method based on flow matching. Applied to the technical field of data processing, the method comprises the following steps: constructing a medical image data set, preprocessing original images in the medical image data set to obtain a training sample set containing images with unified specifications; training a pre-constructed variational autoencoder based on the training sample set to obtain a trained ViT-VAE encoder and a ViT-VAE decoder; constructing a label system containing coarse labels and fine labels, constructing a contrastive language-image pre-training model, and introducing a sparse supervision strategy for training to obtain a trained semantic encoder; extracting latent features based on the ViT-VAE encoder, extracting semantic features based on the semantic encoder, constructing a flow matching network, generating an enhanced image sequence based on the flow matching network, and performing quality screening on the enhanced image sequence. In this way, the enhancement effect of medical images is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a medical image enhancement method based on stream matching. Background Technology

[0002] Medical image enhancement is a core preprocessing step in medical imaging diagnosis and treatment. Its core significance lies in resolving issues such as noise, artifacts, insufficient contrast, and blurred details in original medical images caused by equipment and scanning environment. It clearly restores lesions and tissue anatomical structures, providing clinicians with more accurate observational data and significantly reducing the probability of missed or misdiagnosed cases. Simultaneously, high-quality enhanced images can improve the lesion identification and segmentation accuracy of medical AI algorithms, providing reliable data support for radiomics analysis, surgical planning, and radiotherapy positioning. Furthermore, it can bridge the performance gap in imaging equipment at primary healthcare institutions, facilitate precise remote medical consultations, and provide standardized, high-quality image materials for medical research and teaching, thus comprehensively promoting the implementation of precision medicine and improving diagnostic and treatment efficiency and quality.

[0003] Currently, existing methods suffer from insufficient feature extraction and a lack of semantic supervision mechanisms, which can easily lead to semantic bias in the enhanced images, resulting in poor enhancement effects for medical images. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a medical image enhancement method based on flow matching. The method includes:

[0005] S1, construct a medical image dataset, preprocess the original images in the medical image dataset to obtain a training sample set containing images of uniform specifications;

[0006] S2, the pre-constructed variational autoencoder is trained based on the training sample set to obtain the trained ViT-VAE encoder and ViT-VAE decoder;

[0007] S3. Construct a labeling system that includes coarse and fine labels, build a contrastive language-image pre-trained model, and introduce a sparse supervision strategy for training to obtain a trained semantic encoder.

[0008] S4, extract latent features based on the ViT-VAE encoder, extract semantic features based on the semantic encoder, construct a stream matching network, and generate an enhanced image sequence based on the stream matching network;

[0009] S5, perform quality screening on the enhanced image sequence, specifically: obtain the resource status parameters of the current computing environment, and determine whether the resource status parameters meet the full verification condition; if the resource status parameters meet the full verification condition, extract the generated images corresponding to all time steps in the enhanced image sequence as the verification objects; if the resource status parameters do not meet the full verification condition, extract only the generated images corresponding to the start time step and the end time step in the enhanced image sequence as the verification objects; calculate the semantic similarity between the verification object and the target label; if the semantic similarity is less than a preset semantic consistency threshold, determine that the verification object has semantic deviation and remove it; if the semantic similarity is greater than or equal to the semantic consistency threshold, determine that the verification object is semantically qualified and retain it.

[0010] Further, the preprocessing of the original images in the medical image dataset includes:

[0011] Obtain the size parameters of the original image and determine the model scale type corresponding to the current training task;

[0012] If the model scale type is the first scale, then the resolution of the original image is adjusted to the first resolution, which is used for fast training of the small-scale model.

[0013] If the model scale type is the second scale, then the resolution of the original image is adjusted to the second resolution, which is used for the fine-tuning training of the large-scale model;

[0014] The image after resolution adjustment is subjected to equalization processing to enhance local contrast;

[0015] Apply medium filtering to the equalized image to suppress noise while preserving edge features.

[0016] Furthermore, the training of the pre-constructed variational autoencoder includes:

[0017] Construct a loss function, which is composed of a weighted sum of reconstruction loss, regularization term, adversarial loss and perception loss;

[0018] During training, the loss value of the validation set is monitored in real time;

[0019] If the loss value of the validation set is lower than the convergence threshold for a consecutive preset number of stable rounds, it is determined that the latent spatial distribution of the variational autoencoder has reached a stable state, and training is stopped.

[0020] If the loss value of the validation set does not meet the above conditions, training continues until the maximum number of training rounds is reached;

[0021] The convergence threshold is used to ensure the compactness of potential features, and the preset number of stable rounds is used to prevent accidental fluctuations during model training.

[0022] Furthermore, the construction of the tag system, which includes coarse tags and fine tags, includes:

[0023] Analyze the diagnostic reports in the medical image dataset to extract key semantic information;

[0024] If the key semantic information involves the classification of a disease, it is mapped to a coarse label, which is used to define the major category of the lesion;

[0025] If the key semantic information involves the texture, color, or shape of the lesion, it is mapped to a fine label; the fine label is used to provide fine-grained semantic guidance for the evolution trajectory of potential features in subsequent image generation steps.

[0026] Furthermore, the introduction of a sparse supervision strategy for training includes:

[0027] Get any two samples from the training batch and extract their corresponding coarse labels;

[0028] Determine whether the coarse labels of two samples are the same;

[0029] If the coarse labels are the same, the corresponding element value in the supervision matrix is ​​set as the matching identifier, allowing many-to-many alignment in the semantic space;

[0030] If the coarse labels are different, the corresponding element value in the supervision matrix is ​​set as a non-matching identifier;

[0031] The loss value is calculated based on the supervision matrix, and the parameters of the contrastive language-image pre-trained model are updated according to the loss value.

[0032] Furthermore, the construction of the stream matching network includes:

[0033] Obtain the first feature dimension of the latent feature and the second feature dimension of the semantic feature;

[0034] Determine whether the first feature dimension and the second feature dimension are consistent;

[0035] If they are inconsistent, a dimension mapping module containing convolutional layers and fully connected layers is constructed to map the latent features and the semantic features to a unified third feature dimension.

[0036] The mapped features are fused and used as input conditions for the velocity field network in the flow matching network.

[0037] Further, generating the enhanced image sequence based on the stream matching network includes:

[0038] A source image is randomly selected from the medical image dataset, and its latent features are extracted using the ViT-VAE encoder as the initial state.

[0039] Based on the fine labels, target images are matched from the medical image dataset, and their latent features are extracted as target states;

[0040] The velocity field equations are constructed and integrated using the second-order Heun integral method.

[0041] During the integration process, the time interval is divided into several intermediate time steps. For each intermediate time step, the velocity field is calculated based on the current state and input conditions, the latent features are updated, and the latent features of the current time step are obtained.

[0042] Input the latent features of the current time step into the trained ViT-VAE decoder;

[0043] The decoder reconstructs the latent features into RGB image tensors based on a hierarchical upsampling mechanism and an attention mechanism.

[0044] The reconstructed RGB image tensor is used as a frame in the enhanced image sequence, thereby forming a continuous transformation sequence from the source disease state to the target disease state.

[0045] Furthermore, training the stream matching network includes:

[0046] The mean square error between the predicted output of the velocity field network and the actual velocity field;

[0047] Generative adversarial networks are introduced as discriminators to calculate the adversarial loss of the generated trajectories;

[0048] If the adversarial loss is greater than the preset trajectory quality threshold, the weight of the adversarial loss in the total loss is increased to force the trajectory generated by the velocity field network to be closer to the real data distribution.

[0049] If the adversarial loss is less than or equal to the trajectory quality threshold, then the current weight configuration is maintained.

[0050] Furthermore, S5 also includes:

[0051] Images deemed semantically acceptable are submitted to a manual review process.

[0052] If a manual review determines that an image has structural distortion or artifacts, the image will be marked as an invalid sample.

[0053] If a manual review determines that an image has typical features and is not distorted, then the image is marked as a high-quality sample.

[0054] The high-quality samples are injected into the medical image dataset, and the next round of model iteration training is performed based on the updated dataset.

[0055] This invention obtains a standardized training sample set by preprocessing original medical images, trains the ViT-VAE encoder-decoder, and achieves accurate extraction of latent features from medical images, improving the effectiveness of feature representation. It constructs a coarse-to-fine labeling system, introduces a sparse supervision strategy to train a semantic encoder, and fuses latent and semantic features to build a flow matching network to generate enhanced image sequences, achieving synergistic enhancement of visual and semantic features and improving the semantic consistency and visual expressiveness of the enhanced images. Furthermore, it dynamically adjusts the range of objects to be verified for quality screening based on computational resource status parameters, and combines semantic similarity to remove semantically biased images, achieving efficient and accurate screening of enhanced images and improving the overall quality and screening efficiency of the final output enhanced images.

[0056] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0057] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of the invention. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0058] Figure 1 A flowchart of a flow-matching-based medical image enhancement method according to an embodiment of the present invention is shown. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0061] Figure 1 A flowchart of a flow-matching-based medical image enhancement method according to an embodiment of the present invention is shown, the method comprising:

[0062] S1, construct a medical image dataset, preprocess the original images in the medical image dataset to obtain a training sample set containing images of uniform specifications;

[0063] S2, the pre-constructed variational autoencoder is trained based on the training sample set to obtain the trained ViT-VAE encoder and ViT-VAE decoder;

[0064] S3. Construct a labeling system that includes coarse and fine labels, build a contrastive language-image pre-trained model, and introduce a sparse supervision strategy for training to obtain a trained semantic encoder.

[0065] S4, extract latent features based on the ViT-VAE encoder, extract semantic features based on the semantic encoder, construct a stream matching network, and generate an enhanced image sequence based on the stream matching network;

[0066] S5, perform quality screening on the enhanced image sequence, specifically: obtain the resource status parameters of the current computing environment, and determine whether the resource status parameters meet the full verification condition; if the resource status parameters meet the full verification condition, extract the generated images corresponding to all time steps in the enhanced image sequence as the verification objects; if the resource status parameters do not meet the full verification condition, extract only the generated images corresponding to the start time step and the end time step in the enhanced image sequence as the verification objects; calculate the semantic similarity between the verification object and the target label; if the semantic similarity is less than a preset semantic consistency threshold, determine that the verification object has semantic deviation and remove it; if the semantic similarity is greater than or equal to the semantic consistency threshold, determine that the verification object is semantically qualified and retain it.

[0067] In some embodiments, the preprocessing of the original images in the medical image dataset includes: obtaining the size parameters of the original images and determining the model scale type corresponding to the current training task; if the model scale type is a first scale, adjusting the resolution of the original images to a first resolution, which is used for fast training of small-scale models; if the model scale type is a second scale, adjusting the resolution of the original images to a second resolution, which is used for refined training of large-scale models; performing equalization processing on the image after resolution adjustment to enhance local contrast; and applying medium filtering to the image after equalization processing to suppress noise while preserving edge features. According to embodiments of the present invention, by adjusting the resolution of the original images by scale, the training requirements of models at different scales are adapted, improving the targeting and efficiency of model training; by performing equalization processing on the image after resolution adjustment, local contrast of the image is enhanced, and the recognizability of image details is improved; by applying medium filtering to the image after equalization processing, image noise is suppressed while preserving edge features, improving the quality and usability of the preprocessed image.

[0068] For example, the size parameters of the original image of the upper gastrointestinal endoscopy are obtained, with an initial resolution of 1080×1920 pixels and a pixel depth of 8 bits. The current training task is determined to be rapid verification of model feasibility, corresponding to the first scale type of the model. The original image resolution is adjusted to 256×256 pixels (first resolution), while maintaining the pixel depth at 8 bits. If subsequent refined training is performed (model scale type is second scale), the resolution is adjusted to 512×512 pixels (second resolution). The image after resolution adjustment is then equalized. A 3×3 kernel-size median filter is applied to the equalized image, with one pass of filtering. This effectively suppresses electronic noise and mucosal reflection noise in the endoscopy image while preserving edge features such as mucosal texture and blood vessels.

[0069] In some embodiments, training the pre-constructed variational autoencoder includes: constructing a loss function, which is composed of a weighted sum of reconstruction loss, regularization term, adversarial loss, and perceptual loss; during training, monitoring the loss value of the validation set in real time; if the loss value of the validation set is lower than the convergence threshold for a consecutive preset number of stable epochs, it is determined that the latent space distribution of the variational autoencoder has reached a stable state, and training is stopped; if the loss value of the validation set does not meet the above conditions, training continues until the maximum number of training epochs is reached; the convergence threshold is used to ensure the compactness of latent features, and the preset number of stable epochs is used to prevent accidental fluctuations during model training. According to embodiments of the present invention, by constructing a multi-loss weighted loss function, the training process of the variational autoencoder is fully constrained, improving the accuracy of model training; by monitoring the loss value of the validation set in real time, the model training status is grasped in a timely manner, improving the controllability of the training process; by setting a convergence threshold and a preset number of stable epochs, accidental fluctuations and overfitting / underfitting in model training are avoided, improving the stability of the latent space distribution of the trained model and the compactness of latent features.

[0070] For example, the constructed loss function expression is: Loss = 0.5 × Loss recon + 0.2 × Loss reg +0.2 × Loss adv + 0.1 × Loss perc Loss recon For MSE reconstruction loss, Loss reg For KL divergence regularization term, Loss adv For GAN adversarial loss, loss perc The VGG16 perceptual loss was used. During training, 5000 preprocessed 256×256 pixel gastroscopy images (5% of the training sample set) were selected as the validation set, and their loss values ​​were monitored in real time. The preset convergence threshold was 0.008, the preset stable number of rounds was 20, and the maximum number of training rounds was 500. When training reached round 320, the validation set loss value was 0.0072, and the loss values ​​of the subsequent 20 rounds (rounds 320-339) were between 0.0070 and 0.0078, all below the convergence threshold of 0.008, indicating that the latent space distribution was stable and training was stopped. If the validation set loss value was still 0.0091 (above the convergence threshold) when training reached round 490, training continued until round 500 and then stopped.

[0071] In some embodiments, constructing a labeling system comprising coarse and fine labels includes: parsing diagnostic reports in the medical image dataset to extract key semantic information; if the key semantic information relates to disease category attribution, it is mapped to a coarse label, which is used to define the major category of the lesion; if the key semantic information relates to the texture, color, or shape of the lesion, it is mapped to a fine label; the fine label is used to provide fine-grained semantic guidance for the evolution trajectory of potential features in subsequent image generation steps. According to embodiments of the present invention, by parsing diagnostic reports to extract key semantic information, accurate correspondence between labels and medical images is achieved, improving the effectiveness and relevance of labels; by mapping disease category attribution semantic information to coarse labels, the major category of lesions is defined, improving the macroscopic classification accuracy of semantic features; by mapping lesion detail semantic information to fine labels, fine-grained guidance is provided for the evolution trajectory of potential features, improving the semantic consistency of subsequent enhanced images.

[0072] For example, by analyzing upper gastrointestinal endoscopy diagnostic reports in medical image datasets, key semantic information is extracted. Chronic non-atrophic gastritis, gastric ulcers, esophageal polyps, and duodenal ulcers are mapped as coarse labels (defining the major lesion categories). For the coarse label chronic non-atrophic gastritis, key semantic information such as mucosal congestion, light red texture, punctate erosion, and erosion area ≤5mm² is extracted and mapped as corresponding fine labels. For the coarse label gastric ulcer, key semantic information such as round ulcer surface, diameter 8-12mm, yellowish-white coating, neat edges, and surrounding mucosal edema is extracted and mapped as corresponding fine labels. For the coarse label esophageal polyps, key semantic information such as stalked, diameter 3-6mm, light pink, and smooth surface is extracted and mapped as corresponding fine labels. All fine labels are annotated in text form and correspond one-to-one with the corresponding endoscopy images, providing precise guidance for subsequent potential feature evolution.

[0073] In some embodiments, the sparse supervision strategy for training includes: acquiring any two samples within a training batch and extracting their corresponding coarse labels; determining whether the coarse labels of the two samples are the same; if the coarse labels are the same, setting the corresponding element value in the supervision matrix as a matching identifier, allowing many-to-many alignment in the semantic space; if the coarse labels are different, setting the corresponding element value in the supervision matrix as a non-matching identifier; calculating the loss value based on the supervision matrix, and updating the parameters of the contrastive language-image pre-trained model according to the loss value. According to embodiments of the present invention, by constructing a supervision matrix by comparing the coarse labels of samples within a training batch, sparse supervision constraints are achieved, improving the efficiency of model training and reducing annotation costs; allowing many-to-many alignment in the semantic space through samples with the same coarse label enhances the correlation of semantic features and improves the accuracy of semantic encoding; by calculating the loss value based on the supervision matrix and updating the model parameters, the model training direction is precisely adjusted, improving the semantic extraction capability of the semantic encoder.

[0074] For example, setting the training batch size to 32 samples, randomly selecting samples A and B from the current batch, extracting the coarse label of sample A as chronic non-atrophic gastritis, and the coarse label of sample B as chronic non-atrophic gastritis, then in the 32×32 supervision matrix, the row and column intersection values ​​of samples A and B are set to 1, allowing many-to-many alignment between them in the semantic space; if the coarse label of sample C is gastric ulcer and the coarse label of sample D is esophageal polyp, then the corresponding intersection values ​​of the two in the supervision matrix are set to 0; after constructing the supervision matrix, probability normalization is performed, and using this supervision matrix and the diagonal matrix (for fine labels), the cross-entropy loss function is used to calculate the loss value, with an initial loss value of 0.12. Based on the loss value, the Adam optimizer is used to update the parameters of the contrastive language-image pre-trained model, with a learning rate of 0.0001. After each round of parameter updates, the loss value decreases by 3%-5% until the loss value converges to within 0.01.

[0075] In some embodiments, constructing the flow matching network includes: obtaining a first feature dimension of the latent features and a second feature dimension of the semantic features; determining whether the first feature dimension and the second feature dimension are consistent; if they are inconsistent, constructing a dimension mapping module containing convolutional layers and fully connected layers to map the latent features and the semantic features to a unified third feature dimension; and fusing the mapped features as input conditions for the velocity field network in the flow matching network. According to embodiments of the present invention, by obtaining and determining the dimensional consistency of the two features, accurate identification of feature dimension differences is achieved, improving the rationality of the flow matching network construction; by mapping features of different dimensions to a unified dimension through the dimension mapping module, effective compatibility of the two features is achieved, improving the effect of feature fusion; and by using the fused and mapped features as input to the velocity field network, accurate input conditions are provided for flow matching, improving the generation quality of enhanced image sequences.

[0076] For example, the latent features of gastroscopy images are extracted using a trained ViT-VAE encoder, with a first feature dimension of 1024. The semantic features corresponding to the fine labels are extracted using a trained semantic encoder, with a second feature dimension of 768. The dimensions of the two are inconsistent. A dimension mapping module is constructed, which includes one 3×3 convolutional layer (stride 1, padding 1) and two fully connected layers. The number of input channels for the convolutional layer is 1024 and 768, and the number of output channels is 512 for both. The fully connected layers map the 1024-dimensional latent features to 512-dimensional features and the 768-dimensional semantic features to 512-dimensional features, respectively. The 512-dimensional latent features and 512-dimensional semantic features are fused using element-level addition, with a fusion coefficient of 0.5. The fused feature vector still has a dimension of 512, which is used as the input condition for the velocity field network in the flow matching network.

[0077] In some embodiments, generating an enhanced image sequence based on the flow matching network includes: randomly selecting a source image from the medical image dataset and extracting its latent features as an initial state using the ViT-VAE encoder; matching a target image from the medical image dataset based on the fine label and extracting its latent features as a target state; constructing a velocity field equation and integrating the velocity field equation using the second-order Heun integral method; during the integration process, dividing the time interval into several intermediate time steps, calculating the velocity field based on the current state and input conditions for each intermediate time step, updating the latent features, and obtaining the latent features of the current time step; inputting the latent features of the current time step into the trained ViT-VAE decoder; the decoder reconstructing the latent features into an RGB image tensor based on a hierarchical upsampling mechanism and an attention mechanism; and using the reconstructed RGB image tensor as a frame in the enhanced image sequence, thereby forming a continuous transformation sequence from the source disease state to the target disease state. According to embodiments of the present invention, the evolution range is determined by extracting latent features from the source image and the target image, thereby clarifying the generation direction of the enhanced image sequence and improving the targeting of the sequence generation; the velocity field is calculated and the latent features are updated by the second-order Heun integral method, thereby achieving the smoothness of the evolution of latent features and improving the continuity of the enhanced image sequence; the image is reconstructed by upsampling at the decoder level and the attention mechanism, thereby restoring the detailed features of the image and improving the clarity and realism of the enhanced image.

[0078] For example, a source image of mild chronic non-atrophic gastritis is randomly selected from a medical image dataset. Its latent features (1024-dimensional vector) are extracted using a ViT-VAE encoder and used as the initial state. Based on the fine label indicating aggravated mucosal congestion and erosion area expansion to 8 mm², a target image of moderate chronic non-atrophic gastritis is matched from the dataset and its latent features (1024-dimensional vector) are extracted and used as the target state. The velocity field equation v(t,z)=∇log p is constructed. t(z) Where t is time and z is a latent feature, the equation is integrated using the second-order Heun integral method, with an integration accuracy of 1e. -4The time interval was set to [0,1] and divided into 40 intermediate time steps (step size 0.025). Each intermediate time step calculated the velocity field vector (1024 dimensions) based on the current latent feature state and the 512-dimensional fusion input conditions, and updated the latent features. The updated 1024-dimensional latent features of each time step were input into the ViT-VAE decoder. The decoder adopted a 4-layer hierarchical upsampling mechanism (upsampling from 16×16 pixels to 512×512 pixels) and combined with a channel attention mechanism (attention weight range 0.1-0.9) to reconstruct the latent features into a 512×512×3 RGB image tensor (pixel value range 0-255). The 40 intermediate time steps corresponded to 40 frames of reconstructed images. Together with the initial source image and the target image, a total of 42 frames were formed, creating a continuous enhanced image sequence from mild chronic non-atrophic gastritis to moderate chronic non-atrophic gastritis.

[0079] In some embodiments, training the flow matching network includes: calculating the mean square error between the predicted output of the velocity field network and the real velocity field; introducing a generative adversarial network (GAN) as a discriminator to calculate the adversarial loss of the generated trajectory; if the adversarial loss is greater than a preset trajectory quality threshold, increasing the weight of the adversarial loss in the total loss to force the trajectory generated by the velocity field network to be closer to the real data distribution; if the adversarial loss is less than or equal to the trajectory quality threshold, maintaining the current weight configuration. According to embodiments of the present invention, by calculating the mean square error between the predicted output of the velocity field and the real value, the prediction accuracy of the velocity field network is constrained, improving the accuracy of velocity field generation; by introducing a generative adversarial network to calculate the adversarial loss, the authenticity of the generated trajectory is judged, improving the rationality of the enhanced image sequence trajectory; by dynamically adjusting the weight configuration according to the adversarial loss, the training focus of the model is dynamically optimized, improving the fit between the trajectory generated by the flow matching network and the real data distribution.

[0080] For example, when training the flow matching network, the mean squared error (MSE) between the predicted velocity field and the actual velocity field of the velocity field network is calculated. The initial MSE is 0.012, which is gradually reduced to within 0.005 during training. A generative adversarial network based on CNN is introduced as a discriminator. The discriminator contains 3 convolutional layers and 2 fully connected layers to calculate the adversarial loss of the generated trajectory (the latent feature sequence of 40 intermediate time steps). The preset trajectory quality threshold is 0.012, and the initial weight of the adversarial loss in the total loss is 0.2. When training reaches 80 epochs, the calculated value of the adversarial loss is 0.015 > the threshold of 0.012, so the weight of the adversarial loss is increased to 0.3, while keeping other losses (mean squared error loss weight 0.7) unchanged, forcing the trajectory generated by the velocity field network to be closer to the lesion evolution trajectory of the real gastroscopy image. When training reaches 120 epochs, the calculated value of the adversarial loss is 0.010 ≤ the threshold of 0.012, so the current configuration of the adversarial loss weight of 0.3 is maintained until the end of training (total training epochs 200).

[0081] In some embodiments, S5 further includes: for retained images determined to be semantically acceptable, submitting them to a manual review process to further ensure the reliability and usability of the enhanced images; if the manual review determines that an image has structural distortion or artifacts, then marking the image as an invalid sample; if the manual review determines that an image has typical features and no distortion, then marking the image as a high-quality sample; injecting the high-quality samples into the medical image dataset, and performing the next round of model iteration training based on the updated dataset. According to embodiments of the present invention, by manually reviewing semantically acceptable images, image quality is further filtered, improving the reliability and usability of enhanced images; by marking invalid samples and high-quality samples, sample levels are distinguished, improving the purity of the medical image dataset; by injecting high-quality samples into the dataset and iteratively training, model performance is continuously optimized, improving the quality and stability of subsequently generated enhanced images.

[0082] For example, after semantic screening, 1000 semantically qualified images were retained (all with semantic similarity ≥ preset threshold 0.85). These 1000 images were submitted to a manual review panel composed of 3 senior endoscopists. The review criteria were no mucosal structure distortion, no pixel artifacts, lesion features consistent with the label, and image clarity ≥ 280 dpi. After review, 86 images were found to have slight artifacts (such as blurred edges and local pixel distortion) and were marked as invalid samples. 914 images had typical features, no distortion, and clarity between 280-320 dpi, and were marked as high-quality samples. The original medical image dataset contained 10,000 preprocessed gastroscopy images. After injecting the 914 high-quality samples, the dataset was updated to 10,914 images. The next round of model iteration training was carried out based on the updated dataset.

[0083] It should be understood that the various processes described above can be used to rearrange, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0084] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A medical image enhancement method based on flow matching, characterized in that, include: S1, construct a medical image dataset, preprocess the original images in the medical image dataset to obtain a training sample set containing images of uniform specifications; S2, the pre-constructed variational autoencoder is trained based on the training sample set to obtain the trained ViT-VAE encoder and ViT-VAE decoder; S3. Construct a labeling system that includes coarse and fine labels, build a contrastive language-image pre-trained model, and introduce a sparse supervision strategy for training to obtain a trained semantic encoder. S4, extract latent features based on the ViT-VAE encoder, extract semantic features based on the semantic encoder, construct a stream matching network, and generate an enhanced image sequence based on the stream matching network; S5, perform quality screening on the enhanced image sequence, specifically: obtain the resource status parameters of the current computing environment, and determine whether the resource status parameters meet the full verification conditions; If the resource status parameters meet the full verification conditions, then the generated images corresponding to all time steps in the enhanced image sequence are extracted as the objects to be verified. If the resource status parameters do not meet the full verification conditions, then only the generated images corresponding to the start time step and the end time step in the enhanced image sequence are extracted as the objects to be verified. Calculate the semantic similarity between the object to be verified and the target tag. If the semantic similarity is less than a preset semantic consistency threshold, it is determined that the object to be verified has semantic deviation and is removed. If the semantic similarity is greater than or equal to the semantic consistency threshold, the object to be verified is deemed semantically qualified and retained.

2. The medical image enhancement method based on flow matching according to claim 1, characterized in that, The preprocessing of the original images in the medical image dataset includes: Obtain the size parameters of the original image and determine the model scale type corresponding to the current training task; If the model scale type is the first scale, then the resolution of the original image is adjusted to the first resolution, which is used for fast training of the small-scale model. If the model scale type is the second scale, then the resolution of the original image is adjusted to the second resolution, which is used for the fine-tuning training of the large-scale model; The image after resolution adjustment is subjected to equalization processing to enhance local contrast; Apply medium filtering to the equalized image to suppress noise while preserving edge features.

3. The medical image enhancement method based on flow matching according to claim 2, characterized in that, The training of the pre-constructed variational autoencoder includes: Construct a loss function, which is composed of a weighted sum of reconstruction loss, regularization term, adversarial loss and perception loss; During training, the loss value of the validation set is monitored in real time; If the loss value of the validation set is lower than the convergence threshold for a consecutive preset number of stable rounds, it is determined that the latent spatial distribution of the variational autoencoder has reached a stable state, and training is stopped. If the loss value of the validation set does not meet the above conditions, training continues until the maximum number of training rounds is reached; The convergence threshold is used to ensure the compactness of potential features, and the preset number of stable rounds is used to prevent accidental fluctuations during model training.

4. The medical image enhancement method based on flow matching according to claim 3, characterized in that, The construction of the tag system, which includes coarse and fine tags, includes: Analyze the diagnostic reports in the medical image dataset to extract key semantic information; If the key semantic information involves the classification of a disease, it is mapped to a coarse label, which is used to define the major category of the lesion; If the key semantic information involves the texture, color, or shape of the lesion, it is mapped to a fine label; the fine label is used to provide fine-grained semantic guidance for the evolution trajectory of potential features in subsequent image generation steps.

5. The medical image enhancement method based on flow matching according to claim 4, characterized in that, The introduction of a sparse supervision strategy for training includes: Get any two samples from the training batch and extract their corresponding coarse labels; Determine whether the coarse labels of two samples are the same; If the coarse labels are the same, the corresponding element value in the supervision matrix is ​​set as the matching identifier, allowing many-to-many alignment in the semantic space; If the coarse labels are different, the corresponding element value in the supervision matrix is ​​set as a non-matching identifier; The loss value is calculated based on the supervision matrix, and the parameters of the contrastive language-image pre-trained model are updated according to the loss value.

6. The medical image enhancement method based on flow matching according to claim 5, characterized in that, The construction of the stream matching network includes: Obtain the first feature dimension of the latent feature and the second feature dimension of the semantic feature; Determine whether the first feature dimension and the second feature dimension are consistent; If they are inconsistent, a dimension mapping module containing convolutional layers and fully connected layers is constructed to map the latent features and the semantic features to a unified third feature dimension. The mapped features are fused and used as input conditions for the velocity field network in the flow matching network.

7. The medical image enhancement method based on flow matching according to claim 6, characterized in that, The generation of enhanced image sequences based on the stream matching network includes: A source image is randomly selected from the medical image dataset, and its latent features are extracted using the ViT-VAE encoder as the initial state. Based on the fine labels, target images are matched from the medical image dataset, and their latent features are extracted as target states; The velocity field equations are constructed and integrated using the second-order Heun integral method. During the integration process, the time interval is divided into several intermediate time steps. For each intermediate time step, the velocity field is calculated based on the current state and input conditions, the latent features are updated, and the latent features of the current time step are obtained. Input the latent features of the current time step into the trained ViT-VAE decoder; The decoder reconstructs the latent features into RGB image tensors based on a hierarchical upsampling mechanism and an attention mechanism. The reconstructed RGB image tensor is used as a frame in the enhanced image sequence, thereby forming a continuous transformation sequence from the source disease state to the target disease state.

8. The medical image enhancement method based on flow matching according to claim 7, characterized in that, Training the stream matching network includes: The mean square error between the predicted output of the velocity field network and the actual velocity field; Generative adversarial networks are introduced as discriminators to calculate the adversarial loss of the generated trajectories; If the adversarial loss is greater than the preset trajectory quality threshold, the weight of the adversarial loss in the total loss is increased to force the trajectory generated by the velocity field network to be closer to the real data distribution. If the adversarial loss is less than or equal to the trajectory quality threshold, then the current weight configuration is maintained.

9. The medical image enhancement method based on flow matching according to claim 8, characterized in that, The S5 also includes: Images deemed semantically acceptable are submitted to a manual review process. If a manual review determines that an image has structural distortion or artifacts, the image will be marked as an invalid sample. If a manual review determines that an image has typical features and is not distorted, then the image is marked as a high-quality sample. The high-quality samples are injected into the medical image dataset, and the next round of model iteration training is performed based on the updated dataset.

Images