Chest x-ray data generation method and system, electronic device and storage medium

Abstract

The present application discloses a chest X-ray data generation method and system, electronic equipment and storage medium, which relates to the technical field of medical image processing and deep learning. The method acquires a multi-source heterogeneous chest X-ray dataset; uses a hierarchical prompt engineering strategy to perform semantic mapping conversion on pathological labels to generate structured radiology reports and build a training set; uses a low-rank adaptive fine-tuning strategy to perform parameter incremental update on the cross-attention mechanism and text encoder in the pre-trained latent diffusion model to obtain a domain-adapted diffusion model; constructs a unified classification space and uses a reverse frequency weighted sampling mechanism to calculate a sampling probability to extract a generated category; calls a large language model to generate a clinical descriptive text, inputs the domain-adapted diffusion model under a single-label condition isolation strategy, outputs a high-fidelity chest X-ray image, and constructs an image-text pair dataset. The present application effectively alleviates the long-tail distribution and greatly improves the fidelity and diversity of the generated data.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and deep learning technology, and in particular to a method, system, electronic device and storage medium for generating chest X-ray data. Background Technology

[0002] In the current research field of chest X-ray-assisted diagnosis, the performance of deep learning models highly depends on large-scale, high-quality labeled data. However, there are many challenges in actual clinical applications and data construction. First, existing publicly available chest X-ray datasets generally suffer from severe class imbalance, with abundant samples of common diseases but extremely scarce samples of rare diseases, forming a significant long-tail distribution. This distribution bias causes models to tend to fit the head class during training, while their generalization ability on the tail classes is significantly insufficient, severely restricting the clinical reliability of the assisted diagnostic system. Second, building medical multimodal pre-trained models requires massive amounts of image-text pair data; however, manual annotation is costly and has a high professional threshold, resulting in the limited size of existing image-text pair datasets. In addition, traditional data augmentation methods are difficult to generate high-fidelity medical images with rich semantic information, failing to effectively alleviate the data bottleneck problem faced by multimodal pre-training. Existing general-domain pre-trained latent diffusion models often face domain gaps when directly applied to medical image generation, making it difficult to accurately capture specific anatomical structures and pathological features, and direct full-parameter fine-tuning carries the risk of overfitting and high computational costs. Summary of the Invention

[0003] To address the critical technical problems of severe class imbalance, long-tailed distribution, and extreme scarcity of high-quality image-text pairs in existing publicly available chest X-ray datasets, this invention provides a method, system, electronic device, and storage medium for generating chest X-ray data.

[0004] In a first aspect, the present invention provides a method for generating chest X-ray data, comprising the following steps: Acquire a pre-trained general latent diffusion model, a pre-trained text encoder, and a multi-source heterogeneous chest X-ray image-label dataset; A hierarchical prompting engineering strategy was used to perform semantic mapping transformation on pathology labels in a multi-source heterogeneous chest X-ray image-label dataset to generate structured radiology reports. The real chest X-ray images corresponding to the pathology labels were then paired with the structured radiology reports to construct an image report training set. Based on the image report training set, a low-rank adaptive fine-tuning strategy is used to incrementally update the parameters of the cross-attention mechanism projection matrix and the pre-trained text encoder in the pre-trained general latent diffusion model to obtain a domain-adaptive diffusion model. Construct a unified classification space containing multiple pathological classifications, calculate the sampling probability of each target pathological category in the unified classification space, and extract the target generation category from the unified classification space based on the sampling probability; The large language model is invoked to generate clinical descriptive text covering clinical details for the target category. The clinical descriptive text is then input as a conditional feature vector into the domain adaptation diffusion model, and a single-label conditional isolation strategy is executed to output a synthesized high-fidelity chest X-ray image. Synthesized high-fidelity chest X-ray images are paired and associated with corresponding clinical descriptive texts to construct and output a high-fidelity image-text pair dataset.

[0005] As an optional implementation of the first aspect of this application, the step of using a hierarchical prompting engineering strategy to perform semantic mapping transformation on pathological labels in a multi-source heterogeneous chest X-ray image-label dataset to generate a structured radiology report includes: mapping pathological labels to a preset thoracic anatomical region to obtain anatomical location-level semantics; extracting visual feature descriptions with degree modifiers and morphological descriptive words to obtain pathological characterization-level semantics; adding discovery-definitive statements that conform to radiology report specifications to obtain clinical context-level semantics; and fusing and splicing the anatomical location-level semantics, pathological characterization-level semantics, and clinical context-level semantics to output a structured radiology report.

[0006] As an optional implementation of the first aspect of this application, the incremental parameter update of the cross-attention mechanism projection matrix and the pre-trained text encoder in the pre-trained general latent diffusion model using the low-rank adaptive fine-tuning strategy is as follows: freezing the original weight matrices of the pre-trained general latent diffusion model and the pre-trained text encoder; introducing a first low-rank matrix and a second low-rank matrix multiplied together as incremental weight matrices next to the original weight matrices; adding the product of the input features and the original weight matrices to the product of the input features and the incremental weight matrices as the forward propagation feature output result; using the standard denoising loss function as the optimization objective, iteratively updating the parameters of the first low-rank matrix and the second low-rank matrix through the backpropagation algorithm until convergence.

[0007] As an optional implementation of the first aspect of this application, the calculation expression of the denoising loss function is as follows: In the formula, This represents the value of the denoising loss function. Let z represent the expectation operation, c represent the original image data, c represent the conditional text encoded by CLIP, and t represent the diffusion time. This represents the added real noise. This represents the latent representation at time t. This represents the conditional text embedding obtained by mapping from the pre-trained text encoder. This represents the predicted noise output by the denoising network module in the pre-trained general latent diffusion model.

[0008] As an optional implementation of the first aspect of this application, the step of calculating the sampling probability of each target pathological category within the unified classification space and extracting the target generated category from the unified classification space based on the sampling probability includes: counting the total number of real samples for each pathological category within the unified classification space; and using an inverse frequency-weighted sampling mechanism to calculate the sampling probability of each target pathological category, as shown in the following expression: In the formula, Indicates the target pathology category The sampling probability, Indicates the target pathology category The total number of real samples, The smoothing coefficient is represented by C, which represents the total number of categories in the unified classification space. This represents the category index variable; it performs weighted random sampling using the sampling probability of each target pathology category and outputs the generated target category.

[0009] As an optional implementation of the first aspect of this application, the step of calling the large language model to generate clinical descriptive text covering clinical details for the target generation category includes: inputting the target generation category into a preset medical knowledge base retrieval module to extract the corresponding lesion location distribution data and morphological appearance feature data; encapsulating the target generation category, lesion location distribution data, and morphological appearance feature data into a prompt word template and inputting it into the large language model; and outputting single-label clinical descriptive text containing lesion location and morphological feature details through the autoregressive generation mechanism of the large language model.

[0010] As an optional implementation of the first aspect of this application, the step of inputting clinical descriptive text as a conditional feature vector into a domain-adaptive diffusion model and executing a single-label conditional isolation strategy to output a synthesized high-fidelity chest X-ray image includes: performing feature encoding on the clinical descriptive text to obtain a conditional feature vector and injecting it into the domain-adaptive diffusion model; constructing a negative prompt word set based on a unified classification space, wherein the negative prompt word set contains implicit co-occurring pathological description features of all pathological categories other than the target generation category; during the reverse denoising generation process of the domain-adaptive diffusion model, using the negative prompt word set to suppress the cross-attention map and shield the generation region of co-occurring pathological features; allocating an independent random seed generation space for each target generation category, performing numerical diffusion sampling based on the separated single-label conditions, and outputting a synthesized high-fidelity chest X-ray image.

[0011] Secondly, embodiments of this application provide a chest X-ray data generation system, comprising: The data preparation and preprocessing module is used to acquire a pre-trained general latent diffusion model, a pre-trained text encoder, and a multi-source heterogeneous chest X-ray image-label dataset. The hierarchical prompting engineering strategy is used to perform semantic mapping transformation on the pathology labels in the multi-source heterogeneous chest X-ray image-label dataset to generate structured radiology reports. The real chest X-ray images corresponding to the pathology labels are paired with the structured radiology reports to construct an image report training set. The model fine-tuning and adaptation module is used to incrementally update the parameters of the cross-attention mechanism projection matrix and the pre-trained text encoder in the pre-trained general latent diffusion model based on the image report training set and using a low-rank adaptive fine-tuning strategy to obtain a domain-adaptive diffusion model. The text generation and conditional input module is used to construct a unified classification space containing multiple pathological classifications, calculate the sampling probability of each target pathological category in the unified classification space, and extract the target generated category from the unified classification space based on the sampling probability. The image synthesis and high-fidelity generation module is used to call a large language model to generate clinical descriptive text covering clinical details for the target generation category. The clinical descriptive text is input as a conditional feature vector into the domain adaptation diffusion model, and a single-label conditional isolation strategy is executed to output a synthesized high-fidelity chest X-ray image. The dataset construction and output module is used to pair and associate synthesized high-fidelity chest X-ray images with corresponding clinical descriptive texts to construct and output a high-fidelity image-text pair dataset.

[0012] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0013] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention integrates multi-source heterogeneous datasets and introduces a low-rank adaptive fine-tuning strategy to efficiently fine-tune the parameters of the cross-attention mechanism and text encoder of a general pre-trained latent diffusion model. This overcomes the domain gap in the field of medical imaging, enabling the model to accurately learn the pathological semantic alignment relationships in chest X-rays. This avoids the risk of overfitting and significantly reduces computational costs.

[0015] 2. In response to the long-tail distribution characteristics of existing data, this invention innovatively proposes a reverse frequency-weighted sampling mechanism and a single-label conditional isolation strategy to generate high-fidelity images with balanced class distribution in a targeted manner. This effectively fills the sample gap for rare diseases and significantly alleviates the problem of insufficient generalization ability caused by the long-tail distribution of the dataset.

[0016] 3. This invention introduces a large language model and hierarchical prompting engineering, expanding discrete pathological labels into structured clinical descriptive texts that cover anatomical location, pathological features, and clinical context. This not only enhances the semantic richness of the generated conditional texts but also successfully constructs a large-scale, high-quality image-text pair dataset, providing solid and reliable data support for medical multimodal pre-training models and possessing significant clinical application value. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a method for generating chest X-ray data according to an embodiment of the present invention; Figure 2 A framework diagram for generating high-fidelity chest X-ray data based on a diffusion model, provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a chest X-ray data generation system provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] Example 1 Please see Figure 1This embodiment proposes a method for generating chest X-ray data optimized for long-tail distributions. By integrating heterogeneous medical data sources and efficient parameter fine-tuning strategies, it achieves large-scale generation of high-quality single-label image-text pairs. Figure 2 As shown, this method comprises two core stages: cross-domain fine-tuning based on low-rank adaptation and semantically driven generation with class balance. The former introduces large-scale image-label datasets such as ChestX-ray14 and CheXpert, combined with a GPT-4-driven hierarchical label expansion mechanism, to adapt the pre-trained diffusion model in a parameter-efficient manner; the latter implements a reverse frequency-weighted sampling and single-label conditional isolation strategy to synthesize balanced high-fidelity images of 20 pathological categories and construct a large-scale image-text pair dataset.

[0021] 1. Cross-domain fine-tuning based on low-rank adaptation Existing general-domain pre-trained latent diffusion models often face a domain gap when directly applied to medical image generation, making it difficult to accurately capture specific anatomical structures and pathological features. While existing methods have explored full-parameter fine-tuning of U-Net and CLIP text encoders, they suffer from overfitting risks and high computational costs when handling heterogeneous data sources. In the first stage, this study introduces a low-rank adaptive technique to efficiently fine-tune the parameters of the pre-trained RoentGen model, overcoming the limitation of traditional methods that rely solely on image-report pairs.

[0022] Specifically, this method, while retaining the powerful semantic generation capabilities of the baseline model, integrates two large-scale image-label datasets, ChestXray14 and CheXpert, to address the insufficient sample size for some rare diseases in the original MIMIC-CXR dataset. The MIMIC-CXR dataset provides 38,000 post-anterior view image-report pairs as basic supervision signals; ChestXray14 contributes 112,000 chest X-rays with 14 pathological labels; and CheXpert additionally provides 224,000 multi-institutional data points with labels extracted via natural language processing to expand pathological coverage and enhance label reliability. Addressing the lack of free text descriptions in image-label pairs, this invention employs a hierarchical prompting engineering strategy based on GPT-4 to automatically convert labels into structured radiology reports. This conversion mechanism comprises three semantic levels: the anatomical localization layer maps pathological labels to specific pleural regions, such as "Cardiomegaly" as "cardiac shadow" and "Pneumothorax" as "lung field near the pleura"; the pathological characterization layer introduces degree modifiers ("mild," "moderate," "significant," "diffuse") and morphological descriptive terms ("patchy," "linear," "homogeneous and dense," "luminous area without lung markings") to construct visual feature descriptions; and the clinical context layer adds definitive statements of discovery ("suggests," "considers," "consistent," "recommended to be considered in conjunction with clinical findings") to conform to radiological reporting standards. For example, the label "Pleural Effusion" is converted to "Blunted costophrenic angle, homogeneous and dense shadow visible in the middle and lower lung fields, with an upper margin that is higher on the outside and lower on the inside, suggesting moderate pleural effusion, ultrasound localization and quantification recommended."

[0023] The specific implementation of low-rank adaptive fine-tuning is as follows: assuming the pre-trained diffusion model parameters are... LoRA policy freeze And by introducing two low-rank matrices and To simulate the update of weights, where rank The forward propagation process during fine-tuning can be formalized as follows: in, For input features, The incremental weights are to be trained. This strategy is applied to the projection matrix in the cross-attention mechanism of U-Net. On the text encoder side, CLIP ViT-L / 14 is chosen as the text encoder, and LoRA is used for fine-tuning. This allows the model to map medical text prompts more accurately to the image latent space. The fine-tuning process is performed on the integrated image-label dataset, with the optimization objective being the standard denoising loss function: in, Indicates time The potential representation, For conditional text embedding using CLIP encoding, This is a U-Net used for denoising. In this way, the model accurately learns the pathological semantic alignment relationships in chest X-rays while maintaining its original generative capabilities.

[0024] 2. Category-balanced semantic-driven generation The second phase implements a class-balanced single-label synthesis strategy. To generate high-quality image-text pairs, a multimodal large language model is introduced as a semantic enhancement module. For each disease label, a text report with rich clinical descriptive details is generated using the medical knowledge base of the large language model. These reports not only include disease category information but also cover subtle features such as lesion location and morphology. Subsequently, the generated descriptive text is used as conditional input to a finely tuned diffusion model to synthesize high-fidelity chest X-ray images. This process strictly controls the generation logic to ensure that each generated sample is a high-quality single-label image, thereby constructing a large-scale, semantically rich, and class-balanced image-text pair dataset, effectively filling the data gaps in the tail categories. First, after integrating the label systems of the three datasets and removing duplicates, a unified classification space was constructed, encompassing 21 pathological categories, including Atelectasis, Cardiomegaly, Consolidation, Edma, Effusion, Emphysema, Fibrosis, Hernia, Infiltration, Mass, Nodule, Pleural Thickening, Pneumonia, and Pneumothorax. After removing "No Finding (Normal)" categories, 20 tail-based target classes were generated. Addressing the long-tail distribution characteristics of the original data, an inverse frequency-weighted sampling mechanism was implemented, classifying... The sampling probability is: in For category The number of samples, , Smoothing coefficient To avoid pattern collapse caused by oversampling, the generation process strictly adheres to a single-label conditional isolation protocol: each sample is driven by only a single pathological description, and a set of negative cues is explicitly injected to suppress the implicit generation of co-occurring pathologies. Pre-allocate an independent random seed space for each class To ensure manageable diversity, the inference phase employs a classifier-guided scaling mechanism. The 75-step pseudo-numerical diffusion model sampling process takes approximately 3.2 seconds to generate a single 512×512 image.

[0025] Example Experiment Setup and Evaluation Benchmark The experiment was mainly divided into two parts: (1) image fidelity assessment to verify the consistency between the generated images and the distribution of real medical images; and (2) generation diversity assessment to verify the model's ability to generate differentiated samples under single-text conditions. All experiments were based on the MIMIC-CXR dataset as the real distribution benchmark and compared the generation scenarios of multi-label and single-label. In particular, the image generation quality driven by single-label reports was examined. The specific indicators are as follows: (1) Fraser's initial distance FID is used to measure the distance between the generated image distribution and the real image distribution; a lower value indicates better generation quality. Considering the special characteristics of medical images, a dual feature extraction strategy is adopted: in, and These represent the characteristic mean and covariance matrix, respectively.

[0026] Feature extraction networks: The InceptionV3 model pre-trained on ImageNet and the XRV (DenseNet-121) model pre-trained on a large chest X-ray dataset were used as feature extractors, respectively. InceptionV3 focuses on general visual features, while XRV focuses on medical pathology features.

[0027] Data sampling: 10,000 real images were randomly sampled from the MIMIC-CXR test set as the reference set. The generated sets included samples generated from the baseline and samples generated by the method in this paper. To eliminate randomness, each experiment was repeated 10 times, and the mean and standard deviation were taken.

[0028] (2) Multi-scale structural similarity MS-SSIM is used to evaluate the diversity of generated images. It calculates the MS-SSIM value among multiple images generated under the same text prompt word. The lower the value, the higher the diversity and the more effectively the pattern collapse can be avoided.

[0029] Experimental Procedure: For three different experimental settings (multi-label prompts, single-label prompts, and the single-label report prompts constructed in this paper), 1000 prompts were randomly sampled for each. For each prompt, 100 images were continuously generated using the model, and the average MS-SSIM value between each pair of these 100 images was calculated.

[0030] Comparative analysis: This metric aims to verify whether the fine-tuned model has achieved a balance between "faithfulness to textual conditions" and "generating reasonable random variations".

[0031] Experimental results demonstrate that the proposed fine-tuning method performs exceptionally well in single-label generation tasks. Thanks to the fine-grained text descriptions generated by LLM and the LoRA fine-tuning strategy, the model significantly outperforms the baseline in terms of FID (Fragment ID), particularly in the low FID score under XRV feature extraction, proving the high pathological fidelity of the generated images. Furthermore, MS-SSIM results show that this method maintains reasonable sample diversity while preserving high-quality generation, validating its effectiveness as a data augmentation technique in mitigating the long-tail distribution problem.

[0032] In summary, this invention proposes a chest X-ray data generation framework based on the synergy of a diffusion model and a large language model, effectively addressing the long-standing challenges of class imbalance, long-tailed distribution, and the scarcity of high-quality image-text pairs in medical image analysis. By integrating multi-source heterogeneous datasets and introducing a LoRA fine-tuning strategy, a smooth transfer of the generation model from the general vision domain to the specific medical domain is successfully achieved. Simultaneously, the large language model is used to expand discrete labels into semantic descriptions rich in clinical details, constructing a high-fidelity and class-balanced single-label image-text pair dataset. Quantitative experimental results confirm that the chest X-ray images generated by this method significantly outperform existing baselines in both pathological fidelity and structural consistency. In the future, this framework is expected to be further extended to multimodal medical image generation tasks, providing a solid data foundation for building a more equitable and robust computer-aided diagnostic system.

[0033] Example 2 Please see Figure 3 The diagram shown is a structural schematic of a chest X-ray data generation system according to the second embodiment of this application. The system includes the following key modules: The data preparation and preprocessing module 100 is used to acquire a pre-trained general latent diffusion model, a pre-trained text encoder, and a multi-source heterogeneous chest X-ray image-label dataset; it uses a hierarchical prompting engineering strategy to perform semantic mapping transformation on the pathology labels in the multi-source heterogeneous chest X-ray image-label dataset to generate structured radiology reports, and pairs the real chest X-ray images corresponding to the pathology labels with the structured radiology reports to construct an image report training set; The model fine-tuning and adaptation module 200 is used to incrementally update the parameters of the cross-attention mechanism projection matrix and the pre-trained text encoder in the pre-trained general latent diffusion model based on the image report training set and using a low-rank adaptive fine-tuning strategy to obtain the domain-adaptive diffusion model. The text generation and conditional input module 300 is used to construct a unified classification space containing multiple pathological classifications, calculate the sampling probability of each target pathological category in the unified classification space, and extract the target generation category from the unified classification space based on the sampling probability. The image synthesis and high-fidelity generation module 400 is used to call a large language model to generate clinical descriptive text covering clinical details for the target generation category, input the clinical descriptive text as a conditional feature vector into the domain adaptation diffusion model, and execute a single-label conditional isolation strategy to output a synthesized high-fidelity chest X-ray image. The dataset construction and output module 500 is used to pair and associate synthesized high-fidelity chest X-ray images with corresponding clinical descriptive texts to construct and output a high-fidelity image-text pair dataset.

[0034] The chest X-ray data generation system in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc. This application embodiment does not impose specific limitations.

[0035] The chest X-ray data generation system in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0036] The chest X-ray data generation system provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method embodiment of the chest X-ray data generation method are not described in detail here to avoid repetition.

[0037] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiment of a method for generating chest X-ray data and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0038] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of a method for generating chest X-ray data and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0039] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0040] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0041] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0042] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for generating chest X-ray data, characterized in that, Includes the following steps: Acquire a pre-trained general latent diffusion model, a pre-trained text encoder, and a multi-source heterogeneous chest X-ray image-label dataset; A hierarchical prompting engineering strategy was used to perform semantic mapping transformation on pathology labels in a multi-source heterogeneous chest X-ray image-label dataset to generate structured radiology reports. The real chest X-ray images corresponding to the pathology labels were then paired with the structured radiology reports to construct an image report training set. Based on the image report training set, a low-rank adaptive fine-tuning strategy is used to incrementally update the parameters of the cross-attention mechanism projection matrix and the pre-trained text encoder in the pre-trained general latent diffusion model to obtain a domain-adaptive diffusion model. Construct a unified classification space containing multiple pathological classifications, calculate the sampling probability of each target pathological category within the unified classification space, and extract the target generation category from the unified classification space based on the sampling probability; The large language model is invoked to generate clinical descriptive text covering clinical details for the target category. The clinical descriptive text is then input as a conditional feature vector into the domain adaptation diffusion model, and a single-label conditional isolation strategy is executed to output a synthesized high-fidelity chest X-ray image. Synthesized high-fidelity chest X-ray images are paired and associated with corresponding clinical descriptive texts to construct and output a high-fidelity image-text pair dataset.

2. The method according to claim 1, characterized in that, The hierarchical prompting engineering strategy is used to perform semantic mapping transformation on pathological labels in a multi-source heterogeneous chest X-ray image-label dataset to generate a structured radiology report, including: Map pathology labels to preset thoracic anatomical regions to obtain anatomical location hierarchical semantics; Extract visual feature descriptions with degree modifiers and morphological descriptors to obtain the semantic hierarchy of pathological representations; Add definitive statements of findings that conform to radiology reporting standards to obtain semantic information at the clinical context level; The semantics of anatomical location, pathological features, and clinical context are fused and combined to output a structured radiology report.

3. The method according to claim 1, characterized in that, The process of incrementally updating the parameters of the cross-attention mechanism projection matrix and the pre-trained text encoder in the pre-trained general latent diffusion model using a low-rank adaptive fine-tuning strategy is as follows: Freeze the original weight matrices of the pre-trained general latent diffusion model and the pre-trained text encoder; Introduce a first low-rank matrix and a second low-rank matrix, which are multiplied together, as an incremental weight matrix next to the original weight matrix. The product of the input features and the original weight matrix is ​​added to the product of the input features and the incremental weight matrix, and this is used as the forward propagation feature output. Using the standard denoising loss function as the optimization objective, the parameters of the first and second low-rank matrices are iteratively updated through the backpropagation algorithm until convergence.

4. The method according to claim 3, characterized in that, The calculation expression for the denoising loss function is as follows: ； In the formula, This represents the value of the denoising loss function. Let z represent the expectation operation, c represent the original image data, c represent the conditional text encoded by CLIP, and t represent the diffusion time. This represents the added real noise. This represents the latent representation at time t. This represents the conditional text embedding obtained by mapping from the pre-trained text encoder. This represents the predicted noise output by the denoising network module in the pre-trained general latent diffusion model.

5. The method according to claim 1, characterized in that, The calculation of the sampling probability of each target pathological category within the unified classification space, and the extraction of the target generation category from the unified classification space based on the sampling probability, includes: The total number of real samples for each pathological category within a unified classification space is counted. A reverse frequency-weighted sampling mechanism is used to calculate the sampling probability of each target pathological category. The calculation expression is as follows: ； In the formula, Indicates the target pathology category The sampling probability, Indicates the target pathology category The total number of real samples, The smoothing coefficient is represented by C, which represents the total number of categories in the uniform classification space. Represents a category index variable; Weighted random sampling is performed using the sampling probability of each target pathology category, and the generated target category is output.

6. The method according to claim 5, characterized in that, The invocation of the large language model generates clinical descriptive text covering clinical details for the target category, including: Input the target category into the preset medical knowledge base retrieval module to extract the corresponding lesion location distribution data and morphological appearance feature data; The target generation category, lesion location distribution data, and morphological appearance feature data are encapsulated into prompt word templates and input into the large language model; The autoregressive generation mechanism of the large language model outputs single-label clinical descriptive text containing details of lesion location and morphological features.

7. The method according to claim 6, characterized in that, The process of inputting clinical descriptive text as a conditional feature vector into a domain adaptation diffusion model and executing a single-label conditional isolation strategy to output a synthesized high-fidelity chest X-ray image includes: Conditional feature vectors are obtained by feature encoding of clinical descriptive texts and then injected into a domain-adaptive diffusion model. A set of negative prompt words is constructed based on a unified classification space. The set of negative prompt words contains implicit co-occurrence pathological description features of all other pathological categories except the target generation category. In the process of reverse denoising generation of the domain-adaptive diffusion model, the negative prompt word set is used to suppress the cross-attention map and block the generation region of co-occurring pathologies. An independent random seed generation space is assigned to each target generation category. Numerical diffusion sampling is performed based on the separated single-label conditions to output a synthesized high-fidelity chest X-ray image.

8. A chest X-ray data generation system, characterized in that, The chest X-ray data generation system is applied to the chest X-ray data generation method as described in claim 1, and the chest X-ray data generation system comprises: The data preparation and preprocessing module is used to acquire a pre-trained general latent diffusion model, a pre-trained text encoder, and a multi-source heterogeneous chest X-ray image-label dataset. The hierarchical prompting engineering strategy is used to perform semantic mapping transformation on the pathology labels in the multi-source heterogeneous chest X-ray image-label dataset to generate structured radiology reports. The real chest X-ray images corresponding to the pathology labels are paired with the structured radiology reports to construct an image report training set. The model fine-tuning and adaptation module is used to incrementally update the parameters of the cross-attention mechanism projection matrix and the pre-trained text encoder in the pre-trained general latent diffusion model based on the image report training set and using a low-rank adaptive fine-tuning strategy to obtain a domain-adaptive diffusion model. The text generation and conditional input module is used to construct a unified classification space containing multiple pathological classifications, calculate the sampling probability of each target pathological category in the unified classification space, and extract the target generated category from the unified classification space based on the sampling probability. The image synthesis and high-fidelity generation module is used to call a large language model to generate clinical descriptive text covering clinical details for the target generation category. The clinical descriptive text is input as a conditional feature vector into the domain adaptation diffusion model, and a single-label conditional isolation strategy is executed to output a synthesized high-fidelity chest X-ray image. The dataset construction and output module is used to pair and associate synthesized high-fidelity chest X-ray images with corresponding clinical descriptive texts to construct and output a high-fidelity image-text pair dataset.

9. An electronic device, characterized in that, The method includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, the steps of a method for generating chest X-ray data as described in any one of claims 1-7 are implemented.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the steps of a method for generating chest X-ray data as described in any one of claims 1-7.

Images