A chest X-ray visual positioning method based on semi-supervised learning
By combining semi-supervised learning with anatomical location information from medical text and optimizing pseudo-labels, the problem of insufficient lesion localization accuracy in chest X-ray images is solved, achieving efficient localization under limited annotation conditions, and is applicable to various visual and language models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHILDRENS HOSPITAL OF FUDAN UNIV
- Filing Date
- 2026-01-27
- Publication Date
- 2026-05-29
AI Technical Summary
Existing visual localization methods for lesions or abnormal areas in chest X-ray images rely heavily on a large amount of precisely labeled data, resulting in insufficient localization accuracy. Furthermore, general visual localization models are difficult to adapt to the complexity of medical images.
A semi-supervised learning mechanism is adopted, combining teacher and student models. Spatial priors are constructed using anatomical location information in medical texts, and pseudo-labels are optimized through regional constraint information to improve the localization accuracy of unlabeled samples.
It improves the accuracy and stability of localization under limited annotation conditions, reduces the dependence on manually annotated data, is applicable to various vision and language models, and has good prospects for clinical application.
Smart Images
Figure CN122115332A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and medical image processing technology, specifically to a visual localization method for chest X-ray images based on semi-supervised learning, which is used to automatically locate the spatial position of the lesion or abnormality indicated in the text description in the image by combining the chest X-ray image and its corresponding text description information. Background Technology
[0002] Chest X-ray is one of the most widely used medical imaging techniques in clinical practice due to its advantages such as convenience, low cost, and low radiation dose. It is commonly used for screening and diagnosis of pneumonia, tuberculosis, masses, and other lung diseases. In clinical practice, radiologists typically generate a text-based diagnostic report after reviewing the images, describing the lesion features and their anatomical location observed in the images.
[0003] Establishing a correspondence between the aforementioned unstructured textual descriptions and specific regions in images is a crucial issue in intelligent medical image analysis, with significant implications for applications such as intelligent assisted diagnosis, teaching and training, report structuring, and cross-modal retrieval. This type of task is commonly referred to as visual localization or representational expression understanding in the field of computer vision.
[0004] In recent years, deep learning-based visual localization methods have made some progress in natural scenes, but their direct application to chest X-ray images still faces significant challenges. On the one hand, medical image annotation requires highly specialized medical knowledge, and acquiring large-scale, accurate "image-text-spatial annotation" data is costly and time-consuming, making it difficult to adequately train supervised learning methods. On the other hand, medical texts contain a large number of technical terms and complex spatial descriptions, while chest X-ray images have characteristics such as overlapping tissue structures and low contrast, making it difficult to directly apply general visual localization models and limiting localization accuracy.
[0005] Semi-supervised learning can leverage large amounts of unlabeled data to improve model performance, making it an effective way to alleviate the problem of insufficient labeled data. Among these, the semi-supervised training framework combining teacher and student models has already been applied in some visual tasks. However, current techniques have not fully utilized the inherent anatomical location information in medical texts as spatial priors to introduce into the semi-supervised visual localization process, thereby improving the quality of pseudo-labels and overall localization performance in unlabeled samples.
[0006] Therefore, there is an urgent need for a new technical solution that, based on fully utilizing the location information in medical text, introduces a semi-supervised learning mechanism to achieve accurate localization of lesions or abnormal areas in chest X-ray images. Summary of the Invention
[0007] This invention aims to provide a semi-supervised learning-based visual localization method for chest X-ray images, in order to solve the problems in existing medical image visual localization tasks that rely heavily on a large amount of precisely labeled data and have insufficient localization accuracy under limited labeling conditions.
[0008] To achieve the above objectives, the present invention proposes the following technical solution: A visual localization method for chest X-ray images based on semi-supervised learning includes the following steps: First, obtain chest X-ray images and corresponding text description information, and parse and extract anatomical location information related to lesions or abnormalities from the text description information; Secondly, based on the anatomical location information, regional constraint information corresponding to the chest X-ray image space is constructed to indicate the possible spatial range of the target area in the image, thereby providing spatial prior for subsequent visual localization. Then, the chest X-ray image, text description information, and region constraint information are input into the visual localization model, and the target region localization result corresponding to the semantics of the text description is output. The visual localization model is trained using a semi-supervised learning approach. By introducing a training framework of teacher and student models, the region constraint information is used during training to filter, weight, or constrain candidate localization results generated from unlabeled samples, thereby improving the quality of pseudo-labels and thus enhancing the accuracy and stability of target region localization.
[0009] Compared with the prior art, the present invention has the following beneficial effects: (1) Using the inherent anatomical location information in medical texts to construct spatial priors can effectively guide the visual positioning process and improve the rationality of the positioning results.
[0010] (2) Introduce a semi-supervised learning mechanism to fully explore the value of unlabeled medical image data and significantly reduce the dependence on manually labeled data.
[0011] (3) Optimize pseudo-labels by using region constraint information to improve the stability of semi-supervised training and the localization accuracy of the final model.
[0012] (4) The method has a flexible structure and is applicable to a variety of visual and language models, and has good prospects for clinical application and promotion. Attached Figure Description
[0013] Figure 1 This is a schematic diagram of the overall process of the visual localization method for chest X-ray images based on semi-supervised learning provided in an embodiment of the present invention.
[0014] Figure 2 This is a schematic diagram of the visual positioning model structure provided in an embodiment of the present invention.
[0015] Figure 3 This is a schematic diagram of a semi-supervised training process that combines a teacher model and a student model, as provided in an embodiment of the present invention. Detailed Implementation
[0016] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this invention.
[0017] The semi-supervised learning-based visual localization method for chest X-ray images in this embodiment can be implemented by an electronic device, which includes a processor, a memory, and a computer program that can run on the processor.
[0018] Step S1: Extract anatomical location information Chest X-ray images and their corresponding text descriptions are acquired. The text descriptions are analyzed using natural language processing to identify text fragments that represent anatomical locations of the human body, and these text fragments are mapped to standardized anatomical location information.
[0019] In this embodiment, the natural language processing method can be either a rule-based method or a language model based on deep learning. Those skilled in the art should understand that the specific implementation of this step is not limited to the above example.
[0020] Step S2: Constructing Region Constraint Information Based on the anatomical location information extracted in step S1, regional constraint information corresponding to the chest X-ray image space is constructed to indicate the possible spatial range of the target area in the image.
[0021] In this embodiment, the region constraint information can be obtained by anatomical structure identification of chest X-ray images, and is used to indicate the image region corresponding to the anatomical location information. It should be understood that the region constraint information can be constructed in various ways, and the present invention does not limit it.
[0022] Step S3: Visual localization prediction: The chest X-ray image, text description information and regional constraint information are input into the visual localization model, and the visual localization model outputs the target region localization result corresponding to the semantics of the text description.
[0023] Visual positioning model structure: such as Figure 2As shown, the visual localization model includes an image feature extraction module, a text feature extraction module, a multimodal feature fusion module, and a localization prediction module. The image feature extraction module extracts image features from chest X-ray images; the text feature extraction module extracts text features from text description information; the multimodal feature fusion module fuses image features and text features to generate a joint feature representation; and the localization prediction module outputs the target region localization result based on the joint feature representation. In this embodiment, the multimodal feature fusion module can employ one or more implementation methods, such as attention mechanisms, correlation modeling mechanisms, or feature weighting mechanisms.
[0024] Semi-supervised training process: such as Figure 3 As shown, the visual localization model is trained using a semi-supervised training framework combining a teacher model and a student model. The student model undergoes fully supervised training on labeled samples; the teacher model is used to generate candidate localization results for unlabeled samples. During the training process on unlabeled samples, the region constraint information constructed in step S2 is used to filter, weight, or evaluate the credibility of the candidate localization results generated by the teacher model, thereby determining the pseudo-label localization results used to train the student model.
[0025] By using the above methods, the model can make full use of information in unlabeled data and improve overall positioning performance.
[0026] Functions and effects: According to the semi-supervised learning-based visual localization method for chest X-ray images provided in this embodiment, the anatomical location information parsed from medical text is transformed into spatial priors and introduced into the semi-supervised training process, which effectively improves the quality of pseudo-labels in unlabeled samples and enhances the localization accuracy and stability of the model under limited labeling conditions.
[0027] Furthermore, this method has strong structural versatility, can be adapted to various vision and language models, is suitable for practical clinical applications, and has good practical value and prospects for promotion.
[0028] The above embodiments are merely preferred embodiments of the present invention. Those skilled in the art can make various changes or substitutions without departing from the technical concept of the present invention, and all such changes or substitutions should fall within the protection scope of the present invention.
Claims
1. A visual localization method for chest X-ray images based on semi-supervised learning, characterized in that, Includes the following steps: Step S1: Obtain chest X-ray images and corresponding text description information, and extract anatomical location information related to lesions or abnormalities from the text description information; Step S2: Based on the anatomical location information, construct regional constraint information corresponding to the space of the chest X-ray image. The regional constraint information is used to indicate the possible spatial range of the target area in the image. Step S3: Input the chest X-ray image, the text description information, and the region constraint information into the visual localization model, and output the target region localization result corresponding to the semantics of the text description information; The visual positioning model is trained using a semi-supervised learning method. During the training process, the region constraint information is used to filter, weight, or constrain the candidate positioning results generated in the unlabeled samples, so as to improve the accuracy and stability of the target region positioning.
2. The method according to claim 1, characterized in that, The region constraint information includes any one or more of the following: The bounding box of the image region corresponding to the anatomical location information; Image region mask corresponding to anatomical location information; A weighted map representing the spatial distribution probability of the target region; Spatial constraint parameters generated based on anatomical location information.
3. The method according to claim 1, characterized in that, The semi-supervised learning method includes a training framework for a teacher model and a student model, wherein: The teacher model is used to generate candidate target region localization results for unlabeled samples; The student model is trained based on labeled samples and the unlabeled samples mentioned above; Furthermore, during the training process of the unlabeled samples, the region constraint information is used to screen, weight, or evaluate the credibility of the candidate localization results generated by the teacher model, thereby determining the pseudo-label localization results used to train the student model.
4. The method according to claim 3, characterized in that, The screening, weighting, or credibility assessment of candidate location results is based on at least one of the following factors: Spatial consistency relationship between candidate positioning results and the region constraint information; Prediction confidence of candidate localization results; The degree of consistency between the candidate localization results and the semantic description of the text.
5. The method according to claim 1, characterized in that, The process of extracting anatomical location information from textual description information includes: Natural language processing is performed on the text description information to identify text fragments that represent human anatomical locations, and the text fragments are mapped to predefined anatomical location categories or spatial description information.
6. The method according to claim 1, characterized in that, The visual positioning model includes: The image feature extraction module is used to extract image features from chest X-ray images; The text feature extraction module is used to extract text features from text description information; A multimodal feature fusion module is used to fuse the image features and the text features to generate a joint feature representation; The localization prediction module is used to output the localization result of the target area based on the joint feature representation.
7. The method according to claim 6, characterized in that, The multimodal feature fusion module achieves semantic association between image features and text features through attention mechanisms, correlation modeling mechanisms, or feature weighting mechanisms.
8. The method according to claim 6, characterized in that, The positioning prediction module is used to output target area positioning results in any one or more of the following forms: The bounding box of the target region; Pixel-level region mask for the target area; A probability map representing the spatial distribution of the target region; The location information is obtained by combining the above results.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the visual localization method for chest X-ray images based on semi-supervised learning as described in any one of claims 1 to 8.