Medical record structured and standardized generation method for intelligent analysis of macular diseases

Abstract

The invention discloses a medical record structuring and standardization generation method for intelligent analysis of macular diseases. The method comprises the following steps: acquiring medical report data and a corresponding macular disease OCT (Optical Coherence Tomography) image; performing manual annotation on the medical report data; segmenting and marking the macular disease OCT image; using the marked medical report data to train a MedBERT model; finely adjusting the SAM model by using the segmented and labeled macular disease OCT image; unstructured macular disease OCT image matching texts are collected, and an extended corpus is constructed; using the trained MedBERT model to extract a structured label of the matched text of the macular disease OCT image; according to the trained MedBERT model and the SAM model after fine tuning, an SAMBERT model is constructed; inputting the macular disease OCT image and the structured label into the SAMBERT model, and performing multi-modal alignment training to generate an image label pair; and generating a standardized medical record text according to the image tag pair.

Description

Technical Field

[0001] This invention relates to the field of deep learning applications in medical natural language processing and image segmentation fusion, and particularly to a method for generating structured and standardized medical records for intelligent diagnosis and treatment of macular diseases. Background Technology

[0002] OCT (Optical Computation) is a commonly used ophthalmological examination technique, widely applied in the auxiliary diagnosis of fundus diseases, playing a crucial role in the diagnosis of macular diseases. It acquires detailed images of the retina, aiding doctors in accurate diagnosis. However, the OCT diagnostic process typically involves the analysis of extensive medical images and medical record text. In most medical reports, doctors describe the disease in unstructured natural language text reports of hundreds of words, which is not only time-consuming and labor-intensive, but also susceptible to the influence of the doctor's personal experience and subjective judgment on the diagnostic results. With the continuous development of medical technology and the increasing number of patients, traditional OCT diagnostic methods are struggling to meet the growing medical demands. Therefore, improving the efficiency and accuracy of OCT diagnosis has become an urgent problem to be solved. Summary of the Invention

[0003] This invention provides a method for the structured and standardized generation of medical records for intelligent diagnosis and treatment of macular diseases. It aims to improve the intelligence level of assisted diagnosis based on OCT images of macular diseases and the efficiency of medical record information processing. By integrating deep learning text processing and image segmentation technology, it achieves unified modeling of image and text information and standardized expression of medical record content, thereby effectively realizing the structured and standardized generation of medical record texts related to macular diseases. This improves the accuracy and consistency of OCT-assisted diagnosis and has good clinical application value and potential for integration into intelligent medical systems.

[0004] To achieve the above objectives, the present invention includes the following steps:

[0005] A method for generating structured and standardized medical records for intelligent analysis of macular diseases includes the following steps:

[0006] Step 1: Collect medical report data and corresponding OCT images of macular diseases; manually annotate the medical report data; segment and annotate the OCT images of macular diseases;

[0007] Step 2: Train the MedBERT model using labeled medical report data; fine-tune the SAM model using segmented and labeled OCT images of macular diseases;

[0008] Step 3: Collect unstructured text accompanying OCT images of macular diseases to build an extended corpus; use the trained MedBERT model to extract structured labels from the text accompanying the OCT images of macular diseases.

[0009] Step 4: Construct a SAMBERT model based on the trained MedBERT model and the fine-tuned SAM model; input the macular disease OCT image and the structured labels into the SAMBERT model, perform multimodal alignment training, and generate image label pairs;

[0010] Step 5: Generate standardized medical record text based on the image tags.

[0011] As a preferred option, in step 2, the SAM model is fine-tuned by combining the Dice loss function and the cross-entropy loss function for optimization.

[0012] Preferably, step 4 includes:

[0013] Lesion regions are segmented from OCT images of macular diseases; the lesion regions are encoded to generate region codes; and image features corresponding to each lesion region are extracted.

[0014] Extract text features based on the structured tags;

[0015] The text features are aligned with the image features to generate the image label pairs.

[0016] Preferably, the image features corresponding to each lesion region are extracted as follows:

[0017] For each OCT image I of macular disease, multiple lesion region masks M are obtained using the SAM model. j ∈{0,1} H×W , where j∈{1,2,…,K} represents the j-th region.

[0018] v j =f image (M j ⊙I)

[0019] Among them, f image M is the image encoder, I is the masking operation, and I is the input OCT image of macular disease.

[0020] Preferably, extracting text features based on the structured tags includes:

[0021] For each of the lesion regions r j Construct the corresponding structured label sequence y j This includes medical information such as lesion type, location, and morphology; the structured tag sequence y j Converted into text features using a text encoder:

[0022] t j =f text (y j )

[0023] Among them, f text For text encoders, a structured tag feature library {t1,t2,…,t} is constructed. N}

[0024] Preferably, step 4 also includes:

[0025] Attention calculations are performed on the lesion region using deformable self-attention.

[0026] Preferably, aligning the text features with the image features includes:

[0027] Calculate each of the image features v j With the text feature t k The semantic similarity is measured using cosine similarity:

[0028]

[0029] For each of the lesion regions r j Select the structured tag with the highest similarity:

[0030]

[0031] To obtain the optimal image label pair (v j ,t k ).

[0032] Preferably, step 5 includes:

[0033] The region encoding and the image label are used to standardize the text generated by inputting the T5 model; the region encoding r j The image feature v j The text feature t k The splicing constitutes a fusion representation:

[0034]

[0035] Input into the T5 text generation model:

[0036] x j =GenerateText(z j )

[0037] Output region-level standardized medical record text fragments x j Merging all the standardized medical record text fragments at the region level yields the standardized medical record text:

[0038]

[0039] Compared with the prior art, the beneficial effects of the present invention are mainly reflected in the following aspects:

[0040] 1. Unlike existing OCT diagnostic methods, this invention proposes for the first time to embed a segmentation-guided region numbering mechanism and structured label alignment into a combined image and text modeling framework. This enables the model to accurately generate structured diagnostic text corresponding to specific lesion regions while identifying lesions in OCT images of macular diseases, thereby improving the interpretability, regional traceability, and intelligent generation level of medical record reports.

[0041] 2. Unlike the fixed process of "one-way training" in traditional models, this invention constructs a closed-loop optimization system of image segmentation, structured extraction, multimodal training, and text generation. The generated structured medical record text and segmentation results can not only be used by clinicians, but can also be used in reverse to improve the training data of the extraction model and the multimodal alignment model, so as to achieve continuous optimization and iterative improvement of diagnostic performance.

[0042] 3. Unlike existing methods that target a single disease or a single modality, the three-modal processing framework of image-structured label-text generation proposed in this invention has good versatility and scalability. It can be applied to the diagnosis of other types of retinal diseases and can even be extended to a wider range of medical image-assisted diagnosis and treatment tasks. It has strong cross-scenario transfer capabilities and system-level integration potential. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the method flow of Embodiment 1 of the present invention;

[0044] Figure 2 This is a schematic diagram of the SAMBERT model in Embodiment 1 of the present invention. Detailed Implementation

[0045] To make the technical means, inventive features, objectives, and effects of the invention readily understandable, the invention is further described below with reference to specific illustrations. However, the invention is not limited to the embodiments described below.

[0046] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings of this specification are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0047] Example 1:

[0048] like Figure 1The method for generating structured and standardized medical records for intelligent analysis of macular diseases, as shown, includes the following steps:

[0049] Step 1: Collect medical report data and corresponding OCT images (Optical Coherence Tomography) of macular diseases; manually annotate the medical report data; segment and annotate the OCT images of macular diseases;

[0050] First, medical reports in free text format from multiple hospitals or medical institutions were collected and paired with corresponding labeled OCT images of macular diseases. The collected medical reports were reviewed by a team of ophthalmologists, who manually annotated them with disease-related structured labels, including lesion location, lesion type, and severity. The macular disease OCT images were then normalized, cropped, and segmented to ensure data accuracy and consistency.

[0051] Step 2: Train the MedBERT model (Medical BERT: Biomedical Language Representation Pre-training) using labeled medical report data; fine-tune the SAM model (SegmentAnything Model) using segmented labeled OCT images of macular diseases;

[0052] Based on the annotated structured text data described above, an improved MedBERT model was trained. Cross-validation was used to monitor model performance during training to achieve good semantic understanding and stable information extraction performance. Based on the aforementioned structured corpus, the improved MedBERT model was trained, effectively enhancing its ability to understand fine-grained semantics and implicit diagnostic information in medical texts. The training process iterated until the model achieved stable information extraction performance.

[0053] SAM was fine-tuned using an expert-labeled OCT image segmentation dataset for macular diseases. During fine-tuning, a combination of the Dice loss function and the cross-entropy loss function was used to optimize the model and enhance its accuracy in segmenting macular disease regions, achieving a more suitable segmentation effect for macular diseases.

[0054] Step 3: Collect unstructured text accompanying OCT images of macular diseases to build an expanded corpus; use a trained MedBERT model to extract structured labels from the accompanying text of macular disease OCT images, achieving automatic data expansion and annotation, and generating a macular disease OCT image segmentation dataset. This method significantly improves the size and coverage of the training set.

[0055] Step 4: Construct a SAMBERT model based on the trained MedBERT model and the fine-tuned SAM model; input the macular disease OCT images and structured labels into the SAMBERT model, perform multimodal alignment training, and generate image label pairs;

[0056] like Figure 2 As shown, the segmented macular disease OCT images and their corresponding structured labels are jointly input into the SAMBERT model for multimodal alignment training, specifically including:

[0057] First, lesion regions are segmented from OCT images of macular diseases, and these segmented lesion regions are encoded. Candidate lesion regions are then selected using the segmented OCT images. Because the segmented regions are often irregular, deformable self-attention is used to perform attention calculations only on the segmented regions, saving the global computational burden on normal regions and reducing interference from the introduction of normal region features. Image features corresponding to each candidate lesion region are then extracted using the SAM segmentation model.

[0058] Image feature extraction:

[0059] For each OCT image I of macular disease, multiple lesion region masks M have been obtained through a segmentation model. j ∈{0,1} H×W , where j∈{1,2,…,K} represents the j-th region.

[0060] v j =f image (M j ⊙I)

[0061] Where f image M is the image encoder, I is the masking operation, and I is the input OCT image of macular disease.

[0062] Simultaneously, structured labels are input into the SAMBERT model to extract text features based on the structured labels.

[0063] Structured tag encoding:

[0064] For each region r j Construct the corresponding structured label sequence y j This includes medical information such as lesion type, location, and morphology. The tag sequence is converted into a semantic vector using a text encoder.

[0065] t j =f text (y j )

[0066] Where f textFor the text encoder, construct a structured label feature library {t1,t2,…,t}. N}

[0067] The text features generated by BERT encoding of the structured labels are aligned with the image features calculated by attention using cosine similarity to generate image label pairs:

[0068] Calculate the feature v of each image region j With all structured label features t k The semantic similarity is measured using cosine similarity:

[0069]

[0070] For each image region r j Select the structured tags with the highest similarity:

[0071]

[0072] Obtain the optimal matching pair of region and label (v j ,t k ).

[0073] The SAMBERT model, based on the SAM segmentation model and the BERT encoding model, extracts features from key segmented regions in OCT images of macular diseases and incorporates the location information of the segmented regions. When aligning images and text, it performs alignment at the key region level rather than the entire image and the entire report, making the alignment more accurate.

[0074] Step 5: Generate standardized medical record text based on image tags.

[0075] The core objective of the standardized medical record text generation module is to generate semantically clear and content-standardized medical record text based on image segmentation regions and corresponding structured label information.

[0076] The segmented region numbers and image labels are fed into a T5 model (Text-to-Text Transfer Transformer) for standardized text generation. In the output medical record, the region numbers correspond one-to-one with the lesion locations, facilitating doctors' quick location and understanding of diagnostic points. The final result is a standardized medical record text with a consistent format, standardized content, and clear semantics, providing a one-to-one correspondence between the diagnostic content and the segmented image regions.

[0077] Code the region r j Image features v j Matching tag t k The splicing constitutes a fusion representation:

[0078]

[0079] Input into the T5 text generation model:

[0080] x j =GenerateText(z j )

[0081] Output region-level standardized medical record text fragments x j And merge all the regional texts to obtain the complete standardized medical record text:

[0082]

Claims

1. A method for generating structured and standardized medical records for intelligent analysis of macular diseases, characterized in that, Includes the following steps: Step 1: Collect medical report data and corresponding OCT images of macular diseases; manually annotate the medical report data; segment and annotate the OCT images of macular diseases; Step 2: Train the MedBERT model using labeled medical report data; fine-tune the SAM model using segmented and labeled OCT images of macular diseases; Step 3: Collect unstructured text accompanying OCT images of macular diseases to build an extended corpus; use the trained MedBERT model to extract structured labels from the text accompanying the OCT images of macular diseases. Step 4: Construct a SAMBERT model based on the trained MedBERT model and the fine-tuned SAM model; input the macular disease OCT image and the structured labels into the SAMBERT model, perform multimodal alignment training, and generate image label pairs; Step 5: Generate standardized medical record text based on the image tags.

2. The method for generating structured and standardized medical records for intelligent analysis of macular diseases according to claim 1, characterized in that, In step 2, the SAM model is fine-tuned by combining the Dice loss function and the cross-entropy loss function for optimization.

3. The method for generating structured and standardized medical records for intelligent analysis of macular diseases according to claim 1, characterized in that, Step 4 includes: Lesion regions are segmented from OCT images of macular diseases; the lesion regions are encoded to generate region codes; and image features corresponding to each lesion region are extracted. Extract text features based on the structured tags; The text features are aligned with the image features to generate the image label pairs.

4. The method for generating structured and standardized medical records for intelligent analysis of macular diseases according to claim 3, characterized in that, Extracting the image features corresponding to each of the lesion regions includes: For each OCT image I of macular disease, multiple lesion region masks M are obtained using the SAM model. j ∈{0,1} H×W , where j∈{1,2,…,K} represents the j-th region. v j =f image (M j ⊙I) Among them, f image M is the image encoder, I is the masking operation, and I is the input OCT image of macular disease.

5. The method for generating structured and standardized medical records for intelligent analysis of macular diseases according to claim 3, characterized in that, Extracting text features based on the structured tags includes: For each of the lesion regions r j Construct the corresponding structured label sequence y j This includes medical information such as lesion type, location, and morphology; the structured tag sequence y j Converted into text features using a text encoder: t j =f text (y j ) Among them, f text For text encoders, a structured tag feature library {t1,t2,…,t} is constructed. N } 6. The method for generating structured and standardized medical records for intelligent analysis of macular diseases according to claim 3, characterized in that, Step 4 also includes: Attention calculations are performed on the lesion region using deformable self-attention.

7. The method for generating structured and standardized medical records for intelligent analysis of macular diseases according to claim 3, characterized in that, Aligning the text features with the image features includes: Calculate each of the image features v j With the text feature t k The semantic similarity is measured using cosine similarity: For each of the lesion regions r j Select the structured tag with the highest similarity: To obtain the optimal image label pair (v j ,t k ).

8. The method for generating structured and standardized medical records for intelligent analysis of macular diseases according to claim 3, characterized in that, Step 5 includes: The region encoding and the image label are used to standardize the text generated by inputting the T5 text generation model; The region code r j The image feature v j The text feature t k The splicing constitutes a fusion representation: Input into the T5 text generation model: x j =GenerateText(z j ) Output region-level standardized medical record text fragments x j Merging all the standardized medical record text fragments at the region level yields the standardized medical record text:

Images