An image processing system, method and storage medium

Abstract

The application discloses an image processing system, method and storage medium, and the method comprises the following steps: acquiring an MRI image of a historical patient's ear and a corresponding medical report; generating left ear and right ear image features according to the MRI image coding; constructing an image reconstruction loss according to the left ear and right ear image features; constructing an image contrast learning loss according to anchor point features extracted from the MRI image; determining a specific severity score according to the medical report, performing cross-modal fusion and alignment between the image and text features extracted from the medical report according to the specific severity score, wherein a cross-modal contrast loss is constructed when the cross-modal alignment is performed; constructing an objective function according to each loss, so as to pre-train a diagnosis model through the objective function; and inputting an MRI image of a current patient's ear and a corresponding medical report into the pre-trained diagnosis model to output a Meniere's disease diagnosis result. The application can realize accurate and efficient diagnosis of Meniere's disease through image processing.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to an image processing system, method, and storage medium. Background Technology

[0002] Meniere's disease (MD) is a multifactorial inner ear disorder characterized by recurrent episodes of vertigo and fluctuating sensorineural hearing loss, with endolymphatic hydrops (EH) widely considered its primary pathological feature. While intravenous gadolinium-enhanced MRI (magnetic resonance imaging) has become a key technique for visualizing inner ear lesions, it still faces significant limitations and challenges in routine clinical practice. First, inner ear lesions in early or mild MD are often small and localized, making image resolution and partial volume effects limiting factors and potentially leading to underdiagnosis. Second, although inner ear lesions are a typical pathological feature of MD, similar imaging patterns can occur in other inner ear diseases, potentially affecting diagnostic accuracy. Furthermore, traditional diagnostic methods often overlook subtle pathological clues in the cochlea and vestibular structures, partly because these methods struggle to integrate complementary information from MRI images and radiological reports. Therefore, relying solely on imaging parameters is often insufficient for accurate and efficient diagnosis of MD. Summary of the Invention

[0003] This invention aims to at least partially address the technical problems in related art. Therefore, a first objective of this invention is to provide an image processing method that enables accurate and efficient diagnosis of Meniere's disease by integrating complementary information from MRI images and radiological reports.

[0004] A second objective of this invention is to provide an image processing system.

[0005] A third objective of this invention is to provide a computer-readable storage medium.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0007] An image processing method, comprising:

[0008] Obtain historical MRI images of patients' ears and corresponding medical reports;

[0009] The region of interest (ROI) on the MRI image is labeled, and image patches of the ROI are extracted. The extracted image patches are then encoded using an encoder to generate left ear image features and right ear image features.

[0010] An image reconstruction loss is constructed based on the features of the left and right ear images, and image reconstruction is performed to ensure that no lesion information is lost in the encoded image features;

[0011] An image contrast learning loss is constructed based on anchor point features extracted from MRI images, and image contrast learning is performed to enable the diagnostic model to identify lesions and normal individual variations.

[0012] The specific severity scores of the left and right ears are determined based on the medical report. Cross-modal fusion and alignment between the image and text features extracted from the medical report are performed based on the specific severity scores of the left and right ears. A cross-modal contrast loss is constructed when performing cross-modal alignment.

[0013] An objective function is constructed based on image reconstruction loss, image contrast learning loss, and cross-modal contrast loss, so as to pre-train the diagnostic model using the objective function;

[0014] The MRI image of the patient's ear and the corresponding medical report are input into the pre-trained diagnostic model. The linear classifier in the diagnostic model classifies and diagnoses the fused bilateral image features corresponding to the MRI image of the patient's ear, and outputs the diagnosis result of Meniere's disease.

[0015] In one possible implementation, an image reconstruction loss is constructed based on features from the left and right ear images, and image reconstruction is performed, including:

[0016] Local image patches were reconstructed from the features of the left and right ear images using a linear transformation function, respectively.

[0017] A mean squared error loss function is constructed to measure whether the difference between the reconstructed local image patch and the corresponding original image patch is minimized, so that the encoded image features do not lose lesion information; wherein, the mean squared error loss function is expressed as the sum of squares of the differences between each pixel value between the reconstructed local image patch and the corresponding original image patch.

[0018] In one possible implementation, an image contrast learning loss is constructed based on anchor point features extracted from MRI images, and image contrast learning is performed, including:

[0019] Positive and negative samples are determined based on anchor point features extracted from MRI images. Among them, samples whose spatial location from the anchor point feature is within a preset neighborhood and whose semantic similarity to the anchor point feature is greater than a preset value are classified as positive samples, while samples whose spatial location from the anchor point feature is not within the preset neighborhood and whose semantic similarity to the anchor point feature is less than or equal to a preset value are classified as negative samples.

[0020] Construct an image contrast learning loss based on anchor point features, positive samples, and negative samples;

[0021] The encoder parameters are adjusted to minimize the image contrast learning loss, resulting in an encoder with the minimum image contrast learning loss. The left and right ear image features encoded by the encoder with the minimum image contrast learning loss can identify lesion features or normal individual variation features.

[0022] In one possible implementation, specific severity scores for the left and right ears are determined based on medical reports, including:

[0023] Identify entities in the medical report, the entities being key clinical entities, the entities including anatomical structures, pathological descriptions, and lateral indications, the lateral indications being left, right, and bilateral;

[0024] Calculate the similarity between an entity and the global context in a medical report, so as to determine the context weight of each entity based on the similarity and the presence of negative words in the medical report;

[0025] Determine the basic weight of each entity, determine whether there are severe, moderate, and mild keywords in the medical report, and determine the severity level adjustment coefficient based on the keyword occurrence results;

[0026] The final weight value for each entity is determined based on its base weight, context weight, and severity level adjustment factor, so as to determine the specific severity score for the left and right ears based on the base weight and final weight value of each entity.

[0027] In one possible implementation, after determining the specific severity scores for the left and right ears, the method further includes: determining whether there is at least one bilateral entity in the medical report; if so, increasing the specific severity scores for both the left and right ears by a preset percentage.

[0028] In one possible implementation, cross-modal fusion between images and text features extracted from medical reports is performed based on left and right ear-specific severity scores, including:

[0029] Text features are extracted from medical reports, and left ear image features, right ear image features, and text features are projected into the same shared semantic space.

[0030] The semantic correlation between local image patches and text in the medical report is calculated based on the projected features of the left ear image features, the right ear image features, and the text features. Then, the left ear multi-head attention output and the right ear multi-head attention output are obtained by weighted summation based on the semantic correlation and the corresponding text.

[0031] Based on the specific severity scores of the left and right ears, the multi-head attention output of the left and right ears was modulated to focus on the more severe areas of the lesion, and the final multi-head attention output of the left and right ears was obtained.

[0032] Feature fusion is performed on the final multi-head attention outputs of the left and right ears to obtain fused bilateral image features.

[0033] In one possible implementation, feature fusion is performed on the final left-ear multi-head attention output and the right-ear multi-head attention output to obtain fused bilateral image features, including:

[0034] The preliminary fused features were obtained by weighting and summing the final multi-head attention outputs of the left and right ears based on the specific severity scores of the left and right ears, respectively.

[0035] Asymmetry factors were determined based on specific severity scores for the left and right ears;

[0036] Enhanced features after initial fusion based on asymmetric factors;

[0037] The enhanced initial fused features are subjected to residual connection and layer normalization to obtain the final fused bilateral image features.

[0038] In one possible implementation, performing cross-modal alignment includes:

[0039] The final fused bilateral image features and text features are normalized, and the cosine similarity matrix between the two is determined after normalization.

[0040] A dynamic temperature coefficient is determined based on the cosine similarity matrix of the two ears and the specific severity scores of the left and right ears.

[0041] Determine the positive similarity scores between the final fused bilateral image features and their corresponding text features and other text features;

[0042] A cross-modal contrast loss is constructed based on two positive similarity scores and a dynamic temperature coefficient to guide cross-modal alignment of images and text, ensuring image and text matching. The image is the final fused two-sided image.

[0043] The objective function, constructed based on cross-modal contrast loss, image reconstruction loss, and image contrast learning loss, is expressed as follows:

[0044]

[0045] in, Describe the objective function. These represent the image reconstruction losses for the left and right ears, respectively. These represent the image contrast learning losses for the left and right ears, respectively. λ represents the cross-modal contrast loss. cl λ represents the coefficient used to balance the image reconstruction loss and the image contrast learning loss. gl This represents the coefficient used to control the global alignment loss.

[0046] To achieve the above objectives, a second aspect of the present invention provides an image processing system, comprising:

[0047] The acquisition module is used to acquire historical MRI images of patients' ears and corresponding medical reports;

[0048] The encoding module is used to annotate the region of interest on the MRI image and extract image patches of the region of interest. The encoder encodes the extracted image patches to generate left ear image features and right ear image features.

[0049] The image reconstruction module is used to construct an image reconstruction loss based on the features of the left ear image and the right ear image, and to perform image reconstruction so that the encoded image features do not lose lesion information;

[0050] The contrast learning module is used to construct an image contrast learning loss based on anchor point features extracted from MRI images and to perform image contrast learning so that the diagnostic model can identify lesions and normal individual variations.

[0051] The scoring module is used to determine the specific severity scores of the left and right ears based on the medical report, and to perform cross-modal fusion and alignment between the image and text features extracted from the medical report based on the specific severity scores of the left and right ears, wherein a cross-modal contrast loss is constructed when performing cross-modal alignment;

[0052] A pre-training module is used to construct an objective function based on image reconstruction loss, image contrast learning loss, and cross-modal contrast loss, so as to pre-train the diagnostic model using the objective function;

[0053] The diagnostic module is used to input the current patient's ear MRI image and corresponding medical report into the pre-trained diagnostic model. The linear classifier in the diagnostic model classifies and diagnoses the fused bilateral image features corresponding to the current patient's ear MRI image to output the diagnosis result of Meniere's disease.

[0054] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described above.

[0055] This invention has at least the following technical effects:

[0056] This invention provides a diagnostic method and corresponding image processing system for Meniere's disease using image processing. The method first encodes each MRI pixel and its local neighborhood into a high-dimensional embedding space within a clinically defined region of interest, thereby capturing subtle structural changes. Secondly, in the self-supervised feature refinement stage of the image alone, a dual mechanism of image reconstruction and image contrastive learning is employed. By reconstructing neighboring pixels, the spatial dependence of fine inner ear structures is preserved; by comparing the distribution of lesions with normal tissue, sensitivity to minor abnormalities is improved. Finally, the global lesion representations of both ears are aggregated and aligned with the overall textual report description in a shared semantic space, ensuring patient-level semantic consistency, preserving local lesion details, and enhancing the clinical interpretability of multimodal fusion. This overall approach enables accurate and efficient diagnosis of Meniere's disease.

[0057] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0058] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram illustrating the working principle of the image processing method according to an embodiment of the present invention.

[0060] Figure 3 These are the ROC curves and probability distribution diagrams of embodiments of the present invention.

[0061] Figure 4 This is a structural block diagram of the image processing system according to an embodiment of the present invention. Detailed Implementation

[0062] The following describes this embodiment in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the invention, and should not be construed as limiting the invention.

[0063] An image processing system, method, and storage medium according to this embodiment are described below with reference to the accompanying drawings.

[0064] In terms of imaging, the image processing method provided in this embodiment assists expert radiologists in identifying regions of interest associated with Meniere's disease and encodes each pixel and its local neighborhood into a high-dimensional embedding space to preserve subtle features of the lesion. In terms of text, the scheme also involves a severity assessment method that extracts descriptions of bilateral lesions from standardized imaging reports, i.e., medical or radiological reports, thereby quantifying bilateral pathological differences and enhancing asymmetric modeling. Secondly, MA (a self-supervised learning method) does not rely on image-level enhancements found in many contrastive learning methods, but instead utilizes intra-image region learning. This design focuses on learning small-scale local features rather than invariance to global transformations—which is particularly important for medical images that are globally homogeneous but locally heterogeneous. Furthermore, this invention introduces a severity-aware global cross-modal alignment mechanism to connect local lesion features with patient-level clinical semantics. This mechanism constructs a differentiable shared semantic space by integrating global information, ensuring the preservation of local pathological details by providing bilateral lesion images with complete descriptions, and achieving semantic alignment between image and text modalities, thereby improving the interpretability and consistency of multimodal diagnostic decisions.

[0065] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0066] Step S101: Obtain MRI images of the patient's ear and the corresponding medical reports.

[0067] Specifically, this study included 524 hospitalized patients diagnosed with Meniere's disease at the Vertigo and Hearing Center of a university-affiliated hospital between January 2021 and December 2024. The cohort consisted of 199 male and 325 female patients, aged 12 to 78 years (mean age ± standard deviation: 50.1 ± 11.9 years), with the age of onset also ranging from 12 to 78 years (mean age ± standard deviation: 46.1 ± 16.3 years). All patients presented with typical symptoms of Meniere's disease, including vertigo and ear symptoms, specifically low- to mid-frequency sensorineural hearing loss, with or without tinnitus and / or ear fullness. The diagnosis of endolymphatic hydrops (EH) was confirmed by inner ear imaging. Patients with other diagnoses, such as vestibular migraine, sudden sensorineural hearing loss, central vertigo, or benign paroxysmal positional vertigo, were excluded. All participants received intravenous gadolinium-enhanced inner ear imaging and 3D-FLAIR MRI (three-dimensional liquid attenuation inversion recovery MRI) and provided written informed consent. The study was approved by the medical ethics committee of the university hospital.

[0068] All patients underwent 3D-FLAIR MRI of the inner ear approximately four hours after intravenous administration of gadopentetate dimeglumine. Imaging data were independently evaluated by two senior radiologists unaware of the clinical information, focusing on endolymphatic hydrops within the vestibule and cochlear membranous labyrinth. Most patients presented with varying degrees of endolymphatic hydrops. Following a clinical diagnostic procedure, radiologists manually marked 32×32 pixel regions of interest in the anatomical areas most relevant to typical Meniere's disease, such as cochlear hydrops and vestibular dilatation. These marked regions encompassed clinically predisposed substructures, such as the basal cochlear turn and the lateral portion of the vestibule, rather than strictly adhering to precise anatomical boundaries, to ensure the captured areas had practical diagnostic value.

[0069] In this study, MRI slices of patients diagnosed with Meniere's disease and their corresponding standardized imaging reports (medical reports) were used to investigate the relationship between radiological features and clinical symptoms. Furthermore, a self-supervised image-text pre-training framework was developed, in which the diagnostic model aims to learn multimodal features to improve the automatic analysis and differentiation of MD-related pathological features, thereby enhancing its clinical relevance and interpretability.

[0070] Step S102: Mark the region of interest on the MRI image and extract the image patch of the region of interest. Encode the extracted image patch by an encoder to generate left ear image features and right ear image features.

[0071] In one possible implementation, the region of interest (ROI) on the MRI image is labeled, and an image patch of the ROI is extracted, including: manually labeling the ROI on the MRI image with a bounding box of a preset pixel size, wherein the ROI includes the cochlear basal corner and the lateral vestibular region; and extracting an image patch of a preset pixel size with the center point of the ear as the center.

[0072] Specifically, the ROI (Region of Interest) detection and encoding process based on radiologist annotations is outlined to ensure that these annotations are used only to depict local structural regions, rather than directly providing diagnostic information. Two experienced radiologists independently annotated bounding boxes using anonymized patient data. Since the clinical diagnosis of Meniere's disease typically emphasizes specific anatomical regions, a fixed 32×32 pixel bounding box was used to define ROIs, such as the cochlear basal turn and the lateral vestibular region. This method captures fine-grained structural and textural details crucial for subsequent analysis. After determining the bounding boxes, the x and y coordinates of the center of each ear were calculated as xcenter = x + w / 2 and ycenter = y + h / 2, respectively, where (x, y, w, h) represent the x, y, width, and height of the top-left corner of the bounding box, respectively. Next, a 32×32 pixel patch (i.e., an image block) is extracted from the center position (xcenter, ycenter), with its top-left corner at (xcenter-16, ycenter-16) and its bottom-right corner at (xcenter+16, ycenter+16). If the patch extends beyond the image boundary, zero padding is applied to ensure consistency across all extracted patches.

[0073] After ROI detection, such as Figure 2 As shown, a convolutional neural network (CNN encoder) is used as a patch encoder to map 32×32 image patches into a latent embedding space, generating left ear image features and right ear image features, denoted as zl and zr, respectively. In this embodiment, the encoder includes convolutional layers with batch normalization and non-linear activation functions, but does not perform pooling operations to maintain spatial consistency between the input image and the final feature map. This design effectively captures local structural details, allowing subsequent models to utilize these features for analysis while preserving spatial relationships.

[0074] It should be noted that the encoder encodes image patches extracted from the MRI images of the left and right ears, and then generates image features for the left and right ears.

[0075] Step S103: Construct an image reconstruction loss based on the features of the left ear image and the right ear image, and perform image reconstruction so that the encoded image features do not lose lesion information.

[0076] In one possible implementation, an image reconstruction loss is constructed based on the features of the left ear image and the right ear image, and image reconstruction is performed, including: reconstructing corresponding local image patches based on the features of the left ear image and the right ear image respectively through a linear transformation function; constructing a mean squared error loss function to measure whether the difference between the reconstructed local image patch and the corresponding original image patch is minimized, so that the encoded image features do not lose lesion information; wherein, the mean squared error loss function is expressed as the sum of squares of the differences between each pixel value between the reconstructed local image patch and the corresponding original image patch.

[0077] This embodiment introduces a local patch, or local image patch reconstruction constraint, to enhance the representation capability of small-scale image features. Considering that spatial dependencies between adjacent elements within a 32×32 pixel region are easily lost, an image reconstruction loss centered on pixel-level patches is designed. This constraint ensures that the encoder preserves local structural features in the latent embedding, thereby better maintaining fine-grained contextual information.

[0078] To enhance the representation of small-scale medical image regions, a local patch reconstruction constraint mechanism is introduced. This mechanism addresses the loss of spatial dependencies between adjacent elements in a 32×32 pixel region. By reconstructing pixel-centered patches, this mechanism encourages the encoder to preserve the structural features of local regions in the latent embedding space. Specifically, for lesion regions of the left and right ears, a local patch is constructed around each central pixel. The corresponding embedding vector is represented as z. ij By using the recon function, z ij Map back to the original pixel space. The reconstruction loss is defined as: P ij f represents the pixel features corresponding to the original patch. recon (z ij ) represents a linear projection layer. Where, f recon (z ij )=Wz ij +b, W, and b represent the learnable weights and biases, respectively. This image reconstruction mechanism ensures that the embedding vector preserves the local features of each patch and its surrounding neighborhood by minimizing the reconstruction error. In this embodiment, the encoder adopts a fully convolutional architecture without pooling operations, which maintains the spatial alignment between the input and output feature maps and avoids the loss of structural information due to downsampling. Therefore, the local structural reconstruction losses for the left and right ears are denoted as follows: and

[0079] Step S104: Construct an image contrast learning loss based on the anchor point features extracted from the MRI images, and perform image contrast learning so that the diagnostic model can identify lesions and normal individual variations.

[0080] In one possible implementation, an image contrast learning loss is constructed based on anchor features extracted from MRI images, and image contrast learning is performed, including: determining positive and negative samples based on anchor features extracted from MRI images; wherein, samples whose spatial location from the anchor feature is within a preset neighborhood and whose semantic similarity to the anchor feature is greater than a preset value are classified as positive samples, and samples whose spatial location from the anchor feature is not within the preset neighborhood and whose semantic similarity to the anchor feature is less than or equal to a preset value are classified as negative samples; constructing an image contrast learning loss based on anchor features, positive samples, and negative samples; adjusting encoder parameters to minimize the image contrast learning loss, and obtaining an encoder with the minimum image contrast learning loss; wherein, the left ear image features and right ear image features encoded by the encoder with the minimum image contrast learning loss can identify lesion features or normal individual variation features.

[0081] Specifically, the proposed contrastive learning framework enhances the diagnostic model's ability to identify local pathological features of Meniere's disease by imposing dual constraints of spatial proximity and deep semantic consistency. For anchor patches centered at coordinates (i1, j1)... Embedded features are extracted by the encoder as When constructing the positive sample set, strict spatial and semantic conditions must be met: the geometric center of the positive sample must be located within the anchor point's neighborhood N(i1, j1) = {(I1′, j1′)||i1-i1′|≤p, |j1-j1′|≤p}, and its pathological similarity to the anchor point must be verified using the Deep Image Structural Similarity (DISTS) metric, using δ DISTS A threshold of 0.5 is set, where i1 and j1 are the x and y coordinates of the anchor patch, i1′ and j1′ are the x and y coordinates of other samples, p is the preset neighborhood value, and δ is the threshold value. DISTS This is a preset value. DISTS is based on multi-scale feature alignment of a pre-trained network, which is highly sensitive to subtle pathological changes such as endolymphatic spatial expansion, and outperforms traditional metrics in modeling local deformations.

[0082] The construction of negative samples follows a constraint complementary to that of positive samples: candidate patches must be located outside the anchoring neighborhood. When N(i,j) or the similarity of DISTS is lower than the preset value In these cases, negative samples typically contain anatomical variations, such as physiological narrowing of the cochlear aqueduct or irrelevant spatial deviations. This represents a candidate patch. By imposing symmetry constraints between positive and negative samples, the diagnostic model is forced to distinguish between local pathological features (lesions) and normal anatomical variations (normal individual variations), thereby improving the clinical interpretability of the embedding space. The image contrast learning loss is defined as:

[0083]

[0084] in, Z represents the image contrast learning loss. + Z represents a positive sample. - Ω represents a negative sample. + Ω represents the set of positive samples. - Let represent the set of negative samples, e be the natural constant, sim() represent the cosine similarity, and τ represent the temperature coefficient.

[0085] Step S105: Determine the specific severity scores of the left and right ears based on the medical report, and perform cross-modal fusion and alignment between the image and the text features extracted from the medical report based on the specific severity scores of the left and right ears, wherein a cross-modal contrast loss is constructed when performing cross-modal alignment.

[0086] In one possible implementation, the specific severity scores for the left and right ears are determined based on a medical report, including: identifying entities in the medical report, where entities are key clinical entities, including anatomical structures, pathological descriptions, and lateral indications, which are left, right, and bilateral; calculating the similarity between the entity and the global context in the medical report to determine the contextual weight of each entity based on the similarity and the presence of negative words in the medical report; determining the base weight of each entity, determining the presence of severe, moderate, and mild keywords in the medical report, and determining a severity level adjustment coefficient based on the keyword occurrence results; and determining the final weight value of each entity based on its base weight, contextual weight, and severity level adjustment coefficient to determine the specific severity scores for the left and right ears based on the base weight and final weight value of each entity.

[0087] While standardized radiological reports (i.e., medical reports) for Meniere's disease ensure consistency in description, they often fail to quantitatively capture individual differences in lesion severity and asymmetry between the left and right ears. To address this issue, this embodiment proposes an interpretable severity scoring mechanism that extracts different pathological burdens from homogeneous textual descriptions. This facilitates fine-grained semantic guidance for cross-modal alignment of images and text and enhances the model's ability to distinguish between unilateral and bilateral abnormalities.

[0088] Specifically, a named BioBERT-CRF entity recognition model accurately identifies key clinical entities in reports, including anatomical structures (such as the cochlea and vestibule), pathological descriptions (such as edema and dilatation), and lateral indications (such as left, right, and bilateral). Based on annotation guidelines developed by two senior ENT physicians, a severity vocabulary K is assigned a base weight w to each entity. b(e) The baseline weights strictly follow the core pathological indicators defined in the American Academy of Otolaryngology-HNS (AAO-HNS) clinical grading criteria.

[0089] To capture the dynamic influence of diagnostic context on pathological features, this embodiment designs a context-aware weighted module based on BERT (a pre-trained language model). For entity e, its dynamic context weight w c (e) The calculation formula is as follows:

[0090]

[0091] Where sim(e,c) represents entity embedding t e With global context embedding t c The cosine similarity between the entity embeddings and the global context embeddings. The ReLU function (activation function) suppresses negative correlations by ReLU(x) = max(0, x), where x represents the cosine similarity between the entity embedding and the global context embedding. For example, if "edema" co-occurs with the highly relevant term "significant cochlear expansion" (e.g., sim(e,c) = 0.9), its context weight will increase to w. c (e) = 1.9.

[0092] Furthermore, a severity level adjustment factor is introduced, defined as:

[0093] γ(l)=1.2×δ l (severe) + 1.0 × δ l (moderate) + 0.8 × δ l (mild) (3)

[0094] Where γ(l) represents the severity level adjustment factor, δ l (severe), δ l (moderate), δ l (mild) represents the parameters indicating whether there are heavy, moderate, and light keywords, respectively.

[0095] Therefore, the final weight value w of the entity t (e) The calculation formula is: w t (e)=w b (e)×w c (e)×γ(l). To simulate the spatial distribution of lesions in Meniere's disease, this embodiment employs anatomical co-occurrence rules. For example, when an anatomical region (e.g., the cochlea) and a pathological description (e.g., edema) co-occur in the same semantic unit, the correlation coefficient is set to δ(e) = 1.0. The lateral specificity severity score is calculated as follows:

[0096]

[0097] Among them, E side This represents the set of entities related to the side. This represents the set of anatomical sites on the corresponding side.

[0098] For bilateral expression (e.g., bilateral vestibular dilatation), based on anatomical studies, it can be treated according to E. side ←E side ∪(E B The update rule (×0.7) distributes 70% of the weight evenly to both ears. B This indicates the set of bilateral anatomical entities that appear in the report.

[0099] Finally, through bilateral enhancement factor I bilateral (1 if a bilateral entity exists, 0 otherwise) The standardized left and right scores are adjusted to determine if at least one bilateral entity exists in the medical report. If so, the specific severity scores for both the left and right ears are increased by a preset percentage, such as 30%, as shown below:

[0100] S l =S left ×(1+0.3×I bilateral ),S r =S right ×(1+0.3×I bilateral (5)

[0101] Among them, S l S r The specific severity scores for the left and right ears are S, respectively. left S right The initial specific severity scores for the left and right ears are determined based on the base weight and final weight value of each entity, respectively.

[0102] In one possible implementation, cross-modal fusion between image features and text features extracted from a medical report is performed based on the specific severity scores of the left and right ears. This includes: extracting text features from the medical report and projecting the left ear image features, right ear image features, and text features into the same shared semantic space; calculating the semantic correlation between local image patches and texts in the medical report based on the projected features of the left ear image features, right ear image features, and text features, so as to obtain the left ear multi-head attention output and the right ear multi-head attention output by weighted summation based on the semantic correlation and the corresponding text; modulating the severity of the multi-head attention output of the left and right ears based on the specific severity scores of the left and right ears respectively to focus on areas with more severe lesions, and obtaining the final left ear multi-head attention output and the right ear multi-head attention output; and fusing the final left ear multi-head attention output and the right ear multi-head attention output to obtain fused bilateral image features.

[0103] In one possible implementation, feature fusion is performed on the final left-ear multi-head attention output and the right-ear multi-head attention output to obtain fused bilateral image features. This includes: weighting and summing the final left-ear multi-head attention output and the right-ear multi-head attention output based on the specificity severity scores of the left and right ears to obtain preliminary fused features; determining an asymmetry factor based on the specificity severity scores of the left and right ears; enhancing the preliminary fused features based on the asymmetry factor; and performing residual connection and layer normalization processing on the enhanced preliminary fused features to obtain the final fused bilateral image features.

[0104] Specifically, this embodiment proposes a severity-aware binaural cross-attention fusion mechanism to simulate the heterogeneity of unilateral and bilateral Meniere's disease lesions. This mechanism dynamically integrates imaging features from both ears and their corresponding textual descriptions through cross-modal attention, thereby extracting discriminative representations for subsequent semantic alignment. For unilateral imaging features (left ear image feature zl or right ear image feature zr) and textual features T, these two modalities are first projected into a shared latent space, i.e., a shared semantic space. Cross-modal interaction is then computed through a multi-head attention mechanism:

[0105] Q = z l / r W Q K = TW K V = TW V

[0106]

[0107] Where Q, K, and V represent the target to be retrieved, the identifier stored in the database or memory, and the actual stored information content, respectively. l / rLet T represent the features of the left ear image or the right ear image, and W represent the text features. Q W K and W V These represent the first to third learnable projection matrices, respectively, and d represents the hidden dimension. Representing semantic relevance, Softmax is a mathematical function that transforms a set of arbitrary real numbers into a probability distribution between 0 and 1, with the sum of all outputs being 1. The generated multi-head attention output, Attention(Q,K,V), abbreviated as A, captures the semantic relevance between the local imaging region and the clinical description. To highlight key pathological areas, specific severity scores for the left and right ears are used. l and s r Severity modulation of multi-head attention output in the left and right ears:

[0108] A′ l / r =A l / r ☉log(1+s l / r (7)

[0109] Among them, A l / r S represents the multi-head attention output of either the left or right ear. l / r A′ indicates the specific severity score for the left or right ear. l / r This indicates the final multi-head attention output from either the left or right ear.

[0110] Here, the log(1+s) formula smooths the contribution of severity, avoiding the dominance of extreme values ​​while maintaining the enhancement effect on clinically significant regions, where s is the specific severity score. Subsequently, based on their respective severity ratios, the final multi-head attention output A for the left and right ears is calculated. l 'and A r 'Adaptively fused:'

[0111]

[0112] Among them, a l a r Z represents the weighting factors for the left and right ears, respectively. ε represents a very small positive number, its function being to prevent the denominator from being zero and to avoid numerical instability. Fusion1 These are characteristics after initial fusion.

[0113] To quantify the difference between the two ears, an asymmetric factor Δs = |s| is introduced. l -s r This asymmetric factor is used to enhance the features after initial fusion, adjusting the fused representation as follows:

[0114] z Fusion =z☉(1+Δs) (9)

[0115] This adjustment improves sensitivity to predominantly unilateral lesions (e.g., severe right ear effusion with a normal left ear), aligning with clinical concerns about asymmetric presentations. Finally, training stability is enhanced by applying residual connectivity and layer normalization.

[0116] z' Fusion =LayerNorm(z Fusion +Dropout(A′ l / r (10)

[0117] Among them, z' Fusion For the final fused bilateral image features, Dropout(A′) l / r ) represents a regularization function, indicating that the Dropout operation is applied to the final multi-head attention output of the left or right ear to obtain a feature vector that has been randomly masked. LayerNorm represents the layer normalization function.

[0118] In this embodiment, the severity scoring-guided attention modulation and anatomical information fusion strategy can extract structured pathological semantics from standardized text reports, i.e., medical reports, and provide a quantitative representation of the spatial distribution of lesions, thereby supporting interpretable cross-modal diagnostic modeling.

[0119] In one possible implementation, performing cross-modal alignment includes: normalizing the final fused bilateral image features and text features, and determining their cosine similarity matrix after normalization; determining a dynamic temperature coefficient based on the cosine similarity matrix and the specificity severity scores of the left and right ears; determining the positive similarity scores between the final fused bilateral image features and their corresponding text features and other text features; and constructing a cross-modal contrast loss based on the two positive similarity scores and the dynamic temperature coefficient to guide the cross-modal alignment of the image and text, ensuring image and text matching, with the image being the final fused bilateral image.

[0120] While local feature modeling (such as intra-image patch reconstruction and image contrast learning) can effectively capture subtle pathological clues in regions such as the cochlea and vestibule, the final diagnosis of Meniere's disease still highly depends on global semantic consistency across multiple modalities. To address this issue, this embodiment proposes a cross-modal contrastive alignment strategy based on severity scoring. This method achieves effective semantic alignment of imaging and text features within a shared latent space by dynamically adjusting the strength of contrast constraints.

[0121] To adjust the temperature parameter for image contrast loss and enhance the similarity between opposite images, a severity score for bilateral endolymphatic hydrops was introduced for specific patients (score for left ear). lThe right ear is s r The asymmetry factor Δs is calculated based on the difference between the two ears. This design improves the model's sensitivity to severe unilateral or highly asymmetric cases.

[0122] First, the final fused bilateral image features z' Fusion The text features T are L2 normalized to eliminate scale differences between different modalities. Then, the cosine similarity matrix S between the two is calculated using the following formula:

[0123]

[0124] To simulate the rigorous review of severe or asymmetric pathologies in clinical assessments, a dynamic temperature coefficient τ is defined. i2 :

[0125]

[0126] Where τ0 is the base temperature parameter, and β is the weighting factor for the difference between the two ears. Δs i2 These represent the specific severity scores of the bilateral image features or corresponding samples of the left and right ears after the i2th final fusion, respectively, as well as a measure of the difference between the specific severity scores of the left and right ears. For severe or asymmetrical cases, a lower τ... i2 The value will cause the gradient in the image contrast loss to be steeper, thus forcing more stringent alignment in the feature space.

[0127] Meanwhile, the positive similarity score is further enhanced through a non-linear boosting strategy:

[0128]

[0129] Among them, S i2,i2 γ represents the cross-modal similarity between the i2th final fused two-sided image feature and the corresponding text feature. l and γ r These are weighting factors for the left and right ears, respectively, controlling for similarity enhancement based on severity scores. This mechanism ensures the model focuses more on aligning features of clinically important regions. Subsequently, patient-specific InfoNCE contrast loss, i.e., cross-modal contrast loss, is defined as:

[0130]

[0131] Among them, S i2,j S represents the cross-modal similarity between the i-th final fused two-sided image feature and the j-th text feature. i2,jis the diagonal element in the cosine similarity matrix S, representing positive pairings, while off-diagonal elements correspond to negative pairings between different patients, and B is the total number of image samples.

[0132] This loss utilizes temperature modulation and similarity enhancement to guide diagnostic models to achieve accurate semantic alignment, especially in challenging pathological conditions.

[0133] Step S106: Construct an objective function based on image reconstruction loss, image contrast learning loss, and cross-modal contrast loss, so as to pre-train the diagnostic model using the objective function.

[0134] Specifically, a unified objective function can be constructed by jointly optimizing image reconstruction, intra-modal image contrast learning, and cross-modal alignment tasks:

[0135]

[0136] in, Let represent the unified objective function described above. These represent the image reconstruction losses for the left and right ears, respectively. These represent the image contrast learning losses for the left and right ears, respectively. λ represents the cross-modal contrast loss. cl λ represents the coefficient used to balance the image reconstruction loss and the image contrast learning loss. gl These represent the coefficients used to control the global alignment loss. The selection of these hyperparameters will be discussed in the experimental section.

[0137] Step S107: Input the current patient's ear MRI image and corresponding medical report into the pre-trained diagnostic model. Use the linear classifier in the diagnostic model to classify and diagnose the fused bilateral image features corresponding to the current patient's ear MRI image, and output the Meniere's disease diagnosis result.

[0138] The diagnostic model will be validated through a Meniere's disease diagnostic experiment.

[0139] To validate the diagnostic performance of our diagnostic model (SMAM-MD), it was compared with the following baseline models: ResNet (a classic image classification model based on deep residual networks that extracts global image features through hierarchical convolution), SimCLR (a self-supervised contrastive learning framework that uses image augmentation techniques to construct instance discrimination tasks to learn general visual representations), MoCo (a self-supervised learning method based on momentum contrast that increases the number of negative samples through a dynamic queue mechanism to optimize feature discrimination), CLIP (a large-scale pre-trained visual-language alignment model that achieves semantic mapping between images and text through cross-modal contrastive learning); ALBEF (an attention-based alignment model that integrates visual and linguistic modalities and enhances fine-grained semantic understanding through cross-modal interaction), ConVIRT (a medical image-text contrastive learning framework that optimizes cross-modal feature alignment reports in radiology); and MGCA (a contrastive learning method that incorporates medical anatomical priors and enhances the anatomical effectiveness of image representations through graph structure constraints). To ensure fair comparison, all baseline models used the same linear classifier and underwent end-to-end fine-tuning under a unified data preprocessing workflow.

[0140] Table 1. Classification results of the MD classification task.

[0141]

[0142]

[0143] As shown in Table 1, five widely accepted metrics were used to comprehensively evaluate diagnostic performance: Accuracy or Precision (ACC) measures the overall accuracy of classification; Sensitivity (SEN) reflects the model's ability to identify positive cases (such as Meniere's disease patients) and control the risk of missed diagnoses; Specificity (SPE) assesses the reliability of excluding negative cases (non-patients) and reduces the risk of misdiagnosis; F1 score balances precision and recall, considering the proportion of false positives and false negatives; and Area Under the Receiver Operating Characteristic (AUC) reflects classification stability at different decision thresholds. All metrics are within the range of [0, 1], with higher values ​​indicating better model discrimination ability, clinical safety, and robustness.

[0144] The effectiveness of this method was validated in the diagnostic task of Meniere's disease. Specifically, labeled data was split into training and validation sets at proportions p (ranging from 10% to 40% in 10% increments), with the remaining data serving as an unlabeled set to simulate the challenge of limited labeling resources in the real world. To evaluate the robustness of the model, a five-fold cross-validation strategy was employed, where patient data was randomly split and the experiment was repeated five times. The final performance metrics are reported as mean and standard deviation. All experiments were conducted on the same hardware environment using a fixed random seed to ensure reproducibility of results.

[0145] Table 1 shows the classification performance of various methods for the Meniere's disease diagnosis task at different annotation ratios, with the best results highlighted in bold. All methods showed a trend of continuous performance improvement as the annotation ratio p increased from 10% to 40%. Among them, the accuracy of the diagnostic model increased from 86.35% to 94.36%, and the AUC value improved significantly.

[0146] The accuracy improved from 85.67% to 97.08%, with sensitivity (SEN) and specificity (SPE) increasing by 5.28% and 14.03%, respectively. This demonstrates that more labeled data significantly enhances the model's ability to capture features of cochlear effusion and vestibular-topological anomalies. In contrast, unimodal baseline methods such as ResNet, SimCLR, and MoCo performed poorly in low-label settings (p=10%), achieving accuracy below 83%. Multimodal methods showed moderate improvements due to cross-modal alignment: general visual language models such as CLIP and ALBEF achieved 81.17% and 82.36% accuracy, respectively, while medical-specific contrastive methods such as ConVIRT and MGCA achieved 83.63% and 85.00%, respectively. However, these results are still significantly lower than 86%. Our diagnostic model (SMAM-MD) achieved 35%, highlighting the importance of fine-grained, domain-aware modeling in medical diagnostic tasks.

[0147] This diagnostic model significantly improves the diagnostic performance of Meniere's disease by synergistically integrating local pathological reconstruction, region contrast learning, and global cross-modal alignment mechanisms. Experimental results show that, with a low annotation rate (10%), local pathological reconstruction enhances the model's sensitivity to subtle lesions by encoding pixel-level cochlear effusion features, achieving a 4.3% improvement in specificity (reaching 80.26%) compared to baseline methods. Region contrast learning further reduces the risk of false negatives, achieving a sensitivity of 90.14% at a 20% annotation level by capturing the heterogeneous patterns between unilateral inner ear lesions and normal regions. Building on this, the global cross-modal alignment module integrates the anatomical-semantic associations between image lesions and textual descriptions, constructing a more discriminative cross-modal classification boundary. Therefore, the model achieves an AUC of 97.08% at a 40% annotation rate, reducing the false positive rate by 58.3%.

[0148] Compared to the baseline, this model performs well in predicting overlap across classes. Figure 3 As shown, both the ROC (Receiver Operating Characteristic) curve and probability distribution analysis demonstrate that the model maintains stable performance across different annotation ratios. Furthermore, 95% of the multimodal case predictions fall within the high confidence interval (probability greater than 0.8), validating that the multimodal collaborative framework effectively integrates hierarchical features, optimizing the interpretability and generalization ability of the diagnosis.

[0149] Furthermore, the present invention also provides an image processing system.

[0150] like Figure 4 As shown, the system includes, in sequence, an acquisition module, an encoding module, an image reconstruction module, a contrastive learning module, a scoring module, a pre-training module, and a diagnostic module.

[0151] The system comprises the following modules: an acquisition module for acquiring historical MRI images of patients' ears and corresponding medical reports; an encoding module for annotating regions of interest (ROIs) on the MRI images and extracting image patches from these ROIs, then encoding these patches using an encoder to generate left and right ear image features; an image reconstruction module for constructing an image reconstruction loss based on the left and right ear image features and performing image reconstruction to ensure that no lesion information is lost in the encoded image features; a contrast learning module for constructing an image contrast learning loss based on anchor point features extracted from the MRI images and performing image contrast learning to enable the diagnostic model to identify lesions and normal individual variations; and a scoring module for determining the left and right ear features based on the medical reports. The specific severity score of the right ear is used to perform cross-modal fusion and alignment between the image and text features extracted from the medical report based on the specific severity scores of the left and right ears. A cross-modal contrast loss is constructed during cross-modal alignment. A pre-training module is used to construct an objective function based on the image reconstruction loss, image contrast learning loss, and cross-modal contrast loss, so as to pre-train the diagnostic model using the objective function. The diagnostic module inputs the MRI image of the current patient's ear and the corresponding medical report into the pre-trained diagnostic model. A linear classifier in the diagnostic model classifies and diagnoses the fused bilateral image features corresponding to the MRI image of the current patient's ear, outputting a Meniere's disease diagnosis result.

[0152] It should be noted that the specific implementation of the image processing system in this embodiment of the invention can be found in the specific implementation of the image processing method described above. To avoid redundancy, it will not be repeated here.

[0153] Furthermore, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the above-described image processing method.

[0154] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. An image processing method, characterized in that, include: Obtain historical MRI images of patients' ears and corresponding medical reports; The region of interest (ROI) on the MRI image is labeled, and image patches of the ROI are extracted. The extracted image patches are then encoded using an encoder to generate left ear image features and right ear image features. An image reconstruction loss is constructed based on the features of the left and right ear images, and image reconstruction is performed to ensure that no lesion information is lost in the encoded image features; An image contrast learning loss is constructed based on anchor point features extracted from MRI images, and image contrast learning is performed to enable the diagnostic model to identify lesions and normal individual variations. The specific severity scores of the left and right ears are determined based on the medical report. Cross-modal fusion and alignment between the image and text features extracted from the medical report are performed based on the specific severity scores of the left and right ears. A cross-modal contrast loss is constructed when performing cross-modal alignment. An objective function is constructed based on image reconstruction loss, image contrast learning loss, and cross-modal contrast loss, so as to pre-train the diagnostic model using the objective function; The MRI image of the patient's ear and the corresponding medical report are input into the pre-trained diagnostic model. The linear classifier in the diagnostic model classifies and diagnoses the fused bilateral image features corresponding to the MRI image of the patient's ear, and outputs the diagnosis result of Meniere's disease.

2. The method as described in claim 1, characterized in that, An image reconstruction loss is constructed based on the features of the left and right ear images, and image reconstruction is performed, including: Local image patches were reconstructed from the features of the left and right ear images using a linear transformation function, respectively. A mean squared error loss function is constructed to measure whether the difference between the reconstructed local image patch and the corresponding original image patch is minimized, so that the encoded image features do not lose lesion information; wherein, the mean squared error loss function is expressed as the sum of squares of the differences between each pixel value between the reconstructed local image patch and the corresponding original image patch.

3. The method as described in claim 1, characterized in that, An image contrast learning loss is constructed based on anchor point features extracted from MRI images, and image contrast learning is performed, including: Positive and negative samples are determined based on anchor point features extracted from MRI images. Among them, samples whose spatial location from the anchor point feature is within a preset neighborhood and whose semantic similarity to the anchor point feature is greater than a preset value are classified as positive samples, while samples whose spatial location from the anchor point feature is not within the preset neighborhood and whose semantic similarity to the anchor point feature is less than or equal to a preset value are classified as negative samples. Construct an image contrast learning loss based on anchor point features, positive samples, and negative samples; The encoder parameters are adjusted to minimize the image contrast learning loss, resulting in an encoder with the minimum image contrast learning loss. The left and right ear image features encoded by the encoder with the minimum image contrast learning loss can identify lesion features or normal individual variation features.

4. The method as described in claim 1, characterized in that, Specific severity scores for the left and right ears were determined based on medical reports, including: Identify entities in the medical report, the entities being key clinical entities, the entities including anatomical structures, pathological descriptions, and lateral indications, the lateral indications being left, right, and bilateral; Calculate the similarity between an entity and the global context in a medical report, so as to determine the context weight of each entity based on the similarity and the presence of negative words in the medical report; Determine the basic weight of each entity, determine whether there are severe, moderate, and mild keywords in the medical report, and determine the severity level adjustment coefficient based on the keyword occurrence results; The final weight value for each entity is determined based on its base weight, context weight, and severity level adjustment factor, so as to determine the specific severity score for the left and right ears based on the base weight and final weight value of each entity.

5. The method as described in claim 4, characterized in that, After determining the specific severity scores for the left and right ears, the method further includes: determining whether there is at least one bilateral entity in the medical report; if so, increasing the specific severity scores for both the left and right ears by a preset percentage.

6. The method as described in claim 1, characterized in that, Cross-modal fusion between images and text features extracted from medical reports was performed based on specific severity scores for the left and right ears, including: Text features are extracted from medical reports, and left ear image features, right ear image features, and text features are projected into the same shared semantic space. The semantic correlation between local image patches and text in the medical report is calculated based on the projected features of the left ear image features, the right ear image features, and the text features. Then, the left ear multi-head attention output and the right ear multi-head attention output are obtained by weighted summation based on the semantic correlation and the corresponding text. Based on the specific severity scores of the left and right ears, the multi-head attention output of the left and right ears was modulated to focus on the more severe areas of the lesion, and the final multi-head attention output of the left and right ears was obtained. Feature fusion is performed on the final multi-head attention outputs of the left and right ears to obtain fused bilateral image features.

7. The method as described in claim 6, characterized in that, Feature fusion is performed on the final left-ear multi-head attention output and right-ear multi-head attention output to obtain fused bilateral image features, including: The preliminary fused features were obtained by weighting and summing the final multi-head attention outputs of the left and right ears based on the specific severity scores of the left and right ears, respectively. Asymmetry factors were determined based on specific severity scores for the left and right ears; Enhanced features after initial fusion based on asymmetric factors; The enhanced initial fused features are subjected to residual connection and layer normalization to obtain the final fused bilateral image features.

8. The method as described in claim 7, characterized in that, Performing cross-modal alignment includes: The final fused bilateral image features and text features are normalized, and the cosine similarity matrix between the two is determined after normalization. A dynamic temperature coefficient is determined based on the cosine similarity matrix of the two ears and the specific severity scores of the left and right ears. Determine the positive similarity scores between the final fused bilateral image features and their corresponding text features and other text features; A cross-modal contrast loss is constructed based on two positive similarity scores and a dynamic temperature coefficient to guide cross-modal alignment of images and text, ensuring image and text matching. The image is the final fused two-sided image. The objective function, constructed based on cross-modal contrast loss, image reconstruction loss, and image contrast learning loss, is expressed as follows: Among them, l total Describe the objective function. These represent the image reconstruction losses for the left and right ears, respectively. Let l represent the image contrast learning loss for the left and right ears, respectively. cross λ represents the cross-modal contrast loss. cl λ represents the coefficient used to balance the image reconstruction loss and the image contrast learning loss. gl This represents the coefficient used to control the global alignment loss.

9. An image processing system, characterized in that, include: The acquisition module is used to acquire historical MRI images of patients' ears and corresponding medical reports; The encoding module is used to annotate the region of interest on the MRI image and extract image patches of the region of interest. The encoder encodes the extracted image patches to generate left ear image features and right ear image features. The image reconstruction module is used to construct an image reconstruction loss based on the features of the left ear image and the right ear image, and to perform image reconstruction so that the encoded image features do not lose lesion information; The contrast learning module is used to construct an image contrast learning loss based on anchor point features extracted from MRI images and to perform image contrast learning so that the diagnostic model can identify lesions and normal individual variations. The scoring module is used to determine the specific severity scores of the left and right ears based on the medical report, and to perform cross-modal fusion and alignment between the image and text features extracted from the medical report based on the specific severity scores of the left and right ears, wherein a cross-modal contrast loss is constructed when performing cross-modal alignment; A pre-training module is used to construct an objective function based on image reconstruction loss, image contrast learning loss, and cross-modal contrast loss, so as to pre-train the diagnostic model using the objective function; The diagnostic module is used to input the current patient's ear MRI image and corresponding medical report into the pre-trained diagnostic model. The linear classifier in the diagnostic model classifies and diagnoses the fused bilateral image features corresponding to the current patient's ear MRI image to output the diagnosis result of Meniere's disease.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

Images