Multi-modal data analysis method and device for temporal bone disease diagnosis

Abstract

The invention relates to a multi-modal data analysis method and device for temporal bone disease diagnosis, and the method comprises the steps: obtaining temporal bone multi-modal data, carrying out the modal difference elimination, and obtaining structured feature data; performing cross-modal space registration and target structure mapping on the structured feature data to obtain cross-modal registration data; performing cross-modal correlation calculation on the structural features and the cross-modal registration data to obtain quantitative correlation parameters; and performing multi-modal trend analysis on the quantitative correlation parameters to obtain a temporal bone analysis report. According to the method, the defects of traditional single-mode evaluation can be overcome, and the comprehensiveness and accuracy of disease diagnosis are improved.

Description

Technical Field

[0001] This invention relates to the technical field of air source heat pumps, and in particular to a multimodal data analysis method and apparatus for the diagnosis of temporal bone diseases. Background Technology

[0002] Diagnosis of temporal bone diseases relies on the comprehensive analysis of multimodal medical imaging data, but existing methods struggle to effectively integrate complementary information from different modalities. Due to differences in imaging principles, significant distributional inconsistencies exist between multimodal data, leading to biases in feature extraction and correlation analysis. Spatial registration accuracy across modal data is insufficient, especially for the complex, minute structures within the temporal bone, making precise anatomical alignment difficult. Existing techniques typically employ single-dimensional correlation calculations, failing to fully reflect the deep relationships between multimodal data, thus limiting the accuracy and reliability of disease diagnosis. Summary of the Invention

[0003] The main objective of this invention is to provide a multimodal data analysis method and apparatus for the diagnosis of temporal bone diseases, which can overcome the shortcomings of traditional single-modal assessment and improve the comprehensiveness and accuracy of disease diagnosis.

[0004] To achieve the above objectives, the present invention provides a multimodal data analysis method for the diagnosis of temporal bone diseases, comprising: Acquire temporal bone multimodal data, perform modal difference elimination, and obtain structured feature data; Cross-modal spatial registration and target structure mapping are performed on the structured feature data to obtain cross-modal registration data; The structured features are correlated with the cross-modal registration data to obtain quantized correlation parameters. Multimodal trend analysis was performed on the quantified correlation parameters to obtain a temporal bone analysis report.

[0005] Furthermore, the acquisition of temporal bone multimodal data, modality difference elimination, and obtaining structured feature data include: The temporal bone multimodal data of the target object is obtained through a preset modal database, and the data is split to obtain image data, symptom assessment data and functional measurement data. Image feature extraction and structural standardization are performed on the image data to obtain structured image data; Clinical features were extracted and clinical data were reconstructed from the symptom assessment data to obtain structured clinical data. The functional measurement data is subjected to noise filtering and waveform time-frequency domain extraction to obtain functional structured data; The structured image data, the structured clinical data, and the structured functional data are integrated to obtain the structured feature data.

[0006] Further, the step of performing cross-modal spatial registration and target structure mapping on the structured feature data to obtain cross-modal registration data includes: The structured feature data is subjected to multimodal spatial transformation to obtain registration transformation parameters; Based on the temporal bone multimodal data and the registration transformation parameters, the structured feature data is subjected to rigid registration and non-rigid deformation field optimization to obtain initial registration data; Based on the initial registration data, structural node detection is performed to obtain the target structural feature point set; Cross-modal topological relationship matching is performed on the target structure feature point set to obtain the target structure mapping relationship; The initial registration data is spatially consistent with the target structure mapping relationship to obtain the cross-modal registration data.

[0007] Further, the step of spatially consistentally adjusting the initial registration data according to the target structure mapping relationship to obtain the cross-modal registration data includes: A bidirectional nearest neighbor search is performed on the target structure mapping relationship to obtain a bidirectional matching point set; Local alignment data is obtained by performing a local affine transformation on the initial registration data based on the bidirectional matching point set. Nonlinear deformation analysis is performed on the local alignment data to obtain global deformation correction data; Multimodal voxel intensity similarity analysis was performed on the global deformation correction data to obtain spatially consistent regions; Based on the spatial consistency region, the global deformation correction data is iteratively optimized layer by layer to obtain intermediate registration results; Boundary constraints and spatial adjustments are applied to the intermediate registration results to obtain the cross-modal registration data.

[0008] Further, the step of performing cross-modal association calculations on the structured features and the cross-modal registration data to obtain quantized association parameters includes: The structured feature data is aligned between modal features to obtain a unified dimension feature vector; Based on the unified dimension feature vector, spatial topological relationships are constructed on the registered data to obtain a cross-modal topological correlation matrix; Based on the cross-modal topological correlation matrix, feature fusion calculation is performed on the unified dimension feature vector to obtain a multimodal joint feature tensor; The multimodal joint feature tensor is fused using a dual-path cross-convolution to obtain a fused associated feature map; The fused correlation feature map is subjected to cross-modal correlation extraction and parameterized mapping to obtain the quantized correlation parameters.

[0009] Further, the step of performing dual-path cross-convolution fusion on the multimodal joint feature tensor to obtain a fused associated feature map includes: The multimodal joint feature tensor is initialized with a first path convolution kernel according to a preset convolution database to generate a set of bone feature extraction convolution kernels. The multimodal joint feature tensor is initialized with a second path convolution kernel based on the convolution database to generate a soft tissue feature extraction convolution kernel group; Based on the bone feature extraction convolution kernel group, multi-scale dilated convolution is performed on the multimodal joint feature tensor to obtain a bone feature map sequence; Based on the soft tissue feature extraction convolution kernel group, deformable convolution is performed on the multimodal joint feature tensor to obtain a soft tissue feature map sequence; The bony feature map sequence and the soft tissue feature map sequence are subjected to dual-path cross-fusion to obtain the fused associated feature map.

[0010] Furthermore, the step of performing multimodal trend analysis on the quantified correlation parameters to obtain a quantified diagnostic report includes: Temporal bone target features are extracted from the quantized correlation parameters to obtain the target feature vector; The target feature vector is subjected to cross-modal temporal alignment to obtain a synchronized target sequence; Based on the synchronized target sequence, dynamic correlations between modes are constructed to obtain a temporal bone evolution map; Abnormal pattern detection was performed on the temporal bone evolution map to obtain potential abnormal information of the target. Based on the potential anomaly information of the target, temporal bone target structure mapping and matching are performed to obtain target location distribution data; Based on the temporal bone evolution map, the target potential anomaly information, and the target location distribution data, a report quantification report is constructed to obtain a temporal bone analysis report.

[0011] Furthermore, the abnormal pattern detection of the temporal bone evolution map to obtain target potential abnormal information includes: The temporal bone evolution map is hierarchically divided to obtain a multi-level sub-map set; Node feature calculations are performed on the multi-level subgraph set to obtain anomaly quantification indicators; Based on the aforementioned anomaly metric, nodes are associated to obtain a set of anomaly nodes; Spatial clustering is performed on the set of abnormal nodes to obtain the distribution of abnormal regions; Based on the distribution of the abnormal regions, target structure matching is performed to obtain the abnormal region mapping; Feature integration is performed on the abnormal region mapping to obtain potential abnormal information of the target.

[0012] The present invention also provides a multimodal data analysis device for the diagnosis of temporal bone diseases, applied to any one of the multimodal data analysis methods for the diagnosis of temporal bone diseases described above, comprising: The acquisition module is used to acquire temporal bone multimodal data, perform modal difference elimination, and obtain structured feature data. The analysis module is used to perform cross-modal spatial registration and target structure mapping on the structured feature data to obtain cross-modal registration data; The association module is used to perform cross-modal association calculations between the structured features and the cross-modal registration data to obtain quantized association parameters; The processing module is used to perform multimodal trend analysis on the quantified correlation parameters to obtain a temporal bone analysis report.

[0013] The present invention provides a multimodal data analysis method and apparatus for the diagnosis of temporal bone diseases, which has the following beneficial effects: By extracting structured features and eliminating differences in multimodal data, noise interference between different imaging modalities is effectively reduced, improving the comparability and reliability of feature data and providing more accurate basic data for subsequent analysis. Cross-modal spatial registration and target structure mapping techniques are employed to achieve alignment of fine temporal bone structures and precise localization of target structures, enhancing the spatial consistency of multimodal data and reducing diagnostic errors. Cross-modal correlation calculations reveal the multimodal synergistic characteristics of temporal bone lesions, compensating for the shortcomings of traditional single-modal assessment and improving the comprehensiveness and accuracy of disease diagnosis. Multimodal trend analysis based on quantitative correlation parameters generates a comprehensive temporal bone analysis report. Attached Figure Description

[0014] Figure 1 This is a flowchart of a multimodal data analysis method for diagnosing temporal bone diseases provided by the present invention; Figure 2 This is a structural diagram of a multimodal data analysis device for diagnosing temporal bone diseases provided by the present invention.

[0015] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0018] Reference Figure 1 As shown, this invention provides a multimodal data analysis method for the diagnosis of temporal bone diseases, comprising: Step S1: Obtain temporal bone multimodal data, perform modal difference elimination, and obtain structured feature data; Step S2: Perform cross-modal spatial registration and target structure mapping on the structured feature data to obtain cross-modal registration data; Step S3: Perform cross-modal correlation calculations on the structured features and cross-modal registration data to obtain quantized correlation parameters; Step S4: Perform multimodal trend analysis on the quantitative correlation parameters to obtain a temporal bone analysis report.

[0019] Based on the steps described above, the detailed process is as follows: Step S1: Imaging data, clinical information data, and physiological function data of the temporal bone region were collected from patients using medical imaging equipment and a clinical information system. Imaging data included high-resolution CT bone window images and MRI soft tissue sequences. Clinical information data covered symptom scores and medical history records. Physiological function data included audiological examination and vestibular function test results. Standardization and reconstruction were performed on the multimodal data: CT values ​​in the imaging data were converted to the standard HU interval through linear mapping, and MRI signal intensity was normalized to a fixed grayscale range. For the clinical information data, categorical symptoms were converted into one-hot encoded vectors, continuous scores were Z-score standardized, and the medical history text was generated using binary tags.

[0020] Physiological function data is denoised using bandpass filtering before time-frequency feature parameters are extracted. The output structured feature dataset contains three categories of sub-features: image structured features, clinical structured features, and physiological structured features. Image structured features include bone volume, surface area, and texture statistics; clinical structured features include symptom evolution slope; and physiological structured features include auditory nerve conduction delay rate. This step eliminates the dimensional heterogeneity of multimodal data, providing a unified input format for cross-modal alignment.

[0021] Step S2: Anatomical-driven spatial registration is performed using image features from structured feature data as input. Key anatomical landmarks are located in CT images based on bone density gradients to pinpoint the facial nerve canal entrance coordinates, and in MRI images, signal intensity gradients are used to identify the ossicular chain contour point set. Multimodal spatial alignment uses CT bony landmarks as the reference coordinate system, calculates the spatial offset vector of MRI landmarks using an iterative nearest-point algorithm, optimizes it to sub-millimeter error, and establishes cross-modal point pair mapping relationships.

[0022] The composite transformation model is constructed by deriving the affine transformation matrix based on mapping relationships, and then superimposing spline free deformation corrections to generate a spatial transformation parameter matrix for local soft tissue deformation. Image resampling and validation apply the transformation parameters to the original MRI voxel space, generating registered images through trilinear interpolation, and calculating the Dice coefficient for the internal auditory canal region to verify registration accuracy. The output cross-modal registration data includes spatially aligned CT bone window and MRI soft tissue fusion images, as well as cross-modal coordinate mapping tables for key anatomical structures such as the facial nerve canal and ossicular chain. This step achieves sub-millimeter spatial alignment, providing an anatomical benchmark for cross-modal association.

[0023] Step S3: A joint feature tensor with three-dimensional spatial coordinate binding is constructed by fusing multimodal features based on spatial mapping relationships. Cross-modal association rules are executed to calculate the covariance between bone density change rate and auditory nerve delay rate, establishing symptom-image association models such as linear regression between facial nerve canal stenosis rate and facial paralysis score slope. Parameterized output extracts key correlation indicators, including the nerve compression index and the correlation coefficient between bone destruction rate and hearing threshold shift.

[0024] The final output quantitative correlation parameter vector includes two core indicators: spatial correlation strength and cross-modal dynamic evolution parameters. Spatial correlation strength quantifies the correlation between anatomical structural displacement and functional decline, while cross-modal dynamic evolution parameters characterize the mapping relationship between lesion progression rate and clinical symptom deterioration. This step overcomes the limitations of a single modality, achieving quantitative correlation modeling of structure-function-symptom.

[0025] Step S4: Multimodal temporal evolution modeling and diagnostic decision generation are performed based on quantified correlation parameters. The spatial location correlation strength and cross-modal dynamic evolution parameters in the quantified correlation parameters are aligned along the time axis. Using the patient's initial consultation time as the baseline, missing follow-up interval values ​​are filled in using cubic spline interpolation to generate a synchronized temporal feature matrix. A temporal bone evolution atlas is constructed based on the synchronized temporal feature matrix. This atlas includes node attributes (facial nerve canal stenosis rate, ossicular chain displacement), edge weights (structure-function correlation strength), and temporal state transition probabilities (such as the probability of hearing threshold shift due to the monthly increase in bone destruction volume).

[0026] Anomaly pattern detection was performed on the temporal bone evolution map. The isolated forest algorithm was used to identify low-density evolutionary paths. False anomalies were filtered out by combining temporal bone anatomical constraint rules, and potential abnormal regions and confidence scores were output.

[0027] Based on the anomaly detection results, a predefined template library is matched to calculate the type probability value and severity grading value (Grade III, bone destruction index 7.2 / 10). The evolutionary atlas critical pathways, anomaly region localization, and disease quantification parameters are integrated to generate a structured diagnostic report. The report includes a visualization of the registered image and the overlay of the anomaly region, disease probability determination values, severity grading values, and recommendations.

[0028] This invention provides a multimodal data analysis method for the diagnosis of temporal bone diseases. Through structured feature extraction and difference elimination of multimodal data, it effectively reduces noise interference between different imaging modalities, improves the comparability and reliability of feature data, and provides more accurate basic data for subsequent analysis. By employing cross-modal spatial registration and target structure mapping techniques, it achieves alignment of fine temporal bone structures and precise localization of target structures, enhancing the spatial consistency of multimodal data and reducing diagnostic errors. Through cross-modal correlation calculations, it reveals the multimodal collaborative characteristics of temporal bone lesions, compensating for the shortcomings of traditional single-modal assessment and improving the comprehensiveness and accuracy of disease diagnosis. Based on quantitative correlation parameters, multimodal trend analysis is performed to generate a comprehensive temporal bone analysis report.

[0029] In one embodiment, temporal bone multimodal data is acquired, modality discrepancies are eliminated, and structured feature data is obtained, including: The pre-defined modal database stores a set of raw medical data containing unique identifiers for the target objects. Access commands trigger the database retrieval logic, returning a multimodal data packet bound to the target identifier.

[0030] The data splitting operation automatically performs classification based on differences in data generation device type and storage format: the imaging data classification unit identifies the header file of the medical digital imaging communication protocol and extracts all computed tomography sequences conforming to the DICOM standard. The symptom assessment data classification unit captures structured tabular data and free text clinical records from the electronic medical record system. The functional measurement data classification unit captures binary waveform files and corresponding detection parameter configuration files output by physiological signal acquisition devices.

[0031] The classification process verifies the integrity flag of the original data, discards data streams that fail verification, and ensures that the three data subsets are free from cross-contamination and cover the complete information dimensions required for temporal bone disease assessment.

[0032] Image data is loaded into the image analysis engine for multi-stage processing. The 3D reconstruction algorithm synthesizes an isotropic body data model from the original tomographic sequences, eliminating geometric distortions caused by differences in scan slice thickness. The anatomical structure recognition module performs semi-automatic segmentation based on a temporal bone template atlas, delineating the spatial coordinate boundaries of key regions such as the cochlear bony labyrinth, ossicular chain, facial canal, and vestibular aqueduct. The feature extraction module performs topological analysis on the segmented regions, quantifying geometric parameters such as structural volume, surface area, radius of curvature, and bone density gradient, while simultaneously calculating the spatial relative position vectors between regions.

[0033] The structural normalization module maps individual anatomical structures to a standard temporal bone reference coordinate system using non-rigid registration technology, eliminating spatial biases caused by differences in head posture. The output image structured data is a matrix structure, with row vectors corresponding to target object identifiers and column vectors storing the anatomical structure numbers in the normalized coordinate system and their associated set of quantized feature values.

[0034] The symptom assessment data input into the clinical information standardization pipeline is implemented. The symptom quantification submodule parses standard clinical scales (such as the Tinnitus Disorder Scale and the Vertigo Symptom Scale) and encodes multiple-choice answers into discrete numerical scores. The text mining submodule uses a terminology matching engine to scan free text fields, identify predefined clinical symptom keywords such as "pulsatile tinnitus," "ear fullness," and "facial muscle weakness," and outputs standardized symptom codes validated by a medical terminology ontology database.

[0035] The time series reconstruction submodule integrates multi-timepoint records to generate standardized timeline markers for symptom duration, frequency of attacks, and evolution patterns. The data reconstruction unit merges these outputs to construct a fixed-dimensional clinical feature vector: vector elements include symptom codes, quantified severity values ​​(0-10 levels), and time attribute labels. An outlier detection algorithm corrects contradictory input entries, such as deleting symptom descriptions recorded later than the detection date, ensuring the feature vector conforms to clinical logical time sequence.

[0036] Functional measurement data is input into the biosignal processing channel. The noise filtering module is configured with multi-cascaded digital filters: a 50 / 60Hz notch filter to eliminate power frequency interference, a Butterworth bandpass filter to constrain the signal within the auditory physiological response frequency band (100Hz-8kHz), and an adaptive filtering algorithm to suppress electromyographic artifacts and motion drift. The preprocessed signal enters the time-frequency conversion unit, where a windowed Fourier transform is performed to calculate the short-time energy spectrum, or a Morlet wavelet transform is applied to extract the instantaneous amplitude envelope of a specific frequency band (such as 1-4kHz for otoacoustic emissions).

[0037] Feature localization algorithms detect physiological event markers: auditory brainstem response wave identification of the latency and interpeak periods of waves I-V. Vestibular evoked myogenic potentials (PEPs) are measured for P13 / N23 amplitude. Nystagmography is used to quantify the peak slow-phase angular velocity. The output functional structured data is presented as a parameterized table, with each row containing the target object identifier, detection item name, feature parameter type (e.g., "ABR wave III latency"), and its numerical unit, linked to the timestamp index of the original waveform to ensure data traceability.

[0038] The engine integrates three types of structured data inputs across modalities. The identifier matching unit uses the unique code of the target object as an index to verify the consistency of attribution among imaging anatomical data, clinical symptom vectors, and electrophysiological parameters. The feature alignment module establishes data association rules: morphological parameters of specific temporal bone anatomical structures (such as the facial nerve canal) are bound to ipsilateral nerve function test results. Symptom codes are forcibly matched with corresponding physiological dysfunction detection items.

[0039] The data fusion unit performs a vector concatenation operation, stitching together the anatomical quantification values ​​of the image feature matrix, the symptom severity grading of the clinical feature vector, and the physical quantity values ​​of the functional feature parameters in a preset order to form a complete feature vector. The standardization unit normalizes the concatenated feature vector, achieving dimensionless conversion of parameters with different dimensions. The output structured feature data is a multi-dimensional array, where each row represents the full-dimensional feature vector of a unique target object, and also stores the data source index and integration confidence level markers from the integration process.

[0040] This embodiment utilizes a pre-defined modal database for targeted retrieval and automatic segmentation to ensure strict source differentiation among the acquired images, symptom assessments, and functional measurements, eliminating modal cross-interference caused by mixed raw data. Standardized image structure processing establishes a unified anatomical spatial coordinate system, overcoming the incomparability of morphological features caused by different scanning devices and individual anatomical variations, thus providing a spatially consistent basis for the quantitative analysis of bone lesions. Terminology encoding and timeline reconstruction of symptom assessment data transform fragmented clinical descriptions into standardized feature vectors, enabling objective and quantifiable expression of subjective symptoms and eliminating assessment biases caused by ambiguities in medical terminology and misaligned recording times. Noise filtering and time-frequency feature extraction of functional signals preserve the essential information of physiological responses, filtering out environmental interference and inherent equipment noise, ensuring that electrophysiological parameters reflect the true state of the neuroauditory pathway.

[0041] 3. The multimodal data analysis method for the diagnosis of temporal bone diseases according to claim 1, characterized in that cross-modal spatial registration and target structure mapping are performed on structured feature data to obtain cross-modal registration data, including: The structured feature data input spatial registration preprocessing engine performs coordinate system normalization. The data-driven module parses the standard temporal bone reference coordinate system parameters contained in the image structured data, the spatial markers of symptom areas in the clinical structured data records, and the topological location information of detection points associated with the functional structured data. The transformation parameter generation unit establishes the initial matrix of the three-dimensional affine transformation and uses a feature matching algorithm to calculate the translation vector, rotation angle, and scale factor of the spatial reference points of different modalities.

[0042] The parameter optimization process uses the least squares method to iteratively adjust the matrix coefficients, maximizing the overlap between the center points of bony landmarks, the centroid coordinates of clinical symptom projection areas, and the spatial projection positions of functional detection hotspots in the virtual space. The output registration transformation parameters include homogeneous coordinate transformation matrix elements and spatial deformation residual evaluation values.

[0043] The original spatial information and registration transformation parameters of the temporal bone multimodal data are input into the geometric registration engine. During the rigid registration stage, a spatial rigid body transformation is performed: the cochlear coordinate center point from the original image data, the centroid of the symptom voxel cluster located by the symptom assessment data, and the coordinates of the vestibular core response area marked by the functional measurement data are all translated and rotated based on the transformation matrix. After coordinate transformation, bidirectional distance measures are used to evaluate registration consistency, such as Hausdorff distance to calculate the boundary matching degree between the image bone contour and the symptom projection area.

[0044] Deformation field correction is applied during the non-rigid optimization stage: A thin-plate spline interpolation model is constructed, and a nonlinear displacement vector field is generated based on the local deformation characteristics of the temporal bone air cell system, differences in the course of blood vessels and nerve bundles, and the gradient of symptom area distribution. The displacement field drives topological deformation of movable structures (such as the ossicular chain and facial nerve canal) in the initial registration data, eliminating local spatial offsets caused by individual anatomical differences. The output initial registration data is a multimodal fused three-dimensional data volume, where each voxel contains superimposed CT values, symptom marker intensity, and functional response intensity information.

[0045] Initial registration data is loaded into the anatomical topology analysis module for multi-scale feature point localization. Node definition rules predefine key anatomical intersections of the temporal bone: the cochlear base rotation center, the malleus-incus joint rotation axis, the projection apex of the ampullary crest of the semicircular canals, the geometric center of the geniculate ganglion of the facial nerve, and the junction of the tympanic tegmentum and the anterior wall of the sigmoid sinus. The detection algorithm employs a multimodal response fusion strategy: extracting bony structural corner points in high-density areas of the image; calculating extreme points of symptom intensity gradients in symptom-marked areas; and identifying the spatial accumulation center of neural electrical activity in functional response hotspot areas.

[0046] Feature point selection criteria include geometric constraints and physiological significance verification. The final target structure feature point set records the three-dimensional coordinates of each point, the corresponding anatomical structure number, and the detection confidence value, forming an anchoring marker network for cross-modal space registration.

[0047] The target structural feature point set is input into the topological relation mapper to perform spatial constraint relation modeling. The relation matching unit establishes multiple geometric associations between point sets: analyzes the spatial clustering of cross-modal feature points of the same anatomical structure; constructs fixed distance constraints for adjacent anatomical feature points; and calculates the shortest connected path between functional response points and symptom area points.

[0048] Cross-modal consistency verification rules reject topologically conflicting point pairs: when the anatomical attribution of bony landmarks in imaging contradicts that of functional response points (e.g., auditory brainstem response signals detected in vestibular aqueduct images), a conflict marker re-detection mechanism is triggered. The mapping relationship output uses a directed graph structure: nodes represent feature points detected in each modality, and edges label the distance vector, angle, and associated physiological function type between point pairs. The target structure mapping relationship table records the mapping coordinates of each node in the registration space and the topological constraint weights.

[0049] The initial registration data and the mapping relationship between the target structure are input into the spatial correction engine to perform registration refinement operations. The consistency adjustment algorithm applies strengthening constraints based on the mapping relationship table: it forces feature points of the same anatomical structure to achieve complete spatial overlap; it performs Laplace smoothing constraints on the distance between feature points of adjacent structures; and it adjusts the gradient of the response intensity distribution according to the function-symptom association edge attributes.

[0050] Voxel-level optimization employs an optical flow propagation mechanism to diffuse feature point displacement vectors throughout the entire 3D data field, maintaining local structural continuity. An error control unit monitors registration quality indices during the adjustment process: calculating the bone-nerve interface registration residual, the overlap rate between symptom regions and pathological change areas, and the spatial overlap between functional hotspots and anatomical reaction foci. The refined cross-modal registration data is reconstructed into a multi-channel 3D matrix. Each channel layer stores the spatially integrated mapping results of image grayscale values, symptom quantification intensity, and functional response amplitude. Voxels at the same spatial location in each channel exhibit a strict registration correspondence of anatomical, functional, and clinical attributes.

[0051] This embodiment achieves spatial benchmark unification for imaging, symptom, and functional data through a multimodal spatial transformation parameter generation mechanism, eliminating coordinate system separation issues caused by differences in data acquisition. Rigid registration and deformation field optimization are combined to correct individual temporal bone anatomical heterogeneity, fully preserving the inherent morphology of the bony structure while adapting to the physiological curvature of the neurovascular bundle, thus enhancing the anatomical realism of multimodal spatial fusion. Precise localization of the target structural feature point set constructs a cross-modal topological node network, locking in core anatomical spatial relationships through hard constraint points such as the cochlear base-turn center point and key nodes of the facial nerve canal. A cross-modal topological matching algorithm forces a mapping relationship between symptom distribution areas and functional response hotspots within the bony boundaries, overcoming the spatial separation defect between subjective description and objective detection in traditional methods.

[0052] 4. The multimodal data analysis method for temporal bone disease diagnosis according to claim 3, characterized in that, spatial consistency adjustment is performed on the initial registration data according to the target structure mapping relationship to obtain cross-modal registration data, including: The target structure mapping relationship input point set matching module performs bidirectional topological association verification. The search unit uses image feature points as the reference source point set and functional and clinical feature points as the target point set to build a three-dimensional spatial tree accelerated index structure. The forward matching process calculates the shortest Euclidean distance nearest neighbor point from each image point to the functional point set, and the constraint condition restricts the matching point pairs to belong to the same anatomical structure category.

[0053] The reverse matching process uses functional points as the source to trace back the image point set, implementing a bidirectional verification mechanism with equal distance constraints. Match validity verification criteria include a distance threshold (less than 1.5 times the voxel size) and topological consistency rules. Isolated point pairs that have successfully matched unidirectionally are rejected. The final retained bidirectional matching point set records the strict spatial correspondence between the source and target points, along with their anatomical identifier codes.

[0054] A bidirectional matching point set drives the registration engine to perform partition deformation. The spatial segmentation module divides the initial registration data volume according to the temporal bone partition boundaries. The local transformation unit establishes an affine transformation model within each partition using the matching point set: calculating the average spatial translation deviation of all matching point pairs within the partition as the baseline offset; decomposing the coordinate axis rotation components to correct directional deviations; and using scaling factors to compensate for differences in bony structure dimensions. The transformation parameter optimization process introduces smoothing constraints for overlapping partition areas to prevent abrupt boundary changes in affine transformations of adjacent areas. After transformation, the data undergoes partition resampling to ensure that the image voxel gray values, symptom distribution intensity, and functional response amplitude within each anatomical unit maintain topological integrity during spatial transformation. The output locally aligned data retains the original multi-channel information, eliminating local spatial misalignment of cross-modal data within partitions.

[0055] Locally aligned data is loaded into the deformation field modeling unit to resolve residual spatial distortion. A multi-scale feature extractor identifies three types of nonlinear deformation features: micro-curvature changes in bony structures, elastic deformation of soft tissues, and non-rigid spatial displacement in symptom-function correlation zones. The deformation correction engine constructs a thin-plate spline interpolation model, using bidirectional matching point sets as control nodes to interpolate and generate a continuous displacement field.

[0056] Model constraints are enforced: deformation in bony regions maintains rigidity (deformation gradient tensor eigenvalues ​​approach 1); deformation along neurovascular bundle pathways maintains curve continuity; deformation intensity in symptom distribution areas is positively correlated with functional response amplitude. A global deformation field is applied to locally aligned data for resampling, and the corrected data validates spatial continuity indicators, outputting globally corrected deformation data with anatomical realism.

[0057] Global deformation correction data is input into a multi-channel fusion evaluation module, which performs voxel-level correlation analysis. A similarity calculation unit processes data voxel-by-voxel on a three-dimensional spatial grid, simultaneously calculating the mutual information entropy and correlation coefficient of the three channels: image grayscale value, symptom intensity value, and functional response amplitude. A spatial clustering algorithm identifies mutual information peak regions and merges voxel sets that satisfy a joint distribution: a triple correlation pattern between high-density bone areas in the image and symptom intensity loss and stable functional signals defines a bony consistency zone; a composite pattern between soft tissue areas in the image and high symptom intensity values ​​and functional abnormal fluctuations defines a lesion change consistency zone.

[0058] The similarity verification rule incorporates neighborhood constraints, forcing consistent regions to possess spatial continuity and anatomical boundary compliance. The output spatially consistent region labeling map includes consistent region numbers, spatial mask ranges, and multimodal association strength scores, serving as the baseline region for registration optimization.

[0059] A spatially consistent region-driven registration optimization loop mechanism is employed. The hierarchical processor segments the global data according to the temporal bone anatomical layers, with each layer independently executing optimization iterations. The optimization unit samples a set of control points within the consistency region of each layer and determines the spatial offset by calculating the direction deviation of the multimodal gradient flow field. The iterative update strategy adopts a rigid-flexible coupling mode: the bony labyrinth layer performs rigid transformations to compensate for displacement residuals; the neural canal layer implements elastic displacements to maintain the smoothness of nerve course; and the air cell layer undergoes affine transformations to adapt to changes in air cell morphology. After each iteration, the voxel similarity within the updated region is reassessed, and the loop terminates when the consistency score increment falls below a threshold (e.g., less than 2%) or the maximum number of anatomical layers is reached. Intermediate registration results generate hierarchical registration displacement field records, and the displacement vectors of each anatomical layer are superimposed onto the global deformation correction data for resampling.

[0060] Intermediate registration results are input into the final verification unit for anatomically driven boundary refinement. Hard boundary constraints are applied to the inherent barrier structure of the temporal bone: the bony wall of the internal auditory canal is forced into an indeformable boundary; the bony passage of the facial nerve canal restricts the range of nerve pathway displacement; and the tympanic tegmentum plane constrains vertical displacement. Soft boundary adjustment mechanisms are applied to deformable structures: the incus-staped joint rotation angle maintains freedom of movement through vector rotation; and an elastic displacement transfer model is implemented at the tensor tympani muscle attachment point.

[0061] The spatial adjustment module calculates the multimodal spatial coincidence after displacement. If residuals in a specific region are detected, a local affine transformation is triggered again for fine-tuning. The output cross-modal registration data is a three-dimensional multi-channel matrix with displacement field records. Each voxel achieves sub-millimeter spatial coincidence in image coordinate space, symptom projection space, and functional response space, and is accompanied by a registration quality confidence map.

[0062] This embodiment establishes anatomical constraints through a bidirectional nearest neighbor matching point set, eliminating the spatial bias accumulation problem of unidirectional matching and ensuring a bidirectional correspondence between image feature points and functional and symptom points in key structures. Local affine transformation performs independent calibration of anatomical zones, maintaining the internal structural stability of key units such as the ossicular chain and vestibule, while achieving smooth transitions at zone boundaries. Nonlinear deformation analysis preserves the rigidity of bony regions while adapting to the elasticity of nerve bundles, solving the problem of micro-curvature deformation that traditional rigid registration struggles to adapt to. Multimodal voxel similarity analysis identifies strong response areas with triple correlation between image, symptom, and function, pinpointing the precise spatial location of the core area of ​​lesion changes. A hierarchical iterative optimization mechanism applies rigidity-flexibility differential processing according to the anatomical characteristics of the bony labyrinth, neural canal, and air cell layers, avoiding the destruction of complex structures by a single deformation model.

[0063] In one embodiment, cross-modal correlation calculations are performed between structured features and cross-modal registration data to obtain quantized correlation parameters, including: The structured feature data input dimension alignment engine performs feature space normalization. The modal feature decoupling module separates heterogeneous feature parameters such as bone mineral density gradient values ​​from image structured data, tinnitus severity index from clinical structured data, and auditory nerve response latency from functional structured data. The scale transformation unit performs dimensionless processing: bone parameters are converted to standard values; symptom scores are linearly mapped to the 0-1 interval; and physiological time parameters are logarithmically compressed.

[0064] The feature stitching controller reorganizes features according to spatial affiliation principles: various parameters belonging to the same temporal bone anatomical structure are aggregated into feature tuples. For example, the three parameters of the cochlear basal turn region corresponding to the CT value of the spiral lamina on imaging, the tinnitus symptom intensity coefficient, and the amplitude of the cochlear microphonic potential are combined. The padding mechanism fills in zero values ​​for anatomical structures without matching clinical or functional data to maintain a constant vector dimension. The output uniform-dimensional feature vector has a fixed length, and each element contains multimodal quantified attributes of a specific structure, along with the corresponding temporal bone region code.

[0065] A unified-dimensional feature vector and cross-modal registration data input spatial modeler resolves 3D topological dependencies. The anatomical distance calculation module measures the spatial relationships between structures corresponding to feature vectors: calculating the Euclidean distance vector between the cochlea and vestibule, the relative angles of each component of the ossicular chain, and the position vector product of the facial canal and chorda tympani nerve.

[0066] The dynamic topology analysis unit extracts functional symptom transmission pathways: establishing a temporal correlation between tinnitus symptom intensity and the propagation delay of vestibular aqueduct electrical signals; quantifying the spatial attenuation gradient of hemifacial weakness scores and geniculate ganglion nerve impulse amplitude. The matrix generator integrates static anatomical connectivity and dynamic functional linkages: row / column indices correspond to structure numbers in the feature vectors, and matrix elements store inter-structure correlation strength values, functional transmission coefficients, and symptom influencing factors. The output matrix is ​​a symmetric sparse matrix, where non-zero elements represent structural pairs with significant cross-modal correlations.

[0067] A unified-dimensional feature vector and a cross-modal topological association matrix are input into a tensor synthesizer to perform multi-source information fusion. Neighborhood aggregation is based on matrix connectivity: for each feature vector element, neighboring structural feature values ​​defined by the association matrix are extracted. Modal cross-calculation employs weighted fusion: image feature values ​​and corresponding structural clinical scores are multiplied by weight, and functional time parameters are correlated with symptom intensity through an exponential decay relationship.

[0068] The tensor dimension builder is divided into three layers: the bottom layer is the original feature space; the middle layer records the aggregated features of the same modality (such as the mean of all bony parameters); and the top layer stores cross-modal interaction terms (such as the cross-correlation product of facial nerve canal CT density and facial muscle weakness). The output multimodal joint feature tensor is a three-dimensional array structure, with dimension one indexing the anatomical structure number, dimension two distinguishing the feature type level, and dimension three marking the spatial location coordinates.

[0069] A dual-path convolution engine inputting multimodal joint feature tensors performs heterogeneous information fusion. The spatial path configures a 3D convolution kernel to scan the tensor's bottom layer, extracting spatial distribution patterns of anatomical structures: identifying distance gradient changes between the cochlea and vestibular structures, the ossicular chain alignment vector field, and the spatial interaction patterns between the facial nerve canal and the vascular plexus. The modal path performs one-dimensional convolution along the feature type dimension, capturing cross-modal feature correlation patterns: analyzing the co-fluctuation characteristics of bone density gradient and tinnitus score, and the periodic coupling relationship between neural response delay and vertigo attack frequency.

[0070] The cross-fusion mechanism establishes a dot-product mutual reinforcement connection between the dual-path output feature maps: the tympanic septum location feature map from the spatial path output and the conductive hearing loss score feature map from the modal path are multiplied element-wise to enhance the lesion-related signals in the middle ear abnormality region. The dimensionality reduction unit performs spatial pooling and feature channel compression on the fused feature map, retaining core region data blocks containing significant pathological associations. The output fused association feature map is a two-dimensional matrix, where row vectors represent anatomical structure identifiers and column vectors store the enhanced association strength values ​​after spatial-modal fusion.

[0071] The fusion-related feature map is input into the association parser for pathological quantitative transformation. The association extraction module locates the peak region of the feature map: using the fusion value of the cochlear spiral lamina as the core to detect otosclerosis response clusters, identifying the perilymphatic fistula association area at the highest point of vestibular semicircular canal fusion intensity, and locating the nerve entrapment focus using the facial nerve canal fusion abnormality gradient. The parameterized mapping unit defines three types of diagnostic parameters: structural abnormality parameters calculate the standard deviation multiple of the anatomical region fusion value from the baseline range; functional disharmony parameters measure the Pearson product-moment coefficient of nerve response timing and symptom intensity; and the comprehensive diagnostic index synthesizes the geometrically weighted value of structural abnormality and functional disharmony.

[0072] The parameter calibration process references standard medical diagnostic thresholds: the facial nerve canal entrapment index is mapped to the facial nerve functional grading scale, and the vertigo correlation coefficient is converted into a percentage probability of vestibular dysfunction. The final quantified correlation parameters are output as a structured diagnostic parameter table, containing the target object identifier, correlation parameter type, quantified value, and corresponding anatomical structure code.

[0073] This embodiment achieves scale normalization of imaging bony parameters, symptom scores, and functional temporal indicators through intermodal feature alignment operations, eliminating the correlation calculation bias caused by the dimensional differences of multi-source heterogeneous data. A cross-modal topological correlation matrix accurately characterizes the spatial dependencies and functional conduction paths of various temporal bone anatomical structures, transforming three-dimensional positional information such as cochlear-vestibular distance and neural canal angle into computable correlation weights. A multimodal joint feature tensor integrates original features and cross-modal interaction terms, constructing a fusion feature layer in key dimensions such as facial nerve canal density and facial muscle weakness scores, comprehensively covering the synergistic changes in anatomy, function, and clinical symptoms. A dual-path cross-convolution fusion mechanism captures the topological deformation patterns of the ossicular chain along the spatial dimension and analyzes the correlation between tinnitus intensity and cochlear potential along the modal dimension, achieving complementary verification of spatial distribution and functional features through dot product mutual enhancement.

[0074] In one embodiment, a dual-path cross-convolution fusion is performed on the multimodal joint feature tensor to obtain a fused associated feature map, including: The pre-defined convolutional database stores a dedicated convolutional kernel parameter library for temporal bone bony structures, including core kernel parameter sets for the cochlear spiral plate geometry pattern, ossicular chain spatial configuration template, and semicircular canal curvature feature model. The adapter is initialized to parse the channel dimension composition of the multimodal joint feature tensor and identify feature layers strongly correlated with bony features.

[0075] The kernel parameter matching engine performs a triple mapping for the initialization of the first path convolutional kernels: a radial gradient detection kernel group (3×3×3) is bound to the bone density feature channel; a direction-sensitive convolutional kernel group (5×5×5) is configured for the ossicular chain spatial topology channel; and a multi-directional curvature analysis kernel group (7×7×7) is assigned to the cochlear-vesicle complex channel. The kernel parameter loading process verifies anatomical specificity: cochlear region-specific kernels are forcibly applied to the range of coordinates from the cochlear basal to the apical rotation, while semicircular canal kernels are constrained to the posterior vestibular azimuth angle range. The output bony feature extraction convolutional kernel group includes labels for the scope of each kernel and anatomical applicability verification tags.

[0076] The convolutional database calls upon a soft tissue feature kernel library, containing dynamic kernel parameters for the facial nerve bundle course direction template, vestibular ganglion expansion pattern, and tympanic membrane tension distribution model. The channel correlator locates the functional signal temporal layer, symptom intensity diffusion layer, and vascular density feature layer within the multimodal joint feature tensor.

[0077] Elastic binding for second-path convolutional kernel initialization based on kernel configuration strategy: A deformable neural tracking kernel (reference size 5×5×5, containing 12 control points) is loaded onto the neural conduction direction channel; a thermal gradient-sensitive kernel (reference size 3×3×3, supporting nonlinear stretching) is matched to the symptom dynamic distribution channel; and a fractal dimension analysis kernel (reference size 7×7×7, with porous structure sensing capability) is configured on the microvascular network channel. The kernel deformation degree of freedom parameters are set according to physiological structure. The generated soft tissue feature extraction convolutional kernel group is used to annotate the target anatomical region and its morphological adaptation rules.

[0078] A multimodal joint feature tensor is input into the bony feature extraction channel, and multi-cascaded dilated convolution operations are performed. The scale classification module is configured with a third-order parallel convolutional flow: Microscale level: 3×3×3 dilated convolution kernel (dilation rate 2) scans a specific spiral plate region, extracts anisotropic features, and outputs a feature map with a layer thickness of 0.1mm.

[0079] At a medium scale: 5×5×5 dilated convolution kernels (dilation rate 4) cover joint units to detect joint surface contact angles and gap features, and output a 1mm resolution feature map.

[0080] Macro-scale: 7×7×7 dilated convolution kernel (dilation rate 6) analyzes the overall contour, quantifies the curvature radius consistency parameter, and outputs a 3mm voxel feature map.

[0081] The feature fusion unit integrates and outputs through cross-level skip connections: microscale bone texture features and large-scale morphological features are weighted and superimposed, and the macroscopic positional association information between the incus body microstructure and the stapes footplate is fused in the ossicular chain region. The sequence of bony feature maps is stored in anatomical order, forming a hierarchical response atlas from the microstructure of the bony labyrinth to the composite bony unit, preserving the lesion-sensitive features under spatial topological constraints.

[0082] The multimodal joint feature tensor is loaded into the soft tissue analysis engine, initiating the dynamic convolution process. The deformation field generator calculates the kernel deformation driven by the input data in real time. Facial nerve canal region: The nerve bundle gradient triggers the convolution nucleus to stretch along the long axis of the nerve canal (maximum extension rate 40%), adapting to the curved path from the labyrinthine segment to the tympanic segment.

[0083] Vestibular ganglion area: Fluctuations in lymphatic pressure characteristics guide nuclear topological expansion (radial expansion coefficient 0.3-1.5 times), matching the morphology of endolymphatic hydrops lesions.

[0084] Tensor tympani muscle attachment point: The spatial gradient of symptom intensity drives the displacement of the control point to reconstruct the cone-shaped tension distribution model of the tympanic membrane.

[0085] Post-processing of feature maps performs anatomical rationality verification: path continuity constraints are applied to facial nerve feature maps to eliminate skip artifacts; vestibular feature maps undergo smoothness normalization to suppress high-frequency noise interference. The output soft tissue feature map sequences are stored according to physiological functional areas, forming feature response maps that are highly correlated with the spatial distribution of clinical symptoms.

[0086] The bony and soft tissue feature map sequences are input into a feature interaction furnace to perform a three-dimensional spatial fusion operation. The structural anchoring module establishes fusion base points at bony boundaries: using the cochlear spiral plate feature map as a reference, it aligns the spiral ganglion soft tissue feature map and performs a dot product operation to enforce boundary constraints; using the stapes footplate bony contour as a template, it weights and fuses the tensor tympanic membrane muscle feature map tensor. The pathological response coordination unit detects multimodal associated regions: when the incus-stapedius joint bony feature map shows joint sclerosis and the soft tissue feature map shows muscle attachment point atrophy, a joint feature enhancement coefficient is generated in the middle ear cavity region.

[0087] Channel-level fusion employs a bidirectional attention mechanism: bony paths generate spatial location mask matrices, while soft tissue paths generate feature channel confidence vectors. The final fused associated feature map preserves the high spatial accuracy features of bony structures (such as 0.1mm perforations in the stapes footplate) and the pathological sensitivity characteristics of soft tissues (such as vestibular ganglion edema gradients), and generates anatomical-functional coupling feature values ​​in key regions such as the junction of the facial canal and the bony foramen.

[0088] This embodiment uses a pre-set set of bony feature convolutional kernels to accurately match the complex geometry of the temporal bone, avoiding the recognition bias of general convolutional kernels at bone edges. Deformable soft tissue convolutional kernels dynamically adjust their morphology according to nerve pathways and symptom distribution, solving the problem that traditional fixed convolutional kernels cannot capture physiological deformations. Multi-scale dilated convolutional layers capture bony features at different levels: micro-scale analysis of trabecular bone structure and macro-scale preservation of the overall outline integrity of the semicircular canals, achieving collaborative detection of microscopic lesions and macroscopic morphology. Dual-path output feature maps implement anatomical constraints during the cross-fusion stage: the bony feature map provides precise spatial anchor points, and the soft tissue feature map enhances functional features within the anchor point boundaries, ensuring that the pathological response at the bone-soft tissue junction is both anatomically accurate and reflects functional abnormalities. The final fused feature map simultaneously retains sub-millimeter precision features of bony lesions such as stapes perforation and sensitive gradient features of soft tissue lesions such as nerve entrapment, providing comprehensive structural diagnostic evidence covering both hard and soft tissues for temporal bone diseases.

[0089] In one embodiment, multimodal trend analysis is performed on the quantitative correlation parameters to obtain a quantitative diagnostic report, including: The temporal bone evolution map is input into the abnormal pattern recognition engine to perform multidimensional pathological feature mining. The dynamic threshold model configures adaptive diagnostic criteria based on the time-space dimension: in the acute phase of the disease, a rate of decrease in cochlear bone density gradient exceeding two standard deviations from the baseline value is considered rapid ossification abnormality; in the chronic phase, a persistently higher probability of vestibular dysfunction is marked as permanent vestibular damage.

[0090] Pattern association analysis establishes a cross-modal abnormality chain: when the temporal curve of semicircular canal position deviation and the correlation fluctuation of perilymphatic fistula show a peak simultaneously, a high-risk marker for inner ear fistula is triggered; when the facial nerve entrapment index increases and is accompanied by a sudden increase in the narrowing rate of the bone foramen diameter in the same region, an early warning of nerve entrapment progression is generated. The abnormal data encapsulation unit outputs a combination of three elements: abnormal anatomical structure code, abnormal pattern type, and cross-modal coupling strength value, forming a target potential abnormality information set.

[0091] Potential abnormal information is loaded into the spatial mapper, and anatomical coordinate localization is performed. The structural location matcher calls the standard three-dimensional coordinate database of the temporal bone: cochlear abnormality-related parameters are mapped to the mid-axis coordinates of the cochlear spiral lamina; vestibular abnormality data is bound to the spatial location of the external opening of the vestibular aqueduct; facial nerve abnormality parameters are matched to the coordinate cube of the junction area between the vertical and horizontal segments.

[0092] The pathological gradient diffusion model simulates the spatial influence range of abnormalities: cochlear ossification abnormalities generate an influence sphere with a radius of 5mm from the lesion center, and the bone conduction function attenuation gradient is marked inside; vestibular fistula abnormalities generate a tubular diffusion path along the endolymphatic vessels, and the change value of the vessel diameter expansion rate is marked. The target localization distribution data output is a layered three-dimensional heat map: the basal layer stores the anatomical coordinate matrix, the middle layer records the abnormal intensity distribution, and the top layer marks the interaction index of adjacent abnormal areas.

[0093] The three data inputs form the report generation engine, enabling structured integrated diagnostic operations. The data fusion module cross-validates the temporal features, key parameters of potential abnormal information, and spatial parameters of location distribution in the evolutionary atlas. The diagnostic decision unit applies a clinical pathway rule base: when cochlear ossification abnormalities are combined with an auditory nerve response delay >5ms and an ossification progression rate >0.3mm / month, it outputs the diagnostic entry "Grade III cochlear ossification with nerve compression"; when the spatial overlap between the location distribution area of ​​the vestibular fistula and the frequency of vertigo attacks is >90%, it generates a risk conclusion of "high-risk area for perilymphatic fistula".

[0094] The report formatter organizes content by anatomical section: the cochlea section displays a 3D thermogram of ossification and a time-series curve of functional decline; the vestibular section integrates a tubular model of fistula location and an abnormal spectrum of caloric and thermodynamic tests; the facial nerve section arranges nerve entrapment coordinates and grade prediction values. The final temporal bone analysis report output is a combination of a 3D visualization model with coordinate annotations and a structured diagnostic parameter table.

[0095] This embodiment constructs a target feature vector to focus on core diagnostic parameters such as the cochlear ossification index and the probability of vestibular dysfunction, eliminating interference from irrelevant features and achieving targeted extraction of temporal bone pathological features. A cross-modal temporal alignment mechanism uses an image timeline benchmark to unify symptom fluctuations and functional decline sequences, addressing the problem of fragmented disease progression trajectories caused by differences in follow-up intervals. The temporal bone evolution atlas integrates multi-dimensional dynamic patterns such as structure-symptom time lag correlation and function-structure decline coupling, visually presenting the negative co-evolutionary path of cochlear ossification and nerve compression using a three-dimensional heatmap. Abnormal pattern detection applies staged adaptive thresholds to accurately capture rapid ossification abnormalities in the acute phase and identify permanent vestibular damage in the chronic phase, improving the sensitivity of pathological responses at different disease stages.

[0096] 8. The multimodal data analysis method for diagnosing temporal bone diseases according to claim 7, characterized in that, abnormal pattern detection is performed on the temporal bone evolution map to obtain target potential abnormal information, including: The temporal bone evolution atlas is input into the anatomical layering engine, which performs three-dimensional spatial hierarchical division. The hierarchical structure is constructed based on the anatomical principles of the temporal bone: the bony labyrinth microstructure layer focuses on the cochlear spiral lamina and vestibular microcapsule units, with spatial resolution limited to precise boundaries of 0.1mm; the mesoscopic functional unit layer divides the ossicular chain motor units, vestibular semicircular canal functional groups, and facial nerve canal segmental structures, with spatial scale controlled to 1-3mm functional independent units; the macroscopic composite structure layer integrates the overall contour areas of the cochlea-vestibular complex and the tympanic-mastoid air cell system, covering anatomically integrated units with a scale >5mm.

[0097] The hierarchical boundaries are defined by rigid spatial constraints: the boundary of the bony labyrinth layer is divided by the bony medial wall of the cochlea, and the boundary of the macroscopic layer is naturally isolated by the anatomical plane of the skull base. The division process preserves the hierarchical topological connections, and the ossicular chain incus body is simultaneously mapped to the mesoscopic functional units and the macroscopic middle ear complex. The output multi-level sub-map set contains three layers of independent three-dimensional atlas data, and each sub-map is accompanied by a hierarchical identifier code and a cross-layer mapping index matrix, ensuring that the microstructure of the cochlear basal layer can be traced back to the coordinates of the macroscopic cochlear complex.

[0098] The multi-level subgraph set is loaded into the node quantizer to perform hierarchical differential feature extraction. The node generation strategy follows anatomical functional rules: micro nodes are deployed at key sites such as the spiral lamina of the cochlea and the apex of the ampullary crest of the semicircular canals in the bony labyrinth layer; functional nodes are generated at the rotation center of the incus-stapedius joint, the geniculate ganglion of the facial nerve, and the perilymphatic foramen of the vestibule in the mesoscopic layer; and composite nodes are established at the overall centroid of the cochlea, the projection area of ​​the vestibular nucleus, and the central point of the tympanic tectum in the macroscopic layer.

[0099] Anomaly quantification employs a hierarchical, dedicated calculation strategy: bone labyrinth nodes calculate bone density gradient deviations and implement standard deviation normalization; functional nodes quantify the coefficient of variation of nerve conduction delays using a Gaussian distribution for deviation measurement; composite nodes analyze morphological and volumetric change rates and calculate the percentage difference from the baseline model. Indicator normalization unifies the dimensional system, linearly compressing bone parameters to the [0-1] closed interval and bipolarizing functional parameters to the [-1, 1] range. The output anomaly quantification index table stores node 3D coordinates, hierarchical encoding, anomaly scores, and confidence weights, establishing a quantitative benchmark covering microscopic to macroscopic scales.

[0100] Anomaly metrics are input into the topology association engine to drive cross-level node connection operations. The association rules integrate a dual constraint mechanism: rigid anatomical constraints force the cochlear basal rotation node to establish a connection with the vestibular scala node, and the incus joint node to be bound to the stapes footplate node; flexible functional association is activated based on the neuro-symptom coherence coefficient, and the vestibular ganglion abnormal node and the vertigo symptom temporal node are connected when the coherence is >0.85.

[0101] Anomaly node screening employs a multi-level filtering process: primary threshold screening retains nodes with anomaly scores > 0.8; secondary verification excludes isolated nodes with fewer than 2 associated neighbors; and topological verification ensures that facial nerve nodes are strictly located within the bone canal axis space. The generated set of anomaly nodes records the connection topology between nodes: solid red lines mark bone abnormality conduction chains, indicating the path of cochlear ossification spreading to the ossicular chain; dashed blue lines represent neuro-symptom functional associations, tracing the conduction trajectory from facial nerve entrapment to facial muscle weakness. The set data includes connection strength weights and pathological type labels, constructing a cross-scale abnormal conduction network framework.

[0102] The set of abnormal nodes is input into the spatial clustering engine, which performs density gradient partitioning. Clustering parameters are dynamically configured based on hierarchical characteristics: the bony labyrinth microstructure layer uses a 0.1mm spacing threshold to identify high-density abnormal point clusters in the cochlear spiral lamina corner region; the mesoscopic functional layer sets a 2mm neighborhood radius to merge continuous abnormal nodes in the incus-stapedius joint region to form motor unit abnormal clusters; the macroscopic composite layer applies a 5mm connectivity rule to construct the cochlear-vestibular composite abnormal region. Anatomical constraints are validated during the process: abnormal clusters of the vestibular semicircular canals are prohibited from crossing into the facial nerve canal region, and clusters of the tympanic air cell system must be contained within the boundary of the mastoid bone. The density segmentation algorithm generates three types of pathological regions: calcified core region (node ​​density > 15 points / mm³), transition zone (5-15 points / mm³), and diffuse region (< 5 points / mm³). The output abnormal region distribution map annotates the spatial centroid coordinates, volume, and average abnormality intensity value of each region, forming a pathological distribution heatmap covering the entire temporal bone scale.

[0103] Abnormal region distribution data is loaded into the structural mapper to perform precise matching of anatomical target areas. The coordinate matcher calls the standard temporal bone structural atlas: the centroid coordinates of the calcified core area are matched with the quadrants of the spiral plate of the cochlear base (divided into anterior / posterior / superior / inferior quadrants); the transition zone region is matched with the spatial path of the extravesical lymphatic vessels of the vestibule (located along the three-dimensional curved coordinate axis); the diffusion area is projected onto the tympanic cavity mucosa distribution volume model.

[0104] The pathological-anatomical fusion rules are enforced: ossified areas of the cochlea are only allowed to be mapped to the coordinate range of the bony labyrinth, and the facial nerve entrapment transition zone must be constrained within the space of the facial canal bony wall. The mapping results generate a structured abnormal region mapping table: recording the target structure code (e.g., cochlea CO-B1), abnormal region type (core / transition / diffuse), spatial overlap rate (e.g., 92% overlap between the ossified core area and the anterior quadrant of the spiral lamina), and anatomical matching confidence value.

[0105] Anomaly region mapping is input to the feature integrator, which performs multidimensional pathological feature fusion. The feature aggregation strategy is implemented in a tiered manner. Core region features: Extract the maximum anomaly intensity value and gradient change rate. Transition zone features: Quantize the diffusion direction vector. Diffuse region features: Calculate the spatial volume ratio.

[0106] Cross-regional correlation analysis was used to establish a pathological conduction model: Pearson correlation was established between the calcification intensity of the cochlear core region and the diffusion rate of the vestibular transition zone; an exponential decay function was fitted to the pressure gradient of the facial nerve entrapment core region and the volume change of the middle ear diffusion zone. The feature fusion module outputs a triplet of potential abnormal information: pathological target area encoding, abnormal evolution pattern, and cross-regional correlation coefficient, which is finally encapsulated into machine-resolvable potential abnormal information.

[0107] In this embodiment, the spatial clustering process employs strong anatomical boundary constraints, strictly limiting abnormal areas of the vestibular semicircular canals to the bony labyrinth space, and ensuring that tympanic cavity lesions do not extend beyond the middle ear cavity contour, thus maintaining the anatomical rationality of pathological localization. Abnormal region mapping precisely pinpoints the core lesion area to the anatomical quadrant of the cochlear spiral lamina, and abnormalities in the transition zone are located along neurovascular pathways, achieving substructural-level spatial matching of the lesion target area. The ternary structure output by feature integration simultaneously provides the lesion's spatial encoding, evolutionary pattern type, and cross-regional transmission trend, establishing a comprehensive decision-making basis for clinical intervention, including target localization, pathological nature, and progression risk.

[0108] Reference Figure 2 As shown, the present invention provides a multimodal data analysis device for the diagnosis of temporal bone diseases, and applies the multimodal data analysis method for the diagnosis of temporal bone diseases described in any of the above-mentioned methods, comprising: The acquisition module is used to acquire temporal bone multimodal data, perform modal difference elimination, and obtain structured feature data. The analysis module is used to perform cross-modal spatial registration and target structure mapping on structured feature data to obtain cross-modal registration data. The association module is used to perform cross-modal association calculations between structured features and cross-modal registration data to obtain quantized association parameters. The processing module is used to perform multimodal trend analysis on the quantitative correlation parameters and obtain a temporal bone analysis report.

[0109] This invention provides a multimodal data analysis device for the diagnosis of temporal bone diseases. Through structured feature extraction and difference elimination of multimodal data, it effectively reduces noise interference between different imaging modalities, improves the comparability and reliability of feature data, and provides more accurate basic data for subsequent analysis. By employing cross-modal spatial registration and target structure mapping techniques, it achieves alignment of fine temporal bone structures and precise localization of target structures, enhancing the spatial consistency of multimodal data and reducing diagnostic errors. Through cross-modal correlation calculations, it reveals the multimodal collaborative characteristics of temporal bone lesions, compensating for the shortcomings of traditional single-modal assessment and improving the comprehensiveness and accuracy of disease diagnosis. Based on quantitative correlation parameters, multimodal trend analysis is performed to generate a comprehensive temporal bone analysis report.

[0110] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the system and each module described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0111] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A multimodal data analysis method for the diagnosis of temporal bone diseases, characterized in that, include: Acquire temporal bone multimodal data, perform modal difference elimination, and obtain structured feature data; Cross-modal spatial registration and target structure mapping are performed on the structured feature data to obtain cross-modal registration data; The structured features are correlated with the cross-modal registration data to obtain quantized correlation parameters. Multimodal trend analysis was performed on the quantified correlation parameters to obtain a temporal bone analysis report.

2. The multimodal data analysis method for diagnosing temporal bone diseases according to claim 1, characterized in that, The process of acquiring temporal bone multimodal data, performing modality difference elimination, and obtaining structured feature data includes: The temporal bone multimodal data of the target object is obtained through a preset modal database, and the data is split to obtain image data, symptom assessment data and functional measurement data. Image feature extraction and structural standardization are performed on the image data to obtain structured image data; Clinical features were extracted and clinical data were reconstructed from the symptom assessment data to obtain structured clinical data. The functional measurement data is subjected to noise filtering and waveform time-frequency domain extraction to obtain functional structured data; The structured image data, the structured clinical data, and the structured functional data are integrated to obtain the structured feature data.

3. The multimodal data analysis method for diagnosing temporal bone diseases according to claim 1, characterized in that, The process of performing cross-modal spatial registration and target structure mapping on the structured feature data to obtain cross-modal registration data includes: The structured feature data is subjected to multimodal spatial transformation to obtain registration transformation parameters; Based on the temporal bone multimodal data and the registration transformation parameters, the structured feature data is subjected to rigid registration and non-rigid deformation field optimization to obtain initial registration data; Based on the initial registration data, structural node detection is performed to obtain the target structural feature point set; Cross-modal topological relationship matching is performed on the target structure feature point set to obtain the target structure mapping relationship; The initial registration data is spatially consistent with the target structure mapping relationship to obtain the cross-modal registration data.

4. The multimodal data analysis method for diagnosing temporal bone diseases according to claim 3, characterized in that, The step of adjusting the spatial consistency of the initial registration data according to the target structure mapping relationship to obtain the cross-modal registration data includes: A bidirectional nearest neighbor search is performed on the target structure mapping relationship to obtain a bidirectional matching point set; Local alignment data is obtained by performing a local affine transformation on the initial registration data based on the bidirectional matching point set. Nonlinear deformation analysis is performed on the local alignment data to obtain global deformation correction data; Multimodal voxel intensity similarity analysis was performed on the global deformation correction data to obtain spatially consistent regions; Based on the spatial consistency region, the global deformation correction data is iteratively optimized layer by layer to obtain intermediate registration results; Boundary constraints and spatial adjustments are applied to the intermediate registration results to obtain the cross-modal registration data.

5. The multimodal data analysis method for diagnosing temporal bone diseases according to claim 1, characterized in that, The step of performing cross-modal correlation calculations between the structured features and the cross-modal registration data to obtain quantized correlation parameters includes: The structured feature data is aligned between modal features to obtain a unified dimension feature vector; Based on the unified dimension feature vector, spatial topological relationships are constructed on the registered data to obtain a cross-modal topological correlation matrix; Based on the cross-modal topological correlation matrix, feature fusion calculation is performed on the unified dimension feature vector to obtain a multimodal joint feature tensor; The multimodal joint feature tensor is fused using a dual-path cross-convolution to obtain a fused associated feature map; The fused correlation feature map is subjected to cross-modal correlation extraction and parameterized mapping to obtain the quantized correlation parameters.

6. The multimodal data analysis method for diagnosing temporal bone diseases according to claim 5, characterized in that, The step of performing dual-path cross-convolution fusion on the multimodal joint feature tensor to obtain a fused associated feature map includes: The multimodal joint feature tensor is initialized with a first path convolution kernel according to a preset convolution database to generate a set of bone feature extraction convolution kernels. The multimodal joint feature tensor is initialized with a second path convolution kernel based on the convolution database to generate a soft tissue feature extraction convolution kernel group; Based on the bone feature extraction convolution kernel group, multi-scale dilated convolution is performed on the multimodal joint feature tensor to obtain a bone feature map sequence; Based on the soft tissue feature extraction convolution kernel group, deformable convolution is performed on the multimodal joint feature tensor to obtain a soft tissue feature map sequence; The bony feature map sequence and the soft tissue feature map sequence are subjected to dual-path cross-fusion to obtain the fused associated feature map.

7. The multimodal data analysis method for diagnosing temporal bone diseases according to claim 1, characterized in that, The step of performing multimodal trend analysis on the quantified correlation parameters to obtain a quantified diagnostic report includes: Temporal bone target features are extracted from the quantized correlation parameters to obtain the target feature vector; The target feature vector is subjected to cross-modal temporal alignment to obtain a synchronized target sequence; Based on the synchronized target sequence, dynamic correlations between modes are constructed to obtain a temporal bone evolution map; Abnormal pattern detection was performed on the temporal bone evolution map to obtain potential abnormal information of the target. Based on the potential anomaly information of the target, temporal bone target structure mapping and matching are performed to obtain target location distribution data; Based on the temporal bone evolution map, the target potential anomaly information, and the target location distribution data, a report quantification report is constructed to obtain a temporal bone analysis report.

8. The multimodal data analysis method for diagnosing temporal bone diseases according to claim 7, characterized in that, The abnormal pattern detection of the temporal bone evolution map to obtain target potential abnormal information includes: The temporal bone evolution map is hierarchically divided to obtain a multi-level sub-map set; Node feature calculations are performed on the multi-level subgraph set to obtain anomaly quantification indicators; Based on the aforementioned anomaly metric, nodes are associated to obtain a set of anomaly nodes; Spatial clustering is performed on the set of abnormal nodes to obtain the distribution of abnormal regions; Based on the distribution of the abnormal regions, target structure matching is performed to obtain the abnormal region mapping; Feature integration is performed on the abnormal region mapping to obtain potential abnormal information of the target.

9. A multimodal data analysis device for the diagnosis of temporal bone diseases, characterized in that, The multimodal data analysis method for diagnosing temporal bone diseases according to any one of claims 1-8 includes: The acquisition module is used to acquire temporal bone multimodal data, perform modal difference elimination, and obtain structured feature data. The analysis module is used to perform cross-modal spatial registration and target structure mapping on the structured feature data to obtain cross-modal registration data; The association module is used to perform cross-modal association calculations between the structured features and the cross-modal registration data to obtain quantized association parameters; The processing module is used to perform multimodal trend analysis on the quantified correlation parameters to obtain a temporal bone analysis report.

Images