A multi-modal biosensing fusion optical detection health monitoring method

Abstract

This invention discloses a health monitoring method using multimodal biosensor fusion optical detection, relating to the field of medical diagnostic technology. The specific steps of this method are: data acquisition and localization, intent modeling, spatial registration, weighted fusion, and diagnostic output. This invention introduces a spatial coding mechanism into the health monitoring process using multimodal biosensor fusion optical detection. It aligns the acquisition positions of the biosensor unit and the optical detection unit with the human anatomical structure to generate spatial coordinate codes, which are then bound to the acquired multimodal data. A spatial mapping matrix is ​​then constructed to perform spatial resampling and registration on the multimodal data with spatial coordinate codes. Combined with a spatial attention mechanism, the spatial features of the multimodal data are optimized, ensuring that the biosensor data and the optical detection data maintain consistency in the spatial dimension of the human anatomical structure. This achieves spatially constrained fusion of multimodal data, allowing the integration of biosensor and optical detection technologies to be based on the actual human anatomical structure.

Description

Technical Field

[0001] This invention relates to the field of medical diagnostic technology, specifically to a health monitoring method using multimodal biosensor fusion optical detection. Background Technology

[0002] With the continuous upgrading of residents' health management and clinical diagnosis and treatment needs, single monitoring technologies are no longer sufficient to meet the requirements of comprehensive and accurate perception of human health status. The medical and health monitoring field urgently needs multi-technology collaborative monitoring methods. In order to obtain richer human health information, the industry is gradually exploring multimodal monitoring models that combine different monitoring technologies, in order to provide more comprehensive data support for disease diagnosis and health assessment. Multimodal fusion health monitoring technology has also become an important research and development direction in this field.

[0003] However, while existing technologies attempt to combine biosensing with optical detection techniques for multimodal health monitoring, biosensing includes types such as electrochemical sensing, piezoelectric sensing, and immunosensing, collecting biosensor data including human physiological electrical signals, body fluid composition, and tissue metabolic signals—data that can reflect metabolic, component, and functional abnormalities. Optical detection includes Raman scattering, fluorescence imaging, and near-infrared transmission, collecting optical detection data including human tissue molecular characteristics, cell morphology, and blood vessel distribution—data that can reflect lesion morphology.

[0004] However, existing technologies lack effective solutions that combine both approaches, and still suffer from many key shortcomings: (1) Spatial misalignment problem: The biosensing data and optical detection data are not aligned in space, which makes it impossible for the functional signals and morphological signals of the same anatomical location to correspond accurately during fusion, affecting the diagnostic accuracy; (2) Poor scenario adaptability: The same fusion strategy is used for different diagnostic scenarios (diagnosis of acute and chronic diseases, early screening, daily monitoring, etc.), and the contribution weight of each modality data cannot be dynamically adjusted according to the actual urgency. (3) Insufficient interpretability: The fusion results lack quantitative explanation of the contribution of each modality feature, making it difficult for clinical diagnostic personnel to understand the basis for decision-making. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a health monitoring method based on multimodal biosensing fusion optical detection. It constructs a complete technical solution for data acquisition, spatial alignment, adaptive fusion, and interpretable diagnosis, and solves three major technical problems: spatial misalignment of multimodal data, poor adaptability of fusion strategies to different scenarios, and insufficient interpretability of diagnostic results. It has significant technological advancements and broad application prospects in the field of health monitoring, especially in the field of clinical auxiliary diagnosis.

[0006] To achieve this objective, the present invention adopts the following technical solution.

[0007] This invention discloses a health monitoring method using multimodal biosensor fusion optical detection, comprising the following steps: Data Acquisition and Positioning: Deploy biosensing units and optical detection units to acquire human multimodal data, and introduce a spatial coding mechanism to align the acquisition location with human anatomical structures to generate spatial coordinate codes. Then, bind the spatial coordinate codes with the human multimodal data to form multimodal data with spatial coordinate codes. Spatial registration: Construct a spatial mapping matrix, perform spatial resampling and registration on the multimodal data with spatial coordinate encoding, and complete the spatial feature optimization processing of the multimodal data with spatial coordinate encoding through a spatial attention mechanism; Weighted fusion: Multimodal features are extracted from the spatially optimized multimodal data with spatial coordinate encoding, the fusion weight of each modality is calculated, and the multimodal features are dynamically fused according to the fusion weight to obtain the fused multimodal features; Diagnostic output: Perform feature association analysis on the fused multimodal features, mine the intrinsic correlation features between different modal features, complete diagnostic reasoning, and output the diagnostic results.

[0008] Preferably, in the acquisition and positioning process, the human multimodal data includes biosensor data and optical detection data; the spatial coding mechanism specifically involves: constructing a spatial coordinate system for the standard anatomical structure of the human body, calibrating each acquisition position of the biosensor unit and the optical detection unit, determining the corresponding coordinate value of each acquisition position in the spatial coordinate system as the spatial coordinate code, and then binding the spatial coordinate code of each acquisition position with the corresponding acquired data one by one.

[0009] Preferably, the method further includes an intent modeling step, which is performed after the acquisition and localization and before the spatial registration. Specifically, the intent modeling involves: extracting the biosensing data from the multimodal data with spatial coordinate encoding, determining a diagnostic intent vector based on the current diagnostic scenario, and using the diagnostic intent vector to adjust the feature weights and fusion weights of each modality in the spatial registration and weight fusion.

[0010] Preferably, the biosensing unit includes one or more types of electrochemical sensing, piezoelectric sensing, and immunosensing, and the collected biosensing data includes one or more types of human physiological electrical signals, body fluid components, and tissue metabolic signals; the optical detection unit includes one or more types of near-infrared transmission, Raman scattering, and fluorescence imaging, and the collected optical detection data includes one or more types of optical data of human tissue molecular characteristics, cell morphology, and blood vessel distribution.

[0011] Preferably, in the spatial registration, the spatial mapping matrix is ​​constructed based on the spatial coordinate system of the standard human anatomical structure. Spatial resampling and registration are performed with the goal of unifying the spatial resolution of different modal data, so that the multimodal data with spatial coordinate encoding remains consistent in the spatial dimension of the human anatomical structure. The spatial attention mechanism constructs an attention weight matrix, and the optical feature weight values ​​in the attention weight matrix are the weight values ​​assigned to each feature of the optical detection data. The spatial attention mechanism dynamically adjusts the optical feature weight values ​​to complete the spatial feature optimization processing.

[0012] Preferably, in the intent modeling, the diagnostic intent vector includes two dimensions: urgency and interpretability requirement. The urgency is obtained by calculating the urgency quantification value using the urgency quantification formula, and the interpretability requirement is divided into multiple quantification levels according to the actual degree of need. Both the urgency quantification value and the interpretability requirement quantification value are assigned based on the clinical scenario knowledge base.

[0013] Preferably, the urgency quantification calculation formula is as follows: , Where E is the urgency quantification value, E b This serves as the base score for the urgency of the diagnostic scenario. The clinical weight coefficient for the nth biosensor data item. E represents the single-index urgency fluctuation score of the nth biosensor data item. b and All were retrieved from the clinical scenario knowledge base. It is derived from the degree of deviation between the corresponding biosensor data item and the normal reference range.

[0014] Preferably, the spatial attention mechanism dynamically adjusts the optical feature weight values ​​based on the urgency quantification: in high-urgency scenarios, higher optical feature weight values ​​are assigned to features in the optical detection data that are related to lesion morphology and boundaries; in medium-urgency and low-urgency scenarios, the basic assignment of the optical feature weight values ​​is used to assign various features of the optical detection data.

[0015] Preferably, in the weight fusion, the multimodal feature extraction method is to perform layered extraction of shallow and deep features on the multimodal data with spatial coordinate encoding after spatial feature optimization, and then integrate and splice the shallow and deep features to obtain the multimodal features; the fusion weight is calculated by calculating the fusion weight of optical detection data and biosensing data through the modal fusion weight calculation formula.

[0016] Preferably, the modal fusion weight calculation formula is as follows: , in, Light is used as the fusion weight for optical detection data. As a fusion weight for biosensor data, As the basic fusion weights for optical detection data, This is the urgency-weighted adjustment coefficient. Here, γ is the urgency quantification value, and γ is the interpretability demand weighting adjustment coefficient. This is the quantified value of the interpretability requirement.

[0017] Preferably, in the diagnostic output, a two-layer analysis of spatial correlation and statistical correlation is performed on the fused multimodal features to explore the intrinsic correlation features between different modal features. Based on the feature quantification values ​​obtained from the analysis, diagnostic reasoning is completed, and a diagnostic result including disease type, severity and development trend is output. An interpretable diagnostic report including feature contribution descriptions of key anatomical locations is also generated.

[0018] Preferably, the feature contribution description is obtained through a feature contribution metric calculation formula, which is: , in, The feature contribution quantification value for the i-th key anatomical location. The spatial attention weight coefficient for the i-th key anatomical location. Light is the fusion weight for the optical detection data. Let be the quantized value of the optical detection feature at the i-th key anatomical location. The weights for the fusion of the aforementioned biosensor data are generated. This represents the quantified value of the biosensor characteristics at the i-th key anatomical location.

[0019] The present invention has the following beneficial effects: (1) This invention introduces a spatial coding mechanism into the health monitoring process of multimodal biosensing fusion optical detection. The acquisition positions of the biosensing unit and the optical detection unit are aligned with the human anatomical structure to generate spatial coordinate codes, which are then bound to the acquired multimodal data. A spatial mapping matrix is ​​then constructed to perform spatial resampling and registration on the multimodal data with spatial coordinate codes. Combined with a spatial attention mechanism, the spatial features of the multimodal data are optimized, so that the biosensing data and the optical detection data are consistent in the spatial dimension of the human anatomical structure. This achieves spatially constrained fusion of multimodal data, allowing the integration of biosensing and optical detection technologies to be carried out based on the actual human anatomical structure, which is in line with the physiological reality of health monitoring.

[0020] (2) This invention constructs a diagnostic intent vector by combining clinical diagnostic scenarios in health monitoring with multimodal biosensing and optical detection. It extracts relevant data from multimodal data with spatial coordinate encoding, calculates urgency and determines interpretability requirements, completes quantitative assignment based on clinical scenario knowledge base, calculates the fusion weight of biosensing and optical detection data based on diagnostic intent vector and completes dynamic fusion, and finally completes diagnostic reasoning through two-layer analysis of feature association mining and generates an interpretable diagnostic report with key anatomical location feature contribution description. It integrates the actual needs of clinical scenarios into the entire process of multimodal fusion of biosensing and optical detection, so that the fusion of the two technologies can be adapted to different clinical monitoring scenarios, and at the same time, the fused diagnostic results have clear feature basis support. Attached Figure Description

[0021] Figure 1 A flowchart of a health monitoring method that integrates multimodal biosensing and optical detection; Figure 2 This is a framework diagram for modeling the intent in a health monitoring method that integrates multimodal biosensing and optical detection. Detailed Implementation

[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0023] like Figure 1 As shown, the present invention provides a health monitoring method for multimodal biosensor fusion optical detection, which includes the following steps: Step S1: Data Acquisition and Positioning: Deploy biosensing units and optical detection units to acquire human multimodal data, and introduce a spatial coding mechanism to align the acquisition location with the human anatomical structure to generate spatial coordinate codes. Then, bind the spatial coordinate codes with the human multimodal data to form multimodal data with spatial coordinate codes. Step S2 Intent Modeling: Extract the biosensing data from the multimodal data with spatial coordinate encoding, determine the diagnostic intent vector according to the current diagnostic scenario, and use the diagnostic intent vector to adjust the feature weights and fusion weights of each modality data in the spatial registration and weight fusion.

[0024] Step S3 Spatial Registration: Construct a spatial mapping matrix, perform spatial resampling and registration on the multimodal data with spatial coordinate encoding, and complete the spatial feature optimization processing of the multimodal data with spatial coordinate encoding through a spatial attention mechanism; Step S4 Weight Fusion: Multimodal feature extraction is performed on the multimodal data with spatial coordinate encoding after spatial feature optimization, the fusion weight of each modality is calculated, and the multimodal features are dynamically fused according to the fusion weight to obtain the fused multimodal features; Step S5 Diagnostic Output: Perform feature association analysis on the fused multimodal features, mine the intrinsic correlation features between different modal features, complete diagnostic reasoning, and output the diagnostic results.

[0025] The biosensing unit includes one or more types of electrochemical sensing, piezoelectric sensing, and immunosensing, and the collected biosensing data includes one or more types of human physiological electrical signals, body fluid components, and tissue metabolic signals; the optical detection unit includes one or more types of near-infrared transmission, Raman scattering, and fluorescence imaging, and the collected optical detection data includes one or more types of optical data of human tissue molecular characteristics, cell morphology, and blood vessel distribution.

[0026] In step S1, the human multimodal data includes biosensor data and optical detection data. The spatial coding mechanism is as follows: a spatial coordinate system of the standard human anatomical structure is constructed, the acquisition positions of the biosensor unit and the optical detection unit are calibrated, the corresponding coordinate values ​​of each acquisition position in the spatial coordinate system are determined as the spatial coordinate code, and then the spatial coordinate code of each acquisition position is bound one by one with the corresponding acquired data.

[0027] In step S3, spatial registration, the spatial mapping matrix is ​​constructed based on the spatial coordinate system of the standard human anatomical structure. Spatial resampling and registration are performed to achieve a unified spatial resolution for different modal data, ensuring that the multimodal data with spatial coordinate encoding remains consistent in the spatial dimension of the human anatomical structure. The spatial attention mechanism constructs an attention weight matrix, where the optical feature weight values ​​are weight values ​​assigned to each feature of the optical detection data. The spatial attention mechanism dynamically adjusts the optical feature weight values ​​to complete the spatial feature optimization process.

[0028] In step S2, intent modeling, the diagnostic intent vector includes two dimensions: urgency and interpretability requirement. The urgency is calculated using an urgency quantification formula, and the interpretability requirement is divided into multiple quantification levels based on the actual degree of need. Both the urgency quantification value and the interpretability requirement quantification value are assigned based on a clinical scenario knowledge base.

[0029] The formula for calculating the urgency quantification is as follows: , Where E is the urgency quantification value, E b This serves as the base score for the urgency of the diagnostic scenario. The clinical weight coefficient for the nth biosensor data item. E represents the single-index urgency fluctuation score of the nth biosensor data item. b and All were retrieved from the clinical scenario knowledge base. It is derived from the degree of deviation between the corresponding biosensor data item and the normal reference range.

[0030] The spatial attention mechanism dynamically adjusts the optical feature weight values ​​based on the urgency quantification: in high-urgency scenarios, higher optical feature weight values ​​are assigned to features in the optical detection data that are related to lesion morphology and boundaries; in medium-urgency and low-urgency scenarios, the basic assignment of the optical feature weight values ​​is used to assign various features of the optical detection data.

[0031] In the weighted fusion, the multimodal features are extracted by performing layered extraction of shallow and deep features on the multimodal data with spatial coordinate encoding after spatial feature optimization, and then integrating and splicing the shallow and deep features to obtain the multimodal features; the fusion weight is calculated by calculating the fusion weight of optical detection data and biosensing data through the modal fusion weight calculation formula.

[0032] The formula for calculating the modality fusion weights is as follows: , in, Light is used as the fusion weight for optical detection data. As a fusion weight for biosensor data, As the basic fusion weights for optical detection data, This is the urgency-weighted adjustment coefficient. Here, γ is the urgency quantification value, and γ is the interpretability demand weighting adjustment coefficient. This is the quantified value of the interpretability requirement.

[0033] In the diagnostic output of step S5, a two-layer analysis of spatial correlation and statistical correlation is performed on the fused multimodal features to explore the intrinsic correlation features between different modal features. Based on the feature quantification values ​​obtained from the analysis, diagnostic reasoning is completed, and a diagnostic result containing disease type, severity and development trend is output. An interpretable diagnostic report containing feature contribution descriptions of key anatomical locations is also generated.

[0034] The feature contribution description is obtained through a feature contribution metric calculation formula, which is: , in, The feature contribution quantification value for the i-th key anatomical location. The spatial attention weight coefficient for the i-th key anatomical location. Light is the fusion weight for the optical detection data. Let be the quantized value of the optical detection feature at the i-th key anatomical location. The weights for the fusion of the aforementioned biosensor data are generated. This represents the quantified value of the biosensor characteristics at the i-th key anatomical location.

[0035] The technical solution of this application achieves precise spatial alignment of multimodal data through a spatial coding mechanism, eliminating spatial misalignment between multimodal data from the data source, and enabling biosensing data (functional signals) and optical detection data (morphological signals) to correspond precisely in the same anatomical coordinates, providing a reliable spatial constraint basis for subsequent fusion and significantly improving the accuracy of diagnostic results.

[0036] The diagnostic intent vector enables scenario-adaptive fusion strategies. This allows the fusion strategy to automatically adjust according to the actual needs of the diagnostic scenario—strengthening the weight of optical features related to lesion morphology and boundaries in high-urgency scenarios, and adopting a balanced fusion strategy in low-urgency scenarios. This achieves "scenario-aware" adaptive fusion, significantly improving the intelligence level and clinical applicability of the method.

[0037] By using a spatial attention mechanism to achieve dynamic optimization of optical features, the problem of fixed feature weights in traditional spatial registration, which fails to highlight key lesion areas, is solved. This enables the core morphology and boundary features of lesions in high-urgency scenarios to be highlighted and enhanced in optical detection data, effectively improving the ability of subsequent feature fusion to capture key information about lesions.

[0038] The modal fusion weight calculation formula enables the quantitative controllability of fusion weights, transforming the determination process of fusion weights from empirical setting to quantitative calculation based on clinical knowledge. This makes the fusion results reproducible and verifiable, while providing a unified calculation framework for weight configuration in different clinical scenarios, facilitating the clinical promotion and standard implementation of the method.

[0039] The interpretability of diagnostic results is achieved by quantifying the contribution of features. In the health monitoring method based on multimodal fusion, the contribution of each modality feature to the diagnostic conclusion is quantitatively explained, enabling clinical diagnostic personnel to clearly understand the basis for the diagnostic results. This meets the strict requirements for diagnostic interpretability in high-urgency clinical scenarios and greatly improves the clinical acceptability and trustworthiness of the method.

[0040] By improving the comprehensiveness and accuracy of diagnostic reasoning through a two-layer analysis architecture, spatial correlation analysis ensures the anatomical rationality of diagnostic results in the spatial dimension, while statistical correlation analysis ensures the statistical reliability of diagnostic results in the feature dimension. The two-layer analysis architecture makes the diagnostic reasoning process more comprehensive and rigorous, effectively reducing the risk of misdiagnosis.

[0041] The following is a specific example to further illustrate this: In a hospital endocrinology department, a special health monitoring scenario for acute and chronic pancreatic diseases was conducted on suspected diabetic patients. This monitoring scenario falls under the category of acute and chronic disease diagnosis and is a high-urgency scenario. The key anatomical location monitored is the distribution area of ​​the pancreatic islets. This monitoring adopted a non-invasive multimodal biosensor fusion optical detection method. The entire process revolved around the collection, processing, fusion, and diagnosis of health data from the pancreatic islet distribution area, ultimately providing clinicians with accurate diagnostic results for pancreatic diseases and traceable diagnostic evidence.

[0042] Biosensing units and optical detection units are deployed on the body surface corresponding to the distribution area of ​​the pancreas islets. The biosensing units, in the form of adhesive-type sensing probes, simultaneously acquire physiological electrical signals, body fluid components, and tissue metabolic signals in this area through electrochemical sensing, piezoelectric sensing, and immunosensing. The optical detection units employ a combination of non-invasive near-infrared scanning, Raman scattering imaging, and fluorescence imaging to acquire optical data on the molecular characteristics, cell morphology, and blood vessel distribution of human tissue in this area. These two types of data together constitute multimodal human data, such as… Figure 1 As shown, the multi-dimensional acquisition method can comprehensively capture the physiological metabolism and tissue morphology information of the pancreatic islet distribution area, laying a complete data foundation for subsequent diagnostic analysis. First, a spatial coordinate system of the standard human anatomical structure is constructed. The spatial positions of each patch probe of the biosensor unit and each scanning acquisition point of the optical detection unit corresponding to the pancreatic islet distribution area are precisely calibrated. The corresponding coordinate values ​​of each acquisition position in the spatial coordinate system are determined, and these coordinate values ​​are the spatial coordinate codes. Then, the spatial coordinate codes of each acquisition position are bound one-to-one with the corresponding acquired human multimodal data to form multimodal data with spatial coordinate codes, such as... Figure 2 As shown, this operation enables a precise spatial correspondence between the location of pancreatic lesions captured in optical detection data and the pancreatic metabolic signals detected in biosensor data, thus avoiding diagnostic biases caused by spatial misalignment of multimodal data from the source.

[0043] Biosensor data was extracted from multimodal data with spatial coordinate encoding as relevant data. Considering the high-urgency scenario of this acute and chronic disease diagnosis, the urgency of the extracted biosensor data was calculated. The corresponding quantified urgency value was obtained using the quantification formula: , Where E is the urgency quantification value, E b This serves as the base score for the urgency of the diagnostic scenario. The clinical weight coefficient for the nth biosensor data item. E represents the single-index urgency fluctuation score of the nth biosensor data item. b and All were retrieved from the clinical scenario knowledge base. This is derived from the degree of deviation between the corresponding biosensor data item and the normal reference range. Differential adaptations can be made for different populations: for example, for high-risk groups of cardiovascular disease with high-salt diets, the clinical weight coefficient of biosensor data items can be specifically increased; for people who frequently stay up late, the normal reference range of related indicators such as insulin can be specifically adjusted, and the mapping relationship of the urgency fluctuation score of single indicators can be optimized. This is derived from the deviation of the corresponding biosensor data item from the normal reference range. Simultaneously, considering the actual needs of endocrinology clinical diagnosis and treatment, a high-level interpretability requirement is determined, requiring detailed interpretability evidence for this diagnosis. The urgency and interpretability requirements are quantified and assigned using a clinical scenario knowledge base. This knowledge base pre-stores matching rules for different diagnostic scenarios and urgency baseline scores, assignment rules for clinical weight coefficients of biosensor data items, quantitative grading rules for interpretability requirements, assignment rules for the basic fusion weights of optical detection data under different diagnostic scenarios, and urgency and interpretability requirement weight adjustment coefficients. The matching rules for data types; this clinical scenario knowledge base is built based on multi-center clinical diagnosis and treatment data from the Department of Endocrinology. Among them, the clinical weight coefficient assignment rules for biosensor data are as follows: For the target diagnostic scenario, biosensor data items related to the disease diagnosis outcome are selected, and the corresponding clinical weight coefficients are determined based on the contribution of each data item to the diagnostic result; the single-indicator urgency floating score mapping rules are as follows: Based on the correlation between the abnormality of indicators and the risk of disease in clinical data, a mapping relationship is established between the degree of deviation of biosensor data from the normal range and the single-indicator urgency floating score, achieving standardized quantification of the degree of abnormality; the interpretability requirement quantification and grading rules are as follows: based on internal... Clinical diagnostic data in urology is categorized into quantitative levels and matched with corresponding quantitative values ​​based on the clinical traceability requirements of diagnostic results, adapting to the interpretability output needs of different scenarios. The basic fusion weighting rules for optical detection data are as follows: for target diagnostic scenarios, the diagnostic efficacy of optical detection data and biosensor data for the target disease is verified, and the basic fusion weight of optical detection data is determined based on the diagnostic contribution ratio of the two types of data in the corresponding scenario. The weight adjustment coefficient matching rules are established based on the correlation between clinical scenario needs and the diagnostic weights of optical detection data and biosensor data, establishing a matching between the urgency of the diagnostic scenario, the level of interpretability requirement, and the corresponding weight adjustment coefficients. The rules enable dynamic adaptation of fusion weights to clinical needs; the rules for dynamic adjustment of optical feature weights are as follows: optical features related to the target lesion features are clearly defined, and their basic weights are higher than those of general texture features; at the same time, a positive correlation mapping rule between scene urgency and feature weight adjustment magnitude is established to match the clinical need for precise lesion localization; interpretability requirements are divided into corresponding quantitative values ​​according to high levels, and the quantitative values ​​of the two dimensions are combined to construct a diagnostic intent vector. This vector can accurately transform high-urgency clinical diagnostic needs and high interpretability needs into quantitative parameters that the algorithm can recognize, so that all subsequent data processing links are carried out around the actual clinical diagnosis and treatment needs.

[0044] A spatial mapping matrix is ​​constructed based on the spatial coordinate system of standard human anatomical structures. The standard for achieving spatial resolution unification between biosensor data and optical detection data is to resample and register multimodal data with spatial coordinate encoding. This operation ensures high consistency of the multimodal data with spatial coordinate encoding across the spatial dimensions of the human anatomical structures corresponding to the spatial coordinate system of standard human anatomical structures, allowing data from different acquisition methods to be fused and analyzed in the same spatial dimension. After spatial resampling and registration, an attention weight matrix is ​​constructed using a spatial attention mechanism. The optical feature weight values ​​in this matrix are weights assigned to each feature of the optical detection data. The basic assignment of these optical feature weight values ​​relies on clinical... The scene knowledge base is determined, and the spatial attention mechanism dynamically adjusts the optical feature weight values ​​based on the urgency quantification value corresponding to this high-urgency scene. Higher optical feature weight values ​​are assigned to features in the optical detection data that belong to human tissue molecular features, cell morphology, blood vessel distribution, and are related to the morphology and boundary of pancreatic lesions. These higher optical feature weight values ​​are determined by adjusting the urgency quantification value based on the basic assignment. Through such allocation and dynamic adjustment of optical feature weight values, spatial feature optimization processing of multimodal data with spatial coordinate encoding is completed. This allows the core morphology and boundary features of pancreatic lesions to be highlighted and enhanced in the optical detection data, enabling subsequent feature fusion to accurately capture key information of lesions and improve the accuracy of lesion identification.

[0045] Multimodal feature extraction is performed on multimodal data with spatial coordinate encoding after spatial feature optimization. First, shallow and deep features are extracted hierarchically. Shallow features represent the basic texture and numerical features of the multimodal data, while deep features represent the core correlation features. Then, the shallow and deep features are integrated and concatenated to obtain the multimodal features. This hierarchical extraction method fully explores the surface-level intuitive information and deep-level correlations of the multimodal data, avoiding information omissions caused by single feature extraction. Shallow features can be extracted using statistical calculations, while deep features can be extracted using deep learning models such as convolutional neural networks. Considering the temporal characteristics of biosensing data and the spatial characteristics of optical detection data, a dual-channel network structure can be used to extract deep features separately. Finally, the shallow and deep features are dimensionally matched and then concatenated. Based on the constructed diagnostic intent vector and the extracted multimodal features, the fusion weight of the optical detection data and biosensing data is calculated using the modality fusion weight calculation formula: ,in, For the fusion weights of optical detection data, For the fusion weights of biosensor data, As the basic fusion weights for optical detection data, This is the urgency-weighted adjustment coefficient. For emergency quantification, This is the weighting factor for interpretability requirements. This is a quantified value for the interpretability requirement. , , All features are determined by a clinical scenario knowledge base. Based on the calculated fusion weights of each modality, the multimodal features are dynamically fused to obtain the fused multimodal features. The dynamic fusion method based on the diagnostic intent vector allows the fused features to fully meet the diagnostic needs of high-urgency scenarios, and allows the enhanced lesion optical features to complement the metabolic-related biosensing features, thereby improving the diagnostic value of the fused features.

[0046] Feature association mining analysis is performed on the fused multimodal features. First, a two-layer analysis of spatial correlation and statistical correlation is conducted on the fused multimodal features to uncover the intrinsic correlation features between different modal features. Then, after extraction and quantitative analysis, the biosensor feature quantification values ​​and optical detection feature quantification values ​​of the pancreatic islet distribution area are obtained. The two-layer analysis method can accurately uncover the intrinsic correlation between the biosensor data and optical detection data of the pancreatic islet distribution area, providing clear quantitative data support for diagnostic reasoning. Based on the extracted feature quantification values, the diagnostic reasoning of pancreatic health status is completed, and the diagnostic results including diabetes-related disease types, lesion severity, and disease progression trends are output. Then, based on the two-layer analysis process of diagnostic reasoning and the feature quantification values, an interpretable diagnostic report is generated. This report includes a description of the feature contribution of the key anatomical location of the pancreatic islet distribution area and the degree of support of different modal data for the diagnostic conclusion. The feature contribution description is obtained by obtaining the corresponding feature contribution quantification value through the feature contribution quantification calculation formula. The feature contribution quantification calculation formula is as follows: ,in, For the first The feature contribution of each key anatomical location is quantified numerically. For the first Spatial attention weight coefficients for key anatomical locations are extracted from the constructed attention weight matrix. For the fusion weights of optical detection data, For the first Quantitative values ​​of optical detection features at key anatomical locations For the fusion weights of biosensor data, For the first The system quantifies the biosensor characteristics of key anatomical locations; the generated interpretable diagnostic reports clearly demonstrate the contribution of each characteristic to the diagnostic results and the supporting role of different modal data, enabling clinicians to quickly grasp the core evidence of pancreatic lesions and significantly improve the efficiency and accuracy of clinical diagnosis and treatment in endocrinology.

Claims

1. A health monitoring method using multimodal biosensor fusion optical detection, characterized in that, The method includes the following steps: Data Acquisition and Positioning: Deploy biosensing units and optical detection units to acquire human multimodal data, and introduce a spatial coding mechanism to align the acquisition location with human anatomical structures to generate spatial coordinate codes. Then, bind the spatial coordinate codes with the human multimodal data to form multimodal data with spatial coordinate codes. Spatial registration: Construct a spatial mapping matrix, perform spatial resampling and registration on the multimodal data with spatial coordinate encoding, and complete the spatial feature optimization processing of the multimodal data with spatial coordinate encoding through a spatial attention mechanism; Weighted fusion: Multimodal features are extracted from the spatially optimized multimodal data with spatial coordinate encoding, the fusion weight of each modality is calculated, and the multimodal features are dynamically fused according to the fusion weight to obtain the fused multimodal features; Diagnostic output: Perform feature association analysis on the fused multimodal features, mine the intrinsic correlation features between different modal features, complete diagnostic reasoning, and output the diagnostic results.

2. The health monitoring method for multimodal biosensor fusion optical detection according to claim 1, characterized in that, In the acquisition and positioning process, the human multimodal data includes biosensor data and optical detection data; the spatial coding mechanism specifically involves: constructing a spatial coordinate system for the standard anatomical structure of the human body, calibrating each acquisition position of the biosensor unit and the optical detection unit, determining the corresponding coordinate value of each acquisition position in the spatial coordinate system as the spatial coordinate code, and then binding the spatial coordinate code of each acquisition position with the corresponding acquired data one by one.

3. The health monitoring method for multimodal biosensor fusion optical detection according to claim 1, characterized in that, The method further includes an intent modeling step, which is performed after the acquisition and localization and before the spatial registration. Specifically, the intent modeling involves: extracting the biosensing data from the multimodal data with spatial coordinate encoding, determining a diagnostic intent vector based on the current diagnostic scenario, and using the diagnostic intent vector to adjust the feature weights and fusion weights of each modality in the spatial registration and weight fusion.

4. The health monitoring method for multimodal biosensor fusion optical detection according to claim 2, characterized in that, The biosensing unit includes one or more types of electrochemical sensing, piezoelectric sensing, and immunosensing, and the collected biosensing data includes one or more types of human physiological electrical signals, body fluid components, and tissue metabolic signals; the optical detection unit includes one or more types of near-infrared transmission, Raman scattering, and fluorescence imaging, and the collected optical detection data includes one or more types of optical data of human tissue molecular characteristics, cell morphology, and blood vessel distribution.

5. The health monitoring method for multimodal biosensor fusion optical detection according to claim 1, characterized in that, In the spatial registration, the spatial mapping matrix is ​​constructed based on the spatial coordinate system of the standard human anatomical structure. Spatial resampling and registration are performed with the goal of unifying the spatial resolution of different modal data, so that the multimodal data with spatial coordinate encoding remains consistent in the spatial dimension of the human anatomical structure. The spatial attention mechanism constructs an attention weight matrix, in which the optical feature weight values ​​are the weight values ​​assigned to each feature of the optical detection data. The spatial attention mechanism dynamically adjusts the optical feature weight values ​​to complete the spatial feature optimization process.

6. The health monitoring method for multimodal biosensor fusion optical detection according to claim 3, characterized in that, In the intent modeling, the diagnostic intent vector includes two dimensions: urgency and interpretability requirement. The urgency is calculated using an urgency quantification formula, and the interpretability requirement is divided into multiple quantification levels according to the actual degree of need. Both the urgency quantification value and the interpretability requirement quantification value are assigned based on a clinical scenario knowledge base.

7. A health monitoring method for multimodal biosensor fusion optical detection according to claim 6, characterized in that, The formula for calculating the urgency quantification is as follows: , Where E is the urgency quantification value, E b This serves as the base score for the urgency of the diagnostic scenario. This represents the clinical weight coefficient of the nth biosensor data item. E represents the single-index urgency fluctuation score of the nth biosensor data item. b and All were retrieved from the clinical scenario knowledge base. It is derived from the degree of deviation between the corresponding biosensor data item and the normal reference range.

8. A health monitoring method for multimodal biosensor fusion optical detection according to claims 5 and 6, characterized in that, The spatial attention mechanism dynamically adjusts the optical feature weight values ​​based on the urgency quantification: in high urgency scenarios, higher optical feature weight values ​​are assigned to features in the optical detection data that are related to lesion morphology and boundaries. In medium-urgency and low-urgency scenarios, the basic assignment of optical feature weight values ​​is used to allocate various features of the optical detection data.

9. A health monitoring method for multimodal biosensor fusion optical detection according to claim 1, characterized in that, In the weighted fusion, the multimodal feature extraction method is to perform layered extraction of shallow and deep features on the multimodal data with spatial coordinate encoding after spatial feature optimization, and then integrate and splice the shallow and deep features to obtain the multimodal features. The fusion weight is calculated by using the modal fusion weight calculation formula to calculate the fusion weight of optical detection data and biosensing data.

10. A health monitoring method for multimodal biosensor fusion optical detection according to claim 1, characterized in that, In the diagnostic output, a two-layer analysis of spatial correlation and statistical correlation is performed on the fused multimodal features to explore the intrinsic correlation features between different modal features. Based on the feature quantification values ​​obtained from the analysis, diagnostic reasoning is completed, and a diagnostic result containing disease type, severity and development trend is output. An interpretable diagnostic report containing feature contribution descriptions of key anatomical locations is also generated.

Images