A multimodal image fusion intelligent follow-up management system for chronic diseases

Abstract

This invention relates to the field of smart healthcare technology, and more particularly to a multimodal image fusion intelligent follow-up management system for chronic diseases. It includes: acquiring patient medical images and clinical laboratory data and aligning them temporally; retrieving metabolic half-life parameters and combining them with time differences to construct a nonlinear decay function to weight and attenuate the clinical data; inputting the images and attenuated clinical data into convolutional and deep neural networks respectively to extract feature vectors; calculating image quality and data completeness scores; calculating a baseline probability allocation value based on the scores; and using D-S evidence theory to perform fusion calculations to obtain a comprehensive risk value. Finally, based on the risk grading, it outputs a follow-up intervention strategy including the next follow-up examination time node. This invention effectively reduces the interference of irregular follow-up times and low-quality / incomplete data, improves the accuracy of risk prediction, and achieves a dynamic balance between treatment safety and economic costs.

Description

Technical Field

[0001] This invention relates to the field of smart healthcare technology, and in particular to a multimodal image fusion intelligent follow-up management system for chronic diseases. Background Technology

[0002] Intelligent follow-up of chronic diseases is an important direction for modern smart healthcare. During long-term follow-up, patients continuously generate heterogeneous data such as medical imaging and clinical laboratory tests. Utilizing multimodal fusion technology to integrate this multi-source information and construct disease evolution prediction models can reflect the patient's true pathological state from multiple dimensions, providing crucial technical support for early warning of high-risk disease deterioration and the development of personalized medical intervention strategies.

[0003] However, existing technologies neglect the differences in the metabolic half-life of physiological indicators and the irregularity of follow-up time, and fail to perform time-sensitive weighting on outdated test data. At the same time, traditional fusion mechanisms do not fully quantify the objective quality of the underlying data, and are prone to introducing noise interference when faced with low-quality images or incomplete records, resulting in low accuracy of system risk prediction. Summary of the Invention

[0004] To overcome the above shortcomings, this invention provides a multimodal image fusion intelligent follow-up management system for chronic diseases, which aims to improve the problem that traditional technologies are easily affected by low-quality and incomplete data, resulting in low accuracy of risk prediction.

[0005] This invention provides the following technical solution: a multimodal image fusion intelligent follow-up management system for chronic diseases, comprising:

[0006] The data acquisition and alignment module is used to acquire medical imaging data and clinical test data of patients at multiple historical follow-up nodes, and to align the medical imaging data and the clinical test data according to the time axis.

[0007] The time-related decay processing module is used to retrieve the physiological metabolic half-life parameters corresponding to the clinical test data, combine them with the time difference between the historical follow-up nodes and the current assessment time, construct a non-linear time decay function, and use the non-linear time decay function to perform weighted decay processing on the clinical test data.

[0008] The dual-channel feature extraction module is used to input the time-aligned medical image data into a convolutional neural network to extract image feature vectors, and to input the weighted attenuation-processed clinical test data into a deep neural network to extract indicator feature vectors.

[0009] The data quality assessment module is used to calculate the image quality score of the medical imaging data and to count the effective fields of the clinical test data to calculate and obtain the data completeness score.

[0010] The multi-source information fusion module is used to calculate the basic probability allocation values ​​of the image feature vector and the indicator feature vector based on the image quality score and the data integrity score, respectively, and to perform multi-source information fusion calculation on the basic probability allocation values ​​using the DS evidence theory algorithm to obtain a comprehensive risk value;

[0011] The intelligent follow-up decision module is used to classify risks based on the comprehensive risk value and output follow-up intervention strategies that include the time node for the next follow-up review.

[0012] Preferably, in the data acquisition and alignment module, the step of aligning the medical imaging data and the clinical laboratory data along a time axis includes:

[0013] Extract the original timestamps of the medical imaging data and the clinical test data, and map them to a standard time format to establish a global timeline;

[0014] A preset time tolerance window is set, and the medical image data acquisition time points on the global time axis are clustered to generate discrete follow-up baseline nodes.

[0015] The clinical test data are projected onto the global timeline according to the original timestamps, and then assigned to the corresponding follow-up baseline nodes using the principle of minimizing time distance, thus completing the time series alignment.

[0016] Preferably, in the time-related decay processing module, the step of constructing the nonlinear time decay function includes:

[0017] The clinical test data is analyzed to extract metabolic features, and the absolute metabolic half-life value corresponding to the metabolic features is matched from a preset knowledge base.

[0018] Calculate the scalar difference between the historical follow-up nodes and the current assessment time to generate time offset variables for each clinical data item;

[0019] Using the absolute metabolic half-life value as the rate control factor and combining it with the time offset variable, a continuous mapping equation containing an exponential decay term is constructed to generate the nonlinear time decay function.

[0020] Preferably, in the time-sensitivity attenuation processing module, the step of performing weighted attenuation processing on the clinical test data using the nonlinear time attenuation function includes:

[0021] Substitute each of the time offset variables into the nonlinear time decay function to calculate and obtain the effective retention rate coefficient under the current evaluation node;

[0022] For continuous clinical laboratory data, the effective retention rate coefficient is used to perform scalar multiplication on the original values ​​to calculate the attenuated characteristic values.

[0023] For discrete clinical laboratory data, the effective retention rate coefficient is converted into a weighted penalty term and added to the one-hot encoding representation of the discrete clinical laboratory data to complete the weighted attenuation processing of the data.

[0024] Preferably, in the dual-channel feature extraction module, the step of inputting the time-aligned medical image data into a convolutional neural network to extract image feature vectors includes:

[0025] The metadata of the medical image data is analyzed, and the device modality tags and spatial resolution parameters corresponding to the medical image data are extracted.

[0026] Based on the device modal labels and spatial resolution parameters, the medical image data is uniformly reconstructed into a standard-dimensional three-dimensional data matrix using a resampling algorithm;

[0027] The three-dimensional data matrix is ​​converted into a standard tensor format and input into the convolutional neural network. The texture features of the target region are extracted through forward propagation, and the image feature vector is flattened and output.

[0028] Preferably, in the data quality assessment module, the step of calculating the image quality score of the medical image data includes:

[0029] The medical image data is subjected to a Fourier transform and mapped to the frequency domain. The average value of the high-frequency signal energy ratio is extracted, and a sharpness index representing global sharpness is calculated and generated.

[0030] The difference and variance of the mean gray level between the preset target area and the preset background area are calculated in the spatial domain to construct the contrast-to-noise ratio index, so as to quantify the local resolution capability.

[0031] The sharpness index and the contrast-to-noise ratio index are input into a preset evaluation model for weighted linear combination, and the image quality score is output after normalization.

[0032] Preferably, in the data quality assessment module, the step of statistically analyzing the valid fields of the clinical laboratory data to calculate and obtain a data completeness score includes:

[0033] The pre-defined chronic disease assessment baseline library is parsed to obtain the required core test field set and auxiliary test field set, and differentiated benchmark weights are assigned to them.

[0034] Traverse the clinical test data and count the number of actual non-empty fields in the core test field set and the auxiliary test field set;

[0035] A weighted summation operation is performed based on the actual number of non-empty fields and their corresponding baseline weights, and then divided by the maximum theoretical weight sum to calculate and obtain the data integrity score.

[0036] Preferably, in the multi-source information fusion module, the step of using the DS evidence theory algorithm to perform multi-source information fusion calculation on the basic probability allocation value includes:

[0037] Define an identification framework that includes multiple mutually exclusive disease evolution states, and abstract the image feature vector and the indicator feature vector as independent evidence sources;

[0038] Using orthogonal sum rules, the joint quality matrix of each independent source of evidence under each state combination is calculated, and the global conflict coefficient characterizing the degree of evidence conflict is extracted.

[0039] When the global conflict coefficient is lower than a preset threshold, the joint quality matrix is ​​probabilistically normalized, and the posterior confidence of the target deterioration state is calculated as the comprehensive risk value.

[0040] Preferably, in the intelligent follow-up decision module, the steps of outputting the follow-up intervention strategy for the next follow-up review time point include:

[0041] The comprehensive risk value is input into a preset chronic disease dynamic evolution model to generate the expected risk change curve within the prediction period;

[0042] Construct a joint objective optimization function that includes the cost of missed diagnosis / delay and the cost of follow-up visits.

[0043] The multi-objective particle swarm optimization algorithm is used to perform optimization on the expected risk change curve to find the optimal time interval that minimizes the joint objective optimization function, and the time node for the next follow-up review is calculated accordingly.

[0044] The present invention has the following beneficial effects:

[0045] 1. In this invention, by introducing a time decay function of metabolic half-life to dynamically reduce the weight of test data, and combining image and data quality scores, multimodal information fusion is carried out using DS evidence theory, which effectively reduces the misleading noise interference caused by irregular follow-up time and low-quality incomplete data, and improves the accuracy of chronic disease risk prediction.

[0046] 2. In this invention, by parsing the image metadata and using a resampling algorithm, multi-source heterogeneous images are uniformly reconstructed into a standard-dimensional three-dimensional data matrix, which reduces the spatial scale and physical deviation caused by acquisition from different equipment across different hospital areas, and provides standardized aligned data support for high-dimensional feature extraction of neural networks.

[0047] 3. This invention breaks through the traditional model of relying on fixed time period re-examination. By constructing a joint optimization function that takes into account the cost of missed diagnosis and delay and the resource cost of re-examination, the optimal follow-up interval is dynamically solved by using multi-objective particle swarm optimization algorithm. Under the premise of ensuring that high-risk patients receive timely early warning and intervention, medical resources are avoided and a dynamic balance between diagnosis and treatment safety and economic cost is achieved. Attached Figure Description

[0048] Figure 1 This is an architecture diagram of a multimodal image fusion intelligent follow-up management system for chronic diseases proposed in this invention. Detailed Implementation

[0049] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] In embodiments of the present invention, the present invention provides a multimodal image fusion intelligent follow-up management system for chronic diseases, such as... Figure 1 As shown, it includes:

[0051] The data acquisition and alignment module is used to acquire medical imaging data and clinical laboratory data from multiple historical follow-up nodes of the patient, and to align the medical imaging data and clinical laboratory data according to the time axis.

[0052] Furthermore, in the data acquisition and alignment module, the step of aligning medical imaging data and clinical laboratory data along a timeline includes:

[0053] Extract the raw timestamps from medical imaging data and clinical laboratory data, and map them to a standard time format to establish a global timeline;

[0054] Set a preset time tolerance window, cluster the medical image data acquisition time points on the global time axis, and generate discrete follow-up baseline nodes;

[0055] Clinical laboratory data are projected onto the global timeline according to their original timestamps, and then assigned to the corresponding follow-up baseline nodes using the principle of minimizing time distance, thus completing the time series alignment.

[0056] Specifically, the system first continuously acquires multi-source heterogeneous data generated by patients throughout their long medical history from the hospital's image archiving and communication systems, as well as the laboratory information management system. Medical imaging data specifically includes spatial pixel data such as computed tomography (CT) scans and magnetic resonance imaging (MRI), while clinical laboratory data specifically includes numerical biochemical indicators such as blood biochemical markers and routine urine tests. The system extracts the sequence generation time recorded in the file header of the medical imaging data as the raw timestamp, and extracts the sampling time from the message messages of the clinical laboratory data as the raw timestamp. To eliminate differences in time recording formats between different medical information systems, the system uniformly maps all extracted raw timestamps to Coordinated Universal Time (UTC) format timestamps, thereby constructing a unidirectionally increasing global timeline at the system's underlying layer. Let the set of medical imaging data acquisition time points on the global timeline be denoted as . The set of time points for clinical laboratory data collection is as follows .

[0057] Next, set the preset time tolerance window. The specific value of this window is determined by the lag in the pathological evolution of the target chronic disease. In this scheme, the time tolerance window... The preferred timeframe is 7 days. The system uses a time tolerance window. To determine the threshold, the set of medical image data acquisition time points on the global time axis was analyzed. Perform one-dimensional agglomerative hierarchical clustering. For any two medical image acquisition time points... and If the absolute time difference between the two is less than or equal to If these two time points are grouped into the same set, the local temporal density is clustered. After clustering, several discrete follow-up baseline nodes are generated. The system further calculates the arithmetic mean of all image time points within each follow-up baseline node, and uses this as the central representative time of that baseline node. The specific formula for calculating the center representative time is as follows:

[0058] ;

[0059] in the formula Representing the A number of generated discrete follow-up baseline nodes Representing the The total number of medical imaging time points included within each follow-up baseline node. Represents the number belonging to this node The system calculates the acquisition time points of medical image data. Through this calculation, the system abstracts the imaging examination activities within a dense time interval into a reference node with a unique time coordinate.

[0060] The system then projects the clinical laboratory data onto the global timeline according to the original timestamps. This is done for each data collection point in the clinical laboratory data. The system utilizes the principle of minimizing time distance to iterate through and search for the central representative time with the shortest absolute time interval. And determine whether the shortest distance falls within the effective pathological association period. Within. In this protocol, the effective pathological association period The preferred timeframe is 14 days. The specific matching algorithm formula is as follows:

[0061] ;

[0062] in the formula The index identifier representing the best follow-up baseline node of the final match. Representing the The actual timestamp of each clinical laboratory data collection. Representing the The center representative time of each follow-up baseline node. When the absolute difference Less than or equal to the effective pathological association period At that time, the system establishes the coupling relationship between the clinical laboratory data and the imaging data, and assigns it to the corresponding [missing information]. Time series alignment of cross-modal data was completed among the follow-up baseline nodes.

[0063] The implementation of this step effectively eliminates the asynchronous misalignment caused by the inconsistency between the timing of imaging examinations and biochemical tests during the patient's medical treatment process. It integrates discrete and heterogeneous medical data into multimodal data slices with synchronized time nodes, providing a time-dimensional aligned underlying data structure for the system's subsequent disease evolution analysis.

[0064] The time-related decay processing module is used to retrieve the physiological metabolic half-life parameters corresponding to the clinical test data, combine them with the time difference between the historical follow-up nodes and the current assessment time, construct a non-linear time decay function, and use the non-linear time decay function to perform weighted decay processing on the clinical test data.

[0065] Furthermore, in the time-related decay processing module, the step of constructing the nonlinear time decay function includes:

[0066] Analyze clinical laboratory data to extract metabolic features, and match the absolute metabolic half-life values ​​corresponding to the metabolic features from a pre-set knowledge base;

[0067] Calculate the scalar difference between historical follow-up points and the current assessment time to generate time offset variables for each clinical data item;

[0068] Using the absolute metabolic half-life value as the rate control factor and combining it with the time offset variable, a continuous mapping equation containing an exponential decay term is constructed to generate a nonlinear time decay function.

[0069] Furthermore, in the time-sensitivity attenuation processing module, the step of using a nonlinear time decay function to perform weighted attenuation processing on clinical laboratory data includes:

[0070] Substitute each time offset variable into the nonlinear time decay function to calculate the effective retention rate coefficient at the current evaluation node;

[0071] For continuous clinical laboratory data, the effective retention rate coefficient is used to perform scalar multiplication on the original values ​​to calculate the attenuated characteristic values.

[0072] For discrete clinical laboratory data, the effective retention rate coefficient is transformed into a weighted penalty term and added to the one-hot encoding representation of discrete clinical laboratory data to complete the weighted attenuation processing of the data.

[0073] Specifically, the system first analyzes various clinical test data included in the follow-up baseline node, extracting the metabolic characteristics corresponding to each biochemical test indicator. The system then searches and matches the absolute metabolic half-life value corresponding to this metabolic characteristic from a pre-set medical physiology knowledge base. Taking glycated hemoglobin and fasting blood glucose as specific clinical test data as examples, the pre-set knowledge base records an absolute metabolic half-life of approximately thirty days for glycated hemoglobin, while the absolute metabolic half-life for fasting blood glucose is only half a day. In this protocol, the preferred absolute metabolic half-life for glycated hemoglobin is 30 days, and the preferred absolute metabolic half-life for fasting blood glucose is 0.5 days. Let the first... The absolute metabolic half-life values ​​of the clinical test data are: The system then obtains the absolute time of the current risk prediction as the current assessment time. and extract the first The actual timestamp of clinical laboratory data collection. Calculate the scalar difference between the two to generate the time offset variable for this clinical data item. The calculation formula is as follows:

[0074] ;

[0075] in the formula A standard timestamp indicating the current assessment time. Indicates the first The test index is in the first The actual sampling time during the historical follow-up. This represents the time offset variable, with the unit limited to days.

[0076] Next, the system uses the absolute metabolic half-life as the rate control factor and combines it with time offset variables to construct a continuous mapping equation containing an exponential decay term as a nonlinear time decay function. The system substitutes each calculated time offset variable into this nonlinear time decay function to calculate the effective retention rate coefficients of each test indicator at the current evaluation node. The specific formula for calculating the effective retention rate coefficients is as follows:

[0077] ;

[0078] in the formula Indicates the first The effective retention rate coefficient of clinical laboratory data at the current assessment point, its physical meaning is the confidence level of this indicator in reflecting the patient's current true physiological state. The value range of this coefficient is between zero and one. Represents the natural logarithm constant. This represents the absolute metabolic half-life value. This represents the time offset variable. According to this equation, the effective retention rate coefficient of data older than the current time decreases non-linearly and exponentially, and the shorter the half-life of the test indicator, the faster its decay rate.

[0079] After obtaining the effective retention rate coefficient, the system processes clinical laboratory data with continuous numerical dimensions. When the input clinical laboratory data is a specific continuous raw measurement value such as glycated hemoglobin percentage concentration or blood lipid concentration, the system uses the effective retention rate coefficient to perform a scalar multiplication operation on the raw value to calculate the attenuated characteristic value. The specific formula for attenuating continuous data is as follows:

[0080] ;

[0081] in the formula Represents the raw, true measured values ​​of continuous clinical laboratory data. This represents the characteristic value after time-weighted attenuation.

[0082] For discrete clinical laboratory data with qualitative labeling features, such as structured text data like a patient's smoking status or pathological stage, the system first converts it into one-hot encoded representations. Assume the test item has a total of... In this scheme, there are several discrete state classifications. Preferably, there are four values. The system converts the effective retention rate coefficient into a weight penalty term and performs a scalar multiplication with the original one-hot encoded vector, directly appending the attenuation coefficient to the activated classification dimension in the one-hot encoded representation. The specific discrete data attenuation formula is as follows:

[0083] ;

[0084] in the formula Indicates length is The original one-hot encoded row vector, where the dimension corresponding to the patient's true test state is one and all others are zero. Let represent the decayed feature vector including a weighted penalty term, where the activated dimension value is decayed from one to . The system performs weighted attenuation processing on all clinical data through the above operations, and then sends the processed results to the subsequent dual-channel feature extraction module.

[0085] This step combines the principles of clinical medicine’s physical metabolism to dynamically downweight historical data, which solves the problems of irregular follow-up time and outdated test indicators misleading the assessment of the current disease status, and improves the accuracy of input features in reflecting the patient’s current true physiological status.

[0086] The dual-channel feature extraction module is used to input time-aligned medical image data into a convolutional neural network to extract image feature vectors, and to input weighted and attenuated clinical test data into a deep neural network to extract indicator feature vectors.

[0087] Furthermore, in the dual-channel feature extraction module, the step of inputting the time-aligned medical image data into a convolutional neural network to extract image feature vectors includes:

[0088] Analyze the metadata of medical imaging data and extract the device modality labels and spatial resolution parameters corresponding to the medical imaging data;

[0089] Based on device modal labels and spatial resolution parameters, a resampling algorithm is used to reconstruct medical image data into a standard-dimensional three-dimensional data matrix.

[0090] The 3D data matrix is ​​converted into a standard tensor format and input into a convolutional neural network. The texture features of the target region are extracted through forward propagation, and the image feature vector is flattened and output.

[0091] Specifically, the system first parses the header metadata of the Digital Medical Imaging Communication Standard (DMI) file of the medical image data. From this, it extracts the device modality tag of the image, such as computed tomography (CT) or magnetic resonance imaging (MRI), and simultaneously extracts the three-dimensional spatial resolution parameters, specifically including intra-slice pixel spacing and slice thickness parameters. Let the physical spatial resolution of the original medical image be... Its size includes the three dimensions of length, width, and depth. The physical dimension of the original three-dimensional matrix corresponding to this image is defined as follows: .

[0092] Based on the extracted device modal labels and spatial resolution parameters, the system uses a resampling algorithm to reconstruct medical image data from different devices and at varying scales into a standardized three-dimensional data matrix. The system pre-defines a globally uniform target physical spatial resolution, whose length, width, and depth dimensions are [value missing]. In this scheme, the dimensions of the target physical space resolution are length, width, and depth. The preferred values ​​are 1mm, 1mm, and 1mm. Based on this, the system calculates the physical scaling factor from the original spatial coordinates to the target spatial coordinates, assuming the scaling factors for each dimension are as follows: , and The system then uses a trilinear interpolation algorithm to spatially map and recalculate the grayscale of the original pixels, generating a resampled standard 3D data matrix. The core spatial mapping interpolation formula is as follows:

[0093] ;

[0094] in the formula This represents the calculated grayscale value of a voxel point in the target space coordinate system of the reconstructed standard 3D data matrix. This represents the known gray values ​​of the original medical image matrix at adjacent real discrete coordinate points. This represents a continuous linear interpolation kernel function in three-dimensional space. The system uses this operator to traverse all target space coordinates and complete the three-dimensional volume reconstruction of the medical image.

[0095] Next, the system performs medical window truncation and grayscale normalization on the reconstructed standard 3D data matrix, converting it into a standard multidimensional tensor format that supports deep learning computing frameworks, and feeding it as an input source into a preset 3D convolutional neural network. The specific architecture of the 3D convolutional neural network is configured to include five cascaded 3D convolutional blocks. Each 3D convolutional block consists of a 3D convolutional layer with a kernel size of 3×3×3, a batch normalization layer, and a ReLU activation function connected sequentially. A max-pooling layer with a stride of 2 is set between adjacent 3D convolutional blocks for spatial downsampling. The image tensor undergoes forward propagation within the convolutional neural network through multiple 3D convolutional kernels and activation functions, extracting the deep topological structure and pathological texture features of the target lesion region layer by layer. Let the deep high-dimensional feature map output from the end of the 3D convolutional neural network be... The total number of its channels is The size of the spatial dimension is Multiply Multiply The system employs a global average pooling algorithm to reduce the spatial dimension of the high-dimensional feature map, extract the mean global response of each feature channel, and finally flatten and output a one-dimensional image feature vector. The global average pooling mapping calculation formula is as follows:

[0096] ;

[0097] in the formula The output one-dimensional image feature vector represents the first... scalar feature values ​​of each channel dimension This indicates that the deep high-dimensional feature map is in the first... The activation response values ​​of neurons at their corresponding three-dimensional spatial locations on each channel. , , These represent the height, width, and depth dimensions of the feature map at that layer, respectively. The system will... The aggregated scalar feature values ​​of each channel are concatenated sequentially according to the channel order to form a final length of [length missing]. The system generates a one-dimensional image feature vector. Simultaneously, for the neural network channel processing clinical laboratory data, the system concatenates and reassembles the weighted and attenuated clinical laboratory data into a one-dimensional input vector, which is then fed into a pre-defined deep neural network. The specific architecture of this deep neural network consists of four fully connected hidden layers, with the number of neurons in each hidden layer being 128, 64, 32, and 16 respectively. Each hidden layer is followed by a LeakyReLU activation function, and a regularization layer with a dropout rate of 0.3 is introduced to prevent overfitting. This one-dimensional input vector undergoes a layer-by-layer nonlinear mapping through the multi-layer fully connected network, ultimately outputting a fixed-dimensional indicator feature vector by the final output layer.

[0098] This implementation step eliminates the deviation in spatial scale and physical resolution of medical images caused by acquisition from different devices across different hospital areas, and maps and compresses unstructured 3D images with a large amount of redundant background into feature vectors with high semantic density, clearing the physical dimension barrier for subsequent quantitative cascade fusion of heterogeneous data.

[0099] The data quality assessment module is used to calculate the image quality score of medical imaging data and to statistically analyze the effective fields of clinical laboratory data to calculate the data completeness score.

[0100] Furthermore, in the data quality assessment module, the steps for calculating the image quality score of medical imaging data include:

[0101] Perform a Fourier transform on the medical image data to map it to the frequency domain, extract the average proportion of high-frequency signal energy, and calculate and generate a sharpness index that characterizes global sharpness.

[0102] The difference and variance of the mean gray level between the preset target area and the preset background area are calculated in the spatial domain to construct the contrast-to-noise ratio index, so as to quantify the local resolution capability.

[0103] The sharpness index and contrast-to-noise ratio index are input into a preset evaluation model, weighted and linearly combined, and then normalized before outputting an image quality score.

[0104] Furthermore, in the data quality assessment module, the steps for statistically analyzing the valid fields of clinical laboratory data to calculate and obtain a data completeness score include:

[0105] The pre-defined chronic disease assessment baseline library is parsed to obtain the required core test field set and auxiliary test field set, and differentiated benchmark weights are assigned to them.

[0106] Traverse the clinical laboratory data and count the actual number of non-empty fields in the core test field set and the auxiliary test field set;

[0107] A weighted summation operation is performed based on the actual number of non-empty fields and their corresponding baseline weights, and then divided by the maximum theoretical weight sum to calculate and obtain the data completeness score.

[0108] Specifically, the system first calculates the sharpness index of medical image data from the frequency domain dimension. Taking a single-slice two-dimensional slice of a computed tomography image as an example, the system performs a two-dimensional fast Fourier transform on the pixel matrix of the medical image in the spatial domain, mapping it to the frequency domain to generate a complex spectrum. The system obtains the power spectrum by calculating the square of the amplitude of the complex spectrum and sets a preset center cutoff frequency domain radius. In this scheme, the center cutoff frequency domain radius is preferably 0.25 times the maximum frequency domain radius of the image spectrum, thereby distinguishing low-frequency signals representing smooth background tissue from high-frequency signals representing lesion edge details and high-frequency structures. The system calculates the power spectrum energy integral within the high-frequency signal band and divides it by the global total power spectrum energy integral of the entire frequency domain, thereby extracting the average energy proportion of the high-frequency signal as a sharpness index representing global sharpness. The calculation formula is as follows:

[0109] ;

[0110] in the formula Indicates the clarity index, This represents the coordinates of the medical image spatial matrix in the frequency domain after Fourier transform. and Complex values ​​at that location, This represents a high-frequency masking function. Its value is one when the Euclidean distance from the coordinate point to the center of the frequency domain is greater than the center cutoff frequency domain radius, and zero otherwise.

[0111] Next, the system quantifies the local resolution of medical images in the spatial domain to calculate the contrast-to-noise ratio. The system extracts a pre-defined target area and a pre-defined background area based on specific chronic disease types. For example, for chronic obstructive pulmonary disease (COPD), the target area corresponds to the high-density region of the diseased airway, while the background area corresponds to the surrounding normal low-density lung parenchyma. The system calculates the arithmetic mean of the grayscale values ​​of all pixels in both the target area and the background area, and also calculates the standard deviation of the grayscale values ​​in the background area. Based on these parameters, the system constructs the contrast-to-noise ratio, calculated using the following formula:

[0112] ;

[0113] in the formula This refers to the contrast-to-noise ratio. This represents the arithmetic mean of gray levels within the preset target area. This represents the arithmetic mean of gray levels within a preset background area. This represents the standard deviation of the grayscale values ​​in the preset background region. The system then inputs the sharpness index and contrast-to-noise ratio index into a preset evaluation model for a weighted linear combination. Since the two indices have different dimensions, the system first uses a minimax normalization algorithm to map them to a numerical range of zero to one, then multiplies them by the set sharpness weight coefficient and local contrast weight coefficient respectively, and sums them to finally output a comprehensive image quality score. The fusion evaluation formula is as follows:

[0114] ;

[0115] in the formula Indicates image quality score, This represents the sharpness index after normalization. This represents the normalized contrast-to-noise ratio. and These represent the preset weighting coefficients that sum to one. In this scheme, the sharpness weighting coefficient... The preferred value is 0.4, the local contrast weighting coefficient. The preferred value is 0.6.

[0116] For clinical laboratory data, the system parses a pre-defined chronic disease assessment baseline database to obtain the weight mapping relationships of the required test fields and calculates the data completeness score. Taking diabetes follow-up management as an example, the system obtains a core set of test fields containing core indicators such as fasting blood glucose and glycated hemoglobin, and an auxiliary set of test fields containing auxiliary diagnostic indicators such as triglycerides and low-density lipoprotein from the baseline database. The system assigns a higher differential benchmark weight to the core test field set and a lower differential benchmark weight to the auxiliary test field set. The system iterates through the currently input clinical laboratory data set and counts the number of non-empty fields with actual test values ​​recorded in the core and auxiliary test field sets. Based on the actual number of non-empty fields and their corresponding benchmark weights, the system performs a weighted summation operation, divides the result by the maximum theoretical weight sum of all theoretically required fields in the chronic disease assessment baseline database when they are fully loaded and without missing values, and calculates the data completeness score representing the completeness of the follow-up record. The calculation formula is as follows:

[0117] ;

[0118] in the formula Indicates the data completeness score. and These represent the baseline weight constants assigned to a single mandatory test field and a single auxiliary test field, respectively. In this scheme, the baseline weight constant for a single mandatory test field... Preferably 1.0, the baseline weight constant for a single auxiliary test field. The preferred value is 0.5. and These represent the actual number of non-empty fields in the core test field set and the auxiliary test field set obtained by the system statistics, respectively. and These represent the total number of required test fields and the total number of auxiliary test fields defined in the chronic disease assessment baseline library, respectively.

[0119] This step accurately quantifies the reliability of input data from two dimensions: the spatial frequency characteristics of medical images and the degree of missing text records of clinical indicators. It effectively prevents blurry and distorted low-quality images or test reports with missing core fields from misleading subsequent risk prediction models, thereby improving the noise resistance and robustness of the overall diagnostic analysis system.

[0120] The multi-source information fusion module is used to calculate the basic probability allocation values ​​of image feature vectors and indicator feature vectors based on image quality scores and data integrity scores, respectively, and to perform multi-source information fusion calculation on the basic probability allocation values ​​using the DS evidence theory algorithm to obtain a comprehensive risk value.

[0121] Furthermore, in the multi-source information fusion module, the step of using the DS evidence theory algorithm to perform multi-source information fusion calculation on the basic probability assignment value includes:

[0122] Define an identification framework that includes multiple mutually exclusive disease evolution states, and abstract image feature vectors and indicator feature vectors as independent sources of evidence;

[0123] Using orthogonal sum rules, the joint quality matrix of each independent source of evidence under each state combination is calculated, and the global conflict coefficient, which characterizes the degree of evidence conflict, is extracted.

[0124] When the global conflict coefficient is lower than the preset threshold, the joint quality matrix is ​​probabilistically normalized, and the posterior confidence of the target's deteriorated state is calculated as the comprehensive risk value.

[0125] Specifically, the system first defines a global identification framework that includes multiple mutually exclusive disease evolution states. Let this global identification framework be... Its internal structure includes a stable state of the disease. Slow progression of the disease and the condition deteriorated rapidly. These are three mutually exclusive basic diagnostic hypotheses. The system feeds the image feature vector extracted by the convolutional neural network and the index feature vector extracted by the deep neural network into their respective classification network branches to obtain the initial prediction probabilities of these two independent evidence sources for the three evolutionary states mentioned above. Let the image evidence source be related to the state... The initial prediction probability is Clinical indicator evidence sources for status The initial prediction probability is The system incorporates an image quality score output by a data quality assessment module. Data completeness score As a discount factor characterizing the reliability of the corresponding evidence source, the basic probability allocation values ​​of the image feature vector and the index feature vector are calculated accordingly. Taking the image evidence source as an example, its probability allocation for a specific state... The formulas for calculating the basic probability allocation value and the uncertainty allocation value for the globally unknown state are as follows:

[0126] ;

[0127] ;

[0128] in the formula Indicates the source of image evidence for a single state The base probability allocation value, This represents the base probability assignment value for the global uncertainty of the image evidence source due to defects or noise in image quality. Similarly, the system uses data integrity scoring... Compared with the initial predicted probability The baseline probability allocation values ​​corresponding to the clinical indicator evidence sources were calculated using the same mapping logic. With global uncertainty allocation value .

[0129] After obtaining the base probability assignments for two independent evidence sources, the system uses orthogonal sum rules to calculate the joint quality matrix of each independent evidence source under various state combinations. Before performing the final probability fusion, the system first extracts a global conflict coefficient characterizing the degree of conflict between imaging evidence sources and clinical indicator evidence sources. This global conflict coefficient is used to quantify the conflict measure when imaging features indicate lesion enlargement while biochemical test indicators remain stable, and its calculation formula is as follows:

[0130] ;

[0131] in the formula Represents the global conflict coefficient. Indicates the status of the source of image evidence. The basic probability allocation value, This indicates that clinical indicator evidence sources support different states. The basic probability assignment value. The system iterates through all mutually exclusive state combinations and accumulates the multiplications to obtain the overall measure of evidence conflict.

[0132] The system internally sets a conflict tolerance threshold to determine whether heterogeneous evidence is completely invalid. In this scheme, the conflict tolerance threshold is preferably 0.85. When the calculated global conflict coefficient... When the value is below the preset threshold, it indicates that although the two types of multimodal evidence differ, they do not yet constitute a complete paradox. At this point, the system performs probability normalization on the joint quality matrix and calculates the posterior confidence of each fused state. This applies to any target state within the recognition framework. The multi-source information fusion calculation formula is as follows:

[0133] ;

[0134] in the formula This indicates that the system has assessed the evolution of a specific disease state after multimodal joint evaluation. The final confidence level is calculated. After the calculation is completed, the system extracts the target deterioration state, i.e., the state of rapid deterioration of the condition, from the recognition framework. The corresponding posterior confidence level This is used as a comprehensive risk value to quantify the current risk level of a patient's disease progression.

[0135] This step introduces the objective quality quantification value of the multimodal data itself into the probabilistic decision layer. When there are local missing data in the underlying data or when there are contradictions in the modal prediction results, the system can automatically reduce the interference of low-quality data on the decision results, thereby improving the anti-interference and fusion robustness of the chronic disease risk prediction system.

[0136] The intelligent follow-up decision module is used to classify risks based on comprehensive risk values ​​and output follow-up intervention strategies that include the time node for the next follow-up review.

[0137] Furthermore, in the intelligent follow-up decision module, the steps of outputting the follow-up intervention strategy for the next follow-up review time node include:

[0138] The comprehensive risk value is input into a preset chronic disease dynamic evolution model to generate the expected risk change curve within the prediction period;

[0139] Construct a joint objective optimization function that includes the cost of missed diagnosis / delay and the cost of follow-up visits.

[0140] The multi-objective particle swarm optimization algorithm is used to perform optimization on the expected risk change curve to find the optimal time interval that minimizes the joint objective optimization function, and the time node for the next follow-up review is calculated accordingly.

[0141] Specifically, the system first receives the comprehensive risk value output by the multi-source information fusion module and inputs it as an initial state parameter into the preset chronic disease dynamic evolution model. Taking the progression of chronic obstructive pulmonary disease (COPD) as an example, the system retrieves the preset disease evolution rate constant for this disease, combines it with the comprehensive risk value representing the probability of current disease deterioration, and uses a time-series extrapolation algorithm based on logistic growth mechanism to generate the expected risk change curve within the prediction period. The expected risk change curve represents the dynamic probability of substantial disease deterioration at a future continuous point in time. The specific risk dynamic extrapolation formula is as follows:

[0142] ;

[0143] in the formula Indicates the future number The expected probability of a patient's condition deteriorating rapidly. This represents the overall risk value currently obtained from the system's assessment. This represents the disease evolution rate constant retrieved from a pre-defined chronic disease baseline database. In this scheme, the disease evolution rate constant... The preferred value is 0.015. This represents a continuous time variable that extends backward from the current evaluation time, with its numerical unit being days.

[0144] After obtaining the expected risk change curve, the system constructs a joint objective optimization function to assess the rationality of follow-up timing. This joint objective optimization function consists of a risk cost term for missed diagnosis / delay and a resource cost term for follow-up visits. The risk cost term for missed diagnosis / delay is positively correlated with the cumulative integral of the expected risk probability over time, and is used to penalize the risk of delayed intervention due to excessively long follow-up intervals. The resource cost term for follow-up visits is strictly inversely correlated with the follow-up interval, and is used to penalize the economic burden on patients and the consumption of medical system scheduling resources caused by excessively frequent follow-up visits. The specific calculation formula for the joint objective optimization function is as follows:

[0145] ;

[0146] in the formula This indicates that the follow-up interval will be set to... The system's comprehensive target evaluation value for timing and timing, the integral term on the left side of the plus sign on the right side of the equals sign represents the cumulative cost of missed diagnosis and delay risk, and the fractional term on the right side of the plus sign on the right side of the equals sign represents the cost of follow-up visits resource cost. This represents the preset medical risk penalty weighting coefficient. This represents the preset calibration constant for review resource consumption. In this scheme, the medical risk penalty weighting coefficient... The preferred value is 0.7; the resource consumption calibration constant is then reviewed. The preferred value is 50. This represents the integration variable during the integration process.

[0147] The system then utilizes a multi-objective particle swarm optimization algorithm to iteratively optimize along the expected risk change curve, aiming to find the optimal time interval that minimizes the joint objective optimization function. Within a defined effective follow-up period, the system randomly initializes a swarm of particles, where the spatial position of each particle represents a candidate follow-up time interval. The system directly uses the joint objective optimization function as the fitness evaluation benchmark, calculating the fitness scalar value corresponding to each particle's current spatial position. During each iteration, particles dynamically update their search velocity and spatial position parameters based on their historical best time interval and the population's global best time interval. The core velocity and position iterative update formulas are as follows:

[0148] ;

[0149] ;

[0150] in the formula and They represent the first The particle in the first Next and first The search speed parameter in the next optimization iteration and They represent the first The number of days that each particle represents in the corresponding iteration round for candidate follow-up. The inertia weight factor represents the global exploration capability of the control algorithm. and Representing local self-awareness learning factors and global social awareness learning factors respectively, in this scheme, the inertia weight factor... The optimal value is 0.9, a local self-awareness learning factor. The optimal value is 1.5, representing the global social cognitive learning factor. The preferred value is 1.5. and This represents a floating-point number randomly distributed in the interval between zero and one. This represents the number of days in days that represents the historical best time interval with the minimum fitness value calculated so far for this particle. This represents the number of days that the entire particle swarm has found so far to be the globally optimal time interval. The algorithm terminates when the swarm iterations reach a preset maximum number (preferably 200 in this scheme), or when the globally optimal time interval no longer decays after multiple consecutive iterations. The system then extracts the final globally optimal time interval. The system uses this as the optimal time interval parameter. Finally, it obtains the absolute time reference point of the current computing device and performs a calendar addition operation on it, thereby generating the specific date and time node for the next follow-up examination. This date and time node, which includes the decision-making basis, is then output to the front-end medical management system.

[0151] The implementation of this step transforms the traditional fixed-cycle follow-up model into a quantitative mathematical constraint solution process based on dynamic risk prediction, achieving a precise balance between the safety of medical diagnosis and the economic cost of medical resources, and significantly reducing the rate of delayed or missed diagnosis for high-risk patients with chronic diseases.

[0152] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal image fusion intelligent follow-up management system for chronic diseases, characterized in that, include: The data acquisition and alignment module is used to acquire medical imaging data and clinical test data of patients at multiple historical follow-up nodes, and to align the medical imaging data and the clinical test data according to the time axis. The time-related decay processing module is used to retrieve the physiological metabolic half-life parameters corresponding to the clinical test data, combine them with the time difference between the historical follow-up nodes and the current assessment time, construct a non-linear time decay function, and use the non-linear time decay function to perform weighted decay processing on the clinical test data. The dual-channel feature extraction module is used to input the time-aligned medical image data into a convolutional neural network to extract image feature vectors, and to input the weighted attenuation-processed clinical test data into a deep neural network to extract indicator feature vectors. The data quality assessment module is used to calculate the image quality score of the medical imaging data and to count the effective fields of the clinical test data to calculate and obtain the data completeness score. The multi-source information fusion module is used to calculate the basic probability allocation values ​​of the image feature vector and the indicator feature vector based on the image quality score and the data integrity score, respectively, and to perform multi-source information fusion calculation on the basic probability allocation values ​​using the DS evidence theory algorithm to obtain a comprehensive risk value; The intelligent follow-up decision module is used to classify risks based on the comprehensive risk value and output follow-up intervention strategies that include the time node for the next follow-up review.

2. The multimodal image fusion intelligent follow-up management system for chronic diseases according to claim 1, characterized in that, In the data acquisition and alignment module, the step of aligning the medical imaging data and the clinical laboratory data along a time axis includes: Extract the original timestamps of the medical imaging data and the clinical test data, and map them to a standard time format to establish a global timeline; A preset time tolerance window is set, and the medical image data acquisition time points on the global time axis are clustered to generate discrete follow-up baseline nodes. The clinical test data are projected onto the global timeline according to the original timestamps, and then assigned to the corresponding follow-up baseline nodes using the principle of minimizing time distance, thus completing the time series alignment.

3. The multimodal image fusion intelligent follow-up management system for chronic diseases according to claim 1, characterized in that, In the time-related decay processing module, the step of constructing the nonlinear time decay function includes: The clinical test data is analyzed to extract metabolic features, and the absolute metabolic half-life value corresponding to the metabolic features is matched from a preset knowledge base. Calculate the scalar difference between the historical follow-up nodes and the current assessment time to generate time offset variables for each clinical data item; Using the absolute metabolic half-life value as the rate control factor and combining it with the time offset variable, a continuous mapping equation containing an exponential decay term is constructed to generate the nonlinear time decay function.

4. The multimodal image fusion intelligent follow-up management system for chronic diseases according to claim 1, characterized in that, In the time-sensitivity attenuation processing module, the step of performing weighted attenuation processing on the clinical test data using the nonlinear time attenuation function includes: Substitute each of the time offset variables into the nonlinear time decay function to calculate and obtain the effective retention rate coefficient under the current evaluation node; For continuous clinical laboratory data, the effective retention rate coefficient is used to perform scalar multiplication on the original values ​​to calculate the attenuated characteristic values. For discrete clinical laboratory data, the effective retention rate coefficient is converted into a weighted penalty term and added to the one-hot encoding representation of the discrete clinical laboratory data to complete the weighted attenuation processing of the data.

5. The multimodal image fusion intelligent follow-up management system for chronic diseases according to claim 1, characterized in that, In the dual-channel feature extraction module, the step of inputting the time-aligned medical image data into a convolutional neural network to extract image feature vectors includes: The metadata of the medical image data is analyzed, and the device modality tags and spatial resolution parameters corresponding to the medical image data are extracted. Based on the device modal labels and spatial resolution parameters, the medical image data is uniformly reconstructed into a standard-dimensional three-dimensional data matrix using a resampling algorithm; The three-dimensional data matrix is ​​converted into a standard tensor format and input into the convolutional neural network. The texture features of the target region are extracted through forward propagation, and the image feature vector is flattened and output.

6. The multimodal image fusion intelligent follow-up management system for chronic diseases according to claim 1, characterized in that, In the data quality assessment module, the step of calculating the image quality score of the medical image data includes: The medical image data is subjected to a Fourier transform and mapped to the frequency domain. The average value of the high-frequency signal energy ratio is extracted, and a sharpness index representing global sharpness is calculated and generated. The difference and variance of the mean gray level between the preset target area and the preset background area are calculated in the spatial domain to construct the contrast-to-noise ratio index, so as to quantify the local resolution capability. The sharpness index and the contrast-to-noise ratio index are input into a preset evaluation model for weighted linear combination, and the image quality score is output after normalization.

7. The multimodal image fusion intelligent follow-up management system for chronic diseases according to claim 1, characterized in that, In the data quality assessment module, the step of statistically analyzing the valid fields of the clinical laboratory data to calculate and obtain a data completeness score includes: The pre-defined chronic disease assessment baseline library is parsed to obtain the required core test field set and auxiliary test field set, and differentiated benchmark weights are assigned to them. Traverse the clinical test data and count the number of actual non-empty fields in the core test field set and the auxiliary test field set; A weighted summation operation is performed based on the actual number of non-empty fields and their corresponding baseline weights, and then divided by the maximum theoretical weight sum to calculate and obtain the data integrity score.

8. The multimodal image fusion intelligent follow-up management system for chronic diseases according to claim 1, characterized in that, In the multi-source information fusion module, the step of using the DS evidence theory algorithm to perform multi-source information fusion calculation on the basic probability allocation value includes: Define an identification framework that includes multiple mutually exclusive disease evolution states, and abstract the image feature vector and the indicator feature vector as independent evidence sources; Using orthogonal sum rules, the joint quality matrix of each independent source of evidence under each state combination is calculated, and the global conflict coefficient characterizing the degree of evidence conflict is extracted. When the global conflict coefficient is lower than a preset threshold, the joint quality matrix is ​​probabilistically normalized, and the posterior confidence of the target deterioration state is calculated as the comprehensive risk value.

9. The multimodal image fusion intelligent follow-up management system for chronic diseases according to claim 1, characterized in that, In the intelligent follow-up decision-making module, the output includes the steps of the follow-up intervention strategy for the next follow-up review time point, including: The comprehensive risk value is input into a preset chronic disease dynamic evolution model to generate the expected risk change curve within the prediction period; Construct a joint objective optimization function that includes the cost of missed diagnosis / delay and the cost of follow-up visits. The multi-objective particle swarm optimization algorithm is used to perform optimization on the expected risk change curve to find the optimal time interval that minimizes the joint objective optimization function, and the time node for the next follow-up review is calculated accordingly.

Images