A dementia diagnosis method, system, electronic device and storage medium

Abstract

The application discloses a dementia diagnosis method, system, electronic equipment and storage medium. The method comprises the following steps: obtaining sample data; converting and rewriting table data by using a table encoder to obtain expression vectors of multi-version semantic natural language expressions of the table data; extracting features of imaging data by using an image encoder to obtain image vectors; initializing a CLS vector of each label category by using a label encoder based on a preset number of label categories; training a gate Transformer by using a preset training strategy based on the CLS vector and expression vectors and image vectors corresponding to sample data and label data, to obtain a dementia prediction model; and predicting a target dementia cause of a target patient by using the dementia prediction model based on table data and / or imaging data of the target patient. The application can accurately realize dementia diagnosis. The application can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, system, electronic device and storage medium for diagnosing dementia. Background Technology

[0002] With the widespread use of electronic medical records and scales (such as the MMSE (Minor Mental State Examination) and MoCA (Montreal Cognitive Assessment)), AI models have begun to be applied to dementia diagnosis. However, current models such as ADRD (Nature Medicine, 2024) and TabPFN (Nature, 2025) suffer from problems such as strong field dependencies, weak label sparsity handling, single-dataset dependence, and shallow modality fusion, which severely limit their clinical application and promotion. ADRD has multimodal and multi-label capabilities, but lacks the ability to automatically generalize to multiple datasets; while TabPFN has the capability for small sample sizes, it cannot integrate multiple datasets and lacks field semantic understanding and multi-label modeling capabilities. Summary of the Invention

[0003] The main objective of this application is to provide a method, system, electronic device, and storage medium for diagnosing dementia, aiming to solve at least one problem of the prior art.

[0004] To achieve the above objectives, one aspect of this application provides a method for diagnosing dementia, the method comprising: Acquire sample data; the sample data includes patients' tabular data, imaging data, and label data; A table encoder is used to transform and rewrite the table data to obtain the expression vector of the multi-version semantic natural language representation of the table data; Image vectors are obtained by extracting features from imaging data using an image encoder. Based on a preset number of label categories, a CLS vector for each label category is initialized using a label encoder; where each label category corresponds to a different cause of dementia, and the label data represents the specific cause. Based on the CLS vector and the corresponding expression vector, image vector and label data of the sample data, a gated Transformer is trained using a preset training strategy to obtain a dementia prediction model. Based on tabular data and / or imaging data of the target patients, the target dementia etiology of the target patients is predicted using a dementia prediction model.

[0005] In some embodiments, tabular data represents patient information. A tabular encoder is used to transform and rewrite the tabular data to obtain a multi-version semantic natural language representation vector of the tabular data, including the following steps: Based on the big semantic model, the dictionary fields of the tabular data are rewritten to generate multiple natural language sentences, which are then converted into multiple versions of semantic natural language expressions. The semantic natural language expression of each version is embedded using a language model to obtain the expression vector corresponding to the semantic natural language expression of each version; In the training process of the gated Transformer, each training round randomly selects only one version of the semantic natural language expression vector as the training data.

[0006] In some embodiments, the imaging data characterizes the patient's magnetic resonance imaging. An image encoder is used to extract features from the imaging data to obtain an image vector, including the following steps: Based on the imaging data, imaging features are extracted using a pre-trained segmentation model with frozen parameters. The imaging features are spatially downsampled using a convolution module, and then the image vector is obtained through global averaging.

[0007] In some embodiments, a dementia prediction model is obtained by training a gated Transformer based on the CLS vector and the corresponding expression vector, image vector, and label data of the sample data using a preset training strategy, including the following steps: The CLS vector, the corresponding representation vector and image vector of the sample data are fed into the gated Transformer as input sequences, and the output representation of the CLS vector corresponding to each label category is obtained by using the output of the gated Transformer. In the training process of the gated Transformer, each training round randomly selects only one version of the semantic natural language expression vector as the input sequence. The output representation corresponding to the CLS vector of each label category is connected to a Sigmoid classifier to obtain the probability prediction result of the cause of dementia for each label category. Based on the probability prediction results and label data, a loss function is constructed using a preset training strategy to optimize the gated Transformer and obtain a dementia prediction model.

[0008] In some embodiments, based on probability prediction results and label data, a loss function is constructed using a preset training strategy to optimize the gated Transformer and obtain a dementia prediction model, including the following steps: A multi-label contrastive learning mechanism is used to construct positive and negative sample pairs for each label category; Positive samples were constructed from data of other patients in the same batch of sample data who had the same label category as the label data and were positive; negative samples were constructed from data of other patients in the same batch of sample data who had the same label category as the label data and were negative. A contrastive loss is constructed using the probability prediction results of positive and negative sample pairs and label data. Based on the contrastive loss, a gated Transformer is optimized using a multiple gradient descent algorithm to obtain a dementia prediction model.

[0009] In some embodiments, a dementia prediction model is obtained by optimizing a gated Transformer using a multiple gradient descent algorithm based on contrastive loss, including the following steps: Construct a multi-objective optimization based on contrastive loss; The multi-objective optimization includes a number of binary cross-entropy losses for the classification task and a number of noise contrastive estimation losses for representation learning. The expression for the multi-objective optimization is:

[0010] In the formula, This indicates the shared parameters for the task, which include parameters for both the table encoder and the image encoder. Indicates the first Unique parameters for a specific task; Indicates the first A loss function; Indicates the target quantity; It is the transpose symbol; Based on multi-objective optimization, a quadratic programming solution is obtained by using a multi-gradient descent algorithm to obtain the optimal combination weights of the gradients for each task. A dementia prediction model is obtained by optimizing the gated Transformer using the optimal combination of weights.

[0011] In some embodiments, based on tabular data and / or imaging data of the target patient, a dementia prediction model is used to predict the target dementia cause of the target patient, including the following steps: A table encoder is used to transform and rewrite the tabular data of the target patients to obtain the target expression vector; And / or, use an image encoder to extract features from the imaging data of the target patient to obtain the target image vector; The CLS vector, target representation vector, and target image vector are fed into the dementia prediction model as an input sequence. The output of the dementia prediction model is used to obtain the target output representation corresponding to the CLS vector of each label category. The target output representation corresponding to the CLS vector of each label category is connected to a Sigmoid classifier to obtain the probability prediction result of the cause of dementia for each label category. The cause of dementia corresponding to the highest probability prediction result is taken as the target cause of dementia.

[0012] To achieve the above objectives, another aspect of this application proposes a dementia diagnostic system, the system comprising: The data acquisition module is used to acquire sample data, which includes patient tabular data, imaging data, and label data. The table encoding module is used to transform and rewrite table data using a table encoder to obtain the expression vector of multi-version semantic natural language representation of the table data; The image encoding module is used to extract features from imaging data using an image encoder to obtain image vectors; The label encoding module is used to initialize the CLS vector of each label category based on a preset number of label categories using the label encoder; where each label category corresponds to a different cause of dementia, and the label data represents the specific cause; The model training module is used to train a gated Transformer based on the CLS vector and the corresponding expression vector, image vector and label data of the sample data, using a preset training strategy to obtain a dementia prediction model. The model prediction module is used to predict the target cause of dementia in the target patient based on the target patient's tabular data and / or imaging data using a dementia prediction model.

[0013] To achieve the above objectives, another aspect of the embodiments of this application proposes an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned method.

[0014] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0015] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method. The embodiments of this application include at least the following beneficial effects: This application provides a method, system, electronic device, storage medium, and program product for diagnosing dementia. This solution acquires sample data, including patient tabular data, imaging data, and label data. A tabular encoder is used to transform and rewrite the tabular data to obtain expression vectors of multi-version semantic natural language expressions of the tabular data. An image encoder is used to extract features from the imaging data to obtain image vectors. Based on a preset number of label categories, a label encoder is used to initialize the CLS vector for each label category. Each label category corresponds to a different cause of dementia, and the label data represents the specific cause. Based on the CLS vectors and the expression vectors, image vectors, and label data corresponding to the sample data, a gated Transformer is trained using a preset training strategy to obtain a dementia prediction model. Based on the target patient's tabular data and / or imaging data, the dementia prediction model is used to predict the target dementia cause of the target patient. This application utilizes field semantic modeling and image feature extraction to map the relationship between multimodal data and dementia, and then optimizes the training of a gated Transformer, effectively enabling the transfer and deployment of dementia diagnosis. This application achieves a general and robust multi-label dementia diagnosis, particularly suitable for the heterogeneity of clinical data and the needs of practical deployment. This application can accurately diagnose dementia. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of an implementation environment for the dementia diagnosis method provided in this application embodiment; Figure 2 This is a flowchart illustrating a dementia diagnosis method provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the architecture and principle of the dementia diagnosis method provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of a dementia diagnosis system provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0018] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0019] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0021] The related technologies suffer from problems such as strong field dependencies, weak label sparsity handling, single dataset dependencies, and shallow modality fusion, which severely limit their clinical application and promotion.

[0022] In view of this, this application provides a method for diagnosing dementia. This method involves acquiring sample data, including patient tabular data, imaging data, and label data; transforming and rewriting the tabular data using a tabular encoder to obtain expression vectors of multi-version semantic natural language expressions of the tabular data; extracting features from the imaging data using an image encoder to obtain image vectors; initializing the CLS vector for each label category using a label encoder based on a preset number of label categories; wherein each label category corresponds to a different cause of dementia, and the label data represents the specific cause; training a gated Transformer using a preset training strategy based on the CLS vectors and the expression vectors, image vectors, and label data of the sample data to obtain a dementia prediction model; and predicting the target dementia cause of the target patient using the dementia prediction model based on the target patient's tabular data and / or imaging data. This application utilizes field semantic modeling and image feature extraction to map the relationship between multimodal data and dementia, and then optimizes the training of a gated Transformer, effectively enabling the transfer and deployment of dementia diagnosis. This application achieves a general and robust multi-label dementia diagnosis, particularly suitable for the heterogeneity of clinical data and the needs of practical deployment. This application can accurately diagnose dementia.

[0023] It is understood that the dementia diagnosis method provided in this application can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal can be a smartphone, tablet, laptop, or desktop computer, but it is not limited to these.

[0024] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided in an embodiment of this application. (Refer to...) Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0025] Server 101 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0026] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0027] Terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the application does not impose any limitations.

[0028] For example, based on Figure 1 The implementation environment shown in this application embodiment provides a dementia diagnosis method. The following description uses the application of the dementia diagnosis method in server 101 as an example. It can be understood that the dementia diagnosis method can also be applied in terminal 102. Specifically, terminal 102 or server 101 can be used to execute the relevant data processing logic of the method of this application to implement the corresponding method flow.

[0029] Reference Figure 2 , Figure 2 This is an optional flowchart of the dementia diagnosis method provided in the embodiments of this application. The subject executing the dementia diagnosis method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 The method may include, but is not limited to, steps S100 to S600.

[0030] Step S100: Obtain sample data; The sample data includes patients' tabular data, imaging data, and label data; For example, in some specific implementations, tabular data represents patient information, imaging data represents the patient's magnetic resonance imaging, and label data represents the specific etiology.

[0031] Step S200: Use a table encoder to transform and rewrite the table data to obtain the expression vector of the multi-version semantic natural language expression of the table data; It should be noted that the tabular data represents the patient's information. In some embodiments, step S200 may include the following steps: rewriting the dictionary fields of the tabular data based on the large semantic model to generate multiple natural language sentences, and then converting them into multiple versions of semantic natural language expressions; using the language model to embed each version of the semantic natural language expression to obtain the expression vector corresponding to each version of the semantic natural language expression; wherein, in the training process of the gated Transformer, each training round randomly selects only one version of the semantic natural language expression's expression vector as training data.

[0032] For example, in some specific implementations, the table encoder first rewrites the field dictionary (including the actual meanings corresponding to database column names, and the mapping between numerical values ​​and actual meanings in the value range, such as Gender -> patient gender; 0 -> male, 1 -> female) based on a large language model, generating multiple natural language sentences (patient gender: male / the patient's gender is male / the patient is a man), converting them into multi-version (or even multilingual) semantic natural language expressions, and then using a language model to embed them to obtain embeddings. These multi-version embeddings are randomly selected in each training epoch. The key points are: ① Dictionary rewriting ensures that the backbone model only focuses on semantics and is robust to subtle differences in expression; ② Parameterless normalization (min-max scaling, calculated based on the entire training set during training; calculated based on the test set during inference; normalization can be performed on individual new sample points based on training data; alternative statistical processing methods exist to avoid data leakage into the training set), allowing the model to accept all fields, rather than relying on certain specific fields during training.

[0033] In some specific application scenarios, normalization modifies the original numerical values. Because for "numerical features," the rewritten text doesn't contain numbers, but rather phrases like "years of smoking:" or "patient's height:" are fed into the LLM. These embeddings are multiplied by the normalized value to obtain the final token fed into the Transformer.

[0034] Step S300: Use an image encoder to extract features from the imaging data to obtain an image vector; It should be noted that the imaging data characterizes the patient's magnetic resonance imaging. In some embodiments, step S300 may include the following steps: extracting imaging features based on the imaging data using a pre-trained segmentation model with frozen parameters; spatially downsampling the imaging features using a convolution module, and then obtaining an image vector through global averaging.

[0035] For example, in some specific implementations, the image encoder uses a pre-trained Swin-UNETR encoder (an open-source brain tumor segmentation model) with frozen parameters to extract MRI (Magnetic Resonance Imaging) features, and then downsamples them through a convolutional module (an additional module added outside of Swin-UNETR; a total of 4 layers, each with a kernel size of 2 and a stride of 2), ultimately reducing the size from 768x4x4x4 to 256x4, and then taking a global average to obtain a single 256-length embedding.

[0036] Step S400: Based on a preset number of label categories, initialize the CLS vector for each label category using a label encoder; Each label category corresponds to a different cause of dementia, and the label data represents the specific cause. For example, in some specific implementations, the label encoder constructs an independent CLS vector for each label (a learnable embedding parameter, a single fixed vector for each label; not generated by LLM (Large Language Model). These CLS vectors, along with the embeddings obtained from the image encoder and table encoder, are fed into the main model, i.e., the gated Transformer mentioned below, for forward propagation; finally, these CLS tokens (classification label vectors) can be used in the output layer as a representation of the sample under different labels for comparative learning / direct classification learning). The CLS vector is a special output vector in Transformer-type models such as BERT used to represent the global semantic information of the entire sentence or text, and is typically used for downstream classification tasks.

[0037] Step S500: Based on the CLS vector and the expression vector, image vector and label data corresponding to the sample data, a gated Transformer is trained using a preset training strategy to obtain a dementia prediction model. It should be noted that in some embodiments, step S500 may include the following steps: feeding the CLS vector and the expression vector and image vector corresponding to the sample data as input sequences into a gated Transformer, and using the output of the gated Transformer to obtain the output representation corresponding to the CLS vector of each label category; wherein, in the training process of the gated Transformer, each training round randomly selects only one version of the semantic natural language expression expression vector as the input sequence; feeding the output representation corresponding to the CLS vector of each label category into a Sigmoid classifier to obtain the probability prediction result of the cause of dementia corresponding to each label category; and constructing a loss function using a preset training strategy based on the probability prediction result and the label data to optimize the gated Transformer to obtain a dementia prediction model.

[0038] For example, in some specific implementations, the fusion module uses a gated Transformer structure for intermediate fusion to achieve semantic interaction between modalities. The gating mechanism is implemented in the FFN (Feedforward Neural Network) layer, with self-gating added after dimensionality increase in the intermediate hidden layers. The specific input to the fusion module is directly fed into the Transformer as an input sequence of the table, image, and label CLS vectors (they are already identical 256-dimensional tokens). The output of the fusion module is the output representation corresponding to each label CLS vector; the outputs of other tokens are discarded. After fusion, a Sigmoid classifier is applied to the output of each label CLS vector to obtain the probability prediction result.

[0039] In some embodiments, based on probability prediction results and label data, a loss function is constructed using a preset training strategy to optimize the gated Transformer and obtain a dementia prediction model. This may include the following steps: constructing positive and negative sample pairs for each label category using a multi-label contrastive learning mechanism; wherein, positive samples are constructed from data of other patients in the same batch of sample data whose label category is the same as the label data and has a positive label; negative samples are constructed from data of other patients in the same batch of sample data whose label category is the same as the label data and has a negative label; constructing a contrastive loss using the probability prediction results of the positive and negative sample pairs and label data; and optimizing the gated Transformer using a multiple gradient descent algorithm based on the contrastive loss to obtain a dementia prediction model.

[0040] For example, in some specific implementations, the multi-label contrastive learning mechanism constructs positive and negative sample pairs for each label to improve the diagnostic performance of sparse labels; wherein the positive sample is other samples with the same positive label in the same batch (the representation of the corresponding label), excluding the sample itself; the negative sample is the sample with the same label as negative in the same batch, without using hard negative sample mining; the contrastive loss is based on the CLS vector of the corresponding label, and dual-temperature contrastive loss is used (but infoNCE is also an optional alternative).

[0041] In some embodiments, optimizing a gated Transformer using a multiple gradient descent algorithm based on contrastive loss to obtain a dementia prediction model may include the following steps: constructing a multi-objective optimization based on contrastive loss; wherein, the multi-objective optimization includes a number of objective binary cross-entropy losses for classification tasks and a number of objective noisy contrastive estimation losses for representation learning, and the expression for the multi-objective optimization is:

[0042] In the formula, This indicates the shared parameters for the task, which include parameters for both the table encoder and the image encoder. Indicates the first Unique parameters for a specific task; Indicates the first A loss function; Indicates the target quantity; It is the transpose symbol; Based on multi-objective optimization, a quadratic programming solution is performed using a multi-gradient descent algorithm to obtain the optimal combination weights for each task gradient; the optimal combination weights are then used to optimize the gated Transformer to obtain a dementia prediction model.

[0043] For example, in some specific implementations, the multiple gradient descent algorithm automatically balances the difficulty of label training, thereby improving the overall model performance.

[0044] During model training, this application requires simultaneous optimization of 2L different loss functions, including L binary cross-entropy losses for classification tasks and L noise-contrastive estimation losses for representation learning. In multi-task learning scenarios, gradient directions of different tasks often conflict or compete. Simply adding all gradients directly may cause some tasks (especially simple tasks with large gradient magnitudes) to dominate the entire training process, thereby impairing the learning effect of other tasks (such as difficult tasks corresponding to sparse labels), resulting in significant differences in model performance on different labels.

[0045] To address this problem, this application employs the concept of multi-objective optimization, transforming this multi-task learning problem into a multi-objective optimization problem. Its mathematical form is as follows:

[0046] The goal of this formula is to find a set of parameters that makes the vector composed of all 2L loss functions optimal overall.

[0047] To solve this multi-objective optimization problem, this application employs the Multiple Gradient Descent Algorithm (MGDA). This algorithm does not directly sum the gradients of each task; instead, it dynamically calculates the optimal combination weights of the gradients for each task by solving a quadratic programming problem. :

[0048] The core of this optimization problem lies in finding a set of weights. This makes the gradient of the shared parameters after weighted combination... The L2 norm (i.e., the modulus) is minimized. This set of weights must satisfy the constraint that the sum is 1 and non-negative.

[0049] Step S600: Based on the tabular data and / or imaging data of the target patient, the target dementia etiology of the target patient is predicted using a dementia prediction model; It should be noted that in some embodiments, step S600 may include the following steps: using a table encoder to transform and rewrite the table data of the target patient to obtain a target expression vector; and / or using an image encoder to extract features from the imaging data of the target patient to obtain a target image vector; The CLS vector, target representation vector, and target image vector are fed into the dementia prediction model as an input sequence. The model outputs the target output representation corresponding to the CLS vector for each label category. A Sigmoid classifier is then applied to the target output representation corresponding to the CLS vector for each label category to obtain the probability prediction result of the dementia cause for each label category. The dementia cause corresponding to the highest probability prediction result is taken as the target dementia cause.

[0050] To explain in detail the principles of the technical solution of this application, the overall process of this application will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principles of this application and should not be regarded as a limitation of this application.

[0051] In view of the shortcomings of existing technologies, this application aims to address the key problems existing in current dementia auxiliary diagnostic models in multi-source medical data environments: field heterogeneity, difficulty in multi-label sparse modeling, superficial modality fusion, and poor model transferability, thereby improving the diagnostic ability and adaptability of the model in real clinical scenarios. Figure 3 As shown, the architectural principles of the TabImFormer architecture proposed in this application to implement the method of this application include: The table encoder, based on a large language model, first rewrites the field dictionary (including the actual meanings corresponding to database column names, and the mapping between numerical values ​​and actual meanings in the value range, e.g., Gender -> patient gender; 0 -> male, 1 -> female) to generate multiple natural language sentences (patient gender: male / the patient's gender is male / the patient is a man), converting them into multi-version (even multilingual) semantic natural language expressions. Then, it uses a language model to embed these expressions. These multi-version embeddings are randomly selected in each training epoch. Key points include: ① Dictionary rewriting ensures the backbone model focuses only on semantics, making it robust to subtle differences in expression; ② Parameterless normalization (min-max scaling, calculated based on the entire training set during training; calculated based on the test set during inference; normalization can be performed on individual new sample points based on training data; alternative statistical processing methods exist to avoid data leakage into the training set), allowing the model to accept all fields, rather than relying on specific fields during training. Image encoder: The pre-trained Swin-UNETR encoder (an open-source brain tumor segmentation model) with frozen parameters is used to extract MRI (Magnetic Resonance Imaging) features. The features are then downsampled using a convolutional module (an additional module added outside of Swin-UNETR; a total of 4 layers, each with a kernel size of 2 and a stride of 2). The final result is a 256x4x4 image, which is then averaged globally to obtain a single 256-length embedding. Label Encoder: Constructs an independent CLS vector for each label (learnable embedding parameters, one fixed vector per label; not generated by LLM (Large Language Model)). These CLS vectors, along with the embeddings obtained from the image encoder and table encoder, are fed into the main model, i.e., the gated Transformer mentioned below, for forward propagation; finally, these CLS tokens can be used in the output layer as a representation of the sample under different labels for comparative learning / direct classification learning). CLS vectors are special output vectors in Transformer-type models such as BERT used to represent the global semantic information of the entire sentence or text, and are typically used for downstream classification tasks.

[0052] The fusion module uses a gated Transformer structure for intermediate fusion, enabling semantic interaction between modalities. The gating mechanism is implemented in the FFN (Feedforward Neural Network) layer (forming a gate feedforward network), and self-gating is added after dimensionality increase in the intermediate hidden layers. The specific input to the fusion module is simply the table, image, and label CLS vectors (which are already identical 256-dimensional tokens) fed into the Transformer. The output of the fusion module is the output representation corresponding to each label CLS vector; the outputs of other tokens are discarded.

[0053] Multi-label contrastive learning mechanism: construct positive and negative sample pairs for each label to improve the diagnostic performance of sparse labels; positive samples are other samples with the same positive label in the same batch (representation of the corresponding label), excluding the sample itself; negative samples are samples with the same label that are negative in the same batch, without using hard negative sample mining; the contrastive loss is based on the CLS vector of the corresponding label and uses dual temperature contrastive loss (but infoNCE is also an optional alternative).

[0054] Multiple gradient descent algorithm: automatically balances the difficulty of label training and improves the overall model performance.

[0055] During model training, this application requires simultaneous optimization of 2L different loss functions, including L binary cross-entropy losses for classification tasks and L noise-contrastive estimation losses for representation learning. In multi-task learning scenarios, gradient directions of different tasks often conflict or compete. Simply adding all gradients directly may cause some tasks (especially simple tasks with large gradient magnitudes) to dominate the entire training process, thereby impairing the learning effect of other tasks (such as difficult tasks corresponding to sparse labels), resulting in significant differences in model performance on different labels.

[0056] To address this problem, this application employs the concept of multi-objective optimization, transforming this multi-task learning problem into a multi-objective optimization problem. Its mathematical form is as follows:

[0057] in, This represents parameters shared by all tasks in the model (e.g., parameters of a table encoder or image encoder), while Then it represents the first Unique parameters for a specific task. Representing the There are 2L loss functions. The goal of this formula is to find a set of parameters that makes the vector formed by all 2L loss functions optimal overall.

[0058] To solve this multi-objective optimization problem, this application employs the Multiple Gradient Descent Algorithm (MGDA). This algorithm does not directly sum the gradients of each task; instead, it dynamically calculates the optimal combination weights of the gradients for each task by solving a quadratic programming problem. :

[0059] The core of this optimization problem lies in finding a set of weights. This makes the gradient of the shared parameters after weighted combination... The L2 norm (i.e., the modulus) is minimized. This set of weights must satisfy the constraint that the sum is 1 and non-negative.

[0060] This method has the following key advantages: 1. Finding the Pareto optimal solution: The algorithm's solution process guarantees two possible results: (1) A Pareto stationary point is found, which satisfies the Pareto optimal KKT (Karush-Kuhn-Tucker) condition, meaning that it is impossible to improve the performance of another task without sacrificing the performance of any task; (2) An update direction is found that can reduce the loss of all tasks at the same time.

[0061] 2. Mitigating Task Conflicts and Balancing Training: By minimizing the norm of the combined gradients, MGDA automatically balances gradient conflicts between different tasks. When gradient directions conflict between tasks, the algorithm assigns weights to find a "compromise" update direction. This effectively prevents any single task (especially the task with the larger gradient) from dominating the update of shared parameters, thus ensuring that all tasks are adequately trained. Ultimately, this alleviates the problem of uneven performance across different labels, improving the overall performance and generalization ability of the model.

[0062] Output: Probability predictions of multiple dementia-related causes (a sigmoid classifier is applied to the output of each label CLS vector after fusion).

[0063] In some specific application scenarios, this application can realize the following solutions (for illustrative purposes only and should not be regarded as a limitation of this application): Example 1: Training and Validation with Public Data NACC is trained on the full dataset, and ADNI is fine-tuned using few-shots; NACC and ADNI are reference datasets. It exhibits stable performance and strong transferability, making it suitable for scientific research validation and cross-database learning scenarios.

[0064] Example 2: Hospital Deployment: The hospital submits a CSV form along with field descriptions. The model automatically identifies fields and outputs probability scores for various etiologies. No manual field mapping required, adaptable to multiple hospital systems.

[0065] Example 3: Remote diagnosis and treatment scenario Patients completed the MoCA (Montreal Cognitive Assessment) and other scales in home or community settings; Remote inference after medical applications or community doctors upload structured CSV data; Suitable for remote areas and initial screening.

[0066] In summary, this application proposes the TabImFormer architecture, which overcomes the bottlenecks of existing methods from multiple perspectives, including field semantic modeling, multi-label comparative learning, cross-modal fusion, and transfer deployment capabilities. It aims to build a general, robust, and scalable multi-label diagnostic system, particularly suited to the heterogeneity of clinical data and practical deployment needs. Specifically, this application improves field generalization ability through field semantic rewriting and multi-version generation; furthermore, it effectively models label heterogeneity through independent representation vectors for each label and a multi-label comparative learning mechanism; simultaneously, for the Transformer mid-term fusion mechanism between tables and MRI images, this application utilizes a multi-task optimization mechanism (MGDA) to ensure that weak label learning is not dominated by the main label; the overall architecture supports joint training and deployment inference across datasets, demonstrating practical feasibility.

[0067] Compared with the prior art, this application has at least the following beneficial effects: 1. Sparse label recognition capability: By comparing and learning naturally amplifying positive sample pairs, it exhibits excellent recognition performance in rare etiologies (0.2% positive samples) such as PRD and ODE; 2. Modal fusion enhances stability: It can degenerate when a modality is missing or certain features are missing, and supports table / image single-modality and image joint mode; 3. Multi-tag optimization mechanism: Multiple gradient descent controls the weights of different label loss functions, improving the overall performance of the model across various etiologies; 4. Simple deployment: It supports CSV / XLSX data and can be run directly with a field dictionary without requiring model reconstruction.

[0068] like Figure 4 As shown in the illustration, this application also provides a dementia diagnosis system 900, which can implement the above-described method. The system includes: The data acquisition module 901 is used to acquire sample data, which includes patient tabular data, imaging data, and label data. The table encoding module 902 is used to transform and rewrite table data using a table encoder to obtain the expression vector of multi-version semantic natural language expression of the table data. The image encoding module 903 is used to extract features from the imaging data using the image encoder to obtain an image vector; The tag encoding module 904 is used to initialize the CLS vector of each tag category based on a preset number of tag categories using the tag encoder; where each tag category corresponds to a different cause of dementia, and the tag data represents the specific cause; The model training module 905 is used to train a gated Transformer based on the CLS vector and the corresponding expression vector, image vector and label data of the sample data, using a preset training strategy to obtain a dementia prediction model. The model prediction module 906 is used to predict the target dementia cause of the target patient based on the target patient's tabular data and / or imaging data using a dementia prediction model.

[0069] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0070] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0071] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0072] like Figure 5 As shown, Figure 5 The hardware structure of an electronic device 1000 according to another embodiment is illustrated. The electronic device 1000 includes: The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (aSIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1002 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RaM). The memory 1002 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 using the network node population optimization method of the embodiments of this application. Input / output interface 1003 is used to implement information input and output; The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004); The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0073] The electronic device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0074] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0075] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0076] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0077] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0078] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0079] The dementia diagnosis method, system, electronic device, storage medium, and program product provided in this application embodiment acquire sample data, including patient tabular data, imaging data, and label data. A tabular encoder is used to transform and rewrite the tabular data to obtain expression vectors of multi-version semantic natural language expressions of the tabular data. An image encoder is used to extract features from the imaging data to obtain image vectors. Based on a preset number of label categories, a label encoder is used to initialize the CLS vector for each label category. Each label category corresponds to a different cause of dementia, and the label data represents the specific cause. Based on the CLS vectors and the expression vectors, image vectors, and label data corresponding to the sample data, a gated Transformer is trained using a preset training strategy to obtain a dementia prediction model. Based on the target patient's tabular data and / or imaging data, the dementia prediction model is used to predict the target dementia cause of the target patient. This application utilizes field semantic modeling and image feature extraction to map the relationship between multimodal data and dementia, and then optimizes the training of a gated Transformer, effectively enabling the transfer and deployment of dementia diagnosis. This application achieves a general and robust multi-label dementia diagnosis, particularly suitable for the heterogeneity of clinical data and the needs of practical deployment. This application can accurately diagnose dementia.

[0080] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0081] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0082] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0084] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0085] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0086] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0087] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0089] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for diagnosing dementia, characterized in that, The method includes the following steps: Acquire sample data; wherein, the sample data includes patient tabular data, imaging data, and label data; The table data is transformed and rewritten using a table encoder to obtain the expression vector of the multi-version semantic natural language expression of the table data; The imaging data is processed using an image encoder to extract features, resulting in an image vector. Based on a preset number of label categories, a CLS vector for each label category is initialized using a label encoder; wherein each label category corresponds to a different cause of dementia, and the label data represents the specific cause; Based on the CLS vector and the expression vector, image vector and label data corresponding to the sample data, a gated Transformer is trained using a preset training strategy to obtain a dementia prediction model. Based on tabular data and / or imaging data of the target patients, the dementia prediction model is used to determine the target dementia etiology of the target patients.

2. The method according to claim 1, characterized in that, The tabular data represents patient information. The process of transforming and rewriting the tabular data using a table encoder to obtain a multi-version semantic natural language representation vector of the tabular data includes the following steps: Based on the large semantic model, the dictionary fields of the table data are rewritten to generate multiple natural language sentences, which are then converted into multiple versions of semantic natural language expressions. The semantic natural language expression of each version is embedded using a language model to obtain the expression vector corresponding to the semantic natural language expression of each version. In the training process of the gated Transformer, each training round randomly selects only one version of the semantic natural language expression's expression vector as training data.

3. The method according to claim 1, characterized in that, The imaging data characterizes the patient's magnetic resonance imaging. The step of extracting features from the imaging data using an image encoder to obtain an image vector includes the following steps: Based on the imaging data, imaging features are extracted using a pre-trained segmentation model with frozen parameters. The imaging features are spatially downsampled using a convolution module, and then the image vector is obtained through global averaging.

4. The method according to claim 1, characterized in that, The dementia prediction model is obtained by training a gated Transformer using a preset training strategy based on the CLS vector, the expression vector corresponding to the sample data, the image vector, and the label data, including the following steps: The CLS vector, the expression vector corresponding to the sample data, and the image vector are fed into the gated Transformer as input sequences, and the output representation corresponding to the CLS vector of each label category is obtained by using the output of the gated Transformer. In the training process of the gated Transformer, each training round randomly selects only one version of the semantic natural language expression's expression vector as the input sequence; The output representation corresponding to the CLS vector of each label category is connected to a Sigmoid classifier to obtain the probability prediction result of the cause of dementia corresponding to each label category; Based on the probability prediction results and the label data, a loss function is constructed using a preset training strategy to optimize the gated Transformer and obtain the dementia prediction model.

5. The method according to claim 4, characterized in that, The process of constructing a loss function based on the probability prediction results and the label data using a preset training strategy to optimize the gated Transformer and obtain the dementia prediction model includes the following steps: A multi-label contrastive learning mechanism is used to construct positive and negative sample pairs for each of the label categories; Positive samples are constructed from data of other patients in the same batch of sample data who have the same label category as the label data and are positive; negative samples are constructed from data of other patients in the same batch of sample data who have the same label category as the label data and are negative. The dementia prediction model is obtained by constructing a contrast loss using the probability prediction results of the positive and negative sample pairs and the label data, and then optimizing the gated Transformer using a multiple gradient descent algorithm based on the contrast loss.

6. The method according to claim 5, characterized in that, The dementia prediction model is obtained by optimizing the gated Transformer using a multiple gradient descent algorithm based on the contrastive loss, including the following steps: A multi-objective optimization is constructed based on the contrastive loss; The multi-objective optimization includes a number of binary cross-entropy losses for the classification task and a number of noise contrastive estimation losses for representation learning. The expression for the multi-objective optimization is: In the formula, This indicates the shared parameters for the task, which include parameters for both the table encoder and the image encoder. Indicates the first Unique parameters for a specific task; Indicates the first A loss function; Indicates the target quantity; It is the transpose symbol; Based on the multi-objective optimization, the quadratic programming solution is performed using the multi-gradient descent algorithm to obtain the optimal combination weights of the gradients for each task. The dementia prediction model is obtained by optimizing the gated Transformer using the optimal combination of weights.

7. The method according to any one of claims 1 to 6, characterized in that, The determination of the target dementia cause in the target patient using the dementia prediction model based on the target patient's tabular data and / or imaging data includes the following steps: The table encoder is used to transform and rewrite the table data of the target patient to obtain the target expression vector; And / or, using an image encoder to extract features from the imaging data of the target patient to obtain a target image vector; The CLS vector, the target representation vector, and the target image vector are fed into the dementia prediction model as an input sequence, and the target output representation corresponding to the CLS vector of each label category is obtained by using the output of the dementia prediction model. The target output representation corresponding to the CLS vector of each label category is connected to a Sigmoid classifier to obtain the probability prediction result of the cause of dementia corresponding to each label category; The dementia cause corresponding to the highest predicted probability is taken as the target dementia cause.

8. A dementia diagnostic system, characterized in that, The system includes: A data acquisition module is used to acquire sample data; wherein, the sample data includes patient tabular data, imaging data, and label data; The table encoding module is used to transform and rewrite the table data using a table encoder to obtain the expression vector of the multi-version semantic natural language expression of the table data; The image encoding module is used to extract features from the imaging data using an image encoder to obtain an image vector; The tag encoding module is used to initialize the CLS vector of each tag category based on a preset number of tag categories using a tag encoder; wherein each tag category corresponds to a different cause of dementia, and the tag data represents the specific cause; The model training module is used to train a gated Transformer based on the CLS vector and the expression vector, image vector and label data corresponding to the sample data, using a preset training strategy to obtain a dementia prediction model. The model prediction module is used to determine the target dementia cause of the target patient based on the target patient's tabular data and / or imaging data using the dementia prediction model.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images