Clock drawing task-driven Alzheimer disease early recognition method

Abstract

The invention discloses a clock drawing task-driven Alzheimer's disease early recognition method, and belongs to the technical field of disease early screening, and the method comprises the steps: collecting a static image and eight types of process signal data of a subject in a clock drawing test, and carrying out the preprocessing of the static image and eight types of process signal data; image space structure features and process signal dynamic features are extracted through a double-flow feature extraction module composed of an improved VGGNet16 network and an MLP; generating a joint feature vector through channel attention weighting and full connection layer fusion; and a polynomial loss function optimization model is adopted, and a recognition result is output through a Softmax layer. According to the method, deep fusion of static and dynamic multi-modal features is realized, key features are effectively highlighted, redundant information is inhibited, samples difficult to classify are focused, the recognition accuracy on a DARWIN data set reaches 92.59%, and the method is simple and convenient to operate, low in cost, capable of being deployed on portable equipment and suitable for clinical screening and primary medical popularization.

Description

Technical Field

[0001] This invention relates to the field of early disease screening technology, specifically to an early identification method for Alzheimer's disease driven by a clock drawing task. Background Technology

[0002] Alzheimer's disease (AD) is an irreversible neurodegenerative disease, and its early stages are the golden window for intervention. Timely identification of early AD behaviors is crucial for precise intervention. Currently, early AD identification mainly relies on professional medical tests such as brain CT, MRI, and PET, or on doctors' diagnoses based on genetics and family history. However, these methods have significant limitations: first, they rely on expensive equipment and specialized institutions, resulting in high costs and insufficient accessibility, and diagnostic delays; second, the clinical manifestations of early AD are highly similar to those of normal brain aging, lacking specificity, leading to low willingness to seek medical attention and a high rate of missed diagnoses, with many cases already progressing to the middle or late stages by the time they are diagnosed.

[0003] To address the aforementioned issues, early Alzheimer's disease (AD) detection through daily behavioral observation has become a research hotspot. The clock drawing test (CDT), as a handwriting task less influenced by education and cultural background, is widely used in clinical practice and research. Existing AD identification methods based on the CDT are mainly divided into three categories:

[0004] (I) Single-modal analysis based on static images

[0005] One type is interpretable models that simulate clinical scoring rules. These models extract visual features such as clock face outlines, digit layout, and pointer accuracy using algorithms to generate quantified scores to infer cognitive states. Methods using the ViT model and API-Net network, for example, achieve an accuracy rate of approximately 80.33%. The other type is end-to-end black-box diagnostic models that directly map images to Alzheimer's disease (AD) states using architectures such as CNNs. Methods based on ResNet50 and VGGNet16 achieve accuracy rates between 74.67% and 81.3%. These methods rely solely on the final image information and cannot reflect the dynamic behavioral characteristics during the drawing process.

[0006] (II) Single-mode analysis based on process signals

[0007] This method captures temporal data during the drawing process using a digital tablet and stylus, including timestamps, pen pressure, coordinate trajectories, speed, and acceleration. It then uses machine learning algorithms (such as SVM and random forest) or deep neural networks for classification. Some methods convert one-dimensional signals into two-dimensional images before extracting features. The highest recognition accuracy of this type of method is approximately 90.4%, but its performance drops significantly on a subset of clock-drawing tests, making it difficult to fully characterize the coupling pattern between cognition and movement.

[0008] (III) Multimodal fusion analysis

[0009] To overcome the limitations of single-modality imaging, researchers have begun to combine static images with process signal features. However, existing methods are mostly simple splicing or shallow fusion, such as mapping process signals to image channels to generate multi-channel images. They lack effective cross-modal semantic alignment and collaborative reasoning mechanisms, resulting in a maximum recognition accuracy of only 81.25%, failing to fully explore fine-grained AD biomarkers. In summary, existing technologies suffer from insufficient feature utilization, shallow fusion strategies, and insufficient attention to difficult-to-classify samples, limiting the accuracy and clinical application of the clock drawing test in early AD identification. Summary of the Invention

[0010] The purpose of this invention is to solve the above-mentioned problems by providing a clock-drawing task-driven early identification method for Alzheimer's disease that can achieve deep fusion of static images and process signals, thereby improving recognition accuracy and robustness.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0012] The present invention provides a clock-drawing task-driven method for early identification of Alzheimer's disease, comprising the following steps:

[0013] (1) Data acquisition and preprocessing

[0014] Static image data and process signal data were collected from the subjects during the clock drawing test. The static image was a clock image drawn by the subjects to display a specified time. The process signal included eight key parameters: total duration, pause time in the air, actual writing time, average speed, average acceleration, jerk, average pressure, and number of strokes.

[0015] Image preprocessing: crop and scale to 256×256 pixels, segment and delete irrelevant areas, retain the clock face image with the highest completion, and perform data enhancement by randomly rotating the image within a range of -5~5° and ±20% to adjust brightness / contrast / saturation;

[0016] Process signal preprocessing: Data is processed using Z-score normalization. During the training phase, zero-mean Gaussian noise in the Uniform range [0.001, 0.005] is added to enhance generalization. No noise is added during the verification and testing phases.

[0017] (2) Dual-stream feature extraction (DFCDT module)

[0018] Image feature extraction: An improved VGGNet16 network is used, with the first three layers frozen. ImageNet pre-trained weights are fine-tuned, and the clock face spatial structure features are extracted through convolution and pooling operations. The features are then mapped to a 4096-dimensional vector through a fully connected layer.

[0019] Process signal feature extraction: A multilayer perceptron (MLP) is used to map low-dimensional process signals to a 512-dimensional high-dimensional space to mine fine-grained dynamic patterns;

[0020] (3) Deep feature fusion (SEFF module)

[0021] Channel attention weighting: Channel attention recalibration is performed on image feature vectors and process signal feature vectors respectively to adaptively highlight key feature responses and suppress redundant information;

[0022] Feature concatenation and fusion: The two types of feature vectors after calibration are concatenated, and nonlinear transformation and dimensionality reduction are performed through two fully connected layers to achieve deep fusion of complementary information between modes and generate a joint feature vector;

[0023] (4) Model training and classification

[0024] The model is optimized using a polynomial loss function, by adjusting the hyperparameter ε to give higher weights to hard-to-classify samples and optimize the classification boundary.

[0025] The joint feature vector is input into the Softmax layer, and the AD category prediction result and probability are output.

[0026] The above-mentioned clock drawing task-driven early identification method for Alzheimer's disease includes: the specified time in step (1) includes 11:05 or 11:10, which correspond to the test requirements of the DARWIN dataset and the NHATS dataset, respectively.

[0027] The above-mentioned clock drawing task-driven early identification method for Alzheimer's disease, wherein: in the process of dual-stream feature extraction and subsequent model training in step (2), the data is divided into training set, validation set and test set in a ratio of 70:15:15, and the training parameters are configured as follows: the optimizer is Adam, the initial learning rate is 0.001, the batch size is 32, the number of training rounds is 100, the learning rate adjustment strategy is StepLR, and the early stopping strategy is set to stop training if the accuracy of the validation set does not improve for 10 consecutive rounds.

[0028] The above-mentioned clock-drawing task-driven early Alzheimer's disease identification method, wherein: the expression of the polynomial loss function (PolyLoss) in step (4) is:

[0029] Among them, P tGiven the target label, γ is the predicted probability, ε is the modulation factor (value 2), ε is the hyperparameter (value 1.6), and N is the number of samples.

[0030] Compared with existing technologies, this invention has significant advantages. As can be seen from the above technical solutions: This invention employs a dual-stream feature extraction module (DFCDT), using CNN and MLP to extract static spatial features and dynamic process features respectively, achieving comprehensive capture of multimodal information and overcoming the limitations of single-modal features. A SEFF module based on a channel attention mechanism is designed to achieve adaptive weighting and deep fusion of cross-modal features, enhancing key feature responses, suppressing redundant information, and improving feature discrimination capabilities. A multinomial loss function is introduced to automatically focus on difficult-to-classify samples, optimize classification boundaries, and improve robustness in recognizing early-stage Alzheimer's disease (AD). On the DARWIN dataset, the recognition accuracy reaches 92.59%, precision 93.65%, recall 92.59%, and F1 score 92.61%, all superior to existing methods. Applied to portable systems, it can be used on tablets and digital styluses of different brands, offering ease of operation, low cost, and high accessibility, facilitating clinical screening and promotion in primary healthcare. Attached Figure Description

[0031] Figure 1 is an overall architecture diagram of the multimodal fusion recognition model (MDADR) of the present invention, showing the full process logic of the dual-stream feature extraction module (DFCDT), the deep feature fusion module (SEFF), and the classification output. It also compares the core differences between the existing methods and the method of the present invention.

[0032] Figure 2 shows example tasks of the DARWIN dataset: (a) Task 2 (four-connection task), (b) Task 21 (complex pattern redrawing task), and (c) Task 24 (clock drawing test task); the sample format of the dataset used in the experiment is clarified.

[0033] Figure 3 is a schematic diagram of image data preprocessing, showing the operation standards for clock face area recognition and redundant area segmentation. The red solid box marks the content to be discarded, and the green solid box marks the core area of ​​the clock face to be retained.

[0034] Figure 4 is a visualization of the convolutional filter. From left to right and from top to bottom, it shows the input image and the output feature maps of the 1st to 5th convolutional blocks, which intuitively present the differences in clock face features extracted by each layer of the VGGNet16 network. Detailed Implementation

[0035] The following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and efficacy of the clock-drawing task-driven early Alzheimer's disease identification method proposed in this invention:

[0036] Example 1:

[0037] A clock-drawing task-driven method for early identification of Alzheimer's disease includes the following steps:

[0038] (I) Experimental Environment and Core Parameter Configuration

[0039] Hardware environment: The experiment ran on Windows 11 Professional 64-bit operating system, with a 12th Gen Intel® Core™ i7-12700KF processor (3.60 GHz), 32 GB of memory, and an NVIDIA GeForce RTX4080 SUPER graphics card (16 GB of video memory). The data acquisition device used a pressure-sensitive digital tablet computer and a matching stylus (pressure sensing range 0-16384 levels).

[0040] Software environment: Deep learning architecture is implemented based on Python 3.8, PyTorch 2.4.1 and CUDA Toolkit 11.8. Auxiliary tools include OpenCV (image preprocessing), Pandas (data processing), and Tkinter (visual interface development).

[0041] The core parameters of the model are as follows: loss function is PolyLoss (γ=2, ε=1.6), activation function is ReLU, optimizer is Adam, initial learning rate is 0.001, batch size is 32, training epochs are 100, output layer is Softmax, learning rate adjustment strategy is StepLR (step size 50, decay coefficient 0.1), and early stopping strategy is set to stop training if the accuracy on the validation set does not improve for 10 consecutive epochs.

[0042] (II) 1. Data Acquisition and Preprocessing

[0043] 1.1 Dataset Selection and Partitioning

[0044] The DARWIN dataset: Task 24 (the clock drawing test) was selected, requiring participants to draw a clock displaying 11:05. It contains paired data from 174 participants (85 healthy individuals with HC and 89 AD patients). All AD patients were clinically diagnosed using the MMSE (Mini-Mental State Examination) and MoCA (Montreal Cognitive Assessment). The dataset was divided into a training set (122 samples), a validation set (26 samples), and a test set (26 samples) in a 70:15:15 ratio. A one-to-one pairing dataset of static images and process signals was constructed, and samples containing only single-modal data were removed. See [link to documentation]. Figure 2 (c);

[0045] The NHATS dataset contains 395 clock-drawing test images (11:10 ratio) extracted from 195 AD patients (positive samples) and 200 healthy individuals (negative samples). These images are divided into a training set (277 samples), a validation set (59 samples), and a test set (59 samples) in a 70:15:15 ratio.

[0046] 1.2 Image Preprocessing

[0047] Step 1: Use the YOLO object detection model to automatically identify clock face regions in the image, and remove invalid samples that have no clock face or whose clock face occupies less than 30% of the image area. See [link to relevant documentation]. Figure 3 ;

[0048] Step 2: Crop and scale the identified clock face area to 256×256 pixels, segment and delete redundant areas unrelated to the clock face (such as handwritten signatures, blank backgrounds), mark discarded content with red solid lines, and mark the retained core area of ​​the clock face with green solid lines. See [link to relevant documentation]. Figure 3 ;

[0049] Step 3: Perform data augmentation on the cropped image, sequentially performing: random rotation (range -5~5°, probability 60%), brightness / contrast / saturation adjustment (range ±20%, random values), to enhance the model's generalization ability;

[0050] 1.3 Process Signal Preprocessing

[0051] Step 1: Extract 8 key parameters from the raw time-series data collected by the digital tablet: total time, air time, actual writing time, average speed (Avg_v), average acceleration (Avg_a), jerk, average pressure (Avg_p), and number of strokes. See Table 1 for parameter definitions and units.

[0052] Table 1. Names and Definitions of Writing Behavior Characteristics

[0053]

[0054] Step 2: Calculate the mean (μ) and standard deviation (σ) of each parameter based on the training set data, and use the Z-score standardization formula (x'=(x-μ) / σ) to process the training set, validation set, and test set data in a unified manner to avoid data leakage;

[0055] Step 3: Data augmentation during training: In each batch, noise intensity σ is sampled from the Uniform interval [0.001, 0.005], and independent zero-mean Gaussian noise (N (0, σ²)) is added to the 8 process signal features of each sample. No noise is added to the validation set and the test set.

[0056] 2. Two-stream feature extraction (DFCDT module)

[0057] 2.1 Image Feature Extraction

[0058] Step 1: Using the improved VGGNet16 network, freeze the first three convolutional layers (using ImageNet pre-trained weights), and train only the last three convolutional layers and fully connected layers;

[0059] Step 2: Input the preprocessed 256×256 pixel image into the network, and extract features through 5 convolutional blocks (Conv→ReLU→MaxPool). Blocks 1-2 capture shallow features such as clock face edges and contours; blocks 3-4 extract mid-level features such as digit layout and hand shape; and block 5 integrates global structural features. (See [link to relevant documentation]). Figure 4 (Input image and output feature maps of each convolutional block);

[0060] Step 3: Map the features to a 4096-dimensional vector using a fully connected layer, and set the dropout probability to 0.5;

[0061] 2.2 Process Signal Feature Extraction

[0062] Step 1: Construct a 3-layer MLP network (8-dimensional input layer → 256-dimensional hidden layer → 512-dimensional hidden layer → 512-dimensional output layer), and input the standardized 8 types of process signals into the network;

[0063] Step 2: Implement nonlinear mapping using the ReLU activation function, set the dropout probability to 0.3, and finally output a 512-dimensional process feature vector;

[0064] 3. Deep Feature Fusion (SEFF Module)

[0065] 3.1 Channel Attention Weighting

[0066] Step 1: Perform channel attention calibration on the 4096-dimensional image feature vector and the 512-dimensional process signal feature vector respectively. The process is "global average pooling → fully connected layer (reduced to 1 / 16 of the original dimension) → ReLU → fully connected layer (restored to the original dimension) → Sigmoid", generating channel weights. The feature recalibration logic of this process can be understood with the help of the structural diagram of the SEFF module in Figure 1. See Figure 1.

[0067] Step 2: Use the generated channel weights to weight the original features channel by channel, strengthen the response of key features and suppress redundant information;

[0068] 3.2 Feature splicing and fusion

[0069] Step 1: Concatenate the calibrated image feature vector (4096-dimensional) and the process signal feature vector (512-dimensional) along the channel dimension to obtain a 4608-dimensional joint feature vector. The concatenation method is the same as... Figure 1 The fusion path of dual-modal features is consistent; see [link / reference]. Figure 1 ;

[0070] Step 2: Input the joint feature vector into two fully connected layers (4608-dimensional → 2048-dimensional → 1024-dimensional), and perform nonlinear transformation through the ReLU activation function in between to complete the deep fusion of complementary information between modes;

[0071] 4. Model Training and Classification

[0072] 4.1 Classification Prediction

[0073] Step 1: Input the fused 1024-dimensional feature vector into the Softmax layer and output the predicted probabilities of AD and HC classes. When the AD probability is >50%, it is judged as AD positive; otherwise, it is HC negative.

[0074] 4.2 Model Training

[0075] Step 1: During training, the error between the predicted value and the true label is calculated using the PolyLoss loss function, and the model parameters are updated using the backpropagation algorithm (the weights of the first three layers of the frozen VGGNet16 are not updated).

[0076] Step 2: After each round of training, evaluate the accuracy on the validation set and save the model weights with the best accuracy on the validation set;

[0077] 4.3 Performance Evaluation

[0078] Step 1: In the testing phase, load the optimal model weights, input the test set data, and output the classification results;

[0079] Step 2: Calculate accuracy, precision, recall, and F1 score to evaluate model performance. On the clock drawing test subset of the DARWIN dataset, the model achieved an accuracy of 92.59%, precision of 93.65%, recall of 92.59%, and an F1 score of 92.61%. The overall performance of the model outperforms existing methods. See the comparison results for further details. Figure 1 The architectural differences and performance of the method of this invention compared with existing methods are illustrated in Figure 1.

[0080] Experiment 1: Portable System Deployment and Testing Implementation

[0081] 1. System Module Deployment

[0082] Data acquisition module: The subject is guided to complete the clock drawing test through a tablet computer touch interface (displaying "Please draw the clock at 11:05, including the numbers 1-12 and the hands"). The system automatically records static images (PNG format) and time-series data (CSV format, including timestamps, x / y coordinates, and pressure values) of the drawing process. The interaction logic of each module of the system is consistent with the overall architecture shown in Figure 1. See Figure 1.

[0083] Data processing module: Automatically calls the above preprocessing algorithm to standardize the acquired images and process signals. When there is a single modal input (such as only acquiring images), the other modal data is set to a vector of all zeros.

[0084] Recognition module: Loads the trained MDADR model, performs feature extraction, fusion and classification on the processed data, and outputs AD probability and classification conclusion;

[0085] Visualization module: Real-time display of the acquired clock face image, process signal curves (speed-time, pressure-time), and recognition results (AD probability, classification conclusion);

[0086] 2. Case Test Implementation and Result Verification

[0087] Test Plan A (Real Sample Test): Ten samples each of AD and HC categories were randomly selected from the validation set. Data was re-entered through the system's data acquisition terminal (simulating actual clinical use scenario), and then entered into the system for recognition. The results showed that 9 AD samples were correctly identified (90% accuracy), and 8 HC samples were correctly identified (80% accuracy), with a total accuracy of 85%, verifying the system's reliability in recognizing real samples.

[0088] Test Plan B (Imitation Sample Test): Ten healthy volunteers were recruited to imitate ten clock face images from Plan A (only imitating the static form, unable to replicate the real process signals). The static images after imitation and the volunteers' own process signals were collected and input into the system. The results showed that 8 AD class clocks were correctly identified (80% accuracy), and 6 HC class clocks were correctly identified (60% accuracy), with an overall accuracy of 70%. This verified the crucial role of process signals in the recognition results and demonstrated the system's robustness in non-real sample scenarios.

[0089] Experiment Example 2: Verification of the effectiveness of key steps

[0090] Feature extraction effectiveness verification: The activation feature maps of each convolutional block of VGGNet16 are displayed using visualization tools. See Figure 4. It can be seen that the shallow convolutional blocks (blocks 1-2) clearly capture basic features such as the outline of the clock face and the straight line of the hands. The middle convolutional blocks (blocks 3-4) focus on discriminative features such as the continuity of digit writing and the proportion of the hands. The deep convolutional block (block 5) integrates the features of the rationality of the global layout, proving the effectiveness of feature extraction.

[0091] Validation of the fusion strategy effectiveness: The performance of five fusion functions, concat, max, avg, add, and multi, was compared on the test set. The results show that the concat strategy has the best accuracy (92.59%), precision (93.65%), recall (92.59%), and F1 score (92.61%), proving that the channel splicing fusion method can retain the complementary information of the two modes to the greatest extent. The advantages of this fusion strategy can be further understood in conjunction with the design logic of the SEFF module in Figure 1. See Figure 1.

[0092] Loss function optimization verification: Comparing the model performance under different ε values, when ε=1.6, the model's recognition accuracy for hard-to-classify samples (early AD and normal aging samples) is improved by 3.7 percentage points compared with ε=0 (focus loss), proving the effectiveness of PolyLoss in optimizing hard examples.

[0093] The above are merely preferred embodiments of the present invention and do not impose any limitations on the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from any technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A clock-drawing task-driven early recognition method of Alzheimer's disease, characterized in that, Includes the following steps: (1) Data acquisition and preprocessing: Static image data and process signal data of the subjects drawing clocks were acquired. The static image is a clock image drawn by the subjects to display a specified time. The process signal includes eight key parameters: total duration, pause time in the air, actual writing time, average speed, average acceleration, jerk, average pressure, and number of strokes. The static image was cropped and scaled, redundant region segmented and data augmented. The process signal was Z-score normalized and zero-mean Gaussian noise was added during the training phase. (2) Dual-stream feature extraction: The spatial structure features of static images are extracted using an improved VGGNet16 network and mapped to a 4096-dimensional vector through a fully connected layer; the process signals are mapped to a 512-dimensional high-dimensional feature vector using a multilayer perceptron (MLP); (3) Feature deep fusion: Channel attention recalibration is performed on the image feature vector and the process signal feature vector respectively. The two types of feature vectors after calibration are concatenated and nonlinear transformation and dimensionality reduction are performed through two fully connected layers to generate a joint feature vector. (4) Model training and classification: The model is optimized using a polynomial loss function. The joint feature vector is input into the Softmax layer, and the Alzheimer's disease category prediction results and probabilities are output. 2.The clock-drawing task-driven early identification method of Alzheimer's disease according to claim 1, wherein, The specified time in step (1) is 11:05 or 11:10, which correspond to the test requirements of the DARWIN dataset and the NHATS dataset, respectively. 3.The clock-drawing task-driven early identification method of Alzheimer's disease according to claim 1, wherein, The static image preprocessing in step (1) is as follows: crop and scale to 256×256 pixels, segment and delete irrelevant areas, retain the clock face image with the highest completion, and achieve data enhancement by randomly rotating it within the range of -5~5° and ±20%. 4.The clock-drawing task-driven early identification method of Alzheimer's disease according to claim 1, wherein, In step (1), the zero-mean Gaussian noise added during process signal preprocessing has an intensity range of Uniform [0.001, 0.005]. No noise is added during the verification and testing phases. 5.The clock-drawing task-driven early identification method of Alzheimer's disease according to claim 1, wherein, In step (2), the first three layers of the improved VGGNet16 network are frozen, and the weights are fine-tuned using ImageNet pre-trained weights. Features are extracted through 5 convolutional blocks. The multilayer perceptron has a 3-layer structure, with an 8-dimensional input layer, 256-dimensional and 512-dimensional hidden layers, and a 512-dimensional output layer. 6.The clock-drawing task-driven early identification method of Alzheimer's disease according to claim 1, wherein, In step (3), the channel attention recalibration process is as follows: "Global average pooling → full connected layer dimensionality reduction to 1 / 16 of the original dimension → ReLU activation → full connected layer restoration to the original dimension → Sigmoid generation of weights". 7.The clock-drawing task-driven early identification method of Alzheimer's disease according to claim 1, wherein, In step (4), the modulation factor γ of the polynomial loss function is 2 and the hyperparameter ε is 1.

6.

8. The method for early identification of Alzheimer's disease driven by the clock-drawing task according to claim 1, characterized in that, The data was divided into training, validation, and test sets in a 70:15:15 ratio. The training parameters were configured as follows: the optimizer was Adam, and the initial learning rate was... 0.001, batch size 32, training epochs 100, learning rate adjustment strategy is StepLR, early stopping strategy is set to stop training if the validation set accuracy does not improve for 10 consecutive epochs.

Images