Multitask serial scoring system and method for MoCA scale clock drawing test

Abstract

The invention relates to a multi-task serial scoring system and method for a MoCA scale clock drawing test. The multi-task serial scoring system comprises a clock image preprocessing module, a clock segmentation module, a contour recognition module, a digital recognition module and a pointer recognition module. The image preprocessing module preprocesses the clock image to improve image quality. The clock segmentation module segments the preprocessed clock image into three parts of a clock outline, numbers and pointers. And then the contour recognition module, the number recognition module and the pointer recognition module classify contours, numbers and pointers of the clock in sequence. And finally, results of all the identification modules are fused to obtain a total score of the clock. The problems that a current image analysis method for a clock drawing test lacks a three-score evaluation method, the influence of human factors is large, the recognition rate is low and the efficiency is low are solved, clock image automatic scoring based on the three-score evaluation method is achieved, the evaluation efficiency is improved, the problem of positioning errors caused by different writing habits of testees is avoided, and real-time processing can be achieved.

Description

Technical Field

[0001] The present invention relates to a scoring system technology, and particularly to a system and method for automatically scoring the clock-drawing test task in the MoCA scale. Background Art

[0002] Chronic brain diseases represented by cerebrovascular diseases, Alzheimer's disease, and Parkinson's disease are important factors threatening the health of the elderly and restricting the quality of life of the elderly. Early detection and early intervention are the key strategies to reduce their harm. These brain diseases may cause varying degrees of cognitive impairment. In this regard, the Montreal Cognitive Assessment (MoCA scale) and the clock-drawing test scale are two common scales used for screening patients with cognitive impairment.

[0003] Currently, for the Montreal Cognitive Assessment and the clock-drawing test scale, medical workers usually need to verbally prompt patients to draw a clock on a paper-based scale and score each scoring point. This has the following disadvantages: 1. The subjectivity of medical workers in scoring is relatively large, and different medical workers may have different scoring results for the same clock image; 2. It increases the workload of doctors during outpatient visits, or requires more other medical workers to complete this task. However, there are not many software that can currently automatically judge the drawn clock.

[0004] The Montreal Cognitive Assessment is a tool for evaluating the degree of cognitive impairment in patients. As a sub-scoring item of the Montreal Cognitive Assessment, the clock-drawing test requires patients to draw a clock outline, fill in all the numbers, and make the hands point to 10 minutes past 11. This clock-drawing test uses a 3-point scoring method in the Montreal scale, with 1 point for the outline, 1 point for the numbers, and 1 point for the hands.

[0005] After retrieval, the patent with the Chinese patent application number CN201911300342.4 discloses an automatic scoring method for the clock-drawing test using a 4-point rating method. The 4-point rating method is divided into 1 point for the outline, 1 point for the presence of numbers, 1 point for the correct position of the numbers, and 1 point for the correct pointing of the hands. This method scores the hand-drawn clock at 10 minutes past 11. First, the clock image is processed into a grayscale image, and then the ratio of the area enclosed by the clock outline to the area of the smallest circumscribed circle of the clock outline is calculated to determine whether the clock outline meets the scoring criteria. Then, object detection technology is used to detect whether the numbers and hands exist in the image and obtain their coordinate values. According to the coordinate values of the numbers and the coordinates of the center of the smallest circumscribed circle, the vector from the center to the numbers is obtained, and according to the included angle values between different vectors, it is judged whether the position of the numbers is correct. According to the included angle between the vector of the drawn hand and the vector of the standard hand, it is judged whether the hand meets the scoring criteria.

[0006] The patent with Chinese Patent Application Number CN201911176388.X discloses an automatic scoring method for the clock-drawing test using a 7-point rating method. The 7-point rating method includes 1 point for the outline, 1 point for no omission of numbers, 1 point for correct number positions, 1 point for correct number order, 1 point for the presence of the minute hand and hour hand, 1 point for correct pointer angles, and 1 point for correct relative lengths of the minute hand and hour hand. This method scores the hand-drawn clock at 3:40. First, a device with a photographing function is used to take a picture of the drawn clock image. The first target detection network is used to detect the coordinate range of the clock in the image and crop the clock. Then, the second target detection network is used to obtain the coordinate information of the numbers 1-12 and the hour hand and minute hand. If the coordinate information of all the numbers from 1 to 12 is detected, then there is no omission of numbers. Based on the coordinate information of the 12 numbers, it can be judged whether the relative positions of the numbers conform to the established standards. If the relative positions of the coordinates conform to the standards, then the number positions are correct. The digital coordinate information is converted into polar coordinate form to judge whether the digital coordinate information is within a certain angular range. If it is within the corresponding angular range, then the number order is judged to be correct. If the coordinate information of the minute hand and hour hand is detected, then the minute hand and hour hand are present. By calculating the distances between the hour hand and the numbers 3 and 4, and between the minute hand and the numbers 7, 8, and 9, it is judged whether the hour hand and the minute hand point correctly. Finally, based on the coordinate ranges of the hour hand and the minute hand, their lengths are calculated to judge whether the relative lengths of the hour hand and the minute hand conform to the standards.

[0007] Currently, for the Montreal Cognitive Assessment Scale and the Clock Drawing Test Scale, medical workers usually need to verbally prompt patients to draw a clock on a paper-based scale and manually score each scoring point. This may easily cause subjective judgment errors and increase the workload of medical workers. However, there are not many software that can currently achieve automatic judgment of clock drawing.

[0008] The current image analysis methods for the clock-drawing test mainly target the clock-drawing test scales using the 4-point and 7-point rating methods, and do not analyze the clock-drawing test tasks using the 3-point rating method in the Montreal Scale. These image analysis methods first use object detection technology to identify and obtain the coordinate information of the numbers and the pointers, and then calculate whether the relative coordinate information of the dial, numbers, and pointers conforms to the standards according to the scoring rules. These methods require a large amount of complex coordinate calculations and have poor real-time performance. In addition, there are differences in the writing habits and fonts of different test takers, which will affect the recognition and positioning of numbers, and further lead to inaccurate subsequent coordinate calculations and judgments, thus reducing the scoring accuracy.

[0009] Disadvantages of the existing invention: 1. It is impossible to score the clock drawing test with the 3-point rating method in the MoCA scale; 2. Due to different writing habits of the testees, if the drawn clock outline is a regular shape such as a square and meets the outline score standard, it is impossible to judge as a score. Also, when the numbers are written large or irregularly, misjudgment may occur because accurate coordinates of the numbers cannot be given; 3. After obtaining the coordinate information of the numbers and pointers through target detection, a large number of complex coordinate calculations are required to determine whether the score standard is met. Summary of the Invention

[0010] Aiming at the problems of the current image analysis method for the clock drawing test, such as the lack of the 3-point rating method, great influence of human factors, low recognition rate, and low efficiency, a multi-task serial scoring system and method for the clock drawing test of the MoCA scale are proposed.

[0011] The objective of the present invention is to achieve automatic recognition and scoring of hand-drawn clock images for the 3-point clock drawing test of the Montreal Cognitive Assessment Scale. The present invention proposes a new method based on deep learning for image segmentation and image classification, realizing automatic scoring judgment of hand-drawn clock images with the 3-point scoring method. Doctors only need to let patients draw clock images on the touch screen or input photos of the drawn clock images to obtain the corresponding scores, which can reduce the consumption of manpower for the screening of brain diseases and improve the objectivity and accuracy during screening.

[0012] The technical solution of the present invention is as follows:

[0013] A multi-task serial scoring system for the clock drawing test of the MoCA scale includes two parts: multi-task partitioning of clock recognition and serial lightweight multi-task recognition. The multi-task partitioning of clock recognition includes a clock image preprocessing module and a clock segmentation module. The serial lightweight multi-task recognition includes a contour recognition module, a number recognition module, and a pointer recognition module; the image preprocessing module preprocesses the clock image to improve the image quality; the clock segmentation module segments the preprocessed clock image into three parts: the contour, numbers, and pointers of the clock; then the contour recognition module, number recognition module, and pointer recognition module classify the contour, numbers, and pointers of the clock in sequence; finally, the results of each recognition module are fused to obtain the total score of the clock.

[0014] The purpose of the clock image preprocessing module is to adjust the image size to match the input size required by the clock segmentation module, eliminate the adverse impact of the image background on the image quality, and ensure that the background pixel value is 0 while the foreground pixel value of the clock is 1;

[0015] First, leave a white margin around the input original clock image at 15% of the clock size to obtain an image with a black clock in the middle, a white background, and a white margin around it, and adjust the length and width of the image to 256; then convert the original RGB image into a grayscale image, and convert the grayscale image into a binary image with a black background and a white clock foreground; finally, perform normalization processing on the pixel values of the image to obtain an image with a background pixel value of 0 and a clock pixel value of 1;

[0016] The function of the clock segmentation module is to segment a clock image into three images of the clock's contour, numbers, and hands for subsequent separate recognition of the contour, numbers, and hands in sequence; and the recognition result of the upper-level model in the recognition process is used as the input of the lower-level model;

[0017] The clock segmentation module includes a lightweight neural network for image segmentation; select the UNet, Seaformer, or EGE-UNet segmentation network; and perform model lightweight processing on the segmentation network, reducing the number of convolutional kernels and replacing them with convolutional kernels of smaller sizes; the clock segmentation module segments the image with a background pixel value of 0 and a clock foreground pixel value of 1 into three images with a background pixel value of 0, and the pixel values of the contour, numbers, and hands are 1 respectively;

[0018] The contour recognition module performs binary classification on the contour image with a background pixel of 0 and a contour pixel of 1, divided into two categories: no score or score; the contour recognition module includes a neural network for image binary classification; this neural network selects AlexNet, reduces the number of convolutional kernels on the basis of AlexNet, replaces them with convolutional kernels of smaller sizes, and replaces the fully connected layer with a lightweight attention module to achieve lightweight processing; the contour recognition module can also be replaced by other lightweight neural networks for image classification, such as MobileNet, ShuffleNet; the input data of the contour recognition module is the segmented contour image, and the output is the contour category label and the evaluation score value corresponding to the label; the corresponding evaluation score for the category of no score is 0, and the corresponding evaluation score for the category of score is 1; judge whether to perform number recognition according to the contour category label, and only perform number recognition for those with a score, and directly output a total score of 0 for those with no score;

[0019] The digital recognition module performs three-class classification on digital images with a background pixel value of 0 and a digital pixel value of 1, dividing them into digital category 1, digital category 2, and digital category 3. The digital recognition module includes a lightweight neural network for image three-class classification, with the same architecture as the neural network of the contour recognition module. The digital recognition module can also be replaced by other lightweight neural networks for image classification, such as MobileNet and ShuffleNet. The input of the digital recognition module is the segmented digital image, and the output is the digital category label and the evaluation score value corresponding to the label. When the digital category label value is 1, the evaluation score of the digit is 1 point. When the digital category label value is 2 or 3, the evaluation score of the digit is 0 points. Whether to perform pointer recognition is determined according to the digital category label. When the digital category label value is 1 or 2, pointer recognition continues, otherwise, the calculated total score is directly output as 0.

[0020] The pointer recognition module performs binary classification on pointer images with a background pixel value of 0 and a pointer pixel value of 1, dividing them into two categories: no score or score. The pointer recognition module uses an image binary classification neural network with the same architecture as the contour recognition module but different parameters. Similarly, it can also be replaced by other lightweight neural networks for image classification, such as MobileNet and ShuffleNet. The input of the pointer recognition module is the segmented pointer image, and the output is the evaluation score value corresponding to the pointer category label. The corresponding evaluation score for the category of no score is 0, and the corresponding evaluation score for the category of score is 1. Therefore, the output of the final serial lightweight multi-task recognition module is the sum of the evaluation score values output by the contour, digital, and pointer recognition modules, and this is used as the total score for the clock drawing evaluation.

[0021] Furthermore, when the test subject draws a clock, there are five situations for the drawn contour: the contour is not drawn, the contour is a random graffiti pattern, the contour is a circle, the contour is an ellipse, and the contour is a square. The first two situations belong to the contour not scoring. For a clock with a non-scoring contour, after passing through the clock image preprocessing module and the clock segmentation module, the segmented contour image will be judged as not scoring by the contour recognition module, and thus the total score of the clock is directly obtained as 0 points. The latter three situations belong to the contour scoring. Therefore, the contour recognition module classifies the contour into two categories: scoring and not scoring.

[0022] Further, when the subject draws a clock, there are four situations for the drawn numbers: some numbers are missing and not fully drawn, some numbers are drawn redundantly, the positions of the numbers are drawn wrongly, and the numbers are complete and in the correct positions. Among them, the first three situations result in no score for the contour, and the last situation results in a score for the contour. Among the first three situations where the contour gets no score, if the numbers 11 and 2 are drawn in the correct positions, then it can also be determined whether the pointer points to 11:10. Therefore, the numbers can be divided into three categories: Number Category 1: The numbers are drawn correctly and score; Number Category 2: The numbers are drawn wrongly, but the numbers 11 and 2 are drawn correctly, and the pointer can be further determined; Number Category 3: The numbers are drawn wrongly and the pointer cannot be further determined. So, the number recognition module is used to perform three-classification on the numbers.

[0023] Further, when the subject draws a clock, there are four situations for the drawn pointer: the pointer is not drawn, the number of drawn pointers is wrong, two pointers are drawn but point wrongly, and the pointer points correctly with the correct number of pointers. Among them, the first three situations result in no score for the pointer, and the last situation results in a score for the pointer. Therefore, two-classification is performed on the pointer.

[0024] A multi-task serial scoring method for the clock drawing test of the MoCA scale, which is used to realize automatic scoring for the multi-task serial scoring system for the clock drawing test of the MoCA scale as described above. The automatic scoring process consists of the following 9 steps:

[0025] Step 1: After inputting the original clock image, leave a white margin around the clock according to 15% of the size of the clock in the middle of the image.

[0026] Step 2: Adjust the size of the clock image.

[0027] Step 3: Grayscale the clock image and reverse binary it into a binary image with a black background and a white clock foreground.

[0028] Step 4: Normalize the clock image to obtain a clock image with a background pixel value of 0 and a clock foreground pixel value of 1.

[0029] Step 5: Input the clock image into the clock segmentation module to segment it into three images of the contour, numbers, and pointer with a background pixel value of 0 and a foreground pixel value of 1.

[0030] Step 6: Input the contour image into the contour recognition module to obtain the judgment result of the contour. If the judgment result of the contour is a score, then continue with the recognition of the numbers. If the judgment result of the contour is no score, then by default, the numbers and the pointer are considered no score, and jump to Step 9.

[0031] Step 7: Input the number image into the number recognition module to obtain the category result of the numbers. If the category of the numbers is 1 or 2, then continue with the recognition of the pointer. If the category of the numbers is 3, jump to Step 9.

[0032] Step 8: The pointer image is input into the pointer recognition module to obtain the judgment result of the pointer.

[0033] Step 9: The overall score of the clock-drawing test is obtained by integrating the scoring results of the contour, numbers, and pointer.

[0034] The beneficial effects of the present invention are as follows:

[0035] The present invention designs a multi-task serial lightweight system using image segmentation and image classification technologies, realizing automatic, rapid, and accurate scoring of the drawn clock images, while reducing the occupation of medical staff resources.

[0036] The present invention has the following advantages: 1. It realizes automatic scoring of clock images using the 3-point assessment method, improving the evaluation efficiency and avoiding the subjective errors of manual scoring by medical staff. 2. By separately recognizing a raw image after segmenting it into three parts: contour, numbers, and pointer, it effectively solves the problem of large computational complexity in identifying clock images by target detection technologies in existing methods, and also avoids the positioning error problem caused by different writing habits of the test subjects. 3. The system uses serial recognition of contour, numbers, and pointer. The neural network model used has the characteristics of a small number of parameters and low computational resource occupancy, enabling real-time processing and being suitable for deployment on embedded development boards with limited computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a composition diagram of the multi-task serial lightweight system for the clock-drawing test of the MoCA scale according to the present invention;

[0038] Figure 2 It is a preprocessing module diagram of the clock image according to the present invention;

[0039] Figure 3 It is an identification process diagram of the three lightweight identification tasks according to the present invention;

[0040] Figure 4 It is an automatic scoring process diagram of the clock-drawing test according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0041] The present invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives detailed implementation manners and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.

[0042] The innovation points of the present invention:

[0043] 1. Automatic scoring of the clock-drawing test for multitask implementation: To achieve the automatic scoring of the clock-drawing test in the Montreal Cognitive Assessment, and effectively avoid the problem that the accuracy of object detection technology in identifying clock images in existing methods is easily affected by the writing habits of the test subjects. After preprocessing the clock image, the present invention uses a segmentation model to divide a clock image into three images: a contour image, a digit image, and a pointer image, and divides the single clock scoring task into contour, digit, and pointer recognition tasks. By separately recognizing the contour, digits, and pointer, the recognition process does not require calculating the relative position of the object, and can effectively solve the problem of mispositioning of the contour, digits, and pointer caused by different writing habits of the test subjects during the recognition and positioning process of object detection technology.

[0044] 2. Serial lightweight multitask recognition: To achieve fast and accurate recognition of the contour, digits, and pointer, a lightweight model is used to serially perform real-time recognition on the contour, digits, and pointer of the clock in sequence. The recognition result of the previous-level model in the recognition process is used as the input of the next-level model. Compared with parallel recognition that gives three feature scores simultaneously, this method has the advantages of less computing resource occupancy and faster operation speed, and effectively solves the problem of slow automatic scoring caused by the large amount of complex coordinate calculations required by the object detection method.

[0045] I. Technical Abstract

[0046] The present invention discloses a multitask serial lightweight system and method for the clock-drawing test of the MoCA scale. The system includes five modules: a clock image preprocessing module, a clock segmentation module, a contour recognition module, a digit recognition module, and a pointer recognition module. The image preprocessing module preprocesses the clock image to improve the image quality. The clock segmentation module divides the preprocessed clock image into three parts: the contour, digits, and pointer of the clock. Then, the contour recognition module, the digit recognition module, and the pointer recognition module classify the contour, digits, and pointer of the clock in sequence. Finally, the results of each recognition module are fused to obtain the total score of the clock.

[0047] II. Basic Structure and Function of the System

[0048] The present invention is a multitask serial lightweight system for the clock-drawing test of the MoCA scale. The system mainly includes two parts: multitask division of clock recognition and serial lightweight multitask recognition. The former includes two parts: a clock image preprocessing module and a clock segmentation module; while the serial lightweight multitask recognition includes three modules: a contour recognition module, a digit recognition module, and a pointer recognition module. The system structure is as Figure 1 shown:

[0049] 2.1 Multitask Division of Clock Recognition

[0050] 2.1.1 Clock Image Preprocessing

[0051] The main purpose of the clock image preprocessing module is to adjust the image size to match the input size required by the clock segmentation module, while eliminating the adverse effects of the image background on the image quality and ensuring that the background pixel value is 0 and the foreground pixel value of the clock is 1. The steps are as Figure 2 shown.

[0052] First, leave a white margin around the input original clock image at 15% of the clock size to obtain an image with a black clock in the middle, a white background, and a white margin around it, and adjust the size to the same length and width. Then convert the original RGB image into a grayscale image, and convert the grayscale image into a binary image with a black background and a white clock foreground. Finally, normalize the pixel values of the image to obtain an image with a background pixel value of 0 and a clock pixel value of 1.

[0053] 2.1.2 Clock Image Segmentation

[0054] The main function of the clock segmentation module is to segment a clock image into three images of the clock's outline, numbers, and hands, so as to recognize the outline, numbers, and hands separately one by one later. And the recognition result of the previous-level model is used as the input of the next-level model, as Figure 1 shown.

[0055] The clock segmentation module includes a lightweight neural network for image segmentation. Segmentation networks such as UNet, Seaformer, or EGE-UNet can be selected, and the model of the segmentation network is lightweighted by reducing the number of convolutional kernels and using smaller-sized convolutional kernels, and at the same time adding normalization layers to improve stability. The clock segmentation module segments an image with a background pixel value of 0 and a foreground pixel value of 1 of the clock into three images with a background pixel value of 0 and the pixel values of the outline, numbers, and hands being 1 respectively.

[0056] 2.2 Serial Lightweight Multi-Task Recognition (as Figure 3 shown)

[0057] 2.2.1 Outline Recognition Module

[0058] In clinical practice, when the subject draws a clock, there will be five situations for the drawn outline: the outline is not drawn, the outline is a random graffiti pattern, the outline is a circle, the outline is an ellipse, and the outline is a square. The first two situations belong to the outline not scoring. For the clock with the outline not scoring, after passing through the clock image preprocessing module and the clock segmentation module, the segmented outline image will be judged as not scoring by the outline recognition module, and thus the total score of the clock will directly be 0 points. The latter three situations belong to the outline scoring. Therefore, the outline recognition module is used to classify the outline into two categories: scoring and not scoring.

[0059] The contour recognition module performs binary classification on a contour image with background pixels being 0 and contour pixels being 1, classifying it into two categories: no score or score. The contour recognition module includes a neural network for image binary classification. This neural network can select AlexNet, reduce the number of convolutional kernels based on AlexNet, use convolutional kernels with smaller sizes, and replace the fully connected layer with a lightweight attention module to achieve lightweight processing. The contour recognition module can also be replaced by other lightweight neural networks for image classification, such as MobileNet, ShuffleNet, etc. The input of the contour recognition module is the segmented contour image, and the output is the contour class label and the corresponding evaluation score value. The corresponding evaluation score for the class of no score is 0, and the corresponding evaluation score for the class of score is 1. Whether to perform digit recognition is judged according to the contour class label. Only when the score is obtained will digit recognition be performed, and when the score is not obtained, the calculated total score of 0 will be directly output.

[0060] 2.2.2 Digit Recognition Module

[0061] In clinical practice, when the subject draws a clock, the drawn digits will show four situations: digits are missing and not fully drawn, digits are drawn multiple times with duplicates, the digit positions are drawn wrongly, and the digits are complete and in the correct positions. The first three situations belong to the contour with no score, and the latter situation belongs to the contour with a score. Among the first three situations where the contour has no score, if the digits 11 and 2 are drawn, then it can also be judged whether the pointer points to 11:10. Therefore, the digits can be divided into three categories: ① Digit Category 1: The digits are drawn correctly and score; ② Digit Category 2: The digits are drawn wrongly, but the digits 11 and 2 are drawn correctly, and the pointer can be further judged; ③ Digit Category 3: The digits are drawn wrongly and the pointer cannot be further judged. So the digit recognition module is used to perform three-class classification on the digits.

[0062] The digit recognition module performs three-class classification on a digit image with background pixels being 0 and digit pixels being 1, classifying it into Digit Category 1, Digit Category 2, and Digit Category 3. The digit recognition module includes a lightweight neural network for image three-class classification, and its architecture is the same as that of the neural network in the contour recognition module. The digit recognition module can also be replaced by other lightweight neural networks for image classification, such as MobileNet, ShuffleNet, etc. The input of the digit recognition module is the segmented digit image, and the output is the digit class label and the corresponding evaluation score value. When the digit class label value is 1, the evaluation score of the digit is 1 point. When the digit class label value is 2 or 3, the evaluation score of the digit is 0 points. Whether to perform pointer recognition is judged according to the digit class label. When the digit class label value is 1 or 2, pointer recognition will continue, otherwise the calculated total score of 0 will be directly output.

[0063] 2.2.3 Pointer Recognition Module

[0064] In clinical practice, when the subject draws a clock, there are four situations regarding the drawn hands: no hands are drawn, the number of hands drawn is incorrect, two hands are drawn but the pointing is incorrect, and the pointing and the number of hands are correct. Among them, the first three situations mean the hands do not score, and the last situation means the hands score. Therefore, a binary classification is performed on the hands.

[0065] The hand recognition module performs binary classification on the hand image with a background pixel value of 0 and a hand pixel value of 1, classifying it into two categories: not scoring or scoring. The hand recognition module uses an image binary classification neural network with the same architecture as the contour recognition module but different parameters. Similarly, other lightweight neural networks for image classification can also be used instead, such as MobileNet, ShuffleNet, etc. The input of the hand recognition module is the segmented hand image, and the output is the evaluation score value corresponding to the hand category label. The evaluation score corresponding to the category of not scoring is 0, and the evaluation score corresponding to the category of scoring is 1.

[0066] Therefore, the output of the final serial lightweight multi-task recognition module is the sum of the evaluation score values output by the contour, number, and hand recognition modules, and this is used as the total score for the clock-drawing evaluation.

[0067] III. Implementation Process

[0068] To achieve the automatic scoring of the clock-drawing task in the MoCA cognitive impairment scale, the present invention has developed a multi-task serial lightweight system for the clock-drawing test of the MoCA scale, and its scoring process is as Figure 4 shown.

[0069] The automatic scoring process of the clock-drawing test in the present invention is the following 9 steps:

[0070] 1. After inputting the original clock image, leave a white margin around the clock according to 15% of the size of the clock in the middle of the image;

[0071] 2. Adjust the size of the clock image;

[0072] 3. Grayscale the clock image and reverse binary it into a binary image with a black background and a white clock foreground;

[0073] 4. Normalize the clock image to obtain a clock image with a background pixel value of 0 and a clock foreground pixel value of 1;

[0074] 5. Input the clock image into the clock segmentation module to segment it into three contour, number, and hand images with a background pixel value of 0 and a foreground pixel value of 1;

[0075] 6. The contour image is input into the contour recognition module to obtain the judgment result of the contour. If the judgment result of the contour is a score, the recognition of the number continues. If the judgment result of the contour is not a score, it is defaulted that the number and the pointer are not scored, and it jumps to step 9.

[0076] 7. The digital image is input into the digital recognition module to obtain the category result of the number. If the category of the number is 1 or 2, the recognition of the pointer continues. If the category of the number is 3, it jumps to step 9.

[0077] 8. The pointer image is input into the pointer recognition module to obtain the judgment result of the pointer.

[0078] 9. The total score of the clock-drawing test is obtained by synthesizing the score results of the contour, the number, and the pointer.

[0079] The above-described embodiments only represent one implementation manner of the present invention, and its description is relatively specific and detailed, but it cannot be understood as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the appended claims.

Claims

1. A multi-task serial scoring system for the clock drawing test of the MoCA scale, characterized in that, It includes two parts: clock recognition multi-task division and serial lightweight multi-task recognition. The clock recognition multi-task division includes a clock image preprocessing module and a clock segmentation module. The serial lightweight multi-task recognition includes a contour recognition module, a digit recognition module, and a pointer recognition module. The image preprocessing module preprocesses the clock image to improve the image quality. The clock segmentation module segments the preprocessed clock image into three parts: the contour, digits, and pointers of the clock. Then, the contour recognition module, digit recognition module, and pointer recognition module classify the contour, digits, and pointers of the clock in sequence. Finally, the results of each recognition module are fused to obtain the total score of the clock. The purpose of the clock image preprocessing module is to adjust the image size to match the input size required by the clock segmentation module, eliminate the adverse effects of the image background on the image quality, and ensure that the background pixel value is 0 while the foreground pixel value of the clock is 1. First, leave a white margin around the input original clock image at 15% of the clock size to obtain an image with a black clock in the middle, a white background, and a white margin around it, and adjust the length and width of the image to 256. Then, convert the original RGB image into a grayscale image, and convert the grayscale image into a binary image with a black background and a white foreground for the clock. Finally, normalize the pixel values of the image to obtain an image with a background pixel value of 0 and a clock pixel value of 1. The function of the clock segmentation module is to segment a clock image into three images: the contour, digits, and pointers of the clock, so that the contour, digits, and pointers can be recognized separately in sequence later. And the recognition result of the previous-level model in the recognition process is used as the input of the next-level model. The clock segmentation module includes a lightweight neural network for image segmentation. Select the UNet, Seaformer, or EGE-UNet segmentation network. And perform model lightweight processing on the segmentation network, reducing the number of convolutional kernels and using smaller-sized convolutional kernels instead. The clock segmentation module segments the image with a background pixel value of 0 and a foreground pixel value of 1 for the clock into three images with a background pixel value of 0 for all, and pixel values of 1 for the contour, digits, and pointers respectively. The contour recognition module performs binary classification on the contour image with a background pixel of 0 and a contour pixel of 1, dividing it into two categories: no score or score. The contour recognition module includes a neural network for image binary classification. This neural network selects AlexNet, reduces the number of convolutional kernels on the basis of AlexNet, uses smaller-sized convolutional kernels instead, and replaces the fully connected layer with a lightweight attention module to achieve lightweight processing. The contour recognition module can also be replaced by other lightweight neural networks for image classification, such as MobileNet and ShuffleNet. The input data of the contour recognition module is the segmented contour image, and the output is the contour category label and the evaluation score value corresponding to the label. The corresponding evaluation score for the category of no score is 0, and the corresponding evaluation score for the category of score is 1. Determine whether to perform digit recognition according to the contour category label. Only when it scores can digit recognition be performed, and when it does not score, directly output the calculated total score of 0. The digital recognition module performs three-class classification on digital images with a background pixel value of 0 and a digital pixel value of 1, dividing them into digital category 1, digital category 2, and digital category 3. The digital recognition module includes a lightweight neural network for image three-class classification, with the same architecture as the neural network of the contour recognition module. The digital recognition module can also be replaced by other lightweight neural networks for image classification, such as MobileNet and ShuffleNet. The input of the digital recognition module is the segmented digital image, and the output is the digital category label and the evaluation score value corresponding to the label. When the digital category label value is 1, the evaluation score of the digit is 1 point. When the digital category label value is 2 or 3, the evaluation score of the digit is 0 points. Whether to perform pointer recognition is judged according to the digital category label. When the digital category label value is 1 or 2, pointer recognition continues, otherwise the calculated total score is directly output as 0. The pointer recognition module performs binary classification on pointer images with a background pixel value of 0 and a pointer pixel value of 1, dividing them into two categories: no score or score. The pointer recognition module uses an image binary classification neural network with the same architecture as the contour recognition module but different parameters. Similarly, it can also be replaced by other lightweight neural networks for image classification, such as MobileNet and ShuffleNet. The input of the pointer recognition module is the segmented pointer image, and the output is the evaluation score value corresponding to the pointer category label. The corresponding evaluation score for the category of no score is 0, and the corresponding evaluation score for the category of score is 1. Therefore, the output of the final serial lightweight multi-task recognition module is the sum of the evaluation score values output by the contour, digital, and pointer recognition modules, and this is used as the total score for the clock drawing evaluation.

2. The multi-task serial scoring system for the MoCA scale clock-drawing test according to claim 1, wherein, When the test subject draws a clock, there are five situations for the drawn contour: the contour is not drawn, the contour is a random graffiti pattern, the contour is a circle, the contour is an ellipse, and the contour is a square. The first two situations belong to the contour not scoring. After the clock image preprocessing module and the clock segmentation module, the segmented contour image will be judged as not scoring by the contour recognition module, and thus the total score of the clock is directly obtained as 0 points. The latter three situations belong to the contour scoring. Therefore, the contour recognition module is used to divide the contour into two categories: scoring and not scoring.

3. The multi-task serial scoring system for the clock drawing test of the MoCA scale according to claim 1, wherein When the test subject draws a clock, there are four situations for the drawn digits: the digits are missing and not fully drawn, there are extra and repeated digits, the digit positions are drawn incorrectly, and the digits are complete and in the correct positions. The first three situations belong to the contour not scoring, and the latter situation belongs to the contour scoring. Among the first three situations where the contour does not score, if the digits 11 and 2 are drawn in the correct positions, then it can also be judged whether the pointer points to 11:

10. Therefore, the digits can be divided into three categories: Digital category 1: The digits are drawn correctly and score. Digital category 2: The digits are drawn incorrectly, but the digits 11 and 2 are drawn correctly, and the pointer can continue to be judged. Digital category 3: The digits are drawn incorrectly and the pointer cannot be judged continuously. Therefore, the digital recognition module is used to perform three-class classification on the digits.

4. The multi-task serial scoring system for the clock drawing test of the MoCA scale according to claim 1, wherein When the test subject draws a clock, there are four situations for the drawn hands: the hands are not drawn, the number of drawn hands is incorrect, two hands are drawn but the directions are incorrect, and the directions and the number of hands are correct. Among them, the first three situations mean that the hands do not score, and the last situation means that the hands score. Therefore, a binary classification is performed on the hands.

5. A multi-task serial scoring method for the clock drawing test of the MoCA scale, characterized in that, The multi-task serial scoring system for the MoCA scale clock drawing test as described in any one of claims 1-4 is used to achieve automatic scoring. The automatic scoring process consists of the following 9 steps: Step 1: After inputting the original clock image, leave a white margin around the clock according to 15% of the size of the clock in the middle of the image. Step 2: Adjust the size of the clock image. Step 3: Grayscale the clock image and reverse binary it into a binary image with a black background and a white clock foreground. Step 4: Normalize the clock image to obtain a clock image with a background pixel value of 0 and a clock foreground pixel value of 1. Step 5: Input the clock image into the clock segmentation module and segment it into three images of the outline, numbers, and hands with a background pixel value of 0 and a foreground pixel value of 1. Step 6: Input the outline image into the outline recognition module to obtain the judgment result of the outline. If the judgment result of the outline is a score, continue with the recognition of the numbers. If the judgment result of the outline is no score, default that the numbers and hands are no score and jump to Step 9. Step 7: Input the number image into the number recognition module to obtain the category result of the numbers. If the category of the numbers is 1 or 2, continue with the recognition of the hands. If the category of the numbers is 3, jump to Step 9. Step 8: Input the hand image into the hand recognition module to obtain the judgment result of the hands. Step 9: Combine the scoring results of the outline, numbers, and hands to obtain the total score of the clock drawing test.

Images