Drawing clock test scoring method and device, computer device and storage medium

Abstract

The application relates to a clock-drawing test scoring method and device, computer equipment and a storage medium. The method comprises the following steps: inputting a clock-drawing image dataset into an FCOS network to identify handwritten numerals in the clock-drawing image dataset; performing circle detection on the clock-drawing image dataset by using a maximum connected domain algorithm to obtain a clock region; based on the identification result of the handwritten numerals and the detection result of the clock region, performing pointer detection on the clock-drawing image dataset by using an image processing algorithm to obtain a pointer image of the clock-drawing image dataset, and performing hour hand and minute hand classification on the pointer image by using a breadth-first algorithm; scoring the handwritten numeral identification result, the clock region detection result, the pointer image detection result and the hour hand and minute hand classification result according to a set scoring rule, and adding all the scoring results to generate a clock-drawing test scoring result. The application can avoid evaluation fluctuations and differences caused by subjectivity and bias of artificial evaluation.

Description

Technical Field

[0001] This application belongs to the field of deep learning technology, and specifically relates to a clock drawing test scoring method, device, computer equipment, and storage medium. Background Technology

[0002] The clock drawing test is a complex and comprehensive behavioral activity that requires not only spatial construction skills but also many cognitive functions, such as memory, attention, abstract thinking ability, and time and spatial orientation. The clock drawing test comprehensively reflects cognitive function, is simple and easy to perform, requires no special materials, and is highly accurate with minimal cultural relevance. In recent decades, it has been widely used in screening for diseases such as cognitive decline, making it suitable not only for clinical screening of early-stage dementia patients but also for helping to determine the severity of dementia.

[0003] There are many methods for clock drawing tests, and various scoring standards still exist. Commonly used ones include the 3-point, 4-point, 5-point, 7-point, and 10-point scoring methods. Currently, there is no internationally recognized unified scoring standard. Existing clock drawing test scoring methods mainly include the following four aspects: (1) drawing a closed circle (dial); (2) drawing evenly distributed scales within the circle; (3) marking the numbers corresponding to the scales; and (4) drawing the hour and minute hands at the time specified by the tester. However, these scoring methods neglect the cognitive processing strategies involved in the clock drawing process, especially the order of number writing and the "anchoring strategy" that reflects the clock drawing process and anchors the four key points "12-3-6-9". Therefore, in 2008, Guo Qihao et al. developed a 30-point scoring system. This system begins with the test-taker giving a prompt, such as, "Please draw a clock face on a blank sheet of paper and label all the numbers. The hands should point to 1:50." The test-taker then manually draws the clock according to the prompt, which takes approximately a few minutes. Finally, the test-taker manually scores the drawn clock image according to a scoring standard. However, due to the subjectivity and bias of manual scoring, the final scores vary from person to person, especially regarding the scoring of the clock-drawing process, which shows significant fluctuations and differences. Furthermore, manual scoring is time-consuming and labor-intensive, increasing the workload of the clock-drawing test assessment. Summary of the Invention

[0004] This application provides a clock drawing test scoring method, apparatus, computer device, and storage medium, which aims to at least partially solve one of the aforementioned technical problems in the prior art.

[0005] To address the above problems, this application provides the following technical solution:

[0006] A clock drawing test scoring method includes:

[0007] Handwritten digits from 1 to 12 were generated using the MNIST database, and the handwritten digits and the drawn clock images were randomly placed in a black image with a pixel value of 0 to generate a clock image dataset.

[0008] The clock image dataset is input into the FCOS network, and the FCOS network identifies the handwritten digits in the clock image dataset.

[0009] The maximum connected component algorithm is used to perform circle detection on the clock image dataset to obtain the clock region in the clock image dataset.

[0010] Based on the recognition results of the handwritten digits and the detection results of the clock area, an image processing algorithm is used to detect the hands in the clock image dataset to obtain the hand images of the clock image dataset, and a breadth-first search algorithm is used to classify the hand images into hour and minute hands.

[0011] According to the set scoring rules, the results of handwritten digit recognition, clock area detection, pointer image detection, and hour and minute hand classification are scored separately. All scores are added together to generate the clock drawing test score.

[0012] The technical solution adopted in this application embodiment further includes: generating handwritten digits from 1 to 12 using the MNIST database, and randomly placing the handwritten digits and the drawn clock image into a black image with a pixel value of 0 to generate a clock image dataset specifically as follows:

[0013] The handwritten digits 0 to 9 are obtained from the MNIST database, and the handwritten digit "1" is concatenated with "0", "1", and "2" respectively to generate the handwritten digits "10", "11", and "12".

[0014] Draw a clock image of a specified size; the clock image is circular.

[0015] Image enhancement operations are performed on the handwritten digits and clock images respectively to obtain the handwritten digits and clock images after image enhancement.

[0016] The handwritten digits and clock image after the image enhancement operation are randomly placed in a black image with a pixel value of 0 to generate a single-image clock picture, and the position and size of each handwritten digit in the single-image clock picture are recorded.

[0017] A single image of a clock is generated cyclically according to a set number of iterations to construct a clock image dataset.

[0018] The technical solution adopted in this application embodiment further includes: the step of inputting the clock image dataset into the FCOS network and identifying the handwritten digits in the clock image dataset through the FCOS network specifically involves:

[0019] The input image of the FCOS network is any one of the clock images in the clock image dataset. After calculation, it outputs the recognition result of each handwritten digit in the input image and the location area of ​​the handwritten digit in the clock image.

[0020] The technical solution adopted in this application embodiment further includes: the specific method of using the maximum connected component algorithm to perform circle detection on the clock image dataset to obtain the clock region in the clock image dataset is as follows:

[0021] Assuming that for all clock images drawn by testers, the clock area is larger than any number area, based on this assumption, a connected component detection algorithm is used to arrange the area containing the largest connected component as the clock area, and the clock area is extracted using a filling algorithm and an edge detection algorithm to obtain the clock area that is drawn as a circle.

[0022] The technical solution adopted in this application embodiment also includes: the specific method of using an image processing algorithm to detect the pointer of the clock image dataset based on the recognition result of the handwritten digits and the detection result of the clock area is as follows:

[0023] Fill the pixel values ​​of the positioning area of ​​the handwritten digits and the clock area with 0, filter out the handwritten digits and clock area in the clock image, and obtain the pointer image;

[0024] The Canny operator is used to extract the pointer edges in the pointer image, eliminating noise information other than the pointer itself.

[0025] The technical solution adopted in this application embodiment further includes: the specific method of classifying the pointer image into hour and minute hands using a breadth-first search algorithm is as follows:

[0026] The Hough line detection method is used to find all straight lines in the pointer image, including the hour and minute hands and arrows;

[0027] The BFS algorithm is used to detect the length of each line along its direction, and the length and two vertices of each line are marked. The two longest lines with one vertex located within 30 pixels of the clock center are used as the hour and minute hands, respectively. The remaining lines are paired according to their vertex positions and used as the arrows of the hour and minute hands, respectively.

[0028] The technical solution adopted in this application embodiment further includes: the scoring of handwritten digit recognition results, clock area detection results, pointer image detection results, and hour and minute hand classification results according to the set scoring rules specifically involves:

[0029] All numbers within the clock area score 3 points; numbers outside the clock area deduct 0.5 points per number, up to a total of 0 points.

[0030] Two points are awarded for symmetrical distribution of the four symbolic numbers "12, 3, 6, 9". Specifically, 1 point is awarded for 3 and 9 on the horizontal line and 1 point is awarded for 12 and 6 on the vertical line. Otherwise, 0.5 points are deducted per number, up to a total of 0 points.

[0031] 3 points are awarded for the accurate placement of numbers other than "12, 3, 6, 9". 0.5 points are deducted for each incorrect or biased number, up to a maximum of 0 points.

[0032] First, anchor the 4 key points "12, 3, 6, 9" to get 4 points, otherwise deduct 1 point per key point, until you get 0 points;

[0033] 4 points are awarded for writing 12 numbers accurately; otherwise, 0.5 points are deducted per number, up to a total of 0 points.

[0034] Arranged clockwise, 1 point is awarded.

[0035] 1 point is awarded for correctly ordering the numbers 1 to 12;

[0036] 1 point is awarded for correctly positioning the center point;

[0037] Drawing the hour and minute hands earns 2 points;

[0038] 2 points for the clock hand pointing correctly;

[0039] 2 points are awarded if the minute hand points correctly;

[0040] The minute hand is 2 minutes longer than the hour hand;

[0041] 2 points are awarded if both the hour and minute hands have arrows.

[0042] A complete clock face earns 1 point.

[0043] Another technical solution adopted in this application embodiment is: a clock drawing test scoring device, comprising:

[0044] Data generation module: used to generate handwritten digits from 1 to 12 using the MNIST database, and randomly place the handwritten digits and the drawn clock image into a black image with a pixel value of 0 to generate a clock image dataset;

[0045] The digit recognition module is used to input the clock image dataset into the FCOS network and recognize the handwritten digits in the clock image dataset through the FCOS network.

[0046] Clock detection module: used to perform circle detection on the clock image dataset using the maximum connected component algorithm to obtain the clock region in the clock image dataset;

[0047] Pointer detection module: Based on the recognition results of the handwritten digits and the detection results of the clock area, it uses image processing algorithms to perform pointer detection on the clock image dataset, obtains the pointer images of the clock image dataset, and uses a breadth-first search algorithm to classify the pointer images into hour and minute hands;

[0048] The clock drawing scoring module is used to score the handwritten digit recognition results, clock area detection results, pointer image detection results, and hour and minute hand classification results according to the set scoring rules, and to add up all the scores to generate the clock drawing test score.

[0049] Another technical solution adopted in this application embodiment is: a computer device, the computer device including a processor and a memory coupled to the processor, wherein,

[0050] The memory stores program instructions for implementing the clock drawing test scoring method;

[0051] The processor is used to execute the program instructions stored in the memory to control the clock drawing test scoring method.

[0052] Another technical solution adopted in this application embodiment is: a storage medium storing program instructions that can be executed by a processor, the program instructions being used to execute the clock drawing test scoring method.

[0053] Compared to existing technologies, the beneficial effects of the embodiments of this application are as follows: The clock drawing test scoring method, apparatus, computer equipment, and storage medium of the embodiments of this application adopt a digital recognition and localization method based on a deep learning model. This method can quickly complete the recognition and localization of all numbers in a single clock drawing image, clock area detection, pointer image detection, and classification on the CPU, greatly accelerating the clock drawing test scoring speed and avoiding evaluation fluctuations and differences caused by the subjectivity and bias of human evaluation. At the same time, the embodiments of this application can better simulate the sampling of real data and continuously optimize the deep learning model, making the model more robust. Attached Figure Description

[0054] Figure 1 This is a flowchart of the clock drawing test scoring method according to an embodiment of this application;

[0055] Figure 2 This is a schematic diagram illustrating the construction of the clock image dataset according to an embodiment of this application;

[0056] Figure 3 This is a schematic diagram of handwritten digit recognition and positioning based on FCOS network in an embodiment of this application;

[0057] Figure 4 This is a schematic diagram of the scoring rules for each scoring item in the embodiments of this application;

[0058] Figure 5 This is a schematic diagram of the clock drawing test scoring device according to an embodiment of this application;

[0059] Figure 6 This is a schematic diagram of the computer device structure according to an embodiment of this application;

[0060] Figure 7 This is a schematic diagram of the structure of the storage medium according to an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0062] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or computer device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or computer devices.

[0063] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0064] Please see Figure 1 This is a flowchart of a clock-drawing test scoring method according to an embodiment of this application. The clock-drawing test scoring method according to an embodiment of this application can be deployed on a server, and the server has an API interface for clients to call the clock-drawing test scoring method. Specifically, the clock-drawing test scoring method according to an embodiment of this application includes the following steps:

[0065] S100: Generate handwritten digits from 1 to 12 using the MNIST database, and randomly place the handwritten digits and pre-drawn clock images into a black image with a pixel value of 0 to generate a clock image dataset.

[0066] In this step, to generate a large number of handwritten digits in a short time, this embodiment of the application uses MNIST (Mixed National Institute of Standards and Technology database, which contains 70,000 28×28 grayscale images, each containing a handwritten digit from 0 to 9) as the basic database, and obtains handwritten digits from 1 to 12 from this database. Specifically, the method for generating the clock image dataset includes:

[0067] S101: Obtain handwritten digits 0-9 from the MNIST database, and then concatenate the handwritten digit "1" with "0", "1", and "2" respectively to generate handwritten digits "10", "11", and "12".

[0068] S102: Draw a clock image of a set size; wherein the clock image is circular;

[0069] S103: Perform image enhancement operations on the generated handwritten digits and clock images respectively to obtain the handwritten digits and clock images after image enhancement.

[0070] To more closely resemble a real-world clock-drawing test using a graphics tablet, this embodiment of the application performs image enhancement operations on the generated handwritten digits and clock images, including random angle rotation and random size scaling (the original handwritten digits and clock images were 28×28 pixels, while the enhanced images were between 16 and 40 pixels). Furthermore, to simulate situations where unfamiliarity with the tablet leads to gaps in the digits during clock drawing, this embodiment also incorporates random image erosion during the image enhancement process to simulate digit breakage.

[0071] S104: Randomly place the handwritten digits and clock image after image enhancement into a black image with a pixel value of 0 to generate a single-image clock picture, and record the position and size of each handwritten digit in the single-image clock picture; wherein, the image size of the black image is 1024×1024, which can be set according to the actual application scenario.

[0072] S105: Execute S104 repeatedly according to the set number of iterations to obtain a set number of single clock images and construct a clock image dataset.

[0073] Among them, such as Figure 2 The diagram shown illustrates the construction of the clock image dataset according to an embodiment of this application. The first column contains the generated handwritten digits from 1 to 12, where "10", "11", and "12" are obtained by concatenating two handwritten digits from MNIST; the second column contains the drawn clock image. Then, image enhancement operations are performed on the handwritten digits and the clock image, and they are randomly placed in a 1024x1024 black image with all pixel values ​​of 0 to generate a single clock image.

[0074] S110: Input the clock image dataset into the FCOS (Fully Convolutional One-Stage Object Detection) network, and the FCOS network will automatically identify the handwritten numbers in each clock image;

[0075] In this step, such as Figure 3The diagram illustrates the handwritten digit recognition and localization based on the FCOS network in this embodiment of the application. The FCOS model in this embodiment is trained using Python 3.8 and CUDA 11.4, with an Intel(R) Xeon(R) Silver 4116 CPU at 2.10 GHz. FCOS employs stochastic gradient descent for parameter optimization, undergoing 84,000 training iterations. The training weights and the FCOS model are packaged in ONNX format, providing CPU support scripts for deployment in Linux environments. The input image for the FCOS network is any clock image from the clock image dataset. After computation, the FCOS network outputs two results: the recognition result of each handwritten digit in the input image and the regression value of its localization region (presented as a rectangle) in the clock image. This achieves accurate recognition and localization of handwritten digits even with clock face image interference.

[0076] S120: The maximum connected component algorithm is used to perform circle detection on the clock image dataset to obtain the clock region in the clock image;

[0077] In this step, considering that the clock drawing test scoring criteria do not include a score for the curvature of a circle, it is not necessary to concern ourselves with whether the detected circular region is a perfect circle. This embodiment uses the maximum connected component algorithm to detect circles in the clock drawing image dataset. Specifically, it assumes that for all clock images drawn by test takers, the clock region is larger than any of the digit regions. Based on this assumption, a connected component detection algorithm is used to identify the region containing the maximum connected component as the clock region. Then, a filling algorithm and an edge detection algorithm are used to extract the clock region, resulting in the circular clock region.

[0078] S130: Based on the handwritten digit recognition results and clock area detection results, an image processing algorithm is used to detect the pointers in the clock image dataset to obtain pointer images containing the hour and minute hands;

[0079] In this step, the image processing algorithm is used to detect the pointer in the clock image dataset. Specifically, the pixel values ​​of the handwritten digit location area and the clock area are filled with 0, thereby filtering out the handwritten digits and clock area in the clock image to obtain the pointer image. Then, the Canny operator is used to extract the pointer edge in the pointer image to eliminate other noise information besides the pointer.

[0080] S140: The BFS (Breadth-First-Search) algorithm is used to detect straight lines in the pointer image. Based on the line detection results, the pointers are classified to obtain the hour hand, minute hand, and corresponding arrows.

[0081] In this step, the pointer classification method is as follows: First, the Hough line detection method is used to find all the straight lines in the pointer image, including the hour and minute hands and the arrows; then, the BFS algorithm is used to detect the length of each straight line along its direction, and the length and two vertices of each straight line are marked. Then, the two longest straight lines with one vertex located within 30 pixels of the clock center are used as the hour and minute hands, respectively, and the remaining straight lines are paired according to their vertex positions to serve as the arrows of the hour and minute hands, respectively.

[0082] S150: According to the set scoring rules, the handwritten digit recognition result, clock area detection result, pointer image detection result, and hour and minute hand classification result are scored respectively. The scores of all scoring items are added together to generate the final clock drawing test score.

[0083] In this step, such as Figure 4 The diagram shown illustrates the scoring rules for each scoring item in this application embodiment. Specifically, the scoring rules for handwritten digit recognition results and clock area detection results include:

[0084] (1) All numbers within the clock area receive 3 points, and numbers outside the clock area are deducted 0.5 points per number, up to 0 points. Specifically, the scoring algorithm for scoring rule (1) is as follows: After obtaining the number location area and the clock area, firstly, the clock area is filled, that is, all pixels within the clock area are filled with 255. Then, all numbers are iterated in a loop. In each loop, the pixel values ​​of the number location area and the clock area are added together. Then, the result of the addition is subtracted from the filled image of the clock area to determine whether there are other pixels with a pixel value of 255 in the clock area. If they exist, it proves that the number exceeds the clock area; otherwise, it is determined that the number is completely within the clock area.

[0085] (2) A score of 2 is awarded for symmetrical distribution of the four iconic numbers “12, 3, 6, 9”, i.e., 1 point for 3 and 9 on the horizontal line and 1 point for 12 and 6 on the vertical line; otherwise, 0.5 points are deducted per number, up to 0 points. Specifically, the scoring algorithm for scoring rule (2) is as follows: After obtaining the number positioning and recognition results, the four iconic numbers “12, 3, 6, 9” are extracted respectively. For the symmetry of “12” and “6”, this embodiment sets that the difference in the horizontal coordinate between the two numbers is not allowed to exceed 40% of the number width. If it exceeds this, “12” and “6” are judged to be asymmetrical, otherwise they are symmetrical. Similarly, for the symmetry of “3” and “9”, this embodiment sets that the difference in the vertical coordinate between the two numbers is not allowed to exceed 20% of the number width (because the vertical height of the numbers is greater than the horizontal width). If it exceeds this, “3” and “9” are judged to be asymmetrical, otherwise they are symmetrical.

[0086] (3) 3 points are awarded for the accurate position of numbers other than “12, 3, 6, 9”. 0.5 points are deducted for each number if its position is incorrect or biased, up to 0 points. Specifically, the scoring algorithm for scoring rule (3) is as follows: After obtaining the digital positioning area and the clock area, the center point of the digital positioning area and the clock area are calculated first. The center point of the clock area is used as one vertex and the center point of the digital positioning area is used as the other vertex. The direction of the straight line is calculated respectively. It is determined whether the absolute deviation between the straight line direction and the set direction is within the set degree (the degree is set to 30° in this embodiment, but can be set according to the actual application scenario). If so, the position of the number is determined to be accurate; otherwise, the position of the number is determined to be incorrect.

[0087] (4) First anchor the 4 key points “12, 3, 6, 9” to get 4 points, otherwise deduct 1 point per key point, until 0 points; the scoring algorithm for scoring rule (4) is as follows: retain the image of each stroke of the drawing, and use the digital handwriting recognition and positioning algorithm to identify the number drawn in each stroke of the drawing until the first number appears in the 4 key points “12, 3, 6, 9”. If the first number appears is another number, it is determined that the 4 key points “12, 3, 6, 9” were not anchored first. Among them, for the number “12”, there may be a situation where “1” is detected first and then “12” is detected. In order to avoid judgment errors, this application embodiment makes lenient treatment for the number “1” when detecting numbers, that is, it allows the number “1” to be transformed into the number “12” when it is determined to be the number “1”. If the transformation does not occur after the step of “1”, the number will be considered as “1”.

[0088] (5) 4 points are awarded for writing 12 numbers accurately, otherwise 0.5 points are deducted per number until 0 points are awarded; the scoring algorithm for scoring rule (5) is as follows: after obtaining the number positioning and recognition results, all numbers in the clock area are statistically analyzed to determine whether numbers 1-12 exist, and points are deducted according to the scoring criteria.

[0089] (6) Arranged clockwise, 1 point is awarded;

[0090] (7) 1 point is awarded for the correct order of numbers 1 to 12; Specifically, the scoring algorithm for scoring rules (6) and (7) is as follows: calculate the angle between the straight line and the vertical direction between each number and the center point, and determine whether the angle is monotonically increasing. If so, it is determined that the numbers are arranged clockwise and the order is correct; otherwise, it is determined that the numbers are not arranged clockwise and the order is incorrect.

[0091] The scoring rules for pointer image detection results and hour and minute hand classification results include:

[0092] (8) 1 point is awarded for a correct center point position; Specifically, the scoring algorithm for scoring rule (8) is as follows: After obtaining the detection results of the hour and minute hands, the center point is the intersection of the two pointers. It is determined whether the Euclidean distance between the intersection and the center point of the circular area is within 40 pixels. If so, the center point position is determined to be correct; otherwise, the center point position is determined to be incorrect.

[0093] (9) Drawing the hour and minute hands earns 2 points; The scoring algorithm for scoring rule (9) is as follows: After obtaining the detection results of the hour and minute hands, determine whether there are hour and minute hands in the detection results, and deduct points according to the scoring criteria.

[0094] (10) 2 points are awarded for the correct clock hand direction;

[0095] (11) 2 points are awarded for the correct minute hand pointing. Specifically, the scoring algorithm for scoring rules (10) and (11) is as follows: Considering that the hour hand may be longer than the minute hand during the clock drawing test, it is necessary to judge the direction of the two hands separately, calculate the angle difference between the direction of the two hands and the set direction of the hands, and set the hand category based on the angle difference. For example: Suppose one of the hands is the minute hand, and the angle difference between the direction of the minute hand and the set direction of the minute hand is within the set difference range (30°), then the hand is judged to be the minute hand and the hand is pointing correctly. If there is a mismatch, the hour hand and minute hand are assigned using the minimum phase difference tilt angle method, and the hand that does not meet the tilt angle range is judged to be pointing in the wrong direction.

[0096] (12) The minute hand is 2 minutes longer than the hour hand;

[0097] (13) Both the hour and minute hands have arrows and get 2 points; The scoring algorithm for scoring rules (12) and (13) is as follows: the length of the hour and minute hands is compared according to the division results of scoring rules (10) and (11), and it is determined whether the pointer contains an arrow.

[0098] (14) A complete clock face earns 1 point; The scoring algorithm for scoring rule (14) is as follows: After completing the remaining scoring items, check the numbers, hour hand and minute hand in the clock face area. If the numbers, hour hand and minute hand are all complete, the clock face is judged to be complete; otherwise, the clock face is judged to be incomplete.

[0099] Based on the above, the clock-painting test scoring method of this application adopts a digit recognition and localization method based on a deep learning model. This method can quickly complete the recognition and localization of all digits in a single clock-painting image, clock area detection, pointer image detection, and classification on a CPU, greatly accelerating the clock-painting test scoring speed and avoiding evaluation fluctuations and differences caused by the subjectivity and bias of human evaluation. At the same time, this application embodiment can better simulate the sampling of real data and continuously optimize the deep learning model, making the model more robust.

[0100] Please see Figure 5 This is a schematic diagram of the clock-drawing test scoring device according to an embodiment of this application. The clock-drawing test scoring device 40 according to an embodiment of this application includes:

[0101] Data generation module 41: used to generate handwritten digits from 1 to 12 using the MNIST database, and randomly place the handwritten digits and the drawn clock image into a black image with a pixel value of 0 to generate a clock image dataset.

[0102] Number recognition module 42: used to input the clock image dataset into the FCOS network and recognize the handwritten numbers in the clock image dataset through the FCOS network;

[0103] Clock detection module 43: used to perform circle detection on the clock image dataset using the maximum connected component algorithm to obtain the clock region in the clock image dataset;

[0104] Pointer detection module 44: Based on the recognition results of the handwritten digits and the detection results of the clock area, it uses an image processing algorithm to perform pointer detection on the clock image dataset, obtains the pointer images of the clock image dataset, and uses a breadth-first search algorithm to classify the pointer images into hour and minute hands;

[0105] Clock drawing scoring module 45: It is used to score the handwritten digit recognition results, clock area detection results, pointer image detection results, and hour and minute hand classification results according to the set scoring rules, and add up all the scoring results to generate the clock drawing test score result.

[0106] Please see Figure 6 This is a schematic diagram of a computer device structure according to an embodiment of this application. The computer device 50 includes:

[0107] Memory 51 storing executable program instructions;

[0108] Processor 52 connected to memory 51;

[0109] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: Generate handwritten digits from 1 to 12 using the MNIST database, and randomly place the handwritten digits and a drawn clock image into a black image with a pixel value of 0 to generate a clock image dataset; input the clock image dataset into the FCOS network, and use the FCOS network to identify the handwritten digits in the clock image dataset; perform circle detection on the clock image dataset using the maximum connected component algorithm to obtain the clock region in the clock image dataset; based on the recognition results of the handwritten digits and the detection results of the clock region, perform pointer detection on the clock image dataset using an image processing algorithm to obtain the pointer images of the clock image dataset, and use a breadth-first search algorithm to classify the hour and minute hands of the pointer images; score the handwritten digit recognition results, clock region detection results, pointer image detection results, and hour and minute hand classification results according to the set scoring rules, and add all the scores to generate a clock drawing test score result.

[0110] The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0111] Please see Figure 7 , Figure 7This is a schematic diagram of the storage medium in an embodiment of this application. The storage medium in this embodiment stores program instructions 61 capable of implementing the following steps: generating handwritten digits from 1 to 12 using the MNIST database, and randomly placing the handwritten digits and a drawn clock image into a black image with a pixel value of 0 to generate a clock image dataset; inputting the clock image dataset into an FCOS network, and identifying the handwritten digits in the clock image dataset through the FCOS network; performing circle detection on the clock image dataset using the maximum connected component algorithm to obtain the clock region in the clock image dataset; based on the recognition results of the handwritten digits and the detection results of the clock region, performing pointer detection on the clock image dataset using an image processing algorithm to obtain the pointer images of the clock image dataset, and classifying the hour and minute hands in the pointer images using a breadth-first search algorithm; scoring the handwritten digit recognition results, clock region detection results, pointer image detection results, and hour and minute hand classification results according to the set scoring rules, and adding all the scoring results to generate a clock drawing test score result. The program instructions 61 can be stored in the aforementioned storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network computer device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program instructions, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal computer devices such as computers, servers, mobile phones, and tablets. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0113] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A scoring method for a clock drawing test, characterized in that, include: Handwritten digits from 1 to 12 were generated using the MNIST database, and the handwritten digits and the drawn clock images were randomly placed in a black image with a pixel value of 0 to generate a clock image dataset. The clock image dataset is input into the FCOS network, and the FCOS network identifies the handwritten digits in the clock image dataset. The maximum connected component algorithm is used to perform circle detection on the clock image dataset to obtain the clock region in the clock image dataset. Based on the recognition results of the handwritten digits and the detection results of the clock area, an image processing algorithm is used to detect the hands in the clock image dataset to obtain the hand images of the clock image dataset, and a breadth-first search algorithm is used to classify the hand images into hour and minute hands. According to the established scoring rules, the results of handwritten digit recognition, clock area detection, pointer image detection, and hour and minute hand classification are scored separately. All scores are then summed to generate the clock drawing test score. The process of generating handwritten digits from 1 to 12 using the MNIST database and then randomly placing these handwritten digits and a drawn clock image into a black image with a pixel value of 0 to generate a clock image dataset is as follows: The handwritten digits 0 to 9 are obtained from the MNIST database, and the handwritten digit "1" is concatenated with "0", "1", and "2" respectively to generate the handwritten digits "10", "11", and "12". Draw a clock image of a specified size; the clock image is circular. Image enhancement operations are performed on the handwritten digits and clock images respectively to obtain the handwritten digits and clock images after image enhancement. The handwritten digits and clock image after the image enhancement operation are randomly placed in a black image with a pixel value of 0 to generate a single-image clock picture, and the position and size of each handwritten digit in the single-image clock picture are recorded. A single image of a clock is generated cyclically according to a set number of iterations to construct a clock image dataset.

2. The clock drawing test scoring method according to claim 1, characterized in that, The specific steps of inputting the clock image dataset into the FCOS network and identifying the handwritten digits in the clock image dataset through the FCOS network are as follows: The input image of the FCOS network is any one of the clock images in the clock image dataset. After calculation, it outputs the recognition result of each handwritten digit in the input image and the location area of ​​the handwritten digit in the clock image.

3. The clock drawing test scoring method according to claim 2, characterized in that, The method of using the maximum connected component algorithm to perform circle detection on the clock image dataset specifically yields the clock region in the clock image dataset as follows: Assuming that for all clock images drawn by testers, the clock area is larger than any number area, based on this assumption, a connected component detection algorithm is used to arrange the area containing the largest connected component as the clock area, and the clock area is extracted using a filling algorithm and an edge detection algorithm to obtain the clock area that is drawn as a circle.

4. The clock drawing test scoring method according to claim 3, characterized in that, The specific steps for detecting the pointers in the clock image dataset using image processing algorithms, based on the recognition results of the handwritten digits and the detection results of the clock area, are as follows: Fill the pixel values ​​of the positioning area of ​​the handwritten digits and the clock area with 0, filter out the handwritten digits and clock area in the clock image, and obtain the pointer image; The Canny operator is used to extract the pointer edges in the pointer image, eliminating noise information other than the pointer itself.

5. The clock drawing test scoring method according to claim 4, characterized in that, The specific steps of using a breadth-first search algorithm to classify the pointer image into hour and minute hands are as follows: The Hough line detection method is used to find all straight lines in the pointer image, including the hour and minute hands and arrows; The BFS algorithm is used to detect the length of each line along its direction, and the length and two vertices of each line are marked. The two longest lines with one vertex located within 30 pixels of the clock center are used as the hour and minute hands, respectively. The remaining lines are paired according to their vertex positions and used as the arrows of the hour and minute hands, respectively.

6. The clock drawing test scoring method according to any one of claims 1 to 5, characterized in that, According to the established scoring rules, the results of handwritten digit recognition, clock area detection, pointer image detection, and hour and minute hand classification are scored as follows: All numbers within the clock area score 3 points; numbers outside the clock area deduct 0.5 points per number, up to a total of 0 points. Two points are awarded for symmetrical distribution of the four symbolic numbers "12, 3, 6, 9". Specifically, 1 point is awarded for 3 and 9 on the horizontal line and 1 point is awarded for 12 and 6 on the vertical line. Otherwise, 0.5 points are deducted per number, up to a total of 0 points. 3 points are awarded for the correct placement of numbers other than "12, 3, 6, 9". 0.5 points are deducted for each incorrect or biased number, up to a maximum of 0 points. First, anchor the 4 key points "12, 3, 6, 9" to get 4 points, otherwise deduct 1 point per key point, until you get 0 points; 4 points are awarded for writing 12 numbers accurately; otherwise, 0.5 points are deducted per number, up to a total of 0 points. Arranged clockwise, 1 point is awarded. 1 point is awarded for correctly ordering the numbers 1-12. 1 point is awarded for correctly positioning the center point; Drawing the hour and minute hands earns 2 points; 2 points for the clock hand pointing correctly; 2 points are awarded if the minute hand points correctly; The minute hand is 2 minutes longer than the hour hand; 2 points are awarded if both the hour and minute hands have arrows. A complete clock face earns 1 point.

7. A clock drawing test scoring device utilizing the clock drawing test scoring method of claim 1, characterized in that, include: Data generation module: used to generate handwritten digits from 1 to 12 using the MNIST database, and randomly place the handwritten digits and the drawn clock image into a black image with a pixel value of 0 to generate a clock image dataset; The digit recognition module is used to input the clock image dataset into the FCOS network and recognize the handwritten digits in the clock image dataset through the FCOS network. Clock detection module: used to perform circle detection on the clock image dataset using the maximum connected component algorithm to obtain the clock region in the clock image dataset; Pointer detection module: Based on the recognition results of the handwritten digits and the detection results of the clock area, it uses image processing algorithms to perform pointer detection on the clock image dataset, obtains the pointer images of the clock image dataset, and uses a breadth-first search algorithm to classify the pointer images into hour and minute hands; The clock drawing scoring module is used to score the handwritten digit recognition results, clock area detection results, pointer image detection results, and hour and minute hand classification results according to the set scoring rules. All scores are added together to generate the clock drawing test score.

8. A computer device, characterized in that, The computer device includes a processor and a memory coupled to the processor, wherein, The memory stores program instructions for implementing the clock drawing test scoring method according to any one of claims 1-6; The processor is used to execute the program instructions stored in the memory to control the clock drawing test scoring method.

9. A storage medium, characterized in that, The device stores processor-executable program instructions for performing the clock drawing test scoring method according to any one of claims 1 to 6.

Images