Clock drawing test scoring method and device, computer equipment and storage medium

Abstract

The invention relates to a clock drawing test scoring method and device, computer equipment and a storage medium. The method comprises the following steps: inputting a clock drawing image data set into an FCOS network to identify handwritten digits in the clock drawing image data set; performing circle detection on the clock drawing image data set by adopting a maximum connected domain algorithm to obtain a clock area; based on the recognition result of the handwritten digits and the detection result of the clock area, adopting an image processing algorithm to carry out pointer detection on the clock drawing image data set to obtain pointer images of the clock drawing image data set, and adopting a breadth-first algorithm to carry out hour pointer and minute pointer classification on the pointer images; and according to a set scoring rule, respectively scoring the handwritten number identification result, the clock area detection result, the pointer image detection result and the hour hand and minute hand classification result, and adding all scoring results to generate a clock drawing test scoring result. According to the embodiment of the invention, evaluation fluctuation and difference caused by subjectivity and prejudice of manual evaluation can be avoided.

Description

Technical Field

[0001] The present application belongs to the field of deep learning technology, and in particular, relates to a clock drawing test scoring method, device, computer equipment and storage medium. Background Art

[0002] The clock drawing test is a complex and comprehensive behavioral activity. In addition to spatial construction skills, it also requires many cognitive functions, such as memory, attention, abstract thinking ability, time and space orientation, etc. The clock drawing test can not only reflect cognitive functions more comprehensively, but also is simple and easy to perform, does not require special materials, has high accuracy and low cultural relevance. In recent decades, it has been widely used in the screening of diseases such as cognitive impairment. It is not only suitable for clinical screening of early dementia patients, but also can help determine the severity of dementia.

[0003] There are many methods for clock drawing tests, and there are still various scoring standards. Commonly used ones include 3-point scoring method, 4-point scoring method, 5-point scoring method, 7-point scoring method, 10-point scoring method, etc. At present, there is no internationally recognized unified scoring standard. The existing scoring methods for clock drawing tests mainly include the following four aspects: (1) drawing a closed circle (dial); (2) drawing evenly distributed scales within the circle; (3) marking the numbers corresponding to the scales; (4) drawing the hour hand and minute hand of the time specified by the tester. However, these scoring methods ignore the cognitive processing strategies of the clock drawing process, especially the "anchoring strategy" that reflects the order of writing numbers in the clock drawing process and anchoring the four key points of "12-3-6-9". Therefore, in 2008, Guo Qihao and others developed a 30-point scoring system. The scoring system first requires the tester to say a prompt, such as "Please draw a clock face on a blank piece of paper and mark all the numbers with the pointer pointing to 1:50". Then, the subject draws a clock manually according to the prompt, which takes about a few minutes. Finally, the tester manually scores the clock image according to the scoring criteria. However, due to the subjectivity and bias of manual scoring, the final scoring results vary from person to person, especially the scoring of the clock drawing process has large fluctuations and differences, and manual scoring is time-consuming and labor-intensive, which increases the evaluation workload of the clock drawing test. Summary of the invention

[0004] The present application provides a clock drawing test scoring method, apparatus, computer equipment and storage medium, aiming to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.

[0005] In order to solve the above problems, this application provides the following technical solutions:

[0006] A clock drawing test scoring method, comprising:

[0007] Generate handwritten numbers from 1 to 12 using the MNIST database, and randomly place the handwritten numbers and the drawn clock images in a black image with a pixel value of 0 to generate a clock drawing image dataset;

[0008] Inputting the clock drawing image data set into the FCOS network, and recognizing the handwritten numerals in the clock drawing image data set through the FCOS network;

[0009] Using a maximum connected region algorithm to perform circle detection on the clock drawing image dataset to obtain a clock region in the clock drawing image dataset;

[0010] Based on the recognition result of the handwritten digits and the detection result of the clock area, an image processing algorithm is used to perform pointer detection on the drawn clock image data set to obtain a pointer image of the drawn clock image data set, and a breadth-first algorithm is used to classify the pointer image into hour hands and minute hands;

[0011] According to the set scoring rules, the handwritten digit recognition results, clock area detection results, pointer image detection results and hour hand and minute hand classification results are scored respectively, and all the scoring results are added together to generate the clock drawing test scoring result.

[0012] The technical solution adopted in the embodiment of the present application also includes: using the MNIST database to generate handwritten numbers from 1 to 12, and randomly placing the handwritten numbers and the drawn clock image in a black image with a pixel value of 0, and generating a clock image data set specifically as follows:

[0013] Obtain handwritten digits 0 to 9 from the MNIST database, and concatenate the handwritten digit "1" with "0", "1", and "2" respectively to generate handwritten digits "10", "11", and "12";

[0014] Draw a clock image of a set size; wherein the clock image is a circle;

[0015] Performing image enhancement operations on the handwritten numerals and the clock image respectively to obtain the handwritten numerals and the clock image after the image enhancement operations;

[0016] Randomly placing the handwritten numerals and the clock image after the image enhancement operation in a black image with a pixel value of 0 to generate a single clock image, and recording the position and size of each handwritten numeral in the single clock image;

[0017] A single clock-drawing image is generated cyclically according to the set number of iterations to construct a clock-drawing image dataset.

[0018] The technical solution adopted by the embodiment of the present application also includes: the step of inputting the clock drawing image data set into the FCOS network, and identifying the handwritten numerals in the clock drawing image data set through the FCOS network is specifically:

[0019] The input image of the FCOS network is any clock image in the clock image data set, and after calculation, the recognition result of each handwritten digit in the input image and the positioning area of ​​the handwritten digit in the clock image are output.

[0020] The technical solution adopted by the embodiment of the present application also includes: the maximum connected component algorithm is used to perform circle detection on the clock image data set, and the clock area in the clock image data set is obtained as follows:

[0021] It is assumed that the clock images drawn by all testers follow the rule that the clock area is larger than any digital area. Based on this assumption, a connected domain detection algorithm is used to arrange the area where the largest connected domain is located as the clock area, and the filling algorithm and the edge detection algorithm are used to extract the clock area to obtain the circular clock area.

[0022] The technical solution adopted by the embodiment of the present application also includes: based on the recognition result of the handwritten digits and the detection result of the clock area, the pointer detection of the clock image data set is performed using an image processing algorithm, specifically:

[0023] Filling the pixel values ​​of the positioning area of ​​the handwritten numbers and the clock area with 0, filtering out the handwritten numbers and the clock area in the clock drawing image, and obtaining a pointer image;

[0024] The Canny operator is used to extract the pointer edge in the pointer image, and other noise information except the pointer is eliminated.

[0025] The technical solution adopted by the embodiment of the present application also includes: the use of the breadth-first algorithm to classify the hour hand and minute hand of the pointer image is specifically as follows:

[0026] Using the Hough line detection method to find all the straight lines in the pointer image, the straight lines include the pointers of the hour hand and the minute hand and the arrows;

[0027] The BFS algorithm is used to detect the length of each straight line along the direction of each straight line, and the length of each straight line and the vertices at both ends are marked. The two longest straight lines with one vertex within 30 pixels of the center of the clock are used as the hour hand and minute hand respectively, and the remaining straight lines are paired according to the vertex positions and used as the arrows of the hour hand and minute hand respectively.

[0028] The technical solution adopted by the embodiment of the present application also includes: the handwritten digit recognition result, the clock area detection result, the pointer image detection result and the hour hand and minute hand classification result are scored according to the set scoring rules:

[0029] All numbers within the clock area will be awarded 3 points, and numbers beyond the clock area will be deducted 0.5 points per number, until 0 points;

[0030] The symmetrical distribution of the four iconic numbers "12, 3, 6, 9" will get 2 points, that is, 1 point for 3 and 9 on the horizontal line, and 1 point for 12 and 6 on the vertical line. Otherwise, 0.5 points will be deducted for each number, until 0 points;

[0031] If the positions of the numbers other than "12, 3, 6, 9" are accurate, 3 points will be awarded. If the positions of the numbers are wrong or biased, 0.5 points will be deducted for each number, up to 0 points.

[0032] First anchor the 4 key points "12, 3, 6, 9" to get 4 points, otherwise 1 point will be deducted for each point until 0 points;

[0033] 4 points will be awarded for writing the 12 numbers accurately, otherwise 0.5 points will be deducted for each number, until 0 points;

[0034] 1 point for clockwise arrangement;

[0035] 1 point for the correct order of numbers from 1 to 12;

[0036] 1 point for correct center point location;

[0037] Draw the hour and minute hands for 2 points;

[0038] 2 points if the hour hand points correctly;

[0039] 2 points for the correct pointing of the minute hand;

[0040] The minute hand is 2 minutes longer than the hour hand;

[0041] 2 points if both the hour and minute hands have arrows;

[0042] 1 point for a complete clock face.

[0043] Another technical solution adopted by the embodiment of the present application is: a clock drawing test scoring device, comprising:

[0044] Data generation module: used to generate handwritten numbers from 1 to 12 using the MNIST database, and randomly place the handwritten numbers and the drawn clock images in a black image with a pixel value of 0 to generate a clock drawing image data set;

[0045] Digital recognition module: used for inputting the clock drawing image data set into the FCOS network, and recognizing the handwritten digits in the clock drawing image data set through the FCOS network;

[0046] A clock detection module: used for performing circle detection on the clock image dataset using a maximum connected domain algorithm to obtain a clock region in the clock image dataset;

[0047] Pointer detection module: used for performing pointer detection on the clock image dataset using an image processing algorithm based on the recognition result of the handwritten digits and the detection result of the clock area, obtaining a pointer image of the clock image dataset, and classifying the hour hand and minute hand of the pointer image using a breadth-first algorithm;

[0048] Clock drawing scoring module: used to score the handwritten digit recognition results, clock area detection results, pointer image detection results, and hour and minute hand classification results according to the set scoring rules, and add up all the scoring results to generate the clock drawing test scoring results.

[0049] Another technical solution adopted by the embodiment of the present application is: a computer device, the computer device includes a processor and a memory coupled to the processor, wherein:

[0050] The memory stores program instructions for implementing the clock drawing test scoring method;

[0051] The processor is used to execute the program instructions stored in the memory to control the clock drawing test scoring method.

[0052] Another technical solution adopted by the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the clock drawing test scoring method.

[0053] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows: the clock drawing test scoring method, device, computer equipment and storage medium of the embodiments of the present application adopt a digital recognition and positioning method based on a deep learning model, which can quickly complete the recognition and positioning of all numbers in a single clock drawing image, clock area detection, pointer image detection and classification on the CPU, greatly speeding up the scoring of the clock drawing test and avoiding evaluation fluctuations and differences caused by the subjectivity and bias of manual evaluation. At the same time, the embodiments of the present application can better simulate the sampling of real data, continuously optimize the deep learning model, and make the model more robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of a clock drawing test scoring method according to an embodiment of the present application;

[0055] Figure 2 This is a schematic diagram of constructing a clock drawing image dataset according to an embodiment of the present application;

[0056] Figure 3 This is a schematic diagram of handwritten digit recognition and positioning based on the FCOS network in an embodiment of the present application;

[0057] Figure 4 Schematic diagram of scoring rules for each scoring item in the embodiment of the present application;

[0058] Figure 5 This is a schematic diagram of the structure of a clock drawing test scoring device according to an embodiment of the present application;

[0059] Figure 6 A schematic diagram of the computer device structure of an embodiment of the present application;

[0060] Figure 7 A schematic diagram of the structure of a storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] The terms "first", "second", "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the embodiments of this application, all directional indications (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or computer device that includes a series of steps or units is not limited to the steps or units listed, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or computer devices.

[0063] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0064] See also Figure 1 , is a flow chart of the clock drawing test scoring method of the embodiment of the present application. The clock drawing test scoring method of the embodiment of the present application can be deployed on the server, and the server is provided with an API interface for the client to call the clock drawing test scoring method. Specifically, the clock drawing test scoring method of the embodiment of the present application includes the following steps:

[0065] S100: Generate handwritten numbers from 1 to 12 using the MNIST database, and randomly place the handwritten numbers and pre-drawn clock images in a black image with a pixel value of 0 to generate a clock image dataset;

[0066] In this step, in order to generate a large number of handwritten digits in a short time, the embodiment of the present application uses MNIST (Mixed National Institute of Standards and Technology database, which contains 70,000 28×28 grayscale images, each of which contains a handwritten digit from 0 to 9) as the basic database, and obtains handwritten digits from 1 to 12 from the database. Specifically, the clock drawing image data set generation method includes:

[0067] S101: Obtain handwritten digits 0 to 9 from the MNIST database, and then concatenate the handwritten digit "1" with "0", "1", and "2" respectively to generate handwritten digits "10", "11", and "12";

[0068] S102: Draw a clock image of a set size; wherein the clock image is a circle;

[0069] S103: performing image enhancement operations on the generated handwritten numerals and clock images respectively to obtain the handwritten numerals and clock images after the image enhancement operations;

[0070] Among them, in order to be closer to the actual digital tablet clock drawing test, the embodiment of the present application performs random angle rotation, random size scaling and other image enhancement operations on the generated handwritten digits and clock images (the size of the original handwritten digits and clock images is 28×28, and the size of the handwritten digits and clock images after the image enhancement operation is between 16 and 40). At the same time, in order to simulate the situation where the digits are broken due to the test personnel's unfamiliarity with the digital tablet when drawing a clock on the digital tablet, the embodiment of the present application also adds random image corrosion during the image enhancement operation to simulate the situation of digit breakage.

[0071] S104: Randomly place the handwritten numerals and clock image after the image enhancement operation in a black image with a pixel value of 0 to generate a single clock image, and record the position and size of each handwritten numeral in the single clock image; wherein the image size of the black image is 1024×1024, which can be set according to the actual application scenario.

[0072] S105: S104 is executed repeatedly according to the set number of iterations to obtain a set number of single clock-drawing images and construct a clock-drawing image data set.

[0073] Among them, Figure 2 The figure shows a schematic diagram of the clock image dataset constructed in the embodiment of the present application. The first column is the generated handwritten numbers from 1 to 12, where "10", "11", and "12" are obtained by splicing two handwritten numbers in MNIST; the second column is the drawn clock image. Then, the handwritten numbers and the clock image are enhanced and randomly placed in a 1024x1024 black image with all pixel values ​​​​being 0 to generate a single clock image.

[0074] S110: Input the clock image dataset into the FCOS (Fully Convolutional One-Stage Object Detection) network, and automatically identify the handwritten numbers in each clock image through the FCOS network;

[0075] In this step, if Figure 3As shown, it is a schematic diagram of handwritten digit recognition and positioning based on FCOS network in the embodiment of the present application. The FCOS model training of the embodiment of the present application adopts Python 3.8 version and Cuda 11.4 version. The CPU adopts Intel (R) Xeon (R) Silver 4116 CPU @ 2.10GHz. FCOS uses random gradient descent method for parameter optimization. The number of model training iterations is 84,000 times, and the training weights and FCOS model are packaged in ONNX format, providing CPU support scripts to support Linux environment deployment. The input image of the FCOS network is any clock image in the clock image data set. After the FCOS network is calculated, two outputs are obtained, which are the recognition result of each handwritten digit in the input image and the regression value of the positioning area (presented in the form of a rectangular box) in the clock image where it is located, which realizes the accurate recognition and positioning of handwritten digits in the presence of dial image interference.

[0076] S120: performing circle detection on the clock drawing image data set using a maximum connected region algorithm to obtain a clock region in the clock drawing image;

[0077] In this step, considering that there is no curvature score for a circle in the clock drawing test scoring criteria, there is no need to pay attention to whether the detected circular area is a perfect circle. The embodiment of the present application uses the maximum connected domain algorithm to detect circles on the clock drawing image dataset, that is, it is assumed that all clock images drawn by the testers follow the rule that the clock area is larger than any digital area. Based on this assumption, the connected domain detection algorithm is used to arrange the area where the maximum connected domain is located as the clock area, and the filling algorithm and the edge detection algorithm are used to extract the clock area to obtain the clock area of ​​the circle.

[0078] S130: Based on the handwritten digit recognition result and the clock region detection result, an image processing algorithm is used to perform pointer detection on the clock image dataset to obtain a pointer image including an hour hand and a minute hand;

[0079] In this step, an image processing algorithm is used to detect the pointer of the clock image dataset. Specifically, first, the pixel values ​​of the handwritten digit positioning area and the clock area are filled with 0, thereby filtering out the handwritten digits and the clock area in the clock image to obtain the pointer image, and the Canny operator is used to extract the pointer edge in the pointer image to eliminate other noise information except the pointer.

[0080] S140: using a BFS (Breadth-First-Search) algorithm to perform straight line detection on the pointer image, and classifying the pointer according to the straight line detection result to obtain the hour hand and minute hand and the corresponding arrows;

[0081] In this step, the pointer classification method is specifically as follows: first, the Hough line detection method is used to find all the straight lines in the pointer image, which include the pointers and arrows of the hour hand and minute hand; then, the BFS algorithm is used to detect the length of each straight line along the direction of each straight line, and the length and two end vertices of each straight line are marked, and then the two longest straight lines with one end vertex located within 30 pixels of the center of the clock are used as the hour hand and minute hand respectively, and the remaining straight lines are paired according to the vertex positions as the arrows of the hour hand and minute hand respectively.

[0082] S150: scoring the handwritten digit recognition results, the clock region detection results, the pointer image detection results, and the hour hand and minute hand classification results respectively according to the set scoring rules, adding the scoring results of all scoring items to generate the final clock drawing test scoring result;

[0083] In this step, if Figure 4 , which is a schematic diagram of the scoring rules for each scoring item in the embodiment of the present application. Specifically, the scoring rules for the handwritten digit recognition results and the clock area detection results include:

[0084] (1) All numbers within the clock area are scored 3 points. Numbers beyond the clock area are deducted 0.5 points per number, up to 0 points. Specifically, the scoring algorithm for scoring rule (1) is as follows: after obtaining the digital positioning area and the clock area, first fill the clock area, that is, fill all pixels in the clock area with 255; then use a loop to traverse all numbers, and add the pixel values ​​of the digital positioning area and the clock area in each loop, and then subtract the result of the addition from the filling map of the clock area to determine whether there are other pixels with a pixel value of 255 in the clock area. If so, it proves that the number is beyond the clock area. Otherwise, it is determined that the number is completely within the clock area.

[0085] (2) The symmetrical distribution of the four iconic numbers "12, 3, 6, 9" is scored 2 points, that is, 1 point if 3 and 9 are on the horizontal line, and 1 point if 12 and 6 are on the vertical line. Otherwise, 0.5 points will be deducted for each number, up to 0 points. Specifically, the scoring algorithm for scoring rule (2) is as follows: after obtaining the results of digital positioning and recognition, the four iconic numbers "12, 3, 6, 9" are extracted respectively. For the symmetry of "12" and "6", the embodiment of the present application sets that the horizontal coordinate difference between the two numbers is not allowed to exceed 40% of the width of the numbers. If it exceeds, it is judged that "12" and "6" are asymmetrical, otherwise they are symmetrical. Similarly, for the symmetry of "3" and "9", the embodiment of the present application sets that the vertical coordinate difference between the two numbers is not allowed to exceed 20% of the width of the numbers (because the vertical height of the numbers is greater than the horizontal width). If it exceeds, it is judged that "3" and "9" are asymmetrical, otherwise they are symmetrical.

[0086] (3) 3 points are awarded if the positions of numbers other than "12, 3, 6, 9" are accurate. If the positions of numbers are wrong or biased, 0.5 points are deducted for each number, up to 0 points. Specifically, the scoring algorithm for scoring rule (3) is as follows: after obtaining the digital positioning area and the clock area, the center points of the digital positioning area and the clock area are preferentially calculated, and the center point of the clock area is used as the vertex at one end and the center point of the digital positioning area is used as the vertex at the other end. The directions of the straight lines are calculated respectively to determine whether the absolute deviation between the direction of the straight line and the set direction is within a set degree (the degree is set to 30° in the embodiment of the present application, and can be set according to the actual application scenario). If so, the digital position is determined to be accurate. Otherwise, the digital position is determined to be incorrect.

[0087] (4) First anchor the four key points "12, 3, 6, 9" to get 4 points, otherwise deduct 1 point per point until 0 points; the scoring algorithm for scoring rule (4) is specifically as follows: retain the image of each painting stroke, and use the digital handwriting recognition and positioning algorithm to identify the number drawn by each painting stroke, until the first number that appears is within the four key points "12, 3, 6, 9". If the first number that appears is another number, it is determined that the four key points "12, 3, 6, 9" are not first anchored. Among them, for the number "12", it may happen that "1" is detected first and then "12". In order to avoid misjudgment, the embodiment of the present application performs lenient treatment on the number "1" when detecting numbers, that is, it allows waiting for the number "1" to be transformed into the number "12" when it is determined to be the number "1". If the transformation does not occur after the step of "1", the number is considered to be "1".

[0088] (5) 4 points will be awarded if the 12 numbers are written accurately, otherwise 0.5 points will be deducted for each number, up to 0 points. The scoring algorithm for scoring rule (5) is as follows: after obtaining the digital positioning and recognition results, all numbers in the clock area are counted to determine whether all numbers 1-12 exist, and deduct points according to the scoring criteria.

[0089] (6) 1 point for clockwise arrangement;

[0090] (7) 1 point is awarded for the correct order of numbers 1 to 12. Specifically, the scoring algorithm for scoring rules (6) and (7) is as follows: calculate the inclination of the straight line between each number and the center point and the vertical direction, and determine whether the inclination is monotonically increasing. If so, it is determined that the arrangement is clockwise and the order is correct; otherwise, it is determined that there is no clockwise arrangement and the order is wrong.

[0091] The scoring rules for the pointer image detection results and the hour and minute hand classification results include:

[0092] (8) 1 point is awarded for a correct center point position. Specifically, the scoring algorithm for scoring rule (8) is as follows: after obtaining the hour hand and minute hand detection results, the center point is the intersection of the two pointers. It is determined whether the Euclidean distance between the intersection and the center point of the circular area is within 40 pixels. If so, the center point position is determined to be correct; otherwise, the center point position is determined to be incorrect.

[0093] (9) Draw the hour hand and minute hand to get 2 points; the scoring algorithm for scoring rule (9) is as follows: after obtaining the hour hand and minute hand detection results, determine whether the hour hand and minute hand exist in the detection results, and deduct points according to the scoring criteria.

[0094] (10) 2 points if the hour hand points correctly;

[0095] (11) 2 points are awarded for the correct pointing of the minute hand; Specifically, the scoring algorithm for scoring rules (10) and (11) is as follows: Taking into account the situation that the hour hand may be longer than the minute hand in the clock drawing test, it is necessary to judge the directions of the two pointers separately, calculate the angle difference between the two pointer directions and the set pointer direction respectively, and set the pointer category according to the angle difference. For example: Assuming that one of the pointers is the minute hand, and the angle difference between the pointer direction and the set minute hand direction is within the set difference range (30°), then the pointer is judged to be the minute hand and the pointer is pointing correctly. If there is a mismatch, the hour hand and minute hand are allocated by using the minimum phase difference inclination angle, and the pointer that does not meet the inclination range is judged as not pointing in the correct direction.

[0096] (12) The minute hand is 2 minutes longer than the hour hand;

[0097] (13) If both the hour hand and the minute hand have arrows, 2 points are awarded. The scoring algorithm for scoring rules (12) and (13) is as follows: compare the lengths of the hour hand and minute hand division results according to scoring rules (10) and (11), and determine whether the pointer contains an arrow.

[0098] (14) 1 point is awarded for a complete clock face. The specific scoring algorithm for scoring rule (14) is as follows: after completing the remaining scoring items, check the numbers, hour hand and minute hand in the clock area. If the numbers, hour hand and minute hand are all complete, the clock face is considered complete; otherwise, the clock face is considered incomplete.

[0099] Based on the above, the clock drawing test scoring method of the embodiment of the present application adopts a digital recognition and positioning method based on a deep learning model, which can quickly complete the recognition and positioning of all numbers in a single clock drawing image, clock area detection, pointer image detection and classification on the CPU, greatly speeding up the scoring of the clock drawing test and avoiding evaluation fluctuations and differences caused by the subjectivity and bias of manual evaluation. At the same time, the embodiment of the present application can better simulate the sampling of real data, continuously optimize the deep learning model, and make the model more robust.

[0100] See also Figure 5 , is a schematic diagram of the structure of a clock drawing test scoring device according to an embodiment of the present application. The clock drawing test scoring device 40 according to an embodiment of the present application comprises:

[0101] Data generation module 41: used to generate handwritten numbers from 1 to 12 using the MNIST database, and randomly place the handwritten numbers and the drawn clock image in a black image with a pixel value of 0 to generate a clock drawing image data set;

[0102] The digit recognition module 42 is used to input the clock drawing image data set into the FCOS network, and recognize the handwritten digits in the clock drawing image data set through the FCOS network;

[0103] The clock detection module 43 is used to perform circle detection on the clock image dataset using a maximum connected region algorithm to obtain a clock region in the clock image dataset;

[0104] Pointer detection module 44: used for performing pointer detection on the clock image dataset using an image processing algorithm based on the recognition result of the handwritten digits and the detection result of the clock region, obtaining a pointer image of the clock image dataset, and classifying the hour hand and minute hand of the pointer image using a breadth-first algorithm;

[0105] The clock drawing scoring module 45 is used to score the handwritten digit recognition results, clock area detection results, pointer image detection results and hour hand and minute hand classification results according to the set scoring rules, and add up all the scoring results to generate the clock drawing test scoring results.

[0106] See also Figure 6 , is a schematic diagram of the computer device structure of an embodiment of the present application. The computer device 50 includes:

[0107] A memory 51 storing executable program instructions;

[0108] A processor 52 connected to the memory 51;

[0109] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: using the MNIST database to generate handwritten digits from 1 to 12, and randomly placing the handwritten digits and the drawn clock image in a black image with a pixel value of 0 to generate a clock image data set; inputting the clock image data set into the FCOS network, and identifying the handwritten digits in the clock image data set through the FCOS network; using the maximum connected domain algorithm to perform circle detection on the clock image data set to obtain the clock region in the clock image data set; based on the recognition results of the handwritten digits and the detection results of the clock region, using an image processing algorithm to perform pointer detection on the clock image data set to obtain the pointer image of the clock image data set, and using a breadth-first algorithm to classify the pointer image into hour hands and minute hands; scoring the handwritten digit recognition results, the clock region detection results, the pointer image detection results, and the hour and minute hand classification results according to the set scoring rules, and adding all the scoring results to generate a clock drawing test scoring result.

[0110] The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0111] See also Figure 7 , Figure 7The structure diagram of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores a program instruction 61 capable of implementing the following steps: using the MNIST database to generate handwritten digits from 1 to 12, and randomly placing the handwritten digits and the drawn clock image in a black image with a pixel value of 0 to generate a clock image data set; inputting the clock image data set into the FCOS network, and identifying the handwritten digits in the clock image data set through the FCOS network; using the maximum connected domain algorithm to perform circle detection on the clock image data set to obtain the clock area in the clock image data set; based on the recognition results of the handwritten digits and the detection results of the clock area, using the image processing algorithm to perform pointer detection on the clock image data set to obtain the pointer image of the clock image data set, and using the breadth-first algorithm to classify the hour hand and minute hand of the pointer image; scoring the handwritten digit recognition results, the clock area detection results, the pointer image detection results, and the hour hand and minute hand classification results according to the set scoring rules, and adding all the scoring results to generate the clock drawing test scoring results. Among them, the program instruction 61 can be stored in the above-mentioned storage medium in the form of a software product, including several instructions for a computer device (which can be a personal computer, a server, or a network computer device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program instructions, or terminal computer devices such as computers, servers, mobile phones, tablets, etc. Among them, the server can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0112] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0113] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the description and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A clock drawing test scoring method, characterized in that: include: Generate handwritten numbers from 1 to 12 using the MNIST database, and randomly place the handwritten numbers and the drawn clock images in a black image with a pixel value of 0 to generate a clock drawing image dataset; Inputting the clock drawing image data set into the FCOS network, and recognizing the handwritten numerals in the clock drawing image data set through the FCOS network; Using a maximum connected region algorithm to perform circle detection on the clock drawing image dataset to obtain a clock region in the clock drawing image dataset; Based on the recognition result of the handwritten digits and the detection result of the clock area, an image processing algorithm is used to perform pointer detection on the drawn clock image data set to obtain a pointer image of the drawn clock image data set, and a breadth-first algorithm is used to classify the pointer image into hour hands and minute hands; According to the set scoring rules, the handwritten digit recognition results, clock area detection results, pointer image detection results and hour hand and minute hand classification results are scored respectively, and all the scoring results are added together to generate the clock drawing test scoring result.

2. The clock drawing test scoring method according to claim 1, characterized in that: The MNIST database is used to generate handwritten numbers from 1 to 12, and the handwritten numbers and the drawn clock images are randomly placed in a black image with a pixel value of 0 to generate a clock image dataset: Obtain handwritten digits 0 to 9 from the MNIST database, and concatenate the handwritten digit "1" with "0", "1", and "2" respectively to generate handwritten digits "10", "11", and "12"; Draw a clock image of a set size; wherein the clock image is a circle; Performing image enhancement operations on the handwritten numerals and the clock image respectively to obtain the handwritten numerals and the clock image after the image enhancement operations; Randomly placing the handwritten numerals and the clock image after the image enhancement operation in a black image with a pixel value of 0 to generate a single clock image, and recording the position and size of each handwritten numeral in the single clock image; A single clock-drawing image is generated cyclically according to the set number of iterations to construct a clock-drawing image dataset.

3. The clock drawing test scoring method according to claim 2, characterized in that: The step of inputting the clock drawing image data set into the FCOS network and identifying the handwritten numerals in the clock drawing image data set through the FCOS network is specifically as follows: The input image of the FCOS network is any clock image in the clock image data set, and after calculation, the recognition result of each handwritten digit in the input image and the positioning area of ​​the handwritten digit in the clock image are output.

4. The clock drawing test scoring method according to claim 3, characterized in that: The circle detection is performed on the clock image data set by using the maximum connected region algorithm, and the clock region in the clock image data set is obtained as follows: It is assumed that the clock images drawn by all testers follow the rule that the clock area is larger than any digital area. Based on this assumption, a connected domain detection algorithm is used to arrange the area where the largest connected domain is located as the clock area, and the filling algorithm and the edge detection algorithm are used to extract the clock area to obtain the circular clock area.

5. The clock drawing test scoring method according to claim 4, characterized in that: The method of using an image processing algorithm to perform pointer detection on the clock image data set based on the recognition result of the handwritten digits and the detection result of the clock area is as follows: Filling the pixel values ​​of the positioning area of ​​the handwritten numbers and the clock area with 0, filtering out the handwritten numbers and the clock area in the clock drawing image, and obtaining a pointer image; The Canny operator is used to extract the pointer edge in the pointer image, and other noise information except the pointer is eliminated.

6. The clock drawing test scoring method according to claim 5, characterized in that: The method of using the breadth-first algorithm to classify the hour hand and minute hand of the pointer image is specifically as follows: Using the Hough line detection method to find all the straight lines in the pointer image, the straight lines include the pointers of the hour hand and the minute hand and the arrows; The BFS algorithm is used to detect the length of each straight line along the direction of each straight line, and the length of each straight line and the vertices at both ends are marked. The two longest straight lines with one vertex within 30 pixels of the center of the clock are used as the hour hand and minute hand respectively, and the remaining straight lines are paired according to the vertex positions and used as the arrows of the hour hand and minute hand respectively.

7. The clock drawing test scoring method according to any one of claims 1 to 6, characterized in that: According to the set scoring rules, the handwritten digit recognition results, clock area detection results, pointer image detection results, and hour and minute hand classification results are scored respectively as follows: All numbers within the clock area will be awarded 3 points, and numbers beyond the clock area will be deducted 0.5 points per number, until 0 points; "12, 3, 6, 9" 2 points if the 4 symbolic numbers are symmetrically distributed, i.e. 1 point if 3 and 9 are on the horizontal line, 1 point if 12 and 6 are on the vertical line, otherwise 0.5 points will be deducted for each number, until 0 points; If the positions of the numbers other than "12, 3, 6, 9" are accurate, 3 points will be awarded. If the positions of the numbers are wrong or biased, 0.5 points will be deducted for each number, up to 0 points. First anchor the 4 key points "12, 3, 6, 9" to get 4 points, otherwise 1 point will be deducted for each point until 0 points; 4 points will be awarded for writing the 12 numbers accurately, otherwise 0.5 points will be deducted for each number, until 0 points; 1 point for clockwise arrangement; 1 point for the correct order of numbers from 1 to 12; 1 point for correct center point location; Draw the hour and minute hands for 2 points; 2 points if the hour hand points correctly; 2 points for the correct pointing of the minute hand; The minute hand is 2 minutes longer than the hour hand; 2 points if both the hour and minute hands have arrows; 1 point for a complete clock face.

8. A clock drawing test scoring device, characterized in that: include: Data generation module: used to generate handwritten numbers from 1 to 12 using the MNIST database, and randomly place the handwritten numbers and the drawn clock images in a black image with a pixel value of 0 to generate a clock drawing image data set; Digital recognition module: used for inputting the clock drawing image data set into the FCOS network, and recognizing the handwritten numbers in the clock drawing image data set through the FCOS network; A clock detection module: used for performing circle detection on the clock image dataset using a maximum connected domain algorithm to obtain a clock region in the clock image dataset; Pointer detection module: used for performing pointer detection on the clock image dataset using an image processing algorithm based on the recognition result of the handwritten digits and the detection result of the clock area, obtaining a pointer image of the clock image dataset, and classifying the hour hand and minute hand of the pointer image using a breadth-first algorithm; Clock drawing scoring module: used to score the handwritten digit recognition results, clock area detection results, pointer image detection results, and hour and minute hand classification results according to the set scoring rules, and add up all the scoring results to generate the clock drawing test scoring results.

9. A computer device, characterized in that: The computer device includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the clock drawing test scoring method according to any one of claims 1 to 7; The processor is used to execute the program instructions stored in the memory to control the clock drawing test scoring method.

10. A storage medium, characterized in that: The device stores program instructions executable by a processor, wherein the program instructions are used to execute the clock drawing test scoring method according to any one of claims 1 to 7.

Images