An objective quantitative measurement method for cognitive ability combining finger dexterity and span

Abstract

The present invention discloses an objective quantitative measurement method for cognitive ability that combines finger flexibility and span. The finger flexibility and the span of each finger are calculated by measuring the continuous keystroke speed of the fingers and the difference between the key numbers of different fingers. The reaction speed score and the reaction accuracy score are calculated by measuring the reaction time of the fingering and the accuracy of the fingering. The hand-eye coordination and spatial positioning ability are evaluated, and finally the cognitive ability is evaluated. The steps are as follows: calculate the span of each finger by measuring the difference between the key numbers of different fingers; calculate the flexibility of each finger by measuring the time interval of continuous keystrokes of each finger; display the fingering questions in the form of animation, calculate the reaction time according to the time interval between the measured question notes and the actual playing time interval, calculate the playing accuracy according to the question fingering and the actual key number and the distance between the fingers; quantitatively evaluate cognitive ability through information fusion. The present invention provides an interesting, non-invasive and simple objective detection method for early intervention in brain health.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an objective quantitative measurement method for cognitive ability combining finger dexterity and span. Background Art

[0002] Brain health is the foundation of human health, well-being, and development, and a key pillar of modern social development. With the rapid advancement of modern medicine, people have gained a deeper understanding of the brain's functions and roles, and are increasingly concerned about brain health. Compared to the measurability of other physical health indicators, brain health lacks effective measurement and assessment methods, making effective maintenance and promotion of brain health a challenge.

[0003] Existing objective measurement methods for assessing brain health are categorized as destructive and non-invasive. Destructive testing methods include both invasive and non-invasive methods. Invasive methods, such as anatomical anatomy and implantable electrodes, require the involvement of specialized neurosurgeons and carry high surgical risks. They are generally used for the detection and treatment of patients with moderate to severe brain diseases and are not suitable for general public use. Non-invasive methods, such as magnetic resonance imaging (MRI) and computed tomography (CT), carry radiation hazards and are expensive, making them inadequate for dynamic monitoring of health status. Non-invasive methods include functional near-infrared neurography (fNIRS), neuroelectrophysiological testing, and computerized assessments. Functional near-infrared neurography is susceptible to low signal strength and head movement, requiring a high laboratory environment. Neuroelectrophysiological tests, such as electroencephalography (EEG) and electromyography (EMG), can cause pain or discomfort to the subject. Computerized assessments use digital equipment to diagnose and analyze brain health based on the test results. These tests are susceptible to language and cultural differences, hindering widespread adoption.

[0004] Brain health is difficult to fully and accurately measure and assess through non-invasive and non-destructive methods. However, cognitive ability, as the brain's highest level of ability, is related to brain health and human intelligence. By assessing cognitive ability, we can provide important information for brain health, help people better understand their own cognitive ability levels, and thus provide guidance for the prevention and treatment of brain diseases. Existing methods for assessing cognitive ability are generally scale assessments or behavioral observations. Evaluations are made by observing the subject's response to questions and combining the tester's experience. These are easily influenced by subjective consciousness and the subject's educational level. The test results rely on the tester's professional level and lack objectivity. Misdiagnosis and missed diagnosis are inevitable.

[0005] The present invention's research process has revealed at least one finding: executive function and cognitive ability are proven to be correlated. Since fingers account for the largest portion of the brain's motor and sensory cortices, indirectly measuring cognitive ability by measuring finger executive function overcomes the shortcomings of previously described methods and demonstrates its advanced nature. Furthermore, finger executive function is also influenced by physiological factors unrelated to cognition, such as finger function and structure. Therefore, considering finger information when measuring executive function is scientifically sound. Therefore, indirect and objective measurement of cognitive ability through piano sight-reading, combined with finger information, is of great significance for early intervention in brain health. Summary of the Invention

[0006] The purpose of the present invention is to address the above-mentioned defects in the prior art and to provide an objective quantitative measurement method for cognitive ability that combines finger dexterity and span.

[0007] The purpose of the present invention can be achieved by taking the following technical solutions:

[0008] An objective quantitative measurement method for cognitive ability combining finger dexterity and span, the objective quantitative measurement method for cognitive ability comprising the following steps:

[0009] S1. A question-generating module with a display displays the status of two designated fingers and keys in text and image form. The piano keys are numbered from 1 to 88 from left to right according to their physical positions, which are recorded as key numbers. The acquisition module with a key scanner outputs the key numbers pressed by the fingers, and the difference between the two is used to calculate the span of each finger.

[0010] S2, a question-setting module with a display displays the name of a designated finger and the number of consecutive keystrokes in text form, outputs the finger pressing time through a collection module with a timer, and calculates the dexterity of each finger by measuring the time interval between consecutive keystrokes of each finger;

[0011] S3, a question-setting module with a display displays a fingering question in an animated form, wherein the fingering refers to pressing a piano key with a certain finger, an acquisition module with a key scanner and a timer outputs the finger pressing time and the corresponding key number, a camera takes a photo of the finger pressing moment, the actual finger pressing name is obtained through finger recognition, the reaction time is calculated based on the time interval between the question-setting key presses and the time interval between the actual key presses, and the playing accuracy is calculated based on the distance between the question-setting fingering and the actual fingering;

[0012] S4. Calculate the reaction speed score by combining finger flexibility and reaction time, and calculate the reaction accuracy score by combining finger span and playing accuracy, and quantitatively measure cognitive ability through information fusion.

[0013] Furthermore, the process of step S1 is as follows:

[0014] S101. For ease of labeling, the physical positions of the fingers are sequentially defined from left to right as 1 to 10, which are recorded as finger numbers. Based on the finger position distribution, multiple rounds of finger span measurement tasks are set. Test prompts and finger images are displayed on a module with a display. Each round of the test task measures the finger span in a relaxed state and an exerted state.

[0015] According to the position distribution of fingers, multiple rounds of finger span measurement tasks are set up, including 1 finger 2 fingers, 1 finger 3 fingers, 1 finger 4 fingers, 1 finger 5 fingers, 2 fingers 3 fingers, 2 fingers 4 fingers, 2 fingers 5 fingers, 3 fingers 4 fingers, 3 fingers 5 fingers, 4 fingers 5 fingers, 6 fingers 7 fingers, 6 fingers 8 fingers, 6 fingers 9 fingers, 6 fingers 10 fingers, 7 fingers 8 fingers, 7 fingers 9 fingers, 7 fingers 10 fingers, 8 fingers 9 fingers, 8 fingers 10 fingers, and 9 fingers 10 fingers. Test prompt information and finger images are displayed on a module with a display. Each round of test tasks measures the span of the corresponding fingers in a relaxed state and an exerted state, comprehensively reflecting the span conditions between various fingers in various situations;

[0016] S102: Output the key number pressed in each round of testing through the acquisition module with key scanner, and calculate the maximum key number H max and the minimum key number H min , corresponding to the rightmost and leftmost buttons pressed by the finger in each round of testing;

[0017] S103, collect H max and H min Subtract the finger span S_relax of finger x and finger y in the relaxed state xy and the finger span S_max in the best effort state xy , by measuring the key number difference of the corresponding finger in different states, the span of the corresponding finger in different states is reflected;

[0018] S104, by calculating S_relax xy and S_max xy The average value of finger x and finger y is the span S xy , where x and y are the finger numbers. The calculated finger span takes into account both relaxation and effort, and is more consistent with the actual finger span during playing.

[0019] Furthermore, the process of step S2 is as follows:

[0020] S201. To objectively quantify finger dexterity, set up 10 rounds of dexterity testing tasks to measure the dexterity of each finger. The finger name and the number of consecutive keystrokes are displayed in text form on a module with a display. Each round of testing tasks quantifies the dexterity of the corresponding finger by measuring the time interval between consecutive keystrokes of the corresponding finger. The shorter the time interval, the higher the finger dexterity.

[0021] S202. To reduce the I / O ports, a 6×28 matrix is designed to implement the signal scanning function of 84×2 key contacts of the keyboard. Through this function module, the key number and the pressing time of the finger pressing in each round of test tasks are output. To prevent errors caused by accidental key presses during the test, the mode G of the key numbers collected in this round is calculated by the Moore voting method. mode , and the data of this round is filtered to obtain the key number G mode of the key press time series T = {T i , 1≤i≤n}, where T i represents the i-th data of the key press time series with the key number G mode , and n is the length of the key press time series with the key number G mode ;

[0022] S203. Since the measurement data is inaccurate due to the influence of the brain reaction at the beginning of key pressing and the influence of finger fatigue near the end of the test, to ensure the accuracy and effectiveness of the data, the minimum time interval of pressing the key with the key number G mode multiple times is calculated by the sliding window method to quantify the finger flexibility F j , reflecting the stable and exact flexibility of the finger;

[0023] Among them, the calculation formula for the finger flexibility F j of the finger numbered j is:

[0024] F j = min(T k - T1,..., T n - T n-k+1 ), 1 < j < 10 Formula (1)

[0025] Among them, k is the number of consecutive key presses, and T k , T n-k+1 , T n are the k-th, n-k+1-th, and n-th data of the key press time series of the key with the key number K mode .

[0026] Further, the process of the step S3 is as follows:

[0027] S301. Set multiple fingering difficulties according to the number of keys appearing and the time interval between keys. The more keys there are and the shorter the time interval between keys, the more complex the playing is, and the faster the reaction speed required to complete the question is;

[0028] S302. Multiple rounds of fingering questions are set up with increasing difficulty. To prevent the subject from memorizing the questions, each round of questions is automatically generated according to the difficulty. The question-asking time, question-asking key number, and question-asking finger number sequence are obtained and displayed in an animated form. For example, a moving red vertical line moves to the question-asking finger position to indicate the question-asking time. This method of gradually increasing difficulty allows the subject to become familiar with the fingering test while stimulating the activation of the corresponding brain area, better reflecting the brain's cognitive ability boundaries. S303. The key number and pressing time sequence for each round of test tasks are output through the acquisition module, and a camera is driven by a synchronization signal at the pressing time to take a real-time finger picture, ensuring that each picture corresponds to each pressing time.

[0029] S304: To facilitate calculation of reaction times for multiple and fewer presses, a greedy algorithm is used to calculate a sequence of press times with the minimum reaction time, and the sequence of press times is aligned with a sequence of standard note playing times to calculate the reaction time.

[0030] Among them, the reaction time T react The calculation formula is:

[0031]

[0032] Among them, m is the number of notes in the fingering test, Q l Indicates the standard starting time for playing the first note of the fingering question in this round, P l Indicates the actual time the key is pressed for the first time;

[0033] S305, identifying the name of the pressed finger through the finger recognition image, and objectively quantifying and calculating the playing accuracy by measuring the spatial distance between the given fingering and the actual fingering;

[0034] Among them, the playing accuracy rate A play The calculation formula is:

[0035]

[0036] Among them, N l is the key number of the first note in this round of fingering questions, K l is the key number actually pressed for the first time, M l is the finger number corresponding to the first note of the fingering question, R l The actual number of the finger pressed for the first time.

[0037] Furthermore, the process of step S4 is as follows:

[0038] S401. To eliminate the influence of finger function on the test results, the reaction speed is recalculated by combining the dexterity information of each finger. The revised reaction speed takes into account the differences in finger function among different people and accurately represents the brain's hand-eye coordination and reaction ability.

[0039] Among them, the corrected reaction rate T' react The calculation formula is:

[0040]

[0041] Among them, α j is the flexibility benchmark coefficient of the finger numbered j, and the larger the value, the more flexible the finger;

[0042] S402, combining the span information of each finger, and recalculating the playing accuracy of each round of testing;

[0043] Among them, the corrected playing accuracy A′ play The calculation formula is:

[0044]

[0045] in, Indicates the number is M l Finger and number R l the span of a finger;

[0046] S403. Based on the probability statistics theorem, a reaction time and playing accuracy score mapping relationship table is established by measuring a large number of population data to obtain a reaction speed score and a playing accuracy score. The reaction time and playing accuracy score mapping relationship table establishes a one-to-one mapping relationship between the reaction time and the playing accuracy score, and intuitively quantifies the reaction speed and reaction accuracy of the subject's test in the form of a percentage score. At the same time, the subject's test score corresponds to the subject's ranking in the test population, and the higher the ranking, the higher the score;

[0047] S404. Cognitive ability can be reflected by reaction time and reaction accuracy. Since there is a speed-accuracy trade-off, that is, the subject may sacrifice reaction time to improve reaction accuracy, or may sacrifice reaction accuracy to improve reaction speed. The hand-eye cognition score is obtained by weighting the reaction speed score and playing accuracy score with a certain weight coefficient through information fusion, quantitatively measuring cognitive ability, comprehensively considering reaction speed and accuracy, and scientifically and effectively evaluating cognitive ability.

[0048] Furthermore, the question-setting module includes a display, a web system, a camera, a data transmission interface, and a storage medium, wherein the display is used to display the question and prompt information; the web system is used to control the setting of the question and read the analysis data from the serial port, and at the same time control the camera to shoot and store the test data in the storage medium; the camera is used to capture the image of the finger pressing the key, and can fully capture all the keys; the data transmission interface is used to connect to the external acquisition module to receive key information and key moment data in real time.

[0049] Furthermore, the acquisition module includes a key sensor, a scanning control circuit, and a data transmission serial port, wherein the key sensor includes two high and low contacts for judging the state of each key, and the keystroke force can be calculated by the triggering time difference of the two high and low contacts; the scanning control circuit uses the matrix scanning principle to obtain the key position of the trigger contact signal, and obtains the key number based on the position, saving the I / O port; the data transmission serial port is used to connect to the question-generating module, and transmits the package information of the key number and the pressing time through the synchronization signal, eliminating redundant untriggered key information, and collecting and transmitting data in real time and efficiently.

[0050] Furthermore, the finger recognition process in step S305 includes keyboard registration, finger key point recognition, and finger matching. Keyboard registration is to detect the center position coordinates of each key in the image through Canny edge detection and Hough line detection combined with the physical distribution of the keys. The black key is composed of a rectangle and has a center coordinate, and the white key is composed of two upper and lower rectangles and has two center coordinates; finger key point recognition is to identify the coordinates of the fingertip through the MediaPipe gesture recognition model. MediaPipe is a cross-platform framework based on deep learning open sourced by Google. Its gesture recognition The model can input an image and obtain the coordinates of the key points of the fingers in the image through deep learning methods; finger matching is to obtain the finger closest to the pressed key based on the distance between the coordinates of the fingertip and the center coordinates of the pressed key, which is recorded as the actual key finger. In particular, when the subject taps the upper and lower halves of the white key, the coordinates of the fingertips are far apart, and the distance between each fingertip coordinate and the center coordinates of the upper and lower halves of the white key is calculated respectively. If the fingers closest to the center positions of the upper and lower parts are inconsistent, the depth information of the two is compared, that is, the distance from the camera, and the finger with a larger depth is recorded as the actual key finger, which reduces the finger recognition error when tapping the white key.

[0051] Furthermore, the cognitive ability refers to the ability of the human brain to process, store and retrieve information. The research process of the present invention has at least found that: the finger execution ability is confirmed to be related to cognitive ability, and the finger execution ability is also affected by physiological factors such as finger function and structure that are not related to cognition. By measuring the reaction time and playing accuracy to the specified fingering, combining the finger flexibility and span to calculate the reaction speed score and reaction accuracy score, the influence of irrelevant factors is eliminated, and the hand-eye coordination ability and spatial positioning ability are evaluated.

[0052] The present invention has the following advantages and effects compared to the prior art:

[0053] 1) The present invention proposes an objective quantitative measurement method for cognitive ability based on reaction time and fingering accuracy combined with finger information. This method overcomes the limitations of traditional cognitive assessment methods and is not affected by factors such as subjective consciousness and the subject's educational level. At the same time, the test is automatically scored through a designed program without relying on professional testers, providing a practical response to problems such as early intervention in brain health and popularization of smart aesthetic education.

[0054] 2) The present invention collects the reaction time series and reaction fingering series of the subjects, compares them with the actual question-posing time series and question-posing fingering series, obtains reaction time and playing accuracy, combines statistics and cognitive psychology theories, and uses information fusion to perform weighted calculation on the two to obtain an objective score of brain cognitive ability. Through big data methods, the trade-off problem between reaction speed and reaction accuracy in cognitive assessment is solved.

[0055] 3) This method calculates finger dexterity by measuring the time intervals between consecutive keystrokes of different fingers, and calculates finger span by measuring the distance between keystrokes made by different fingers. During cognitive assessment, the reaction speed score is modified based on finger dexterity, and the playing accuracy score is modified based on finger span. This modification eliminates the influence of physiological factors unrelated to cognition, such as finger function and structure, and can more scientifically and accurately reflect cognitive ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0057] Figure 1 is a flow chart of a method for objectively and quantitatively measuring cognitive ability combining finger dexterity and span disclosed in an embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of an objective quantitative measurement system for cognitive ability in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of finger numbering in an embodiment of the present invention;

[0060] Figure 4 is a reference diagram of finger dexterity distribution in Example 1 of the present invention;

[0061] Figure 5 1 is a process diagram of fingering recognition in an embodiment of the present invention;

[0062] Figure 6 1 is a reaction time statistical histogram and a curve fitting schematic diagram in an embodiment of the present invention;

[0063] Figure 7 is a statistical histogram of playing accuracy in an embodiment of the present invention;

[0064] Figure 8 1 is a statistical histogram and curve fitting diagram of finger dexterity in Example 2 of the present invention;

[0065] Figure 9 This is an animated schematic diagram of a left-hand fingering problem of difficulty level 3 in Example 2 of the present invention. DETAILED DESCRIPTION

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0067] Example 1

[0068] This embodiment discloses an objective quantitative measurement method for cognitive ability combining finger dexterity and span, such as Figure 1 The specific steps are as follows:

[0069] S1. The question-generating module displays the status of two specified fingers and keys in the form of text and pictures. The piano keys are numbered from 1 to 88 from left to right according to their physical positions, which are recorded as key numbers. The acquisition module with a key scanner outputs the key numbers pressed by the fingers, and the difference between the two is calculated to obtain the span of each finger.

[0070] like Figure 2The figure shows a schematic diagram of the objective quantitative measurement system for cognitive ability. The system consists of an acquisition module and a test generation module. The test generation module includes a display, a web system, a camera, a serial data transmission port, and a storage medium. The display displays test questions and prompts; the web system controls the setting of test questions and reads parsed data from the serial port. It also controls the camera to capture test data and store it in the storage medium. The camera captures images of fingers pressing keys, fully capturing all keystrokes. The serial data transmission port connects to the acquisition module to obtain real-time keystroke information and keystroke timing.

[0071] The acquisition module includes a key sensor, a scanning control circuit, and a data transmission serial port. The key sensor includes two high and low contacts for determining the status of each key. The scanning control circuit utilizes the matrix scanning principle to obtain the key position that triggers the contact signal and then obtains the key number based on the position. The data transmission serial port is used to connect to the question-generating module, transmitting the packaged information of the key number and the time it was pressed in real time. The question-generating time, i.e., the time when the note to be played needs to be played, is displayed on the screen as the moment when the vertical progress line scrolls to the left edge of the image of the note to be played.

[0072] like Figure 3 The figure shows a schematic diagram of finger numbering, with the fingers designated as 1 to 10 from left to right according to their physical positions. Based on the finger distribution, multiple rounds of finger span measurement tasks were set up, including tasks like 1 finger 2 fingers, 1 finger 3 fingers, 1 finger 4 fingers, 1 finger 5 fingers, 2 fingers 3 fingers, 2 fingers 4 fingers, 2 fingers 5 fingers, 3 fingers 4 fingers, 3 fingers 5 fingers, 4 fingers 5 fingers, 6 fingers 7 fingers, 6 fingers 8 fingers, 6 fingers 9 fingers, 6 fingers 10 fingers, 7 fingers 8 fingers, 7 fingers 9 fingers, 7 fingers 10 fingers, 8 fingers 9 fingers, 8 fingers 10 fingers, and 9 fingers 10 fingers. Test prompts and finger images were displayed on a module with a display. Each round of the test task measured the span of the corresponding finger in both a relaxed and exerted state.

[0073] The key number pressed in each round of testing is output by the acquisition module with a key scanner. In order to avoid accidental touch, after the end of the round of testing, the key number sequence H={H a ,1≤a≤b} to obtain the maximum bond number H max and the minimum key number H min , where H a is the key number pressed for the ath time, b is the number of times the key is pressed in this test, and the key numbers of the acquisition module keys increase from left to right according to their physical positions.

[0074] The H collected in each round of test tasks max and H min Subtract the finger span S_relax of finger x and finger y in the relaxed state xyand the finger span S_max in the effort state xy . According to the calculation of the finger span S_relax in the relaxed state xy and the finger span S_max in the best effort state xy The average value of finger x and finger y is the span S xy The calculated spans of each finger are shown in Tables 1 and 2 below:

[0075] Table 1. Left hand finger span record in Example 1

[0076]

[0077]

[0078] The values in the table represent the span of the fingers, and “\” indicates that there is no span for the same finger.

[0079] Table 2. Record of right hand finger span in Example 1

[0080] Finger number 6 7 8 9 10 6 \ 7.5 8.5 9.5 10.5 7 7.5 \ 4.5 6 8 8 8.5 4.5 \ 3 4.5 9 9.5 6 3 \ 3.5 10 10.5 8 4.5 3.5 \

[0081] The values in the table represent the span of the fingers, and “\” indicates that there is no span for the same finger.

[0082] Finger span is expressed as piano interval difference, which facilitates the calculation of playing accuracy later. It can be seen that there are differences in the finger spans of the subjects, and it is necessary to measure finger span.

[0083] S2. The question-setting module displays the name of the designated finger and the number of consecutive keystrokes in text form, outputs the finger pressing time through the acquisition module with a timer, and calculates the flexibility of each finger by measuring the time interval between consecutive keystrokes of each finger.

[0084] A total of 10 test tasks are set for each finger of the left and right hands according to the finger number from small to large. The finger name and the number of consecutive keystrokes are displayed in text form on the module with a display. The keystroke time interval of the corresponding finger is measured in each test task.

[0085] The acquisition module with key scanner outputs the key number and pressing time of each round of test fingers. When the test ends, the web system of the question module obtains the test data of this round and obtains the key number sequence G={G v ,1≤v≤s} and the corresponding pressing time series T′={T′ v ,1≤v≤s}, s is the number of key presses in this round of testing, G v is the key number pressed for the vth time in this round of flexibility test, T′ vIt is the v - th pressing time in this round of flexibility test. To avoid errors and ensure the validity of data, the mode G in the sequence of button numbers collected in this round is calculated by the Moore voting method. mode , and calculate the number of occurrences n of the button number G mode . If the number of occurrences is less than the set number of consecutive key presses, the data is invalid and this round of test is re - executed. Otherwise, filter the sequence of pressing times to obtain the sequence of pressing times T = {T mode , 1 ≤ i ≤ n} of the button with button number G i . T i represents the i - th data in the sequence of pressing times of the button with button number G mode .

[0086] According to the times of pressing the button with button number G mode multiple times, calculate the flexibility of each finger. The flexibility reflects the finger's motor function.

[0087] Among them, the formula for the finger flexibility F j is:

[0088] F j = min(T k - T1, …, T n - T n-k+1 ), 1 < j < 10 Formula (1)

[0089] Among them, k is the number of consecutive key presses, and T k , T n-k+1 , T n are the k - th, n - k + 1 - th, and n - th data in the sequence of pressing times of the button with button number K mode .

[0090] Figure 4 is the reference diagram of finger flexibility distribution. It can be seen that the finger flexibility conforms to a normal distribution with a mean μ of 45330 and a standard deviation σ of 5038. Here, the value is the unit time, which is determined by the timer clock of the scan control circuit. Each unit time is converted into standard time as 31.25 microseconds. Then, the flexibility score is calculated according to the progressive score formula.

[0091] Among them, the progressive score formula is:

[0092] y = cD 2 - Z

[0093] Among them, y is the score, c is the coefficient, D is the time variable, and Z is the constant.

[0094] Among them, D is defined as:

[0095] D = 5+(t - u) / σ

[0096] Where t is the continuous keystroke time, u is the mean, and σ is the standard deviation.

[0097] S3. The question-setting module displays fingering questions in animated form. Fingering refers to pressing a key with a specific finger. The acquisition module, equipped with a key scanner and timer, outputs the finger pressing time and the corresponding key number. A camera takes a photo of the finger pressing moment, and the actual finger key name is obtained through finger recognition. Reaction time is calculated by measuring the time interval between the question key press and the time interval between the actual key press. Playing accuracy is calculated based on the distance between the question fingering and the actual fingering.

[0098] Set multiple fingering difficulties according to the number of keys that appear. Set the number of notes to 3 as difficulty 1, and the fingering playing order meets the conditions of finger numbering from small to large and key numbering from small to large, that is, the fingering order is from left to right in physical position, requiring fingers to run and play; set the number of notes to 5 as difficulty 2, and the fingering playing order meets the conditions of key numbering from small to large and the finger appearance order is 1 to 5 for the left hand and 6 to 10 for the right hand, that is, the fingering order is from left to right in physical position and all fingers of the tested hand are tested; set the number of notes to 8 as difficulty 3, and the fingering playing order meets the conditions of finger numbering from small to large and then from large to small and key numbering from small to large and then from large to small, that is, the fingering appearance order is from left to right and then to left in physical position, requiring fingers to run and play back and forth.

[0099] Three rounds of fingering questions are set for each hand according to increasing difficulty. Each round of questions will be automatically generated according to the difficulty and the test hand. The question time and the fingering sequence are obtained and displayed in the form of animation. The question time is displayed by moving a moving red vertical line to the fingering position of the question. The movement of the vertical line is controlled by the CSS animation of the web system. The width of the fingering question is used to set the pixel movement speed of the vertical line.

[0100] The acquisition module outputs the series of key numbers and pressing times for each test round. The data transmission serial port of the acquisition module communicates with the data transmission interface of the question-generating module at a certain baud rate, and uses synchronization signals to ensure synchronous data transmission. When the question-generating module receives the key signal, it drives the camera to take a real-time finger picture. The picture fully records the entire keyboard and finger information.

[0101] Align the first data of the pressing time series of each round of test with the standard note playing time series, and align the remaining data in sequence. If the series sizes are inconsistent, truncate or fill the pressing time series to calculate the reaction time T of each round of test. react .

[0102] Among them, the reaction time T react The calculation formula is:

[0103]

[0104] Among them, m is the number of notes in the fingering test, Q l Indicates the standard starting time for playing the first note of the fingering question in this round, P l Indicates the actual time the key is pressed for the first time;

[0105] The finger recognition image is used to identify the name of the pressed finger, and the playing accuracy is objectively quantified by measuring the spatial distance between the given fingering and the actual fingering.

[0106] Among them, the playing accuracy rate A play The calculation formula is:

[0107]

[0108] Among them, N l is the key number of the first note in this round of fingering questions, K l is the key number actually pressed for the first time, M l is the finger number corresponding to the first note of the fingering question, R l The actual number of the finger pressed for the first time.

[0109] Further, if Figure 5 The figure shows the finger recognition flow chart. Finger recognition includes keyboard registration, finger key point recognition, and finger matching. Keyboard registration uses Canny edge detection and Hough line detection combined with the physical distribution of keys to determine the coordinates of each key in the image. Finger key point recognition uses the gesture recognition model in the MediaPipe framework to identify the coordinates of the fingertips. Finger matching uses the distance between the coordinates of the fingertips and the coordinates of the pressed key to determine the finger closest to the pressed key, which is recorded as the actual finger.

[0110] S4. Calculate the reaction speed score by combining finger flexibility and reaction time, and calculate the reaction accuracy score by combining finger span and playing accuracy, and quantitatively measure cognitive ability through information fusion.

[0111] Combined with the dexterity information of the pressing finger, the reaction speed T' is recalculated react ,The corrected reaction speed takes into account the impairment of finger function and accurately reflects the hand-eye coordination ability.

[0112] Among them, the reaction rate T' react The calculation formula is:

[0113]

[0114] Among them, αj is the flexibility benchmark coefficient of the finger numbered j, and the larger the value, the more flexible the finger;

[0115] Combined with the span information of each finger, the playing accuracy of each round of testing is recalculated;

[0116] Among them, the corrected playing accuracy A′ play The calculation formula is:

[0117]

[0118] in, Indicates the number is M l Finger and number R l the span of a finger;

[0119] According to the probability statistics theorem, a mapping relationship table between reaction time and playing accuracy score is established by measuring a large number of people's data, and the reaction speed score and playing accuracy score are obtained.

[0120] Figure 6 The following figure shows the histogram of the reaction time distribution of the test subjects and a curve fit diagram, which shows the difference between the test subjects' playing times and the actual standard time. As can be seen from the figure, the majority of the subjects' reaction times were between -500ms and 500ms. A very small number of subjects had reaction times outside of -800ms to the left or outside of 800ms to the right, significantly lagging behind the others. Analysis of the reaction time distribution reveals that it peaks at around 22ms and gradually decreases towards the left and right, forming a bell-shaped distribution with a high center and low sides, similar to a normal distribution.

[0121] Based on this distribution, we initially developed a reaction speed scoring rule: 0ms was considered a speed standard of 100 points, and reaction times of 800ms to the left and -800ms to the right were both considered a speed standard of 0 points. Between the standard of 100 points and the standard of 0 points, we used a linear scoring system: within the range of -800ms to 800ms, we subtracted 1 point for every 8ms of distance, ultimately deriving a 100-point speed scoring standard.

[0122] Figure 7 This is a histogram of the test subjects' playing accuracy, showing the difference between their fingerings and the standard fingering. As can be seen, the vast majority of subjects had an accuracy rate between 0 and 6, with over 70% achieving 0. A very small number of subjects had an accuracy rate between 8 and 10. The overall distribution shows that the accuracy rate peaks at 0 and then continuously decreases to the left, forming a roughly half-bell shape, similar to a half-normal distribution.

[0123] Based on this distribution, we initially developed a scoring rule for playing accuracy, treating a playing accuracy of 0 as equivalent to a standard accuracy of 100 points, and treating a playing accuracy of 10 or greater as equivalent to a standard accuracy of 0 points. Between the standard accuracy of 100 points and the standard accuracy of 0 points, we used a linear evaluation method, subtracting 10 points for every unit of distance between 0 and 10, ultimately arriving at a percentage-based accuracy standard.

[0124] Table 3. Overall speed and accuracy scores of subjects in the tracking test

[0125]

[0126] Table 3 shows the overall speed scores and accuracy scores of the subjects in the tracking test. The optimal weight coefficient for score improvement is found through long-term tracking measurement. The reaction speed score and playing accuracy score are weighted by the weight coefficient through information fusion to obtain the hand-eye cognition score, which quantitatively measures cognitive ability.

[0127] The hand-eye cognition scores of the subjects in the tracking test with equal weights for speed and accuracy are as follows:

[0128] Table 4. Overall hand-eye cognition scores of subjects in the tracking test

[0129]

[0130] As shown in the table above, the scores of the first three tests showed a steady upward trend, and the upward trend began to slow down in the fourth and fifth tests, which better demonstrated the slow change trend of the subjects' cognitive abilities gradually improving with the increase in the number of practice.

[0131] Example 2

[0132] This embodiment continues to disclose an objective quantitative measurement method for cognitive ability that combines finger dexterity and span, such as Figure 1 The specific steps are as follows:

[0133] S1. Referring to the corresponding steps in Example 1, in particular, in the finger span test, it is stipulated that the finger with the larger number is fixed on key 53, and the finger with the smaller number is naturally and stretched to the left as much as possible, and the difference in key numbers is the corresponding finger span. When testing the span of any two fingers of the right hand, it is stipulated that the finger with the smaller number is fixed on key 37, and the finger with the larger number is naturally and stretched to the right as much as possible, and the difference in key numbers is the corresponding finger span. In this way, the span measurement of both hands is consistent and the calculation is simplified. The collected key numbers are processed to obtain the finger span S. xy .

[0134] S2. Refer to the corresponding steps in Example 1. In particular, set the number of consecutive keystrokes to 10, and calculate the dexterity F of each finger by measuring the time interval between consecutive keystrokes of each finger. j .

[0135] Figure 8 Statistical histogram and curve fitting diagram of finger dexterity in this embodiment. After the KS significance normality test, the distribution can be considered to obey the normal distribution with a mean μ of 1469ms and a standard deviation σ of 180ms.

[0136] Set μ-2.5σ, i.e. 1019ms, as the scoring point of 100 points, and set μ+2σ, i.e. 1829ms, as the scoring point of 50 points. Substitute them into the cumulative scoring formula to obtain the finger dexterity score y for number j. j Calculation formula:

[0137] y j =1.06×[5-(F j -1469) / 180] 2 +40.46,1≤j≤10

[0138] In particular, when F j When the finger dexterity score was greater than 2041ms, it was directly evaluated as 0 points.

[0139] The dexterity and flexibility scores of each finger are shown in Table 5 below:

[0140] Table 5. Statistics of finger dexterity and dexterity scores in Example 2

[0141] Finger number mean Standard deviation Average flexibility score 1 1530 171 63 2 1642 164 58 3 1510 152 65 4 1487 178 66 5 1528 172 64 6 1406 175 71 7 1322 136 76 8 1365 133 73 9 1486 157 66 10 1420 164 70

[0142] As shown in the table above, all the experimental subjects were right-handed. The flexibility scores of the fingers on their right hands were significantly higher than those on their left hands. At the same time, the index finger of each hand was the most flexible and had the best independence. The ring finger was the least flexible due to the balance between the little finger and the middle finger. The standard deviation of the little finger was the largest. The scoring results were consistent with reality and could well reflect the flexibility of each finger.

[0143] S3. Referring to the corresponding steps in Example 1, the question-setting module displays the fingering question in an animated form, outputs the finger pressing time and the corresponding key number through the acquisition module with a key scanner and a timer, takes a photo of the finger pressing moment through a camera device, obtains the actual key name through finger recognition, and calculates the reaction time T based on the time interval between the question-setting key presses and the time interval between the actual key presses. react Calculate the playing accuracy A based on the distance between the given fingering and the actual fingering play .

[0144] Table 6 is a schematic diagram of a set of left-hand fingering questions in the cognitive test. Each set of questions contains 4 rounds of questions of different difficulty levels. Rounds 1 and 2 are randomly selected from the question bank of difficulty 1, round 2 is randomly selected from the question bank of difficulty 2, and round 3 is randomly selected from the question bank of difficulty 3.

[0145] Table 6. Schematic table of a group of left-hand fingering problems in Example 2

[0146]

[0147] As shown in the table above, to ensure that each finger is adequately tested at each difficulty level, Difficulty 1 consists of two rounds of testing. To allow participants to better utilize their cognitive abilities while avoiding the influence of inertial playing, each round of the task is set to change the playing speed from slow to fast. Initially, each note is played for 1 second, and then the speed is changed twice, to 1.5 times and 2.0 times the initial speed respectively. This allows participants to familiarize themselves with the fingering while stimulating activation in the corresponding brain areas, better reflecting the boundaries of the brain's cognitive ability.

[0148] Figure 9 This is an animated diagram of a left-hand fingering task of difficulty 3. The task consists of five notes at a tempo of 60 beats per minute. The animation controls the speed of the task by moving a vertical line. The dotted horizontal line represents the playing timeline. The vertical line is initially located to the left of the first note, allowing the subject a certain amount of preparation time for playing. The timeline position of the left edge of each note indicates the standard start time of the note. When the vertical line moves to the left edge of the note, it indicates the start of playing the note.

[0149] S4. Referring to the corresponding steps in Example 1, the reaction speed score is calculated by combining finger flexibility and reaction time, and the reaction accuracy score is calculated by combining finger span and playing accuracy. In the difficulty test, cognitive ability is quantitatively measured through information fusion.

[0150] Table 7 shows the overall speed scores and accuracy scores of the subjects in the difficulty test. It is not difficult to see that there is a trade-off between speed and accuracy in the score changes in the two rounds of tests of difficulty 1. As the difficulty of playing increases, the scores show a downward trend, which basically conforms to the objective law.

[0151] Table 7. Overall speed scores and accuracy scores of subjects in the difficulty test of Example 2

[0152]

[0153]

[0154] The hand-eye cognition scores of different subjects in the difficulty test were calculated based on the optimal weight system in Example 1 and are shown in Table 8 below:

[0155] Table 8. Cognitive scores of 5 different subjects in the difficulty test in Example 2

[0156]

[0157] As shown in the table above, with the exception of Round 1, which was affected by factors such as the subjects' unfamiliarity with fingering and piano equipment, the subjects' cognitive scores gradually decreased as the difficulty of playing increased, which was generally consistent with the overall trend of change. Some subjects performed better in Round 3 than in Round 2, and their cognitive scores in Round 4 exceeded those in Round 3. This fluctuation may be affected by various factors such as psychological quality, difficulty adaptation, and concentration, but it is common in experiments and is normal. This cognitive ability assessment method can not only clearly demonstrate the changes in the hand-eye cognitive ability of the same subject at different difficulty levels, but also show the differences in cognitive ability of different subjects when performing fingering sight-reading in the same situation.

[0158] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. An objective quantitative measurement method for cognitive ability combining finger dexterity and span, characterized by: The objective quantitative measurement method of cognitive ability comprises the following steps: S1. A question-generating module with a display displays the status of two designated fingers and keys in text and image form. The piano keys are numbered from 1 to 88 from left to right according to their physical positions, which are recorded as key numbers. The acquisition module with a key scanner outputs the key numbers pressed by the fingers, and the difference between the two is used to calculate the span of each finger. S2, a question-setting module with a display displays the name of a designated finger and the number of consecutive keystrokes in text form, outputs the finger pressing time through a collection module with a timer, and calculates the dexterity of each finger by measuring the time interval between consecutive keystrokes of each finger; S3. A question-setting module with a display displays fingering questions in an animated form. The fingering refers to pressing a piano key with a certain finger. An acquisition module with a key scanner and a timer outputs the finger pressing time and the corresponding key number. A camera takes a photo of the finger pressing moment. The actual finger name of the key is obtained through finger recognition. The reaction time is calculated based on the time interval between the question-setting keys and the time interval between the actual keys. The playing accuracy is calculated based on the distance between the question-setting fingering and the actual fingering. The process is as follows: S301, setting multiple fingering difficulties according to the number of keys that appear and the time interval between keys; S302, setting multiple rounds of fingering questions according to increasing difficulty, each round of questions will be automatically generated according to the difficulty, and the question time, question key number and question finger number sequence will be obtained and displayed in the form of animation; S303: Output the key number and pressing time sequence of each round of test task through the acquisition module, and drive the camera device to take a real-time finger picture at the pressing moment; S304, aligning the pressing time sequence with the standard note playing time sequence to calculate the reaction time; Among them, the reaction time T react The calculation formula is: Among them, m is the number of notes in the fingering test, Q l Indicates the standard starting time for playing the first note of the fingering question in this round, P l Indicates the actual time the key is pressed for the first time; S305, identifying the name of the pressed finger through the finger recognition image, and calculating the playing accuracy based on the distance between the questioned fingering and the actual fingering; Among them, the playing accuracy rate A play The calculation formula is: Among them, N l is the key number of the first note in this round of fingering questions, K l is the key number actually pressed for the first time, M l is the finger number corresponding to the first note of the fingering question, R l The number of the finger that actually pressed the first time; S4. Calculate the reaction speed score by combining finger dexterity and reaction time, and calculate the reaction accuracy score by combining finger span and playing accuracy. This quantitatively measures cognitive ability through information fusion. The process is as follows: S401, recalculating the reaction speed based on the dexterity information of each finger pressed; Among them, the corrected reaction rate T' react The calculation formula is: Among them, α j is the flexibility benchmark coefficient of the finger numbered j, and the larger the value, the more flexible the finger; S402, combining the span information of each finger, and recalculating the playing accuracy of each round of testing; Among them, the corrected playing accuracy A′ play The calculation formula is: in, Indicates the number is M l Finger and number R l the span of a finger; S403. Based on the probability statistics theorem, a reaction time and playing accuracy score mapping relationship table is established by measuring a large amount of population data to obtain a reaction speed score and a playing accuracy score, wherein the reaction time and playing accuracy score mapping relationship table establishes a one-to-one mapping relationship between the reaction time and the playing accuracy score; S404. Calculate the reaction speed score and the playing accuracy score by using a certain weight coefficient in an information fusion method to obtain a hand-eye cognition score, thereby quantitatively measuring cognitive ability.

2. The objective quantitative measurement method for cognitive ability combining finger dexterity and span according to claim 1, characterized in that: The process of step S1 is as follows: S101. Define the physical positions of the fingers from left to right as 1 to 10, record them as finger numbers, set multiple rounds of finger span measurement tasks based on the finger position distribution, display test prompts and finger images on a module with a display, and measure the finger span in a relaxed state and a stressed state in each round of the test task; S102: Output the key number pressed in each round of testing through the acquisition module with key scanner, and calculate the maximum key number H max and the minimum key number H min ; S103, collect H max and H min Subtract the finger span S_relax of finger x and finger y in the relaxed state xy and the finger span S_max in the best effort state xy ; S104, according to the calculation S_relax xy and S_max xy The average value of finger x and finger y is the span S xy , where x and y are the finger numbers.

3. The objective quantitative measurement method for cognitive ability combining finger dexterity and span according to claim 1, characterized in that: The process of step S2 is as follows: S201, setting 10 rounds of dexterity test tasks to measure the dexterity of each finger individually, displaying the finger name and the number of consecutive keystrokes in text form on a module with a display, and measuring the keystroke time interval of the corresponding finger in each round of the test task; S202: Output the key number and pressing time of the finger in each round of test task through the acquisition module with key scanner, and calculate the mode G of the key number collected in this round mode , and the key number is G mode The key pressing time series T={T i ,1≤i≤n}, where T i Indicates the key number is G mode The i-th data of the key pressing time series, n is the key number G mode The length of the key pressing time sequence; S203, according to the multiple pressing of the key number G mode The finger dexterity F is calculated by the pressing time of the key j ; Among them, the dexterity F of the finger numbered j j The calculation formula is: F j = min(T k - T1, …, T n - T n-k+1 ), 1 < j < 10 Equation (1) Among them, k is the number of consecutive keystrokes, T k 、T n-k+1 、T n Key number K mode The kth, n-k+1th, and nth data of the key pressing time series.

4. The objective quantitative measurement method for cognitive ability combining finger dexterity and span according to claim 1, characterized in that: The question-generating module includes a display, a web system, a camera, a data transmission interface, and a storage medium. The display is used to display the question and prompt information; the web system is used to control the setting of the question and read the analysis data from the serial port, while controlling the camera to shoot and store the test data in the storage medium; the camera is used to capture the image of the finger pressing the key, and can fully capture all the keys; the data transmission interface is used to connect to the external acquisition module to receive key information and key moment data in real time.

5. The objective quantitative measurement method for cognitive ability combining finger dexterity and span according to claim 1, characterized in that: The acquisition module includes a key sensor, a scanning control circuit, and a data transmission serial port. The key sensor includes two high and low contacts for judging the status of each key; the scanning control circuit uses the matrix scanning principle to obtain the key position that triggers the contact signal, and obtains the key number based on the position; the data transmission serial port is used to connect to the question-generating module and transmit the packaged information of the key number and the pressing time in real time.

6. The objective quantitative measurement method for cognitive ability combining finger dexterity and span according to claim 1, characterized in that: The finger recognition process in step S305 includes keyboard registration, finger key point recognition, and finger matching. Among them, keyboard registration is to detect the position coordinates of each key in the image through Canny edge detection and Hough line detection combined with the physical distribution of the keys; finger key point recognition is to identify the coordinates of the fingertip; finger matching is to obtain the finger closest to the pressed key based on the distance between the coordinates of the fingertip and the coordinates of the pressed key, and record it as the actual key finger.

7. The objective quantitative measurement method for cognitive ability combining finger dexterity and span according to claim 1, characterized in that: The cognitive ability refers to the ability of the human brain to process, store and retrieve information. By measuring the reaction time to the specified fingering and the playing accuracy, the reaction speed score and finger ability score are calculated in combination with the finger flexibility and span, and the hand-eye coordination ability and spatial positioning ability are evaluated.

Images