Method and system for improving students' concentration in answering questions through their gestures when doing questions

Abstract

The present invention provides a method and system for improving students' concentration in answering questions through their gestures when answering questions. The method takes students' action of answering questions during the question-answering process as the shooting object to determine the students' corresponding hand activity data, and further analyzes the hand activity data to judge whether the students have stopped answering questions during the question-answering process and the duration corresponding to the stopped answering state. Finally, the students' concentration in answering questions is determined and adaptive reminders are given to the students. The method utilizes the fact that when students are not focused on answering questions, their gestures when answering questions will be abnormal. By analyzing the students' gestures when answering questions, the method can effectively urge students to answer questions efficiently and attentively, and improve the intelligence and automation level of monitoring the students' question-answering process.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent question answering management, and in particular to a method and system for improving students' concentration in answering questions through their gestures when answering questions. Background Art

[0002] Currently, after completing classroom learning, students need to review and consolidate the knowledge learned in class by doing exercises. Generally speaking, students do exercises independently, but during the exercise process, students cannot always maintain a high level of concentration, and they are inevitably distracted. Although existing technologies have used video surveillance to monitor students' exercise status in real time, this video surveillance method is limited to monitoring students' actual exercise time and whether they are exercising in the designated location. It cannot analyze students' concentration during the exercise process, which is not conducive to effectively urging students to do exercises efficiently and attentively, and it also cannot improve the intelligence and automation level of monitoring students' exercise process. Summary of the Invention

[0003] In response to the defects of the existing technology, the present invention provides a method and system for improving students' concentration in answering questions through their gestures when answering questions. The method captures the students' action images during the problem-solving process and analyzes the action images to determine the students' hand movement data during the problem-solving process. Then, based on the hand movement data, the students' gesture information during the problem-solving process is determined. Then, based on the gesture information, it is determined whether the students have stopped answering questions and the duration of the stopped answering state during the problem-solving process. Finally, based on the judgment result of the stopped answering state, the students' concentration in answering questions during the problem-solving process is determined. Then, based on the concentration level, the students are evaluated at different time periods during the problem-solving process. Remind; It can be seen that the method and system for improving students' concentration in answering questions through their gestures when answering questions takes students' action in answering questions as the shooting object, thereby determining the students' corresponding hand activity data, and further analyzing the hand activity data to judge whether the students have stopped answering questions in the process of answering questions and the duration corresponding to the said stopped answering state, and finally determining the students' concentration in answering questions and giving adaptive reminders to the students. It utilizes the fact that when students are not focused on answering questions, their gestures in answering questions will have corresponding abnormalities. By analyzing students' gestures in answering questions, it can effectively urge students to answer questions efficiently and attentively, and improve the intelligence and automation level of monitoring students' answering process.

[0004] The present invention provides a method for improving students' concentration in answering questions by using their gestures when answering questions, which is characterized by comprising the following steps:

[0005] Step S1, capturing an image of a student's action during a problem-solving process, and analyzing the image to determine hand movement data of the student during the problem-solving process;

[0006] Step S2: determining the student's gesture information during the problem-solving process based on the hand movement data; and then determining whether the student has stopped solving the problem and the duration of the stopped problem-solving state based on the gesture information.

[0007] Step S3, determining the student's concentration level during the problem-solving process based on the judgment result of the problem-solving stop state; and then reminding the student at different times during the problem-solving process based on the concentration level;

[0008] Furthermore, in step S1, the image of the student's action during the problem-solving process is captured, and the image of the action is analyzed to determine the hand movement data of the student during the problem-solving process, which specifically includes:

[0009] Step S101, taking binocular photos of the student's hands and the surrounding area during the student's problem-solving process, thereby obtaining a binocular image of the student's problem-solving action during the problem-solving process;

[0010] Step S102, calculating the binocular image parallax of the binocular image of the problem-solving action, and then generating a three-dimensional image of the student in the problem-solving process according to the binocular image parallax;

[0011] Step S103: extracting a plurality of image frames from the three-dimensional image of the student at predetermined time intervals, and identifying the hand movement position and hand movement amplitude of the student in each image frame as the hand movement data;

[0012] Furthermore, in step S2, determining the student's gesture information during the problem-solving process based on the hand movement data; and then judging whether the student has stopped solving the problem and the duration of the stopped problem-solving state based on the gesture information specifically includes:

[0013] Step S201: determining, based on the hand movement posture included in the hand movement data, first gesture information corresponding to the student's hand holding the pen to solve the problem; and determining, based on the hand movement amplitude included in the hand movement data, second gesture information corresponding to the relative position between the student's hand and the answer sheet during the problem solving process.

[0014] Step S202: Analyze the first gesture information to determine whether the student's actual hand gesture of holding the pen during the test-solving process is a preset pen-holding gesture. If not, determine that the student is currently in a stopped state. Simultaneously, determine the duration of the student's actual hand gesture of holding the pen during the test-solving process that is not a preset pen-holding gesture, and use this duration as the duration corresponding to the stopped state.

[0015] Step S203: Analyze the second gesture information to determine whether the student's hand has left the plane coverage area corresponding to the answer book itself during the test-taking process. If so, determine that the student is currently in a stopped state. At the same time, determine the duration of the student's hand leaving the plane coverage area corresponding to the answer book itself during the test-taking process, and use this as the duration corresponding to the stopped state.

[0016] Furthermore, in step S202, analyzing the first gesture information to determine whether the actual gesture of the student holding the pen during the test is the preset pen-holding gesture specifically includes:

[0017] In step S2021, the coordinate position of each fingertip, the coordinate position of the palm center, and the normal vector of the palm surface are collected using the hand posture sensor as the first gesture information, and the angle value between each fingertip and the normal vector of the palm surface is determined using the following formula (1):

[0018]

[0019] In the above formula (1), α a represents the angle between the tip of the ath finger and the normal vector of the palm, (x a ,y a ,z a ) represents the coordinate value of the fingertip of the ath finger, (x0, y0, z0) represents the coordinate value of the palm center, and (x, y, z) represents the normal vector of the palm surface;

[0020] In step S2022, the following formula (2) is used to determine whether the student's actual pen-holding gesture during the test is the preset pen-holding gesture based on the angle between the fingertip of each finger and the normal vector of the palm:

[0021]

[0022] In the above formula (2), P represents the judgment value of whether the actual gesture of the student holding the pen during the problem-solving process corresponds to the preset pen-holding gesture, β a,min Indicates the minimum preset angle between the tip of the ath finger and the normal vector of the palm in the preset pen-holding gesture, β a,maxrepresents the maximum preset angle between the fingertip of the ath finger and the normal vector of the palm in the preset pen-holding gesture of the student, n represents the number of fingers in contact with the pen when the student holds the pen, u() represents a step function, when the value in the brackets is greater than or equal to 0, the step function takes the value of 1, when the value in the brackets is less than 0, the step function takes the value of 0, δ{} represents a unit impulse function, when the value in the brackets is equal to 0, the unit impulse function takes the value of 1, when the value in the brackets is not equal to 0;

[0023] If P=0, it is determined that the actual hand gesture of the student holding the pen during the problem-solving process does not belong to the preset pen-holding gesture, and the student is determined to be in the stopped problem-solving state. At the same time, the duration of the actual hand gesture of the student holding the pen during the problem-solving process that does not belong to the preset pen-holding gesture is determined, and this is used as the duration corresponding to the student being in the stopped problem-solving state;

[0024] If P=1, it is determined that the actual pen-holding gesture of the student during the test is a preset pen-holding gesture. Then, in step S2023, it is determined whether the student's pen-holding posture is standard. If not, the smart pen held by the student is controlled to vibrate to remind the student that the current pen-holding posture is not standard.

[0025] In step S2023, if it is determined that the actual pen-holding gesture of the student during the test-taking process does not conform to the preset pen-holding gesture, the smart pen held by the student is controlled to vibrate to remind the student that the current pen-holding posture is not standard. The vibration frequency of the smart pen held by the student is determined using the following formula (3):

[0026]

[0027] In the above formula (3), f represents the vibration frequency of the smart pen, f max Indicates the maximum vibration frequency that the smart pen can reach when vibrating. Indicates taking the maximum value of the values ​​in the brackets;

[0028] Furthermore, in step S3, the student's concentration level during the problem-solving process is determined based on the result of the judgment of the stopped problem-solving state; and then, based on the concentration level, the student is reminded at different times during the problem-solving process, specifically including:

[0029] Step S301: summing the durations corresponding to any two chronologically adjacent stop states to obtain a corresponding total duration; then comparing the total duration with a preset time threshold; if the total duration exceeds the preset time threshold, determining that the student was in a state of not concentrating on solving problems during the time periods corresponding to the two stop states; otherwise, determining that the student was in a state of concentrating on solving problems during the time periods corresponding to the two stop states;

[0030] Step S302: Play a voice reminder to the student during the period when the student is not concentrating on the test.

[0031] The present invention also provides a system for improving students' concentration in answering questions by using their gestures when answering questions, characterized in that it includes a module for capturing and analyzing the action of answering questions, a module for determining the gesture information of answering questions, a module for determining the stopped state of the work, and a reminder module; wherein,

[0032] The question-solving action shooting and analysis module is used to shoot the student's question-solving action images during the question-solving process and analyze the question-solving action images to determine the student's hand movement data during the question-solving process;

[0033] The question-solving gesture information determination module is used to determine the question-solving gesture information of the student during the question-solving process based on the hand movement data;

[0034] The stopped homework state determination module is used to determine whether the student has stopped doing the homework state and the duration of the stopped doing the homework state according to the question-solving gesture information;

[0035] The reminder module is used to determine the student's concentration level during the problem-solving process based on the judgment result of the stopped problem-solving state; and then remind the student at different times during the problem-solving process based on the concentration level;

[0036] Furthermore, the question-solving action shooting and analysis module shoots the student's question-solving action images during the question-solving process and analyzes the question-solving action images to determine the student's hand movement data during the question-solving process, specifically including:

[0037] Taking binocular photos of the student's hands and the surrounding area while the student is doing the test, thereby obtaining binocular images of the student's action during the test;

[0038] Then, the binocular image parallax of the binocular image of the problem-solving action is calculated, and then a three-dimensional image of the student in the problem-solving process is generated according to the binocular image parallax;

[0039] Finally, according to a preset time interval, a plurality of image frames are sequentially extracted from the three-dimensional image of the student, and the hand movement posture and hand movement amplitude of the student in each image frame are identified to serve as the hand activity data;

[0040] Furthermore, the question-solving gesture information determination module determines the question-solving gesture information of the student during the question-solving process based on the hand movement data, specifically including:

[0041] Determine, based on the hand movement posture included in the hand movement data, first gesture information corresponding to the student's hand holding the pen to solve the problem; and determine, based on the hand movement amplitude included in the hand movement data, second gesture information corresponding to the relative position between the student's hand and the answer book during the problem solving process;

[0042] as well as,

[0043] The stopped homework state determination module determines whether the student is in a stopped homework state and the duration of the stopped homework state according to the question-solving gesture information, specifically including:

[0044] Analyzing the first gesture information to determine whether the student's actual hand gesture of holding the pen during the problem-solving process is a preset pen-holding gesture; if not, determining that the student is currently in a stopped problem-solving state; and simultaneously determining a duration during which the student's actual hand gesture of holding the pen during the problem-solving process is not a preset pen-holding gesture, and using this duration as the duration corresponding to the stopped problem-solving state;

[0045] Analyzing the second gesture information to determine whether the student's hand has left the plane coverage area corresponding to the answer book itself during the test-taking process; if so, determining that the student is currently in a stopped test-taking state; and determining a duration of time that the student's hand has left the plane coverage area corresponding to the answer book itself during the test-taking process, which serves as the duration corresponding to the stopped test-taking state;

[0046] Furthermore, the reminder module determines the student's concentration level during the problem-solving process based on the judgment result of the stopped problem-solving state; and then, based on the concentration level, reminds the student at different times during the problem-solving process, specifically including:

[0047] The durations corresponding to any two chronologically adjacent stop-work states are summed to obtain a corresponding total duration; the total duration is then compared with a preset time threshold; if the total duration exceeds the preset time threshold, it is determined that the student is in a state of not concentrating on the work during the period corresponding to the two stop-work states as a whole; otherwise, it is determined that the student is in a state of concentrating on the work during the period corresponding to the two stop-work states as a whole;

[0048] Finally, during the period when the student is not concentrating on the test, a voice reminder is played to the student.

[0049] Compared with the existing technology, the method and system for improving students' concentration in answering questions through their gestures when answering questions captures images of students' action during the question-answering process and analyzes the images to determine the students' hand movement data during the question-answering process. Then, based on the hand movement data, the system determines the students' gesture information during the question-answering process. Then, based on the gesture information, the system determines whether the students have stopped answering questions and the duration of the stopped answering state. Finally, based on the determination result of the stopped answering state, the system determines the students' concentration during the question-answering process. Then, based on the students' concentration, the system reminds the students at different times during the question-answering process. It can be seen that the method and system for improving students' concentration in answering questions through their gestures when answering questions takes the students' action of answering questions during the question-answering process as the shooting object, thereby determining the students' corresponding hand activity data, and further analyzing the hand activity data to determine whether the students have stopped answering questions during the question-answering process and the duration corresponding to the stopped answering state, and finally determining the students' concentration in answering questions and giving adaptive reminders to the students. It utilizes the fact that when students are not focused on answering questions, their gestures when answering questions will be abnormal. By analyzing the students' gestures when answering questions, it can effectively urge students to answer questions efficiently and attentively, and improve the intelligence and automation level of monitoring the students' question-answering process.

[0050] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0051] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A flow chart of a method provided by the present invention for improving students' concentration in answering questions by using their gestures when answering questions.

[0054] Figure 2 This is a structural diagram of the system provided by the present invention for improving students' concentration in answering questions through their gestures when answering questions. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0056] See Figure 1 , is a flow chart of a method for improving students' concentration in answering questions by using their gestures when answering questions, provided by an embodiment of the present invention. The method for judging students' answering status by gestures includes the following steps:

[0057] Step S1, capturing an image of a student's action during the problem-solving process, and analyzing the image to determine hand movement data of the student during the problem-solving process;

[0058] Step S2: determining the student's gesture information during the problem-solving process based on the hand movement data; and then determining whether the student has a pause state during the problem-solving process and the duration of the pause state based on the gesture information;

[0059] Step S3, according to the judgment result of the stopped state, determine the concentration level of the student in the process of doing the questions; then, according to the concentration level, remind the student at different time periods in the process of doing the questions.

[0060] The beneficial effects of the above technical solution are: the method of improving students' concentration in answering questions through their gestures when answering questions takes the students' action of answering questions during the question-solving process as the shooting object, thereby determining the students' corresponding hand activity data, and further analyzing the hand activity data to judge whether the students have stopped answering questions during the question-solving process and the duration corresponding to the stopped state, and finally determining the students' concentration in answering questions and giving adaptive reminders to the students. It utilizes the fact that when students are not focused on answering questions, their gestures when answering questions will have corresponding abnormalities. By analyzing the students' gestures when answering questions, it can effectively urge students to answer questions efficiently and attentively, and improve the intelligence and automation level of monitoring the students' question-answering process.

[0061] Preferably, in step S1, capturing an image of the student's action during the problem-solving process, and analyzing the image of the action to determine the student's hand movement data during the problem-solving process specifically includes:

[0062] Step S101, taking binocular photos of the student's hands and the surrounding area while the student is doing the test, thereby obtaining a binocular image of the student's action during the test;

[0063] Step S102, calculating the binocular image parallax of the binocular image of the student doing the exercise, and then generating a three-dimensional image of the student doing the exercise according to the binocular image parallax;

[0064] In step S103, a plurality of image frames are sequentially extracted from the three-dimensional image of the student doing the test at preset time intervals, and the hand movement posture and hand movement amplitude of the student in each image frame are identified as the hand activity data.

[0065] The beneficial effects of the above technical solution are: when students do questions in answer books in the form of exercise books, they need to use their hands to hold the pen to write answers. When students are distracted or not concentrating on doing questions, their hands cannot always maintain a standard posture to hold the pen (for example, spinning the pen or tapping the table with the pen, etc.), or the relative distance between the student's hand and the answer book cannot always be maintained within the preset range (for example, the hand directly touches the answer book or the distance between the hand and the answer book is too far to write). By collecting the student's corresponding question-doing action image through binocular shooting and performing further image parallax conversion processing to obtain a three-dimensional question-doing image, the accuracy and reliability of identifying the student's hand movement posture and hand movement amplitude during the question-doing process can be greatly improved.

[0066] Preferably, in step S2, determining the student's gesture information during the problem-solving process based on the hand movement data; and then judging whether the student has stopped solving the problem and the duration of the stopped problem-solving state based on the gesture information specifically includes:

[0067] Step S201: Determine, based on the hand movement posture included in the hand movement data, first gesture information corresponding to the student's hand holding the pen to solve the problem; and determine, based on the hand movement amplitude included in the hand movement data, second gesture information corresponding to the relative position between the student's hand and the answer sheet during the problem solving process.

[0068] Step S202: Analyze the first gesture information to determine whether the student's actual pen-holding gesture during the problem-solving process corresponds to a preset pen-holding gesture. If not, determine that the student is currently in a stopped problem-solving state. Also, determine the duration of the time during which the student's actual pen-holding gesture during the problem-solving process does not correspond to the preset pen-holding gesture, and use this duration as the duration corresponding to the stopped problem-solving state.

[0069] Step S203: Analyze the second gesture information to determine whether the student's hand has left the plane coverage area corresponding to the answer book itself during the test-taking process. If so, determine that the student is currently in a state of stopping test-taking. At the same time, determine the duration of the student's hand leaving the plane coverage area corresponding to the answer book itself during the test-taking process, and use this as the duration corresponding to the stop test-taking state.

[0070] The beneficial effect of the above technical solution is: when the student's hand movement posture is not standard during the test process or the hand leaves the plane coverage of the answer book itself, the student will not be able to write the test normally on the answer book. The first gesture information is used to represent the posture of the student's hand holding the pen, and the second gesture information is used to represent the relative position relationship between the student's hand and the answer book. The first gesture information and the second gesture information are both important influencing factors for the student to be able to write the test normally and attentively on the answer book. Specifically, when the actual gesture of the student's hand holding the pen is inconsistent with the preset pen-holding gestures (the preset pen-holding gestures can be obtained by counting the commonly used pen-holding gestures of different people and performing six-degree-of-freedom analysis. This is a conventional method in this field and will not be repeated here), it indicates that the student is not currently writing to answer questions. Similarly, when the student's hand is away from the plane coverage range corresponding to the answer book itself, it indicates that the student's current relative position to the answer book cannot meet the requirements for writing on the answer book. The above two situations both indicate that the student is in a state of stopping answering questions. Through further analysis and processing, it is possible to accurately and quickly determine whether the student is in a state of stopping answering questions and how long the state of stopping answering questions lasts during the process of answering questions.

[0071] Preferably, in step S202, analyzing the first gesture information to determine whether the actual gesture of the student holding the pen during the test is the preset pen-holding gesture specifically includes:

[0072] In step S2021, the coordinate position of each fingertip, the coordinate position of the palm center, and the normal vector of the palm surface are collected using the hand posture sensor as the first gesture information, and the angle value between each fingertip and the normal vector of the palm surface is determined using the following formula (1):

[0073]

[0074] In the above formula (1), α a represents the angle between the tip of the ath finger and the normal vector of the palm, (x a ,y a ,z a ) represents the coordinate value of the fingertip of the ath finger, (x0, y0, z0) represents the coordinate value of the palm center, and (x, y, z) represents the normal vector of the palm surface;

[0075] In step S2022, the following formula (2) is used to determine whether the student's actual pen-holding gesture during the test is the preset pen-holding gesture based on the angle between the fingertip of each finger and the normal vector of the palm:

[0076]

[0077] In the above formula (2), P represents the judgment value of whether the actual gesture of the student holding the pen during the problem-solving process corresponds to the preset pen-holding gesture, β a,min Indicates the minimum preset angle between the tip of the ath finger and the normal vector of the palm in the preset pen-holding gesture, β a,max represents the maximum preset angle between the fingertip of the ath finger and the normal vector of the palm in the preset pen-holding gesture of the student, n represents the number of fingers in contact with the pen when the student holds the pen, u() represents a step function, when the value in the brackets is greater than or equal to 0, the step function takes the value of 1, when the value in the brackets is less than 0, the step function takes the value of 0, δ{} represents a unit impulse function, when the value in the brackets is equal to 0, the unit impulse function takes the value of 1, when the value in the brackets is not equal to 0;

[0078] If P=0, it is determined that the actual hand gesture of the student holding the pen during the problem-solving process does not belong to the preset pen-holding gesture, and the student is determined to be in the stopped problem-solving state. At the same time, the duration of the actual hand gesture of the student holding the pen during the problem-solving process that does not belong to the preset pen-holding gesture is determined, and this is used as the duration corresponding to the student being in the stopped problem-solving state;

[0079] If P=1, it is determined that the actual pen-holding gesture of the student during the test is a preset pen-holding gesture. Then, in step S2023, it is determined whether the student's pen-holding posture is standard. If not, the smart pen held by the student is controlled to vibrate to remind the student that the current pen-holding posture is not standard.

[0080] In step S2023, if it is determined that the actual pen-holding gesture of the student during the test-taking process does not conform to the preset pen-holding gesture, the smart pen held by the student is controlled to vibrate to remind the student that the current pen-holding posture is not standard. The vibration frequency of the smart pen held by the student is determined using the following formula (3):

[0081]

[0082] In the above formula (3), f represents the vibration frequency of the smart pen, f max Indicates the maximum vibration frequency that the smart pen can reach when vibrating. Indicates that the maximum value of the values ​​in the brackets is calculated.

[0083] The beneficial effect of the above technical solution is: through the above formula and steps, it is determined whether the actual gesture of the student's hand holding the pen during the problem-solving process is the preset pen-holding gesture. If it is determined that the actual gesture of the student's hand holding the pen during the problem-solving process is not the preset pen-holding gesture, it is determined that the student is currently in a state of stopping problem-solving. At the same time, the duration of the student's actual hand holding the pen during the problem-solving process is not the preset pen-holding gesture, and this is used as the duration corresponding to the stop problem-solving state; if it is determined that the actual gesture of the student's hand holding the pen during the problem-solving process is the preset pen-holding gesture, it is determined that the student's hand holding the pen is not standard, and the smart pen is controlled to vibrate accordingly to remind the student that the current pen holding is not standard.

[0084] Preferably, in step S3, the student's concentration level during the problem-solving process is determined based on the result of the judgment of the stopped problem-solving state; and then, based on the concentration level, the student is reminded at different times during the problem-solving process, specifically including:

[0085] Step S301: summing the durations corresponding to any two chronologically adjacent stop states to obtain a corresponding total duration; then comparing the total duration with a preset time threshold; if the total duration exceeds the preset time threshold, determining that the student was in a state of not concentrating on the problem solving during the period corresponding to the two stop states; otherwise, determining that the student was in a state of concentrating on the problem solving during the period corresponding to the two stop states;

[0086] Step S302: Play a voice reminder to the student during the period when the student is not concentrating on the test.

[0087] The beneficial effect of the above technical solution is: the total duration obtained by summing up the durations corresponding to any two adjacent stop-questioning states in chronological order is used as the basis for judging whether the student is focused on doing the questions. This can effectively avoid the misjudgment caused by the student being unable to do the questions normally due to other objective factors. This can accurately judge whether the student is in a focused state on doing the questions without the need for frequent time threshold comparisons for each stop-questioning state, thereby greatly reducing the workload of time threshold comparisons.

[0088] See Figure 2 , is a schematic diagram of the structure of a system for improving students' concentration in answering questions through their gestures, provided by an embodiment of the present invention. The system for judging students' answering status through gestures includes a question-taking action capture and analysis module, a question-taking gesture information determination module, a work stop status determination module, and a reminder module; wherein,

[0089] The question-solving action shooting and analysis module is used to shoot the student's question-solving action images during the question-solving process and analyze the question-solving action images to determine the student's hand movement data during the question-solving process;

[0090] The question-solving gesture information determination module is used to determine the question-solving gesture information of the student during the question-solving process based on the hand movement data;

[0091] The stop-work state determination module is used to determine whether the student has stopped working on the problem and the duration of the stopped working on the problem according to the problem-solving gesture information;

[0092] The reminder module is used to determine the student's concentration level during the problem-solving process based on the judgment result of the stopped problem-solving state; and then remind the student at different times during the problem-solving process based on the concentration level.

[0093] The beneficial effects of the above technical solution are: the system for improving students' concentration in answering questions through their gestures when answering questions takes the students' action of answering questions during the question-solving process as the shooting object, thereby determining the students' corresponding hand activity data, and further analyzing the hand activity data to determine whether the students have stopped answering questions during the question-solving process and the duration corresponding to the stopped state, and finally determining the students' concentration in answering questions and giving adaptive reminders to the students. It utilizes the fact that when students are not focused on answering questions, their gestures when answering questions will be abnormal. By analyzing the students' gestures when answering questions, it can effectively urge students to answer questions efficiently and attentively, and improve the intelligence and automation level of monitoring the students' question-answering process.

[0094] Preferably, the question-solving action shooting and analysis module shoots the student's question-solving action image during the question-solving process and analyzes the question-solving action image to determine the student's hand movement data during the question-solving process, specifically including:

[0095] Taking binocular photos of the student's hands and the surrounding area while the student is doing the test, thereby obtaining binocular images of the student's action during the test;

[0096] Then, the binocular image parallax of the binocular image of the problem-solving action is calculated, and then a three-dimensional image of the student in the problem-solving process is generated based on the binocular image parallax;

[0097] Finally, according to the preset time interval, several image frames are extracted from the three-dimensional test-taking image in turn, and the hand movement posture and hand movement amplitude of the student in each image frame are identified as the hand activity data.

[0098] The beneficial effects of the above technical solution are: when students do questions in answer books in the form of exercise books, they need to use their hands to hold the pen to write answers. When students are distracted or not concentrating on doing questions, their hands cannot always maintain a standard posture to hold the pen (for example, spinning the pen or tapping the table with the pen, etc.), or the relative distance between the student's hand and the answer book cannot always be maintained within the preset range (for example, the hand directly touches the answer book or the distance between the hand and the answer book is too far to write). By collecting the student's corresponding question-doing action image through binocular shooting and performing further image parallax conversion processing to obtain a three-dimensional question-doing image, the accuracy and reliability of identifying the student's hand movement posture and hand movement amplitude during the question-doing process can be greatly improved.

[0099] Preferably, the gesture information determination module determines the gesture information of the student during the question-solving process based on the hand movement data, specifically including:

[0100] Determine, based on the hand movement posture included in the hand movement data, first gesture information corresponding to the student's hand holding the pen to solve the problem; and determine, based on the hand movement amplitude included in the hand movement data, second gesture information corresponding to the relative position between the student's hand and the answer sheet during the problem solving process;

[0101] as well as,

[0102] The stopped homework state determination module determines whether the student is in a stopped homework state and the duration of the stopped homework state according to the question-solving gesture information. Specifically, the module determines:

[0103] Analyze the first gesture information to determine whether the student's actual hand gesture of holding the pen during the problem-solving process is a preset pen-holding gesture; if not, determine that the student is currently in a stopped problem-solving state; and simultaneously determine the duration of the student's actual hand gesture of holding the pen during the problem-solving process that is not a preset pen-holding gesture, and use this duration as the duration corresponding to the stopped problem-solving state;

[0104] Analyze the second gesture information to determine whether the student's hand has left the plane coverage area corresponding to the answer book itself during the process of answering the questions. If so, determine that the student is currently in a state of stopping answering the questions. At the same time, determine the duration of the student's hand leaving the plane coverage area corresponding to the answer book itself during the process of answering the questions, and use this as the duration corresponding to the stop answering the questions state.

[0105] The beneficial effect of the above technical solution is: when the student's hand movement posture is not standard during the test process or the hand leaves the plane coverage of the answer book itself, the student will not be able to write the test normally on the answer book. The first gesture information is used to represent the posture of the student's hand holding the pen, and the second gesture information is used to represent the relative position relationship between the student's hand and the answer book. The first gesture information and the second gesture information are both important influencing factors for the student to be able to write the test normally and attentively on the answer book. Specifically, when the actual gesture of the student's hand holding the pen is inconsistent with the preset pen-holding gestures (the preset pen-holding gestures can be obtained by counting the commonly used pen-holding gestures of different people and performing six-degree-of-freedom analysis. This is a conventional method in this field and will not be repeated here), it indicates that the student is not currently writing to answer questions. Similarly, when the student's hand is away from the plane coverage range corresponding to the answer book itself, it indicates that the student's current relative position to the answer book cannot meet the requirements for writing on the answer book. The above two situations both indicate that the student is in a state of stopping answering questions. Through further analysis and processing, it is possible to accurately and quickly determine whether the student is in a state of stopping answering questions and how long the state of stopping answering questions lasts during the process of answering questions.

[0106] Preferably, the reminder module determines the student's concentration level during the problem-solving process based on the result of the judgment of the stopped problem-solving state; and then, based on the concentration level, reminds the student at different times during the problem-solving process, specifically including:

[0107] The durations corresponding to any two chronologically adjacent stop-work states are summed to obtain a corresponding total duration; the total duration is then compared with a preset time threshold; if the total duration exceeds the preset time threshold, it is determined that the student is in a state of not concentrating on the work during the period corresponding to the two stop-work states as a whole; otherwise, it is determined that the student is in a state of concentrating on the work during the period corresponding to the two stop-work states as a whole;

[0108] Finally, during the period when the student is not concentrating on the test, a voice reminder will be played to the student.

[0109] The beneficial effect of the above technical solution is: the total duration obtained by summing up the durations corresponding to any two adjacent stop-questioning states in chronological order is used as the basis for judging whether the student is focused on doing the questions. This can effectively avoid the misjudgment caused by the student being unable to do the questions normally due to other objective factors. This can accurately judge whether the student is in a focused state on doing the questions without the need for frequent time threshold comparisons for each stop-questioning state, thereby greatly reducing the workload of time threshold comparisons.

[0110] From the contents of the above embodiments, it can be seen that the method and system for improving students' concentration in answering questions through their gestures when answering questions captures images of students' action in the process of answering questions and analyzes the images of their action in answering questions, thereby determining the students' hand activity data in the process of answering questions, and then determining the students' gesture information in the process of answering questions based on the hand activity data; then, based on the gesture information, determining whether the students have stopped answering questions and the duration corresponding to the stopped answering questions state in the process of answering questions, and finally, based on the judgment result of the stopped answering questions state, determining the students' concentration in answering questions in the process of answering questions; and then, based on the concentration level, conducting assessments on students at different time periods in the process of answering questions. Reminder; It can be seen that the method and system for improving students' concentration in answering questions through their gestures in answering questions takes the students' action in answering questions as the shooting object, thereby determining the students' corresponding hand activity data, and further analyzing the hand activity data to judge whether the students have stopped answering questions in the process of answering questions and the duration corresponding to the stopped state, and finally determining the students' concentration in answering questions and giving adaptive reminders to the students. It utilizes the fact that when students are not focused on answering questions, their gestures in answering questions will have corresponding abnormalities. By analyzing the students' gestures in answering questions, it can effectively urge students to answer questions efficiently and attentively, and improve the intelligence and automation level of monitoring the students' answering process.

[0111] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for improving students' concentration in answering questions by using their gestures when doing questions, characterized in that: It includes the following steps: Step S1, capturing an image of a student's action during a problem-solving process, and analyzing the image to determine hand movement data of the student during the problem-solving process; Step S2: determining the student's gesture information during the problem-solving process based on the hand movement data; and then determining whether the student has stopped solving the problem and the duration of the stopped problem-solving state based on the gesture information. Step S3, determining the student's concentration level during the problem-solving process based on the judgment result of the problem-solving stop state; Then, according to the concentration level of the student in doing the test, the student is reminded at different times during the test; Wherein, the step S2 specifically includes: Step S201: determining, based on the hand movement posture included in the hand movement data, first gesture information corresponding to the student's hand holding the pen to solve the problem; and determining, based on the hand movement amplitude included in the hand movement data, second gesture information corresponding to the relative position between the student's hand and the answer sheet during the problem solving process. Step S202: Analyze the first gesture information to determine whether the student's actual hand gesture of holding the pen during the test-solving process is a preset pen-holding gesture. If not, determine that the student is currently in a stopped state. Simultaneously, determine the duration of the student's actual hand gesture of holding the pen during the test-solving process that is not a preset pen-holding gesture, and use this duration as the duration corresponding to the stopped state. Step S203: Analyze the second gesture information to determine whether the student's hand has left the plane coverage area corresponding to the answer book itself during the test-taking process. If so, determine that the student is currently in a stopped state. At the same time, determine the duration of the student's hand leaving the plane coverage area corresponding to the answer book itself during the test-taking process, and use this as the duration corresponding to the stopped state. The step S202 specifically includes: In step S2021, the coordinate position of each fingertip, the coordinate position of the palm center, and the normal vector of the palm surface are collected using the hand posture sensor as the first gesture information, and the angle value between each fingertip and the normal vector of the palm surface is determined using the following formula (1): In the above formula (1), α a represents the angle between the tip of the ath finger and the normal vector of the palm, (x a ,y a ,z a ) represents the coordinate value of the fingertip of the ath finger, (x0, y0, z0) represents the coordinate value of the palm center, and (x, y, z) represents the normal vector of the palm surface.

2. The method for improving students' concentration in answering questions by using their hand gestures when answering questions as claimed in claim 1, characterized in that: In step S1, the image of the student's action during the problem-solving process is captured, and the image of the action is analyzed to determine the hand movement data of the student during the problem-solving process, which specifically includes: Step S101, taking binocular photos of the student's hands and the surrounding area during the student's problem-solving process, thereby obtaining a binocular image of the student's problem-solving action during the problem-solving process; Step S102, calculating the binocular image parallax of the binocular image of the problem-solving action, and then generating a three-dimensional image of the student in the problem-solving process according to the binocular image parallax; In step S103, a plurality of image frames are sequentially extracted from the three-dimensional image of the test-taking according to a preset time interval, and the hand movement posture and hand movement amplitude of the student in each image frame are identified as the hand activity data.

3. The method of improving students' concentration in answering questions by using their hand gestures as claimed in claim 1, characterized in that: After step S2021, the following steps are also included: In step S2022, the following formula (2) is used to determine whether the student's actual pen-holding gesture during the test is the preset pen-holding gesture based on the angle between the fingertip of each finger and the normal vector of the palm: In the above formula (2), P represents the judgment value of whether the actual gesture of the student holding the pen during the problem-solving process corresponds to the preset pen-holding gesture, β a,min Indicates the minimum preset angle between the tip of the ath finger and the normal vector of the palm in the preset pen-holding gesture, β a,max represents the maximum preset angle between the fingertip of the ath finger and the normal vector of the palm in the preset pen-holding gesture of the student, n represents the number of fingers in contact with the pen when the student holds the pen, u() represents a step function, when the value in the brackets is greater than or equal to 0, the step function takes the value of 1, when the value in the brackets is less than 0, the step function takes the value of 0, δ{} represents a unit impulse function, when the value in the brackets is equal to 0, the unit impulse function takes the value of 1, when the value in the brackets is not equal to 0; If P=0, it is determined that the actual hand gesture of the student holding the pen during the problem-solving process does not belong to the preset pen-holding gesture, and the student is determined to be in the stopped problem-solving state. At the same time, the duration of the actual hand gesture of the student holding the pen during the problem-solving process that does not belong to the preset pen-holding gesture is determined, and this is used as the duration corresponding to the student being in the stopped problem-solving state; If P=1, it is determined that the actual pen-holding gesture of the student during the test is a preset pen-holding gesture. Then, in step S2023, it is determined whether the student's pen-holding posture is standard. If not, the smart pen held by the student is controlled to vibrate to remind the student that the current pen-holding posture is not standard. In step S2023, if it is determined that the actual pen-holding gesture of the student during the test-taking process does not conform to the preset pen-holding gesture, the smart pen held by the student is controlled to vibrate to remind the student that the current pen-holding posture is not standard. The vibration frequency of the smart pen held by the student is determined using the following formula (3): In the above formula (3), f represents the vibration frequency of the smart pen, f max Indicates the maximum vibration frequency that the smart pen can reach when vibrating. Indicates that the maximum value of the values ​​in the brackets is calculated.

4. The method of improving students' concentration in answering questions by using their hand gestures as claimed in claim 1, characterized in that: In step S3, the student's concentration level during the problem-solving process is determined based on the result of the judgment of the problem-solving stop state; Then, according to the student's concentration level during the test, the reminders to the student at different times during the test include: Step S301: summing the durations corresponding to any two chronologically adjacent stop states to obtain a corresponding total duration; then comparing the total duration with a preset time threshold; if the total duration exceeds the preset time threshold, determining that the student was in a state of not concentrating on solving problems during the time periods corresponding to the two stop states; otherwise, determining that the student was in a state of concentrating on solving problems during the time periods corresponding to the two stop states; Step S302: Play a voice reminder to the student during the period when the student is not concentrating on the test.

5. A system that improves students' concentration in answering questions by using their gestures when doing questions, characterized by: It includes a module for shooting and analyzing the action of doing the question, a module for determining the gesture information of doing the question, a module for determining the stopped state of the task and a reminder module; wherein, The question-solving action shooting and analysis module is used to shoot the student's question-solving action images during the question-solving process and analyze the question-solving action images to determine the student's hand movement data during the question-solving process; The question-solving gesture information determination module is used to determine the question-solving gesture information of the student during the question-solving process based on the hand movement data; The stopped-work state determination module is used to determine whether the student has stopped working on the problem and the duration of the stopped working state according to the problem-solving gesture information; the reminder module is used to determine the student's concentration level during the problem-solving process according to the result of the stopped-work state determination; and then, based on the concentration level, remind the student at different time periods during the problem-solving process; The question-solving gesture information determination module is further configured to: determine, based on the hand movement posture included in the hand activity data, first gesture information corresponding to the student's hand holding the pen to solve the problem; and determine, based on the hand movement amplitude included in the hand activity data, second gesture information corresponding to the relative position between the student's hand and the answer book during the problem-solving process; The stopped work state determination module is further configured to: analyze the first gesture information to determine whether the actual hand gesture of the student holding the pen during the problem-solving process is a preset pen-holding gesture; if not, determine that the student is currently in a stopped work state; and simultaneously determine a duration during which the actual hand gesture of the student holding the pen during the problem-solving process is not a preset pen-holding gesture, and use this duration as the duration corresponding to the stopped work state; Analyzing the second gesture information to determine whether the student's hand has left the plane coverage area corresponding to the answer book itself during the test-taking process; if so, determining that the student is currently in a stopped test-taking state; and determining a duration of time that the student's hand has left the plane coverage area corresponding to the answer book itself during the test-taking process, which serves as the duration corresponding to the stopped test-taking state; The analyzing the first gesture information to determine whether the actual pen-holding gesture of the student during the test-taking process is the preset pen-holding gesture specifically includes: In step S2021, the coordinate position of each fingertip, the coordinate position of the palm center, and the normal vector of the palm surface are collected using the hand posture sensor as the first gesture information, and the angle value between each fingertip and the normal vector of the palm surface is determined using the following formula (1): In the above formula (1), α a represents the angle between the tip of the ath finger and the normal vector of the palm, (x a ,y a ,z a ) represents the coordinate value of the fingertip of the ath finger, (x0, y0, z0) represents the coordinate value of the palm center, and (x, y, z) represents the normal vector of the palm surface.

6. The system for improving students' concentration in answering questions by using their hand gestures as claimed in claim 5, characterized in that: The question-solving action shooting and analysis module shoots the student's question-solving action images during the question-solving process and analyzes the question-solving action images to determine the student's hand movement data during the question-solving process. Specifically, the data includes: Taking binocular photos of the student's hands and the surrounding area while the student is doing the test, thereby obtaining binocular images of the student's action during the test; Then, the binocular image parallax of the binocular image of the problem-solving action is calculated, and then a three-dimensional image of the student in the problem-solving process is generated according to the binocular image parallax; Finally, according to the preset time interval, several image frames are extracted from the three-dimensional test-taking image in turn, and the hand movement posture and hand movement amplitude of the student in each image frame are identified and used as the hand activity data.

7. The system for improving students' concentration in answering questions by using their hand gestures as claimed in claim 5, characterized in that: After step S2021, the following steps are also included: In step S2022, the following formula (2) is used to determine whether the student's actual pen-holding gesture during the test is the preset pen-holding gesture based on the angle between the fingertip of each finger and the normal vector of the palm: In the above formula (2), P represents the judgment value of whether the actual gesture of the student holding the pen during the problem-solving process corresponds to the preset pen-holding gesture, β a,min Indicates the minimum preset angle between the tip of the ath finger and the normal vector of the palm in the preset pen-holding gesture, β a,max represents the maximum preset angle between the fingertip of the ath finger and the normal vector of the palm in the preset pen-holding gesture of the student, n represents the number of fingers in contact with the pen when the student holds the pen, u() represents a step function, when the value in the brackets is greater than or equal to 0, the step function takes the value of 1, when the value in the brackets is less than 0, the step function takes the value of 0, δ{} represents a unit impulse function, when the value in the brackets is equal to 0, the unit impulse function takes the value of 1, when the value in the brackets is not equal to 0; If P=0, it is determined that the actual hand gesture of the student holding the pen during the problem-solving process does not belong to the preset pen-holding gesture, and the student is determined to be in the stopped problem-solving state. At the same time, the duration of the actual hand gesture of the student holding the pen during the problem-solving process that does not belong to the preset pen-holding gesture is determined, and this is used as the duration corresponding to the student being in the stopped problem-solving state; If P=1, it is determined that the actual pen-holding gesture of the student during the test is a preset pen-holding gesture. Then, in step S2023, it is determined whether the student's pen-holding posture is standard. If not, the smart pen held by the student is controlled to vibrate to remind the student that the current pen-holding posture is not standard. In step S2023, if it is determined that the actual pen-holding gesture of the student during the test-taking process does not conform to the preset pen-holding gesture, the smart pen held by the student is controlled to vibrate to remind the student that the current pen-holding posture is not standard. The vibration frequency of the smart pen held by the student is determined using the following formula (3): In the above formula (3), f represents the vibration frequency of the smart pen, f max Indicates the maximum vibration frequency that the smart pen can reach when vibrating. Indicates that the maximum value of the values ​​in the brackets is calculated.

8. The system for improving students' concentration in answering questions by using their hand gestures as claimed in claim 5, characterized in that: The reminder module determines the student's concentration level during the test-taking process according to the result of the judgment of the stopped test-taking state; Then, according to the student's concentration level during the test, the reminders to the student at different times during the test include: The durations corresponding to any two chronologically adjacent stop-work states are summed to obtain a corresponding total duration; the total duration is then compared with a preset time threshold; if the total duration exceeds the preset time threshold, it is determined that the student is in a state of not concentrating on the work during the period corresponding to the two stop-work states as a whole; otherwise, it is determined that the student is in a state of concentrating on the work during the period corresponding to the two stop-work states as a whole; Finally, during the period when the student is not concentrating on the test, a voice reminder is played to the student.

Images