Student concentration evaluation method, device, equipment and medium

Abstract

The invention relates to a student concentration evaluation method and device, computer equipment and a storage medium. The method comprises the following steps: a data acquisition step: acquiring facial data and screen display content of a student; a data processing step: processing the face data of the student to obtain a view field range of the student on a screen; a reading capability obtaining step of obtaining the reading capability of the student; and a concentration generation step of obtaining a concentration score of the student according to the screen display content, the view field of the student on the screen and the reading ability of the student. According to the invention, the concentration performance conditions of different students at different time can be quantitatively displayed, and help is provided for optimizing teaching methods of different students.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and medium for evaluating student concentration. Background Art

[0002] At present, the main research direction of human facial data is to obtain human facial expression changes through facial data. The most common solution is to define human expressions into 7 types, namely: sadness, fear, disgust, happiness, anger, surprise and neutrality. This also constitutes the seven basic facial expressions in today's facial expression recognition research. Related technologies are mainly used in social network analysis, telemedicine and criminal investigation monitoring.

[0003] The existing AI concentration recognition technology is an extension of face recognition technology, which uses a convolutional neural network to identify and judge the collected expression data based on the expression data set. In the application of this technology in the field of education and teaching, it cannot accurately capture the students' visual focus range, nor can it quantify the students' concentration based on their individual differences.

[0004] To address the above problems, no effective solution has been proposed yet. Summary of the invention

[0005] Based on this, in order to solve the above problems, this paper proposes a student concentration evaluation method, device, equipment and medium. The invention can quantitatively display the concentration performance of different students at different times, and provide help for optimizing the teaching methods for different students.

[0006] According to a first aspect of the present invention, a method for evaluating student concentration is provided, comprising:

[0007] Data acquisition step, acquiring students' facial data and screen display content;

[0008] Data processing step, processing the student's facial data to obtain the student's field of view on the screen;

[0009] Reading ability acquisition steps, acquire students' reading ability;

[0010] A concentration generating step is to obtain the student's concentration score according to the screen display content, the student's field of vision on the screen, and the student's reading ability.

[0011] In some embodiments, the processing of the student's facial data to obtain the student's field of view on the screen includes:

[0012] According to the facial data of the students, the facial key points of the students are located, the sight angle of the eyes is obtained according to the eye key point data, the distance of the face from the screen is obtained according to the data changes of the facial key points, and the field of view of the students on the screen is obtained according to the distance from the face to the screen and the sight angle of the eyes.

[0013] In some embodiments, the facial key points are 68 facial key points, and the eye key points are 6 eye key points.

[0014] In some embodiments, the processing of the student's facial data to obtain the student's field of view on the screen includes:

[0015] A correspondence between facial expressions and visual fields is established, the facial expressions of students are obtained according to the facial data of the students, and the visual fields of students on the screen are obtained through the facial expressions of the students according to the correspondence.

[0016] In some embodiments, establishing the correspondence between facial expressions and visual fields includes: taking the student's expressions as an input sequence, taking the student's visual field on the screen as a target sequence, performing model training through a Transformer model, and obtaining the correspondence between facial expressions and visual fields.

[0017] In some embodiments, the method further includes: a concentration curve generating step, generating a concentration curve that changes over time based on the student's concentration score.

[0018] In some embodiments, the method for acquiring the student's reading ability includes: testing the area that the student can see without eye movement, and generating the corresponding student's reading ability according to the area.

[0019] According to a second aspect of the present invention, there is provided a device for evaluating student concentration, comprising:

[0020] A data acquisition module is used to acquire students' facial data and screen display content;

[0021] A data processing module is used to process the student's facial data and obtain the student's field of view on the screen;

[0022] A reading ability acquisition module is used to acquire students' reading ability;

[0023] The concentration generation module is used to obtain the student's concentration score based on the screen display content, the student's field of vision on the screen, and the student's reading ability.

[0024] According to a third aspect of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0025] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned embodiments of the method are implemented.

[0026] According to a fifth aspect of the present invention, a use is provided, wherein any of the above methods or devices are used by teachers and / or students and / or parents and / or AI to formulate corresponding learning plans based on the students' concentration levels.

[0027] The technical solution of the present invention can realize the quantitative evaluation of the concentration of different students in each class, so as to adjust the corresponding teaching plan for different students and improve the teaching effect. Taking into account the differences between different students, a more accurate concentration evaluation can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of some embodiments of the student concentration evaluation method of the present invention;

[0029] Figure 2 is a schematic diagram of six key points of the eye in the present invention;

[0030] Figure 3 It is a multi-color concentric circle test chart in the present invention;

[0031] Figure 4 It is the color horizontal bar test chart of the present invention;

[0032] Figure 5 is a schematic diagram of the structure of some embodiments of the student concentration evaluation device of the present invention;

[0033] Figure 6 It is a diagram of the internal structure of a computer device used to implement some embodiments of the present invention. DETAILED DESCRIPTION

[0034] Embodiments of the present invention will be described more fully below with reference to the accompanying drawings, in which embodiments of the present invention are shown. However, the present invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

[0035] The terms used herein are used only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an", "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that when used herein, the term "comprising" specifies the presence of the claimed features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof.

[0036] Unless otherwise defined, the terms (including technical and scientific terms) used herein have the same meanings as commonly understood by those of ordinary skill in the art to which the present disclosure belongs. The terms used herein should be interpreted as having the meanings consistent with their meanings in the context of this specification and in the relevant field, and should not be interpreted in an idealized or overly formal sense, unless specifically defined as such herein.

[0037] Reading ability: refers to the ability of each person to read screen information when reading the screen, or the size of the area that each person can read when the line of sight is fixed. This ability will show obvious differences among different people, for example: A reads quickly line by line, while B can only read word by word.

[0038] 68 facial key points: The specific locations include 5 key points for one eyebrow, 6 key points for the eye, and other key points. This annotation scheme was first proposed in the Xm2vtsdb dataset in 1999, and has been widely used in the subsequent 300W dataset and XM2VTS dataset. The Dlib algorithm is a well-known algorithm that uses this 68-point annotation. It can accurately locate the positions of these 68 key points by analyzing facial images.

[0039] 6 eye key points: that is, 6 eye key points among the 68 facial key points.

[0040] Concentration is of vital importance in the learning process. Concentration helps to improve learning efficiency. When students can concentrate on learning, they can understand and absorb knowledge faster, reduce the waste of learning time, complete more learning tasks in the same time, and achieve better academic results. Concentration can cultivate the ability to think deeply. It enables students to explore problems in depth, conduct logical reasoning and analysis, not only know the facts, but also know the reasons, which helps to cultivate innovative thinking and the ability to solve complex problems. Concentration helps to enhance memory. Concentrated attention allows the brain to encode and store information more effectively, thereby improving the effect of memory and allowing the knowledge learned to remain in the mind longer. The present invention can effectively improve the teaching effect in the subsequent teaching process by quantitatively evaluating the concentration of different students in the classroom.

[0041] Reading range: The range on the screen that students can read without moving their eyes. Select the corresponding area extending outward from the center of the student's field of view on the screen based on the student's reading ability, and the size of the area is consistent with the student's reading ability.

[0042] Figure 1 A flow chart showing some embodiments of the method for evaluating student concentration of the present invention.

[0043] like Figure 1 As shown, the method includes:

[0044] Data acquisition step S102, acquiring the student's facial data and screen display content;

[0045] Data processing step S104, processing the student's facial data to obtain the student's field of view on the screen;

[0046] In some embodiments, the specific steps include:

[0047] (1) Based on the student's facial data, use the dlib machine learning framework to specify parameters and locate the 68 key points of the face in dlib. Import the dlib library and import the parameters.

[0048] (2) Set the parameters and the corresponding ROI area. Detect the position of a region where the face is located, and then detect the position of the nose relative to the face frame, such as: the left eye of a person is at the face frame of 0.2w and 0.2h.

[0049] (3) The positions of the key points relative to the face frame, which correspond to the mouth, left eye, right eye, left eyebrow, right eyebrow, nose, and chin.

[0050] (4) Load face detection and key point location. Then return the coordinates of the four points of the face detection rectangle through the passed parameters. Take a certain area of ​​the image as input and output a series of points to represent the posture in this image.

[0051] (5) Process the data and convert it into a grayscale image. Call the face frame detector, use the grayscale image for detection, and traverse the detection frame. Traverse the area and draw a circle around it. Extract the ROI area and display it. Detect the face and locate it based on the key points. After locating, apply the positioning information to the student's line of sight angle detection.

[0052] (6) Determine the sight angle through the six key points of the glasses. Import the Euclidean distance toolkit and locate the 68 key points. Figure 2As shown in the figure, among the 6 key points of the eye, when the sight angle changes, the Euclidean distance between P2 and P6 and P3 and P5 will change, and the distance between P1 and P4 will also change slightly. A formula is set based on the relationship between the 6 key points of the glasses, that is, subtract P2 from P6, subtract P3 from P5, and then take the absolute value of the difference between P1 and P4 times the previous value. When the sight angle changes, the value will change accordingly, and the sight angle of the human eye can be judged based on this value.

[0053] The calculation steps are as follows:

[0054] Calculate the Euclidean distance between points P2 and P6:

[0055]

[0056] Calculate the Euclidean distance between points P3 and P5:

[0057]

[0058] Calculate the Euclidean distance between points P1 and P4:

[0059]

[0060] Substitute into the formula to calculate:

[0061]

[0062] Through this formula, we can calculate a value s, which will change with the change of the viewing angle, and can be used to determine the viewing angle of the human eye.

[0063] (7) Locate the chin ROI area. The key points of the chin are marked at points 0-17.

[0064] (8) Solve the Euler angles based on the rotation matrix. The posture of an object relative to the camera can be represented by a rotation matrix and a translation matrix.

[0065] (9) Using rectangles and rectangular connecting frames, the key points are located and the world coordinate system is converted into 2D coordinates. Finally, the Euler angle is calculated through CV2 to grasp the three-dimensional sight angle of the face.

[0066] (10) Set face data. Import the 68 key point data of the face to set the face size. Determine the distance between the face and the screen based on the changes in the values ​​of the 68 key points.

[0067] (11) Import the Euclidean distance toolkit. Import the face 3D sight angle numerical tool. According to the 3D sight angle range and screen distance, select a 25-degree range of focused attention on the screen as the student's field of vision on the screen.

[0068] In other embodiments, the specific steps include: establishing a correspondence between facial expressions and visual fields, obtaining the student's facial expressions based on the student's facial data, and obtaining the student's visual field on the screen through the student's facial expressions based on the correspondence.

[0069] Establishing the correspondence between facial expressions and visual fields includes: using the student's expression as an input sequence, using the student's visual field on the screen as a target sequence, and training the model through a Transformer model to obtain the correspondence between facial expressions and visual fields. The student's visual field on the screen is obtained through the above steps (1)-(11).

[0070] Reading ability acquisition step S106, acquiring the student's reading ability;

[0071] In some embodiments, the area that the student can see without eye movement is tested, and the corresponding reading ability of the student is generated according to the area.

[0072] Test method:

[0073] Use multi-colored concentric circles to test students' focus range, such as Figure 3 As shown, when a student can see a sphere of one color clearly without moving his eyes, the circular screen area of ​​that color is regarded as the focusing range of the student, and the focusing range is regarded as the reading ability of the student.

[0074] The colored horizontal bars are used to test the students’ focus range and determine the effective reading length of the text. Figure 4 As shown, the color sequence of the colored horizontal bars is said from left to right. When the eyeball moves (to the right), it means that it has exceeded the range of the visual focus point. The previous range is taken as the effective reading length, and the effective reading length is taken as the student's reading ability. Since the learning content is usually presented in horizontal text, the effective reading length is preferably used as the student's reading ability.

[0075] The concentration generating step S108 obtains the student's concentration score according to the screen display content, the student's field of vision on the screen, and the student's reading ability.

[0076] In some embodiments, the screen display content, the student's field of view on the screen, and the student's reading ability are used as vectors, and a multidimensional array is created using the Numpy library, and the student's concentration score is obtained by output. The student's reading range on the screen is obtained by the student's field of view on the screen and the student's reading ability, and the student's concentration score is obtained by comparing the reading range with the screen display content (such as: concentration score = reading range area of ​​screen display content / screen display content area). Among them, the reading range can be the reading area of ​​the student on the screen within a certain period of time (the screen display content has not changed) obtained by the student's field of view on the screen and the student's reading ability within a certain period of time.

[0077] In some embodiments, a concentration curve that changes over time can be generated based on the student's concentration score.

[0078] The students' concentration obtained by the present invention can be used in multiple directions such as teachers formulating learning plans and students' self-evaluation, which can greatly help improve teaching effects.

[0079] In some embodiments, on the teacher side: teachers can deeply understand students' concentration through the concentration curve chart, and can customize learning plans according to individual concentration differences. For students whose concentration curve is at a high level for a long time, provide more challenging and in-depth learning tasks, encourage independent learning, organize group discussions or cooperative learning, etc. For students whose concentration curve fluctuates normally, break down learning tasks into smaller, manageable parts; adopt a variety of teaching methods, such as explanation, demonstration, practice, discussion, etc.; regularly conduct short concentration training activities, such as meditation, concentration exercises, etc. For students whose concentration curve is at a low level for a long time, design interesting and interactive learning activities, such as gamified learning and experimental operations; shorten learning time, increase the interval between rest and activities, and avoid excessive fatigue; cooperate with parents to develop a concentration training plan at home, such as limiting the time of using electronic products and conducting parent-child activities for concentration training. In the process of teachers implementing learning plans, the concentration curve chart helps teachers adjust and optimize the plan according to actual conditions to ensure that each student can gradually improve their learning effects based on their concentration level.

[0080] Student side: After students self-assess their concentration through the concentration curve, they can customize learning methods based on individual concentration differences. Including: Observe how often you are easily distracted when completing homework or learning tasks. Record the length of time you can maintain continuous concentration. Pay attention to the differences in concentration performance in different subjects and different learning environments. For example, the concentration curve shows the student's concentration: 1. Can concentrate on subjects of interest for about 40 minutes, and can only maintain 20 minutes on subjects of no interest. 2. Easily distracted by mobile phones, TVs, etc. when studying at home, and perform better in quiet environments such as libraries. Based on the above self-understanding, the study plan that can be formulated through AI is as follows: Monday to Friday, in the morning, use the time period with better memory to recite Chinese ancient poems or English words for 20 minutes each time; in the afternoon after school, first complete the more difficult and uninteresting subject homework in the library, such as mathematics, concentrate for 25 minutes each time, rest for 5 minutes, and repeat twice. Then go home to complete the homework of the subject of interest, such as history, for 30 minutes each time. On weekends, on Saturday morning, review the key knowledge learned this week, change a subject every 30 minutes, and take a 10-minute break in between. On Saturday afternoon, attend extracurricular interest classes to cultivate interest and exercise concentration. On Sunday morning, conduct reading training, choose books with a certain depth, read for 30 minutes each time, and take notes. On Sunday afternoon, summarize the study situation of this week and make a study plan for next week. At the same time, students can also make some rules to help themselves improve their concentration, such as: 1. Set the mobile phone to silent mode and put it away from yourself during study. 2. When studying at home, tell your family in advance not to disturb you. 3. Conduct a self-assessment of concentration once a week and adjust the study plan according to the situation.

[0081] Parent side: After parents understand their children's concentration through the concentration model curve, they can take corresponding measures according to the specific situation. If the student's concentration curve is at a high position for a long time: 1. Give affirmation and encouragement to strengthen this good performance. 2. Provide more challenging tasks and activities to further improve the child's concentration level. 3. Continue to create an environment conducive to concentration and maintain a regular rhythm of life and study. If the student's concentration curve fluctuates normally, parents can follow the fluctuation range: 1. Analyze the reasons for the lack of concentration with the child, such as environmental interference, fatigue, lack of interest, etc. 2. Adjust the child's schedule to ensure adequate sleep and rest. 3. If necessary, seek professional help, such as psychological counseling or concentration training courses. 4. Accompany your child more often in activities that require patience and concentration, such as handicrafts, playing chess, etc.

[0082] Figure 5 The schematic diagrams of the structures of some embodiments of the device for evaluating the concentration of students of the present invention are shown.

[0083] like Figure 5 As shown, including:

[0084] The data acquisition module 100 is used to acquire the student's facial data and screen display content;

[0085] The data processing module 200 is used to process the facial data of the student and obtain the visual field of the student on the screen;

[0086] The reading ability acquisition module 300 is used to acquire the reading ability of the students;

[0087] The concentration generating module 400 is used to obtain the student's concentration score according to the screen display content, the student's field of vision on the screen, and the student's reading ability.

[0088] For the specific definition of a student concentration evaluation device, please refer to the definition of a student concentration evaluation method above, which will not be repeated here. Each module in the above-mentioned student concentration evaluation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0089] The present invention also provides a computer device, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned student concentration evaluation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc. Those skilled in the art can understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0090] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned student concentration evaluation method is implemented.

[0091] Those skilled in the art will appreciate that all or part of the processes in the above method embodiments can be accomplished by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0092] So far, the embodiments of the present invention have been described in detail. In order to avoid obscuring the concept of the present invention, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solutions disclosed herein.

[0093] Although some specific embodiments of the present invention have been described in detail by way of examples, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It should be understood by those skilled in the art that the above embodiments may be modified or some technical features may be replaced by equivalents without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A method for evaluating student concentration, characterized in that: The following steps are involved: Data acquisition step, acquiring students' facial data and screen display content; Data processing step, processing the student's facial data to obtain the student's field of view on the screen; Reading ability acquisition steps, acquire students' reading ability; A concentration generating step is to obtain the student's concentration score according to the screen display content, the student's field of vision on the screen, and the student's reading ability.

2. The method for evaluating student concentration according to claim 1, characterized in that: The processing of the student's facial data to obtain the student's field of view on the screen includes: According to the facial data of the students, the facial key points of the students are located, the sight angle of the eyes is obtained according to the eye key point data, the distance of the face from the screen is obtained according to the data changes of the facial key points, and the field of view of the students on the screen is obtained according to the distance from the face to the screen and the sight angle of the eyes.

3. The method for evaluating student concentration according to claim 2, characterized in that: The facial key points are 68 facial key points, and the eye key points are 6 eye key points.

4. The method for evaluating student concentration according to claim 1, characterized in that: The processing of the student's facial data to obtain the student's field of view on the screen includes: A correspondence between facial expressions and visual fields is established, the facial expressions of students are obtained according to the facial data of the students, and the visual fields of students on the screen are obtained through the facial expressions of the students according to the correspondence.

5. The method for evaluating student concentration according to claim 4, characterized in that: The method of establishing the correspondence between facial expressions and visual fields includes: taking the student's expression as an input sequence, taking the student's visual field on the screen as a target sequence, performing model training through a Transformer model, and obtaining the correspondence between facial expressions and visual fields.

6. The method for evaluating student concentration according to claim 1, characterized in that: Also includes: A concentration curve generation step is to generate a concentration curve that changes over time based on the student's concentration score.

7. The method for evaluating student concentration according to claim 1, characterized in that: The method for acquiring the student's reading ability comprises: testing the area that the student can see without eyeball rotation, and generating the corresponding student's reading ability according to the area.

8. A device for evaluating student concentration, characterized in that: include: A data acquisition module is used to acquire students' facial data and screen display content; A data processing module is used to process the student's facial data and obtain the student's field of view on the screen; A reading ability acquisition module is used to acquire students' reading ability; The concentration generation module is used to obtain the student's concentration score based on the screen display content, the student's field of vision on the screen, and the student's reading ability.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. Use of the method according to any one of claims 1 to 7 or the device according to claim 8 for teachers and / or students and / or parents and / or AI to formulate corresponding learning plans according to students' concentration.

Images