An online education supervision method and system based on artificial intelligence

Abstract

The present invention relates to the technical field of online education supervision, and specifically discloses an online education supervision method and system based on artificial intelligence, which first obtains the initial pupil data of a user; then obtains a to-be-learned video that the user needs to learn, and obtains standard pupil data corresponding to the to-be-learned video; the standard pupil data here refers to the change of pupil position in the process of carefully watching the to-be-learned video, and then obtains a second to-be-analyzed video when the user actually watches the to-be-learned video for online learning, and analyzes the second to-be-analyzed video to obtain the change of the user's pupil in the learning process, and compares the user's pupil change with the standard pupil data to generate a concentration score for expressing the user's concentration in the learning process; if the actual change of the user's pupil in the learning process is closer to the standard pupil data, the user is more serious in learning at this time; otherwise, the user needs to be reminded.

Description

Technical Field

[0001] The present invention relates to the technical field of online education supervision, and in particular to an online education supervision method and system based on artificial intelligence. Background Art

[0002] With the widespread popularity of modern computer networks and the rapid development of home electronic equipment terminals, online education using modern technologies such as computer networks and artificial intelligence has become an important part of family education. Online education can alleviate the teaching pressure caused by the inability to carry out offline teaching due to various factors; online education is flexible in form and can meet the learning and education requirements of users of all ages and classes. However, in online education programs, since learners learn independently by following teaching videos, compared with traditional offline teaching models, learners are more likely to feel relaxed, which leads to poor learning results; that is, the existing online teaching programs lack technical means to effectively supervise learners. Summary of the invention

[0003] The main purpose of the present invention is to provide an online education supervision method and system based on artificial intelligence, aiming to solve the problem that existing online teaching schemes lack technical means for effective supervision of learners.

[0004] The technical solution proposed by the present invention is:

[0005] An online education supervision method based on artificial intelligence is applied to an online education supervision system based on artificial intelligence; the system includes an education terminal and a cloud server communicatively connected to the education terminal; the education terminal includes a first processing module, and a first display screen and a first camera module communicatively connected to the first processing module; the method includes:

[0006] When the educational terminal is started, it receives the viewing angle positioning video from the cloud server, and plays the viewing angle positioning video through the first display screen for the user to watch;

[0007] During the process of playing the viewing angle positioning video on the first display screen, the first camera module captures the user's face image video in real time and marks it as the first video to be analyzed;

[0008] The first processing module determines the user's initial pupil data based on the view positioning video and the first video to be analyzed, wherein the initial pupil data is used to describe the pupil position when the user is gazing at the center of the first display screen;

[0009] The cloud server obtains a video to be learned determined by a user through an education terminal, obtains standard pupil data corresponding to the video to be learned, and then sends the video to be learned and the corresponding standard pupil data to the education terminal, wherein the standard pupil data is used to express changes in pupil position during the process of carefully watching the video to be learned;

[0010] The first processing module controls the first display screen to play the video to be learned, and during the playing process, obtains in real time a video of a user's face image captured in real time by the first camera module, and marks it as a second video to be analyzed;

[0011] The first processing module generates in real time a concentration score for expressing the concentration of the user during the learning process based on the second video to be analyzed and the standard pupil data.

[0012] Preferably, when the viewing angle positioning video is played on the first display screen, the display content of the viewing angle positioning video includes an identification block at the center of the first display screen and a first text message for prompting the user to look at the identification block; the viewing angle positioning video and the first video to be analyzed have the same duration and are greater than a first preset duration; the first processing module determines the user's initial pupil data based on the viewing angle positioning video and the first video to be analyzed, including:

[0013] The first processing module marks the last video frame image in the first video to be analyzed as a first target frame image;

[0014] The first processing module determines a first reference line and a second reference line of the first target frame image, wherein a straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the first target frame image is the first reference line, and a straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the first reference line in the first target frame image is the second reference line;

[0015] The first processing module marks the distance between the pupil center point in the first target frame image and the first reference line as the first distance, and marks the distance between the pupil center point in the first target frame image and the second reference line as the second distance;

[0016] The first processing module uses the first distance and the second distance as initial pupil data of the user.

[0017] Preferably, the system further comprises a test terminal connected to the cloud server in communication; the test terminal comprises a second processing module, and a second display screen and a second camera module connected to the second processing module in communication; the cloud server stores a plurality of different videos to be learned; the method further comprises:

[0018] The cloud server sends the video to be learned and the view positioning video to the test terminal;

[0019] The second display screen plays the viewing angle positioning video for the tester to watch, and during the viewing angle positioning video, the second camera module takes a real-time video of the tester's face image, and marks it as the third video to be analyzed;

[0020] The second processing module determines the initial pupil data of the test person based on the perspective positioning video and the third video to be analyzed;

[0021] The second display screen plays the video to be learned for the tester to watch and learn, and during the playing of the video to be learned, the second camera module takes a real-time video of the tester's face image and marks it as a fourth video to be analyzed, wherein the tester stays focused while watching and learning the video to be learned;

[0022] The second processing module generates standard pupil data corresponding to the video to be learned based on the fourth video to be analyzed and the initial pupil data of the test person.

[0023] Preferably, the second processing module determines the initial pupil data of the test person based on the viewing angle positioning video and the third video to be analyzed, including:

[0024] The second processing module marks the last video frame image in the third video to be analyzed as a second target frame image;

[0025] The second processing module determines a third reference line and a fourth reference line of the second target frame image, wherein a straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the second target frame image is the third reference line, and a straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the second reference line in the first target frame image is the fourth reference line;

[0026] The second processing module marks the distance between the pupil center point and the third reference line in the second target frame image as a third distance, and marks the distance between the pupil center point and the fourth reference line in the second target frame image as a fourth distance;

[0027] The second processing module uses the third distance and the fourth distance as initial pupil data of the test person.

[0028] Preferably, the recording duration of the fourth video to be analyzed is consistent with the playback duration of the video to be learned, and the recording start time of the fourth video to be analyzed is consistent with the time when the video to be learned starts to be played on the test terminal; the second processing module generates standard pupil data corresponding to the video to be learned based on the fourth video to be analyzed and the initial pupil data of the tester, including:

[0029] The second processing module determines a plurality of preset time nodes in the video to be learned, wherein the interval between any two adjacent preset time nodes is the second preset time, the first preset time node is the start time of the video to be learned, and the interval between the last preset time node and the end time of the video to be learned is less than the second preset time;

[0030] Whenever the video to be learned is played on the second display screen to any preset time node, the second processing module obtains a video frame image of the fourth video to be analyzed at the same preset time node, marks it as a third target frame image, thereby obtaining a plurality of third target frame images, wherein the i-th third target frame image is a video frame image in the fourth video to be analyzed corresponding to when the video to be learned is played to the i-th preset time node, and the number of the third target frame images is consistent with the number of preset time nodes in the video to be learned, and satisfies: 1≤i≤K, K is the total number of preset time nodes in the video to be learned;

[0031] The second processing module determines a fifth reference line and a sixth reference line of the third target frame image, wherein a straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the third target frame image is the fifth reference line, and a straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the fifth reference line in the third target frame image is the sixth reference line;

[0032] The second processing module marks the point in the third target frame image whose distance from the fifth reference line is the third distance, whose distance from the sixth reference line is the fourth distance, and which is located in the right eye as the first origin;

[0033] The second processing module establishes a first polar coordinate system in the third target frame image, wherein the origin of the first polar coordinate system is the first origin, and the polar axis of the first polar coordinate system is a ray extending away from the inner canthus of the right eye and parallel to the sixth reference line;

[0034] The second processing module obtains the polar coordinates of the center point of the right pupil in the third target frame image in the first polar coordinate system and marks them as first polar coordinates. is the polar coordinate of the center point of the right pupil in the i-th third target frame image in the first polar coordinate system, is the polar diameter of the center point of the right pupil in the i-th third target frame image in the first polar coordinate system, is the polar angle of the center point of the right eye pupil in the i-th third target frame image in the first polar coordinate system;

[0035] The second processing module uses the preset time nodes in the video to be learned and the first polar coordinates of each third target frame image as standard pupil data corresponding to the video to be learned.

[0036] Preferably, the recording start time of the second video to be analyzed is consistent with the time when the video to be learned starts to be played on the education terminal; the first processing module generates a concentration score for expressing the concentration of the user in the learning process in real time based on the second video to be analyzed and the standard pupil data, including:

[0037] Whenever the video to be learned is played on the first display screen to any preset time node, the first processing module obtains a video frame image of the second video to be analyzed at the same preset time node, marks it as a fourth target frame image, to obtain a plurality of fourth target frame images, wherein the i-th fourth target frame image is a video frame image in the fourth video to be analyzed corresponding to when the video to be learned is played to the i-th preset time node, and the number of the fourth target frame images is consistent with the number of preset time nodes in the video to be learned;

[0038] Whenever a new fourth target frame image is acquired, the first processing module determines a seventh reference line and an eighth reference line of the new fourth target frame image, wherein a straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the fourth target frame image is the seventh reference line, and a straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the seventh reference line in the fourth target frame image is the eighth reference line;

[0039] The first processing module marks the point in the new fourth target frame image whose distance from the seventh reference line is the first distance, whose distance from the eighth reference line is the second distance, and which is located in the right eye as the second origin;

[0040] The first processing module establishes a second polar coordinate system in the new fourth target frame image, wherein the origin of the second polar coordinate system is the second origin, and the polar axis of the second polar coordinate system is a ray extending away from the inner canthus of the right eye and parallel to the eighth reference line;

[0041] The first processing module obtains the polar coordinates of the center point of the right pupil in the new fourth target frame image in the second polar coordinate system and marks them as second polar coordinates. is the polar coordinate of the center point of the right pupil in the i-th fourth target frame image in the second polar coordinate system, is the polar diameter of the center point of the right pupil in the i-th fourth target frame image in the second polar coordinate system, is the polar angle of the center point of the right eye pupil in the i-th fourth target frame image in the second polar coordinate system;

[0042] The first processing module calculates the user's current concentration score based on the second polar coordinates of the new fourth target frame image and the first polar coordinates of the third target frame image.

[0043] Preferably, the preset time node closest to the current moment is assumed to be the kth preset time node, and 1≤k≤K is satisfied; the first processing module calculates the user's current concentration score based on the second polar coordinates of the new fourth target frame image and the first polar coordinates of the third target frame image, including:

[0044] The first processing module calculates the user's current concentration score by formula (1):

[0045] (1),

[0046] In the formula, Indicates the user's current concentration score. The closer the concentration score is to 1, the higher the user's concentration when studying. represents the polar angle of the center point of the right pupil in the kth fourth target frame image in the second polar coordinate system; Represents the polar angle of the right eye pupil center point in the kth third target frame image in the first polar coordinate system.

[0047] Preferably, the first processing module calculates the user's current concentration score by formula (1), which includes:

[0048] The first processing module determines whether k is greater than a first preset value;

[0049] If yes, the first processing module calculates the user's current concentration score using formula (2) and formula (3):

[0050] (2),

[0051] (3),

[0052] In the formula, Indicates the user's current concentration score. The closer the concentration score is to 2, the higher the user's concentration is when studying. represents the polar diameter of the center point of the right pupil in the kth fourth target frame image in the second polar coordinate system; represents the polar diameter of the center point of the right pupil in the kth third target frame image in the first polar coordinate system;

[0053] If not, the first processing module is executed to calculate the user's current concentration score by formula (1).

[0054] Preferably, the first processing module generates a concentration score for expressing the concentration of the user during the learning process in real time based on the second video to be analyzed and the standard pupil data, and then further includes:

[0055] The first processing module obtains all concentration scores within the past third preset time period and calculates the average value, and uses the average value as the feedback score value;

[0056] When the feedback score value is lower than a second preset value, the first processing module generates a second text message for reminding the user to concentrate, and displays the second text message on the first display screen.

[0057] The present invention also proposes an online education supervision system based on artificial intelligence, which applies an online education supervision method based on artificial intelligence; the system includes an education terminal, and a cloud server communicatively connected to the education terminal; the education terminal includes a first processing module, and a first display screen and a first camera module communicatively connected to the first processing module.

[0058] Through the above technical solution, the following beneficial effects can be achieved:

[0059] The present invention proposes an online education supervision method based on artificial intelligence, which can effectively supervise learners who are doing online education. The specific scheme is as follows: firstly, a viewing angle positioning video is played through a first display screen for users to watch, and a first video to be analyzed in the process of users watching the viewing angle positioning video is obtained, and the initial pupil data of the user is obtained by analyzing the first video to be analyzed; then, a video to be learned that the user needs to learn is obtained, and standard pupil data corresponding to the video to be learned is obtained; the standard pupil data here refers to the change of pupil position in the process of carefully watching the video to be learned. When the user subsequently watches the video to be learned for online learning, the first camera module will shoot the user's pupils in real time. The face image video of the user (i.e., the second video to be analyzed) is analyzed, and the changes in the user's pupil during the learning process can be obtained, and the user's pupil changes are compared with the standard pupil data to generate a concentration score for expressing the user's concentration during the learning process; the specific principle is: if the actual changes in the user's pupil during the learning process are closer to the standard pupil data, the corresponding concentration score will be higher, which means that the user is studying more seriously at this time; conversely, the lower the corresponding concentration score is, the lower the user's concentration on learning at this time, and the user needs to be reminded to achieve the purpose of effectively supervising the user's online education. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0061] Figure 1 A flowchart of a first embodiment of an online education supervision method based on artificial intelligence proposed by the present invention;

[0062] Figure 2 A partial image of the first target frame image in the second embodiment of the online education supervision method based on artificial intelligence proposed by the present invention;

[0063] Figure 3 It is a partial image of any one of the fourth target frame images in the sixth embodiment of the online education supervision method based on artificial intelligence proposed by the present invention. DETAILED DESCRIPTION

[0064] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0065] The present invention proposes an online education supervision method and system based on artificial intelligence.

[0066] As attached Figure 1 As shown, in a first embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, the method is applied to an online education supervision system based on artificial intelligence; the system includes an education terminal, and a cloud server communicatively connected to the education terminal; the education terminal includes a first processing module, and a first display screen and a first camera module communicatively connected to the first processing module; this embodiment includes the following steps:

[0067] Step S110: When the educational terminal is started, it receives the perspective positioning video from the cloud server, and plays the perspective positioning video through the first display screen for the user to watch.

[0068] Step S120: While the first display screen is playing the viewing angle positioning video, the first camera module captures the user's facial image video in real time and marks it as the first video to be analyzed.

[0069] Specifically, the first video to be analyzed includes a facial image of a user.

[0070] Step S130: The first processing module determines the user's initial pupil data based on the viewing angle positioning video and the first video to be analyzed, wherein the initial pupil data is used to express the pupil position when the user looks at the center position of the first display screen.

[0071] Specifically, the user's eyes are guided to focus on the center of the first display screen through the perspective positioning video, so as to facilitate the subsequent eye movement angle of the user, thereby judging whether the user maintains concentration during the online learning process.

[0072] Step S140: The cloud server obtains the video to be learned determined by the user through the education terminal, and obtains the standard pupil data corresponding to the video to be learned, and then sends the video to be learned and the corresponding standard pupil data to the education terminal, wherein the standard pupil data is used to express the changes in pupil position during the process of carefully watching the video to be learned.

[0073] Specifically, the standard pupil data is obtained by recording the eye movement of the test subject while watching the video to be learned, thereby reflecting the change of the pupil position while carefully watching the video to be learned.

[0074] Step S150: the first processing module controls the first display screen to play the video to be learned, and during the playback process, obtains in real time a user's facial image video captured in real time by the first camera module, and marks it as a second video to be analyzed.

[0075] Step S160: The first processing module generates in real time a concentration score for expressing the concentration of the user during the learning process based on the second video to be analyzed and the standard pupil data.

[0076] The present invention proposes an online education supervision method based on artificial intelligence, which can effectively supervise learners who are doing online education. The specific scheme is as follows: firstly, a viewing angle positioning video is played through a first display screen for users to watch, and a first video to be analyzed in the process of users watching the viewing angle positioning video is obtained, and the initial pupil data of the user is obtained by analyzing the first video to be analyzed; then, a video to be learned that the user needs to learn is obtained, and standard pupil data corresponding to the video to be learned is obtained; the standard pupil data here refers to the change of pupil position in the process of carefully watching the video to be learned. When the user subsequently watches the video to be learned for online learning, the first camera module will shoot the user's pupils in real time. The face image video of the user (i.e., the second video to be analyzed) is analyzed, and the changes in the user's pupil during the learning process can be obtained, and the user's pupil changes are compared with the standard pupil data to generate a concentration score for expressing the user's concentration during the learning process; the specific principle is: if the actual changes in the user's pupil during the learning process are closer to the standard pupil data, the corresponding concentration score will be higher, which means that the user is studying more seriously at this time; conversely, the lower the corresponding concentration score is, the lower the user's concentration on learning at this time, and the user needs to be reminded to achieve the purpose of effectively supervising the user's online education.

[0077] In a second embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, based on the first embodiment, when the perspective positioning video is played on the first display screen, the display content of the perspective positioning video includes an identification block at the center of the first display screen, and a first text message for prompting the user to look at the identification block (for example: please look at the center of the screen); the perspective positioning video and the first video to be analyzed have the same duration, and are greater than a first preset duration (for example, 3 seconds); step S130 includes the following steps:

[0078] Step S210: the first processing module marks the last video frame image in the first video to be analyzed as a first target frame image.

[0079] Step S220: The first processing module determines the first reference line and the second reference line of the first target frame image, wherein the straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the first target frame image is the first reference line, and the straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the first reference line in the first target frame image is the second reference line.

[0080] Specifically, as attached Figure 2 As shown, attached Figure 2 The dotted frame in the middle is a partial image of the first target frame image.

[0081] Step S230: the first processing module marks the distance between the pupil center point in the first target frame image and the first reference line as the first distance, and marks the distance between the pupil center point in the first target frame image and the second reference line as the second distance.

[0082] Step S240: The first processing module uses the first distance and the second distance as initial pupil data of the user.

[0083] Specifically, the first distance and the second distance can reflect the position of the center point of the pupil of the user's right eye (at this time, the user is gazing at the center of the first display screen).

[0084] In a third embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, based on the second embodiment, the system further includes a test terminal communicatively connected to the cloud server; the test terminal includes a second processing module, and a second display screen and a second camera module communicatively connected to the second processing module; the cloud server stores a plurality of different videos to be learned; this embodiment further includes the following steps:

[0085] Step S310: The cloud server sends the video to be learned and the view positioning video to the test terminal.

[0086] Step S320: The second display screen plays the perspective positioning video for the test person to watch, and during the playing of the perspective positioning video, the second camera module takes real-time video of the test person's facial image, and marks it as the third video to be analyzed, wherein when the perspective positioning video is played on the second display screen, the display content of the perspective positioning video includes an identification block at the center of the second display screen, and a first text message for prompting the user to look at the identification block; the perspective positioning video and the second video to be analyzed have the same duration.

[0087] Step S330: The second processing module determines the initial pupil data of the test person based on the viewing angle positioning video and the third video to be analyzed.

[0088] Specifically, the initial pupil data of the tester here is the pupil position corresponding to when the tester views the central position of the second display screen.

[0089] Step S340: The second display screen plays the video to be learned for the tester to watch and learn, and during the playing of the video to be learned, the second camera module captures the tester's facial image video in real time and marks it as the fourth video to be analyzed, wherein the tester stays focused while watching and learning the video to be learned.

[0090] Step S350: The second processing module generates standard pupil data corresponding to the video to be learned based on the fourth video to be analyzed and the initial pupil data of the test person.

[0091] Specifically, the standard pupil data corresponding to the video to be learned here refers to the position change of the pupil of the test person when the test person is carefully watching and learning the video to be learned.

[0092] In a fourth embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, based on the third embodiment, step S330 includes the following steps:

[0093] Step S410: the second processing module marks the last video frame image in the third video to be analyzed as a second target frame image.

[0094] Specifically, the second target frame image is a face image of the tester when he or she is gazing at the center of the second display screen.

[0095] Step S420: The second processing module determines the third reference line and the fourth reference line of the second target frame image, wherein the straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the second target frame image is the third reference line, and the straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the second reference line in the first target frame image is the fourth reference line.

[0096] Step S430: the second processing module marks the distance between the pupil center point in the second target frame image and the third reference line as the third distance, and marks the distance between the pupil center point in the second target frame image and the fourth reference line as the fourth distance.

[0097] Specifically, the third distance and the fourth distance can reflect the position of the center point of the pupil of the right eye of the tester (at this time, the tester is gazing at the center of the second display screen).

[0098] Step S440: The second processing module uses the third distance and the fourth distance as initial pupil data of the test person.

[0099] In a fifth embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, based on the fourth embodiment, the recording duration of the fourth video to be analyzed is consistent with the playback duration of the video to be learned, and the recording start time of the fourth video to be analyzed is consistent with the time when the video to be learned starts to be played on the test terminal; step S350 includes the following steps:

[0100] Step S510: The second processing module determines multiple preset time nodes in the video to be learned, wherein the interval between any two adjacent preset time nodes is a second preset time length (for example, 5 seconds), the first preset time node is the start time of the video to be learned, and the interval between the last preset time node and the end time of the video to be learned is less than the second preset time length.

[0101] Step S520: Whenever the video to be learned is played on the second display screen to any preset time node, the second processing module obtains the video frame image of the fourth video to be analyzed at the same preset time node, marks it as the third target frame image, thereby obtaining multiple third target frame images, wherein the i-th third target frame image is the video frame image in the fourth video to be analyzed corresponding to when the video to be learned is played to the i-th preset time node, and the number of third target frame images is consistent with the number of preset time nodes in the video to be learned, and satisfies: 1≤i≤K, K is the total number of preset time nodes in the video to be learned.

[0102] Specifically, the face image in each third target frame image can reflect the change in pupil position of the tester when the tester is carefully watching the video to be learned.

[0103] Step S530: The second processing module determines the fifth reference line and the sixth reference line of the third target frame image, wherein the straight line in the third target frame image that passes through the inner canthus point of the right eye and is parallel to the vertical midline of the face is the fifth reference line, and the straight line in the third target frame image that passes through the lowest point of the lower eyelid of the right eye and is perpendicular to the fifth reference line is the sixth reference line.

[0104] Step S540: the second processing module marks the point in the third target frame image whose distance from the fifth reference line is the third distance, whose distance from the sixth reference line is the fourth distance, and which is within the right eye as the first origin.

[0105] Step S550: The second processing module establishes a first polar coordinate system in the third target frame image, wherein the origin of the first polar coordinate system is the first origin, and the polar axis of the first polar coordinate system is a ray extending away from the inner canthus of the right eye and parallel to the sixth reference line.

[0106] Step S560: the second processing module obtains the polar coordinates of the center point of the right pupil in the third target frame image in the first polar coordinate system, and marks them as first polar coordinates. is the polar coordinate of the center point of the right pupil in the i-th third target frame image in the first polar coordinate system, is the polar diameter of the center point of the right pupil in the i-th third target frame image in the first polar coordinate system, is the polar angle of the center point of the right eye pupil in the i-th third target frame image in the first polar coordinate system.

[0107] Specifically, as the video to be learned is played, the tester watches the video to be learned carefully throughout the whole process, and will inevitably move his eyes according to the explanation content in the video to be learned, so as to follow the explanation of the key content in real time; then, a first polar coordinate system is established on the third target frame image, and the first origin is used as the origin of the first polar coordinate system; then the polar coordinates of the center point of the pupil of the right eye in the third target frame image (that is, the center point of the pupil of this tester) in the first polar coordinate system are obtained, and the polar coordinates can express the position deviation of the pupil of the tester relative to the initial moment (that is, when watching the perspective positioning video) at the moment corresponding to the third target frame image; thereby reflecting the eye position of the tester at this moment (specifically including the distance between the center point of the pupil and the first origin, and the angle between the center point of the pupil and the polar axis of the first polar coordinate system).

[0108] Step S570: the second processing module uses the preset time nodes in the video to be learned and the first polar coordinates of each third target frame image as standard pupil data corresponding to the video to be learned.

[0109] Specifically, the first polar coordinates of the third target frame image are used as standard pupil data corresponding to the video to be learned, and are subsequently compared with the actual pupil data of the user.

[0110] In a sixth embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, based on the fifth embodiment, the recording start time of the second video to be analyzed is consistent with the time when the video to be learned starts to be played on the education terminal; step S160 includes the following steps:

[0111] Step S610: Whenever the video to be learned is played on the first display screen to any preset time node, the first processing module obtains the video frame image of the second video to be analyzed at the same preset time node, marks it as the fourth target frame image, to obtain multiple fourth target frame images, wherein the i-th fourth target frame image is the video frame image in the fourth video to be analyzed corresponding to when the video to be learned is played to the i-th preset time node, and the number of fourth target frame images is consistent with the number of preset time nodes in the video to be learned.

[0112] Specifically, when the video to be learned is played on the first display screen, the number of fourth target frame images is consistent with the number of third target frame images. The face images in each fourth target frame image can reflect the changes in the pupil position of the user when actually watching the video to be learned.

[0113] Step S620: Whenever a new fourth target frame image is acquired, the first processing module determines the seventh reference line and the eighth reference line of the new fourth target frame image, wherein the straight line in the fourth target frame image that passes through the inner canthus point of the right eye and is parallel to the vertical midline of the face is the seventh reference line, and the straight line in the fourth target frame image that passes through the lowest point of the lower eyelid of the right eye and is perpendicular to the seventh reference line is the eighth reference line.

[0114] Step S630: the first processing module marks the distance between the new fourth target frame image and the seventh reference line as the first distance, the distance between the new fourth target frame image and the eighth reference line as the second distance, and the point in the right eye as the second origin.

[0115] Step S640: The first processing module establishes a second polar coordinate system in the new fourth target frame image, wherein the origin of the second polar coordinate system is the second origin, and the polar axis of the second polar coordinate system is a ray extending away from the inner canthus of the right eye and parallel to the eighth reference line.

[0116] Step S650: the first processing module obtains the polar coordinates of the center point of the right pupil in the new fourth target frame image in the second polar coordinate system, and marks them as second polar coordinates. is the polar coordinate of the center point of the right pupil in the i-th fourth target frame image in the second polar coordinate system, is the polar diameter of the center point of the right pupil in the i-th fourth target frame image in the second polar coordinate system, is the polar angle of the right eye pupil center point in the i-th fourth target frame image in the second polar coordinate system.

[0117] Specifically, as the video to be learned is played, the user watches the video to be learned. If the user's concentration is relatively concentrated, he will also move his eyes along with the explanation in the video to be learned, just like the tester, so as to follow the explanation of the key content in real time; then a second polar coordinate system is established on the fourth target frame image, and the second origin is used as the origin of the second polar coordinate system; then the polar coordinates of the center point of the right eye pupil in the fourth target frame image (that is, the center point of the user's pupil at this time) in the second polar coordinate system are obtained, and the polar coordinates can express the position deviation of the user's pupil at the moment corresponding to the fourth target frame image relative to the initial moment (that is, when watching the perspective positioning video); thereby reflecting the moment and the user's eye position (specifically including the distance between the center point of the pupil and the second origin, and the angle between the center point of the pupil and the polar axis of the second polar coordinate system).

[0118] Specifically, as attached Figure 3 As shown, attached Figure 3The dotted frame in is a local image of any fourth target frame image, in which a second polar coordinate system is established. From the figure, we can know the positional relationship between the center point of the right eye pupil (point B in the figure) and the second origin (point A in the figure).

[0119] Step S660: The first processing module calculates the user's current concentration score based on the second polar coordinates of the new fourth target frame image and the first polar coordinates of the third target frame image.

[0120] Specifically, the second polar coordinates of the fourth target frame image are compared with the aforementioned standard pupil data to determine the user's concentration; the specific principle is: if the second polar coordinates of the fourth target frame image corresponding to the same preset time point are closer to the first polar coordinates of the third target frame image, it proves that the user's eye movement data (angle and distance) is closer to the eye movement data of the tester who is carefully watching the video to be learned, it proves that the user is more serious at this moment, and the corresponding user's current concentration score is higher.

[0121] In the seventh embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, based on the sixth embodiment, the preset time node closest to the current moment is set as the kth preset time node, and 1≤k≤K is satisfied; step S660 includes the following steps:

[0122] Step S710: The first processing module calculates the user's current concentration score using formula (1):

[0123] (1),

[0124] In the formula, Indicates the user's current concentration score. The closer the concentration score is to 1, the higher the user's concentration when studying. represents the polar angle of the center point of the right pupil in the kth fourth target frame image in the second polar coordinate system; Represents the polar angle of the right eye pupil center point in the kth third target frame image in the first polar coordinate system.

[0125] Specifically, this embodiment provides a specific formula for calculating the concentration score. Because the sizes of the first display screen and the second display screen may be different, or the distance between the user and the first display screen when watching the video to be learned, and the distance between the tester and the second display screen when watching the video to be learned may be different, even at the same preset time node, the distance value of the first polar coordinate in the third target frame image corresponding to The distance value from the second polar coordinate in the fourth target frame image There may be differences, but the angle value of the first polar coordinate in the third target frame image corresponding to the same preset time node The angle value with the second polar coordinate in the fourth target frame image But they should be consistent (assuming the user watches the video to be learned carefully).

[0126] Therefore, the angle value of the second polar coordinate in the fourth target frame image corresponding to the same preset time node is The angle value with the first polar coordinate in the third target frame image The absolute value of the ratio is used as the concentration score, which can reflect the real-time concentration of the user when learning; the closer the concentration score is to 1, the closer the user's eye movement data (specifically, the eye movement angle) is to the eye movement data (specifically, the eye movement angle) of the tester who carefully watches the video to be learned, which proves that the user is more serious at that moment, indicating that the user has a higher concentration when learning.

[0127] In an eighth embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, based on the seventh embodiment, step S660 includes the following steps:

[0128] Step S810: The first processing module determines whether k is greater than a first preset value (eg, 4).

[0129] If yes, execute step S820: the first processing module calculates the user's current concentration score by formula (2) and formula (3):

[0130] (2),

[0131] (3),

[0132] In the formula, Indicates the user's current concentration score. The closer the concentration score is to 2, the higher the user's concentration is when studying. represents the polar diameter of the center point of the right pupil in the kth fourth target frame image in the second polar coordinate system; Represents the polar diameter of the right eye pupil center point in the kth third target frame image in the first polar coordinate system.

[0133] Specifically, if k is greater than 4, then k is at least 5, then the first polar coordinate and the polar diameter in the second polar coordinate (i.e., the distance between the center point of the right pupil and the first origin or the second origin) need to be considered when calculating the concentration score; because although the distance value of the second polar coordinate in the fourth target frame image corresponding to the same preset time node is The distance value from the first polar coordinate in the third target frame image There may be differences, but the distance values ​​of the second polar coordinates in the fourth target frame image corresponding to different preset time points The distance value from the first polar coordinate in the third target frame image The ratio of should be consistent, so when there are enough samples (that is, when k is greater than the first preset value in this embodiment), the calculation formula of the concentration score is corrected to formula (2) and formula (3). Formula (2) and formula (3) take into account the distance value of the second polar coordinate. Distance value from the first polar coordinate The ratio, under normal circumstances, The value of should be close to 1. The closer it is to 1, the closer the user's eye movement data (specifically, the distance between the current position of the pupil center and the second origin) is to the eye movement data of the tester who carefully watches the video to be learned (specifically, the distance between the current position of the pupil center and the first origin). This proves that the user is more serious at that moment, indicating that the user has a higher degree of concentration when learning.

[0134] If not, execute step S660.

[0135] Specifically, if k is not greater than 4, the sample is insufficient and it is difficult to obtain the accurate distance value of the second polar coordinate in the fourth target frame image corresponding to the same preset time point. The distance value from the first polar coordinate in the third target frame image Therefore, in order to ensure the accuracy of the concentration score, formula (1) is directly used to calculate the concentration score.

[0136] In the ninth embodiment of an online education supervision method based on artificial intelligence proposed by the present invention, based on the seventh embodiment, step S160 further includes the following steps:

[0137] Step S910: The first processing module obtains all concentration scores within a third preset time period (for example, 3 minutes) in the past and calculates the average value, which is used as the feedback score value.

[0138] Step S920: When the feedback score value is lower than a second preset value (eg, 0.5), the first processing module generates a second text message for reminding the user to concentrate, and displays the second text message on the first display screen.

[0139] Specifically, when the feedback score value is lower than the second preset value, it means that the user's concentration in the past 3 minutes is not high enough, and the user needs to be reminded to concentrate more, so as to supervise the user's online learning.

[0140] The present invention also proposes an online education supervision system based on artificial intelligence, which applies an online education supervision method based on artificial intelligence; the system includes an education terminal, and a cloud server communicatively connected to the education terminal; the education terminal includes a first processing module, and a first display screen and a first camera module communicatively connected to the first processing module.

[0141] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0142] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. An online education supervision method based on artificial intelligence, It is characterized in that Applicable to an online education supervision system based on artificial intelligence; the system includes an education terminal and a cloud server communicatively connected to the education terminal; the education terminal includes a first processing module, and a first display screen and a first camera module communicatively connected to the first processing module; the method includes: When the educational terminal is started, it receives the viewing angle positioning video from the cloud server, and plays the viewing angle positioning video through the first display screen for the user to watch; During the process of playing the viewing angle positioning video on the first display screen, the first camera module captures the user's face image video in real time and marks it as the first video to be analyzed; The first processing module determines the user's initial pupil data based on the view positioning video and the first video to be analyzed, wherein the initial pupil data is used to describe the pupil position when the user is gazing at the center of the first display screen; The cloud server obtains a video to be learned determined by a user through an education terminal, obtains standard pupil data corresponding to the video to be learned, and then sends the video to be learned and the corresponding standard pupil data to the education terminal, wherein the standard pupil data is used to express changes in pupil position during the process of carefully watching the video to be learned; The first processing module controls the first display screen to play the video to be learned, and during the playing process, obtains in real time a video of a user's face image captured in real time by the first camera module, and marks it as a second video to be analyzed; The first processing module generates in real time a concentration score for expressing the concentration of the user during the learning process based on the second video to be analyzed and the standard pupil data; When the viewing angle positioning video is played on the first display screen, the display content of the viewing angle positioning video includes an identification block at the center of the first display screen and a first text message for prompting a user to look at the identification block; the viewing angle positioning video and the first video to be analyzed have the same duration and are greater than a first preset duration; the first processing module determines the user's initial pupil data based on the viewing angle positioning video and the first video to be analyzed, including: The first processing module marks the last video frame image in the first video to be analyzed as a first target frame image; The first processing module determines a first reference line and a second reference line of the first target frame image, wherein a straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the first target frame image is the first reference line, and a straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the first reference line in the first target frame image is the second reference line; The first processing module marks the distance between the pupil center point in the first target frame image and the first reference line as the first distance, and marks the distance between the pupil center point in the first target frame image and the second reference line as the second distance; The first processing module uses the first distance and the second distance as initial pupil data of the user; The system further includes a test terminal connected to the cloud server in communication; the test terminal includes a second processing module, and a second display screen and a second camera module connected to the second processing module in communication; the cloud server stores a plurality of different videos to be learned; the method further includes: The cloud server sends the video to be learned and the view positioning video to the test terminal; The second display screen plays the viewing angle positioning video for the tester to watch, and during the viewing angle positioning video, the second camera module takes a real-time video of the tester's face image, and marks it as the third video to be analyzed; The second processing module determines the initial pupil data of the test person based on the perspective positioning video and the third video to be analyzed; The second display screen plays the video to be learned for the tester to watch and learn, and during the playing of the video to be learned, the second camera module takes a real-time video of the tester's face image and marks it as a fourth video to be analyzed, wherein the tester stays focused while watching and learning the video to be learned; The second processing module generates standard pupil data corresponding to the video to be learned based on the fourth video to be analyzed and the initial pupil data of the tester; The recording start time of the second video to be analyzed is consistent with the time when the video to be learned starts to be played on the education terminal; the first processing module generates a concentration score for expressing the concentration of the user in the learning process in real time based on the second video to be analyzed and the standard pupil data, including: Whenever the video to be learned is played on the first display screen to any preset time node, the first processing module obtains a video frame image of the second video to be analyzed at the same preset time node, marks it as a fourth target frame image, to obtain a plurality of fourth target frame images, wherein the i-th fourth target frame image is a video frame image in the fourth video to be analyzed corresponding to when the video to be learned is played to the i-th preset time node, and the number of the fourth target frame images is consistent with the number of preset time nodes in the video to be learned; Whenever a new fourth target frame image is acquired, the first processing module determines a seventh reference line and an eighth reference line of the new fourth target frame image, wherein a straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the fourth target frame image is the seventh reference line, and a straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the seventh reference line in the fourth target frame image is the eighth reference line; The first processing module marks the point in the new fourth target frame image whose distance from the seventh reference line is the first distance, whose distance from the eighth reference line is the second distance, and which is located in the right eye as the second origin; The first processing module establishes a second polar coordinate system in the new fourth target frame image, wherein the origin of the second polar coordinate system is the second origin, and the polar axis of the second polar coordinate system is a ray extending away from the inner canthus of the right eye and parallel to the eighth reference line; The second processing module determines the initial pupil data of the test person based on the perspective positioning video and the third video to be analyzed, including: The second processing module marks the last video frame image in the third video to be analyzed as a second target frame image; The second processing module determines a third reference line and a fourth reference line of the second target frame image, wherein a straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the second target frame image is the third reference line, and a straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the third reference line in the first target frame image is the fourth reference line; The second processing module marks the distance between the pupil center point and the third reference line in the second target frame image as a third distance, and marks the distance between the pupil center point and the fourth reference line in the second target frame image as a fourth distance; The second processing module uses the third distance and the fourth distance as initial pupil data of the test person; The recording duration of the fourth video to be analyzed is consistent with the playback duration of the video to be learned, and the recording start time of the fourth video to be analyzed is consistent with the time when the video to be learned starts to be played at the test terminal; the second processing module generates standard pupil data corresponding to the video to be learned based on the fourth video to be analyzed and the initial pupil data of the tester, including: The second processing module determines a plurality of preset time nodes in the video to be learned, wherein the interval between any two adjacent preset time nodes is the second preset time, the first preset time node is the start time of the video to be learned, and the interval between the last preset time node and the end time of the video to be learned is less than the second preset time; Whenever the video to be learned is played on the second display screen to any preset time node, the second processing module obtains a video frame image of the fourth video to be analyzed at the same preset time node, marks it as a third target frame image, thereby obtaining a plurality of third target frame images, wherein the i-th third target frame image is a video frame image in the fourth video to be analyzed corresponding to when the video to be learned is played to the i-th preset time node, and the number of the third target frame images is consistent with the number of preset time nodes in the video to be learned, and satisfies: 1≤i≤K, K is the total number of preset time nodes in the video to be learned; The second processing module determines a fifth reference line and a sixth reference line of the third target frame image, wherein a straight line passing through the inner canthus point of the right eye and parallel to the vertical midline of the face in the third target frame image is the fifth reference line, and a straight line passing through the lowest point of the lower eyelid of the right eye and perpendicular to the fifth reference line in the third target frame image is the sixth reference line; The second processing module marks the point in the third target frame image whose distance from the fifth reference line is the third distance, whose distance from the sixth reference line is the fourth distance, and which is located in the right eye as the first origin; The second processing module establishes a first polar coordinate system in the third target frame image, wherein the origin of the first polar coordinate system is the first origin, and the polar axis of the first polar coordinate system is a ray extending away from the inner canthus of the right eye and parallel to the sixth reference line; The second processing module obtains the polar coordinates of the center point of the right pupil in the third target frame image in the first polar coordinate system and marks them as first polar coordinates. is the polar coordinate of the center point of the right pupil in the i-th third target frame image in the first polar coordinate system, is the polar diameter of the center point of the right pupil in the i-th third target frame image in the first polar coordinate system, is the polar angle of the center point of the right eye pupil in the i-th third target frame image in the first polar coordinate system; The second processing module uses the preset time node in the video to be learned and the first polar coordinates of each third target frame image as the standard pupil data corresponding to the video to be learned; The first processing module establishes a second polar coordinate system in the new fourth target frame image, and then further includes: The first processing module obtains the polar coordinates of the center point of the right pupil in the new fourth target frame image in the second polar coordinate system and marks them as second polar coordinates. is the polar coordinate of the center point of the right pupil in the i-th fourth target frame image in the second polar coordinate system, is the polar diameter of the center point of the right pupil in the i-th fourth target frame image in the second polar coordinate system, is the polar angle of the center point of the right pupil in the i-th fourth target frame image in the second polar coordinate system; The first processing module calculates the user's current concentration score based on the second polar coordinates of the new fourth target frame image and the first polar coordinates of the third target frame image.

2. According to the online education supervision method based on artificial intelligence according to claim 1, It is characterized in that Let the preset time node closest to the current time be the kth preset time node, and satisfy 1≤k≤K; The first processing module calculates the user's current concentration score based on the second polar coordinates of the new fourth target frame image and the first polar coordinates of the third target frame image, including: The first processing module calculates the user's current concentration score by formula (1): （1）， In the formula, Indicates the user's current concentration score. The closer the concentration score is to 1, the higher the user's concentration when studying. represents the polar angle of the center point of the right pupil in the kth fourth target frame image in the second polar coordinate system; Represents the polar angle of the right eye pupil center point in the kth third target frame image in the first polar coordinate system.

3. According to the online education supervision method based on artificial intelligence according to claim 2, It is characterized in that The first processing module calculates the user's current concentration score through formula (1), which includes: The first processing module determines whether k is greater than a first preset value; If yes, the first processing module calculates the user's current concentration score using formula (2) and formula (3): （2）， （3）， In the formula, Indicates the user's current concentration score. The closer the concentration score is to 2, the higher the user's concentration is when studying. represents the polar diameter of the center point of the right pupil in the kth fourth target frame image in the second polar coordinate system; represents the polar diameter of the center point of the right pupil in the kth third target frame image in the first polar coordinate system; If not, the first processing module is executed to calculate the user's current concentration score by formula (1).

4. According to the online education supervision method based on artificial intelligence as claimed in claim 2, It is characterized in that The first processing module generates a concentration score for expressing the concentration of the user in the learning process in real time based on the second video to be analyzed and the standard pupil data, and then further includes: The first processing module obtains all concentration scores within the past third preset time period and calculates the average value, and uses the average value as the feedback score value; When the feedback score value is lower than a second preset value, the first processing module generates a second text message for reminding the user to concentrate, and displays the second text message on the first display screen.

5. An online education supervision system based on artificial intelligence, It is characterized in that An artificial intelligence-based online education supervision method as described in any one of claims 1 to 4 is applied; the system includes an education terminal, and a cloud server communicatively connected to the education terminal; the education terminal includes a first processing module, and a first display screen and a first camera module communicatively connected to the first processing module.

Images