Learning state detection method and apparatus, computer device, and storage medium

Abstract

The present scheme relates to a learning state detection method and device, computer equipment and a storage medium. The method comprises: acquiring learning video data of a learner in real time to obtain posture data of the learner; when the posture data corresponds to a learning state of the learner, determining a breathing determination region in the learning video data, and obtaining the breathing frequency of the learner according to the breathing determination region; collecting pupil data of the learner in the learning process in real time, and determining the visual range of the learner according to the pupil data; determining the current learning state of the learner according to the posture data, the breathing frequency and the visual range, and counting the state duration of the current learning state. The breathing detection and learning posture detection are performed through the learning video, the eye movement tracking is realized by collecting the pupil data, the learning state of the learner is detected, and the detection accuracy of the learning state can be improved. The duration of various learning states is calculated, and the learner can be self-driven.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a learning state detection method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of computer technology, more and more users are learning on computer devices via the internet, and these users come from all age groups. Traditional learning processes primarily rely on accumulating time and time management. However, a person's initiative and concentration often play a crucial and decisive role in the learning process. The detection of learning status mainly focuses on two aspects: the learner's fatigue level and attention level. To detect the learning status of each user, it is common practice to capture the user's learning posture and detect behaviors such as dozing off, thereby determining the user's learning status. For example, related methods for detecting learning status often involve recognizing facial images and determining whether the eyes are open and whether the person is awake to determine if the person corresponding to the face is in a learning state.

[0003] However, traditional learning state detection methods are inadequate and suffer from low detection accuracy. Summary of the Invention

[0004] Therefore, in order to solve the above-mentioned technical problems, a learning state detection method, apparatus, computer equipment and storage medium are provided, which can improve the detection accuracy of learning state.

[0005] A learning state detection method, the method comprising:

[0006] The system collects learner video data in real time and obtains learner posture data based on the video data.

[0007] When the learner state corresponding to the posture data is a learning state, the breathing determination region in the learning video data is determined, and the learner's breathing frequency is obtained based on the breathing determination region.

[0008] The learner's pupil data is collected in real time during the learning process, and the learner's visual range is determined based on the pupil data;

[0009] The learner's current learning state is determined based on the posture data, breathing frequency, and visual range, and the duration of the current learning state is calculated.

[0010] In one embodiment, after obtaining the learner's posture data based on the learning video data, the method further includes:

[0011] The learner's learning posture is determined based on the posture data;

[0012] When the learning posture is incorrect, a posture error message is issued, and the learner's state is determined to be non-learning state.

[0013] When the learning posture is the correct posture, the learner's state is determined to be a learning state.

[0014] In one embodiment, obtaining the learner's breathing frequency based on the breathing determination region includes:

[0015] The rise and fall of the human chest is determined based on the breathing determination area to obtain the determination result;

[0016] The determination result is subjected to digital signal processing and Fourier transform, and frequency domain analysis is performed to obtain the analysis result;

[0017] The analysis results are input into the respiratory rate algorithm calculation model to obtain the learner's respiratory rate.

[0018] In one embodiment, after obtaining the learner's respiratory rate based on the respiratory determination region, the method further includes:

[0019] Obtain the respiratory rate threshold;

[0020] When the breathing rate is greater than the breathing rate threshold, a learner tension prompt message is issued, and the learner's state is determined to be tense.

[0021] When the breathing rate is less than or equal to the breathing rate threshold, the learner's state is determined to be a learning state.

[0022] In one embodiment, the pupil data includes pupil position and Pulchin spot position;

[0023] Determining the learner's visual range based on the pupil data includes:

[0024] Calculate eye movement velocity based on the pupil position and the location of the Purkinje spot;

[0025] The learner's reading speed is calculated based on the eye movement speed, and the visual range is calculated based on the reading speed.

[0026] In one embodiment, the method further includes:

[0027] The learner's eye status is monitored in real time during the learning process to obtain eye monitoring results;

[0028] When the eye monitoring results include saccades, the saccade paradigm corresponding to the saccades is determined;

[0029] The learner's current learning state is determined based on the eye saccade paradigm.

[0030] In one embodiment, the step of calculating the duration of the current learning state includes:

[0031] Obtain the various current learning states and find the duration of each state corresponding to each different current learning state;

[0032] The duration of each state corresponding to the same current learning state is summed to obtain the state duration.

[0033] A learning state detection device, the device comprising:

[0034] The posture data acquisition module is used to collect learner's learning video data in real time and acquire the learner's posture data based on the learning video data.

[0035] The breathing determination module is used to determine the breathing determination region in the learning video data when the learner state corresponding to the posture data is a learning state, and to obtain the learner's breathing frequency based on the breathing determination region.

[0036] The pupil data acquisition module is used to collect the learner's pupil data in real time during the learning process and determine the learner's visual range based on the pupil data.

[0037] The state duration statistics module is used to determine the learner's current learning state based on the posture data, the breathing frequency, and the visual range, and to count the state duration of the current learning state.

[0038] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0039] The system collects learner video data in real time and obtains learner posture data based on the video data.

[0040] When the learner state corresponding to the posture data is a learning state, the breathing determination region in the learning video data is determined, and the learner's breathing frequency is obtained based on the breathing determination region.

[0041] The learner's pupil data is collected in real time during the learning process, and the learner's visual range is determined based on the pupil data;

[0042] The learner's current learning state is determined based on the posture data, breathing frequency, and visual range, and the duration of the current learning state is calculated.

[0043] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0044] The system collects learner video data in real time and obtains learner posture data based on the video data.

[0045] When the learner state corresponding to the posture data is a learning state, the breathing determination region in the learning video data is determined, and the learner's breathing frequency is obtained based on the breathing determination region.

[0046] The learner's pupil data is collected in real time during the learning process, and the learner's visual range is determined based on the pupil data;

[0047] The learner's current learning state is determined based on the posture data, breathing frequency, and visual range, and the duration of the current learning state is calculated.

[0048] The aforementioned learning state detection method, apparatus, computer equipment, and storage medium acquire learner's learning video data in real time and obtain the learner's posture data based on the learning video data. When the learner's state corresponding to the posture data is a learning state, a breathing determination area in the learning video data is identified, and the learner's breathing frequency is obtained based on the breathing determination area. Pupil data of the learner during the learning process is acquired in real time, and the learner's visual range is determined based on the pupil data. The learner's current learning state is determined based on the posture data, breathing frequency, and visual range, and the duration of the current learning state is calculated. By detecting breathing and learning posture through learning videos and achieving eye tracking through pupil data acquisition, the learner's learning state can be detected, improving the accuracy of learning state detection. Calculating the duration of various learning states facilitates learner self-motivation, thereby forming an efficient learning state. Attached Figure Description

[0049] Figure 1 This is a diagram illustrating the application environment of the learning state detection method in one embodiment;

[0050] Figure 2 This is a flowchart illustrating a learning state detection method in one embodiment;

[0051] Figure 3 This is a flowchart illustrating the learning state detection method in another embodiment;

[0052] Figure 4 This is a structural block diagram of a learning state detection device in one embodiment;

[0053] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] The learning state detection method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1 As shown, the application environment includes a computer device 110, which may be equipped with a camera. The computer device 110 can collect learner's learning video data in real time and obtain the learner's posture data based on the learning video data. When the learner's state corresponding to the posture data is a learning state, the computer device 110 can determine the breathing detection area in the learning video data and obtain the learner's breathing rate based on the breathing detection area. The computer device 110 can also collect the learner's pupil data in real time during the learning process and determine the learner's visual range based on the pupil data. The computer device 110 can determine the learner's current learning state based on the posture data, breathing rate, and visual range, and calculate the duration of the current learning state. The computer device 110 can be, but is not limited to, various personal computers, laptops, smartphones, robots, tablets, and other devices.

[0056] In one embodiment, such as Figure 2 As shown, a learning state detection method is provided, including the following steps:

[0057] Step 202: Collect learner's learning video data in real time, and obtain learner's posture data based on the learning video data.

[0058] Computer devices can be intelligent hardware terminals with NPU capabilities. The Neural Network Processing Unit (NPU) adopts a "data-driven parallel computing" architecture, which can perform multiplication and addition operations in parallel. It can decompose many operation methods into multiplication and addition algorithms, such as convolution, dot product, matrix operation, digital filter operation, and even polynomial evaluation, thereby greatly improving the computational efficiency through parallelism.

[0059] The computer device can be equipped with a camera to collect learners' video data in real time. The device can also include a posture recognition system that uses a posture recognition algorithm to detect and correct poor posture in learners. This algorithm can extract posture features from the camera-captured video data using OpenCV, then extract them using OpenPose, and finally train the dataset using a convolutional neural network built with the Keras deep learning framework. The computer device can then use this trained algorithm to identify the learner's posture, thus obtaining the learner's posture data.

[0060] Step 204: When the learner state corresponding to the posture data is the learning state, determine the breathing determination area in the learning video data, and obtain the learner's breathing frequency based on the breathing determination area.

[0061] Different learner posture data can correspond to different learning states. Computer devices can determine whether a learner is in the correct learning posture. If the posture is incorrect, the learner's state can be a distracted state; if the posture is correct, the learner's state can be either a deep learning state or a shallow learning state, both of which are considered learning states.

[0062] When posture data corresponds to a learning state, the computer device can use a posture recognition algorithm to help delineate the chest area where breathing fluctuations are most pronounced, designating this chest area as the breathing determination region. Then, the computer device can obtain the learner's breathing rate based on this region. In this embodiment, in addition to using a camera on the computer device for breathing determination, millimeter-wave radar can also be used as a tool for breathing determination and breathing rate calculation.

[0063] Step 206: Collect pupil data of learners in real time during the learning process, and determine the learner's visual range based on the pupil data.

[0064] Since the position of the camera on the computer device is fixed, and the position of the book the learner is studying is also fixed, the visual range can be determined by the learner's eye movements. In this embodiment, the computer device can collect the learner's pupil data to determine the visual range.

[0065] Step 208: Determine the learner's current learning state based on posture data, breathing rate, and visual range, and calculate the duration of the current learning state.

[0066] The learner's current learning state can be a distracted state, a deep learning state, or a shallow learning state. The computer device can determine the learner's current learning state based on posture data, breathing frequency, and visual range. For example, when the learner's posture data is abnormal, or when the learner's visual range is outside the reasonable range, the current learning state is a distracted state.

[0067] During a learning period, learners may experience various different learning states, and computer devices can track the learning time corresponding to each current learning state.

[0068] In this embodiment, the computer device acquires learner's learning video data in real time and obtains learner's posture data based on the learning video data. When the learner's state corresponding to the posture data is a learning state, the breathing judgment area in the learning video data is determined, and the learner's breathing frequency is obtained based on the breathing judgment area. The device also acquires learner's pupil data in real time during the learning process and determines the learner's visual range based on the pupil data. Based on posture data, breathing frequency, and visual range, the learner's current learning state is determined, and the duration of the current learning state is calculated. By detecting breathing and learning posture through the learning video and achieving eye tracking through pupil data acquisition, the learner's learning state can be detected, improving the accuracy of learning state detection. The calculation of the duration of various learning states facilitates learner self-motivation, thereby forming an efficient learning state.

[0069] In one embodiment, the provided learning state detection method may further include a posture error prompting process, specifically including: determining the learner's learning posture based on posture data; when the learning posture is an incorrect posture, issuing a posture error prompt message and determining the learner's state as a non-learning state; when the learning posture is a correct posture, determining the learner's state as a learning state.

[0070] When the learner's posture data indicates that the learner's posture is incorrect, the computer device can issue a posture error prompt to remind the learner to pay attention to their posture. At the same time, the computer device can determine that the learner with incorrect posture is in a non-learning state, which may be a state of distraction.

[0071] In one embodiment, a learning state detection method may further include a process of calculating respiratory rate, specifically including: determining the rise and fall of the human chest based on the respiratory determination area to obtain a determination result; performing digital signal processing and Fourier transform on the determination result, and performing frequency domain analysis to obtain an analysis result; and inputting the analysis result into the respiratory rate algorithm calculation model to obtain the learner's respiratory rate.

[0072] Breathing changes during the learning process are somewhat correlated with the depth of learning. Learners' subconscious breath-holding and exhalation can serve as key characteristic indicators of the learning process. Computer equipment can normalize the learning video data based on the regular fluctuations in the chest area (the breathing detection region), calculate the breathing frequency model using a Fast Fourier Transform algorithm, and use the dominant frequency as a reference for the learner's breathing frequency data.

[0073] In one embodiment, a learning state detection method may further include a process of alerting the learner to be tense, specifically including: acquiring a respiratory rate threshold; when the respiratory rate is greater than the respiratory rate threshold, issuing a learner tension alert and determining that the learner is in a tense state; when the respiratory rate is less than or equal to the respiratory rate threshold, determining that the learner is in a learning state.

[0074] The respiratory rate threshold can be 20 breaths per minute. When the respiratory rate exceeds the threshold, it indicates that the learner is relatively tense and breathing rapidly. At this time, the computer device can issue a tension alert to help the learner relax.

[0075] In one embodiment, a learning state detection method may further include a process for determining the learner's visual range, specifically including: calculating eye movement speed based on pupil position and Purkinje spot position; calculating the learner's reading speed based on eye movement speed; and calculating the visual range based on reading speed.

[0076] The pupil data can include pupil position and Purkinje image position. In this embodiment, the computer device can employ pupil-corneal reflection spot recognition technology. Utilizing the characteristic that eye movements remain constant, the Purkinje image, a bright spot on the outer surface of the cornea, is produced by the reflection (CR) of light entering the pupil on the outer surface of the cornea (assuming the eyeball is spherical and the head remains stationary). Therefore, the relative position of the pupil's center and the light spot will naturally track the eye movement speed during reading. This allows the learner's focused learning duration to be determined based on the reasonable reading speed. Furthermore, clockwise and counterclockwise eye movements can be used to determine whether the learner is recalling or thinking. Both recalling and thinking are deep learning processes, indicating that the learner's learning state is a deep learning state.

[0077] Specifically, because the positions of the camera, the learning light source, and the center of the eyeball are fixed on the computer device, the absolute position of the Pulcim spot does not change with eye movement. However, its position relative to the pupil and the eyeball is constantly changing—for example, when staring at the light source, the Pulcim spot is between the pupils; and when thinking requires the pupil to be tilted upwards, the Pulcim spot is positioned relatively downwards from the pupil. Therefore, by real-time positioning of the pupil and Pulcim spot on the eye image, simulating the reading speed based on the natural rate of pupil rotation, and considering the duration and orientation of pupil-target switching, it is possible to determine whether the learner is distracted.

[0078] In one embodiment, a learning state detection method may further include a process of detecting saccades, specifically including: real-time monitoring of the learner's eye state during the learning process to obtain eye monitoring results; when the eye monitoring results contain saccades, determining the saccade paradigm corresponding to the saccades; and determining the learner's current learning state based on the saccade paradigm.

[0079] Besides pupillary fixation, saccades are another behavior related to human thought. Fixation refers to the activity of aligning the fovea of ​​the eye with a target stimulus. When fixing on a stationary target, three types of micro-movement saccades still occur: drift, nystagmus, and involuntary saccades. Drift of the visual axis is an irregular, slow change in the visual axis. Involuntary micro-saccades are induced when fixation on a point on a stationary object exceeds a certain time, approximately 300–500 ms, or when the image of the fixation point on the retina tends to move away from the fovea due to drift. Generally, most fixations are accompanied by drift and nystagmus. When fixing on a stationary object for a prolonged period, drift, nystagmus, and micro-involuntary saccades occur simultaneously. Saccades are rapid shifts in the fixation point, thus keeping the object to be fixed in the fovea region of the retina, thereby obtaining clear vision.

[0080] Saccades have two main characteristics: first, saccades in both eyes are almost synchronized; second, they occur very quickly. Generally, a saccade of 5–20° horizontally takes 35–96 ms; a saccade of 5–20° vertically takes 35–108 ms. In daily life, there are approximately three saccades per second. Research indicates that saccades are significantly influenced by various cognitive factors, including attention, working memory, learning, long-term memory, and decision-making. Therefore, many researchers believe that the saccade system provides a powerful tool for exploring the cognitive control mechanisms of behavior and have designed saccade paradigms targeting different cognitive functions, such as visually guided saccades, predictive saccades, and memory-guided saccades. When a test subject is in a normal eye-tracking state and exhibits both predictive and memory-guided saccades regularly, the behavior is classified as a deep learning state; conversely, a regular saccade accompanied by a visually guided saccade paradigm indicates a shallow learning state.

[0081] In one embodiment, a learning state detection method may further include a process of calculating the duration of a state. The specific process includes: obtaining each different current learning state and finding the duration of each state corresponding to each different current learning state; and adding up the durations of each state corresponding to the same current learning state to obtain the state maintenance duration.

[0082] Learners will experience various learning states during the learning process. Computer devices can identify and time the learner's learning state in real time. Finally, by adding up the duration of the same learning state, the duration of each learning state can be obtained, which facilitates more reasonable, energy-management-based control of the learning state and thus forms an efficient learning state.

[0083] By analyzing learners' periods of deep learning, course content requiring in-depth study can be prioritized during these peak times, achieving twice the learning results with half the effort. Meanwhile, during periods of shallow learning, it's suitable to schedule open-ended reading. Furthermore, the study can measure students' self-motivated learning in specific subjects and assess whether effective external environments, such as musical accompaniment, can appropriately enhance learning depth, thus fostering a more personalized and efficient learning style.

[0084] In one embodiment, such as Figure 3 As shown, a learning state detection method is provided, which specifically includes:

[0085] Computer devices can capture learners' learning video data in real time via cameras;

[0086] Next, the computer device acquires the learner's posture data based on the learning video data;

[0087] The computer device can determine whether the learner is in the correct learning posture by using posture data. If not, it will remind the learner to maintain the correct learning posture and identify the learner's state as a distracted state, and record the duration of the distracted state.

[0088] If the learner is in the correct learning posture, the computer device can further determine the breathing judgment area in the learning video data and obtain the learner's breathing frequency based on the breathing judgment area.

[0089] When the respiratory rate exceeds the respiratory rate threshold, a learner tension cue message is issued, and the learner's state is determined to be tense.

[0090] When the breathing rate is the correct breathing rate, the computer equipment can determine the learner's visual range based on pupil data and define a reasonable eye movement range.

[0091] When the eye movement range is outside the reasonable eye movement range, the computer device can issue a distraction reminder, identify the learner's state as distraction, and record the duration of the distraction state;

[0092] When the eye movement range is within a reasonable range, the computer device can determine the learner's state as a shallow learning state based on the eye movement tracking status and record the duration of the shallow learning state.

[0093] At the same time, when the eye movement range is within a reasonable range, the computer device can determine the eye saccade state, thereby identifying the learner's state as a deep learning state and recording the duration of the deep learning state.

[0094] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0095] In one embodiment, such as Figure 4 As shown, a learning state detection device is provided, including: a posture data acquisition module 410, a breathing determination module 420, a pupil data acquisition module 430, and a state duration statistics module 440, wherein:

[0096] The posture data acquisition module 410 is used to collect learner's learning video data in real time and obtain learner's posture data based on the learning video data.

[0097] The breathing determination module 420 is used to determine the breathing determination area in the learning video data when the learner state corresponding to the posture data is the learning state, and to obtain the learner's breathing frequency based on the breathing determination area.

[0098] The pupil data acquisition module 430 is used to collect the learner's pupil data in real time during the learning process and determine the learner's visual range based on the pupil data.

[0099] The state duration statistics module 440 is used to determine the learner's current learning state based on posture data, breathing frequency, and visual range, and to count the duration of the current learning state.

[0100] In one embodiment, the posture data acquisition module 410 is further configured to determine the learner's learning posture based on the posture data; when the learning posture is an incorrect posture, a posture error prompt message is issued, and the learner's state is determined to be a non-learning state; when the learning posture is a correct posture, the learner's state is determined to be a learning state.

[0101] In one embodiment, the breathing determination module 420 is further configured to determine the rise and fall of the human chest based on the breathing determination area and obtain a determination result; perform digital signal processing and Fourier transform on the determination result and perform frequency domain analysis to obtain an analysis result; and input the analysis result into the breathing frequency algorithm calculation model to obtain the learner's breathing frequency.

[0102] In one embodiment, the breathing determination module 420 is further configured to obtain a breathing frequency threshold; when the breathing frequency is greater than the breathing frequency threshold, a learner tension prompt message is issued, and the learner's state is determined to be a tension state; when the breathing frequency is less than or equal to the breathing frequency threshold, the learner's state is determined to be a learning state.

[0103] In one embodiment, the pupil data acquisition module 430 is further configured to calculate eye movement velocity based on pupil position and Purkinje spot position; calculate the learner's reading speed based on eye movement velocity; and calculate the visual range based on reading speed.

[0104] In one embodiment, the pupil data acquisition module 430 is also used to monitor the learner's eye state in real time during the learning process and obtain eye monitoring results; when the eye monitoring results include saccades, the saccade paradigm corresponding to the saccades is determined; and the learner's current learning state is determined based on the saccade paradigm.

[0105] In one embodiment, the state duration statistics module 440 is further used to obtain each different current learning state and find the state duration corresponding to each different current learning state; and add up the state durations corresponding to the same current learning state to obtain the state maintenance duration.

[0106] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a learning state detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0107] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0108] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0109] Real-time acquisition of learners' learning video data, and acquisition of learners' posture data based on the learning video data;

[0110] When the learner state corresponding to the posture data is the learning state, determine the breathing judgment area in the learning video data, and obtain the learner's breathing frequency based on the breathing judgment area.

[0111] Real-time collection of learners' pupil data during the learning process, and determination of learners' visual range based on pupil data;

[0112] The learner's current learning state is determined based on posture data, breathing rate, and visual range, and the duration of the current learning state is calculated.

[0113] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the learner's learning posture based on posture data; when the learning posture is an incorrect posture, issuing a posture error message and determining the learner's state as a non-learning state; when the learning posture is a correct posture, determining the learner's state as a learning state.

[0114] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the rise and fall of the human chest based on the breathing determination area to obtain a determination result; performing digital signal processing and Fourier transform on the determination result, and performing frequency domain analysis to obtain an analysis result; inputting the analysis result into the respiratory frequency algorithm calculation model to obtain the learner's respiratory frequency.

[0115] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining a respiratory rate threshold; when the respiratory rate is greater than the respiratory rate threshold, issuing a learner tension prompt message and determining that the learner's state is a tension state; when the respiratory rate is less than or equal to the respiratory rate threshold, determining that the learner's state is a learning state.

[0116] In one embodiment, the pupil data includes pupil position and Pulchin spot position; when the processor executes the computer program, it also performs the following steps: calculating eye movement speed based on pupil position and Pulchin spot position; calculating the learner's reading speed based on eye movement speed; and calculating the visual range based on reading speed.

[0117] In one embodiment, when the processor executes the computer program, it further implements the following steps: real-time monitoring of the learner's eye state during the learning process to obtain eye monitoring results; when the eye monitoring results contain saccades, determining the saccade paradigm corresponding to the saccades; and determining the learner's current learning state based on the saccade paradigm.

[0118] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining each different current learning state and finding the duration of each state corresponding to each different current learning state; adding up the durations of each state corresponding to the same current learning state to obtain the state maintenance duration.

[0119] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0120] Real-time acquisition of learners' learning video data, and acquisition of learners' posture data based on the learning video data;

[0121] When the learner state corresponding to the posture data is the learning state, determine the breathing judgment area in the learning video data, and obtain the learner's breathing frequency based on the breathing judgment area.

[0122] Real-time collection of learners' pupil data during the learning process, and determination of learners' visual range based on pupil data;

[0123] The learner's current learning state is determined based on posture data, breathing rate, and visual range, and the duration of the current learning state is calculated.

[0124] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the learner's learning posture based on posture data; when the learning posture is an incorrect posture, issuing a posture error message and determining the learner's state as a non-learning state; when the learning posture is a correct posture, determining the learner's state as a learning state.

[0125] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the rise and fall of the human chest based on the breathing determination area to obtain a determination result; performing digital signal processing and Fourier transform on the determination result, and performing frequency domain analysis to obtain an analysis result; inputting the analysis result into the respiratory rate algorithm calculation model to obtain the learner's respiratory rate.

[0126] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining a respiratory rate threshold; when the respiratory rate is greater than the respiratory rate threshold, issuing a learner tension prompt message and determining that the learner's state is a tension state; when the respiratory rate is less than or equal to the respiratory rate threshold, determining that the learner's state is a learning state.

[0127] In one embodiment, the pupil data includes pupil position and Pulchin spot position; when the computer program is executed by the processor, it also performs the following steps: calculating eye movement speed based on pupil position and Pulchin spot position; calculating the learner's reading speed based on eye movement speed; and calculating the visual range based on reading speed.

[0128] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: real-time monitoring of the learner's eye state during the learning process to obtain eye monitoring results; when the eye monitoring results contain saccades, determining the saccade paradigm corresponding to the saccades; and determining the learner's current learning state based on the saccade paradigm.

[0129] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining each different current learning state and finding the duration of each state corresponding to each different current learning state; adding up the durations of each state corresponding to the same current learning state to obtain the state maintenance duration.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0131] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A learning state detection method, characterized in that, The method includes: The system collects learner video data in real time and obtains learner posture data based on the video data; it determines the learner's learning posture based on the posture data; when the learning posture is incorrect, it issues a posture error prompt and determines that the learner is in a non-learning state; when the learning posture is correct, it determines that the learner is in a learning state. When the learner state corresponding to the posture data is a learning state, the breathing determination region in the learning video data is determined, and the learner's breathing frequency is obtained based on the breathing determination region, and a breathing frequency threshold is obtained. The system collects pupil data of the learner in real time during the learning process. The pupil data includes pupil position and Pulchin spot position. When the respiratory rate is less than or equal to the respiratory rate threshold, the system determines the learner's visual range based on the pupil data, including: calculating eye movement velocity based on the pupil position and Pulchin spot position; calculating the learner's reading speed based on the eye movement velocity; and calculating the visual range based on the reading speed. The system also includes: real-time monitoring of the learner's eye state during the learning process to obtain eye monitoring results; when the visual range is within a reasonable visual range, determining whether the eye monitoring results contain saccades; when the eye monitoring results contain saccades, determining the saccade paradigm corresponding to the saccades; and determining the learner's current learning state based on the saccade paradigm. The learner's current learning state is determined based on the posture data, breathing frequency, and visual range, and the duration of the current learning state is calculated.

2. The learning state detection method according to claim 1, characterized in that, The step of obtaining the learner's breathing rate based on the breathing determination region includes: The rise and fall of the human chest is determined based on the breathing determination area to obtain the determination result; The determination result is subjected to digital signal processing and Fourier transform, and frequency domain analysis is performed to obtain the analysis result; The analysis results are input into the respiratory rate algorithm calculation model to obtain the learner's respiratory rate.

3. The learning state detection method according to claim 1, characterized in that, After obtaining the learner's respiratory rate based on the respiratory determination region, the method further includes: When the breathing rate is greater than the breathing rate threshold, a learner tension prompt message is issued, and the learner's state is determined to be tense. When the breathing rate is less than or equal to the breathing rate threshold, the learner's state is determined to be a learning state.

4. The learning state detection method according to claim 1, characterized in that, The duration of the current learning state is statistically analyzed, including: Obtain the various current learning states and find the duration of each state corresponding to each different current learning state; The duration of each state corresponding to the same current learning state is summed to obtain the state duration.

5. A learning state detection device, characterized in that, The device includes: The posture data acquisition module is used to collect learner's learning video data in real time, and acquire learner's posture data based on the learning video data; determine learner's learning posture based on posture data; when the learning posture is an incorrect posture, issue posture error prompt information and determine learner's state as non-learning state; when the learning posture is a correct posture, determine learner's state as learning state. The breathing determination module is used to determine the breathing determination region in the learning video data when the learner state corresponding to the posture data is a learning state, and to obtain the learner's breathing frequency based on the breathing determination region and obtain a breathing frequency threshold. A pupil data acquisition module is used to acquire pupil data of the learner in real time during the learning process. The pupil data includes pupil position and Pultchin spot position. When the respiratory rate is less than or equal to the respiratory rate threshold, the module determines the learner's visual range based on the pupil data, including: calculating eye movement velocity based on the pupil position and Pultchin spot position; calculating the learner's reading speed based on the eye movement velocity; and calculating the visual range based on the reading speed. The module also includes: real-time monitoring of the learner's eye state during the learning process to obtain eye monitoring results; when the visual range is within a reasonable visual range, determining whether the eye monitoring results contain saccades; when the eye monitoring results contain saccades, determining the saccade paradigm corresponding to the saccades; and determining the learner's current learning state based on the saccade paradigm. The state duration statistics module is used to determine the learner's current learning state based on the posture data, the breathing frequency, and the visual range, and to count the state duration of the current learning state.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images