A job concentration supervision method and system based on random triggering and multi-modal verification

Abstract

The application discloses a kind of based on random trigger and multi-modal verification's job concentration supervision method and system.Method includes: after starting supervision mode, multiple shooting moments are randomly generated in preset period;In each shooting moment, job area behavior image sequence is collected, and random verification symbol is output simultaneously, verification data containing user response information is collected;Whether the user response is consistent with the verification symbol is judged by identification model, whether learning related feature exists in behavior image sequence is simultaneously identified;If verification fails or continuous multiple times are missing learning features, it is determined as "non-concentrated state", and real-time push reminder is pushed to specified terminal, and concentration report is generated simultaneously.The application realizes real, effective concentration supervision by random trigger and multi-modal verification mechanism, has strong anti-cheating ability, and can work with intelligent tutoring system, and builds solid foundation for intelligent learning ecology.

Description

Technical Field

[0001] This invention relates to the field of intelligent education hardware technology, and more specifically, to a method and system for monitoring student focus during assignments on intelligent devices. Background Technology

[0002] Currently, insufficient concentration among primary and secondary school students during homework is a common problem troubling many parents. Existing study lamps on the market mainly provide auxiliary functions such as lighting, fingertip word lookup, and video calls, lacking effective supervision of students' homework-writing process. While some products have simple timer functions, they only record time and cannot determine whether students are truly focused on learning. Existing patent applications propose abstract concepts of learning task supervision and management, but they do not disclose specific, implementable technical means, particularly lacking effective technical solutions for "whether students are truly engaged in learning" and "how to prevent students from cheating." Therefore, there is an urgent need for an intelligent method and system that can truly and effectively supervise students' concentration during homework and has anti-cheating capabilities. Summary of the Invention

[0003] The technical problem to be solved by this invention is to address the shortcomings of existing technologies in effectively and realistically monitoring students' concentration during homework and in lacking anti-cheating measures, by providing a homework concentration monitoring method and system based on random triggering and multimodal verification, which can be applied to smart devices.

[0004] The technical solution adopted by this invention to solve its technical problem is: to construct a method for monitoring job attention based on random triggering and multimodal verification, including the following steps:

[0005] S1: Supervision mode is activated, responding to user operation commands or preset timed tasks, and real-time image data of the work area is collected through the camera set on the smart device;

[0006] S2: Randomly trigger sampling. Within a preset supervision period, multiple non-fixed shooting times are dynamically generated through a random number generation algorithm. At each shooting time, the camera is controlled to capture a frame of behavioral image sequence of the current working area.

[0007] S3: Real-time anti-cheating verification. While executing step S2, the output device of the intelligent device is controlled to randomly output a verification symbol, and the acquisition device is controlled to synchronously acquire verification data containing user response information. The verification data is then identified by an identification model to determine whether the user response information matches the verification symbol.

[0008] S4: Focused state recognition, inputting the behavior image sequence into a pre-trained image recognition model to detect whether there are preset learning-related features in the image; the learning-related features include at least one or more of the following: notebook, handwriting, and writing hand.

[0009] S5: Status determination and intervention. If the response information and verification symbol are inconsistent in step S3, or the learning-related features are not detected for a preset number of consecutive times in step S4, the user is determined to be in a "distracted state".

[0010] S6: Real-time notification and report generation. In response to the determination of "non-focused state", an alert message is immediately generated and pushed to a preset designated terminal via wireless network. At the same time, based on all determination results during the entire monitoring period, a focus report is generated and pushed to the designated terminal periodically.

[0011] Furthermore, in a preferred embodiment of the present invention, the method is applied to an intelligent learning desk lamp. In step S2, a true random number generator is used to generate unpredictable shooting times, which can generate random trigger times based on time intervals following a Poisson distribution, fundamentally preventing students from cheating by predicting the shooting time.

[0012] Furthermore, in step S3, the verification methods include gesture verification and voice verification. During gesture verification, the display screen randomly shows verification symbols (such as the number "3" or the geometric shape "○"), and the camera simultaneously captures images of the user's hand gestures, which are then compared using a gesture recognition model. During voice verification, the speaker randomly plays verification words (such as the number "5" or the word "apple"), and the microphone simultaneously captures the user's repeated speech, which is then compared using a voice recognition model. The two verification methods can be used individually or in combination to improve the reliability of anti-cheating measures.

[0013] Furthermore, the image recognition model in step S4 adopts a lightweight MobileNet-SSD architecture and is trained through transfer learning using a sample set containing different writing postures, different lighting conditions, and different types of exercise books, so as to achieve efficient and accurate local inference on edge devices.

[0014] Furthermore, this invention also provides an intelligent learning desk lamp system that implements the above-mentioned method. Its hardware structure includes a lamp body, a camera module, a microphone module, a display screen, a control motherboard, and a speaker. As a preferred hardware solution, the camera module employs two fixed cameras. The first camera is fixedly mounted on the top of the lamp body with its lens pointing vertically downwards, used for continuously capturing images of the work area. The second camera is fixedly mounted on the upper part of the lamp post with its lens facing the user, used for capturing the user's upper body and gestures. The two cameras are functionally decoupled, requiring no mechanical movement, resulting in lower cost and noiseless operation, without disturbing the user. As an alternative, the camera module can also employ a single rotatable camera, driven by a micro-motor, allowing adjustment of the shooting angle to cover different areas.

[0015] Furthermore, the system described in this invention can work in conjunction with another patent application filed on the same day by the applicant (invention title: A Multimodal Limited-Time Heuristic Homework Tutoring Method and System). The focus data identified by the monitoring module can serve as the basis for triggering or adjusting tutoring strategies. For example, when it is detected that a user is frequently distracted, the number of tutoring sessions available that day can be appropriately reduced to encourage the user to improve their focus.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] 1. Achieve real and effective supervision: By using a random-triggered photo-taking mechanism, the shortcomings of fixed-time interval supervision are overcome, making it impossible for students to predict the supervision time, thereby obtaining more realistic data on their concentration status.

[0018] 2. Possesses strong anti-cheating capabilities: It innovatively introduces a method that combines random verification symbols with multimodal verification (gestures / voice), effectively preventing students from deceiving the system by using pre-stored photos, staged photos of books, or recording playback, thus ensuring the authenticity and credibility of the monitoring data.

[0019] 3. Designated terminals participate in real time to form a closed-loop supervision system: The "inattentive state" reminder is pushed to the designated terminal in real time through the wireless network, and periodic attention reports are generated, so that parents or teachers can intervene in a timely manner and understand the child's learning habits in the long term, thus building a dual guarantee system of "device supervision - remote reminder".

[0020] 4. Controllable hardware costs and easy promotion: Flexible hardware solutions such as dual fixed cameras or single rotatable camera can be adopted, which effectively controls product costs while achieving powerful functions, thus facilitating market promotion.

[0021] 5. Collaborate with tutoring systems to build a complete learning ecosystem: Data interfaces are reserved for intelligent tutoring systems, laying the technical foundation for building a complete intelligent learning ecosystem that integrates "supervision, tutoring, and correction". Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the structure of the intelligent study lamp according to an embodiment of the present invention (single rotating camera scheme).

[0023] Figure 2 This is a flowchart of the task focus monitoring method based on random triggering and multimodal verification according to the present invention.

[0024] Figure 3 This is a schematic diagram of the interface for a specified terminal to receive reminder information in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the gesture verification process in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of the dual-fixed-camera smart learning desk lamp structure according to Embodiment 4 of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Example 1: Surveillance System Based on a Single Rotatable Camera

[0029] Reference Figure 1 As shown, the intelligent study desk lamp of this embodiment includes a lamp body (1), a rotatable camera module (2) set on the top of the lamp head, a touch screen (3) embedded in the front of the lamp head, a built-in control motherboard (4), a speaker (5), and a microphone module (6). The camera module (2) is driven by a micro stepper motor and can rotate in the vertical direction within a range of 0-90 degrees.

[0030] Reference Figure 2 The flowchart shown illustrates the working process of this embodiment:

[0031] Parents can set the daily monitoring period from 19:00 to 21:00 via WeChat mini-program and activate the monitoring mode. The system's built-in random number generator will randomly generate 20 shooting times within this period, such as 3 minutes 15 seconds, 7 minutes 42 seconds, 12 minutes 08 seconds, etc.

[0032] At 3 minutes and 15 seconds, the system first controls the display screen (3) to randomly display the number "3", and at the same time controls the camera (2) to rotate downwards by 90 degrees to take a picture of the work area (the image to be inspected), and simultaneously takes a picture containing the user's hand posture (the verification image). The entire acquisition process is completed within 0.5 seconds.

[0033] Subsequently, the system uploads the verification image to the cloud server (or performs local edge computing), calls the gesture recognition model based on MediaPipe, recognizes the gesture as "3", which matches the displayed verification symbol, and the verification is successful.

[0034] Meanwhile, the image to be inspected was input into a pre-trained YOLOv5 image recognition model, which was trained to recognize objects such as "workbook," "pen," and "hand." The detection results showed that the image contained a clear workbook and hand, the user's concentration was normal, and the system did not intervene.

[0035] If, in three consecutive random shooting sessions (e.g., at 12:08, 15:30, and 20:45), the image recognition model fails to detect any learning-related features, and the gesture verification passes (proving the user is in front of the camera), the system determines that the user is in a "distracted state" (e.g., the user is daydreaming or doing something unrelated to homework). At this time, the system plays a first-level voice reminder through the speaker (5): "Little friend, please concentrate on your homework." Simultaneously, a WeChat message is immediately generated and pushed to the preset designated terminal (parent's WeChat): "Your child may be distracted, please pay attention."

[0036] After the monitoring period ends, the system will calculate the total learning time, effective focus time, number of distractions and their distribution time for the day, and generate a "Focus Daily Report" with pictures and text, which will be pushed to the designated terminal.

[0037] Example 2: Supervision Scheme Based on Voice Verification

[0038] The difference between this embodiment and embodiment 1 is that the anti-cheating verification method uses voice verification. The system integrates a microphone module (6), which is electrically connected to the control motherboard (4).

[0039] When the random trigger time arrives at 3 minutes and 15 seconds, the system first controls the speaker (5) to randomly broadcast the number "5", and at the same time controls the camera (2) to capture the image of the work area to be inspected, and simultaneously collects the user's voice segment repeating "5" through the microphone module (6). The entire acquisition process is completed within 1 second.

[0040] The system then inputs the speech segment into a pre-trained speech recognition model (such as an acoustic model based on a convolutional neural network). It recognizes the user's repeated content as "5," which matches the broadcast verification symbol, thus passing the verification. If the recognition is inconsistent or no valid speech is detected within the time limit, the anomaly is recorded and subsequent intervention procedures are triggered.

[0041] This embodiment solves the recognition rate problem of gesture verification in insufficient lighting or when the hand is obscured by voice verification, and further improves the reliability of anti-cheating verification.

[0042] Example 3: Supervision Scheme Linked with Tutoring System

[0043] This embodiment, based on Embodiment 1 above, works in conjunction with the "Multimodal Limited-Time Heuristic Homework Tutoring Method and System" filed by the applicant on the same day. After generating a focus report, the system transmits data such as the frequency of distraction as input parameters to the tutoring system. Based on this, the tutoring system dynamically adjusts its tutoring strategy for the following day: if the number of distractions exceeds 5 times on a given day, the number of available tutoring sessions for the next day is automatically adjusted from the default 3 to 2, thus incentivizing the user to be more focused the following day to receive more tutoring assistance. This linkage mechanism further enhances the overall effectiveness of supervision and tutoring.

[0044] Example 4: Surveillance System Based on Dual Fixed Cameras

[0045] Reference Figure 5 As shown, the intelligent study desk lamp of this embodiment includes a lamp body (1), a first camera (2a) fixedly mounted on the top of the lamp head with its lens pointing vertically downwards, a second camera (2b) fixedly mounted on the upper part of the lamp pole with its lens facing the user, a touch display screen (3) embedded in the front of the lamp head, a built-in control motherboard (4), a speaker (5), and a microphone module (6). The first camera (2a) is specifically used to capture images of the work area to be inspected, and the second camera (2b) is specifically used to capture images of the user's gestures or upper body. Both cameras have a fixed focal length and a fixed angle, requiring no mechanical moving parts.

[0046] Reference Figure 2 The flowchart shown illustrates the working process of this embodiment:

[0047] When the random trigger time of 3 minutes and 15 seconds arrives, the system first controls the display screen (3) to randomly display the number "3", and at the same time controls the first camera (2a) to take a picture of the work area (the image to be inspected), and controls the second camera (2b) to simultaneously take a picture containing the user's hand posture (the verification image). The entire acquisition process is completed within 0.3 seconds, and the two cameras work in parallel without interfering with each other.

[0048] The subsequent processes, such as image recognition, status determination, and notification push, are the same as in Example 1.

[0049] This embodiment eliminates mechanical moving parts, reduces hardware costs, and improves system reliability through a dual fixed camera solution. At the same time, it avoids the interference of camera rotation on students' attention, achieving truly "unobtrusive" attention monitoring.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring work attention based on random triggering and multimodal verification, applied to intelligent devices, characterized in that: Includes the following steps: S1: Supervision mode is activated, responding to user operation commands or preset timed tasks, and real-time image data of the work area is collected through the camera set on the smart device; S2: Randomly trigger sampling. Within a preset supervision period, multiple non-fixed shooting times are dynamically generated through a random number generation algorithm. At each shooting time, the camera is controlled to capture a frame of behavioral image sequence of the current working area. S3: Real-time anti-cheating verification. While executing step S2, the output device of the smart device is controlled to randomly output a verification symbol, and the acquisition device is controlled to synchronously acquire verification data containing user response information. The verification data is identified by a recognition model to determine whether the user response information matches the verification symbol. S4: Focus on state recognition, input the behavior image sequence into a pre-trained image recognition model, and detect whether there are preset learning-related features in the image; The learning-related features include at least one or more of the following: workbook, handwriting, and writing hand. S5: Status determination and intervention. If the response information and the verification symbol are inconsistent in step S3, or if the learning-related features are not detected for a preset number of consecutive times in step S4, then the user is determined to be in a "distracted state". S6: Real-time notification and report generation. In response to the determination of "non-focused state", an alert message is immediately generated and pushed to a preset designated terminal via wireless network. At the same time, based on all determination results during the entire monitoring period, a focus report is generated and pushed to the designated terminal periodically.

2. The task attention monitoring method based on random triggering and multimodal verification according to claim 1, characterized in that, The method is applied to a smart study lamp, which is equipped with the camera and the output device.

3. The task attention monitoring method based on random triggering and multimodal verification according to claim 1, characterized in that, In step S2, the random number generation algorithm is a true random number generator or a pseudo random number generator, ensuring that the time interval between adjacent shooting moments is a non-fixed value and unpredictable.

4. The task attention monitoring method based on random triggering and multimodal verification according to claim 1, characterized in that, In step S3, the verification symbol is output through a display screen or speaker, and the verification data is collected through a camera or microphone. The verification method is gesture verification, voice verification, or a combination of both. During gesture verification, the verification symbol is one of numbers, letters, or geometric shapes. During voice verification, the verification symbol is one of numbers, letters, or words. The verification data and the behavior image sequence are collected at the same time or within a time difference of less than 1 second.

5. The task attention monitoring method based on random triggering and multimodal verification according to claim 4, characterized in that, The voice verification includes: controlling a speaker to randomly play a verification word, collecting the user's repeated speech through a microphone, and using a speech recognition model to determine whether the user's repeated content matches the played verification word.

6. The task attention monitoring method based on random triggering and multimodal verification according to claim 1, characterized in that, The image recognition model in step S4 is a deep learning model based on a convolutional neural network architecture, trained using a sample dataset containing different lighting conditions, angles, and work scenarios.

7. The task attention monitoring method based on random triggering and multimodal verification according to claim 1, characterized in that, The "number of consecutive preset times" in step S5 is dynamically adjusted based on the user's age or historical attention data, and its value ranges from 2 to 5 times.

8. The task attention monitoring method based on random triggering and multimodal verification according to claim 1, characterized in that, In step S6, the reminder information includes local voice reminders and remote notifications; the remote notifications are pushed through at least one of WeChat mini-programs, mobile apps, or SMS.

9. A smart study lamp system implementing the method of any one of claims 1-8, characterized in that, include: Hardware unit, including: Lamp body (1); A camera module (2) is mounted on the lamp body (1) and is used to collect images of the work area and user response. A microphone module (6) is mounted on the lamp body (1) and is used to collect user voice. The output device includes a display screen (3) and / or a speaker (5) for outputting verification symbols and interactive information; The control motherboard (4) is embedded in the lamp body (1) and includes a wireless communication module, a memory and a processor. The processor is electrically connected to the camera module (2), the microphone module (6), the output device and the memory. A software unit, stored in the memory, includes a computer program that, when executed by the processor, implements the job attention monitoring method based on random triggering and multimodal verification as described in any one of claims 1-8.

10. The intelligent study lamp system according to claim 9, characterized in that, The camera module (2) includes two fixed cameras, wherein the first camera is fixedly installed on the top of the lamp body with the lens pointing vertically downward to capture the work area, and the second camera is fixedly installed on the upper part of the lamp pole with the lens facing the user to capture the user's upper body and gestures.

11. The intelligent study lamp system according to claim 9, characterized in that, The camera module (2) is a single rotatable camera driven by a micro motor, which can adjust the shooting angle to cover the work area or the upper body area of ​​the user.

12. The intelligent study lamp system according to claim 9, characterized in that, It also includes an ambient light sensor installed on the lamp body, which is electrically connected to the control motherboard (4) and is used to automatically adjust the lighting brightness according to the ambient light intensity.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the job focus monitoring method based on random triggering and multimodal verification as described in any one of claims 1 to 8.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the job focus monitoring method based on random triggering and multimodal verification as described in any one of claims 1 to 8.

Images