Intelligent self-study room seat state recognition management method based on edge vision

Abstract

The application discloses a wisdom self-study room seat state recognition management method based on edge vision, relates to the wisdom management and edge artificial intelligence technical field, and comprises the following steps: collecting self-study room scene real-time video images through a camera, and transmitting the real-time video images to an edge computing terminal; the edge computing terminal performs human body target detection on a video frame, and obtains human body target bounding box position information; the edge computing terminal performs mobile phone target detection on the same video frame, and obtains mobile phone target bounding box position information; a seat object is established according to the human body target position, a seat reference area is generated, and the seat identity is maintained stable in continuous frames; according to the spatial relationship between the human body target and the seat reference area, in-situ, off-seat or overtime off-seat states are determined; based on the spatial correlation between the mobile phone target and the human body target, the mobile phone target is attributed to the corresponding seat, and when the determination condition is met, it is determined that there is mobile phone abnormal behavior; when the seat state or the mobile phone abnormal state changes, an event record is generated.

Description

Technical Field

[0001] This invention belongs to the field of intelligent management and edge artificial intelligence technology, and in particular relates to a method for recognizing and managing the seat status of intelligent study rooms based on edge vision. Background Technology

[0002] Currently, study rooms, reading rooms, laboratories, and other learning spaces generally suffer from problems such as low personnel management efficiency, high patrol costs, and difficulty in timely detection of abnormal behavior. Traditional management methods mainly rely on manual patrols, which have shortcomings such as low patrol frequency, inability to continuously record data, strong subjectivity, and incomplete evidence chains.

[0003] In practice, managers can typically only check the situation at specific times, making it difficult to continuously monitor whether a particular seat is occupied for an extended period, whether there is any unauthorized use of a seat, or whether behaviors such as using mobile phones disrupt the learning order. Furthermore, manual management often fails to automatically generate visual evidence and structured records, hindering subsequent traceability, statistical analysis, and management optimization.

[0004] Existing intelligent monitoring solutions largely rely on cloud servers for image processing and data analysis. While they possess a certain level of intelligence, they also suffer from high deployment costs, strong network dependence, significant privacy risks, and insufficient adaptability in closed environments. For scenarios like study rooms where real-time performance, stability, and privacy are critical, there is an urgent need for a smart study room management terminal that can operate independently locally and balances recognition accuracy with closed-loop system management capabilities.

[0005] Therefore, developing a smart study room management terminal based on edge vision, which integrates video recognition, status judgment, event recording and local visual management into the same edge device, has significant practical significance and application value. Summary of the Invention

[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0007] This invention relates to a smart study room seat status recognition and management method based on edge vision, comprising the following steps:

[0008] Step S1: Capture real-time video images of the study room scene using a camera and transmit the image data to the edge computing terminal;

[0009] Step S2: The edge computing terminal performs human target detection on the input video frame and obtains the bounding box position information of the human target in the picture;

[0010] Step S3: The edge computing terminal performs mobile phone target detection on the same video frame and obtains the bounding box position information of the mobile phone target in the picture;

[0011] Step S4: Establish corresponding seat objects based on the detected human target positions, generate seat reference areas corresponding to each seat object, and maintain stable seat identities during continuous video frame processing;

[0012] Step S5: Based on the spatial relationship between the human target and the corresponding seat reference area, determine whether the corresponding seat is in a seated state or an unseated state. When the unseated duration exceeds a preset threshold, determine that the seat is in an overdue unseated state.

[0013] Step S6: Based on the spatial relationship between the mobile phone target and the human body target, assign the mobile phone target to the corresponding seat object. When the abnormal judgment condition is met, determine that there is abnormal mobile phone behavior in that seat.

[0014] Step S7: When a change in seat status or abnormal change in mobile phone status is detected, a corresponding event record is generated, and a screenshot of the video frame when the event is triggered is saved simultaneously, so that the event record and the screenshot are associated.

[0015] Further, step S1 includes the following steps:

[0016] Step S11: The camera continuously captures video of the target area, with the capture frequency set to 10 to 30 frames per second;

[0017] Step S12: The edge computing terminal acquires video stream data through a USB camera interface or a MIPI camera interface.

[0018] Furthermore, step S2 includes the following steps:

[0019] Step S21: Use a lightweight target detection model to identify and infer human targets in video frames;

[0020] Step S22: Output the bounding box coordinates, confidence score, and target location information of the human target.

[0021] Furthermore, step S3 includes the following steps:

[0022] Step S31: Use a mobile phone detection model optimized for small target scenes to identify and infer mobile phone targets in video frames;

[0023] Step S32: Output the bounding box coordinates, confidence level, and spatial location information of the mobile phone target.

[0024] Furthermore, step S4 includes the following steps:

[0025] Step S41: Generate a reference area for the corresponding seat by expanding the human body detection frame downwards or outwards;

[0026] Step S42: In continuous video frame processing, maintain the identity of each seat object based on the continuity of the target location, the correspondence between regions and historical status information to avoid frequent switching of seat identity.

[0027] Furthermore, step S5 includes the following steps:

[0028] Step S51: When the human target leaves the reference area of ​​the corresponding seat, start the departure timer.

[0029] Step S52: Set the threshold for the timeout duration to 10 to 300 seconds. When the timeout duration exceeds this threshold, update the seat status to the timeout status and generate a corresponding abnormal event.

[0030] Furthermore, step S6 includes the following steps:

[0031] Step S61: Determine if the mobile phone is abnormal based on one or a combination of the following conditions: the center point of the mobile phone target is located within a preset range around the human body detection box, the mobile phone target and the human body detection box have overlapping areas, and the mobile phone target stably appears near the same seat object in multiple consecutive frames.

[0032] Step S62: Set the number of consecutive abnormal frames to 3 to 30 frames. Only when the number of consecutive frames that meet the abnormal judgment condition reaches the set value is it determined that there is abnormal mobile phone behavior in the seat.

[0033] Furthermore, step S7 includes the following steps:

[0034] Step S71: The generated event log includes five types of events: start of leaving the seat, end of leaving the seat, trigger of timeout leaving the seat, start of abnormal mobile phone operation, and end of abnormal mobile phone operation. Each event log includes the event time, event category, seat number, status change information, and duration. When the event is triggered, the system synchronously saves the corresponding video frame screenshot. The screenshot file is associated with the log record to form a traceable chain of evidence.

[0035] Step S72: Save the event log in a structured table format, and save the on-site screenshots in JPG or PNG format.

[0036] Furthermore, it also includes step S8, local data storage and webpage visualization, which specifically includes the following steps:

[0037] Step S81: After a management scenario ends, an event summary is automatically generated based on the event records. The event summary includes the number of times the person left their seat, the number of times they left their seat beyond the time limit, the number of times their mobile phone malfunctioned, the duration of various malfunctions, and an overview of changes in seat status.

[0038] Step S82: Build a local web service based on the Flask framework. The web interface includes a console page, a real-time status page, a history page, and a details summary page. It supports real-time viewing and historical querying via local area network or device access.

[0039] Furthermore, it also includes step S9, accelerating edge model deployment, which specifically includes the following steps:

[0040] Step S91: Export the trained detection model to ONNX format, and then convert it to RKNN format;

[0041] Step S92: Deploy the converted model to the Orange Pi 5Pro edge computing platform, call the NPU to perform inference acceleration, and support the selection of CPU mode, GPU assisted mode or NPU accelerated mode according to computing power.

[0042] The present invention has the following beneficial effects:

[0043] 1. This invention operates in a local closed loop: video acquisition, detection inference, status judgment, event logging, and page display can all be completed locally on the edge device, reducing dependence on external networks and cloud platforms.

[0044] 2. This invention offers good privacy protection: The system primarily processes and stores data locally, making it suitable for learning environments where privacy is a primary concern.

[0045] 3. The invention has a high degree of automation: it can automatically identify statuses such as being in place, leaving the seat, leaving the seat for an extended period of time, and abnormal mobile phone status, reducing the pressure of manual inspection.

[0046] 4. The evidence chain of this invention is complete: a traceable management closed loop is formed through event logs, screenshots, and summary statistics.

[0047] 5. This invention is scalable: It has been successfully validated in a dual-seat management scenario and can be smoothly extended to multi-seat management, more abnormal behavior recognition, and more complex management scenarios.

[0048] 6. The invention has a low deployment cost: By using edge devices and a lightweight model, the smart study room terminal can be deployed at a low cost.

[0049] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating the intelligent study room seat status recognition and management method based on edge vision according to the present invention.

[0052] Figure 2 This is a schematic diagram of the control panel page of the smart study room management terminal of the present invention;

[0053] Figure 3 This is a schematic diagram of the history page of the smart study room management terminal of the present invention;

[0054] Figure 4 This is a schematic diagram of the dual-seat detection and mobile phone anomaly identification of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Please see Figure 1-4 As shown, this invention is a smart study room seat status recognition and management method based on edge vision, including the following steps:

[0057] Step S1: Capture real-time video images of the study room scene using a camera and transmit the image data to the edge computing terminal;

[0058] Step S2: The edge computing terminal performs human target detection on the input video frame and obtains the bounding box position information of the human target in the picture;

[0059] Step S3: The edge computing terminal performs mobile phone target detection on the same video frame and obtains the bounding box position information of the mobile phone target in the picture;

[0060] Step S4: Establish corresponding seat objects based on the detected human target positions, generate seat reference areas corresponding to each seat object, and maintain stable seat identities during continuous video frame processing;

[0061] Step S5: Based on the spatial relationship between the human target and the corresponding seat reference area, determine whether the corresponding seat is in a seated state or an unseated state. When the unseated duration exceeds a preset threshold, determine that the seat is in an overdue unseated state.

[0062] Step S6: Based on the spatial relationship between the mobile phone target and the human body target, assign the mobile phone target to the corresponding seat object. When the abnormal judgment condition is met, determine that there is abnormal mobile phone behavior in that seat.

[0063] Step S7: When a change in seat status or abnormal change in mobile phone status is detected, a corresponding event record is generated, and a screenshot of the video frame when the event is triggered is saved simultaneously, so that the event record and the screenshot are associated.

[0064] Step S1 includes the following steps:

[0065] Step S11: The camera continuously captures video of the target area, with the capture frequency set to 10 to 30 frames per second;

[0066] Step S12: The edge computing terminal acquires video stream data through a USB camera interface or a MIPI camera interface.

[0067] Step S2 includes the following steps:

[0068] Step S21: Use a lightweight target detection model to identify and infer human targets in video frames;

[0069] Step S22: Output the bounding box coordinates, confidence score, and target location information of the human target.

[0070] Step S3 includes the following steps:

[0071] Step S31: Use a mobile phone detection model optimized for small target scenes to identify and infer mobile phone targets in video frames;

[0072] Step S32: Output the bounding box coordinates, confidence level, and spatial location information of the mobile phone target.

[0073] Step S4 includes the following steps:

[0074] Step S41: Generate a reference area for the corresponding seat by expanding the human body detection frame downwards or outwards;

[0075] Step S42: In continuous video frame processing, maintain the identity of each seat object based on the continuity of the target location, the correspondence between regions and historical status information to avoid frequent switching of seat identity.

[0076] Step S5 includes the following steps:

[0077] Step S51: When the human target leaves the reference area of ​​the corresponding seat, start the departure timer.

[0078] Step S52: Set the threshold for the timeout duration to 10 to 300 seconds. When the timeout duration exceeds this threshold, update the seat status to the timeout status and generate a corresponding abnormal event.

[0079] Step S6 includes the following steps:

[0080] Step S61: Determine if the mobile phone is abnormal based on one or a combination of the following conditions: the center point of the mobile phone target is located within a preset range around the human body detection box, the mobile phone target and the human body detection box have overlapping areas, and the mobile phone target stably appears near the same seat object in multiple consecutive frames.

[0081] Step S62: Set the number of consecutive abnormal frames to 3 to 30 frames. Only when the number of consecutive frames that meet the abnormal judgment condition reaches the set value is it determined that there is abnormal mobile phone behavior in the seat.

[0082] Step S7 includes the following steps:

[0083] Step S71: The generated event log includes five types of events: start of leaving the seat, end of leaving the seat, trigger of timeout leaving the seat, start of abnormal mobile phone operation, and end of abnormal mobile phone operation. Each event log includes the event time, event category, seat number, status change information, and duration. When the event is triggered, the system synchronously saves the corresponding video frame screenshot. The screenshot file is associated with the log record to form a traceable chain of evidence.

[0084] Step S72: Save the event log in a structured table format, and save the on-site screenshots in JPG or PNG format.

[0085] It also includes step S8, local data storage and webpage visualization, which specifically includes the following steps:

[0086] Step S81: After a management scenario ends, an event summary is automatically generated based on the event records. The event summary includes the number of times the person left their seat, the number of times they left their seat beyond the time limit, the number of times their mobile phone malfunctioned, the duration of various malfunctions, and an overview of changes in seat status.

[0087] Step S82: Build a local web service based on the Flask framework. The web interface includes a console page, a real-time status page, a history page, and a details summary page. It supports real-time viewing and historical querying via local area network or device access.

[0088] It also includes step S9, which accelerates edge model deployment, specifically including the following steps:

[0089] Step S91: Export the trained detection model to ONNX format, and then convert it to RKNN format;

[0090] Step S92: Deploy the converted model to the Orange Pi 5Pro edge computing platform, call the NPU to perform inference acceleration, and support the selection of CPU mode, GPU assisted mode or NPU accelerated mode according to computing power.

[0091] One specific application of this embodiment is:

[0092] Example 1: Standard Management Example for Two Seats

[0093] In a two-seat study room scenario, a camera is fixedly mounted in front of and above the seats, with a capture frequency set to 15 to 25 frames per second. The system creates two seat objects, P1 and P2, and generates corresponding reference areas for each.

[0094] When P1 leaves his seat, the system immediately starts the departure timer; if P1 returns within 30 seconds, a normal departure event is recorded; if P1 is away from his seat for more than 30 seconds, an overdue departure event is recorded and displayed on the history page.

[0095] When a mobile phone target is continuously detected near P2 for a duration of 1 to 5 seconds, the system determines that P2 has abnormal mobile phone behavior and records the start time, end time and screenshot of the abnormality.

[0096] Example 2: Low Threshold Quick Reminder Example

[0097] In scenarios where real-time management is critical, the departure threshold can be set to 10 to 20 seconds, and the number of frames for continuous mobile phone anomaly detection can be set to 3 to 10 frames to improve response speed.

[0098] This embodiment is suitable for scenarios that require rapid prompts, such as pre-exam study areas and short-term duty areas, but it has relatively high requirements for suppressing false detections.

[0099] Example 3: High Stability Screening Example

[0100] In scenarios where system stability is critical, the off-seat threshold can be set to 60 to 300 seconds, and the number of frames for continuous mobile phone anomaly detection can be set to 10 to 30 frames to reduce false alarms caused by single-frame false detection.

[0101] This embodiment is applicable to scenarios with complex personnel activities, such as open reading rooms and laboratories.

[0102] Example 4: Edge NPU Acceleration Example

[0103] During deployment, the detection model is first exported to ONNX format, then converted to RKNN format, and loaded into the NPU inference environment of the OrangePi 5 Pro device for execution.

[0104] In this embodiment, the system still performs the complete process of video acquisition, target detection, seat status determination, anomaly recognition, event logging, and webpage display, but the inference part is handled by the NPU, thereby improving operating efficiency and enhancing long-term operational stability.

[0105] Example 5: Multi-person Extended Example

[0106] After verification is completed in the dual-seat mode, the number of seat objects can be expanded to 3 to 20 or more. The system can manage multi-person seat scenarios by increasing the number of seat objects, expanding the reference area generation strategy, and optimizing the identity retention rules.

[0107] In this embodiment, human detection, mobile phone detection, event logging, and webpage display framework can all be implemented using the dual-seat embodiment, with only the seat allocation and personnel identity association strategies needing to be extended.

[0108] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0109] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for recognizing and managing the seat status of a smart study room based on edge vision, characterized in that, Includes the following steps: Step S1: Capture real-time video images of the study room scene using a camera and transmit the image data to the edge computing terminal; Step S2: The edge computing terminal performs human target detection on the input video frame and obtains the bounding box position information of the human target in the picture; Step S3: The edge computing terminal performs mobile phone target detection on the same video frame and obtains the bounding box position information of the mobile phone target in the picture; Step S4: Establish corresponding seat objects based on the detected human target positions, generate seat reference areas corresponding to each seat object, and maintain stable seat identities during continuous video frame processing; Step S5: Based on the spatial relationship between the human target and the corresponding seat reference area, determine whether the corresponding seat is in a seated state or an unseated state. When the unseated duration exceeds a preset threshold, determine that the seat is in an overdue unseated state. Step S6: Based on the spatial relationship between the mobile phone target and the human body target, assign the mobile phone target to the corresponding seat object. When the abnormal judgment condition is met, determine that there is abnormal mobile phone behavior in that seat. Step S7: When a change in seat status or abnormal change in mobile phone status is detected, a corresponding event record is generated, and a screenshot of the video frame when the event is triggered is saved simultaneously, so that the event record and the screenshot are associated.

2. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: The camera continuously captures video of the target area, with the capture frequency set to 10 to 30 frames per second; Step S12: The edge computing terminal acquires video stream data through a USB camera interface or a MIPI camera interface.

3. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Use a lightweight target detection model to identify and infer human targets in video frames; Step S22: Output the bounding box coordinates, confidence score, and target location information of the human target.

4. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Use a mobile phone detection model optimized for small target scenes to identify and infer mobile phone targets in video frames; Step S32: Output the bounding box coordinates, confidence level, and spatial location information of the mobile phone target.

5. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Generate a reference area for the corresponding seat by expanding the human body detection frame downwards or outwards; Step S42: In continuous video frame processing, maintain the identity of each seat object based on the continuity of the target location, the correspondence between regions and historical status information to avoid frequent switching of seat identity.

6. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, Step S5 includes the following steps: Step S51: When the human target leaves the reference area of ​​the corresponding seat, start the departure timer. Step S52: Set the threshold for the timeout duration to 10 to 300 seconds. When the timeout duration exceeds this threshold, update the seat status to the timeout status and generate a corresponding abnormal event.

7. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, Step S6 includes the following steps: Step S61: Determine if the mobile phone is abnormal based on one or a combination of the following conditions: the center point of the mobile phone target is located within a preset range around the human body detection box, the mobile phone target and the human body detection box have overlapping areas, and the mobile phone target stably appears near the same seat object in multiple consecutive frames. Step S62: Set the number of consecutive abnormal frames to 3 to 30 frames. Only when the number of consecutive frames that meet the abnormal judgment condition reaches the set value is it determined that there is abnormal mobile phone behavior in the seat.

8. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, Step S7 includes the following steps: Step S71: The generated event log includes five types of events: start of leaving the seat, end of leaving the seat, trigger of timeout leaving the seat, start of abnormal mobile phone operation, and end of abnormal mobile phone operation. Each event log includes the event time, event category, seat number, status change information, and duration. When the event is triggered, the system synchronously saves the corresponding video frame screenshot. The screenshot file is associated with the log record to form a traceable chain of evidence. Step S72: Save the event log in a structured table format, and save the on-site screenshots in JPG or PNG format.

9. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, It also includes step S8, local data storage and webpage visualization, which specifically includes the following steps: Step S81: After a management scenario ends, an event summary is automatically generated based on the event records. The event summary includes the number of times the person left their seat, the number of times they left their seat beyond the time limit, the number of times their mobile phone malfunctioned, the duration of various malfunctions, and an overview of changes in seat status. Step S82: Build a local web service based on the Flask framework. The web interface includes a console page, a real-time status page, a history page, and a details summary page. It supports real-time viewing and historical querying via local area network or device access.

10. The method for recognizing and managing the seat status of a smart study room based on edge vision according to claim 1, characterized in that, It also includes step S9, which accelerates edge model deployment, specifically including the following steps: Step S91: Export the trained detection model to ONNX format, and then convert it to RKNN format; Step S92: Deploy the converted model to the Orange Pi 5Pro edge computing platform, call the NPU to perform inference acceleration, and support the selection of CPU mode, GPU assisted mode or NPU accelerated mode according to computing power.

Images