A human behavior analysis system based on densely populated classrooms

By using a classroom crowd behavior analysis system, image processing and analysis technologies have solved the problem of students' body movements being obscured, enabling accurate identification and avoidance of misjudgments of classroom behavior, and improving teaching quality.

CN114037954BActive Publication Date: 2024-11-15JIANGXI NORMAL UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111320917.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-09
Publication Date
2024-11-15
Estimated Expiration
2041-11-09

AI Technical Summary

Technical Problem

In existing technologies, due to the obstruction between desks, chairs, and students, it is difficult to accurately identify the body movements of students in densely packed classrooms, leading to misjudgments in behavior analysis.

Method used

The system employs a human behavior analysis system based on densely packed classroom crowds, including a central processing system, an image system, surveillance cameras, a prediction system, and an external environment detection module. Through image processing and analysis, combined with limb feature extraction, particle filling, and audio recognition, it achieves accurate judgment of student behavior.

Benefits of technology

By repairing occluded parts of the image, accurate judgment of a person's behavior can be made, avoiding misjudgments and improving the accuracy of classroom behavior analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114037954B_ABST
    Figure CN114037954B_ABST
Patent Text Reader

Abstract

The invention discloses a human behavior analysis system based on a classroom with dense crowds, and relates to the field of classroom crowd behavior analysis. The technical scheme comprises an analysis system, wherein the interior of the analysis system comprises a central processing system, an image system, a monitoring camera, a prediction system, a judgment module and an external environment detection module, and the external communication connection of the analysis system is connected with a cloud storage module, and the cloud storage module synchronizes the data inside the analysis system in real time; the image system is communicated with the monitoring camera, and the interior of the image system comprises an image acquisition module, an image processing module and an image recognition module, and the image system acquires images taken by the monitoring camera, and can realize the judgment of the surroundings of a person whose action is inconsistent with others through the coordinated use of an association module and a regional positioning module, and at the same time, can realize the comparison of images within 5s before and after the image, and thus can realize the accurate judgment of students with abnormal actions and avoid misjudgment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of classroom crowd behavior analysis, and in particular to a human behavior analysis system based on a densely populated classroom crowd. Background Art

[0002] During class, students whisper to each other or are distracted by external things, which leads to students not paying attention in class and reduces the quality of classroom teaching;

[0003] After searching, the invention patent with Chinese patent number CN109284737A discloses a student behavior analysis and recognition system for smart classrooms, including: a coordinate and posture capture module, which obtains the position coordinates of each student and captures each student's facial orientation posture and hand movement posture; an attention point calculation module, which obtains the student's facial orientation posture and calculates the student's classroom attention point through the facial orientation posture; a behavior recognition module, which obtains the student's attention point and hand movement posture, and recognizes the student's classroom behavior through the student's classroom attention point and hand movement posture; a behavior comparison module, which obtains each student's classroom behavior and compares it with the classroom behavior of its neighboring students to determine each student's classroom status; a result feedback module, which is used to feedback the student's classroom status and provide real-time feedback to the teacher. The present invention solves the problem of inaccurate recognition and judgment of students' classroom status in the prior art;

[0004] However, in the process of acquiring student behaviors, the above system can only determine whether the student is in a playful state by acquiring the facial expressions and body behaviors of a single student. However, when the recognition device detects the student's body, it is difficult to accurately recognize the student's movements due to the obstruction of tables, chairs and students. In addition, it is easy to make wrong judgments by only analyzing a single student. Therefore, a human behavior analysis system based on densely populated classrooms is needed. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcoming in the prior art that when detecting students' limbs, it is difficult to accurately identify students' movements due to the obstruction between tables, chairs and students, and to propose a human behavior analysis system based on densely populated classrooms.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A human behavior analysis system based on a densely populated classroom includes an analysis system, wherein the analysis system includes a central processing system, an image system, a monitoring camera, a prediction system, a judgment module, and an external environment detection module, and the external communication of the analysis system is connected to a cloud storage module, and the cloud storage module synchronizes the data inside the analysis system in real time;

[0008] The image system is connected to the monitoring camera through communication. The image system includes an image acquisition module, an image processing module and an image recognition module. The image system acquires images taken by the monitoring camera, processes and recognizes the acquired images, and transmits the processed and recognized image data to the central processing system. The central processing system includes a facial feature extraction module and a limb feature extraction module. The limb feature extraction module includes a skeleton extraction module, a feature acquisition module and a particle filling module. The limb feature extraction module splits and pre-judges and fills the skeleton of the character through the image data transmitted by the image system. The central processing system transmits the facial and limb action image data of the character to the prediction system. The prediction system includes an audio recognition module, an association module and a regional positioning module. The prediction system is connected to the external environment detection module through communication. The prediction system associates according to the received image data, compares and analyzes the behavior of the character within a certain area and time, and transmits the result of the comparison and analysis to the judgment module. By comparing with the predetermined data in the judgment module, the discreteness between the characters is judged.

[0009] The above technical solution further includes:

[0010] The surveillance camera is installed inside the classroom. The surveillance camera monitors the students inside the classroom in real time. The image acquisition module acquires the images taken by the surveillance camera. The image acquisition module transmits the acquired images to the image processing module. The image processing module pre-processes the acquired images to remove image impurities and noise.

[0011] The image processing module transmits the processed image to the image recognition module, and the image recognition module recognizes the image based on the data processed in the image processing module. The content recognized by the image recognition module includes the body and facial movements of the person, and the image recognition module transmits the recognized image data to the central processing system.

[0012] The facial feature extraction module acquires the facial features of the person according to the image data transmitted by the image recognition module, and the limb feature extraction module acquires the limb features of the person according to the image data transmitted by the image recognition module.

[0013] The skeleton extraction module extracts the skeleton of the character according to the acquired body features of the character and establishes joint activity points. The feature acquisition module acquires the filling features around the skeleton according to the character skeleton established by the skeleton extraction module.

[0014] When the particle filling module performs expected positioning of the running track of the subsequent character according to the data in the skeleton extraction module and the feature acquisition module, the skeleton is filled with particles according to the filling features around the skeleton.

[0015] The audio recognition module acquires and recognizes the audio inside the classroom, and the area positioning module locates the area where the person's behavior is abnormal, and defines the area radius of the center of the person's position as Q;

[0016] Detect the first interval 0m<Q<1m and analyze the shapes of other characters in the interval;

[0017] Detect the second interval 1m≤Q<3m and analyze the forms of other characters in the interval;

[0018] Then the third interval 3m≤Q<5m is tested, and the forms of other characters in this interval are analyzed;

[0019] At the same time, the external environment detection module transmits the external environment data to the prediction system. At the same time, the association module compares the image data of the students with abnormal behavior with the image content within 5 seconds before and after that time point, and transmits the comparison results to the judgment module.

[0020] The judgment module judges the discreteness between the characters according to the detection results of the first interval, the second interval and the third interval, and the comparison results of the images within 5 seconds before and after.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. In the present invention, when used, the coordinated use of the skeleton extraction module, the feature acquisition module and the particle filling module inside the limb feature extraction module can realize the repair of the occluded part of the image movement. First, the skeleton of the character is extracted, and then when the character moves to the occluded part, the image of the skeleton movement is restored through the trajectory of the character's movement, and then the skeleton can be filled with particles through the particle filling module, thereby realizing the restoration of the occluded part of the character, which is convenient for judging the character's behavior.

[0023] 2. In the present invention, the coordinated use of the association module and the area positioning module can realize the judgment of the surroundings of the person whose actions are inconsistent with others, and at the same time, by comparing the images within 5 seconds before and after the image, it is possible to accurately judge the students with abnormal actions and avoid misjudgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1A system block diagram of a human behavior analysis system based on a classroom with dense crowds proposed by the present invention;

[0025] Figure 2 is a system block diagram of the image system in the present invention;

[0026] Figure 3 is a system block diagram of the central processing system in the present invention;

[0027] Figure 4 It is a system block diagram of the limb feature extraction module in the present invention;

[0028] Figure 5 It is a system block diagram of the prediction system in the present invention.

[0029] In the figure: 1. Central processing system; 2. Image system; 3. Surveillance camera; 4. Prediction system; 5. Judgment module; 6. External environment detection module; 7. Cloud storage module; 8. Image acquisition module; 9. Image processing module; 10. Image recognition module; 11. Facial feature extraction module; 12. Limb feature extraction module; 13. Skeleton extraction module; 14. Feature acquisition module; 15. Particle filling module; 16. Audio recognition module; 17. Association module; 18. Regional positioning module. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0031] Embodiment 1

[0032] like Figure 1-5 As shown, the present invention proposes a human behavior analysis system based on a densely populated classroom, including an analysis system, wherein the interior of the analysis system includes a central processing system 1, an image system 2, a monitoring camera 3, a prediction system 4, a judgment module 5 and an external environment detection module 6, and the external communication of the analysis system is connected to a cloud storage module 7, and the cloud storage module 7 synchronizes the data inside the analysis system in real time;

[0033] The image system 2 is connected to the monitoring camera 3 by communication. The image system 2 includes an image acquisition module 8, an image processing module 9 and an image recognition module 10. The image system 2 acquires the image taken by the monitoring camera 3, processes and recognizes the acquired image, and transmits the processed and recognized image data to the central processing system 1. The central processing system 1 includes a facial feature extraction module 11 and a limb feature extraction module 12. The limb feature extraction module 12 includes a skeleton extraction module 13, a feature acquisition module 14 and a particle filling module 15. The limb feature extraction module 12 is connected to the monitoring camera 3 by communication. The image system 2 includes an image acquisition module 8, an image processing module 9 and an image recognition module 10. The image system 2 acquires the image taken by the monitoring camera 3, processes and recognizes the acquired image, and transmits the processed and recognized image data to the central processing system 1. The central processing system 1 includes a facial feature extraction module 11 and a limb feature extraction module 12. The limb feature extraction module 12 includes a skeleton extraction module 13, a feature acquisition module 14 and a particle filling module 15. The image data transmitted by the image system 2 is used to split and pre-judge the skeleton of the character. The central processing system 1 transmits the facial and body movement image data of the character to the prediction system 4. The prediction system 4 includes an audio recognition module 16, an association module 17 and a regional positioning module 18. The prediction system 4 is connected to the external environment detection module 6 for communication. The prediction system 4 associates the received image data and compares and analyzes the behavior of the character within a certain area and time. The prediction system 4 transmits the comparison and analysis results to the judgment module 5, and compares them with the predetermined data in the judgment module 5 to judge the discreteness between the characters.

[0034] The monitoring camera 3 is installed inside the classroom. The monitoring camera 3 monitors the students inside the classroom in real time. The image acquisition module 8 acquires the image taken by the monitoring camera 3. The image acquisition module 8 transmits the acquired image to the image processing module 9. The image processing module 9 pre-processes the acquired image to remove image impurities and noise.

[0035] The image processing module 9 transmits the processed image to the image recognition module 10. The image recognition module 10 recognizes the image according to the data processed in the image processing module 9. The content recognized by the image recognition module 10 includes the body and facial movements of the person. The image recognition module 10 transmits the recognized image data to the central processing system 1.

[0036] The working principle of the human behavior analysis system based on the classroom dense crowd according to the first embodiment is that during class, the monitoring camera 3 monitors the students in the classroom in real time, the image acquisition module 8 collects the images taken by the monitoring camera 3, and the image acquisition module 8 transmits the collected images to the image processing module 9. The image processing module 9 pre-processes the collected images, removes image impurities and noise, and decomposes the images;

[0037] The image processing module 9 transmits the processed image to the image recognition module 10. The image recognition module 10 recognizes the image according to the data processed in the image processing module 9. The content recognized by the image recognition module 10 includes the body and facial movements of the person. Finally, the image recognition module 10 transmits the recognized image data to the central processing system 1.

[0038] Embodiment 2

[0039] like Figure 1-5 As shown, based on the first embodiment, the facial feature extraction module 11 acquires the facial features of the person according to the image data transmitted by the image recognition module 10, and the limb feature extraction module 12 acquires the limb features of the person according to the image data transmitted by the image recognition module 10;

[0040] The skeleton extraction module 13 extracts the skeleton of the character according to the acquired limb features of the character and establishes joint activity points. The feature acquisition module 14 acquires filling features around the skeleton according to the character skeleton established by the skeleton extraction module 13.

[0041] When the particle filling module 15 performs expected positioning of the subsequent running track of the character according to the data in the skeleton extraction module 13 and the feature acquisition module 14 , the skeleton is filled with particles according to the filling features around the skeleton.

[0042] In this embodiment, when working, the facial feature extraction module 11 acquires the facial features of the character according to the image data transmitted by the image recognition module 10, and the limb feature extraction module 12 acquires the limb features of the character according to the image data transmitted by the image recognition module 10. Then, the skeleton extraction module 13 extracts the skeleton of the character according to the acquired limb features of the character, and establishes joint activity points. The feature acquisition module 14 acquires the filling features around the skeleton according to the character skeleton established by the skeleton extraction module 13. When the limbs of the character are blocked, the skeleton extraction module 13 can locate the skeleton of the character according to the movement trajectory of the limbs of the character, and then fills the features with particles around the skeleton through the particle filling module 15 to restore the authenticity of the character.

[0043] Embodiment 3

[0044] like Figure 1-5 As shown, based on the above-mentioned embodiment 1 or 2, the audio recognition module 16 obtains and recognizes the audio inside the classroom, and the area positioning module 18 locates the abnormal behavior area of ​​the character, and defines the area radius of the center of the position of the character as Q;

[0045] Detect the first interval 0m<Q<1m and analyze the shapes of other characters in the interval;

[0046] Detect the second interval 1m≤Q<3m and analyze the forms of other characters in the interval;

[0047] Then the third interval 3m≤Q<5m is tested, and the forms of other characters in this interval are analyzed;

[0048] At the same time, the external environment detection module 6 transmits the external environment data to the prediction system 4, and the association module 17 compares the image data of the student with abnormal behavior with the image content within 5 seconds before and after the time point, and transmits the comparison result to the judgment module 5;

[0049] The judgment module 5 judges the discreteness between the characters according to the detection results of the first interval, the second interval and the third interval, and the comparison results of the images within 5 seconds before and after.

[0050] In this embodiment, when working, the audio recognition module 16 obtains and recognizes the audio inside the classroom, and the area positioning module 18 locates the area where the person's behavior is abnormal, and defines the area radius of the center of the person's position as Q, and then detects the intervals respectively;

[0051] Detect the first interval 0m<Q<1m and analyze the shapes of other characters in the interval;

[0052] Detect the second interval 1m≤Q<3m and analyze the forms of other characters in the interval;

[0053] Then the third interval 3m≤Q<5m is tested, and the forms of other characters in this interval are analyzed;

[0054] When other abnormal behaviors occur in the first interval, the abnormal behaviors are recorded and marked;

[0055] When the first interval does not appear but the second interval shows abnormality, it is necessary to compare the facial expressions of the two in the second interval to determine the student's class status;

[0056] When there is no abnormality in the first and second intervals, it is necessary to compare the facial expressions and body parts of the two students in the third interval, and introduce the external environment to determine whether the abnormality is caused by the external environment, so as to determine the student's class status;

[0057] At the same time, when performing the comparison, the images within 5 seconds before and after the abnormal behavior image need to be compared, and then the prediction system 4 transmits the comparison results to the judgment module 5. The judgment module 5 judges the discreteness between the characters based on the detection results of the first interval, the second interval and the third interval, and the comparison results of the images within 5 seconds before and after.

[0058] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A human behavior analysis system based on a classroom with dense crowds, comprising an analysis system, characterized in that: The analysis system comprises a central processing system (1), an image system (2), a monitoring camera (3), a prediction system (4), a judgment module (5) and an external environment detection module (6); the analysis system is externally connected to a cloud storage module (7); the cloud storage module (7) performs real-time synchronization of data within the analysis system; The image system (2) is communicatively connected to the monitoring camera (3). The image system (2) includes an image acquisition module (8), an image processing module (9) and an image recognition module (10). The image system (2) acquires images captured by the monitoring camera (3). The image system (2) processes and recognizes the acquired images. The image system (2) transmits the processed and recognized image data to the central processing system (1). The central processing system (1) includes a facial feature extraction module (11) and a limb feature extraction module (12). The limb feature extraction module (12) includes a skeleton extraction module (13), a feature acquisition module (14) and a particle filling module (15). The block (12) splits and pre-judges the skeleton of the character through the image data transmitted by the image system (2); the central processing system (1) transmits the facial and body movement image data of the character to the prediction system (4); the prediction system (4) internally includes an audio recognition module (16), an association module (17) and a region positioning module (18); the prediction system (4) is connected to the external environment detection module (6); the prediction system (4) performs association based on the received image data, and compares and analyzes the behavior of the character within a certain area and time; the prediction system (4) transmits the comparison and analysis results to the judgment module (5), and judges the discreteness between the characters by comparing with the predetermined data in the judgment module (5); The monitoring camera (3) is installed inside the classroom, and the monitoring camera (3) monitors the students inside the classroom in real time. The image acquisition module (8) acquires the image captured by the monitoring camera (3), and the image acquisition module (8) transmits the acquired image to the image processing module (9). The image processing module (9) pre-processes the acquired image to remove image impurities and noise points; The image processing module (9) transmits the processed image to the image recognition module (10); the image recognition module (10) recognizes the image based on the data processed by the image processing module (9); the content recognized by the image recognition module (10) includes body and facial movements of a person; the image recognition module (10) transmits the recognized image data to the central processing system (1); The facial feature extraction module (11) acquires facial features of a person based on the image data transmitted by the image recognition module (10), and the limb feature extraction module (12) acquires limb features of a person based on the image data transmitted by the image recognition module (10); The skeleton extraction module (13) extracts the skeleton of the character according to the acquired limb features of the character and establishes joint activity points, and the feature acquisition module (14) acquires filling features around the skeleton according to the character skeleton established by the skeleton extraction module (13); The audio recognition module (16) acquires and recognizes the audio inside the classroom, and the area positioning module (18) locates the area where the person's behavior is abnormal, and defines the area radius of the center of the person's position as Q; For the first interval Conduct detection and analyze the forms of other characters in the interval; For the second interval Conduct detection and analyze the forms of other characters in the interval; For the third interval Conduct detection and analyze the forms of other characters in the interval; At the same time, the external environment detection module (6) transmits the external environment data to the prediction system (4), and the association module (17) compares the image data of the student with abnormal behavior with the image content within 5 seconds before and after the time point, and transmits the comparison result to the judgment module (5); The judgment module (5) judges the discreteness between the characters based on the detection results of the first interval, the second interval and the third interval, and the comparison results of the images within 5 seconds before and after.

2. The human behavior analysis system based on densely populated classrooms according to claim 1 is characterized in that: When the particle filling module (15) performs expected positioning of the subsequent running trajectory of the character based on the data in the skeleton extraction module (13) and the feature acquisition module (14), the skeleton is filled with particles based on the filling features around the skeleton.

Citation Information

Patent Citations

  • A student behavior analysis and identification system for a smart classroom

    CN109284737A

  • Novel video recording and playing system

    CN108243327A

  • Learning assistance method and system based on classroom emotion and behavior analysis

    CN110992741A

  • Method and system for operation management based on student postures and student face recognition

    CN111931608A

  • Distributed monitoring-based violent behavior detection system and detection method

    CN113269076A