Method and system for detecting students playing with mobile phones

Abstract

The application discloses a method and system for detecting students playing mobile phones. The system comprises a pan-tilt camera arranged at the left and right ends of the back side of a classroom, a shooting control module and an analysis module. The analysis method used by the analysis module comprises the following steps: obtaining close-up pictures of positions of students; performing mobile phone detection and screen-on detection on the close-up pictures; if the mobile phone detection result is that a mobile phone exists and the screen-on detection result is that the mobile phone is in a screen-on state, marking that a student at the position corresponding to the close-up picture is playing a mobile phone. The two pan-tilt cameras are arranged at the left and right ends of the back side of the classroom, so that positions that are difficult to shoot can be ensured to be shot, and the mobile phone detection and the screen-on detection are used for filtering before posture detection, so that the cost can be reduced.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a method and system for detecting students using mobile phones. Background Technology

[0002] The classroom is the core arena for education and teaching, and its quality directly affects teaching effectiveness and the cultivation of students' comprehensive qualities. Paying attention to students' classroom learning behaviors and performance is an important basis for constructing a scientific teaching evaluation system. Traditional student classroom behavior evaluation relies on manual observation and recording, which is not only time-consuming and labor-intensive but also inevitably subject to subjectivity. Today, with the rapid development of artificial intelligence technology, more and more AI technologies are being applied to the automated recognition of classroom student behavior.

[0003] At the same time, the widespread use of mobile phones has made them a necessity in people's daily lives. While bringing convenience, people's dependence on mobile phones has also deepened, a phenomenon particularly prominent in classroom settings—students' use of mobile phones in class has had a significant negative impact on normal teaching order and effectiveness.

[0004] Currently, student behavior detection primarily employs vision-based recognition technologies, which typically include two core components: behavior representation and target detection. Among these, joint-based behavior representation methods obtain the position and movement information of various joints in the human body through pose estimation, thereby characterizing human behavior. However, in classroom settings, this method faces several challenges: firstly, students' movements are generally small in scale, and overlapping occlusion between students is common in video images, significantly increasing the difficulty of recognizing mobile phone use; secondly, the joint diagrams extracted from mobile phone use and writing are highly similar, making it difficult to effectively distinguish them based solely on skeletal key points.

[0005] Therefore, developing a method and system that can accurately detect students' mobile phone usage behavior in the classroom has become an urgent need. Summary of the Invention

[0006] To overcome at least one of the defects described in the prior art, the present invention provides a method and system for detecting students using mobile phones. The technical solution adopted by the present invention is as follows.

[0007] In a first aspect, the present invention provides a method for detecting students using mobile phones, comprising:

[0008] Get close-up shots of each student's location;

[0009] The close-up image is subjected to mobile phone detection and screen-on detection; wherein, the mobile phone detection is the detection of whether a mobile phone is present, and the screen-on detection is the detection of whether the mobile phone is in a screen-on state;

[0010] If the mobile phone detection result indicates the presence of a mobile phone and the screen-on detection result indicates the screen is on, then the student at the location corresponding to the close-up image is playing on their mobile phone.

[0011] In one implementation, before the student marking the student's position corresponding to the close-up image plays with their phone, the process further includes:

[0012] If the mobile phone detection result indicates that a mobile phone is present and the screen-on detection result indicates that the screen is on, then the posture of the student at the student position corresponding to the close-up image is detected to obtain the posture detection result.

[0013] The process of marking the student's position corresponding to the close-up image as a student playing on their phone is only executed when the posture detection result indicates that the student is playing on their phone.

[0014] In one implementation, the process of performing posture detection on the student at the student's position corresponding to the close-up shot and obtaining the posture detection result includes:

[0015] The main key points of the student's upper body are detected, including: head point, neck point, wrist point, and elbow point;

[0016] Calculate the angle between the line connecting the head point and the neck point and the horizontal plane, and denote it as the first angle;

[0017] Calculate the ratio between the length of the line connecting the center of the phone to the wrist and the length of the line connecting the head to the neck, and denote it as the first ratio;

[0018] Calculate the ratio between the length of the line connecting the center of the phone to the wrist point and the length of the line connecting the wrist point to the elbow point, and denote it as the second ratio;

[0019] If the first ratio is less than the set first ratio threshold, then determine whether the first included angle is less than the set angle threshold and the second ratio is less than the set second ratio threshold; if so, determine the posture detection result as playing with a mobile phone.

[0020] Secondly, the present invention provides a system for detecting students playing with mobile phones, comprising: two pan-tilt cameras, a shooting control module and an analysis module;

[0021] Two PTZ cameras were positioned at the left and right ends of the back of the classroom.

[0022] The shooting control module is used to control the shooting of two PTZ cameras;

[0023] The analysis module is used to analyze the images captured by the two PTZ cameras;

[0024] The analysis module uses the method described above for detecting students playing with their mobile phones to analyze close-up footage captured by two PTZ cameras.

[0025] The shooting control module controls the shooting process of the two pan-tilt cameras, including:

[0026] Obtain student location distribution information;

[0027] Based on the student location distribution information, calculate the planned shooting area that each of the two cameras needs to patrol and shoot.

[0028] Control two cameras to patrol and film within their respective planned shooting areas;

[0029] During the patrol and shooting process, based on the student location distribution information, the camera is controlled to shoot in close-up mode each time it reaches a student's location, capturing close-up images of each student's location.

[0030] In one embodiment, the process of the shooting control module controlling the two pan-tilt cameras to shoot further includes:

[0031] Control two PTZ cameras to each capture a rear-view image in panoramic mode;

[0032] The head detection algorithm is used to detect heads in two rear view images, and the head detection results of the two rear view images are obtained.

[0033] By fusing the head detection results from the two rear-view images, the student location distribution information is obtained.

[0034] In one implementation, the process of calculating the planned shooting area to be patrolled by each of the two cameras based on the student location distribution information includes:

[0035] Based on the student location distribution information, calculate the area that each of the two cameras needs to patrol and film.

[0036] By merging the two areas that each camera needs to patrol and film, and removing overlapping areas, the planned shooting areas calculated by each camera are obtained.

[0037] In one implementation, the process of merging two areas that each needs to be patrolled, removing overlapping areas, and obtaining the planned shooting areas calculated by each of the two cameras includes:

[0038] If each group in the classroom has two columns, the planned shooting area of ​​the left pan-tilt camera is the left column of each group, and the planned shooting area of ​​the right pan-tilt camera is the right column of each group.

[0039] In one implementation, the process of merging two areas that each needs to be patrolled, removing overlapping areas, and obtaining the planned shooting areas calculated by each of the two cameras includes:

[0040] If the number of groups in the classroom is an odd number M and each group has only one column, then the planned shooting area of ​​the left pan-tilt camera is (M-1) / 2 groups on the left, and the planned shooting area of ​​the right pan-tilt camera is (M-1) / 2 groups on the right.

[0041] Calculate the centerline of the student position distribution, wherein the centerline is perpendicular to the blackboard;

[0042] Using the center line as a reference, among the students in the middle group, if a student's head is to the left of the center line, the student's position will be photographed by the camera on the left rear side; if a student's head is to the right of the center line, the student's position will be photographed by the camera on the right rear side.

[0043] Thirdly, the present invention provides a device for detecting students using mobile phones, characterized in that it comprises:

[0044] The acquisition module is used to capture close-up images of each student's location;

[0045] The detection module is used to perform mobile phone detection and screen-on detection on close-up images; wherein, the mobile phone detection is the detection of whether a mobile phone is present, and the screen-on detection is the detection of whether the mobile phone is in a screen-on state;

[0046] The judgment module is used to mark the student at the student's location corresponding to the close-up image as playing on a mobile phone if the mobile phone detection result indicates that a mobile phone is present and the screen light-up detection result indicates that the screen is on.

[0047] Fourthly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any of the above embodiments.

[0048] Fifthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the method of any of the above embodiments.

[0049] In this invention, two PTZ cameras are positioned at the left and right ends of the rear side of the classroom. This ensures that positions such as desks and storage spaces, which are difficult to capture from the front, are covered, guaranteeing the quality of close-up footage of students using their phones. During the analysis of these close-up images, the presence or absence of a phone screen is detected to confirm whether a student is using their phone, thus eliminating actions such as writing that are highly similar to the human joint diagrams of phone use and avoiding misjudgments. Furthermore, phone detection and screen illumination detection are used for filtering before posture detection, reducing overhead and the time spent on patrol filming. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the overall process of one embodiment of the present invention.

[0051] Figure 2 This is a schematic diagram of the overall structure of the system for detecting students playing with mobile phones according to Embodiment 2 of the present invention.

[0052] Figure 3 This is a schematic diagram of the overall structure of Embodiment 3 of the present invention. Detailed Implementation

[0053] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0054] It should be noted that the terms "first, second, ..." used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, ..." can be interchanged in a specific order or sequence where permissible. It should be understood that the objects distinguished by "first, second, ..." can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0055] Example 1

[0056] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for detecting students using mobile phones according to Embodiment 1 of the present invention. The method includes steps S110, S120, and S140. It should be noted that steps S110, S120, and S140 are merely reference numerals used to clearly explain the embodiment and the accompanying drawings. Figure 1 The correspondence is not intended to limit the order of steps in this embodiment.

[0057] Step S110: Obtain close-up images of each student's location;

[0058] Step S120: Perform mobile phone detection and screen-on detection on the close-up image; wherein, the mobile phone detection is the detection of whether a mobile phone exists, and the screen-on detection is the detection of whether the mobile phone is in a screen-on state;

[0059] Step S140: If the mobile phone detection result indicates the presence of a mobile phone and the screen-on detection result indicates the screen is on, then mark the student at the student's location corresponding to the close-up image as playing on a mobile phone.

[0060] In a classroom environment, one or more PTZ cameras can be used to target each area in the classroom where students may appear, i.e., the student's location. Pre-set shooting positions can be manually or automatically set in advance, and the pan-tilt camera's navigation path can be planned accordingly to shoot separately and obtain close-up images of the student's location with sufficient clarity.

[0061] Step S110: Obtain close-up images of each student's position after close-up shooting.

[0062] The prerequisite for detecting whether a student is using a mobile phone is whether they have a mobile phone. Therefore, in step S120, a mobile phone detection is first performed on the close-up image. If there is no mobile phone, then it can be determined that the student is not using a mobile phone.

[0063] Detecting a phone does not necessarily mean the student is using it, so further judgment is needed. Step S120 of this method further detects whether the phone's screen is on. Most phones have a screen-off timer; when not in use, the screen is usually off. If the phone's screen is on, there is a high probability the student is using it.

[0064] In step S130, the condition that a mobile phone is present and the screen is on is defined as having played with the mobile phone. Then, the student at the student position corresponding to the close-up image is marked as playing with the mobile phone.

[0065] In one implementation, before the process of marking the student's position corresponding to the close-up image in step S130, the process further includes step S140.

[0066] Step S140: If the mobile phone detection result is that a mobile phone exists and the screen light detection result is that the screen is on, perform posture detection on the student at the student position corresponding to the close-up image to obtain the posture detection result.

[0067] The process of marking the student's position corresponding to the close-up image as a student playing on their phone is only executed when the posture detection result indicates that the student is playing on their phone.

[0068] This implementation further confirms the student's behavior of using a mobile phone. As mentioned earlier, if the phone screen is on, there is a high probability that the student is using it. However, sometimes the phone receives a message or is moved, which can also cause the screen to light up. Therefore, this implementation further confirms the student's behavior of using a mobile phone. Specifically, this is confirmed through posture detection.

[0069] First, detect whether the phone screen is on, and then use posture detection to confirm whether the student is using the phone. The advantages of doing this are: it can eliminate actions that are highly similar to the human joint diagram of playing with a phone, such as writing, thus avoiding misjudgment; in addition, posture detection is generally much more expensive than target detection and screen detection. If posture detection is performed at every location, each patrol will take longer. Therefore, phone detection and screen detection are used for filtering before posture detection.

[0070] In one implementation, step S140, which involves performing posture detection on the student at the student's position corresponding to the close-up shot and obtaining the posture detection result, includes:

[0071] Step S141: Detect the main key points of the student's upper body, including: head point, neck point, wrist point, and elbow point;

[0072] Step S142: Calculate the angle between the line connecting the head point and the neck point and the horizontal plane, and record it as the first angle.

[0073] Step S143: Calculate the ratio between the length of the line connecting the center of the phone to the wrist point and the length of the line connecting the head point to the neck point, and record it as the first ratio.

[0074] Step S144: Calculate the ratio between the length of the line connecting the center of the phone to the wrist point and the length of the line connecting the wrist point to the elbow point, and record it as the second ratio.

[0075] Step S145: If the first ratio is less than the set first ratio threshold, then determine whether the first included angle is less than the set angle threshold and the second ratio is less than the set second ratio threshold; if so, determine the posture detection result as playing with a mobile phone.

[0076] Step S141 detects the main key points of the student's upper body, including: head point, neck point, wrist point, and elbow point.

[0077] Step S142 calculates the angle alpha between the line connecting the head point and the neck point and the horizontal plane, which is the first angle.

[0078] Step S143 calculates the ratio beta between (the length of the line connecting the center of the phone and the wrist point) and (the length of the line connecting the head point and the neck point), and records it as the first ratio;

[0079] Step S144 calculates the ratio gamma between (the length of the line connecting the center of the phone and the wrist point) and (the length of the line connecting the wrist point and the elbow point), and records it as the second ratio;

[0080] Step S145: If beta is less than the set first ratio threshold, then determine whether alpha is less than the set angle threshold and gamma is less than the set second ratio threshold. If so, mark the student at that position as playing on their phone.

[0081] In this embodiment, to reduce the false recognition rate of key points in posture detection, the ratio beta of the length from the phone to the wrist to the length from the head to the neck is used as the judgment criterion:

[0082] If beta is greater than the set percentage threshold, it indicates that there may be misidentification of key points or that there is a mobile phone at the current location but the student at that location is not using the mobile phone.

[0083] If beta is less than the set ratio threshold, the system uses the head angle alpha to determine whether the student at that position is looking down, and uses the ratio of the length from the phone to the wrist to the length from the elbow to the wrist to determine whether the student at that position is holding or playing with the phone.

[0084] This method detects whether a student is using their phone by checking if the screen is on, thus eliminating actions such as writing that are highly similar to the human joint diagrams of phone use and avoiding false positives. Furthermore, this method uses phone detection and screen on / off detection for filtering before posture detection, which reduces overhead and the time spent on patrol photography.

[0085] Example 2

[0086] Please see Figure 2 , Figure 2 The diagram below shows the overall structure of a system for detecting students playing with mobile phones according to Embodiment 1 of the present invention. The system (2) for detecting students playing with mobile phones includes: two pan-tilt cameras (210, 220), a shooting control module (230), and an analysis module (240).

[0087] Two PTZ cameras were positioned at the left and right ends of the back of the classroom.

[0088] The shooting control module is used to control the shooting of two PTZ cameras;

[0089] The analysis module is used to analyze the footage captured by the two PTZ cameras.

[0090] The analysis module uses the method for detecting students playing with mobile phones as described in Embodiment 1 to analyze close-up images captured by two PTZ cameras.

[0091] The process of the shooting control module controlling the shooting of two PTZ cameras includes: steps S210, S220, S230 and S240.

[0092] Step S210: Obtain student location distribution information;

[0093] Step S220: Based on the student location distribution information, calculate the planned shooting area that each of the two cameras needs to shoot.

[0094] Step S230: Control the two cameras to perform patrol shooting in their respective planned shooting areas;

[0095] Step S240: During the patrol shooting process, based on the student location distribution information, the camera is controlled to shoot in close-up mode each time it reaches a student's location, capturing close-up images of each student's location.

[0096] In a typical classroom setting, cameras installed at the front of the classroom often result in overlapping and obstruction of images from students, making it difficult to capture students using their phones. However, when cameras are positioned at the left and right ends of the rear of the classroom, they can ensure that they can capture areas that are difficult to film from the front, such as desks and storage spaces. Therefore, in this system for detecting students using their phones, two PTZ cameras are positioned at the left and right ends of the rear of the classroom.

[0097] Step S210: Obtain student location distribution information. In this system, two PTZ cameras need to perform patrol shooting to capture close-up images of each student's location. Therefore, obtaining student location distribution information is necessary to plan the patrol shooting process.

[0098] In one embodiment, the process of the shooting control module controlling the two pan-tilt cameras to shoot further includes steps S310, S320 and S330.

[0099] Step S310: Control the two PTZ cameras to each take a rear-view image in panoramic mode;

[0100] Step S320: Use a head detection algorithm to detect heads in the two rear view images to obtain the head detection results for the two rear view images;

[0101] Step S330: The head detection results of the two rear view images are fused to obtain the student location distribution information.

[0102] This implementation describes the process of obtaining individual student location information. Two PTZ cameras are positioned at the left and right ends of the rear side of the classroom. Therefore, the images captured by the two PTZ cameras are partial views, but when combined, they can show the entire classroom. Thus, the two PTZ cameras are controlled to capture a rear-view image in panoramic mode.

[0103] After capturing two rear-view images, head detection is performed on each image. The detected heads indicate the students' location distribution. However, since the two images overlap, the head detection results from the two rear-view images need to be fused. The fused result is the final, accurate location distribution information.

[0104] After obtaining the student location distribution information through step S210, two cameras need to be arranged to calculate the planned shooting area that needs to be patrolled.

[0105] In one implementation, step S220, which involves calculating the planned shooting area to be patrolled by each of the two cameras based on the student location distribution information, includes steps S221 and S222.

[0106] Step S221: Calculate the area that each of the two cameras needs to patrol based on the student location distribution information;

[0107] Step S222: Merge the two areas that each needs to be patrolled, remove the overlapping areas, and obtain the planned shooting areas that each of the two cameras needs to patrol.

[0108] In general education settings, student seating arrangements typically fall into two categories: ① multiple single-column arrangements (e.g., 1×N×M, meaning there are M groups, each with only one column, and each column has N rows); ② multiple double-column arrangements (e.g., 2×N×M, meaning there are M groups, each with two columns, and each column has N rows).

[0109] In this case, for multiple double-column configurations: the left rear camera captures the left column of each group, and the right rear camera captures the right column of each group.

[0110] For multiple single-column formats:

[0111] If it is an even number of groups, the left camera will shoot the left M / 2 group, and the right camera will shoot the right M / 2 group;

[0112] If the number of students is odd, the left camera will film the left (M-1) / 2 group, and the right camera will film the right (M-1) / 2 group. For the middle group, first calculate the center line of the student position distribution, which is perpendicular to the blackboard. Then, based on the center line, if a student's head is to the left of the center line, the student's position will be filmed by the camera on the left rear side; if a student's head is to the right of the center line, the student's position will be filmed by the camera on the right rear side.

[0113] This ensures that each PTZ camera only films one row during patrol filming.

[0114] After step S220, the planned shooting areas that each of the two cameras needs to shoot are calculated. Then, step S230 controls the two cameras to start the shooting function and use the close-up mode to shoot according to the shooting area they are responsible for calculated in the previous step. During the shooting process, every time a student is reached, the close-up mode is used to shoot and capture close-up images of each student's location.

[0115] After obtaining the close-up footage, the analysis module is used to analyze it. The analysis module uses the method for detecting students playing with mobile phones as described in Example 1 to analyze the close-up footage captured by the two PTZ cameras, thereby obtaining the result of detecting students playing with mobile phones.

[0116] It should be noted that since students' mobile phone use is intermittent, the shooting control module also operates intermittently, controlling the two PTZ cameras to shoot close-up shots every time interval T. After obtaining the close-up footage, the analysis module immediately performs analysis.

[0117] In this system, two PTZ cameras are positioned at the left and right ends of the back of the classroom, ensuring that they can capture images of locations that are difficult to film from the front, such as desks and storage spaces, thus guaranteeing the quality of close-up footage of students using their mobile phones.

[0118] Example 3

[0119] Corresponding to the method in Example 1, such as Figure 3 As shown, the present invention also provides a device 3 for detecting students playing with mobile phones, including: an acquisition module 310, a detection module 320, and a judgment module 330.

[0120] The acquisition module is used to capture close-up images of each student's location;

[0121] The detection module is used to perform mobile phone detection and screen-on detection on close-up images; wherein, the mobile phone detection is the detection of whether a mobile phone is present, and the screen-on detection is the detection of whether the mobile phone is in a screen-on state;

[0122] The judgment module is used to mark the student at the student's location corresponding to the close-up image as playing on a mobile phone if the mobile phone detection result indicates that a mobile phone is present and the screen light-up detection result indicates that the screen is on.

[0123] In one embodiment, the detection module is further configured to perform posture detection on the student at the student position corresponding to the close-up image if the mobile phone detection result is that a mobile phone exists and the screen-on detection result is that the screen is on, and obtain the posture detection result.

[0124] The judgment module is also used to execute the process of marking the student's position corresponding to the close-up image as playing on a mobile phone only when the posture detection result is that the student is playing on a mobile phone.

[0125] In one implementation, the detection module performs the process of performing posture detection on the student at the student's position corresponding to the close-up shot and obtaining the posture detection result, including:

[0126] The main key points of the student's upper body are detected, including: head point, neck point, wrist point, and elbow point;

[0127] Calculate the angle between the line connecting the head point and the neck point and the horizontal plane, and denote it as the first angle;

[0128] Calculate the ratio between the length of the line connecting the center of the phone to the wrist and the length of the line connecting the head to the neck, and denote it as the first ratio;

[0129] Calculate the ratio between the length of the line connecting the center of the phone to the wrist point and the length of the line connecting the wrist point to the elbow point, and denote it as the second ratio;

[0130] If the first ratio is less than the set first ratio threshold, then determine whether the first included angle is less than the set angle threshold and the second ratio is less than the set second ratio threshold; if so, determine the posture detection result as playing with a mobile phone.

[0131] This device detects whether a student is using their phone by checking if the screen is on, thus eliminating actions such as writing that are highly similar to the human joint diagrams of phone use and avoiding false positives. Furthermore, the device uses phone detection and screen on / off detection for filtering before posture detection, reducing overhead and time spent on patrol photography.

[0132] Example 4

[0133] This invention also provides a storage medium storing computer instructions that, when executed by a processor, implement the method for detecting students using mobile phones as described in any of the above embodiments.

[0134] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, random access memory (RAM), read-only memory (ROM), magnetic disks, or optical disks.

[0135] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, RAM, ROM, magnetic disks, or optical disks.

[0136] Corresponding to the computer storage medium described above, one embodiment also provides a computer device, which includes a memory, an encoder, and a computer program stored in the memory and executable on the encoder, wherein the encoder executes the program to implement any of the methods for detecting students playing with mobile phones as described in the above embodiments.

[0137] The aforementioned computer equipment detects whether a student is using a mobile phone by checking if the screen is on, thus eliminating actions such as writing that are highly similar to the human joint diagrams of mobile phone use and avoiding false positives. Furthermore, the equipment uses mobile phone detection and screen on / off detection for filtering before posture detection, reducing overhead and time spent on patrol photography.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting students' mobile phone use, characterized in that, include: Get close-up shots of each student's location; The close-up image is subjected to mobile phone detection and screen-on detection; wherein, the mobile phone detection is the detection of whether a mobile phone is present, and the screen-on detection is the detection of whether the mobile phone is in a screen-on state; If the mobile phone detection result indicates the presence of a mobile phone and the screen-on detection result indicates the screen is on, then the student at the location corresponding to the close-up image is playing on their mobile phone.

2. The method for detecting students using mobile phones according to claim 1, characterized in that, Before the student whose position corresponds to the close-up shot was playing on their phone, the process also included: If the mobile phone detection result indicates that a mobile phone is present and the screen-on detection result indicates that the screen is on, then the posture of the student at the student position corresponding to the close-up image is detected to obtain the posture detection result. The process of marking the student's position corresponding to the close-up image as a student playing on their phone is only executed when the posture detection result indicates that the student is playing on their phone.

3. The method for detecting students using mobile phones according to claim 1, characterized in that, The process of performing posture detection on the student at the corresponding student position in the close-up shot and obtaining the posture detection result includes: The main key points of the student's upper body are detected, including: head point, neck point, wrist point, and elbow point; Calculate the angle between the line connecting the head point and the neck point and the horizontal plane, and denote it as the first angle; Calculate the ratio between the length of the line connecting the center of the phone to the wrist and the length of the line connecting the head to the neck, and denote it as the first ratio; Calculate the ratio between the length of the line connecting the center of the phone to the wrist point and the length of the line connecting the wrist point to the elbow point, and denote it as the second ratio; If the first ratio is less than the set first ratio threshold, then determine whether the first included angle is less than the set angle threshold and the second ratio is less than the set second ratio threshold; if so, determine the posture detection result as playing with a mobile phone.

4. A system for detecting students' mobile phone use, characterized in that, include: Two PTZ cameras, a shooting control module, and an analysis module; Two PTZ cameras were positioned at the left and right ends of the back of the classroom. The shooting control module is used to control the shooting of two PTZ cameras; The analysis module is used to analyze the images captured by the two PTZ cameras; The analysis module uses the method for detecting students playing with mobile phones as described in any one of claims 1-3 to analyze close-up images captured by two PTZ cameras. The shooting control module controls the shooting process of the two pan-tilt cameras, including: Obtain student location distribution information; Based on the student location distribution information, calculate the planned shooting area that each of the two cameras needs to patrol and shoot. Control two cameras to patrol and film within their respective planned shooting areas; During the patrol and shooting process, based on the student location distribution information, the camera is controlled to shoot in close-up mode each time it reaches a student's location, capturing close-up images of each student's location.

5. The system for detecting students using mobile phones according to claim 4, characterized in that, The process of the shooting control module controlling the two pan-tilt cameras to shoot also includes: Control two PTZ cameras to each capture a rear-view image in panoramic mode; The head detection algorithm is used to detect heads in two rear view images, and the head detection results of the two rear view images are obtained. By fusing the head detection results from the two rear-view images, the student location distribution information is obtained.

6. The system for detecting students using mobile phones according to claim 4, characterized in that, The process of calculating the planned shooting area to be patrolled by each of the two cameras based on the student location distribution information includes: Based on the student location distribution information, calculate the area that each of the two cameras needs to patrol and film. By merging the two areas that each camera needs to patrol and film, and removing overlapping areas, the planned shooting areas calculated by each camera are obtained.

7. The system for detecting students using mobile phones according to claim 6, characterized in that, The process of merging two areas that each needs to be patrolled, removing overlapping areas, and obtaining the planned shooting areas calculated by each camera includes: If each group in the classroom has two columns, the planned shooting area of ​​the left pan-tilt camera is the left column of each group, and the planned shooting area of ​​the right pan-tilt camera is the right column of each group.

8. The system for detecting students using mobile phones according to claim 6, characterized in that, The process of merging two areas that each needs to be patrolled, removing overlapping areas, and obtaining the planned shooting areas calculated by each camera includes: If the number of groups in the classroom is an odd number M and each group has only one column, then the planned shooting area of ​​the left pan-tilt camera is (M-1) / 2 groups on the left, and the planned shooting area of ​​the right pan-tilt camera is (M-1) / 2 groups on the right. Calculate the centerline of the student position distribution, wherein the centerline is perpendicular to the blackboard; Using the center line as a reference, among the students in the middle group, if a student's head is to the left of the center line, the student's position will be photographed by the camera on the left rear side; if a student's head is to the right of the center line, the student's position will be photographed by the camera on the right rear side.

9. A device for detecting students' mobile phone use, characterized in that, include: The acquisition module is used to capture close-up images of each student's location; The detection module is used to perform mobile phone detection and screen-on detection on close-up images; wherein, the mobile phone detection is the detection of whether a mobile phone is present, and the screen-on detection is the detection of whether the mobile phone is in a screen-on state; The judgment module is used to mark the student at the student's location corresponding to the close-up image as playing on a mobile phone if the mobile phone detection result indicates that a mobile phone is present and the screen light-up detection result indicates that the screen is on.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-3.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-3.

Images