Teacher status detection method, device, and storage medium

Abstract

The present application discloses a teacher status detection method, device, and storage medium, and relates to the field of recording and broadcasting technology. The method includes: identifying a first target object in a teacher panoramic image, and identifying a second target object in a student panoramic image, wherein the first target object and the second target object are both imaging results with the teacher as the shooting target; determining the change trend of the first target object according to the teacher panoramic image and the historical teacher panoramic image, and determining the change trend of the second target object according to the student panoramic image and the historical student panoramic image; determining the target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend, and the target object state includes the state of going up the podium and the state of going down the podium. Through the above-mentioned technical means, the error caused by the difference in the height of the teacher when detecting the state of going up and down the podium by a single panoramic image can be eliminated, and the problem that the state of going up and down the podium of the teacher cannot be accurately judged in the prior art can be solved.

Description

Technical Field

[0001] The present application relates to the field of recording and broadcasting technology, and in particular to a teacher status detection method, device, and storage medium. Background Art

[0002] The educational recording and broadcasting system has broad application prospects and important value in the field of education. It can record the teacher's teaching process for students to review or remote learning, transmit educational recording videos to remote areas to achieve the sharing and balance of high-quality educational resources, and record the teaching meeting process for post-meeting sorting and sharing. With the development of artificial intelligence technology, the educational recording and broadcasting system will become more intelligent, which can automatically identify and track recording targets and improve recording efficiency and quality.

[0003] The educational recording and broadcasting system includes a teacher camera and a student camera. The teacher camera shoots a panoramic picture of the teacher, and the student camera shoots a panoramic picture of the student. The teacher's close-up picture and the student's close-up picture can be cut out from the teacher's panoramic picture and the student's panoramic picture respectively. During the classroom recording and broadcasting process, the teacher's close-up picture is recorded when the teacher goes to the podium and moves with the teacher, and the student's panoramic picture is recorded when the teacher leaves the podium.

[0004] In related technologies, in order to determine whether a teacher is going up to the podium or down from the podium, a tracking frame can be set based on the blackboard and the interactive tablet. When the teacher's shoulders in the teacher's panoramic picture exceed the tracking frame, it is determined that the teacher is going up to the podium, and when half of the teacher's head is below the tracking frame, it is determined that the teacher is leaving the podium. However, due to the different heights of different teachers, when the tracking frame is fixed, it is impossible to accurately determine whether the teacher is going up to the podium or down from the podium based solely on the positional relationship between the teacher's body and the tracking frame, which affects the accuracy of the recording target tracking, resulting in poor quality of teaching recording. Summary of the invention

[0005] The present application provides a teacher status detection method, device, and storage medium, which detect the teacher area and the change trend of the teacher area in the corresponding image through the teacher panoramic image and the student panoramic image, and jointly detect the teacher's up and down podium status through the teacher area and the corresponding change trend in the two panoramic images, thereby eliminating the error caused by the height difference of the teacher when detecting the teacher's up and down podium status with a single panoramic image, solving the problem that the prior art cannot accurately judge the teacher's up and down podium status, improving the detection accuracy of the teacher's status, thereby improving the accuracy of recording and broadcasting target tracking, and optimizing the quality of teaching recording and broadcasting.

[0006] In a first aspect, the present application provides a teacher status detection method, comprising:

[0007] A first target object is identified in the teacher's panoramic image, and a second target object is identified in the student's panoramic image, wherein the first target object and the second target object are both imaging results with the teacher as the shooting target;

[0008] Determine the change trend of the first target object according to the teacher panoramic image and the history teacher panoramic image, and determine the change trend of the second target object according to the student panoramic image and the history student panoramic image;

[0009] The target object state is determined according to the first target object and the corresponding change trend and the second target object and the corresponding change trend, and the target object state includes an on-podium state and a off-podium state.

[0010] Through the above technical means, the first target object and the second target object can correspond to the position of the teacher relative to the teacher camera and the student camera. The first target object and the second target object can be used to jointly analyze the position of the teacher in the classroom, ensure the accuracy of the teacher's position estimation, and eliminate the positioning error caused by the difference in teacher height. The change trend of the first target object and the change trend of the second target object can represent the movement direction of the teacher relative to the teacher camera and the student camera. The change trend of the first target object and the change trend of the second target object can be used to jointly analyze the movement direction of the teacher in the classroom. While ensuring the accuracy of the teacher's movement direction estimation, the change trend is used to eliminate the problem of frequent switching of the recording and broadcasting screen caused by the teacher's repeated movement near the podium. Therefore, combined with the first target object and the corresponding change trend and the second target object and the corresponding change trend, it can be accurately analyzed whether the teacher is on the podium or off the podium, eliminating the error caused by the difference in teacher height when a single panoramic image detects the teacher's up and down podium state, solving the problem that the teacher's up and down podium state cannot be accurately judged in the prior art, improving the detection accuracy of the teacher's state, thereby improving the accuracy of recording and broadcasting target tracking, and optimizing the quality of teaching recording and broadcasting.

[0011] Optionally, determining the target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend includes:

[0012] In a case where the first target object overlaps with the blackboard image area in the teacher panoramic image, determining a first body part of the first target object in the blackboard image area;

[0013] determining a second body part of the second target object;

[0014] The target object state is determined according to the change trend of the first body part, the second body part, the first target object, and the change trend of the second target object.

[0015] Through the above-mentioned technical means, the teacher's position can be detected jointly through the first body part and the second body part, the teacher's behavior and actions can be detected jointly through the changing trend of the first target object and the changing trend of the second target object, and the teacher's state can be detected jointly through the teacher's position and behavior and actions, thereby eliminating the error introduced by single data detection and improving the detection accuracy of the teacher's state.

[0016] Optionally, determining the target object state according to the change trend of the first body part, the second body part, the first target object, and the change trend of the second target object includes:

[0017] When the change trends of the first target object and the second target object meet a first condition, and the first body part and the second body part do not meet a second condition, determining that the target object state is a podium state, the first condition including that the change trend of the first target object is getting smaller, and the change trend of the second target object is getting larger;

[0018] When the change trend of the first target object and the change trend of the second target object satisfy the third condition, and the first body part and the second body part satisfy the second condition, the target object state is determined to be the down-podium state, and the third condition includes that the change trend of the first target object is increasing, and the change trend of the second target object is decreasing.

[0019] Through the above-mentioned technical means, by using the changing trend of the first target object becoming larger or smaller and the trend of the second target object becoming smaller or larger, it is possible to accurately judge whether the teacher is walking towards or away from the podium, eliminating the motion detection error caused by using static images to judge the teacher's dynamic state, and then combining the high-precision teacher position determined jointly by the first body part and the second body part to accurately detect the teacher's state, thereby improving the detection accuracy of the teacher's state.

[0020] Optionally, determining the target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend includes:

[0021] In a case where the first target object overlaps with the blackboard image area in the teacher panoramic image, determining a first body part of the first target object in the blackboard image area;

[0022] determining a second body part of the second target object;

[0023] Determine a first ratio of the first target object to the teacher panoramic image, and determine a second ratio of the second target object to the student panoramic image;

[0024] The state of the target object is determined according to the first body part, the second body part, the first ratio, the second ratio, a change trend of the first target object, and a change trend of the second target object.

[0025] Through the above-mentioned technical means, the teacher's position is jointly detected by the first body part, the second body part, the first ratio and the second ratio, the teacher's behavior and action are jointly detected by the changing trend of the first target object and the changing trend of the second target object, and the teacher's state is jointly detected by the teacher's position and behavior and action, thereby eliminating the error introduced by single data detection and improving the detection accuracy of the teacher's state.

[0026] Optionally, determining the target object state according to the first body part, the second body part, the first ratio, the second ratio, a change trend of the first target object, and a change trend of the second target object includes:

[0027] When the change trends of the first target object and the change trends of the second target object meet the first condition, the first body part and the second body part do not meet the second condition, and the first ratio and the second ratio meet the fourth condition, determine that the state of the target object is the podium state;

[0028] When the change trend of the first target object and the change trend of the second target object meet the third condition, the first body part and the second body part meet the second condition, and the first ratio and the second ratio do not meet the fourth condition, the target object state is determined to be the down-podium state.

[0029] Through the above-mentioned technical means, by using the changing trend of the first target object becoming larger or smaller and the trend of the second target object becoming smaller or larger, it is possible to accurately judge whether the teacher is walking towards or away from the podium, thereby eliminating the motion detection error caused by using static images to judge the teacher's dynamic state. The high-precision teacher position determined by combining the first body part, the second body part, the first ratio and the second ratio can be used to accurately detect the teacher's state, thereby improving the detection accuracy of the teacher's state.

[0030] Optionally, determining the target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend includes:

[0031] In a case where the first target object does not overlap with the blackboard image area in the teacher panoramic image, determining a second body part of the second target object;

[0032] When the second body part includes a foot, and the change trend of the first target object and the change trend of the second target object meet a third condition, determining that the target object state is a state of getting off the podium;

[0033] When the second body part does not include feet, and the change trends of the first target object and the second target object satisfy a first condition, it is determined that the state of the target object is the state of going up to the podium.

[0034] Through the above-mentioned technical means, by detecting the teacher's position based on the second body part when the first target object does not overlap with the blackboard image area, and jointly detecting the teacher's behavior and actions based on the change trend of the first target object and the change trend of the second target object, and jointly detecting the teacher's status through the teacher's position and behavior and actions, the error introduced by single data detection is eliminated, thereby improving the detection accuracy of the teacher's status.

[0035] Optionally, the first condition also includes that the duration of the first target object becoming smaller reaches a preset duration, and the duration of the second target object becoming larger reaches a preset duration; the third condition also includes that the duration of the first target object becoming larger reaches a preset duration, and the duration of the second target object becoming smaller reaches a preset duration.

[0036] Through the above-mentioned technical means, through the duration of the change in the decreasing trend or increasing trend of the first target object and the duration of the change in the increasing trend or decreasing trend of the second target object, the teacher's behavior can be accurately judged as walking towards the podium, moving away from the podium, or making repeated movements on the edge of the podium, thereby avoiding misidentifying the teacher's repeated movements on the edge of the podium as movements of going up and down the podium, thereby avoiding frequent switching of teacher status, reducing the frequency of screen cutting of the recording and broadcasting screen and the proportion of invalid screens in the recording and broadcasting process, and greatly optimizing the quality of teaching recording and broadcasting.

[0037] Optionally, the second condition is that the head of the first body part is larger than the non-head part, and the second body part includes the upper body and / or the foot of the second body part does not touch the bottom of the student panoramic image.

[0038] Through the above-mentioned technical means, it is possible to accurately judge whether the teacher is on the podium or under the podium through the size relationship between the head and non-head in the first body part and the type of parts included in the second body part, eliminate the position detection error caused by the height difference of teachers, and thus improve the detection accuracy of the teacher's status.

[0039] Optionally, the fourth condition is that the first ratio is less than or equal to a first preset threshold and the second ratio is greater than or equal to a second preset threshold.

[0040] Through the above technical means, the relationship between the first ratio and the first preset threshold and the second ratio and the second preset threshold can be used to accurately determine whether the teacher is on the podium or under the podium, thereby improving the detection accuracy of the teacher's status.

[0041] Optionally, before the first target object is identified in the teacher panoramic image, the method further includes:

[0042] Taking an initial panoramic image by a teacher's camera, and detecting a blackboard image area in the initial panoramic image;

[0043] According to the blackboard image area, the screen center of the teacher's camera is adjusted to be aligned with the center of the blackboard image area.

[0044] Through the above-mentioned technical means, the teacher's camera is accurately reset to the initial state through the blackboard image area in the initial panoramic image taken by the teacher's camera before the recording starts, so as to ensure that the teacher's camera starts to shoot the teacher's panoramic image in the initial state, so that the teacher's panoramic image is displayed with the interactive large board and the teacher in the center and includes the entire picture of the podium, thereby optimizing the recording effect of the classroom picture and ensuring the accuracy of subsequent teacher status detection.

[0045] Optionally, identifying the first target object in the teacher panoramic image includes:

[0046] Performing standing human body detection on the teacher panoramic image to obtain a first standing human body frame;

[0047] Performing Kalman filtering on the first standing human body frame to obtain a first target object;

[0048] The step of identifying a second target object in the student panoramic image includes:

[0049] Perform standing human body detection on the student panoramic image to obtain a second standing human body frame;

[0050] A Kalman filter process is performed on the second standing human body frame to obtain a second target object.

[0051] Through the above-mentioned technical means, by detecting the first standing human frame including the area where the teacher is located in the teacher's panoramic image, and performing Kalman filtering on the first standing human frame, a high-precision first target object is obtained, which is beneficial to improving the accuracy of subsequent teacher status detection; by detecting the second standing human frame including the teacher's area in the student's panoramic image, and performing Kalman filtering on the second standing human frame, a high-precision second target object is obtained, which is beneficial to improving the accuracy of subsequent teacher status detection.

[0052] Optionally, performing standing human body detection on the student panoramic image to obtain a second standing human body frame includes:

[0053] When multiple standing human frames are identified in the student panoramic image, the historical back-turning time of the objects in the multiple standing human frames is determined, and the standing human frame corresponding to the object with the longest historical back-turning time is determined as the second standing human frame; or

[0054] When multiple standing human frames are identified in the student panoramic image, an initial frame in which the objects in the multiple standing human frames first appear in the student camera is determined, and a standing human frame corresponding to the objects in a standing state in the initial frame is determined as a second standing human frame.

[0055] Through the above-mentioned technical means, the second standing human frame including the teacher's area is accurately identified based on the historical back-turning time of the object in the standing human frame, thereby improving the detection accuracy of the second target object, thereby improving the accuracy of subsequent teacher status detection; based on the state of the object in each standing human frame when it first appears in the student's camera, the second standing human frame including the teacher's area is accurately identified, thereby improving the detection accuracy of the second target object, thereby improving the accuracy of subsequent teacher status detection.

[0056] In a second aspect, the present application provides a teacher status detection device, comprising:

[0057] One or more processors; a memory storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the teacher status detection method as described in the first aspect.

[0058] In a third aspect, the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the teacher status detection method as described in the first aspect.

[0059] In the present application, the first target object is obtained by identifying the teacher in the currently collected panoramic image of the teacher, and the second target object is obtained by identifying the teacher in the currently collected panoramic image of the student. The change trend of the first target object is determined based on the currently collected and historically collected panoramic images of the teacher, and the change trend of the second target object is determined based on the currently collected and historically collected panoramic images of the student. The teacher is jointly determined to be in the state of going up to the podium or going down the podium according to the first target object and the corresponding change trend and the second target object and the corresponding change trend. Through the above technical means, the first target object and the second target object can correspond to the position of the teacher relative to the teacher camera and the student camera. The first target object and the second target object can be used to jointly analyze the position of the teacher in the classroom, ensure the accuracy of the teacher's position estimation, and eliminate the positioning error caused by the height difference of the teacher. The change trend of the first target object and the change trend of the second target object can represent the movement direction of the teacher relative to the teacher camera and the student camera. The change trend of the first target object and the change trend of the second target object can be used to jointly analyze the movement direction of the teacher in the classroom, ensure the accuracy of the teacher's movement direction estimation, and use the change trend to eliminate the problem of frequent switching of the recording screen caused by the teacher's repeated movement near the podium. Therefore, by combining the first target object and the corresponding change trend as well as the second target object and the corresponding change trend, it is possible to accurately analyze whether the teacher is in the state of going up to the podium or down from the podium, eliminating the error caused by the difference in teachers' heights when detecting the teacher's state of going up or down the podium with a single panoramic image, solving the problem in the prior art of being unable to accurately judge the teacher's state of going up or down the podium, and improving the detection accuracy of the teacher's state, thereby improving the accuracy of recording and broadcasting target tracking and optimizing the quality of teaching recording and broadcasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a top view schematic diagram of the recording classroom provided in the embodiment of the present application;

[0061] Figure 2 It is one of the schematic diagrams of the panoramic image of the teacher provided in the embodiment of the present application;

[0062] Figure 3 is a flow chart of a teacher status detection method provided by an embodiment of the present application;

[0063] Figure 4 is a cross-sectional schematic diagram of a recording and broadcasting classroom provided in an embodiment of the present application;

[0064] Figure 5 This is the second schematic diagram of the panoramic image of the teacher provided in the embodiment of the present application;

[0065] Figure 6 This is the third schematic diagram of the panoramic image of the teacher provided in the embodiment of the present application;

[0066] Figure 7 It is one of the schematic diagrams of the panoramic image of students provided in the embodiment of the present application;

[0067] Figure 8 This is the second schematic diagram of the panoramic image of students provided in the embodiment of the present application;

[0068] Fig. 9 This is the third schematic diagram of the panoramic image of students provided in the embodiment of the present application;

[0069] Fig.10 It is a structural diagram of a teacher status detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only part of the present application is shown in the accompanying drawings, but not all of the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0071] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0072] In the more common existing implementation methods, Figure 1 Schematic diagram of a top view of a recording classroom provided in an embodiment of the present application. Figure 1As shown, the teacher camera 13 is installed at the rear of the recording classroom, the student camera 12 is installed at the front of the recording classroom, and the smart blackboard 11 is installed below the student camera 12. The smart blackboard 11 consists of blackboards on the left and right sides and an interactive large board in the middle. After the class is opened, the teacher camera 13 and the student camera 12 are started. The teacher camera 13 captures the teacher's panoramic image and obtains the teacher's close-up picture by cropping. The teacher's panoramic image includes the whole picture of the teacher, the smart blackboard 11 and the podium 15. The teacher's close-up picture is a small picture centered on the teacher cut from the teacher's panoramic image. The teacher's close-up picture has the effect of tracking the teacher. The student camera 12 captures the student's panoramic image and obtains the student's close-up picture by cropping. The student's panoramic image includes all students in the student area 14. The student's close-up picture is a small picture of the area where the standing students are cut from the student's panoramic image after detecting the standing students in the student's panoramic image. The student's close-up picture has the effect of detecting students standing up to answer questions. The recording and broadcasting system also includes a recording and broadcasting host. The interaction in the smart blackboard can transmit the corresponding screen image to the recording and broadcasting host. The teacher camera and the student camera transmit the captured teacher panoramic image and student panoramic image to the recording and broadcasting host. The recording and broadcasting host obtains the teacher close-up image and the student close-up image based on the teacher panoramic image and the student panoramic image respectively, and finally selects one or more synthesized recording and broadcast images from the screen image, teacher panoramic image, student panoramic image, teacher close-up image and student close-up image.

[0073] The teacher's behavior in the classroom can be divided into the state of going on the podium and the state of leaving the podium. When generating the recording and broadcasting screen, the recording and broadcasting host can record the teacher's close-up picture when the teacher goes on the podium and follow the teacher's movement, and record the panoramic image of the students when the teacher leaves the podium. Figure 2 FIG. 1 is one of the schematic diagrams of the panoramic image of the teacher provided in the embodiment of the present application. Figure 2As shown, the tracking area 16 in the teacher's panoramic image is obtained by manually drawing the image area where the smart blackboard is located during the deployment stage of the recording and broadcasting system. When the teacher walks into the tracking area 16, it is determined that the teacher is in the state of going to the podium, and when the teacher leaves the tracking area 16, it is determined that the teacher is in the state of going to the podium. Because the teacher's camera converts the physical world coordinate system into the camera coordinate system during the process of shooting from the back of the classroom to the front, the depth information (the distance between the teacher and the lens) is lost. Therefore, the teacher's panoramic image captured by the teacher's camera will appear in the case where the teacher leaves the podium and comes to the front row of students, and part of the teacher's limbs are still in the tracking area. In this regard, the teacher's up and down podium state can be distinguished by a clearer detection boundary. Specifically, when the teacher's shoulder part in the teacher's panoramic image exceeds the tracking area, the teacher is determined to be in the state of going to the podium, and when half of the teacher's head is lower than the tracking area, the teacher is determined to be in the state of going to the podium. However, due to the different heights of different teachers, when the tracking area is fixed, it is impossible to accurately judge whether the teacher is going to the podium or going down the podium only by the positional relationship between the teacher's body and the tracking area. Moreover, the tracking area is drawn manually by the deployment personnel, which is affected by the deployment personnel's professionalism and has the problem of inaccurate and unreasonable drawing. Therefore, when the recording and broadcasting host tracks the corresponding recording and broadcasting target based on the teacher's up and down podium status, the recording and broadcasting target tracking problem will appear, resulting in poor quality of teaching recording and broadcasting.

[0074] Moreover, when the teacher moves repeatedly near the detection boundary, the recording and broadcasting host will recognize that the teacher is repeatedly switching between the upper and lower podium states, thereby frequently switching the content of the recording and broadcasting screen, further reducing the quality of teaching recording and broadcasting. At present, the solution adopted by the industry to solve the problem of frequent switching of recording and broadcasting screens by the recording and broadcasting host is mainly based on the transformation of the director logic. For example, the teacher's panoramic image is fixed as the main image, and the switching of the teacher's close-up image is cancelled, so as to avoid the side effects of judging the upper and lower podium states. Alternatively, a certain delay strategy can be implemented to avoid the chaotic picture caused by the teacher's repeated movement near the detection boundary, but this will be affected by the accuracy of the tracking area drawing and the teacher's height. Therefore, the debugged delay strategy cannot be applied to all scenarios, resulting in each teacher having to do a debugging before class, which increases the recording and broadcasting cost.

[0075] To solve the above problems, the present embodiment provides a teacher status detection method, which detects the teacher area and the change trend of the teacher area in the corresponding images through the teacher panoramic image and the student panoramic image, and jointly detects the teacher's up and down podium status through the teacher area in the two panoramic images and the corresponding change trends, thereby eliminating the error caused by the height difference of teachers when detecting the teacher's up and down podium status with a single panoramic image, and utilizes the change trend to eliminate the problem of frequent switching of the recording and broadcasting screen caused by the teacher repeatedly moving near the podium, thereby improving the detection accuracy of the teacher's status, thereby improving the accuracy of recording and broadcasting target tracking, and optimizing the quality of teaching recording and broadcasting.

[0076] The teacher status detection method provided in this embodiment can be executed by a teacher status detection device, which can be implemented by software and / or hardware, and can be composed of two or more physical entities, or one physical entity. For example, the teacher status detection device can be a recording and broadcasting system, or a recording and broadcasting host of the recording and broadcasting system.

[0077] The teacher status detection device is installed with at least one type of operating system, wherein the operating system includes but is not limited to Android system, Linux system and Windows system. The teacher status detection device can install at least one application based on the operating system, and the application can be an application that comes with the operating system or an application downloaded from a third-party device or server. In this embodiment, the teacher status detection device has at least an application that can execute the teacher status detection method.

[0078] For ease of understanding, this embodiment is described by taking the recording and broadcasting host as an example of the main body executing the teacher status detection method.

[0079] Figure 3 Flowchart of a method for detecting teacher status provided by an embodiment of the present application. Figure 3 As shown, the steps of the teacher status detection method include:

[0080] S110. Identify a first target object in the teacher's panoramic image, and identify a second target object in the student's panoramic image. Both the first target object and the second target object are imaging results with the teacher as the shooting target.

[0081] Among them, the teacher panoramic image is an image taken by the teacher camera located at the back of the classroom, which includes the whole picture of the teacher, the smart blackboard and the podium. The student panoramic image is an image taken by the student camera located at the front of the classroom, which includes all students and may even include the teacher. Figure 4 Schematic diagram of the cross section of the recording classroom provided in the embodiment of the present application. Figure 4 As shown, the shooting area of ​​the teacher camera 13 covers the entire area of ​​the podium 15 and the front part of the student area 14. When the teacher 17 is on the podium 15, the teacher camera 13 can shoot the whole body of the teacher 17. When the teacher 17 is under the podium 15, the teacher camera 13 can shoot the whole body or upper body of the teacher 17. The shooting area of ​​the student camera 12 covers the area above the podium 15 and the entire student area 14. When the teacher 17 is on the podium 15, the student camera 12 can shoot the head of the teacher 17 or cannot shoot the teacher 17. When the teacher 17 is under the podium 15, the student camera 12 can shoot the upper body or even the whole body of the teacher 17.

[0082] After the class starts, the recording and broadcasting host controls the teacher camera and the student camera to turn on. The teacher camera captures the teacher's panoramic image and transmits it to the recording and broadcasting host, and the student camera captures the student's panoramic image and transmits it to the recording and broadcasting host. The recording and broadcasting host identifies the first target object and the second target object in the teacher's panoramic image and the student's panoramic image respectively. Among them, the target object is the teacher teaching in the classroom, the first target object is the pixel area corresponding to the teacher detected in the teacher's panoramic image, and the second target object is the pixel area corresponding to the teacher detected in the student's panoramic image.

[0083] Need to explain, by Figure 4 As shown in the content, when the teacher stands on the podium and is close to the smart blackboard, the student camera cannot capture the teacher, so the student panoramic image does not contain the second target object. Accordingly, the second target object may not be recognized in the student panoramic image. If the student panoramic image cannot recognize the second target object, it can be directly determined that the teacher is on the podium, and then according to the first target object in the teacher panoramic image and the corresponding change trend, it can be determined whether the teacher is statically on the podium or dynamically on and off the podium.

[0084] In one embodiment, the teacher's camera does not rotate but shoots at a fixed angle, and the center of the teacher's camera is always aligned with the center of the blackboard image area. The blackboard image area is the image area where the teacher's camera captures the smart blackboard, that is, the blackboard image area includes the interactive board and the image areas corresponding to the blackboards on both sides. When the center of the teacher's camera is aligned with the center of the blackboard image area, it indicates that the teacher's camera is aimed at the interactive board to capture the entire podium.

[0085] In another embodiment, the teacher camera will rotate with the teacher, resulting in the camera not being in the initial state at the beginning of the class. At the beginning of the class, the teacher will stand in the middle of the podium, that is, in front of the center of the interactive board. In this regard, for the rotating teacher camera, the teacher camera can be reset to the initial state before the formal recording, so that the center of the teacher camera in the initial state is aligned with the center of the blackboard image area, and the teacher is located in the center of the teacher camera's picture.

[0086] Optionally, when resetting the teacher's camera, the teacher's camera can be used to shoot an initial panoramic image, and the blackboard image area in the initial panoramic image can be detected; the center of the teacher's camera's screen can be adjusted to align with the center of the blackboard image area according to the blackboard image area. For example, before the recording and broadcasting is officially started, the recording and broadcasting host obtains the first image shot by the teacher's camera as the initial panoramic image. Since the interactive large board is a rectangle, the largest rectangular area in the initial panoramic image can be detected as the blackboard image area, and then the center coordinates of the blackboard image area are identified. The shooting area of ​​the teacher's camera is adjusted according to the coordinate deviation between the center coordinates of the blackboard image area and the center coordinates of the initial panoramic image, so that the center of the teacher's camera's screen after adjustment is aligned with the center of the blackboard image area. After resetting the teacher's camera to the initial state, the recording and broadcasting host can officially start recording the classroom screen, and in the process of recording the classroom screen, the teacher's up and down podium status is determined according to the teacher's panoramic image captured by the teacher's camera and the student's panoramic image captured by the student's camera, so as to switch the classroom screen according to the teacher's up and down podium status. This embodiment accurately resets the teacher's camera to the initial state through the blackboard image area in the initial panoramic image taken by the teacher's camera before the recording starts, so as to ensure that the teacher's camera starts to capture the teacher's panoramic image in the initial state, so that the teacher's panoramic image is displayed with the interactive large board and the teacher in the center and includes the entire picture of the podium, thereby optimizing the recording effect of the classroom picture and ensuring the accuracy of subsequent teacher status detection.

[0087] In one embodiment, the recording and broadcasting host can identify the first target object and the second target object in the teacher panoramic image and the student panoramic image respectively through a semantic segmentation model or a target detection model. For example, based on a preset teacher face image, the human body frame corresponding to the teacher can be identified in the teacher panoramic image and the student panoramic image respectively through a pre-trained target detection model, and the human body frame corresponding to the teacher in the teacher panoramic image is used as the first target object, and the human body frame corresponding to the teacher in the student panoramic image is used as the second target object. When the teacher panoramic image and the student panoramic image contain the teacher's face picture, the first target object and the second target object can be accurately detected in the teacher panoramic image and the student panoramic image through the preset teacher face image, but the teacher in the classroom is either facing the teacher camera or the student camera, that is, there is always one camera that cannot capture the teacher's face picture, resulting in a pixel area in which the teacher cannot be identified in the panoramic image, affecting the detection accuracy. Moreover, the detection of the teacher in the panoramic image based on the teacher's face image requires a high computing power for the recording and broadcasting host, which affects the recording efficiency of the recording and broadcasting picture.

[0088] Optionally, since the teacher always stands during the class, the standing human body in the teacher's panoramic image and the student's panoramic image can be identified through a semantic segmentation model or a target detection model, and the identified standing human body is used as the first target object and the second target object respectively. Compared with the method of detecting the target object in the panoramic image through the face image, directly detecting the standing human body in the panoramic image as the target object requires less computing power, and the corresponding standing human body can be effectively identified in both panoramic images.

[0089] In this embodiment, the steps of identifying the first target object and the second target object specifically include S1101-S1104:

[0090] S1101, performing standing human body detection on the teacher panoramic image to obtain a first standing human body frame.

[0091] Exemplarily, the teacher's panoramic image is input into a pre-trained standing human body detection model to obtain the first standing human body frame output by the standing human body detection model. The first standing human body frame circles the area where the teacher is located in the teacher's panoramic image. The training process of the first standing human body frame is: collect multiple teacher panoramic sample images, the teacher panoramic sample images mark the standing human body in the image through the detection frame, input the teacher panoramic sample image into the untrained neural network model to obtain the detection result output by the neural network model, calculate the loss value of the detection result and the detection frame marked in the teacher panoramic sample image, and reversely optimize the model parameters of the neural network model through the loss value. When the model parameters converge or the number of training times reaches the upper limit, the neural network model has the function of accurately marking the standing human body in the image through the detection frame. At this time, the neural network model can be used as a standing human body detection model.

[0092] Generally, only the teacher who is giving the lecture is standing in the classroom. Therefore, when the standing human body detection model detects a standing human body frame, the standing human body frame can be determined as the first standing human body frame that circles the area where the teacher is located. In special cases, such as when a student stands to answer a question, the standing human body detection model can detect multiple standing human body frames. In this case, it is necessary to select the first standing human body frame that circles the area where the teacher is located from the multiple standing human body frames.

[0093] Optionally, when teaching, the teacher generally faces the teacher camera, and the students face away from the teacher camera. A standing human frame containing a face can be identified in multiple standing human frames, so that the standing human frame containing the face is determined as the first standing human frame. Of course, when the teacher walks back to the podium from the student area, he will face away from the teacher camera. If there happens to be a student standing to answer questions at this time, none of the multiple standing human frames contain faces. At this time, the historical facing time of the objects in each standing human frame can be counted. The historical facing time can be understood as the cumulative time from the beginning of the class recording by the teacher camera to the current recording of the teacher and the student facing the teacher camera. The historical facing time can be obtained by accumulating the number of frames in which the faces of the objects in each standing human frame appear in the teacher's panoramic image collected by the teacher camera in history. Since the teaching teacher faces the teacher camera most of the time in the classroom, the standing human frame corresponding to the object with the longest historical facing time can be determined as the first standing human frame.

[0094] In addition, at the beginning of the class, the teacher stands on the podium and the students sit in their respective seats. Therefore, in the first panoramic image of the teacher captured by the teacher's camera, only the teacher is standing, that is, when the teacher and the students first appear in the teacher's camera, only the teacher is standing. In this regard, the initial screen in which the objects in multiple standing human frames first appear in the teacher's camera in the current class can be obtained, and the standing human frame corresponding to the object in the standing state in the initial screen is determined as the first standing human frame.

[0095] It should be noted that if a student in the front row is standing and is mistakenly identified as a teacher, the student's standing state may be mistakenly determined as the teacher's state on the podium, resulting in the problem of incorrect target tracking during recording. However, this embodiment can accurately identify the first standing human frame containing the teacher's area in the teacher's panoramic image, avoiding the problem of subsequent misjudgment of state, and improving the accuracy of target tracking during recording.

[0096] S1102, performing Kalman filtering on the first standing human body frame to obtain a first target object.

[0097] Exemplarily, the human body may have limb shaking. In order to improve the detection accuracy of the first standing human body frame, the first standing human body frame may be subjected to a Kalman filter process to obtain a stable first standing human body frame, and the first standing human body frame after filtering is determined as the first target object. When the first standing human body frame is subjected to Kalman filtering, the state transfer equation and observation equation of the first standing human body frame are obtained, and the first standing human body frame at the current moment is predicted by the state transfer equation and the first standing human body frame obtained after filtering at the previous moment. Based on the first standing human body frame detected in the teacher's panoramic image at the current moment and the observation equation, the first standing human body frame predicted at the current moment is corrected to obtain the first standing human body frame after filtering at the current moment. The first standing human body frame after Kalman filtering maintains a relatively stable change on the time axis with the first standing human body frame detected in history, reducing the fluctuation of the detection frame caused by limb shaking, ensuring the accuracy of the change trend of the first target object determined subsequently, thereby improving the accuracy of teacher state detection.

[0098] This embodiment detects the first standing human frame including the area where the teacher is located in the teacher's panoramic image, and performs Kalman filtering on the first standing human frame to obtain a high-precision first target object, which is beneficial to improving the accuracy of subsequent teacher status detection.

[0099] S1103, performing standing human body detection on the student panoramic image to obtain a second standing human body frame.

[0100] Exemplarily, the student panoramic image is input into a pre-trained standing human body detection model to obtain a second standing human body frame output by the standing human body model, wherein the second standing human body frame encircles the area where the teacher is located in the student panoramic image.

[0101] When the standing human body detection model detects a standing human body frame, the standing human body frame may be determined as a second standing human body frame that encloses the area where the teacher is located.

[0102] When teaching, the teacher usually faces away from the student camera, while the students face the student camera. Therefore, when the standing human body detection model detects multiple standing human body frames, a standing human body frame that does not contain a face can be identified from the multiple standing human body frames, and the standing human body frame that does not contain a face can be determined as the second standing human body frame.

[0103] Optionally, when multiple standing human frames are identified in the student panoramic image, the historical back-to-back time of the objects in the multiple standing human frames is determined, and the standing human frame corresponding to the object with the longest historical back-to-back time is determined as the second standing human frame. For example, when the teacher walks back to the podium from the student area, he will face the student camera. If there is a student standing to answer questions at this time, the multiple standing human frames all contain faces, and the historical back-to-back time of the objects in each standing human frame can be counted. The historical back-to-back time can be understood as the cumulative time from the beginning of the class recording and broadcasting of the student camera to the current recording of the teacher and the student with their backs to the student camera. The historical back-to-back time can be obtained by accumulating the number of frames in which the faces of the objects in each standing human frame do not appear in the student panoramic image historically collected by the student camera. Since the teaching teacher is facing away from the student camera most of the time in the classroom, the standing human frame corresponding to the object with the longest historical back-to-back time can be determined as the second standing human frame. In this embodiment, when multiple standing human frames are detected in the student panoramic image, the second standing human frame containing the teacher's area is accurately identified based on the historical back-to-back time of the objects in the standing human frame, thereby improving the detection accuracy of the second target object, thereby improving the accuracy of subsequent teacher status detection.

[0104] In addition, when multiple standing human frames are identified in the student panoramic image, the initial screen in which the objects in the multiple standing human frames first appear in the student camera is determined, and the standing human frame corresponding to the object in the standing state in the initial screen is determined as the second standing human frame. Exemplarily, the time when the teacher first appears in the student camera is when the teacher walks to the student area, and the time when the student first appears in the student camera is before the class starts, that is, when the teacher and the student first appear in the student camera, only the teacher is standing. In this regard, the initial screen in which the objects in the multiple standing human frames first appear in the student camera can be obtained, where the initial screen corresponding to the student and the initial screen corresponding to the teacher are not the same initial screen, because the teacher stands on the podium at the beginning of the class and the student camera does not capture the teacher. The standing human frame corresponding to the object in the standing state in the corresponding initial screen can be determined as the second standing human frame. In this embodiment, when multiple standing human frames are detected in the student panoramic image, the second standing human frame containing the area where the teacher is located is accurately identified based on the state in which the objects in each standing human frame first appear in the student camera, thereby improving the detection accuracy of the second target object and thus improving the accuracy of subsequent teacher state detection.

[0105] It should be noted that if the student in the front row is standing and the student is mistakenly identified as a teacher, the student's standing state may be mistakenly determined as the teacher's state on the podium, resulting in the problem of incorrect target tracking in the recording. However, this embodiment can accurately identify the second standing human frame containing the teacher's area in the student panoramic image, avoiding the problem of subsequent misjudgment of state, and improving the accuracy of target tracking in the recording.

[0106] S1104, performing Kalman filtering on the second standing human body frame to obtain a second target object.

[0107] Exemplarily, in order to improve the detection accuracy of the second standing human frame, the second standing human frame may be subjected to a Kalman filter process to obtain a stable second standing human frame, and the second standing human frame after the filtering process is determined as the second target object. The second standing human frame after the Kalman filter process maintains a relatively stable change on the time axis with the second standing human frame detected historically, reducing the fluctuation of the detection frame caused by limb shaking, ensuring the accuracy of the change trend of the second target object determined subsequently, thereby improving the accuracy of the teacher status detection.

[0108] This embodiment detects a second standing human frame including the teacher's area in the student panoramic image and performs Kalman filtering on the second standing human frame to obtain a high-precision second target object, which is beneficial to improving the accuracy of subsequent teacher status detection.

[0109] S120. Determine a change trend of a first target object based on the teacher panoramic image and the historical teacher panoramic image, and determine a change trend of a second target object based on the student panoramic image and the historical student panoramic image.

[0110] The history teacher panoramic image is the remaining teacher panoramic images except the teacher panoramic image captured by the teacher camera at the current time point within the preset time window. The preset time window takes the current time point as the end time point and the time point obtained by moving the current time point forward by a preset time length as the start time point. The change trend of the first target object represents the change of the size of the first target object or the proportion of the first target object in the teacher panoramic image within the preset time window. Specifically, the size or proportion of the first target object may increase, decrease, or change back and forth within the preset time window.

[0111] Exemplarily, a panoramic image of the history teacher is obtained according to a preset time window, and the first target object detected in the panoramic image of the history teacher in the past is obtained. According to the size or proportion of the first target object in each teacher panoramic image within the preset time window, it is determined that the change trend of the first target object is getting bigger, getting smaller, or changing back and forth in size.

[0112] The historical student panoramic images are the remaining student panoramic images except the student panoramic images captured by the student camera at the current time point within the preset time window. The change trend of the second target object represents the change of the size of the second target object or the proportion of the second target object in the student panoramic image within the preset time window, which may be that the size or proportion of the second target object increases, decreases, or changes back and forth within the preset time window.

[0113] Exemplarily, historical student panoramic images are obtained according to a preset time window, and the second target objects detected in the past by the historical student panoramic images are obtained. According to the size or proportion of the second target objects in each student panoramic image within the preset time window, it is determined that the change trend of the second target object is getting bigger, getting smaller, or changing back and forth in size.

[0114] S130. Determine a target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend, wherein the target object state includes an on-stage state and a off-stage state.

[0115] Exemplarily, the first target object and the second target object can respectively represent the position of the teacher relative to the teacher camera and the student camera, and the first target object and the second target object can be used to jointly analyze whether the teacher is on the podium. The change trend of the first target object and the change trend of the second target object can represent the movement direction of the teacher relative to the teacher camera and the student camera, and the change trend of the first target object and the change trend of the second target object can be used to jointly analyze whether the teacher's action is walking towards the podium or away from the podium. Combining the first target object and the corresponding change trend and the second target object and the corresponding change trend, it can be determined that the teacher is in the state of going up to the podium or going down to the podium. For example, if the first target object and the second target object analyze that the teacher's position is on the podium, and the change trend of the first target object and the change trend of the second target object analyze that the teacher's action is walking towards the podium, then it is determined that the teacher is in the state of going up to the podium. If the first target object and the second target object analyze that the teacher's position is under the podium, and the change trend of the first target object and the change trend of the second target object analyze that the teacher's action is away from the podium, then it is determined that the teacher is in the state of going down to the podium.

[0116] It should be noted that the state of going up to the podium and the state of going down the podium are dynamic states, that is, the teacher is in motion when he is in the state of going up to the podium or going down the podium. The existing technology uses real-time panoramic images of teachers to detect when the teacher is in the state of going up to the podium and when he is going down the podium, which belongs to the use of static data to judge the dynamic state of the teacher. Static data cannot accurately reflect the current teacher's behavior and actions. Therefore, determining the teacher's state through a single panoramic image of the teacher will also introduce motion detection errors. The change trend of the first target object and the change trend of the second target object used in this embodiment are dynamic data, which can accurately reflect the current teacher's behavior and actions, ensuring the accuracy of teacher status detection.

[0117] refer to Figure 4 When the teacher is located at the podium, the teacher is close to the student camera and far away from the teacher camera, the first target object accounts for a smaller proportion in the teacher's panoramic image, and the second target object accounts for a larger proportion in the student's panoramic image. When the teacher is located in the student area, the teacher is close to the teacher camera and far away from the student camera, the first target object accounts for a larger proportion in the teacher's panoramic image, and the second target object accounts for a smaller proportion in the student's panoramic image. In this regard, the teacher's status can be determined based on a first ratio of the first target object to the teacher's panoramic image, a change trend of the first target object, a second ratio of the second target object to the student's panoramic image, and a change trend of the second target object.

[0118] For example, when the first ratio is less than or equal to the first preset threshold and the second ratio is greater than or equal to the second preset threshold, and the change trend of the first target object is decreasing and the change trend of the second target object is increasing, the teacher's state is determined to be on the podium; when the first ratio is greater than the first preset threshold and the second ratio is less than the second preset threshold, and the change trend of the first target object is increasing and the change trend of the second target object is decreasing, the teacher's state is determined to be off the podium. Among them, the first preset threshold is the maximum proportion of the corresponding imaging result in the teacher's panoramic image when the teacher is on the podium, and the second preset threshold is the minimum proportion of the corresponding imaging result in the student's panoramic image when the teacher is on the podium. When the first ratio is less than or equal to the first preset threshold and the second ratio is greater than or equal to the second preset threshold, it indicates that the teacher is currently on the podium. When the change trend of the first target object is decreasing and the change trend of the second target object is increasing, it indicates that the teacher is walking towards the student camera and away from the teacher camera, so as to determine that the teacher has made an action of walking towards the podium within the preset time window, combined with the teacher's current position on the podium, it can be determined that the teacher is currently in the state of going to the podium. When the first ratio is greater than the first preset threshold and the second ratio is less than the second preset threshold, it indicates that the teacher is not currently on the podium. When the change trend of the first target object is increasing and the change trend of the second target object is decreasing, it indicates that the teacher is walking towards the teacher camera and away from the student camera, thereby determining that the teacher has made an action away from the podium within the preset time window. Combined with the teacher's current position not on the podium, it can be determined that the teacher is currently off the podium.

[0119] refer to Figure 4 When the teacher is at the podium, the student camera can capture the teacher's upper body, and when the teacher is in the student area, the student camera can capture the teacher's whole body. When the teacher is at the podium, the teacher camera can capture the teacher's upper body in the interactive large board area, and when the teacher is in the student area, the teacher camera can capture the teacher's head in the interactive large board area or the teacher is not in the interactive large board area. In this regard, the body parts of the first target object and the second target object can also be analyzed to determine the teacher's location, and then the teacher's status can be determined in combination with the change trend of the first target object and the change trend of the second target object. Optionally, the steps of determining the teacher's status based on body parts and change trends specifically include S1301-S1303:

[0120] S1301. When a first target object overlaps with a blackboard image region in a teacher's panoramic image, determine a first body part of the first target object in the blackboard image region.

[0121] Exemplarily, the largest rectangular area in the teacher's panoramic image is identified as the blackboard image area, and whether there is overlap between the blackboard image area and the first target object is determined based on the pixel coordinates of the blackboard image area and the pixel coordinates of the first target object. When the first target object overlaps with the blackboard image area, the first body part of the first target object in the blackboard image area can be determined.

[0122] S1302: Determine a second body part of a second target object.

[0123] Exemplarily, a second body part of the second target object in the student panoramic image is determined.

[0124] S1303: Determine the state of the target object according to the change trend of the first body part, the second body part, the first target object, and the change trend of the second target object.

[0125] Exemplarily, it is possible to determine whether the teacher is on the podium or under the podium based on the first body part and the second body part, and determine whether the teacher makes an action away from the podium or close to the podium based on the change trend of the first target object and the change trend of the second target object. The teacher is determined to be in the state of being on the podium or being off the podium based on the position and action of the teacher. When determining the position of the teacher based on the first body part and the second body part, a first confidence score of the teacher being on the podium can be evaluated by the first body part, and a second confidence score of the teacher being on the podium can be evaluated by the second body part. The two confidence scores are weighted and summed to obtain a weighted value. When the weighted value is greater than a preset confidence threshold, it is determined that the teacher is on the podium, otherwise it is determined that the teacher is under the podium.

[0126] Optionally, the position of the teacher can also be determined based on the type of parts included in the first body part and the second body part and the size relationship of different types of parts, and the teacher's state can be determined in combination with the change trend of the first target object and the change trend of the second target object. The specific implementation process is: when the change trend of the first target object and the change trend of the second target object meet the first condition, and the first body part and the second body part meet the second condition, the state of the target object is determined to be the state of going up to the podium; when the change trend of the first target object and the change trend of the second target object meet the third condition, and the first body part and the second body part do not meet the second condition, the state of the target object is determined to be the state of going down to the podium. Among them, the first condition includes that the change trend of the first target object is getting smaller, and the change trend of the second target object is getting larger; the third condition includes that the change trend of the first target object is getting larger, and the change trend of the second target object is getting smaller; the second condition is that the head of the first body part is larger than the non-head, and the second body part includes the upper body and / or the feet of the second body part do not touch the bottom of the student's panoramic image.

[0127] For example, Figure 5 and Figure 6 It is a schematic diagram of a panoramic image of a teacher provided in an embodiment of the present application. Figure 7-Figure 9 It is a schematic diagram of a panoramic image of students provided in an embodiment of the present application. Figure 5 and Figure 7 Teacher 17 is on the podium. Figure 6 , Figure 8 and Fig. 9 Teacher 17 is under the podium. Figure 5 and Figure 6 As shown, the first body part in the blackboard image area 18 includes the head and the non-head area. When the teacher 17 is on the podium, the non-head area is greater than or equal to the head area. When the teacher 17 is under the podium, the head area is greater than the non-head area. Figure 7-Figure 8 As shown, when the teacher 17 is on the podium, the second body part does not include a complete upper body, and when the teacher 17 is under the podium, the second body part includes a complete upper body. Fig. 9 , when the teacher 17 is shorter, the student camera may also capture the complete upper body of the teacher 17 on the podium, but when the teacher walks into the student area, the student camera can capture the teacher 17's feet and the teacher 17's feet do not touch the bottom of the student panoramic image. Therefore, when the non-head part of the first body part is greater than or equal to the head and the second body part does not contain the complete upper body, it is determined that the teacher is on the podium, or, when the non-head part of the first body part is greater than or equal to the head and the second body part does not contain the feet or the feet touch the bottom of the student panoramic image, it is determined that the teacher is on the podium. When the head part of the first body part is greater than the non-head part and the second body part contains the complete upper body, it is determined that the teacher is under the podium, or, when the head part of the first body part is greater than the non-head part and the feet of the second body part do not touch the bottom of the student panoramic image, it is determined that the teacher is under the podium.

[0128] It should be noted that there will be certain errors when judging the teacher's position based on the first body part or the second body part alone, but when the two are combined to judge the teacher's position, most of the errors can be eliminated, thereby accurately detecting the teacher's position. In addition, when determining the teacher's position, the positional relationship between the foot in the second body part and the bottom of the student's panoramic image can also be used to determine the teacher's position, so as to convert the judgment field of view from the top to the bottom. Even if the teacher's height is different, the measurement basis of their feet on the ground is consistent and there is no difference. Therefore, the detection error caused by the height difference can be eliminated, and the detection accuracy of the teacher's position is improved.

[0129] When it is determined that the teacher is located at the podium based on the first body part and the second body part, if the changing trend of the first target object is decreasing and the changing trend of the second target object is increasing, it can be determined that the teacher is in the state of going up to the podium; when it is determined that the teacher is located below the podium based on the first body part and the second body part, if the changing trend of the first target object is increasing and the changing trend of the second target object is decreasing, it can be determined that the teacher is in the state of going down from the podium.

[0130] This embodiment detects the position of the teacher by combining the first body part and the second body part, detects the teacher's behavior by combining the change trend of the first target object and the change trend of the second target object, and detects the teacher's state by combining the position and behavior of the teacher, thereby eliminating the error introduced by single data detection and improving the detection accuracy of the teacher's state. In addition, this embodiment accurately determines whether the teacher is walking towards the podium or away from the podium through the change trend of the first target object becoming larger or smaller and the trend of the second target object becoming smaller or larger, thereby eliminating the motion detection error caused by using static images to judge the dynamic state of the teacher, and then combines the high-precision teacher position determined by combining the first body part and the second body part to accurately detect the teacher's state, thereby improving the detection accuracy of the teacher's state. In addition, this embodiment accurately determines whether the teacher is on the podium or under the podium through the size relationship between the head and the non-head in the first body part and the part type contained in the second body part, thereby eliminating the position detection error caused by the height difference of the teacher, thereby improving the detection accuracy of the teacher's state.

[0131] Optionally, the first condition also includes that the duration of the first target object's shrinking reaches a preset duration, and the duration of the second target object's enlargement reaches a preset duration; the third condition also includes that the duration of the first target object's enlargement reaches a preset duration, and the duration of the second target object's shrinking reaches a preset duration. Exemplarily, the length corresponding to the preset time window can be set to be equal to the preset duration. When the size or proportion of the first target object in each teacher's panoramic image collected by the teacher's camera in the preset time window gradually decreases, it is determined that the change trend of the first target object is shrinking and the duration of shrinking reaches the preset duration. When the size or proportion of the first target object in each teacher's panoramic image collected by the teacher's camera in the preset time window gradually increases, it is determined that the change trend of the first target object is enlarging and the enlargement duration reaches the preset duration. When the size or proportion of the second target object in each student's panoramic image collected by the student's camera in the preset duration window gradually decreases, it is determined that the change trend of the second target object is shrinking and the duration of shrinking reaches the preset duration. When the size or proportion of the second target object in each student's panoramic image collected by the student's camera in the preset time window gradually decreases, it is determined that the change trend of the second target object is enlarging and the enlargement duration reaches the preset duration.

[0132] When determining that the teacher is on the podium, if the change trend of the first target object is getting smaller and the duration of the change reaches the preset duration, and the change trend of the second target object is getting larger and the duration of the increase reaches the preset duration, it indicates that the teacher has been taking the action of walking towards the podium, so it can be determined that the teacher is in the state of going up to the podium. When determining that the teacher is on the podium, if the change trend of the first target object and the second target object is alternating between getting larger and getting smaller, it indicates that the teacher has been making repeated movements at the edge of the podium, so the teacher's state can be kept at the state of the previous moment, avoiding frequent switching of the recording and broadcasting screen, and optimizing the quality of the teaching recording and broadcasting. When determining that the teacher is under the podium, if the change trend of the first target object is getting larger and the duration of the increase reaches the preset duration, and the change trend of the second target object is getting smaller and the duration of the increase reaches the preset duration, it indicates that the teacher has been taking the action of moving away from the podium, so it can be determined that the teacher is in the state of getting down from the podium. When determining that the teacher is under the podium, if the change trend of the first target object and the second target object is alternating between increasing and decreasing, it indicates that the teacher has been making repeated movements at the edge of the podium. Therefore, the teacher's state can be maintained at the state of the previous moment, avoiding frequent switching of the recording and broadcasting screen, and optimizing the quality of the teaching recording and broadcasting. This embodiment accurately judges the teacher's behavior as walking towards the podium, moving away from the podium, or making repeated movements at the edge of the podium through the duration of the change of the decreasing trend or increasing trend of the first target object and the duration of the change of the increasing trend or decreasing trend of the second target object, avoiding misidentifying the teacher's repeated movements at the edge of the podium as movements of going up and down the podium, thereby avoiding frequent switching of the teacher's state, reducing the frequency of screen cutting of the recording and broadcasting screen and the proportion of invalid screens in the recording and broadcasting process, and greatly optimizing the quality of the teaching recording and broadcasting.

[0133] In another embodiment, based on the change trend of the first body part, the second body part, the first target object and the change trend of the second target object, the teacher status can also be determined in combination with the proportion of the first target object in the teacher's panoramic image and the proportion of the second target object in the student's panoramic image. The specific implementation steps include S1304-S1307:

[0134] S1304: When the first target object overlaps with the blackboard image area in the teacher's panoramic image, determine a first body part of the first target object in the blackboard image area.

[0135] S1305: Determine a second body part of the second target object.

[0136] Steps S1304 and S1305 may refer to steps S1301 and S1302.

[0137] S1306. Determine a first ratio between the first target object and the teacher's panoramic image, and determine a second ratio between the second target object and the student's panoramic image.

[0138] Exemplarily, the pixel area of ​​the first target object is determined, and the ratio of the pixel area of ​​the first target object to the pixel area of ​​the teacher's panoramic image is used as the first ratio. The pixel area of ​​the second target object is determined, and the ratio of the pixel area of ​​the second target object to the pixel area of ​​the student's panoramic image is used as the second ratio. The first ratio and the second ratio can be respectively regarded as the proportion of the first target object in the teacher's panoramic image and the proportion of the second target object in the student's panoramic image.

[0139] S1307: Determine the state of the target object according to the first body part, the second body part, the first ratio, the second ratio, the change trend of the first target object, and the change trend of the second target object.

[0140] Exemplarily, it is possible to determine whether the teacher is on the podium or under the podium based on the first body part, the second body part, the first ratio and the second ratio, and determine whether the teacher makes an action away from the podium or close to the podium based on the change trend of the first target object and the change trend of the second target object. The teacher is determined to be on the podium or off the podium based on the position and action of the teacher. Among them, when the teacher's position is determined based on the first body part, the second body part, the first ratio and the second ratio, the first confidence score of the teacher on the podium can be evaluated by the first body part, the second confidence score of the teacher on the podium can be evaluated by the second body part, the third confidence score of the teacher on the podium can be evaluated by the first ratio, and the fourth confidence score of the teacher on the podium can be evaluated by the second ratio. The four confidence scores are weighted and summed to obtain a weighted value. When the weighted value is greater than a preset confidence threshold, it is determined that the teacher is on the podium, otherwise it is determined that the teacher is under the podium.

[0141] Optionally, the teacher's location can be determined based on the type of parts included in the first body part and the second body part and the size relationship of different types of parts, as well as the size relationship between the first ratio and the second ratio and the corresponding preset threshold, and the teacher's state can be determined in combination with the change trend of the first target object and the change trend of the second target object. The specific implementation process is: when the change trend of the first target object and the change trend of the second target object meet the first condition, the first body part and the second body part do not meet the second condition, and the first ratio and the second ratio meet the fourth condition, the target object state is determined to be on the podium state; when the change trend of the first target object and the change trend of the second target object meet the third condition, the first body part and the second body part meet the second condition, and the first ratio and the second ratio do not meet the fourth condition, the target object state is determined to be off the podium state. Among them, the first condition includes that the change trend of the first target object is to become smaller, and the change trend of the second target object is to become larger; the third condition includes that the change trend of the first target object is to become larger, and the change trend of the second target object is to become smaller; the second condition is that the head of the first body part is larger than the non-head, and the second body part includes the upper body and / or the feet of the second body part do not touch the bottom of the student panoramic image; the fourth condition is that the first ratio is less than or equal to the first preset threshold and the second ratio is greater than or equal to the second preset threshold.

[0142] Exemplarily, when the non-head in the first body part is greater than or equal to the head and the second body part does not include a complete upper body, or when the non-head in the first body part is greater than or equal to the head and the second body part does not include a foot or the foot touches the bottom of the student's panoramic image, it indicates that the teacher is on the podium; when the first ratio is less than or equal to the first preset threshold and the second ratio is greater than or equal to the second preset threshold, it also indicates that the teacher is on the podium. Therefore, when the non-head in the first body part is greater than or equal to the head, and the second body part does not include a complete upper body or the second body part does not include a foot or the foot touches the bottom of the student's panoramic image, and the first ratio is less than or equal to the first preset threshold and the second ratio is greater than or equal to the second preset threshold, it can be determined that the teacher is on the podium. When the head in the first body part is greater than the non-head and the second body part includes a complete upper body, or when the head in the first body part is greater than the non-head and the foot of the second body part does not touch the bottom of the student's panoramic image, it indicates that the teacher is on the podium; when the first ratio is greater than the first preset threshold and the second ratio is less than the second preset threshold, it also indicates that the teacher is on the podium. Therefore, when the head is larger than the non-head part in the first body part, and the second body part includes the complete upper torso or the feet of the second body part touch the bottom of the student's panoramic image, and the first ratio is greater than the first preset threshold and the second ratio is less than the second preset threshold, it can be determined that the teacher is under the podium.

[0143] It can be understood that the more data used to detect the teacher's position, the higher the accuracy of the corresponding detected teacher's position. Therefore, when using the first body part, the second body part, the first ratio and the second ratio to detect the teacher's position, the detection accuracy of the teacher's position can be further improved, thereby improving the detection accuracy of the teacher's status.

[0144] When it is determined that the teacher is located at the podium based on the first body part and the second body part, if the changing trend of the first target object is decreasing and the changing trend of the second target object is increasing, it can be determined that the teacher is in the state of going up to the podium; when it is determined that the teacher is located below the podium based on the first body part and the second body part, if the changing trend of the first target object is increasing and the changing trend of the second target object is decreasing, it can be determined that the teacher is in the state of going down from the podium.

[0145] This embodiment detects the position of the teacher by combining the first body part, the second body part, the first ratio and the second ratio, detects the behavior of the teacher by combining the change trend of the first target object and the change trend of the second target object, and detects the state of the teacher by combining the position and the behavior of the teacher, thereby eliminating the error introduced by single data detection and improving the detection accuracy of the teacher's state. In addition, this embodiment accurately determines whether the teacher is walking towards the podium or away from the podium through the change trend of the first target object becoming larger or smaller and the trend of the second target object becoming smaller or larger, thereby eliminating the motion detection error caused by using static images to judge the dynamic state of the teacher, and then combines the high-precision teacher position determined by combining the first body part, the second body part, the first ratio and the second ratio to accurately detect the teacher's state, thereby improving the detection accuracy of the teacher's state. In addition, this embodiment accurately determines whether the teacher is on the podium or under the podium through the size relationship between the first ratio and the first preset threshold and the second ratio and the second preset threshold, thereby improving the detection accuracy of the teacher's state.

[0146] It should be noted that when the teacher is short, the first target object in the teacher's panoramic image does not overlap with the blackboard image area, that is, the blackboard image area contains the body part of the first target object. At this time, the teacher's location can be determined based on the second body part of the second target object, and the teacher's status can be determined based on the change trend of the first target object and the change trend of the second target object. The specific steps include S1308-S1310:

[0147] S1308. When the first target object does not overlap with the blackboard image area in the teacher's panoramic image, determine a second body part of the second target object.

[0148] Exemplarily, the largest rectangular area in the teacher's panoramic image is identified as the blackboard image area, and whether the two overlap is determined based on the pixel coordinates of the blackboard image area and the pixel coordinates of the first target object. When the first target object does not overlap with the blackboard image area, the second body part of the second target object in the student's panoramic image is determined, so that the teacher's location can be subsequently determined based on the second body part.

[0149] S1309: When the second body part includes the feet and the change trend of the first target object and the change trend of the second target object meet the third condition, determine that the state of the target object is the state of getting off the podium.

[0150] refer to Fig. 9 , when the second body part of the teacher 17 includes the feet, whether the feet are in contact with the bottom of the student image or not, it indicates that the teacher 17 has merged into the student area, thereby determining that the teacher 17 is under the podium. If the change trend of the first target object is increasing and the change trend of the second target object is decreasing, then it is determined that the teacher has made an action to move away from the podium, thereby determining that the teacher is in the state of leaving the podium.

[0151] S1310. When the second body part does not include feet and the change trend of the first target object and the change trend of the second target object meet the first condition, determine that the state of the target object is the state of going up to the podium.

[0152] refer to Figure 7 and Figure 8 , when the second body part of the teacher 17 does not include the feet, it can be indicated that the teacher 17 has not yet integrated into the student area, so it is determined that the teacher 17 is on the podium. If the change trend of the first target object is decreasing and the change trend of the second target object is increasing, it is determined that the teacher has made an action of walking towards the podium, so it is determined that the teacher is in the state of going to the podium.

[0153] This embodiment detects the teacher's position based on the second body part when the first target object does not overlap with the blackboard image area, and jointly detects the teacher's behavior and actions based on the change trend of the first target object and the change trend of the second target object. The teacher's status is detected by jointly detecting the teacher's position and behavior and actions, thereby eliminating the error introduced by single data detection and improving the detection accuracy of the teacher's status.

[0154] In addition, based on the second body part, the teacher's position can also be determined by combining the proportion of the first target object in the teacher's panoramic image and the proportion of the second target object in the student's panoramic image. For example, when the second body part includes the feet, and the first ratio of the first target object to the teacher's panoramic image is greater than the first preset threshold and the second ratio of the second target object to the student's panoramic image is less than the second preset threshold, it is determined that the teacher is located under the podium. When the second body part does not include the feet, and the first ratio is less than or equal to the first preset threshold and the second ratio is greater than or equal to the second preset threshold, it is determined that the teacher is located on the podium.

[0155] In summary, the teacher status detection method provided by the embodiment of the present application obtains a first target object by identifying the teacher in the currently collected teacher panoramic image, obtains a second target object by identifying the teacher in the currently collected student panoramic image, determines the change trend of the first target object based on the currently collected and historically collected teacher panoramic images, determines the change trend of the second target object based on the currently collected and historically collected student panoramic images, and jointly determines whether the teacher is in the state of going up to the podium or going down the podium according to the first target object and the corresponding change trend and the second target object and the corresponding change trend. Through the above technical means, the first target object and the second target object can correspond to the position of the teacher relative to the teacher camera and the student camera, and the first target object and the second target object can be used to jointly analyze the position of the teacher in the classroom, ensure the accuracy of the teacher's position estimation, and eliminate the positioning error caused by the height difference of the teacher. The change trend of the first target object and the change trend of the second target object can represent the movement direction of the teacher relative to the teacher camera and the student camera, and the change trend of the first target object and the change trend of the second target object can be used to jointly analyze the movement direction of the teacher in the classroom, ensure the accuracy of the teacher's movement direction estimation, and use the change trend to eliminate the problem of frequent switching of the recording and broadcasting screen caused by the teacher's repeated movement near the podium. Therefore, by combining the first target object and the corresponding change trend as well as the second target object and the corresponding change trend, it is possible to accurately analyze whether the teacher is in the state of going up to the podium or down from the podium, eliminating the error caused by the difference in teachers' heights when detecting the teacher's state of going up or down the podium with a single panoramic image, solving the problem in the prior art of being unable to accurately judge the teacher's state of going up or down the podium, and improving the detection accuracy of the teacher's state, thereby improving the accuracy of recording and broadcasting target tracking and optimizing the quality of teaching recording and broadcasting.

[0156] Fig.10 This is a structural diagram of a teacher status detection device provided in an embodiment of the present application, with reference to Fig.10The teacher status detection device includes: a processor 21, a memory 22, a communication device 23, an input device 24 and an output device 25. The number of processors 21 in the teacher status detection device can be one or more, and the number of memories 22 in the teacher status detection device can be one or more. The processor 21, memory 22, communication device 23, input device 24 and output device 25 of the teacher status detection device can be connected through a bus or other means.

[0157] The memory 22, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the teacher status detection method of any embodiment of the present application. The memory 22 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] The communication device 23 is used for data transmission.

[0159] The processor 21 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 22, that is, implements the above-mentioned teacher status detection method.

[0160] The input device 24 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 25 may include a display device such as a display screen.

[0161] The teacher status detection device provided above can be used to execute the teacher status detection method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0162] An embodiment of the present application also provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute a teacher status detection method, the teacher status detection method comprising: identifying a first target object in a teacher panoramic image, and identifying a second target object in a student panoramic image, the first target object and the second target object being imaging results with the teacher as the shooting target; determining a change trend of the first target object based on the teacher panoramic image and historical teacher panoramic images, and determining a change trend of the second target object based on the student panoramic image and historical student panoramic images; determining a target object state based on the first target object and the corresponding change trend and the second target object and the corresponding change trend, the target object state comprising an on-podium state and a off-podium state.

[0163] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0164] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the above-mentioned teacher status detection method, can also execute the relevant operations in the teacher status detection method provided in any embodiment of the present application.

[0165] The teacher status detection device, storage medium and teacher status detection equipment provided in the above embodiments can execute the teacher status detection method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the teacher status detection method provided in any embodiment of the present application.

[0166] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A teacher status detection method, characterized in that: include: A first target object is identified in the teacher's panoramic image, and a second target object is identified in the student's panoramic image, wherein the first target object and the second target object are imaging results of the teacher; Determine the change trend of the first target object based on the teacher panoramic image and the history teacher panoramic image, and determine the change trend of the second target object based on the student panoramic image and the history student panoramic image, the history teacher panoramic image and the history student panoramic image are acquired through a preset time window, the preset time window takes the current time point as the end time point, and takes the time point obtained by moving the current time point forward by a preset time length as the start time point; Determine a target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend, wherein the target object state includes an on-stage state and a off-stage state; The method includes: determining a teacher's position according to the first target object and the second target object, wherein the teacher's position includes being on a podium and being under a podium; When the changing trend of the first target object is decreasing and the duration of the decrease reaches a preset time, and the changing trend of the second target object is increasing and the increasing time reaches a preset time, it is determined that the teacher's action is walking towards the podium; when the changing trend of the first target object is increasing and the increasing time reaches a preset time, and the changing trend of the second target object is decreasing and the duration of the decrease reaches a preset time, it is determined that the teacher's action is moving away from the podium; the target object state is determined according to the teacher's position and the teacher's action.

2. The teacher status detection method according to claim 1, characterized in that: The determining the target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend includes: In a case where the first target object overlaps with the blackboard image area in the teacher panoramic image, determining a first body part of the first target object in the blackboard image area; determining a second body part of the second target object; The target object state is determined according to the change trend of the first body part, the second body part, the first target object, and the change trend of the second target object.

3. The teacher status detection method according to claim 2, characterized in that: The determining the target object state according to the change trend of the first body part, the second body part, the first target object, and the change trend of the second target object includes: In the case where the change trends of the first target object and the second target object meet a first condition, and the first body part and the second body part do not meet a second condition, determining that the target object state is on the podium state, the first condition comprising that the change trend of the first target object is getting smaller and the duration of getting smaller reaches a preset time, and the change trend of the second target object is getting larger and the duration of getting larger of the second target object reaches a preset time; When the changing trend of the first target object and the changing trend of the second target object satisfy the third condition, and the first body part and the second body part satisfy the second condition, the target object state is determined to be the down-podium state, and the third condition includes that the changing trend of the first target object is increasing and the increasing duration reaches a preset duration, and the changing trend of the second target object is decreasing and the decreasing duration reaches a preset duration.

4. The teacher status detection method according to claim 1, characterized in that: The determining the target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend includes: In a case where the first target object overlaps with the blackboard image area in the teacher panoramic image, determining a first body part of the first target object in the blackboard image area; determining a second body part of the second target object; Determine a first ratio of the first target object to the teacher panoramic image, and determine a second ratio of the second target object to the student panoramic image; The state of the target object is determined according to the first body part, the second body part, the first ratio, the second ratio, a change trend of the first target object, and a change trend of the second target object.

5. The teacher status detection method according to claim 4, characterized in that: Determining the state of the target object according to the first body part, the second body part, the first ratio, the second ratio, the change trend of the first target object, and the change trend of the second target object includes: When the change trends of the first target object and the change trends of the second target object meet the first condition, the first body part and the second body part do not meet the second condition, and the first ratio and the second ratio meet the fourth condition, determine that the state of the target object is the podium state; When the change trend of the first target object and the change trend of the second target object meet the third condition, the first body part and the second body part meet the second condition, and the first ratio and the second ratio do not meet the fourth condition, the target object state is determined to be the down-podium state.

6. The teacher status detection method according to claim 1, characterized in that: The determining the target object state according to the first target object and the corresponding change trend and the second target object and the corresponding change trend includes: In a case where the first target object does not overlap with the blackboard image area in the teacher panoramic image, determining a second body part of the second target object; When the second body part includes a foot, and the change trend of the first target object and the change trend of the second target object meet a third condition, determining that the target object state is a state of getting off the podium; When the second body part does not include feet, and the change trends of the first target object and the second target object satisfy a first condition, it is determined that the state of the target object is the state of going up to the podium.

7. The teacher status detection method according to claim 3 or 5, characterized in that: The second condition is that the head of the first body part is larger than the non-head part, and the second body part includes the upper body and / or the foot of the second body part does not touch the bottom of the student panoramic image.

8. The teacher status detection method according to claim 5, characterized in that: The fourth condition is that the first ratio is less than or equal to a first preset threshold and the second ratio is greater than or equal to a second preset threshold.

9. The teacher status detection method according to claim 1, characterized in that: Before the first target object is identified in the teacher panoramic image, the method further includes: Taking an initial panoramic image by a teacher's camera, and detecting a blackboard image area in the initial panoramic image; According to the blackboard image area, the screen center of the teacher's camera is adjusted to be aligned with the center of the blackboard image area.

10. The teacher status detection method according to claim 1, characterized in that: The step of identifying a first target object in the teacher panoramic image includes: Performing standing human body detection on the teacher panoramic image to obtain a first standing human body frame; Performing Kalman filtering on the first standing human body frame to obtain a first target object; The step of identifying a second target object in the student panoramic image includes: Perform standing human body detection on the student panoramic image to obtain a second standing human body frame; A Kalman filter process is performed on the second standing human body frame to obtain a second target object.

11. The teacher status detection method according to claim 10, characterized in that: The step of performing standing human body detection on the student panoramic image to obtain a second standing human body frame includes: When multiple standing human frames are identified in the student panoramic image, the historical back-turning time of the objects in the multiple standing human frames is determined, and the standing human frame corresponding to the object with the longest historical back-turning time is determined as the second standing human frame; or When multiple standing human frames are identified in the student panoramic image, an initial frame in which the objects in the multiple standing human frames first appear in the student camera is determined, and a standing human frame corresponding to the objects in a standing state in the initial frame is determined as a second standing human frame.

12. A teacher status detection device, characterized in that: include: one or more processors; A memory stores one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the teacher status detection method as described in any one of claims 1-11.

13. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the teacher status detection method as described in any one of claims 1-11 when executed by a computer processor.

Images