Screen directing method, device, storage medium and electronic device

Abstract

The embodiments of the present application provide a screen directing method, device, storage medium and electronic device, wherein the method comprises: a recording and broadcasting host in each classroom generates a trigger instruction in response to a trigger operation of answering questions in a blackboard competition, and sends the trigger instruction to a teacher camera in each classroom; the teacher camera in each classroom collects a panoramic picture of the teacher in real time according to the trigger instruction; the position area of ​​each target student is identified from the teacher's panoramic picture, and each target student picture is determined from the teacher's panoramic picture according to the position area of ​​each target student, and each target student picture is sent to a target recording and broadcasting host; the target recording and broadcasting host receives each target student picture; each target student picture is synthesized to obtain a synthesized picture, and the synthesized picture is sent to a directing platform so that the directing platform displays the synthesized picture, thereby realizing interaction between students in multiple classrooms and improving the teaching interaction effect.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of interactive teaching technology, and in particular to a picture directing method, device, storage medium and electronic device. Background Art

[0002] In the application scenario of interactive teaching, teachers and students in multiple classrooms can interact with each other, thereby improving the teaching interaction effect. For example, students in multiple classrooms watch a teacher in one classroom giving a lecture together.

[0003] However, in the related art, there is little interaction between students in multiple classrooms, and the interactive teaching form is single. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present application provides a picture directing method, device, storage medium and electronic device, which can realize interaction between students in multiple classrooms and improve the teaching interaction effect.

[0005] According to a first aspect of an embodiment of the present application, a method for directing a video is provided, comprising the following steps:

[0006] The recording and broadcasting host in each classroom responds to the trigger operation of the blackboard competition answering question, generates a trigger instruction, and sends the trigger instruction to the teacher's camera in each classroom;

[0007] The teacher's camera in each classroom collects the teacher's panoramic picture in real time according to the trigger command; identifies the location area of ​​each target student from the teacher's panoramic picture, and determines each target student's picture from the teacher's panoramic picture according to the location area of ​​each target student, and sends each target student's picture to the target recording and broadcasting host; wherein the target students are the students answering questions in the competition on the blackboard in each classroom, and the target recording and broadcasting host is the recording and broadcasting host in any classroom in each classroom;

[0008] The target recording and broadcasting host receives the screens of each target student; synthesizes the screens of each target student to obtain a synthesized screen, and sends the synthesized screen to the director platform so that the director platform displays the synthesized screen.

[0009] According to a second aspect of an embodiment of the present application, a picture directing device is provided, including:

[0010] A trigger instruction generation module is used for the recording and broadcasting host in each classroom to respond to the trigger operation of the blackboard competition answering question, generate a trigger instruction, and send the trigger instruction to the teacher camera in each classroom;

[0011] The panoramic picture acquisition module is used for the teacher's camera in each classroom to collect the teacher's panoramic picture in real time according to the trigger instruction; identify the location area of ​​each target student from the teacher's panoramic picture, determine each target student's picture from the teacher's panoramic picture according to the location area of ​​each target student, and send each target student's picture to the target recording and broadcasting host; wherein the target students are the students answering questions in the competition on the blackboard in each classroom, and the target recording and broadcasting host is the recording and broadcasting host in any classroom in each classroom;

[0012] The screen synthesis module is used for the target recording and broadcasting host to receive the screens of each target student; synthesize the screens of each target student to obtain a synthesized screen, and send the synthesized screen to the director platform so that the director platform displays the synthesized screen.

[0013] According to a third aspect of an embodiment of the present application, there is provided an electronic device, comprising a display, a processor and a memory; the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the picture directing method as described above.

[0014] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the picture directing method as described above is implemented.

[0015] In the embodiment of the present application, the recording and broadcasting host in each classroom responds to the trigger operation of the blackboard competition answering, generates a trigger instruction, and sends the trigger instruction to the teacher camera in each classroom; the teacher camera in each classroom collects the teacher's panoramic picture in real time according to the trigger instruction; identifies the position area of ​​each target student from the teacher's panoramic picture, and determines each target student picture from the teacher's panoramic picture according to the position area of ​​each target student, and sends each target student picture to the target recording and broadcasting host; wherein the target students are students answering questions in the competition on the blackboard in each classroom, and the target recording and broadcasting host is the recording and broadcasting host in any classroom in each classroom; the target recording and broadcasting host receives each target student picture; synthesizes each target student picture to obtain a synthesized picture, and sends the synthesized picture to the director platform so that the director platform displays the synthesized picture. The embodiment of the present application realizes the interaction between students in multiple classrooms by synthesizing the target student pictures of students in each classroom answering questions in the competition on the blackboard, and displays the synthesized picture on the director platform, thereby improving the teaching interaction effect.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0017] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is a flow chart of a method for directing pictures according to an embodiment of the present application;

[0020] Figure 2 This is a flowchart of step S10 in the picture directing method according to one embodiment of the present application;

[0021] Figure 3 This is a flowchart of step S20 in the picture directing method according to one embodiment of the present application;

[0022] Figure 4 This is a flowchart of step S23 in the picture directing method according to one embodiment of the present application;

[0023] Figure 5 This is a flowchart of step S20 in the picture directing method according to another embodiment of the present application;

[0024] Figure 6 This is a flowchart of step S30 in the picture directing method according to one embodiment of the present application;

[0025] Figure 7 This is a schematic block diagram of the structure of a picture directing device according to an embodiment of the present application;

[0026] Figure 8 The present invention is a block diagram showing the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0028] It should be clear that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present application.

[0029] When the following description relates to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. The singular forms of "a", "said" and "the" used in the present application and the appended claims are also intended to include the majority form, unless the context clearly indicates other meanings. The words "if" / "if" used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0030] In addition, in the description of this application, unless otherwise specified, "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0031] The screen directing method of the embodiment of the present application can be applied to application scenarios such as teaching scenarios and conference scenarios. In one application scenario of the present application: a student is selected from each of the multiple classrooms, and goes to the podium to answer questions in a competition facing the smart blackboard. Taking classroom A and classroom B as examples, for classroom A, it is hoped that the actual situation of students answering questions in classroom B will be seen, and for classroom B, it is hoped that the actual situation of students answering questions in classroom A will be seen, and for the directing platform other than classroom A / B, it is hoped that the actual situation of two students answering questions in classroom A and classroom B will be seen at the same time.

[0032] Based on this, the present application proposes a picture directing method, device, storage medium and electronic device.

[0033] The following will be combined with the attached Figures 1 to 6 , the picture directing method provided in the embodiment of the present application is introduced in detail.

[0034] See also Figure 1 The screen directing method provided in the embodiment of the present application comprises the following steps:

[0035] S10: The recording and broadcasting host in each classroom generates a trigger instruction in response to the trigger operation of answering questions in the blackboard competition, and sends the trigger instruction to the teacher's camera in each classroom.

[0036] Among them, each classroom is equipped with a recording and broadcasting host and multiple video source devices, and multiple video source devices are connected to the recording and broadcasting host. The video source devices include but are not limited to teacher cameras, student cameras and classroom computer multimedia. The recording and broadcasting host is used to receive video images transmitted by multiple video source devices. The teacher camera is used to capture the teacher's panoramic picture. The teacher camera is generally installed at the back of the classroom, and the lens is aimed at the classroom blackboard and podium area. Specifically, the blackboard can be a smart tablet, an interactive tablet and a smart blackboard. The student camera is used to capture the student's panoramic picture. The student camera is generally installed in the front of the classroom, and the lens is aimed at the student's seat area. The classroom computer multimedia is used to display the courseware screen.

[0037] In the embodiment of the present application, after the blackboard contest begins, the teacher in each classroom can start the course recording operation of the recording and broadcasting host. The recording and broadcasting host in each classroom detects the course recording operation, generates a trigger instruction, and sends the trigger instruction to the teacher's camera in the classroom. Alternatively, the recording and broadcasting host in each classroom recognizes the courseware screen, and if it recognizes that there is courseware content of the blackboard contest in the courseware screen, generates a trigger instruction, and sends the trigger instruction to the teacher's camera in the classroom.

[0038] S20: The teacher's camera in each classroom collects the teacher's panoramic picture in real time according to the trigger instruction; identifies the location area of ​​each target student from the teacher's panoramic picture, and determines each target student's picture from the teacher's panoramic picture according to the location area of ​​each target student, and sends each target student's picture to the target recording and broadcasting host; wherein, the target students are the students answering questions in the competition on the blackboard in each classroom, and the target recording and broadcasting host is the recording and broadcasting host in any classroom in each classroom.

[0039] The teacher's panoramic picture includes the teacher, blackboard, podium and students answering questions. After the blackboard competition begins, at least one student in each classroom stands on the podium and answers questions facing the blackboard. The target students are the students who are answering questions in the classroom.

[0040] In an embodiment of the present application, after the teacher's camera receives a trigger command, it can capture a frame of the teacher's panoramic picture, and the location area of ​​the target student can be identified from the frame of the teacher's panoramic picture. It is also possible to capture multiple frames of continuous teacher's panoramic pictures at preset time intervals, and identify the location area of ​​the target student from multiple frames of continuous teacher's panoramic pictures. Specifically, the teacher's panoramic picture can be input into a target detection model to identify the location area of ​​the target student. Among them, the target detection model is obtained by training based on sample student images. The teacher's panoramic picture can also be input into a blackboard detection model to identify the blackboard area in the teacher's panoramic picture, and then identify the location area of ​​the target student from the blackboard area. Among them, the blackboard detection model is obtained by training based on sample blackboard images.

[0041] After obtaining the position area of ​​the target student, the screen covering the position area of ​​the target student may be cut out from the teacher's panoramic screen to obtain the target student screen. Specifically, the position area of ​​the target student may be cut out from the teacher's panoramic screen to obtain the target student screen. Alternatively, the position area of ​​the target student may be expanded outward by a preset distance to obtain the expanded position area, and the expanded position area may be cut out from the teacher's panoramic screen to obtain the target student screen.

[0042] Send each target student's image to the target recording and broadcasting host. The teacher's camera in each classroom can directly send the target student's image to the directory recording and broadcasting host, or the teacher's camera in each classroom can send the target student's image to the recording and broadcasting host in the same classroom, and the recording and broadcasting host in the same classroom will then send the target student's image to the target recording and broadcasting host.

[0043] S30: The target recording and broadcasting host receives the images of each target student; synthesizes the images of each target student to obtain a synthesized image, and sends the synthesized image to the director platform so that the director platform displays the synthesized image.

[0044] The director platform can be a server or a multimedia display platform.

[0045] In an embodiment of the present application, the target recording and broadcasting host can synthesize each target student screen according to a preset frame rate to obtain a synthesized screen with a preset frame rate. Specifically, the frame rates of each target student screen may be different. For example, the frame rate of the target student screen in classroom A is 10 frames per second, and the frame rate of the target student screen in classroom B is 30 frames per second. During the screen synthesis process, the preset frame rate is 20 frames per second. The two target student screens are synthesized according to the preset frame rate of 20 frames per second to obtain a synthesized screen with a preset frame rate of 20 frames per second.

[0046] The target recording and broadcasting host can also synthesize the target student screens according to the preset screen resolution to obtain a synthesized screen with the preset screen resolution. Specifically, the screen resolutions of the target student screens may be different. For example, the screen resolution of the target student screen in classroom A is 300*400, and the screen resolution of the target student screen in classroom B is 320*480. During the screen synthesis process, the preset screen resolution is 350*450. The two target student screens are synthesized according to the preset screen resolution of 350*450 to obtain a synthesized screen with the preset screen resolution of 350*450.

[0047] After obtaining the synthesized image, the target recording and broadcasting host sends the synthesized image to the director platform, and the director platform can display the synthesized image according to a preset frame rate or a preset image resolution.

[0048] Using the embodiment of the present application, the recording and broadcasting host in each classroom responds to the trigger operation of the blackboard competition answering, generates a trigger instruction, and sends the trigger instruction to the teacher camera in each classroom; the teacher camera in each classroom collects the teacher's panoramic picture in real time according to the trigger instruction; identifies the position area of ​​each target student from the teacher's panoramic picture, and determines each target student picture from the teacher's panoramic picture according to the position area of ​​each target student, and sends each target student picture to the target recording and broadcasting host; wherein the target students are students answering questions in the competition on the blackboard in each classroom, and the target recording and broadcasting host is the recording and broadcasting host in any classroom in each classroom; the target recording and broadcasting host receives each target student picture; synthesizes each target student picture to obtain a synthesized picture, and sends the synthesized picture to the director platform so that the director platform displays the synthesized picture. The embodiment of the present application realizes the interaction between students in multiple classrooms by synthesizing the target student pictures of students in each classroom answering questions in the competition on the blackboard, and displays the synthesized picture on the director platform, thereby improving the teaching interaction effect.

[0049] In an alternative embodiment, see Figure 2 , step S10, including steps S11 to S12, are as follows:

[0050] S11: The recording and broadcasting host in each classroom obtains the courseware images sent in real time by the classroom computer multimedia.

[0051] The classroom computer multimedia is connected to the recording and broadcasting host. Specifically, the classroom computer multimedia is connected to the recording and broadcasting host via an HDMI line, and the courseware screen can be sent to the recording and broadcasting host via the HDMI line.

[0052] S12: The recording and broadcasting host in each classroom identifies the courseware content on the courseware screen, and generates a trigger instruction if the courseware content includes the courseware content of the blackboard competition question answering.

[0053] The courseware content includes but is not limited to text and formulas. The text in the courseware screen can be recognized by optical character recognition (OCR) technology, and the formula in the courseware screen can be recognized by a formula recognition model. The formula recognition model is trained based on sample formula images and their corresponding sample formula recognition results.

[0054] In the embodiment of the present application, the teacher opens the blackboard contest answering courseware in the classroom computer multimedia, and the blackboard contest answering courseware is displayed in the courseware screen sent by the classroom computer multimedia to the recording and broadcasting host. The recording and broadcasting host can detect whether the blackboard contest answering has started by identifying the courseware content on the courseware screen. Specifically, if it is detected that the courseware content includes the content of the blackboard contest answering question, it means that the student's blackboard contest answering question has started, and a trigger instruction is generated.

[0055] By identifying the courseware screen, trigger instructions can be automatically and quickly generated.

[0056] In an alternative embodiment, see Figure 3 , step S20, including steps S211 to S213, are as follows:

[0057] S211: extracting the brightness channel value of the teacher's panoramic picture to obtain a brightness channel picture.

[0058] Among them, the pixel value of each pixel in the brightness channel image is the brightness value.

[0059] In the embodiment of the present application, the teacher's panoramic picture is an RGB image, the RGB image is converted into a YUV image, the Y component of each pixel in the YUV image is extracted, and the brightness channel picture is obtained.

[0060] S212: The area where the pixels whose brightness values ​​in the brightness channel image are greater than the preset brightness threshold are located is used as the blackboard area.

[0061] The preset brightness threshold can be set as required. Specifically, the average brightness value of each pixel on the smart blackboard screen when the smart blackboard is turned on can be used as the preset brightness threshold.

[0062] Among them, the blackboard area is the screen area of ​​the smart blackboard, interactive tablet or smart tablet.

[0063] In an embodiment of the present application, students are answering questions on the smart blackboard, and the smart blackboard is in a bright screen state. The brightness value of the screen area of ​​the smart blackboard is significantly higher than the brightness value of the podium area. By comparing the brightness value of each pixel point in the teacher's panoramic picture with a preset brightness threshold, the area where the pixels with brightness values ​​greater than the preset brightness threshold are located is determined as the smart blackboard area.

[0064] S213: Determine the location area of ​​each target student based on the blackboard area.

[0065] In the embodiment of the present application, the smart blackboard area can be directly used as the location area of ​​each target student. The location of the target student can also be identified in the smart blackboard area, and the location area of ​​each target student can be determined from the smart blackboard area according to the location of the target student.

[0066] By identifying the brightness value of each pixel in the teacher's panoramic image, the location area of ​​each target student can be automatically and quickly determined.

[0067] In an alternative embodiment, see Figure 4 Step S213 includes steps S2131 to S2132, which are as follows:

[0068] S2131: Perform human shape detection on the blackboard area to obtain human shape detection results; wherein the human shape detection results include target student position information.

[0069] The human figure detection model can be used to detect the human figure in the blackboard area. The human figure detection model can be trained by sample pictures containing students answering questions on the blackboard. The human figure detection model includes but is not limited to a machine learning model and a deep learning model.

[0070] The human figure detection result includes the target student human figure area and the target student position information. The target student human figure area includes the target student full body area, the target student upper body area and the target student head and shoulder area. The target student position information includes the position coordinates of the target student in the blackboard area. Specifically, the target student position information can be the position coordinates of the center point of the target student's head.

[0071] S2132: According to the target student location information, determine the location area of ​​each target student from the blackboard area.

[0072] In an embodiment of the present application, the location area of ​​the target student may be a preset area including the target student. The preset area may be a rectangular area. Specifically, a rectangular area is constructed with the location coordinates of the target student as the center and according to a preset width and a preset height. The preset area may be a circular area. Specifically, a circular area is constructed with the location coordinates of the target student as the center and according to a preset radius.

[0073] By performing human figure detection on the blackboard area, the location area of ​​each target student can be automatically and quickly determined from the blackboard area.

[0074] In another alternative embodiment, see Figure 5 Step S20 includes steps S221 to S224, which are specifically as follows:

[0075] S221: extracting brightness channel values ​​of a plurality of frames of continuous teacher panoramic pictures to obtain a plurality of brightness channel pictures.

[0076] In the embodiment of the present application, in each classroom, the teacher camera can collect several consecutive frames of teacher panoramic pictures, extract the brightness channel value of each frame of the teacher panoramic picture, and obtain the brightness channel picture corresponding to each frame of the teacher panoramic picture. The specific process can be referred to step S21, which will not be repeated here.

[0077] S222: The areas where the pixels having brightness values ​​greater than a preset brightness threshold in each brightness channel image are located are all taken as blackboard areas to obtain a plurality of blackboard areas.

[0078] In the embodiment of the present application, the blackboard area in each frame of the teacher's panoramic picture can be determined according to the brightness channel picture corresponding to each frame of the teacher's panoramic picture. The specific process can be referred to step S22, which will not be repeated here.

[0079] S223: Determine the overlapping area of ​​several blackboard areas.

[0080] In the embodiment of the present application, the common area of ​​all blackboard areas is calculated to obtain the overlapping area. Specifically, the position coordinates of the pixels in each blackboard area are obtained, and the area where the pixels with the same position coordinates are located is determined as the overlapping area.

[0081] S224: Determine the location area of ​​each target student based on the overlapping area.

[0082] In the embodiment of the present application, the overlapping area can be directly used as the location area of ​​the target student in the current classroom. It is also possible to perform human figure detection on the overlapping area to obtain the location information of the target student, and determine the location area of ​​each target student from the blackboard area according to the location information of the target student. For the specific process, refer to steps S231 to S232, which will not be repeated here.

[0083] By continuously taking multiple frames of the teacher's panoramic picture, the overlapping area of ​​the blackboard area in each frame of the teacher's panoramic picture is determined, and the position area of ​​the target student is determined from the overlapping area, thereby improving the accuracy of determining the position area of ​​the target student.

[0084] In an alternative embodiment, see Figure 6 , step S30, including steps S301 to S303, are as follows:

[0085] S301: The target recording and broadcasting host obtains the screen resolution of each target student screen.

[0086] In the embodiment of the present application, the screen resolutions of the target student screens may be the same or different. For example, for classroom A, the screen resolution of the target student screen is 300*400, and for classroom B, the screen resolution of the target student screen is 320*480.

[0087] S302: The target recording and broadcasting host adjusts the screen resolutions of each target student screen to the same resolution to obtain each first target student screen.

[0088] In an embodiment of the present application, the target recording and broadcasting host can scale the screens of each target student so that the screen resolutions of each scaled target student screen are consistent.

[0089] S303: The target recording and broadcasting host splices the images of the first target students to obtain a composite image.

[0090] In an embodiment of the present application, the target recording and broadcasting host can splice each first target student screen according to a preset screen layout to obtain a composite screen. The preset screen layout includes the position information of each first target student screen. For example, the display area can be divided into left and right areas of equal size, wherein the left area can display a first target student screen, and the right area can display another first target student screen.

[0091] By uniformly adjusting the image resolutions of the target student images to the same resolution and then stitching them together, the quality of the composite image can be improved.

[0092] The present application also provides a device embodiment, which can be used to execute the contents of the screen directing method in the embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the contents of the screen directing method in the embodiment of the present application.

[0093] See also Figure 7 , a picture directing device 4 disclosed in an embodiment of the present application includes:

[0094] A trigger instruction generating module 41 is used for the recording and broadcasting host in each classroom to generate a trigger instruction in response to the trigger operation of the blackboard competition answering operation, and send the trigger instruction to the teacher camera in each classroom;

[0095] The panoramic picture acquisition module 42 is used for the teacher camera in each classroom to collect the teacher's panoramic picture in real time according to the trigger instruction; identify the position area of ​​each target student from the teacher's panoramic picture, determine each target student's picture from the teacher's panoramic picture according to the position area of ​​each target student, and send each target student's picture to the target recording and broadcasting host; wherein the target students are students answering questions in the competition on the blackboard in each classroom, and the target recording and broadcasting host is the recording and broadcasting host in any classroom in each classroom;

[0096] The picture synthesis module 43 is used for the target recording and broadcasting host to receive the pictures of each target student; synthesize the pictures of each target student to obtain a synthesized picture, and send the synthesized picture to the director platform so that the director platform displays the synthesized picture.

[0097] Using the embodiment of the present application, the recording and broadcasting host in each classroom responds to the trigger operation of the blackboard competition answering, generates a trigger instruction, and sends the trigger instruction to the teacher camera in each classroom; the teacher camera in each classroom collects the teacher's panoramic picture in real time according to the trigger instruction; identifies the position area of ​​each target student from the teacher's panoramic picture, and determines each target student picture from the teacher's panoramic picture according to the position area of ​​each target student, and sends each target student picture to the target recording and broadcasting host; wherein the target students are students answering questions in the competition on the blackboard in each classroom, and the target recording and broadcasting host is the recording and broadcasting host in any classroom in each classroom; the target recording and broadcasting host receives each target student picture; synthesizes each target student picture to obtain a synthesized picture, and sends the synthesized picture to the director platform so that the director platform displays the synthesized picture. The embodiment of the present application realizes the interaction between students in multiple classrooms by synthesizing the target student pictures of students in each classroom answering questions in the competition on the blackboard, and displays the synthesized picture on the director platform, thereby improving the teaching interaction effect.

[0098] The embodiment of the picture directing device of the embodiment of the present application only uses the division of the above-mentioned functional modules as an example when executing the picture directing method. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0099] The present application also provides a device embodiment, which can be used to execute the contents of the screen directing method in the embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the contents of the screen directing method in the embodiment of the present application.

[0100] See also Figure 8 The present application also provides an electronic device 300, which can be a computer, a screen director device, etc. In the embodiment of the present application, the electronic device 300 is a screen director device. The screen director device may include: at least one processor 310, at least one memory 320, at least one display 330, at least one network interface 340, a user interface 350, and at least one communication bus 360.

[0101] Among them, the display 330 includes an electronic ink screen; the electronic ink screen can receive touch operations or writing operations input by the user through fingers or input devices.

[0102] The user interface 350 is mainly used to provide an input interface for the user and obtain data input by the user. Optionally, the user interface 350 may also include a standard wired interface or a wireless interface.

[0103] The network interface 340 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0104] The communication bus 360 is used to realize the connection and communication between these components.

[0105] Among them, the processor 310 may include one or more processing cores. The processor 310 uses various interfaces and lines to connect various parts of the entire electronic device 300, and executes various functions and processes data of the electronic device 300 by running or executing instructions, programs, code sets or instruction sets stored in the memory 320, and calling data stored in the memory 320. Optionally, the processor 310 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 310 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed by the display layer; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 310, and it can be implemented separately through a chip.

[0106] Among them, the memory 320 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 320 includes a non-transitory computer-readable storage medium. The memory 320 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 320 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 320 may also be optionally at least one storage device located away from the aforementioned processor 310. As Figure 8As shown, the memory 320 as a computer storage medium may include an operating system, a network communication module, and a user.

[0107] The processor 310 can be used to call the screen directing application stored in the memory 320, and specifically execute the steps of the screen directing method of the embodiment. The specific execution process can be found in the specific description shown in the embodiment, and will not be repeated here.

[0108] The present application also provides a computer-readable storage medium on which a computer program is stored, and the instructions are suitable for being loaded by a processor and executing the method steps of the above-mentioned embodiment. The specific execution process can refer to the specific description shown in the embodiment, and will not be repeated here. The device where the storage medium is located can be a personal computer, a laptop computer, a smart phone, a tablet computer, a picture director device and other electronic devices.

[0109] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0110] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the function selected in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 function selected in a box or multiple boxes.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 steps for the function selected in a box or multiple boxes.

[0113] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0114] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0115] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0116] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0117] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for directing a picture, characterized in that: The steps include: The recording and broadcasting host in each classroom responds to the trigger operation of the blackboard competition answering question, generates a trigger instruction, and sends the trigger instruction to the teacher camera in each classroom; The teacher camera in each classroom collects the teacher's panoramic picture in real time according to the trigger instruction; By using a target detection model or a blackboard detection model, the location area of ​​each target student is identified from the teacher's panoramic picture, and according to the location area of ​​each target student, each target student screen is determined from the teacher's panoramic picture, and each target student screen is sent to a target recording and broadcasting host; wherein the target students are students answering questions in a competition on the blackboard in each classroom, and the target recording and broadcasting host is a recording and broadcasting host in any one of the classrooms; The target recording and broadcasting host receives the images of each target student; synthesizes the images of each target student to obtain a synthesized image, and sends the synthesized image to the director platform so that the director platform displays the synthesized image.

2. The image directing method according to claim 1, characterized in that: The step of identifying the location area of ​​each target student from the teacher's panoramic picture includes: Extract the brightness channel value of the teacher's panoramic picture to obtain the brightness channel picture; The area where the pixels whose brightness values ​​in the brightness channel image are greater than a preset brightness threshold are located is used as a blackboard area; According to the blackboard area, the location area of ​​each target student is determined.

3. The image directing method according to claim 2, characterized in that: The step of determining the location area of ​​each target student according to the blackboard area includes: Performing human shape detection on the blackboard area to obtain a human shape detection result; wherein the human shape detection result includes the position information of the target student; According to the target student position information, the position area of ​​each target student is determined from the blackboard area.

4. The image directing method according to claim 1, characterized in that: The step of identifying the location area of ​​each target student from the teacher's panoramic picture includes: Extract the brightness channel values ​​of several frames of continuous teacher panoramic pictures to obtain several brightness channel pictures; The areas where the pixels having brightness values ​​greater than a preset brightness threshold in each brightness channel image are located are all taken as blackboard areas to obtain a plurality of blackboard areas; Determine overlapping areas of a plurality of the blackboard areas; The location area of ​​each target student is determined based on the overlapping area.

5. The image directing method according to any one of claims 1 to 4, characterized in that: The target recording and broadcasting host receives the images of each target student; The step of synthesizing the target student images to obtain a synthesized image includes: The target recording and broadcasting host obtains the screen resolution of each target student screen; The target recording and broadcasting host adjusts the image resolutions of the target student images to the same resolution to obtain the first target student images; The target recording and broadcasting host splices the first target student images to obtain a composite image.

6. The image directing method according to any one of claims 1 to 4, characterized in that: The step of generating a trigger instruction by the recording and broadcasting host in each classroom in response to the trigger operation of answering questions in the blackboard competition includes: The recording and broadcasting host in each classroom obtains the courseware images sent in real time by the classroom computer multimedia; The recording and broadcasting host in each classroom identifies the courseware content of the courseware screen, and generates a trigger instruction if the courseware content includes the courseware content of the blackboard competition question answering.

7. A picture directing device, characterized in that: include: A trigger instruction generation module is used for the recording and broadcasting host in each classroom to respond to the trigger operation of the blackboard competition answering question, generate a trigger instruction, and send the trigger instruction to the teacher camera in each classroom; A panoramic picture acquisition module, which is used for the teacher camera in each classroom to collect the teacher's panoramic picture in real time according to the trigger instruction; By using a target detection model or a blackboard detection model, the location area of ​​each target student is identified from the teacher's panoramic picture, and according to the location area of ​​each target student, each target student screen is determined from the teacher's panoramic picture, and each target student screen is sent to a target recording and broadcasting host; wherein the target students are students answering questions in a competition on the blackboard in each classroom, and the target recording and broadcasting host is a recording and broadcasting host in any one of the classrooms; The screen synthesis module is used for the target recording and broadcasting host to receive the screens of each target student; synthesize the screens of each target student to obtain a synthesized screen, and send the synthesized screen to the director platform so that the director platform displays the synthesized screen.

8. An electronic device comprising a display, a processor and a memory; characterized in that: The memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the picture directing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image directing method as described in any one of claims 1 to 6 is implemented.

Images