A method, device and system for generating classroom videos

Abstract

The present invention relates to the field of video generation, and in particular to a method, device and system for generating classroom videos. The method for generating classroom videos comprises: obtaining a real-time display image of a first display board and real-time audio collected by a sound collection device; respectively obtaining real-time trajectory images of the first display board and the second display board; superimposing the real-time trajectory images of the first display board and the second display board on the real-time display image, and combining the real-time audio to obtain a first video displayed on a mobile terminal; obtaining a real-time trajectory image of a mobile terminal; superimposing the real-time trajectory image of the mobile terminal on the first video to obtain a second video; deleting the second video according to the changes in the picture and audio of the second video to obtain a new second video; and editing all real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video. The present invention solves the problem of poor interactivity of offline videos.

Description

Technical Field

[0001] The present invention relates to the field of video generation, and in particular to a method, device and system for generating classroom videos. Background Art

[0002] In the era of "Internet +", the combination of education and the Internet is an inevitable trend. The emergence of online classes can enable students in remote areas to receive instruction from famous teachers, making online teaching as effective as on-site teaching, and better solving the time and space limitations of education today. The emergence of online classes has realized a true combination of online and offline.

[0003] At present, the combination of online videos and offline videos is to use video recording equipment to record online teaching videos and then produce offline videos.

[0004] The offline video produced in this way only has the output of the teacher, but no interactive content of the students, and the courseware content in the screen may be blurred due to the alteration of the output of the teacher. Therefore, the offline video will have no interactive content of the students or the output of the teacher cannot be interactive, which leads to the problem of poor interactivity of the offline video. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device and system for generating classroom videos to address the above-mentioned problems.

[0006] The embodiment of the present invention is implemented as follows: a method for generating a classroom video, the method for generating a classroom video comprising:

[0007] S101, obtaining a real-time display image of a first display board and real-time audio collected by a sound collection device;

[0008] S102, respectively obtaining real-time trajectory images of the first display board and the second display board;

[0009] S103, superimposing the real-time trajectory images of the first display board and the second display board onto the real-time display image, and combining with the real-time audio to obtain a first video displayed on the mobile terminal;

[0010] S104, obtaining a real-time trajectory image of the mobile terminal;

[0011] S105, superimposing the real-time trajectory image of the mobile terminal onto the first video to obtain a second video;

[0012] S106, deleting the second video according to changes in the picture and audio of the second video to obtain a new second video;

[0013] S107, editing all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video.

[0014] In one embodiment, the present invention provides a device for generating a classroom video, the device for generating a classroom video comprising:

[0015] An image and sound acquisition module, used to acquire the real-time display image of the first display board and the real-time audio collected by the sound acquisition device;

[0016] A first trajectory module, used to obtain real-time trajectory images of the first display board and the second display board respectively;

[0017] A first superposition module, used to superimpose all real-time trajectory images onto the real-time display image, and combine the real-time audio to obtain a first video displayed on the mobile terminal;

[0018] The second trajectory module is used to obtain a real-time trajectory image of the mobile terminal;

[0019] A second superposition module is used to superimpose the real-time trajectory image of the mobile terminal on the first video to obtain a second video;

[0020] A video deletion module, used for deleting the second video according to changes in the picture and audio of the second video to obtain a new second video;

[0021] The video editing module is used to edit the new second video according to the display mode of the mobile terminal to obtain a third video.

[0022] In one embodiment, the present invention provides a system for generating classroom videos, the system comprising: a first display board, a second display board, a mobile terminal, a sound collection device, and a computer device;

[0023] The first display board is connected to the computer device and is used to display the content of the electronic courseware and generate a real-time display image, obtain the writing content of the lecturer on the first display board and generate a real-time trajectory image of the first display board;

[0024] The second display board is connected to the computer device and is used to obtain the writing content of the lecturer on the second display board and generate a real-time trajectory image of the second display board;

[0025] The mobile terminal is connected to the computer device, and is used to obtain the written content of the lecturer on the mobile terminal and generate a real-time trajectory image of the mobile terminal;

[0026] The sound collection device is connected to the computer device and is used for collecting real-time audio;

[0027] The computer device is used to execute the steps of the above-mentioned classroom video generation method.

[0028] A method for generating a classroom video provided by an embodiment of the present invention obtains a real-time display image of a first display board and real-time audio collected by a sound collection device; obtains real-time trajectory images of the first display board and the second display board respectively; superimposes the real-time trajectory images of the first display board and the second display board on the real-time display image, and obtains a first video displayed on a mobile terminal in combination with real-time audio; obtains a real-time trajectory image of a mobile terminal; superimposes the real-time trajectory image of the mobile terminal on the first video to obtain a second video; deletes the second video according to the picture and audio changes of the second video to obtain a new second video; and edits all real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video. In this way, the interactive content of the students can be obtained through the mobile terminal and generated and merged into the first video to obtain the second video; when making the third video, all real-time trajectory images in the new second video are edited according to the display mode of the mobile terminal, so that the output content of the teacher on the first display board or the second display board can be edited through the interaction of the mobile terminal, thereby eliminating the situation where the courseware content is blurred due to the alteration of the output content of the teacher, and solving the problem of poor interactivity of offline videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flowchart of a method for generating a classroom video provided in one embodiment;

[0030] Figure 2 It is a structural block diagram of a device for generating a classroom video in one embodiment;

[0031] Figure 3 A structural block diagram of a system for generating classroom videos in one embodiment;

[0032] Figure 4 is a schematic diagram of a second display board;

[0033] Figure 5 FIG. 4 is a block diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0035] It is understood that the terms "first", "second", etc. used in the present invention may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script without departing from the scope of the present invention.

[0036] like Figure 1 As shown, in one embodiment, a method for generating a classroom video is proposed, which may specifically include the following steps:

[0037] S101, obtaining a real-time display image of a first display board and real-time audio collected by a sound collection device;

[0038] S102, respectively obtaining real-time trajectory images of the first display board and the second display board;

[0039] S103, superimposing the real-time trajectory images of the first display board and the second display board onto the real-time display image, and combining with the real-time audio to obtain a first video displayed on the mobile terminal;

[0040] S104, obtaining a real-time trajectory image of the mobile terminal;

[0041] S105, superimposing the real-time trajectory image of the mobile terminal onto the first video to obtain a second video;

[0042] S106, deleting the second video according to changes in the picture and audio of the second video to obtain a new second video;

[0043] S107, editing all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video.

[0044] In this embodiment, the first display board can be an all-in-one teaching machine, which integrates multiple technologies such as infrared touch technology, intelligent office teaching software, multimedia network communication technology, high-definition flat-panel display technology, etc. It integrates multiple devices such as projectors, electronic whiteboards, computers, televisions, touch functions, etc. into one multifunctional interactive teaching device, which upgrades the traditional display terminal into a full-featured human-computer interaction device.

[0045] In this embodiment, the second display board is composed of plastic corners, nano green board, aluminum frame, infrared filter strip, right angle iron, and printed circuit board (PCBA, Printed Circuit Board + Assembly).

[0046] In this embodiment, the mobile terminal may be a smart terminal such as a mobile phone, a tablet, or a computer.

[0047] In this embodiment, for S101-S105, what is obtained are real-time images or sounds, and the first video and the second video are also real-time videos. Therefore, S101-S105 is a repeated and continuous process. The first video is a real-time video synchronously displayed on the mobile terminal.

[0048] In this embodiment, the real-time trajectory images of the first display board and the second display board are superimposed on the real-time display image. The real-time trajectory images of the first display board and the second display board do not cover the real-time display image. Instead, a track or layer is set on the real-time trajectory image, and the real-time trajectory image is placed in the set track or layer for mixed display.

[0049] In this embodiment, the image area of ​​the real-time trajectory image of the first display board is the display area of ​​the first display board, and the real-time trajectory image of the second display board is the writing area of ​​the second display board. Generally speaking, the writing area of ​​the second display board is not larger than the display area of ​​the first display board, and preferably, the writing area of ​​the second display board is equal to the display area of ​​the first display board.

[0050] In this embodiment, the image area of ​​the real-time trajectory image of the first display board is the display area of ​​the entire first display board, and the area without the trajectory in the real-time trajectory image is represented by transparent pixels. In this way, it is not necessary to determine the coordinates of the trajectory during superposition, and it is only necessary to determine that the coordinates of the four vertices of the real-time trajectory image and the coordinates of the four vertices of the real-time display image are consistent.

[0051] In this embodiment, for the second display board and the real-time trajectory image of the mobile terminal, the area without the trajectory in the real-time trajectory image is represented by transparent pixels. The writing area of ​​the second display board is not necessarily the same size as the display area of ​​the first display board. A fixed area is determined in the display area of ​​the first display board as the synchronization area of ​​the writing area of ​​the second display board. Preferably, the center point of the fixed area is the center point of the display area of ​​the first display board. The image area of ​​the real-time trajectory image of the mobile terminal is the display area of ​​the mobile terminal. The display area of ​​the mobile terminal is generally much smaller than the display area of ​​the first display board. When displaying the same video, the video of the first display board is generally reduced. During the reduction process, the aspect ratio of the video is kept unchanged. In S105, the real-time trajectory image of the mobile terminal is superimposed on the first video. The first video can be firstly reduced in proportion, and then the real-time trajectory image of the mobile terminal is superimposed on the first video. The superimposition method can be the same as the superimposition method of the real-time trajectory image of the first display board and the second display board, except that the layer where the real-time trajectory image of the mobile terminal is located is at the top layer.

[0052] In this embodiment, the first video is a real-time video. As for the second video, it is essentially a collection of all first videos superimposed with the real-time trajectory image of the mobile terminal. It can be understood that after the first video stops, for the first video recording video superimposed with the real-time trajectory image of the mobile terminal, before executing S106, it can be considered that the first video has stopped generating, and the duration of the second video is determined; of course, the second video is the first video partially superimposed with the real-time trajectory image of the mobile terminal. For example, a preset duration is set, and the collection of first videos superimposed with the real-time trajectory image of the mobile terminal within each preset duration is the second video. The preset duration can be 1 hour, so there are multiple second videos, that is, the second video is generated and processed at the same time, which can reduce the waiting time for generating the third video.

[0053] In this embodiment, the first video is essentially a synchronous real-time video of the students in class, the second video is essentially a video recorded from the synchronous real-time video, and the third video is a review video of the students after class.

[0054] A method for generating a classroom video provided by an embodiment of the present invention obtains a real-time display image of a first display board and real-time audio collected by a sound collection device; obtains real-time trajectory images of the first display board and the second display board respectively; superimposes the real-time trajectory images of the first display board and the second display board on the real-time display image, and obtains a first video displayed on a mobile terminal in combination with real-time audio; obtains a real-time trajectory image of a mobile terminal; superimposes the real-time trajectory image of the mobile terminal on the first video to obtain a second video; deletes the second video according to the picture and audio changes of the second video to obtain a new second video; and edits all real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video. In this way, the interactive content of the students can be obtained through the mobile terminal and generated and merged into the first video to obtain the second video; when making the third video, all real-time trajectory images in the new second video are edited according to the display mode of the mobile terminal, so that the output content of the teacher on the first display board or the second display board can be edited through the interaction of the mobile terminal, thereby eliminating the situation where the courseware content is blurred due to the alteration of the output content of the teacher, and solving the problem of poor interactivity of offline videos.

[0055] In one embodiment, obtaining a real-time trajectory image of the first display board includes:

[0056] At first preset time intervals, determining whether there is a first touch point in a preset area of ​​the first display board, and if so, obtaining all first touch points in the preset area of ​​the first display board;

[0057] sorting the first contacts according to the generation time of the first contacts;

[0058] For each first touch point, determine whether the distance between the first touch point and the first touch point before the first touch point in the sequence is greater than a first preset value; if so, determine the first touch point and the first touch point before the first touch point in the sequence as being in the same track; if not, determine the first touch point as the starting point of the next track;

[0059] Determine the displayed image of the tracks formed by all the first contact points in the transparent image as a real-time track image of the first display board;

[0060] If the first contact point does not exist in the preset area of ​​the first display board, the transparent image is determined as the real-time trajectory image of the first display board.

[0061] In this embodiment, the first preset time should be as short as possible, and the first preset time can be set to 0.5s.

[0062] In this embodiment, the preset area of ​​the first display board refers to the writing area of ​​the first display board, which may be a display area or may be smaller than the display area. The display area may also include an option area. The touch point in the option area is generally to select a specific option, which is not a type of handwriting.

[0063] In this embodiment, the first preset value may be the spacing of 3-5 first contacts. If two adjacent first contacts are in the same track, then the two adjacent first contacts should have one side or one point in contact, that is, the spacing of the two adjacent first contacts is 0. The first contact is essentially an area. If two adjacent first contacts are in contact, then the two areas are connected. Multiple connected areas constitute a track, so it is only necessary to determine the starting point and end point of each track. For sorting, the first contact in the sorting is the starting point of the first track, and the last first contact in the sorting is the end point of the last track.

[0064] In this embodiment, all the first touch points in the preset area of ​​the first display board are obtained, including both the first touch points that appear in the latest first preset time and the first touch points that appear in the previous first preset time. Determining the first touch point is essentially to determine a coordinate point. Once the first touch point appears, it will not be eliminated unless it is cleared. The appearance of the first touch point is formed by the contact between the pen tip and the first display board, but the elimination of the first touch point has nothing to do with the pen tip leaving the first display board. If the first touch point is not eliminated, it will always exist. The first display board has a clearing option. Only when the clearing option is selected, the first touch point will be cleared. At this time, there will be a situation where the first touch point does not exist in the preset area of ​​the first display board, and the transparent image is determined as the real-time trajectory image of the first display board. There are many clearing options for the first display board, such as full screen clearing, area clearing or eraser clearing. When clearing, you only need to eliminate the first touch point in the selected area.

[0065] In this embodiment, there is no text recognition involved, only the movement trajectory of the handwriting is recognized, which can be understood as what you write is what you get.

[0066] In one embodiment, obtaining a real-time trajectory image of the second display board includes:

[0067] At every second preset time, obtaining the infrared light group that is blocked within the second preset time;

[0068] Determine whether the number of infrared groups in the blocked state is greater than a first preset number, and if not, determine a second contact point according to each infrared group in the blocked state;

[0069] If the number of infrared groups in the blocked state is greater than a first preset number, determining an elimination point according to each infrared group in the blocked state;

[0070] For each group of elimination points, determine whether there is a second contact point at the position of the elimination point, and if so, eliminate the second contact point corresponding to the position of the elimination point;

[0071] Determine whether there is a second touch point in the preset area of ​​the second display board, and if so, obtain all the second touch points of the second display board;

[0072] sorting the second contacts according to the generation time of the second contacts;

[0073] For each second touch point, determine whether the distance between the second touch point and the second touch point before the second touch point in the sequence is greater than a second preset value; if so, determine the second touch point and the second touch point before the second touch point in the sequence as being in the same track; if not, determine the second touch point as the starting point of the next track;

[0074] Determine the displayed image of the tracks formed by all the second contact points in the transparent image as a real-time track image of the second display board;

[0075] If there is no second touch point in the preset area of ​​the second display board, the transparent image is determined as the real-time trajectory image of the second display board.

[0076] In this embodiment, the preset area of ​​the second display board is the writing area of ​​the second display board.

[0077] In this embodiment, the second touch point is similar to the first touch point. Acquiring all the second touch points in the preset area of ​​the second display board includes both the second touch points that appear in the latest second preset time and the second touch points that appear in the previous second preset time. Determining the second touch point is essentially determining a coordinate point. The appearance of the second touch point is formed by the contact between the chalk tip and the second display board, thereby blocking the infrared group, but the elimination of the second touch point has nothing to do with the chalk tip leaving the second display board. If the second touch point is not eliminated, it will always exist.

[0078] In this embodiment, unlike the first display board, the second display board is composed of plastic corners, nano green board, aluminum frame, infrared filter strip, right angle iron, and printed circuit board (PCBA, Printed Circuit Board + Assembly). The clearing on the first display board is a function. For the second display board, it is similar to a blackboard. The lecturer uses chalk and other tools to write. When writing, the tip of the chalk blocks a group of infrared groups, causing the signal of the infrared group to be lost, thereby determining a second contact point, that is, determining a second contact point according to each group of infrared groups that appear in a blocked state. When clearing, use tools such as blackboard erasers to clear. At this time, the number of infrared groups that appear in a blocked state within the second preset time is much greater than the number of infrared groups blocked by the chalk tip when writing. The first preset number can be set to 10-20. At this time, a elimination point is determined according to each group of infrared groups that appear in a blocked state, not the second contact point. The function of the elimination point is to eliminate the second contact point. The elimination point will not be retained, and the elimination point will be cleared after each second preset time. After the elimination point is cleared, the position of the elimination point can be re-determined as the second contact point.

[0079] In this embodiment, the infrared ray group in the blocked state will be restored to the unblocked state, but the determined second contact point will be retained until the second contact point is eliminated.

[0080] In this embodiment, the second preset time may be the same as the first preset time.

[0081] In this embodiment, the second preset value can be set to a distance of 3-5 second contact points.

[0082] In this embodiment, the second touch point determines the trajectory in the same manner as the first touch point, that is, the starting point and the end point of each trajectory are determined.

[0083] In one embodiment, the step of acquiring the real-time trajectory image of the mobile terminal includes:

[0084] At every third preset time, determine whether there is a third touch point in the preset area of ​​the mobile terminal, and if so, obtain all third touch points in the preset area of ​​the third display board within the third preset time;

[0085] sorting the third contacts according to the generation time of the third contacts;

[0086] For each third contact point, determine whether the distance between the third contact point and the third contact point before the third contact point in the sequence is greater than a third preset value; if so, determine the third contact point and the third contact point before the third contact point in the sequence as being in the same track; if not, determine the third contact point as the starting point of the next track;

[0087] Determine the trajectory formed by all the third touch points in the display image of the transparent image as a real-time trajectory image of the mobile terminal;

[0088] If there is no third touch point in the preset area of ​​the third display board, the transparent image is determined as the real-time trajectory image of the third display board.

[0089] In this embodiment, the preset area of ​​the mobile terminal refers to the writing area of ​​the mobile terminal, which may be the display area or may be smaller than the display area. The display area may also include an option area.

[0090] In this embodiment, the third preset time may be the same as the first preset time.

[0091] In this embodiment, the third preset value may be a spacing of 3-5 third contact points. The third contact points determine the trajectory in the same manner as the first contact points, that is, the starting point and the end point of each trajectory are determined.

[0092] In this embodiment, the mobile terminal is an electronic device like the first display board, so the elimination method of the third touch point is the same as the elimination method of the first touch point. The mobile terminal also has a clear option.

[0093] In one embodiment, the real-time trajectory images of the first display board and the second display board are superimposed on the real-time display image, and combined with the real-time audio to obtain the first video displayed on the mobile terminal, including:

[0094] Setting the real-time display image of the first display board at the first level;

[0095] Setting the real-time trajectory image of the first display board at the second level;

[0096] The real-time track image of the second display board is set at the third level;

[0097] Set the real-time audio to level 4;

[0098] The first level, the second level, the third level, and the fourth level are arranged on the same timeline for superposition to obtain a first video displayed on a mobile terminal.

[0099] In this embodiment, the real-time audio has only sound but no image, and can be considered as a transparent image. Therefore, the real-time audio does not affect the images of other layers when superimposed, which can be understood as adding sound to the video.

[0100] In one embodiment, the deleting the second video according to changes in the picture and audio of the second video includes:

[0101] Obtain each frame of the second video and the audio corresponding to each frame;

[0102] Determine continuous picture frames with unchanged picture pixels according to the pixels of each picture frame;

[0103] For each continuous picture frame, determining whether the number of frames of the continuous picture frame is greater than a preset number of frames, and if so, recording the continuous picture frame as a first picture to be determined;

[0104] Determine whether the audios corresponding to the picture frames in the first picture to be determined are all silent audios, and if so, record the first picture to be determined as the second picture to be determined;

[0105] Obtain a second preset number of picture frames before the second picture to be determined, determine whether the audio corresponding to the second preset number of picture frames before the second picture to be determined recognizes a preset word, and if not, delete the picture frames of the second picture to be determined except the first picture frame.

[0106] In this embodiment, one frame includes the real-time display image and real-time trajectory image of the first display board, the real-time trajectory image of the second display board, and the real-time trajectory image of the mobile terminal. A continuous frame includes at least two frames. The unchanged picture pixel means that each pixel in the picture has not changed.

[0107] In this embodiment, the preset number of frames can be 900 frames. The playback speed of a video is usually measured by playing a certain number of frames per second, that is, the frame rate, and the frame rate is usually 30 frames per second (FPS, i.e., Frames PerSecond). This standard applies to most videos, including movies, TV programs, and most digital video content. 900 frames correspond to 30 seconds of pictures. If the number of frames of continuous picture frames is greater than the preset number of frames, such as 900 frames, it means that the picture of the second video has not changed within 30 seconds. If the picture of the second video of 30 seconds or even more seconds has not changed, and the audio corresponding to this continuous picture that has not changed is silent audio, then this picture may only retain the first frame to reduce the length of the second video. The second video is mainly used for students to review and watch, and unnecessary waiting time can be deleted, such as break time between classes.

[0108] In this embodiment, the second preset number can be set to 120-150 frames, corresponding to a 4-5s screen. The purpose of the preset words is to determine the purpose of the second screen to be determined. If the purpose is to keep the screen unchanged, the second screen to be determined cannot be deleted. For example, the preset words are "Give students X minutes to answer" and "Let's take a look at this question". Such words require the screen to remain unchanged.

[0109] In one embodiment, the step of editing all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain the third video includes:

[0110] Determining a display mode of the mobile terminal according to an option of the mobile terminal;

[0111] If the display mode is the first mode, all the real-time track images of the second display board in the new second video are hidden to obtain a third video, and all the real-time track images of the hidden second display board are separately composed into a fourth video;

[0112] Dividing a display area of ​​the mobile terminal into two parts, respectively displaying the third video and the fourth video;

[0113] If the display mode is the second mode, determining the new second video as the third video, and displaying the third video in the display area of ​​the mobile terminal;

[0114] If the display mode is the third mode, all real-time trajectory images in the new second video are hidden to obtain a third video, and the third video is displayed in the display area of ​​the mobile terminal.

[0115] In this embodiment, the first mode can distinguish the content of the first display board from the content of the second display board to obtain the third video and the fourth video. For example, the content of the first display board is a math problem, and the content of the second display board is the problem-solving process.

[0116] In this embodiment, for example, if the student's review video needs to display the teacher's blackboard content and the student's own notes while watching the first video, the second mode can be selected.

[0117] In this embodiment, for example, if the student's review video only requires the courseware and does not require the teacher's blackboard content and the student's own notes when watching the first video, the third mode can be selected.

[0118] In this embodiment, other modes are also included.

[0119] like Figure 2 As shown, in one embodiment, a device for generating a classroom video is provided, which may specifically include:

[0120] An image and sound acquisition module, used to acquire the real-time display image of the first display board and the real-time audio collected by the sound acquisition device;

[0121] A first trajectory module, used to obtain real-time trajectory images of the first display board and the second display board respectively;

[0122] A first superposition module, used to superimpose all real-time trajectory images onto the real-time display image, and combine the real-time audio to obtain a first video displayed on the mobile terminal;

[0123] The second trajectory module is used to obtain a real-time trajectory image of the mobile terminal;

[0124] A second superposition module is used to superimpose the real-time trajectory image of the mobile terminal on the first video to obtain a second video;

[0125] A video deletion module, used for deleting the second video according to changes in the picture and audio of the second video to obtain a new second video;

[0126] The video editing module is used to edit the new second video according to the display mode of the mobile terminal to obtain a third video.

[0127] In this embodiment, each module of the classroom video generation device is the modularization of the method part of the present invention. For the specific explanation of each module, please refer to the corresponding content of the method part of the present invention, and the embodiment of the present invention will not be repeated here.

[0128] like Figure 3As shown, in one embodiment, a system for generating classroom videos is provided, which may specifically include: a first display board, a second display board, a mobile terminal, a sound collection device, and a computer device;

[0129] The first display board is connected to the computer device and is used to display the content of the electronic courseware and generate a real-time display image, obtain the writing content of the lecturer on the first display board and generate a real-time trajectory image of the first display board;

[0130] The second display board is connected to the computer device and is used to obtain the writing content of the lecturer on the second display board and generate a real-time trajectory image of the second display board;

[0131] The mobile terminal is connected to the computer device, and is used to obtain the written content of the lecturer on the mobile terminal and generate a real-time trajectory image of the mobile terminal;

[0132] The sound collection device is connected to the computer device and is used for collecting real-time audio;

[0133] The computer device is used to execute the steps of the above-mentioned classroom video generation method.

[0134] In this embodiment, the classroom video generation system may further include a communication device, which is respectively connected to the first display board, the second display board, the mobile terminal, the sound collection device and the computer device, so that all data can be interacted through the communication device.

[0135] In this embodiment, the computer device can be either a hardware device or a cloud server.

[0136] In this embodiment, the first display board can be an all-in-one teaching machine, which integrates multiple technologies such as infrared touch technology, intelligent office teaching software, multimedia network communication technology, high-definition flat-panel display technology, etc. It integrates multiple devices such as projectors, electronic whiteboards, computers, televisions, touch functions, etc. into one multifunctional interactive teaching device, which upgrades the traditional display terminal into a full-featured human-computer interaction device.

[0137] In this embodiment, the second display board is composed of plastic corners, nano green board, aluminum frame, infrared filter strip, right angle iron, and printed circuit board (PCBA, Printed Circuit Board + Assembly).

[0138] In this embodiment, the number of the first display board and the second display board and their spatial positions during the implementation are not limited. For example, there may be two second display boards, which are respectively located on the left and right sides of the first display board.

[0139] The classroom video generation system provided by the embodiment of the present invention obtains the real-time display image of the first display board and the real-time audio collected by the sound collection device; obtains the real-time trajectory images of the first display board and the second display board respectively; superimposes the real-time trajectory images of the first display board and the second display board on the real-time display image, and obtains the first video displayed on the mobile terminal in combination with the real-time audio; obtains the real-time trajectory image of the mobile terminal; superimposes the real-time trajectory image of the mobile terminal on the first video to obtain the second video; deletes the second video according to the picture and audio changes of the second video to obtain a new second video; edits all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video. In this way, the interactive content of the students can be obtained through the mobile terminal and generated and merged into the first video to obtain the second video; when making the third video, all the real-time trajectory images in the new second video are edited according to the display mode of the mobile terminal, so that the output content of the teacher on the first display board or the second display board can be edited through the interaction of the mobile terminal, thereby eliminating the situation that the courseware content is blurred due to the alteration of the output content of the teacher, and solves the problem of poor interactivity of offline videos.

[0140] like Figure 4 As shown, in one embodiment, the second display board includes: a first infrared module and a second infrared module;

[0141] The first infrared module is arranged in a first direction of the second display board, and the second infrared module is arranged in a second direction of the second display board;

[0142] The first infrared module and the second infrared module are used to determine the location of the landing point when the lecturer writes content on the second display board, so as to generate a real-time trajectory image of the second display board according to the location of the landing point.

[0143] In this embodiment, the first direction may be a horizontal direction, and the second direction may be a vertical direction between the vertical and horizontal directions. The first infrared module and the second infrared module are used in conjunction, that is, one determines the coordinates in the horizontal direction, and the other determines the coordinates in the vertical direction, thereby obtaining a coordinate point, which is the location of the landing point. A two-dimensional coordinate system can be established with any vertex of the second display board as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis, and such a coordinate point is specifically reflected in the two-dimensional coordinate system.

[0144] In this embodiment, the first direction may also be a direction that forms an angle with the horizontal direction, and it only needs to keep the first direction and the second direction perpendicular.

[0145] In this embodiment, the first infrared module and the second infrared module include several groups.

[0146] Figure 5 FIG. 2 shows an internal structure diagram of a computer device in one embodiment. Figure 5 As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement a method for generating a classroom video provided in an embodiment of the present invention. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute a method for generating a classroom video provided in an embodiment of the present invention. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0147] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] In one embodiment, a classroom video generation device provided by an embodiment of the present invention can be implemented in the form of a computer program. The computer program can be used in Figure 5 The computer device shown in the figure is run. The memory of the computer device can store various program modules constituting the generation device of the classroom video, for example, Figure 2 The picture and sound acquisition module, the first track module, the first superposition module, the second track module, the second superposition module, the video deletion module and the video editing module are shown. The computer program composed of each program module enables the processor to execute the steps in the method for generating a classroom video of each embodiment of the present invention described in this specification.

[0149] For example, Figure 5 The computer device shown can be Figure 2 The picture and sound acquisition module in the classroom video generation device shown executes step S101; the computer device can execute step S102 through the first trajectory module; the computer device can execute step S103 through the first superposition module; the computer device can execute step S104 through the second trajectory module; the computer device can execute step S105 through the second superposition module; the computer device can execute step S106 through the video deletion module; the computer device can execute step S107 through the video editing module.

[0150] In one embodiment, a computer device is provided, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0151] S101, obtaining a real-time display image of a first display board and real-time audio collected by a sound collection device;

[0152] S102, respectively obtaining real-time trajectory images of the first display board and the second display board;

[0153] S103, superimposing the real-time trajectory images of the first display board and the second display board onto the real-time display image, and combining with the real-time audio to obtain a first video displayed on the mobile terminal;

[0154] S104, obtaining a real-time trajectory image of the mobile terminal;

[0155] S105, superimposing the real-time trajectory image of the mobile terminal onto the first video to obtain a second video;

[0156] S106, deleting the second video according to changes in the picture and audio of the second video to obtain a new second video;

[0157] S107, editing all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video.

[0158] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor performs the following steps:

[0159] S101, obtaining a real-time display image of a first display board and real-time audio collected by a sound collection device;

[0160] S102, respectively obtaining real-time trajectory images of the first display board and the second display board;

[0161] S103, superimposing the real-time trajectory images of the first display board and the second display board onto the real-time display image, and combining with the real-time audio to obtain a first video displayed on the mobile terminal;

[0162] S104, obtaining a real-time trajectory image of the mobile terminal;

[0163] S105, superimposing the real-time trajectory image of the mobile terminal onto the first video to obtain a second video;

[0164] S106, deleting the second video according to changes in the picture and audio of the second video to obtain a new second video;

[0165] S107, editing all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video.

[0166] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0167] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0168] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for generating a classroom video, characterized in that: The method for generating the classroom video includes: S101, obtaining a real-time display image of a first display board and real-time audio collected by a sound collection device; S102, respectively obtaining real-time trajectory images of the first display board and the second display board; S103, superimposing the real-time trajectory images of the first display board and the second display board onto the real-time display image, and combining with the real-time audio to obtain a first video displayed on the mobile terminal; S104, obtaining a real-time trajectory image of the mobile terminal; S105, superimposing the real-time trajectory image of the mobile terminal onto the first video to obtain a second video; S106, deleting the second video according to changes in the picture and audio of the second video to obtain a new second video; S107, editing all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain a third video; The step of editing all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain the third video includes: Determining a display mode of the mobile terminal according to an option of the mobile terminal; If the display mode is the first mode, all the real-time track images of the second display board in the new second video are hidden to obtain a third video, and all the real-time track images of the hidden second display board are separately composed into a fourth video; Dividing a display area of ​​the mobile terminal into two parts, respectively displaying the third video and the fourth video; If the display mode is the second mode, determining the new second video as the third video, and displaying the third video in the display area of ​​the mobile terminal; If the display mode is the third mode, all real-time trajectory images in the new second video are hidden to obtain a third video, and the third video is displayed in the display area of ​​the mobile terminal.

2. The method for generating classroom videos according to claim 1, characterized in that: Get the real-time trajectory image of the first display board, including: At first preset time intervals, determining whether there is a first touch point in a preset area of ​​the first display board, and if so, obtaining all first touch points in the preset area of ​​the first display board; sorting the first contacts according to the generation time of the first contacts; For each first touch point, determine whether the distance between the first touch point and the first touch point before the first touch point in the sequence is greater than a first preset value; if so, determine the first touch point and the first touch point before the first touch point in the sequence as being in the same track; if not, determine the first touch point as the starting point of the next track; Determine the displayed image of the tracks formed by all the first contact points in the transparent image as a real-time track image of the first display board; If the first contact point does not exist in the preset area of ​​the first display board, the transparent image is determined as the real-time trajectory image of the first display board.

3. The method for generating classroom videos according to claim 1, characterized in that: Get the real-time trajectory image of the second display board, including: At every second preset time, obtaining the infrared light group that is blocked within the second preset time; Determine whether the number of infrared groups in the blocked state is greater than a first preset number, and if not, determine a second contact point according to each infrared group in the blocked state; If the number of infrared groups in the blocked state is greater than a first preset number, determining an elimination point according to each infrared group in the blocked state; For each group of elimination points, determine whether there is a second contact point at the position of the elimination point, and if so, eliminate the second contact point corresponding to the position of the elimination point; Determine whether there is a second touch point in the preset area of ​​the second display board, and if so, obtain all the second touch points of the second display board; sorting the second contacts according to the generation time of the second contacts; For each second touch point, determine whether the distance between the second touch point and the second touch point before the second touch point in the sequence is greater than a second preset value; if so, determine the second touch point and the second touch point before the second touch point in the sequence as being in the same track; if not, determine the second touch point as the starting point of the next track; Determine the displayed image of the tracks formed by all the second contact points in the transparent image as a real-time track image of the second display board; If there is no second touch point in the preset area of ​​the second display board, the transparent image is determined as the real-time trajectory image of the second display board.

4. The method for generating classroom videos according to claim 1, characterized in that: The step of obtaining a real-time trajectory image of a mobile terminal includes: At every third preset time, determine whether there is a third touch point in the preset area of ​​the mobile terminal, and if so, obtain all third touch points in the preset area of ​​the third display board within the third preset time; sorting the third contacts according to the generation time of the third contacts; For each third contact point, determine whether the distance between the third contact point and the third contact point before the third contact point in the sequence is greater than a third preset value; if so, determine the third contact point and the third contact point before the third contact point in the sequence as being in the same track; if not, determine the third contact point as the starting point of the next track; Determine the trajectory formed by all the third touch points in the display image of the transparent image as a real-time trajectory image of the mobile terminal; If there is no third touch point in the preset area of ​​the third display board, the transparent image is determined as the real-time trajectory image of the third display board.

5. The method for generating classroom videos according to claim 1, characterized in that: The method of superimposing the real-time trajectory images of the first display board and the second display board on the real-time display image and combining the real-time audio to obtain a first video displayed on the mobile terminal includes: Setting the real-time display image of the first display board at the first level; Setting the real-time trajectory image of the first display board at the second level; The real-time track image of the second display board is set at the third level; Set the real-time audio to level 4; The first level, the second level, the third level, and the fourth level are arranged on the same timeline for superposition to obtain a first video displayed on a mobile terminal.

6. The method for generating classroom videos according to claim 1, characterized in that: The deleting the second video according to the changes in the picture and audio of the second video includes: Obtain each frame of the second video and the audio corresponding to each frame; Determine continuous picture frames with unchanged picture pixels according to the pixels of each picture frame; For each continuous picture frame, determining whether the number of frames of the continuous picture frame is greater than a preset number of frames, and if so, recording the continuous picture frame as a first picture to be determined; Determine whether the audios corresponding to the picture frames in the first picture to be determined are all silent audios, and if so, record the first picture to be determined as the second picture to be determined; Obtain a second preset number of picture frames before the second picture to be determined, determine whether the audio corresponding to the second preset number of picture frames before the second picture to be determined recognizes a preset word, and if not, delete the picture frames of the second picture to be determined except the first picture frame.

7. A device for generating classroom videos, characterized in that: The classroom video generation device comprises: An image and sound acquisition module, used to acquire the real-time display image of the first display board and the real-time audio collected by the sound acquisition device; A first trajectory module, used to obtain real-time trajectory images of the first display board and the second display board respectively; A first superposition module, used to superimpose all real-time trajectory images onto the real-time display image, and combine the real-time audio to obtain a first video displayed on the mobile terminal; The second trajectory module is used to obtain a real-time trajectory image of the mobile terminal; A second superposition module is used to superimpose the real-time trajectory image of the mobile terminal on the first video to obtain a second video; A video deletion module, used to delete the second video according to changes in the picture and audio of the second video to obtain a new second video; A video editing module, used for editing the new second video to obtain a third video according to the display mode of the mobile terminal; The step of editing all the real-time trajectory images in the new second video according to the display mode of the mobile terminal to obtain the third video includes: Determining a display mode of the mobile terminal according to an option of the mobile terminal; If the display mode is the first mode, all the real-time track images of the second display board in the new second video are hidden to obtain a third video, and all the real-time track images of the hidden second display board are separately composed into a fourth video; Dividing a display area of ​​the mobile terminal into two parts, respectively displaying the third video and the fourth video; If the display mode is the second mode, determining the new second video as the third video, and displaying the third video in the display area of ​​the mobile terminal; If the display mode is the third mode, all real-time trajectory images in the new second video are hidden to obtain a third video, and the third video is displayed in the display area of ​​the mobile terminal.

8. A classroom video generation system, characterized in that: The classroom video generation system includes: a first display board, a second display board, a mobile terminal, a sound collection device and a computer device; The first display board is connected to the computer device and is used to display the content of the electronic courseware and generate a real-time display image, obtain the writing content of the lecturer on the first display board and generate a real-time trajectory image of the first display board; The second display board is connected to the computer device and is used to obtain the writing content of the lecturer on the second display board and generate a real-time trajectory image of the second display board; The mobile terminal is connected to the computer device, and is used to obtain the written content of the lecturer on the mobile terminal and generate a real-time trajectory image of the mobile terminal; The sound collection device is connected to the computer device and is used for collecting real-time audio; The computer device is used to execute the steps of the method for generating a classroom video as described in any one of claims 1 to 6.

9. The classroom video generation system according to claim 8, characterized in that: The second display board includes: a first infrared module and a second infrared module; The first infrared module is arranged in a first direction of the second display board, and the second infrared module is arranged in a second direction of the second display board; The first infrared module and the second infrared module are used to determine the location of the landing point when the lecturer writes content on the second display board, so as to generate a real-time trajectory image of the second display board according to the location of the landing point.

Images