Video processing method, recording and broadcasting system, electronic equipment and readable storage medium

Abstract

The invention provides a video processing method, a recording and broadcasting system, electronic equipment and a computer readable storage medium, and relates to the technical field of video processing, the method comprises the following steps: obtaining a panoramic classroom video and video editing information, the video editing information comprising related description information of a target video expected to be generated through the panoramic classroom video by a user; generating a mirror moving parameter and a shooting parameter of the virtual camera according to the video editing information; determining a first interested picture which accords with the description of the video editing information from the panoramic classroom video according to the mirror operation parameters; performing image reconstruction processing on the first interested picture according to the mirror operation parameter and the shooting parameter to obtain a second interested picture; and generating a target video according to the second interested picture. According to the method and the device, the teaching video which has the mirror moving effect and meets the requirements of the user is quickly generated under the conditions that manpower is not consumed and a mirror moving technology is not needed.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and more specifically, to a video processing method, a recording and broadcasting system, an electronic device and a computer-readable storage medium in the field of video processing technology. Background Art

[0002] At present, when shooting classroom videos, in addition to using multiple fixed positions for multi-directional video shooting, multi-directional video shooting is also carried out by moving the camera. The second shooting method is not only labor-intensive, but also poses a great challenge to the camera movement technology of the shooting personnel. If the camera movement technology is not up to standard, it is easy to make the classroom video shot unusable, and the classroom video shooting work needs to be carried out again, resulting in a waste of resources. Therefore, how to obtain the video shot by moving the camera without consuming manpower and without the need for camera movement technology has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides a video processing method, a recording and broadcasting system, an electronic device and a computer-readable storage medium. The present application utilizes panoramic classroom video data and video editing information provided by users according to their own needs to automatically create teaching videos with professional camera movement effects, thereby realizing the rapid generation of teaching videos that meet user needs without consuming manpower or requiring camera movement technology.

[0004] In a first aspect, a video processing method is provided, which includes: obtaining a panoramic classroom video and video editing information; wherein the video editing information includes relevant description information of a target video that a user expects to generate through the panoramic classroom video; generating camera movement parameters and shooting parameters of a virtual camera according to the video editing information; wherein the camera movement parameters correspond to the shooting parameters, and according to the camera movement parameters, determining a first picture of interest that meets the description of the video editing information from the panoramic classroom video; performing image reconstruction processing on the first picture of interest according to the camera movement parameters and the shooting parameters to obtain a second picture of interest; and generating a target video according to the second picture of interest.

[0005] In the above technical scheme, the embodiment of the present application adopts the method of obtaining panoramic classroom video and video editing information, wherein the video editing information includes relevant description information of the target video that the user expects to generate through the panoramic classroom video, generating the mirror movement parameters and shooting parameters of the virtual camera according to the video editing information, determining the first interested picture that meets the description of the video editing information from the panoramic classroom video according to the mirror movement parameters, performing image reconstruction processing on the first interested picture according to the mirror movement parameters and the shooting parameters, obtaining the second interested picture, and generating the target video according to the second interested picture. By combining the "shooting first and then moving the mirror" technology with the user's video needs, the panoramic classroom video data and the video editing information provided by the user according to their own needs are used to automatically create a teaching video with a professional mirror movement effect, thereby realizing the rapid generation of teaching videos that meet the user's needs without consuming manpower and without the need for mirror movement technology. On the one hand, because the present application omits the manual mirror movement link during video shooting, it can save the cost of video shooting. On the other hand, teaching videos with professional camera effects are generated based on the video editing information provided by users. Users only need to input the corresponding video editing information according to their own needs. The generation of corresponding videos is automatically completed by the system, and users do not need to perform additional operations, which improves the convenience of teaching video generation.

[0006] In combination with the first aspect, in some possible implementations, determining a first screen of interest that meets the description of the video editing information from the panoramic classroom video based on the camera movement parameters includes: obtaining multiple director codes corresponding to the panoramic classroom video; wherein each director code corresponds to description information about the video screen; based on the description information corresponding to each director code, obtaining a target director code corresponding to the video editing information from the multiple director codes; extracting a partial video screen of a classroom character corresponding to the target director code from the panoramic classroom video; and screening the partial video screen according to the camera movement trajectory in the camera movement parameters to obtain the first screen of interest.

[0007] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the local video screen is screened according to the camera movement trajectory in the camera movement parameters to obtain the first picture of interest, including: calculating the distance between each pixel point in the local video screen and the camera movement trajectory to obtain multiple distances; determining the distance less than a threshold among the multiple distances as the target distance; determining the pixel point corresponding to the target distance as the target pixel point intersecting with the camera movement trajectory; and obtaining the video screen including the target pixel point from the local video screen to obtain the first picture of interest.

[0008] Based on the above technical solution, on the one hand, by using the director code, the key local video screen in the video can be quickly and accurately located, thereby improving the accuracy of video screen positioning; on the other hand, by combining the director code with the camera movement parameters, the video screen can be screened, so that the video screen that meets the user's needs can be quickly and accurately obtained, which is conducive to improving the accuracy of generating teaching videos that meet user needs.

[0009] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the video processing method also includes: tracking and identifying the panoramic classroom video to obtain a recognition result; wherein the tracking and identification processing includes target detection, key point detection, single target tracking, multi-target tracking and target behavior recognition, and the recognition result includes the target detection frame, tracking identifier and behavior category of the classroom character; and generating multiple director codes corresponding to the panoramic classroom video according to the recognition result.

[0010] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, performing image reconstruction processing on the first picture of interest according to the camera movement parameters and the shooting parameters to obtain the second picture of interest includes: acquiring image parameters of the first picture of interest; adjusting the image parameters according to the camera movement parameters and the shooting parameters to obtain target image parameters; and replacing the image parameters with the target image parameters to obtain the second picture of interest.

[0011] Based on the above technical solution, the image parameters are replaced with the target image parameters, which actually applies the target image parameters to the first picture of interest, thereby generating a new visual expression, that is, modifying the first picture of interest to generate a modified first picture of interest, and the modified first picture of interest is the second picture of interest. The second picture of interest can reflect the visual effect brought about by the change in camera movement, that is, a video picture with a professional camera movement effect is obtained, which can more flexibly control and optimize the expression of the video picture content, and is conducive to improving the user's visual viewing experience.

[0012] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the target image parameters include target pixel parameters and target camera parameters; adjusting the image parameters according to the camera movement parameters and the shooting parameters to obtain the target image parameters includes: using the rotation and translation parameters in the camera movement parameters to adjust the pixel parameters in the image parameters to obtain the target pixel parameters; using the shooting parameters to adjust the camera parameters in the image parameters to obtain the target camera parameters; replacing the image parameters with the target image parameters to obtain the second picture of interest includes: replacing the pixel parameters with the target pixel parameters, and replacing the camera parameters with the target camera parameters to obtain the second picture of interest.

[0013] Based on the above technical solution, after calculating the target pixel parameters and the target camera parameters, the pixel parameters of the first picture of interest are replaced with the target pixel parameters, and the camera parameters of the first picture of interest are replaced with the target camera parameters, so that the first picture of interest is adjusted to the second picture of interest, thereby obtaining a video picture with a professional camera effect and meeting user needs. From the user's viewing visual perspective, the viewing experience of the second picture of interest is higher than that of the first picture of interest. It can be seen that it is conducive to improving the user's viewing experience of the video picture.

[0014] In a second aspect, a recording and broadcasting system is provided, the recording and broadcasting system comprising: a virtual reality camera and a broadcast director host, the virtual reality camera is connected to the broadcast director host;

[0015] Virtual reality camera, used to collect panoramic classroom videos and transmit them to the director host;

[0016] A director host is used to obtain video editing information, the video editing information includes relevant descriptive information of a target video that a user expects to generate through a panoramic classroom video, and to generate camera movement parameters and shooting parameters of a virtual camera according to the video editing information, wherein the camera movement parameters correspond to the shooting parameters, and according to the camera movement parameters, a first picture of interest that meets the description of the video editing information is determined from the panoramic classroom video, and image reconstruction processing is performed on the first picture of interest according to the camera movement parameters and the shooting parameters to obtain a second picture of interest, and a target video is generated according to the second picture of interest; wherein the video editing information includes relevant descriptive information of a target video that a user expects to generate through a panoramic classroom video.

[0017] Based on the above technical scheme, the director system provided by the embodiment of the present application has the following advantages: 1. By combining the "shoot first and then move the camera" technology with the user's video needs, the panoramic classroom video data and the video editing information provided by the user according to their own needs are used to automatically create teaching videos with professional camera movement effects, thereby realizing the rapid generation of teaching videos that meet user needs without consuming manpower and without the need for camera movement technology. 2. Because the present application omits the manual camera movement link during video shooting, the cost of video shooting can be saved. 3. The teaching video with professional camera movement effect is generated based on the video editing information provided by the user. For the user, the user only needs to input the corresponding video editing information according to his own needs. The generation of the corresponding video is automatically completed by the system, and the user does not need to perform additional operations, which improves the convenience of teaching video generation.

[0018] In combination with the second aspect, in some possible implementations, the virtual reality camera includes at least two fisheye lenses, the field of view angles of the at least two fisheye lenses are directed towards different viewing angles, and the field of view angles of two adjacent fisheye lenses of the at least two fisheye lenses overlap; the virtual reality camera is used to obtain classroom images taken by the at least two fisheye lenses at each shooting moment, to obtain classroom images of at least two viewing angles at each shooting moment, to splice the classroom images of at least two viewing angles at each shooting moment, to obtain a panoramic image of the classroom at each shooting moment, and to arrange the panoramic images of the classroom at each shooting moment in order of shooting time to obtain a panoramic classroom video.

[0019] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the director host is also used to send the panoramic classroom video to the live broadcast server, so that the live broadcast server distributes the panoramic classroom video to the client that supports playing virtual reality videos.

[0020] Based on the above technical solution, no matter where the user is, as long as there is a network, the user can remotely participate in classroom teaching through a client that supports playing virtual reality videos. It can also immerse the user in a panoramic teaching environment, as if they were in the classroom, that is, immersively experience live teaching.

[0021] In a third aspect, an electronic device is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the video processing method in the first aspect or any possible implementation of the first aspect.

[0022] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the video processing method in the first aspect or any possible implementation of the first aspect.

[0023] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the video processing method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic flow chart of a video processing method provided in an embodiment of the present application is shown;

[0025] Figure 2 A schematic diagram showing the setting of a virtual reality camera provided in an embodiment of the present application in a classroom is shown;

[0026] Figure 3 A schematic diagram of a partial video drawing showing classroom characters is shown;

[0027] Figure 4 A schematic diagram of the field of view of a virtual reality camera is shown;

[0028] Figure 5 A structural diagram of a recording and broadcasting system provided in an embodiment of the present application is shown;

[0029] Figure 6 A schematic diagram showing the transmission of panoramic classroom video by the director host;

[0030] Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0032] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0033] At present, when shooting classroom videos, in addition to using multiple fixed positions for multi-directional video shooting, the camera movement method is also used for multi-directional video shooting. The second shooting method is not only labor-intensive, but also poses a great challenge to the camera movement technology of the shooting personnel. If the camera movement technology is not up to standard, it is easy to make the classroom video shot unusable, and the classroom video shooting work needs to be carried out again, resulting in a waste of resources.

[0034] Based on the problems existing in the above-mentioned related technologies, the present application provides a video processing method, a recording and broadcasting system, an electronic device and a computer-readable storage medium. The video processing method provided by the present application belongs to an intelligent post-video editing process. By combining the "shoot first and then move the camera" technology with the user's video needs, the panoramic classroom video data and the video editing information provided by the user according to their own needs are used to automatically create a teaching video with a professional camera effect, so that the teaching video that meets the user's needs can be quickly generated without consuming manpower and without the need for camera technology. On the one hand, because the present application omits the manual camera movement link during video shooting, the video shooting cost can be saved. On the other hand, the teaching video with a professional camera effect is generated according to the video editing information provided by the user. For the user, the user only needs to input the corresponding video editing information according to his own needs, and the generation of the corresponding video is automatically completed by the system, without the user having to perform additional operations, which improves the convenience of generating teaching videos.

[0035] The following is an embodiment of a video processing method provided in an embodiment of the present application.

[0036] Figure 1 A schematic flow chart of a video processing method provided by an embodiment of the present application is shown. Figure 1 As shown, the video processing method provided in the embodiment of the present application is applied to an electronic device with computing power, such as a computer, and the video processing method includes the following schemes:

[0037] S110: Obtain panoramic classroom video and video editing information.

[0038] In an exemplary embodiment, if Figure 2 As shown, Figure 2 The diagram of the setting of the virtual reality camera provided by the embodiment of the present application in the classroom is shown, J represents the podium, K represents the desk, 101 represents the virtual reality camera, and the virtual reality camera is installed in the center of the classroom ceiling. The field of view of the virtual reality camera can cover the entire classroom without blind spots. Because the virtual reality camera is installed in the center of the classroom ceiling, it can capture the dynamics of the entire classroom with the best viewing angle, while considering reducing the unnecessary space intake on the ground and ceiling, so as to focus mainly on the classroom characters participating in classroom activities.

[0039] During the class, a virtual reality camera is used to capture a panoramic classroom video of the students in the class. The panoramic classroom video can be understood as a 360° panoramic video. Each frame of the video image in the panoramic classroom video is a classroom panoramic image, wherein the aspect ratio of the classroom panoramic image is a preset ratio, for example, the preset ratio is 2:1. Through the classroom panoramic image, various positions in the classroom can be seen, that is, the classroom panoramic image covers the entire classroom environment at 360°.

[0040] like Figure 3 As shown, Figure 3 The schematic diagram of the field of view of a virtual reality camera is shown. 101 represents a virtual reality camera, which includes two fisheye lenses, FOV1 represents the field of view of the first fisheye lens, FOV2 represents the field of view of the second fisheye lens, the two fisheye lenses are set at different viewing angles in the classroom, the horizontal field of view of the two fisheye lenses is greater than 180 degrees, and the field of view of the two fisheye lenses overlap, that is, the field of view of the two fisheye lenses after splicing can cover the entire classroom environment without blind spots, so that a panoramic image of the classroom environment can be obtained.

[0041] The scheme for collecting panoramic classroom videos of students in class by using a virtual reality camera including two fisheye lenses is as follows:

[0042] Obtain classroom images taken by two fisheye lenses at each shooting moment, and obtain classroom images of two viewing angles at each shooting moment;

[0043] The classroom images from two perspectives at each shooting moment are stitched together to obtain a panoramic image of the classroom at each shooting moment;

[0044] Arrange the classroom panoramic images at each shooting moment in order of shooting time to obtain a panoramic classroom video.

[0045] Each shooting moment is called Ti, and the classroom images taken by the two fisheye lenses at Ti are obtained. Since the field of view angle of each fisheye lens is in a different direction, that is, it corresponds to different viewing angles, the classroom images of two viewing angles under Ti can be obtained. Then the classroom images of the two viewing angles under Ti are spliced ​​to obtain a panoramic image of the classroom under Ti, thereby obtaining a multi-frame panoramic image of the classroom, that is, for the same acquisition moment, the classroom images of the two viewing angles at the same acquisition moment are spliced ​​to obtain a panoramic image of the classroom at the same acquisition moment, thereby obtaining a multi-frame panoramic image of the classroom.

[0046] After obtaining multiple frames of classroom panoramic images, the multiple frames of classroom panoramic images are sorted according to the shooting time sequence to generate a classroom panoramic image sequence, and a panoramic classroom video is generated through the classroom panoramic image sequence. For each frame of the classroom panoramic image in the panoramic classroom video, a 360° all-round view of the classroom can be displayed. Users can drag or rotate to view any direction in the video and experience an immersive viewing effect.

[0047] The video editing information includes the relevant description information of the target video that the user expects to generate through the panoramic classroom video. The video editing information can be input by the user through the keyboard or by voice. The above-mentioned relevant description information includes the user's requirements for the video theme, video content, camera movement, perspective change, visual effect, editing rhythm, emphasis, etc. For example, the video theme is required to be "teacher's blackboard writing", the video content is required to be "highlight the blackboard writing on the blackboard in the video, and ensure that all important formulas are clearly captured", the camera movement is required to be "add a smooth panning lens to the video, and the camera moves in the direction of the teacher's walking", the perspective change is required to be "watch the teacher's blackboard writing from the student's perspective", the visual effect is required to be "use a blurred background in some parts to highlight the teacher in front of the podium", the editing rhythm is required to be "insert some short animation transitions in the video to distinguish different course chapters", and the emphasis is required to be "for complex concepts, use arrows or highlights to mark key points", etc.

[0048] S120: Generate camera movement parameters and shooting parameters of the virtual camera according to the video editing information.

[0049] The camera movement parameters can be understood as instructing the virtual camera to simulate the movement of the real camera. The camera movement parameters include camera movement trajectory, camera movement speed, rotation and translation parameters, etc. The camera movement trajectory includes camera movement direction, camera movement angle, etc. The shooting parameters can be understood as the shooting parameters of the virtual camera during the movement of the real camera. The shooting parameters include parameters such as focal length change, white balance, exposure, and color change. The camera movement parameters correspond to the shooting parameters, that is, when the virtual camera moves based on the camera movement parameters, it shoots according to the shooting parameters.

[0050] After obtaining the video editing information, the video editing information is input into the machine learning model, and the camera movement parameters and shooting parameters of the virtual camera are predicted by the machine learning model. Among them, the machine learning model is trained by learning a large amount of high-quality sample data. The machine learning model can understand what kind of video editing information corresponds to what kind of camera movement parameters and shooting parameters, and what kind of camera movement parameters and shooting parameters can obtain a specific type of video content that meets the video editing information.

[0051] S130: Determine, according to the camera movement parameters, a first frame of interest from the panoramic classroom video that meets the description of the video editing information.

[0052] Because the camera movement parameters are used to indicate how the virtual camera moves, the video images that the virtual camera can capture can be obtained from the video images corresponding to different perspectives included in the panoramic classroom video according to the camera movement parameters. That is, the video images that the virtual camera can capture are called the first images of interest described by the video editing information.

[0053] S140: performing image reconstruction processing on the first picture of interest according to the camera movement parameters and the shooting parameters to obtain a second picture of interest.

[0054] Because the first interesting picture is obtained from the video pictures corresponding to different perspectives included in the panoramic classroom video, the image parameters of the first interesting picture are unchanged. After obtaining the first interesting picture, the image of the first interesting picture is reconstructed according to the camera movement parameters and the shooting parameters, that is, the image parameters of the first interesting picture are adjusted to obtain the second interesting picture captured by the virtual camera during the movement of the camera movement parameters and the shooting parameters, that is, the image parameters of the second interesting picture are different from the image parameters of the first interesting picture.

[0055] S150: Generate a target video according to the second interesting picture.

[0056] The movement of the virtual camera according to the camera movement parameters is a continuous process, that is, the virtual camera moves according to the camera movement parameters, and the second pictures of interest captured with the shooting parameters include several pictures, and then the several pictures of second pictures of interest are spliced ​​according to the chronological order of shooting, so as to automatically generate a target video that meets the video editing information, that is, a specific type of target video that meets user needs is obtained.

[0057] The embodiment of the present application adopts the method of obtaining panoramic classroom video and video editing information, wherein the video editing information includes the relevant description information of the target video that the user expects to generate through the panoramic classroom video, generating the mirror movement parameters and shooting parameters of the virtual camera according to the video editing information, determining the first interested picture that meets the description of the video editing information from the panoramic classroom video according to the mirror movement parameters, performing image reconstruction processing on the first interested picture according to the mirror movement parameters and the shooting parameters, obtaining the second interested picture, and generating the target video according to the second interested picture. The technical solution combines the "shooting first and then moving the mirror" technology with the user's video needs, and automatically creates a teaching video with a professional mirror movement effect by using the panoramic classroom video data and the video editing information provided by the user according to their own needs, so as to realize the rapid generation of teaching videos that meet the user's needs without consuming manpower and without the need for mirror movement technology. On the one hand, because the present application omits the manual mirror movement link during the video shooting, it can save the video shooting cost. On the other hand, the teaching video with a professional mirror movement effect is generated according to the video editing information provided by the user. For the user, the user only needs to input the corresponding video editing information according to his own needs, and the generation of the corresponding video is automatically completed by the system, without the user performing additional operations, thereby improving the convenience of generating teaching videos. On the other hand, the panoramic classroom video data used to generate a specific type of teaching video is obtained by shooting with a single virtual reality camera. By using a single virtual reality camera to replace multiple cameras for classroom video acquisition, only a single virtual reality camera needs to be installed, which is simple to install and helps to reduce installation costs. In addition, when switching between different perspectives based on the panoramic classroom video data, smooth switching of the images can be achieved, which is conducive to improving the continuity of viewing and avoiding the abruptness and visual jump caused by frequent switching of traditional multiple cameras.

[0058] The following Figure 1 The specific implementation methods of each step in the illustrated embodiment are described as follows:

[0059] In a possible implementation, the above S130, according to the camera movement parameter, determining the first interesting scene that meets the description of the video editing information from the panoramic classroom video includes the following scheme:

[0060] Obtain multiple director codes corresponding to the panoramic classroom video; wherein each director code corresponds to description information about the video screen;

[0061] Based on the description information corresponding to each director code, a target director code corresponding to the video editing information is obtained from the multiple director codes;

[0062] Extract the partial video picture of the classroom role corresponding to the target director code from the panoramic classroom video;

[0063] The local video picture is screened according to the camera movement track in the camera movement parameters to obtain a first picture of interest.

[0064] The director code is used to track the behavior of each classroom role in the panoramic classroom video. Each director code is represented by the director code Di. Then, the partial video picture of the classroom role corresponding to the director code Di is extracted from the panoramic classroom video according to the director code Di. The partial video picture of the classroom role corresponding to the director code Di can be understood as the focus picture of the classroom role corresponding to the director code Di, or it can be understood as the close-up picture of the classroom role corresponding to the director code Di. The entire behavior process of the classroom role corresponding to the director code Di is tracked through the partial video picture of the classroom role corresponding to the director code Di. Figure 3 As shown, Figure 3 A schematic diagram of a partial video picture showing classroom characters is shown, where F1 represents a frame of a panoramic classroom image in a panoramic classroom video, and F2 represents a partial video picture of a classroom character corresponding to director code Di. The partial video picture of a classroom character corresponding to director code Di is displayed in a picture-in-picture manner.

[0065] The generation process of multiple director codes corresponding to the panoramic classroom video is as follows:

[0066] Tracking and identifying the panoramic classroom video to obtain identification results; wherein the tracking and identification processing includes target detection, key point detection, single target tracking, multi-target tracking and target behavior recognition, and the recognition results include target detection frames, tracking identifiers and behavior categories of classroom characters;

[0067] Multiple director codes corresponding to the panoramic classroom video are generated according to the recognition results.

[0068] The multiple director codes include director codes about teachers and director codes about students. The director code about teachers is called a teacher director code, and the director code about students is called a student director code.

[0069] For the generation of student director codes: perform in-depth analysis of the panoramic classroom video, such as target detection, key point detection, multi-target tracking, and target behavior recognition, and output AI structured data about students. The AI ​​structured data about students includes the target detection frame, tracking mark, and behavior category of each student. The behavior category is, for example, a student raising his hand, a student standing up, etc. Then, the student director code is generated through the AI ​​structured data about students. The student director code includes the target detection frame, tracking mark, and behavior category of each student.

[0070] For the generation of teacher director codes: perform in-depth analysis of the panoramic classroom video, such as target detection, key point detection, single target tracking, and target behavior recognition, and output AI structured data about the teacher. The AI ​​structured data about the teacher includes the teacher's target detection frame, tracking mark, and behavior category. Behavior categories include writing on the blackboard, moving, and going up and down the podium. Then, the teacher director code is generated through the AI ​​structured data about the teacher, that is, the teacher director code includes the teacher's target detection frame, tracking mark, and behavior category.

[0071] For the application of director codes, the system uses the specific classroom roles that indicate the corresponding classroom behaviors indicated by the director codes, for example, "Student A raises his hand." The system will cut out a partial image of Student A from the panoramic classroom video and make Student A the center of the screen, that is, focus on Student A, thereby obtaining a partial video screen of Student A. When it is necessary to switch to the screen of "Teacher writing on the blackboard," the focus of the screen will smoothly move to the teacher and the blackboard, as if a virtual camera is tracking the behavior of students and teachers in real time. However, this is actually achieved through partial rendering of the panoramic image, which avoids the visual jump caused by traditional multi-camera switching and is conducive to improving viewing fluency.

[0072] Because the director code is generated after analyzing the panoramic classroom video, the director code identifies the classroom behavior and the location coordinates of the classroom characters in the classroom. The director code can be used not only for real-time live broadcast, but also for offline video editing. In real-time live broadcast, the director code can instantly trigger the director strategy and automatically switch to the partial screen of the relevant classroom behavior, improving the interactivity and viewing experience of the live broadcast. When replaying, the director code can generate the student perspective screen or the teacher perspective screen that the user chooses to focus on, which is conducive to improving the viewing experience. In offline video editing, the director code can quickly obtain the required partial video screen.

[0073] Each director code corresponds to description information about the video screen, and each director code corresponds to different description information, that is, director code Di corresponds to a description information about the video screen, and the description information about the video screen corresponding to director code Di is used to describe the local video screen of the classroom role corresponding to director code Di.

[0074] After obtaining the video editing information, the description information that is the same or similar to the video editing information can be queried through the description information about the video screen corresponding to each director code, which is called the target description information. The director code corresponding to the target description information is the target director code corresponding to the video editing information. Then, according to the target director code, the local video screen of the classroom character corresponding to the target director code is extracted from the panoramic classroom video. Among them, the target director code includes multiple, and there are also multiple local video screens obtained according to the target director code. Then, according to the camera movement trajectory in the camera movement parameter, the multiple local video screens obtained by the target director code are screened to obtain the first interested screen. Among them, the mirror trajectory represents the shooting path of the virtual camera, and the local video screens that can be captured by the virtual camera in the multiple local video screens when moving according to the camera movement trajectory are screened to obtain the first interested screen.

[0075] The above-mentioned method of filtering the local video picture according to the camera movement track in the camera movement parameter to obtain the first interesting picture includes the following schemes:

[0076] Calculate the distance between each pixel point in the local video screen and the camera trajectory to obtain multiple distances;

[0077] Determine a distance smaller than a threshold value among the multiple distances as a target distance;

[0078] The pixel point corresponding to the target distance is determined as the target pixel point intersecting with the mirror movement trajectory;

[0079] A video frame including a target pixel point is acquired from a local video frame to obtain a first frame of interest.

[0080] The process of obtaining the first interesting picture can be understood as a process of screening the video pictures that have intersections with the camera movement trajectory in the local video pictures, which is specifically as follows:

[0081] For each pixel point in the local video picture, the distance between each pixel point and the camera movement trajectory is calculated to obtain multiple distances, and then the size relationship between each distance and the threshold is compared, the distance less than the threshold is determined as the target distance, and the pixel point corresponding to the target distance is determined as the target pixel point, and then the video picture including the target pixel point in the local video picture is retained, and the video picture not including the target pixel point in the local video picture is filtered out, that is, the retained video picture is the first picture of interest, which is also the video picture in the local video picture that intersects with the camera movement trajectory.

[0082] Based on the above technical solution, on the one hand, by using the director code, the key local video screen in the video can be quickly and accurately located, thereby improving the accuracy of video screen positioning; on the other hand, by combining the director code with the camera movement parameters, the video screen can be screened, so that the video screen that meets the user's needs can be quickly and accurately obtained, which is conducive to improving the accuracy of generating teaching videos that meet user needs.

[0083] In a possible implementation, the above S140, performing image reconstruction processing on the first picture of interest according to the camera movement parameters and the shooting parameters to obtain the second picture of interest includes the following scheme:

[0084] Acquire image parameters of a first picture of interest;

[0085] Adjust the image parameters according to the camera movement parameters and the shooting parameters to obtain the target image parameters;

[0086] The target image parameters are used to replace the image parameters to obtain a second picture of interest.

[0087] The process of generating the second interesting picture includes: obtaining the image parameters of the first interesting picture, the image parameters refer to the parameters describing the various contents in the first interesting picture, after obtaining the image parameters, calculating the new image parameters according to the camera movement parameters, the shooting parameters and the image parameters, so as to adjust the image parameters, and the new image parameters are the target image parameters. The target image parameters are used to generate a new picture, and the image parameters are replaced with the target image parameters, which actually applies the target image parameters to the first interesting picture, so as to generate a new visual expression, that is, to modify the first interesting picture, and generate the modified first interesting picture, the modified first interesting picture is the second interesting picture, the second interesting picture can reflect the visual effect brought by the camera movement change, that is, to obtain a video picture with a professional camera movement effect, which can more flexibly control and optimize the expression form of the video picture content, and is conducive to improving the user's visual viewing experience.

[0088] The target image parameters include target pixel parameters and target camera parameters. The target pixel parameters include grayscale value (RGB value), pixel coordinates, etc. The target camera parameters include size, focal length change, white balance, exposure, color change, etc. The above-mentioned adjustment of image parameters according to the camera movement parameters and shooting parameters to obtain the target image parameters includes the following schemes:

[0089] The pixel point parameters in the image parameters are adjusted by using the rotation and translation parameters in the camera movement parameters to obtain the target pixel point parameters;

[0090] The shooting parameters are used to adjust the camera parameters in the image parameters to obtain the target camera parameters.

[0091] The rotation and translation parameters are used to instruct the virtual camera to rotate and shoot at a corresponding rotation angle and to move and shoot at a corresponding horizontal displacement, so as to obtain pictures shot at different camera angles. The adjustment of the image parameters of the first picture of interest includes adjusting the pixel parameters in the image parameters and adjusting the camera parameters in the image parameters. The adjustment of the pixel parameters can be understood as calculating the new pixel parameters through the rotation and translation parameters and the pixel parameters in the mirror movement parameters to obtain the target pixel parameters. The adjustment of the camera parameters can be understood as calculating the new camera parameters through the shooting parameters and the camera parameters to obtain the target camera parameters. In this way, different visual video pictures under different camera perspectives can be obtained, thereby meeting the visual needs of users.

[0092] The above-mentioned method of replacing the image parameters with the target image parameters to obtain the second interesting picture includes the following schemes:

[0093] The pixel parameters are replaced by the target pixel parameters, and the camera parameters are replaced by the target camera parameters to obtain a second picture of interest.

[0094] After calculating the target pixel parameters and the target camera parameters, the pixel parameters of the first picture of interest are replaced with the target pixel parameters, and the camera parameters of the first picture of interest are replaced with the target camera parameters, so that the first picture of interest is adjusted to the second picture of interest, thereby obtaining a video picture with a professional camera effect and meeting user needs. From the user's viewing visual perspective, the viewing experience of the second picture of interest is higher than that of the first picture of interest, which is conducive to improving the user's viewing experience of the video picture.

[0095] The following is an embodiment of a recording and broadcasting system provided in an embodiment of the present application.

[0096] Figure 5 A structural diagram of a recording and broadcasting system provided in an embodiment of the present application is shown. Figure 5 As shown, the recording and broadcasting system 100 includes: a virtual reality camera 101 and a director host 102, and the virtual reality camera 101 is connected to the director host 102.

[0097] The virtual reality camera 101 is used to collect the panoramic classroom video and transmit the panoramic classroom video to the director host 102; wherein each frame of the panoramic classroom video is a panoramic image of the classroom.

[0098] It is understandable that if Figure 2As shown, 101 represents a virtual reality camera, which is installed at the center of the classroom ceiling. The field of view of the virtual reality camera 101 can cover the entire classroom without blind spots. Because the virtual reality camera is installed at the center of the classroom ceiling, it can capture the entire classroom dynamics with the best viewing angle, while considering reducing unnecessary space intake on the ground and ceiling, so as to focus mainly on the classroom characters participating in classroom activities.

[0099] During the class, the virtual reality camera 101 collects the panoramic classroom video of the students in the class, and then transmits the panoramic classroom video to the director host 102 through the High-Definition Multimedia Interface (HDMI). The panoramic classroom video can be understood as a 360° panoramic video, and each frame of the video image in the panoramic classroom video is a classroom panoramic image, wherein the aspect ratio of the classroom panoramic image is a preset ratio, and each position in the classroom can be seen through the classroom panoramic image, that is, the classroom panoramic image covers the entire classroom environment at 360°.

[0100] The director host 102 is used to obtain video editing information, which includes relevant descriptive information of a target video that a user expects to generate through the panoramic classroom video, and generates camera movement parameters and shooting parameters of a virtual camera based on the video editing information. The camera movement parameters correspond to the shooting parameters, and a first picture of interest that meets the description of the video editing information is determined from the panoramic classroom video based on the camera movement parameters, and an image reconstruction process is performed on the first picture of interest based on the camera movement parameters and the shooting parameters to obtain a second picture of interest, and a target video is generated based on the second picture of interest; wherein, the video editing information includes relevant descriptive information of a target video that a user expects to generate through the panoramic classroom video.

[0101] It can be understood that: after receiving the panoramic classroom video transmitted by the virtual reality camera 101, the director host 102 analyzes the panoramic classroom video to obtain a director code for tracking the behavior of classroom roles, and the classroom roles include students and / or teachers. Among them, the director code obtained by the director host 102 for tracking the behavior of classroom roles according to the panoramic classroom video includes multiple director codes, and the multiple director codes include director codes about teachers and director codes about students. The director code about teachers is called a teacher director code, and the director code about students is called a student director code.

[0102] For the generation of student director codes: the director host 102 performs in-depth analysis of the panoramic classroom video, such as target detection, key point detection, multi-target tracking, and target behavior recognition, and outputs AI structured data about students. The AI ​​structured data about students includes the target detection frame, tracking mark, and behavior category of each student. The behavior category is, for example, a student raising his hand, a student standing up, etc. The director host 102 then generates a student director code based on the AI ​​structured data about the students. The student director code includes the target detection frame, tracking mark, and behavior category of each student.

[0103] For the generation of the teacher's director code: the director host 102 performs in-depth analysis of the panoramic classroom video, such as target detection, key point detection, single target tracking, and target behavior recognition, and outputs AI structured data about the teacher. The AI ​​structured data about the teacher includes the teacher's target detection frame, tracking mark, and behavior category. The behavior category is, for example, writing on the blackboard, moving, going up and down the podium, etc. The director host 102 then generates the teacher's director code through the AI ​​structured data about the teacher, that is, the teacher's director code includes the teacher's target detection frame, tracking mark, and behavior category.

[0104] After the teacher director code and the student director code are generated, each director code can indicate the specific classroom role where the corresponding classroom behavior occurs, such as indicating "student A raises his hand" or indicating "the teacher writes on the blackboard."

[0105] Regarding the application of director codes, the director host 102 uses the specific classroom roles that have corresponding classroom behaviors indicated by the director codes, for example, "Student A raises his hand. The system will cut out a partial image of Student A from the panoramic classroom video and make Student A the center of the screen, that is, focus on Student A, thereby obtaining a partial video screen of Student A. When it is necessary to switch to the screen of "Teacher writing on the blackboard", the focus of the screen will move smoothly to the teacher and the blackboard, as if a virtual camera is tracking the behavior of students and teachers in real time, but it is actually achieved through partial rendering of the panoramic image, which avoids the visual jump caused by traditional multi-camera switching and is conducive to improving the smoothness of viewing.

[0106] Because the director code is generated after analyzing the panoramic classroom video, the director code identifies the classroom behavior and the location coordinates of the classroom characters in the classroom. The director code can be used not only for real-time live broadcast, but also for video replay. In real-time live broadcast, the director code can instantly trigger the director strategy and automatically switch to the local screen of the relevant classroom behavior, improving the interactivity and viewing experience of the live broadcast. During replay, the director code can generate the student perspective screen or teacher perspective screen that the user chooses to focus on, which is conducive to improving the viewing experience.

[0107] After the director host 102 generates multiple director codes for tracking the behavior of classroom characters, each director code is represented by a director code Di. The director host 102 extracts the partial video picture of the classroom character corresponding to the director code Di from the panoramic classroom video according to the director code Di. The partial video picture of the classroom character corresponding to the director code Di can be understood as the focus picture of the classroom character corresponding to the director code Di, and can also be understood as the close-up picture of the classroom character corresponding to the director code Di. The entire behavior process of the classroom character corresponding to the director code Di is tracked through the partial video picture of the classroom character corresponding to the director code Di.

[0108] After the director host 102 extracts the local video picture corresponding to the director code Di from the panoramic classroom video, it uses a machine learning model to describe the local video picture corresponding to the director code Di to obtain the description information of the local video picture corresponding to the director code Di, and associates the description information of the local video picture corresponding to the director code Di with the director code Di to obtain the description information about the video picture corresponding to the director code Di, and then associates the corresponding camera movement parameters and shooting parameters for each type of description information about the video picture.

[0109] After obtaining the video editing information input by the user, the director host 102 inputs the video editing information into the machine learning model, and predicts the camera movement parameters and shooting parameters of the virtual camera through the machine learning model. Because the camera movement parameters are used to indicate how the virtual camera moves, the video images that the virtual camera can shoot can be obtained from the video images corresponding to different perspectives included in the panoramic classroom video according to the camera movement parameters, that is, the video images that the virtual camera can shoot are called the first interesting images that meet the description of the video editing information.

[0110] Because the first interesting picture is obtained from the video pictures corresponding to different perspectives included in the panoramic classroom video, the image parameters of the first interesting picture are unchanged. After obtaining the first interesting picture, the director host 102 performs image reconstruction processing on the first interesting picture according to the camera movement parameters and the shooting parameters, that is, adjusts the image parameters of the first interesting picture to obtain the second interesting picture captured by the virtual camera during the movement of the camera movement parameters and the shooting parameters, that is, the image parameters of the second interesting picture are different from the image parameters of the first interesting picture.

[0111] The movement of the virtual camera according to the camera movement parameters is a continuous process, that is, during the period when the virtual camera moves according to the camera movement parameters, and the second pictures of interest captured with the shooting parameters include several pictures, the director host 102 then splices the several pictures of second interest according to the chronological order of shooting, so as to automatically generate a target video that meets the video editing information, that is, obtain a specific type of target video that meets the user's needs.

[0112] The broadcast director system provided in the embodiment of the present application has the following advantages:

[0113] 1. By combining the "shoot first, then move the camera" technology with the user's video needs, the panoramic classroom video data and the video editing information provided by the user according to their own needs are used to automatically create teaching videos with professional camera effects, which realizes the rapid generation of teaching videos that meet user needs without consuming manpower or requiring camera movement technology.

[0114] 2. Because the present application eliminates the manual camera movement during video shooting, it can save the cost of video shooting.

[0115] 3. Teaching videos with professional camera effects are generated based on the video editing information provided by users. Users only need to input the corresponding video editing information according to their own needs. The generation of corresponding videos is automatically completed by the system, and users do not need to perform additional operations, which improves the convenience of teaching video generation.

[0116] 4. The panoramic classroom video data used to generate specific types of teaching videos is obtained by shooting with a single virtual reality camera. A single virtual reality camera is used to replace multiple cameras for classroom video acquisition. Only a single virtual reality camera needs to be installed, which is simple to install and helps reduce installation costs.

[0117] 5. When switching between images of different perspectives based on panoramic classroom video data, smooth switching of images can be achieved, which is conducive to improving viewing continuity and avoiding the abruptness and visual jump caused by frequent switching of traditional multiple cameras.

[0118] In a possible implementation, the virtual reality camera 101 includes at least two fisheye lenses, each of which has a field of view facing a different viewing angle, and the field of view of two adjacent fisheye lenses of the at least two fisheye lenses overlaps. Figure 4 As shown, the virtual reality camera 101 includes two fisheye lenses, FOV1 represents the field of view of the first fisheye lens, FOV2 represents the field of view of the second fisheye lens, the two fisheye lenses are set at different viewing angles in the classroom, the horizontal field of view of the two fisheye lenses is greater than 180 degrees, and the field of view of the two fisheye lenses overlap, that is, the field of view of the two fisheye lenses after splicing can cover the entire classroom environment without blind spots, so that a panoramic image of the classroom environment can be obtained.

[0119] The virtual reality camera 101 is used to obtain classroom images taken by at least two fisheye lenses at each shooting moment, obtain classroom images from at least two perspectives at each shooting moment, stitch the classroom images from at least two perspectives at each shooting moment to obtain a panoramic image of the classroom at each shooting moment, and arrange the panoramic images of the classroom at each shooting moment in order of shooting time to obtain a panoramic classroom video.

[0120] It can be understood that: each shooting moment is referred to as Ti, and the virtual reality camera 101 obtains classroom images shot by at least two fisheye lenses at Ti, because the field of view angle of each fisheye lens is in a different direction, that is, corresponding to different viewing angles, and at least two viewing angles of the classroom images under Ti can be obtained. The virtual reality camera 101 then splices the classroom images of at least two viewing angles under Ti to obtain a panoramic image of the classroom under Ti, thereby obtaining a multi-frame panoramic image of the classroom, that is, for the same acquisition moment, the classroom images of at least two viewing angles at the same acquisition moment are spliced ​​to obtain a panoramic image of the classroom at the same acquisition moment, thereby obtaining a multi-frame panoramic image of the classroom.

[0121] After the virtual reality camera 101 obtains multiple frames of classroom panoramic images, it sorts the multiple frames of classroom panoramic images in the order of shooting time to generate a classroom panoramic image sequence, and generates a panoramic classroom video through the classroom panoramic image sequence. Each frame of the classroom panoramic image in the panoramic classroom video can show a 360° all-round view of the classroom, and the user can drag or rotate to view any direction in the video to experience an immersive viewing effect.

[0122] In one possible implementation, Figure 6 The schematic diagram of the director host transmitting the panoramic classroom video is shown in FIG. Figure 6 As shown, the director host 102 is also used to send the panoramic classroom video to the live broadcast server 200, so that the live broadcast server 200 distributes the panoramic classroom video to the client 300 that supports playing virtual reality videos.

[0123] It can be understood that: after the director host 102 receives the panoramic classroom video transmitted by the virtual reality camera 101, it sends the panoramic classroom video to the live broadcast server 200 through the Real Time Messaging Protocol (RTMP), and distributes the panoramic classroom video to the client 300 that supports playing virtual reality videos through the live broadcast server 200. This means that no matter where the user is, as long as there is a network, the user can remotely participate in classroom teaching through the client 300 that supports playing virtual reality videos, and can also immerse the user in the panoramic teaching environment, as if in the classroom, that is, immersive experience teaching live broadcast. In addition, the user can also automatically adjust the video viewing angle according to the classroom activities to experience the teaching live broadcast experience from different perspectives.

[0124] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. Figure 7 As shown, the electronic device 700 includes: a memory 701 and a processor 702, wherein the memory 701 stores an executable program code 7011, and the processor 702 is used to call and execute the executable program code 7011 to perform a video processing method.

[0125] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be corresponded to, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0126] In the case of dividing each functional module according to each function, the electronic device may include: a virtual reality camera, a broadcast host, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0127] The electronic device provided in this embodiment is used to execute the above-mentioned video processing method, and thus can achieve the same effect as the above-mentioned implementation method.

[0128] In the case of an integrated unit, the electronic device may include a processing module and a storage module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device in executing related program codes and data.

[0129] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0130] This embodiment further provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement a video processing method in the above-mentioned embodiment.

[0131] This embodiment further provides a computer program product. When the computer program product runs on a computer, the computer is enabled to execute the above-mentioned related steps to implement a video processing method in the above-mentioned embodiment.

[0132] In addition, the electronic device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the electronic device may include a connected processor and a memory; wherein the memory is used to store instructions, and when the electronic device is running, the processor may call and execute instructions so that the chip executes a video processing method in the above-mentioned embodiments.

[0133] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding video processing method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding video processing method provided above, and will not be repeated here.

[0134] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0135] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0136] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video processing method, characterized in that: The video processing method comprises: Obtaining a panoramic classroom video and video editing information; wherein the video editing information includes relevant description information of a target video that a user expects to generate through the panoramic classroom video; Generate camera movement parameters and shooting parameters of a virtual camera according to the video editing information; wherein the camera movement parameters correspond to the shooting parameters; According to the camera movement parameters, determining a first interesting scene that meets the description of the video editing information from the panoramic classroom video; Performing image reconstruction processing on the first picture of interest according to the camera movement parameters and the shooting parameters to obtain a second picture of interest; The target video is generated according to the second interesting picture.

2. The video processing method according to claim 1, characterized in that: Determining, according to the camera movement parameters, from the panoramic classroom video a first interesting scene that meets the description of the video editing information includes: Obtain multiple director codes corresponding to the panoramic classroom video; wherein each director code corresponds to description information about the video screen; Based on the description information corresponding to each of the director codes, acquiring a target director code corresponding to the video editing information from the multiple director codes; Extracting a partial video picture of the classroom role corresponding to the target director code from the panoramic classroom video; The local video picture is screened according to the camera movement track in the camera movement parameters to obtain the first picture of interest.

3. The video processing method according to claim 2, characterized in that: The filtering of the local video picture according to the camera movement track in the camera movement parameter to obtain the first interesting picture includes: Calculating the distance between each pixel point in the local video screen and the camera movement trajectory to obtain multiple distances; Determine a distance among the multiple distances that is smaller than a threshold as a target distance; Determine the pixel point corresponding to the target distance as the target pixel point intersecting with the camera movement trajectory; A video frame including the target pixel point is acquired from the local video frame to obtain the first frame of interest.

4. The video processing method according to claim 2, characterized in that: The video processing method further includes: Performing tracking and recognition processing on the panoramic classroom video to obtain a recognition result; wherein the tracking and recognition processing includes target detection, key point detection, single target tracking, multi-target tracking and target behavior recognition, and the recognition result includes a target detection frame, tracking identifier and behavior category of the classroom character; A plurality of director codes corresponding to the panoramic classroom video are generated according to the recognition result.

5. The video processing method according to any one of claims 1 to 4, characterized in that: The performing image reconstruction processing on the first picture of interest according to the camera movement parameter and the shooting parameter to obtain the second picture of interest comprises: Acquire image parameters of the first picture of interest; Adjust the image parameters according to the camera movement parameters and the shooting parameters to obtain target image parameters; The target image parameters are used to replace the image parameters to obtain the second picture of interest.

6. The video processing method according to any one of claim 5, characterized in that: The target image parameters include target pixel parameters and target camera parameters; The adjusting the image parameters according to the camera movement parameters and the shooting parameters to obtain target image parameters comprises: The pixel point parameters in the image parameters are adjusted by using the rotation and translation parameters in the camera movement parameters to obtain the target pixel point parameters; The camera parameters in the image parameters are adjusted by using the shooting parameters to obtain the target camera parameters; The step of replacing the image parameters with the target image parameters to obtain the second interesting picture comprises: The pixel parameters are replaced by the target pixel parameters, and the camera parameters are replaced by the target camera parameters to obtain the second picture of interest.

7. A recording and broadcasting system, characterized in that: The recording and broadcasting system comprises: a virtual reality camera and a broadcast director host, wherein the virtual reality camera is connected to the broadcast director host; The virtual reality camera is used to collect the panoramic classroom video and transmit the panoramic classroom video to the director host; The director host is used to obtain video editing information, which includes relevant descriptive information of a target video that a user expects to generate through the panoramic classroom video, and generates camera movement parameters and shooting parameters of a virtual camera according to the video editing information, wherein the camera movement parameters correspond to the shooting parameters, and according to the camera movement parameters, a first picture of interest that meets the description of the video editing information is determined from the panoramic classroom video, and image reconstruction processing is performed on the first picture of interest according to the camera movement parameters and the shooting parameters to obtain a second picture of interest, and the target video is generated according to the second picture of interest.

8. The recording and broadcasting system according to claim 7, characterized in that: The virtual reality camera comprises at least two fisheye lenses, wherein the field of view of each of the at least two fisheye lenses is oriented toward different viewing angles, and the field of view of two adjacent fisheye lenses of the at least two fisheye lenses overlap; The virtual reality camera is used to obtain the classroom images taken by the at least two fisheye lenses at each shooting moment, obtain the classroom images from at least two perspectives at each shooting moment, splice the classroom images from at least two perspectives at each shooting moment to obtain a panoramic image of the classroom at each shooting moment, and arrange the panoramic images of the classroom at each shooting moment in order of shooting time to obtain the panoramic classroom video.

9. The recording and broadcasting system according to claim 7 or 8, characterized in that: The director host is also used to send the panoramic classroom video to a live broadcast server, so that the live broadcast server can distribute the panoramic classroom video to a client that supports playing virtual reality videos.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the electronic device executes the video processing method as described in any one of claims 1 to 6.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the video processing method according to any one of claims 1 to 6 is implemented.

Images