Virtual camera template generation method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a virtual camera dolly template generation method and device, electronic equipment and storage medium. The method comprises: obtaining a plurality of shooting images of a target object obtained by a real camera array at the same time; performing parameter calibration processing on a plurality of real cameras in the real camera array to obtain first camera parameters of each real camera; performing curve fitting on the spatial positions of the plurality of real cameras according to the first camera parameters to obtain a spatial curve; based on the plurality of shooting images and the first camera parameters, performing virtual camera configuration processing on the spatial curve to obtain a plurality of virtual cameras and second camera parameters corresponding to the plurality of virtual cameras; generating a virtual camera dolly template according to the second camera parameters corresponding to the plurality of virtual cameras; and the virtual camera dolly template is used for generating a bullet time video. According to the technical scheme provided by the present disclosure, the virtual camera dolly template can have more virtual lenses, and the generation efficiency of the bullet time video can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for generating virtual camera movement templates. Background Technology

[0002] With the development of computer technology and the widespread application of the Internet, six-degrees-of-freedom (6DOF) video is becoming increasingly popular, for example, in the display of bullet-time video in live sports broadcasts. Bullet-time video is an application form of 6DOF video that makes the target scene appear to be in a state of time freeze or slow motion, enabling the viewpoint to move continuously in space, thus presenting more information and creating a more impactful effect.

[0003] In related technologies, a camera array consisting of dozens or even hundreds of cameras is generally used to simultaneously capture the scene. The lenses of these real cameras are paused and zoomed in and out to obtain bullet time video, but this is prone to problems such as video stuttering or jumping. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and storage medium for generating virtual camera movement templates. The technical solution of this disclosure is as follows:

[0005] According to a first aspect of the present disclosure, a method for generating a virtual camera movement template is provided, comprising:

[0006] Acquire multiple images of a target object captured by a real camera array at the same time.

[0007] The parameters of the multiple real cameras in the real camera array are calibrated to obtain the first camera parameters of each real camera.

[0008] Based on the first camera parameters, the spatial positions of the plurality of real cameras are curve-fitted to obtain a spatial curve;

[0009] Based on the multiple captured images and the first camera parameters, virtual camera configuration processing is performed on the spatial curve to obtain multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras;

[0010] A virtual camera movement template is generated based on the second camera parameters corresponding to the plurality of virtual cameras. The virtual camera movement template includes the second camera parameters corresponding to the plurality of virtual cameras and is used to generate bullet-time video at any progress point in the video to be processed.

[0011] In one possible implementation, the step of configuring virtual cameras on the spatial curve based on the plurality of captured images and the first camera parameters to obtain a plurality of virtual cameras and second camera parameters corresponding to the plurality of virtual cameras includes:

[0012] Add a key virtual camera as the current key virtual camera to be configured;

[0013] Obtain the current video frame number of the currently configured key virtual camera and the current position information of the currently configured key virtual camera on the spatial curve;

[0014] Based on the current location information, the current video frame number, and the first camera parameters of the multiple real cameras, obtain the multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras.

[0015] In one possible implementation, obtaining the multiple virtual cameras and their corresponding second camera parameters based on the current location information, the current video frame number, and the first camera parameters of the multiple real cameras includes:

[0016] Select a first auxiliary real camera associated with the current key virtual camera from among the plurality of real cameras;

[0017] Using the first camera parameters of the first auxiliary real camera and the current position information, generate the first virtual camera parameters of the current key virtual camera;

[0018] Based on the first virtual camera parameters, the first camera parameters of the first reference real camera, and the captured image of the first reference real camera, a first-view image corresponding to the current key virtual camera is generated; the first reference real camera is a real camera among multiple real cameras whose distance from the current key virtual camera meets the distance condition.

[0019] When the first viewpoint image meets the image configuration conditions, the multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras are obtained based on the first virtual camera parameters corresponding to the image configuration conditions and the current video frame number.

[0020] In one possible implementation, the method further includes:

[0021] If the first viewpoint image does not meet the image configuration conditions, reconfigure the current position information of the current key virtual camera on the spatial curve; return to the step of obtaining the current position information of the current key virtual camera on the spatial curve until the first viewpoint image meets the image configuration conditions.

[0022] In one possible implementation, obtaining the plurality of virtual cameras and the second camera parameters corresponding to the plurality of virtual cameras based on the first virtual camera parameters corresponding to the image configuration conditions and the current video frame number includes:

[0023] If the current video frame number is not the preset initial number, obtain the adjacent historical key virtual camera of the previous configuration of the current key virtual camera;

[0024] Virtual camera interpolation is performed between the current key virtual camera and the historical key virtual camera to obtain at least one interpolated virtual camera and interpolation position information of each interpolated virtual camera. The number of the at least one interpolated virtual camera is the number of video frame numbers between the current video frame number and the historical video frame number corresponding to the adjacent historical key virtual camera.

[0025] Based on the multiple real cameras and the interpolated position information, the multiple virtual cameras and the corresponding second camera parameters are obtained.

[0026] In one possible implementation, obtaining the multiple virtual cameras and their corresponding second camera parameters based on the multiple real cameras and the interpolated position information includes:

[0027] Select a second auxiliary real camera associated with each of the interpolated virtual cameras from the plurality of real cameras;

[0028] Using the first camera parameters of the second auxiliary real camera and the interpolated position information, the second virtual camera parameters of each of the interpolated virtual cameras are generated;

[0029] Based on the second virtual camera parameters, the first camera parameters of the second reference real camera, and the captured images of the second reference real camera, a second perspective image of each of the at least one interpolated virtual camera is generated; the second reference real camera is a real camera among a plurality of real cameras whose distance from the interpolated virtual camera satisfies a distance condition.

[0030] Displaying the historical perspective image, the second perspective image, and the first perspective image yields a first playback result; the historical perspective image is the perspective image corresponding to any one of the configured historical key virtual cameras.

[0031] If the first playback result satisfies the preset playback result, the current key virtual camera, the at least one interpolated virtual camera, and the configured historical key virtual camera and historical interpolated virtual camera are used as the plurality of virtual cameras; and the historical virtual camera parameters of each of the configured historical key virtual cameras, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the historical interpolated virtual camera are used as the second camera parameters corresponding to the plurality of virtual cameras.

[0032] In one possible implementation, when the first playback result satisfies a preset playback result, the current key virtual camera, the at least one interpolated virtual camera, and the full set of historical key virtual cameras and historical interpolated virtual cameras are taken as the plurality of virtual cameras; and the historical virtual camera parameters of each of the full set of historical key virtual cameras, the first virtual camera parameters, at least one second virtual camera parameter, and the interpolated virtual camera parameters of the historical interpolated virtual cameras are taken as the second camera parameters corresponding to the plurality of virtual cameras, including:

[0033] If the first playback result satisfies the preset playback result, obtain the end indication information;

[0034] When the end indication information indicates the end, the current key virtual camera, the at least one interpolated virtual camera, and the full historical key virtual camera and the historical interpolated virtual camera are taken as the plurality of virtual cameras; and the historical virtual camera parameters, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the historical interpolated virtual camera are taken as the second camera parameters corresponding to the plurality of virtual cameras.

[0035] In one possible implementation, the method further includes:

[0036] If the end indication message indicates that the process has not ended, return to the step of adding a key virtual camera as the current key virtual camera to be configured.

[0037] In one possible implementation, the method further includes:

[0038] If the first playback result does not meet the preset playback result, reset the video frame number of the current key virtual camera as the current video frame number configured for the current key virtual camera; return to the step of obtaining the current video frame number configured for the current key virtual camera.

[0039] In one possible implementation, the method further includes:

[0040] If the current video frame number is the preset initial number, return to the step of adding a key virtual camera as the current key virtual camera to be configured.

[0041] In one possible implementation, generating a virtual camera movement template based on the second camera parameters corresponding to the plurality of virtual cameras includes:

[0042] Based on the video frame number corresponding to each of the multiple virtual cameras, the second camera parameters corresponding to the multiple virtual cameras are sorted to obtain a camera parameter sequence;

[0043] The camera parameter sequence is processed to generate a file, thereby obtaining the virtual camera movement template.

[0044] In one possible implementation, the step of configuring virtual cameras on the spatial curve based on the plurality of captured images and the first camera parameters to obtain a plurality of virtual cameras and second camera parameters corresponding to the plurality of virtual cameras includes:

[0045] A preset number of virtual cameras are configured on the spatial curve, and the virtual camera parameters of the preset number of virtual cameras are set;

[0046] Based on the multiple captured images, the first camera parameters, and the virtual camera parameters, generate a preset number of third-view images corresponding to each of the virtual cameras;

[0047] According to the video frame number configured for each of the third-view images, the video stitched together from the third-view images is played to obtain a second playback result;

[0048] If the second playback result does not meet the preset playback result, the virtual camera parameters of the preset number of virtual cameras are reset, and the generation step of the third-view image is returned until the playback result meets the preset playback result. The virtual camera parameters corresponding to the preset playback result are then used as the second camera parameters corresponding to the preset number of virtual cameras.

[0049] According to a second aspect of the present disclosure, a video processing method is provided, comprising:

[0050] Multiple videos to be processed are acquired; these videos are obtained by shooting the target object using multiple real cameras.

[0051] Obtain the target progress time of the video to be processed, wherein the target progress time is used to indicate the target progress of the video to be processed for bullet time video;

[0052] Extract the image corresponding to the target progress time from the plurality of videos to be processed to obtain a plurality of images to be processed;

[0053] Obtain the target virtual camera movement template and the camera parameter information of each of the plurality of real cameras corresponding to the target progress; the target virtual camera movement template includes a plurality of target virtual cameras and target virtual camera parameters corresponding to each target virtual camera;

[0054] Based on the multiple images to be processed, the multiple camera parameter information, and the target virtual camera parameters, generate target view images corresponding to each of the multiple target virtual cameras;

[0055] Based on multiple target perspective images, a bullet-time video corresponding to the target progress time is generated.

[0056] According to a third aspect of the present disclosure, a virtual camera movement template generation apparatus is provided, comprising:

[0057] The image capture module is configured to acquire multiple images of the target object captured by a real camera array at the same time.

[0058] The parameter calibration module is configured to perform parameter calibration processing on multiple real cameras in the real camera array to obtain the first camera parameters of each real camera.

[0059] The curve fitting module is configured to perform curve fitting on the spatial positions of the plurality of real cameras based on the first camera parameters to obtain a spatial curve;

[0060] The virtual camera configuration module is configured to perform virtual camera configuration processing on the spatial curve based on the multiple captured images and the first camera parameters, to obtain multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras;

[0061] The virtual camera movement template generation module is configured to generate a virtual camera movement template based on the second camera parameters corresponding to the plurality of virtual cameras; the virtual camera movement template includes the second camera parameters corresponding to the plurality of virtual cameras, and the virtual camera movement template is used to generate bullet time video at any progress moment in the video to be processed.

[0062] In one possible implementation, the virtual camera configuration module includes:

[0063] The key virtual camera adding unit is configured to add a key virtual camera as the current key virtual camera to be configured;

[0064] The numbering and location acquisition unit is configured to acquire the current video frame number of the current key virtual camera and the current location information of the current key virtual camera located on the spatial curve.

[0065] The virtual camera configuration unit is configured to perform operations based on the current location information, the current video frame number, and the first camera parameters of the plurality of real cameras to obtain the plurality of virtual cameras and the second camera parameters corresponding to the plurality of virtual cameras.

[0066] In one possible implementation, the virtual camera configuration unit includes:

[0067] The first auxiliary real camera selection unit is configured to select a first auxiliary real camera associated with the current key virtual camera from the plurality of real cameras;

[0068] The first virtual camera parameter generation unit is configured to generate the first virtual camera parameters of the current key virtual camera using the first camera parameters of the first auxiliary real camera and the current position information.

[0069] The first-view image generation unit is configured to generate a first-view image corresponding to the current key virtual camera based on the first virtual camera parameters, the first camera parameters of the first reference real camera, and the captured image of the first reference real camera; the first reference real camera is a real camera among a plurality of real cameras whose distance from the current key virtual camera meets the distance condition.

[0070] The second camera parameter acquisition unit is configured to, when the first viewpoint image meets the image configuration conditions, obtain the plurality of virtual cameras and the second camera parameters corresponding to the plurality of virtual cameras based on the first virtual camera parameters corresponding to the image configuration conditions and the current video frame number.

[0071] In one possible implementation, the device further includes:

[0072] The position reconfiguration module is configured to, when the first view image does not meet the image configuration conditions, reconfigure the position information of the current key virtual camera on the spatial curve as the current position information; and return to the step of obtaining the current position information of the current key virtual camera on the spatial curve until the first view image meets the image configuration conditions.

[0073] In one possible implementation, the second camera parameter acquisition unit includes:

[0074] The adjacent historical key virtual camera acquisition subunit is configured to acquire the adjacent historical key virtual camera configured before the current key virtual camera when the current video frame number is not the preset initial number.

[0075] The virtual camera interpolation processing subunit is configured to perform virtual camera interpolation processing between the current key virtual camera and the historical key virtual camera to obtain at least one interpolated virtual camera and interpolation position information of each interpolated virtual camera. The number of the at least one interpolated virtual camera is the number of video frame numbers between the current video frame number and the historical video frame number corresponding to the adjacent historical key virtual camera.

[0076] The virtual camera configuration subunit is configured to perform operations based on the plurality of real cameras and the interpolated position information to obtain the plurality of virtual cameras and the second camera parameters corresponding to the plurality of virtual cameras.

[0077] In one possible implementation, the virtual camera configuration subunit includes:

[0078] The second auxiliary real camera selection subunit is configured to perform the selection of a second auxiliary real camera associated with each of the plurality of real cameras from the interpolated virtual cameras;

[0079] The second virtual camera parameter generation subunit is configured to generate second virtual camera parameters for each of the interpolated virtual cameras using the first camera parameters of the second auxiliary real camera and the interpolation position information.

[0080] The second-view image generation subunit is configured to generate a second-view image for each of the at least one interpolated virtual camera based on the second virtual camera parameters, the first camera parameters of the second reference real camera, and the captured image of the second reference real camera; the second reference real camera is a real camera among a plurality of real cameras whose distance from the interpolated virtual camera satisfies a distance condition.

[0081] The first playback result acquisition subunit is configured to display the historical view image, the second view image, and the first view image to obtain the first playback result; the historical view image is the view image corresponding to any one of the historical key virtual cameras in the configured full set of historical key virtual cameras;

[0082] The second camera parameter acquisition subunit is configured to, when the first playback result satisfies the preset playback result, take the current key virtual camera, the at least one interpolated virtual camera, and the configured historical key virtual camera and historical interpolated virtual camera as the plurality of virtual cameras; and take the historical virtual camera parameters of each of the configured historical key virtual cameras, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the historical interpolated virtual camera as the second camera parameters corresponding to the plurality of virtual cameras.

[0083] In one possible implementation, the second camera parameter acquisition subunit includes:

[0084] The end indication information acquisition subunit is configured to acquire end indication information when the first playback result satisfies the preset playback result.

[0085] The second camera parameter determination subunit is configured to, when the end indication information indicates the end, take the current key virtual camera, the at least one interpolated virtual camera, and the full historical key virtual camera and the historical interpolated virtual camera as the plurality of virtual cameras; and take the historical virtual camera parameters of each of the full historical key virtual cameras, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the historical interpolated virtual camera as the second camera parameters corresponding to the plurality of virtual cameras.

[0086] In one possible implementation, the device further includes:

[0087] The first iteration module is configured to, if the end indication information indicates that the process has not ended, return to the step of adding a key virtual camera as the current key virtual camera to be configured.

[0088] In one possible implementation, the device further includes:

[0089] The second iteration module is configured to, if the first playback result does not meet the preset playback result, reset the video frame number of the current key virtual camera as the current video frame number configured for the current key virtual camera; and return to the step of obtaining the current video frame number configured for the current key virtual camera.

[0090] In one possible implementation, the device further includes:

[0091] The third iteration module is configured to, when the current video frame number is the preset initial number, return to the step of adding a key virtual camera as the current key virtual camera to be configured.

[0092] In one possible implementation, the virtual camera movement template generation module includes:

[0093] The camera parameter sequence acquisition unit is configured to sort the second camera parameters corresponding to the plurality of virtual cameras according to the video frame numbers corresponding to each of the plurality of virtual cameras, thereby obtaining a camera parameter sequence.

[0094] The virtual camera movement template generation unit is configured to perform file generation processing on the camera parameter sequence to obtain the virtual camera movement template.

[0095] In one possible implementation, the virtual camera configuration module includes:

[0096] The virtual camera and parameter setting unit is configured to configure a preset number of virtual cameras on the space curve and set the virtual camera parameters of the preset number of virtual cameras.

[0097] The third-view image generation unit is configured to generate a preset number of third-view images corresponding to each of the virtual cameras based on the multiple captured images, the first camera parameters, and the virtual camera parameters.

[0098] The second playback result acquisition unit is configured to play the video stitched together from the third-view images according to the video frame numbers configured for each of the third-view images, and obtain the second playback result.

[0099] The second camera parameter determination unit is configured to, when the second playback result does not meet the preset playback result, reset the virtual camera parameters of the preset number of virtual cameras, return to the third-view image generation step, until the playback result meets the preset playback result, and use the virtual camera parameters corresponding to the preset playback result as the second camera parameters corresponding to the preset number of virtual cameras.

[0100] According to a fourth aspect of the present disclosure, a video processing apparatus is provided, comprising:

[0101] The video acquisition module is configured to acquire multiple videos to be processed; these multiple videos are obtained by capturing images of the target object using multiple real cameras.

[0102] The bullet time determination module is configured to acquire the target progress time of the video to be processed, wherein the target progress time is used to indicate the target progress of the video to be processed for bullet time video.

[0103] The image extraction module is configured to extract the image corresponding to the target progress time from the plurality of videos to be processed, thereby obtaining a plurality of images to be processed;

[0104] The template and camera parameter acquisition module is configured to acquire the target virtual camera movement template and the camera parameter information of each of the plurality of real cameras corresponding to the target progress; the target virtual camera movement template includes a plurality of target virtual cameras and target virtual camera parameters corresponding to each target virtual camera.

[0105] The camera movement template module is configured to generate target view images corresponding to each of the multiple target virtual cameras based on the multiple images to be processed, the multiple camera parameter information, and the target virtual camera parameters.

[0106] The bullet time video generation module is configured to generate a bullet time video corresponding to the target progress time based on multiple target viewpoint images.

[0107] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method as described in any one of the first or second aspects above.

[0108] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided such that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the methods described in the first or second aspect of the present disclosure.

[0109] According to a seventh aspect of the present disclosure, a computer program product is provided, including computer instructions that, when executed by a processor, cause a computer to perform the method described in any one of the first or second aspects of the present disclosure.

[0110] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0111] By configuring multiple virtual cameras on the spatial curve fitted by a real camera array, second camera parameters corresponding to multiple virtual cameras are obtained. Based on these second camera parameters, a virtual camera movement template is generated. This allows the number of virtual cameras in the template to be unlimited, enabling the production of any number of virtual shots. Virtual shots offer greater freedom, richer camera language, and an infinitely high video frame rate, resulting in smoother bullet-time video. Furthermore, it avoids the explosive growth in data volume, system complexity, deployment difficulty, cost, and maintenance associated with adding real cameras. Additionally, subsequent videos can be quickly and efficiently generated into bullet-time videos based on the virtual camera movement template, meeting the real-time requirements of applications such as live streaming.

[0112] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0113] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0114] Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment.

[0115] Figure 2 This is a flowchart illustrating a virtual camera movement template generation method according to an exemplary embodiment.

[0116] Figure 3 This is a partial flowchart illustrating a virtual camera movement template generation method according to an exemplary embodiment.

[0117] Figure 4 This is a flowchart illustrating another method for generating virtual camera movement templates according to an exemplary embodiment.

[0118] Figure 5 This is a flowchart illustrating yet another method for generating virtual camera movement templates according to an exemplary embodiment.

[0119] Figure 6 This is a block diagram of a virtual camera movement template generation device according to an exemplary embodiment.

[0120] Figure 7 This is a block diagram illustrating an electronic device for generating virtual camera movement templates according to an exemplary embodiment. Detailed Implementation

[0121] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0122] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0123] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment, such as... Figure 1 As shown, the application environment may include server 01, terminal 02, and camera array 03.

[0124] In an optional embodiment, server 01 can be used for generating virtual camera movement templates. Specifically, server 01 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0125] In an optional embodiment, terminal 02 can be used to configure the current video frame number, current location information, etc. Specifically, terminal 02 can be, but is not limited to, electronic devices such as smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, and smart wearable devices. Optionally, the operating system running on the electronic device can be, but is not limited to, Android, iOS, Linux, Windows, etc.

[0126] In an optional embodiment, camera array 03 may include multiple real cameras, which can be used to simultaneously capture multiple images or videos of the same object. Here, the same object can be a person or scene, etc. Figure 1As shown, cameras 03 can be deployed in the same scene according to a preset arrangement to achieve synchronous shooting of the same object from different perspectives. Optionally, cameras 03 can transmit the captured sample video to server 01, so that server 01 can generate a virtual camera movement template. Alternatively, cameras 03 can transmit the captured video to be processed to terminal 02, so that terminal 02 can process the video to generate a bullet-time video corresponding to the target progress time. The preset arrangement may include, but is not limited to, the positions and camera parameters of each real camera in camera array 03.

[0127] In addition, it should be noted that, Figure 1 The example shown is merely one application environment of the virtual camera movement template generation method provided in this disclosure.

[0128] In the embodiments described in this specification, the server 01, terminal 02, and camera array 03 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this.

[0129] It should be noted that the following diagram illustrates one possible sequence of steps, and it is not strictly required to follow this order. Some steps can be performed in parallel without interdependence. The user information (including but not limited to user device information, user personal information, user behavior information, etc.) and data (including but not limited to data used for display, training data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0130] Figure 2 This is a flowchart illustrating a virtual camera movement template generation method according to an exemplary embodiment. For example... Figure 2 As shown, it may include the following steps.

[0131] In step S201, multiple images of the target object captured by the real camera array at the same time are obtained.

[0132] In the embodiments of this specification, in order to generate a virtual camera movement template for subsequent use in bullet-time video, synchronous screenshots of the target object can be captured by a real camera array. For example, sample videos of the target object captured by a real camera array at the same time can be obtained. The real camera array includes multiple real cameras, and correspondingly, multiple sample videos can be obtained. As an example, the number of real cameras can be less than a certain threshold to reduce the deployment difficulty and data processing complexity of the real cameras. For example, the number of real cameras can be 10, and this disclosure does not limit this.

[0133] The target object can be a target person, a target scene, etc., such as at least one athlete in a sports event or the entire venue of a sports event, etc. This disclosure does not limit it.

[0134] Furthermore, images corresponding to the same moment can be extracted from multiple sample videos to obtain multiple captured images. Here, "the same moment" can refer to a specific point in the video's progress; this moment can be user-specified or randomly selected, and this disclosure does not impose any limitations. It should be noted that the starting times for multiple real cameras in the real camera array to capture the target object can be the same, meaning they are captured synchronously.

[0135] In step S203, parameter calibration is performed on multiple real cameras in the real camera array to obtain the first camera parameters of each real camera.

[0136] In the embodiments of this specification, parameter calibration processing is performed on multiple real cameras in a real camera array to obtain the first camera parameters of each real camera. This parameter calibration processing can be based on a preset camera parameter calibration method, and this disclosure does not limit the preset camera parameter calibration method.

[0137] The first camera parameters and the virtual camera parameters in the embodiments of this specification can both include camera intrinsic parameters and extrinsic parameters, and this disclosure does not limit them.

[0138] As an example, the first camera parameters may include a rotation matrix, a translation vector, and a focal length. The rotation matrix can be used to describe the camera's attitude information in space, and the translation vector can describe the camera's position information in space.

[0139] In step S205, based on the parameters of the first camera, the spatial positions of multiple real cameras are curve-fitted to obtain a spatial curve.

[0140] In the embodiments of this specification, the camera positions of real cameras in space can be extracted from the first camera parameters, thereby allowing the camera positions to be fitted to form a spatial curve. Any position in this spatial curve can be represented by three-dimensional coordinates. The positions of multiple real cameras can lie on this spatial curve. The spatial curve can be a closed curve or an open curve, and this disclosure does not limit this. When the spatial curve is an open curve, the starting position of the spatial curve can be the first real camera in the real camera array, and the ending position can be the last real camera in the real camera array.

[0141] The fitting methods mentioned above may include connecting adjacent cameras with straight lines, connecting adjacent cameras with smoothed Bayesian curves, forcibly fitting into an arc, forcibly fitting into a straight line, etc., and this disclosure does not limit them.

[0142] In step S207, based on multiple captured images and first camera parameters, virtual camera configuration processing is performed on a spatial curve to obtain multiple virtual cameras and corresponding second camera parameters.

[0143] In one possible implementation, step S207 can be achieved through the following steps:

[0144] A preset number of virtual cameras are configured on a spatial curve, and the virtual camera parameters of the preset number of virtual cameras are set; the preset number can be pre-set, such as 300, etc., and this disclosure does not limit it. In one example, the preset number of virtual cameras can be aligned with a predetermined target in the captured image, and the virtual camera parameters of the preset number of virtual cameras can be automatically generated according to bullet time effects (such as time freeze, slow motion, focus, etc.).

[0145] This allows for the generation of a predetermined number of third-view images corresponding to each virtual camera, based on multiple captured images, first camera parameters, and virtual camera parameters, using a depth image rendering algorithm. For example, the stereo matching algorithm (patchmatch) can be used to convert captured images into depth images, enabling the generation of a predetermined number of third-view images corresponding to each virtual camera based on the depth images, first camera parameters, and virtual camera parameters. The depth image rendering algorithm can be the DIBR (depth image based rendering) technique in OpenCV, as detailed below.

[0146] The video, stitched together from the third-view images, is played according to their respective configured video frame numbers to obtain the playback result. For example, the video can be played in ascending order of video frame numbers to obtain the playback result. The configured video frame numbers for each third-view image are also the video frame numbers corresponding to each virtual camera. These video frame numbers can be set by the operator who triggered the virtual camera movement template generation, or they can be automatically set according to the order of a preset number of virtual cameras on the spatial curve from the start position to the end position. Virtual cameras closer to the start position can have smaller video frame numbers. The video frame numbers can be used as the sequence numbers for the virtual cameras in the virtual camera movement template. The start position of the spatial curve can be the position of any real camera on the spatial curve, or it can be a specified preset position; this disclosure does not limit this.

[0147] Furthermore, it can be determined whether the playback result meets the preset playback result. If the playback result does not meet the preset playback result, the virtual camera parameters of the preset number of virtual cameras can be reset, and the generation step of the third-view image can be returned until the playback result meets the preset playback result. The virtual camera parameters corresponding to the preset playback result are then used as the second camera parameters corresponding to the preset number of virtual cameras.

[0148] By configuring a preset number of virtual cameras on a spatial curve and setting the virtual camera parameters of the preset number of virtual cameras, the playback effect of the viewpoint images of the preset number of virtual cameras is used to adjust the overall second camera parameters to meet the preset playback result. This can improve the smoothness of the second camera parameters, thereby improving the smoothness of the camera movement template. The process is also relatively simple.

[0149] In step S209, a virtual camera movement template is generated based on the second camera parameters corresponding to the multiple virtual cameras.

[0150] In the embodiments described in this specification, the virtual camera movement template may include second camera parameters corresponding to multiple virtual cameras, which may be arranged in sequence. The virtual camera movement template can be used to generate bullet-time video at any progress point in the video to be processed.

[0151] In one optional implementation, the second camera parameters corresponding to the multiple virtual cameras can be sorted according to the video frame numbers corresponding to each virtual camera to obtain a camera parameter sequence; and the camera parameter sequence can be processed to generate a file to obtain a virtual camera movement template. As an example, the virtual camera movement template can be as follows:

[0152] Virtual camera 1, camera parameter 1; Virtual camera 2, camera parameter 2; ...; Virtual camera M, camera parameter M.

[0153] Where M can be the total number of virtual cameras, and M can be an integer greater than 1. Camera parameter 1 to camera parameter M can refer to multiple second camera parameters.

[0154] In the embodiments of this specification, the video frame numbers can be dynamically generated for multiple virtual cameras or manually set by the user; this disclosure does not limit this. The order of 1 to M can be arranged according to the sequence of the video frames. As an example, the video frame numbers can be 1 to M, meaning that the video frame numbers are used to characterize the order of the M virtual cameras in the virtual camera movement template during camera movement.

[0155] Furthermore, virtual camera movement templates can be stored and named with unique identifiers to facilitate subsequent retrieval of the virtual camera movement templates.

[0156] Optionally, at least one virtual camera and its corresponding camera parameters may be deleted from the virtual camera movement template to update the virtual camera movement template; this disclosure does not limit this.

[0157] By configuring multiple virtual cameras on the spatial curve fitted by a real camera array, second camera parameters corresponding to multiple virtual cameras are obtained. Based on these second camera parameters, a virtual camera movement template is generated. This allows the number of virtual cameras in the template to be unlimited, enabling the production of any number of virtual shots. Virtual shots offer greater freedom, richer camera language, and an infinitely high video frame rate, resulting in smoother bullet-time video. Furthermore, it avoids the explosive growth in data volume, system complexity, deployment difficulty, cost, and maintenance associated with adding real cameras. Additionally, subsequent videos can be quickly and efficiently generated into bullet-time videos based on the virtual camera movement template, meeting the real-time requirements of applications such as live streaming.

[0158] Figure 3 This is a partial flowchart illustrating a virtual camera movement template generation method according to an exemplary embodiment. In one possible implementation, such as... Figure 3 As shown, step S207 may include:

[0159] In step S301, a key virtual camera is added as the current key virtual camera to be configured.

[0160] In practical applications, after configuring a key virtual camera in the initial stage or afterward, another key virtual camera can be added as the current key virtual camera to be configured. In the embodiments of this specification, a counter, such as N, can be maintained for each key virtual camera, where N can be an integer greater than 0. In the initial stage, when a key virtual camera is added, N = 1. Afterward, for each additional key virtual camera added, N = N + 1, meaning N increments by 1. This allows for both counting and assigning numbers to the key virtual cameras. For example, if a key virtual camera is added and N is updated to 3, then the added key virtual camera can be recorded as C3.

[0161] In step S303, the current video frame number of the current key virtual camera and the current position information of the current key virtual camera located on the spatial curve are obtained.

[0162] In one example, if it's the first key virtual camera configured in the initial phase (denoted as C1), the current video frame number can be configured as 1. If it's not the first key virtual camera configured in the initial phase, let's say it's the second key virtual camera C2, then the current video frame number can be configured to be greater than 1. That is, for the key virtual camera C to be configured... N You can get CN-1 The video frame number, such as C N-1 The video frame number is A, and an integer greater than A can be set to C. N The video frame number.

[0163] As an alternative solution, it is also possible to obtain in C N With C N-1 The number of virtual cameras that need interpolation, for example, R and C. N-1 If the video frame number is A, then the current video frame number B = A + R + 1 can be configured. Here, R can be automatically generated or configured by the user, and this disclosure does not limit it.

[0164] In the embodiments of this specification, the current position information of the key virtual camera configured on the spatial curve can be represented by a position weight. For example, the position weight of the key virtual camera on the spatial curve can be obtained and set, with a range of 0.0-1.0, where 0.0 can represent the starting position of the spatial curve and 1.0 can represent the ending position of the spatial curve. The position weight can represent the proportion of the distance from the corresponding position on the spatial curve to the starting position to the total distance of the spatial curve. For example, a position weight of 0.1 can represent that the distance from the position on the spatial curve to the starting position is 0.1 of the total distance of the spatial curve.

[0165] The configuration of the current video frame number and the current location information can be configured manually, such as by providing a visual interface, or it can be configured automatically. This disclosure does not limit this.

[0166] In step S305, based on the current location information, the current video frame number, and the first camera parameters of multiple real cameras, multiple virtual cameras and their corresponding second camera parameters are obtained.

[0167] In one possible implementation, the virtual camera parameters of a key virtual camera can be determined based on the current location information and the first camera parameters of multiple real cameras. This allows for the acquisition of configured adjacent historical key virtual cameras based on the current video frame number, where the video frame number corresponding to the adjacent historical key virtual camera has the smallest difference from the current video frame number. This allows for the estimation of a certain number of interpolated virtual cameras between the adjacent historical key virtual cameras and the current key virtual camera. The virtual camera parameters of each interpolated virtual camera can then be determined based on the first camera parameters of the real cameras adjacent to each interpolated virtual camera. In this way, the configured adjacent historical key virtual cameras, the current key virtual camera, and a certain number of interpolated virtual cameras can be considered as multiple virtual cameras, and the virtual camera parameters corresponding to each of these multiple virtual cameras can be used as the second camera parameters for the multiple virtual cameras.

[0168] In another possible implementation, step S305 may include:

[0169] In step S3051, a first auxiliary real camera associated with the current key virtual camera is selected from multiple real cameras.

[0170] In one example, you can select a first real camera and a last real camera. The first real camera can be the one closest to the start of the space curve, and the last real camera can be the one closest to the end of the space curve.

[0171] In another example, the real camera closest to the current key virtual camera can be selected as the first auxiliary real camera associated with the current key virtual camera.

[0172] In another example, two adjacent real cameras can be selected as the first auxiliary real cameras associated with the current key virtual camera. Specifically, the space curve can be divided into multiple line segments, with each line segment having two adjacent real cameras at its two endpoints. Based on this, the real cameras corresponding to the two endpoints of the line segments whose position weights fall into can be selected as the first auxiliary real cameras associated with the current key virtual camera. For example, if the position weight of the current key virtual camera is set to 0.45, and the position weight of the 4th real camera is 0.4, and the position weight of the 5th real camera is 0.5, then the 4th and 5th cameras will be selected.

[0173] In another example, a focus mode can be used to select the first auxiliary real camera. For instance, a target in a captured image, such as a person, can be selected, and the real camera corresponding to this selected captured image can be used as the first auxiliary real camera associated with the current key virtual camera.

[0174] In step S3053, the first camera parameters of the current key virtual camera are generated using the first camera parameters of the first auxiliary real camera and the current position information.

[0175] In the embodiments of this specification, the parameters of the first virtual camera may include the translation vector, rotation matrix, and focal length of the current key virtual camera. The translation vector (position information) of the key virtual camera can be calculated from the current position information and the spatial curve. The method of obtaining the rotation matrix and focal length of the key virtual camera can correspond to the method of selecting the first auxiliary real camera. For example, it can be implemented in the following ways.

[0176] 1. When the first auxiliary real camera is the first real camera and the last real camera, linear interpolation can be used to generate the virtual camera based on the first camera parameters of the first auxiliary real camera and the current position information of the key virtual camera.

[0177] 2. If the first auxiliary real camera is the real camera closest to the current key virtual camera, directly assign the rotation matrix and focal length of the real camera closest to the current key virtual camera to the key virtual camera, thereby obtaining the rotation matrix and focal length in the first virtual camera parameters of the current key virtual camera.

[0178] 3. When the first auxiliary real camera is two adjacent real cameras, linear interpolation can be performed based on the first camera parameters of these two real cameras and the local weights (position weights on the line segment formed by the two real cameras) converted from the position weights of the key virtual camera.

[0179] 4. When the first auxiliary real camera is selected using a focus mode, a point can be chosen as the target point in the image captured by the first auxiliary real camera. The lens of the key virtual camera is then aligned with the spatial location of this target point, thereby obtaining the rotation matrix of the key virtual camera. Furthermore, the size of the target point in the viewpoint image generated by the key virtual camera can be kept consistent with its size in the captured image, thus obtaining the focal length of the key virtual camera.

[0180] Optionally, the parameters of the first virtual camera can also be adjusted to obtain the parameters for subsequent use. For example, the rotation matrix can be adjusted by modifying the correction values ​​of Euler angles (Pitch, Yaw, Roll), and the focal length can be adjusted by setting the scaling factor. Here, Pitch can refer to the pitch angle, which is the angle of rotation of an object around the X-axis, used to describe the degree of pitch. Yaw can refer to the yaw angle, which is the angle of rotation of an object around the Y-axis, used to describe the degree of yaw. Roll can refer to the roll angle, which is the angle of rotation of an object around the Z-axis, used to describe the degree of rotation.

[0181] In step S3055, a first-view image corresponding to the current key virtual camera is generated based on the first virtual camera parameters, the first camera parameters of the first reference real camera, and the captured image of the first reference real camera.

[0182] The first reference real camera can be any real camera among multiple real cameras whose distance from the current key virtual camera satisfies a distance condition. For example, the first reference real camera can be either of the two real cameras closest to the current key virtual camera, and these two real cameras can be located on either side of the current key virtual camera.

[0183] In one example, the patchmatch algorithm can be used to convert the image captured by the first reference real camera into a depth image. It should be noted that this depth image conversion step can be performed at any time before generating the first-view image corresponding to the current key virtual camera. For example, it can be performed when the image captured by the real camera is obtained, i.e., the image is converted for use in subsequent view image generation. This disclosure does not limit this.

[0184] Accordingly, a first-view image corresponding to the current key virtual camera can be generated using a depth image rendering algorithm based on the parameters of the first virtual camera, the parameters of the first reference real camera, and the image captured by the first reference real camera. In the embodiments of this specification, the depth image rendering algorithm, namely DIBR technology, can take the parameters of the first virtual camera, the parameters of the first reference real camera, the image captured by the first reference real camera, and the depth image corresponding to the captured image as input to the DIBR technology to generate a view image under the parameters of the first virtual camera, that is, the first-view image corresponding to the current key virtual camera.

[0185] In step S3057, if the first view image meets the image configuration conditions, multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras are obtained based on the first virtual camera parameters corresponding to the image configuration conditions and the current video frame number.

[0186] In the embodiments of this specification, the configured historical key virtual cameras and the current key virtual cameras can be obtained together as multiple virtual cameras. The parameters of the first virtual camera are arranged in the camera parameters corresponding to the historical key virtual cameras according to the current video frame number, resulting in the second camera parameters corresponding to the multiple virtual cameras. As an optional example, it can be determined whether the count of the key virtual cameras has reached a counting threshold. If it has, the above steps can be performed; this disclosure does not limit this. If it has not reached the threshold, the process can return to step S301 to continue adding new key virtual cameras.

[0187] Optionally, if the first-view image does not meet the image configuration conditions, the position information of the current key virtual camera located on the spatial curve can be reconfigured as the current position information; the step of obtaining the current position information of the current key virtual camera configured on the spatial curve can be returned, that is, the step of obtaining the current position information of the current key virtual camera configured on the spatial curve in step S303 can be returned, such as... Figure 4 As shown (for ease of description, Figure 4 The acquisition of S303 is divided into two steps (as shown in the diagram) until the first-view image meets the image configuration conditions. The reconfiguration can be performed manually or automatically, and this disclosure does not limit this. By adding a key virtual camera and using the assistance of a real camera, the accuracy of the second camera parameters can be improved; and during this process, the current position information is readjusted to ensure that the first-view image meets the image configuration conditions.

[0188] Reference Figure 4 In one possible implementation, step S3057 may include:

[0189] In step S401, if the current video frame number is not the preset initial number, the adjacent historical key virtual camera of the previous configuration of the current key virtual camera is obtained.

[0190] In the embodiments of this specification, the preset initial number may refer to the video frame number configured for the first key virtual camera, that is, the video frame number pre-assigned to the key virtual camera when N=1. For example, the preset initial number may be 1, and this disclosure does not limit it.

[0191] If the current video frame number is not the preset initial number, the adjacent historical key virtual cameras of the previous configuration of the current key virtual camera can be retrieved. For example, if the current key virtual camera is C... N You can get C N-1 As the adjacent historical key virtual camera of the previous configuration.

[0192] Optionally, if the current video frame number is a preset initial number, the step of adding a key virtual camera as the current key virtual camera to be configured can be returned, that is, step S301 can be returned.

[0193] In step S403, virtual camera interpolation is performed between the current key virtual camera and the historical key virtual camera to obtain at least one interpolated virtual camera and the interpolated position information of each interpolated virtual camera.

[0194] The number of at least one interpolated virtual camera is the number of video frame numbers between the current video frame number and the historical video frame numbers corresponding to adjacent historical key virtual cameras. For example, the current key virtual camera C N的 The current video frame number is B; the previously configured adjacent historical key virtual camera is C. N-1 If the video frame number is A, then the number of at least one interpolated virtual camera is R = BA-1.

[0195] In the embodiments of this specification, a preset interpolation method can be used for the interpolation processing. This preset interpolation method includes, but is not limited to, uniform speed interpolation, uniform acceleration interpolation, or directly drawing an interpolation curve. After interpolation processing, at least one interpolated virtual camera can be obtained, thereby acquiring the spatial position of each interpolated virtual camera as interpolation position information. The interpolation position information of each interpolated virtual camera lies on the spatial curve. That is, performing virtual camera interpolation processing between the current key virtual camera and the historical key virtual camera refers to performing virtual camera interpolation processing between line segments on the spatial curve whose endpoints are the positions of the current key virtual camera and the historical key virtual camera, respectively. For example, the positions of the current key virtual camera and the historical key virtual camera on the spatial curve can be determined, and virtual camera interpolation processing can be performed on the line segment between these two positions.

[0196] It should be noted that in the embodiments of this specification, a large number of interpolated virtual cameras are obtained by interpolating a small number of key virtual cameras. Based on this, the count of key virtual cameras is different from the video frame number. The multiple virtual cameras in the obtained virtual camera movement template are composed of key virtual cameras and interpolated virtual cameras that conform to the preset playback results. The video frame number is used to distinguish and sort the multiple virtual cameras. The count of key virtual cameras is used to count the number of virtual cameras that are turned off, and it can also be used to distinguish different key virtual cameras.

[0197] In step S405, based on multiple real cameras and interpolated position information, multiple virtual cameras and their corresponding second camera parameters are obtained.

[0198] In the embodiments of this specification, adjacent real cameras can be selected from multiple real cameras based on interpolation position information. The number of adjacent real cameras can be two or four, and this disclosure does not limit this. Therefore, based on the first camera parameters of the adjacent real cameras and the interpolation position information, the virtual camera parameters corresponding to the interpolated virtual camera can be determined. For details, please refer to the corresponding processing described above, which will not be repeated here. Thus, a key virtual camera, an interpolated virtual camera, and configured historical virtual cameras (including adjacent historical key virtual cameras) can be used as multiple virtual cameras, and the virtual camera parameters corresponding to each of the key virtual camera, the interpolated virtual camera, and the configured historical virtual camera can be used as the second camera parameters corresponding to the multiple virtual cameras.

[0199] Reference Figure 5 In one possible implementation, step S405 may include:

[0200] In step S4051, a second auxiliary real camera associated with each of the interpolated virtual cameras is selected from the plurality of real cameras. The specific method of this step can be found in step S305 above, specifically in step S3051, and will not be repeated here.

[0201] In step S4053, the second virtual camera parameters for each of the interpolated virtual cameras are generated using the first camera parameters of the second auxiliary real camera and the interpolated position information. The specific method for this step can be found in step S3053 above, and will not be repeated here.

[0202] In step S4055, based on the parameters of the second virtual camera, the parameters of the first camera of the second reference real camera, and the images captured by the second reference real camera, at least one second-view image of each interpolated virtual camera is generated. The second reference real camera is one of the real cameras whose distance from the interpolated virtual camera satisfies a distance condition, such as minimum distance. The specific method of this step can be found in step S3055 above, and will not be repeated here.

[0203] In step S4057, the historical view image, the second view image, and the first view image are displayed to obtain the first playback result.

[0204] The historical viewpoint image can be the viewpoint image corresponding to any one of the configured historical key virtual cameras. The configured historical key virtual cameras can refer to all configured historical key virtual cameras.

[0205] In practical applications, historical viewpoint images, second-viewpoint images, and first-viewpoint images can be displayed sequentially according to the video frame numbers to obtain the first playback result. For example, historical viewpoint images, second-viewpoint images, and first-viewpoint images can be stitched together into a video according to the video frame numbers, and then played to obtain the first playback result. The first playback result may include bullet time effects, whether the focused object is accurate, etc., but this disclosure does not limit these aspects and can be set according to actual needs.

[0206] In step S4059, if the first playback result satisfies the preset playback result, the current key virtual camera, at least one interpolated virtual camera, and the configured historical key virtual camera and historical interpolated virtual camera are used as multiple virtual cameras; and the historical virtual camera parameters, first virtual camera parameters, at least one second virtual camera parameters, and interpolated virtual camera parameters of the configured historical key virtual cameras are used as the second camera parameters corresponding to the multiple virtual cameras.

[0207] In other words, the camera parameters corresponding to all configured historical key virtual cameras, all historical interpolated virtual cameras, and the current key virtual camera can be used as the second camera parameters.

[0208] In one example, a camera movement template file can be set up to store continuously configured historical key virtual cameras and historical interpolated virtual cameras. If the first playback result meets the preset playback result, the current key virtual camera can be added to this camera movement template file. Each camera movement template file can be distinguished by unique identification information, so that the resulting virtual camera movement templates can also be distinguished by this unique identification information for easy retrieval in subsequent use.

[0209] By adding key virtual cameras and interpolating between two key virtual cameras, a larger number of interpolated virtual cameras can be obtained. This method of interpolating a large number of virtual cameras using a small number of key virtual cameras can improve camera movement efficiency and enhance the playback effect of bullet time videos.

[0210] Optionally, such as Figure 4 As shown, the method may further include: if the first playback result does not meet the preset playback result, the video frame number of the current key virtual camera can be reset as the current video frame number configured for the current key virtual camera; returning to the step of obtaining the current video frame number configured for the current key virtual camera, that is, returning to the step of obtaining the current video frame number configured for the current key virtual camera in step S303. By adjusting the current video frame number when the first playback result does not meet the preset playback result, the number of interpolated virtual cameras can be made more reasonable, thereby enabling the playback result to meet the preset playback result.

[0211] In one possible implementation, step S4059 may include: if the first playback result satisfies the preset playback result, obtaining end indication information; if the end indication information indicates the end, taking the current key virtual camera, at least one interpolated virtual camera, and all historical key virtual cameras and historical interpolated virtual cameras, as multiple virtual cameras; and taking the historical virtual camera parameters, first virtual camera parameters, at least one second virtual camera parameter, and interpolated virtual camera parameters of each of the all historical key virtual cameras as the second camera parameters corresponding to the multiple virtual cameras.

[0212] Optionally, if the end indication message indicates that the process has not ended, the process can return to the step of adding a key virtual camera as the current key virtual camera to be configured, i.e., it can return to step S301, as shown below. Figure 4 As shown. By setting up the reception of instruction information, the flexibility of virtual camera movement can be improved.

[0213] This specification also provides a video processing method in its embodiments, which may include:

[0214] Multiple videos to be processed are acquired; these videos are obtained by capturing the target object using multiple real cameras. It should be noted that these multiple real cameras may be different from those mentioned above. The target object here may be the same as or different from the target object in the virtual camera movement template; this disclosure does not impose any limitations on this.

[0215] The target progress time configured for the video to be processed is determined. The target progress time is used to indicate the target progress of the video to be processed for bullet time video, that is, it can determine which frame of the video to be processed for bullet time video.

[0216] Extract the image corresponding to the target progress time from multiple videos to be processed to obtain multiple images to be processed;

[0217] Furthermore, the target virtual camera movement template and the camera parameter information of each of the multiple real cameras corresponding to the target progress can be obtained; the target virtual camera movement template may include multiple target virtual cameras and the target virtual camera parameters corresponding to each target virtual camera. The target virtual camera movement template may be one of multiple saved virtual camera movement templates, may be selected by the user, or may be the default virtual camera movement template selected by the bullet; this disclosure does not limit this.

[0218] Then, based on multiple images to be processed, multiple camera parameter information, and target virtual camera parameters, multiple target viewpoint images corresponding to each target virtual camera can be generated. Alternatively, the method described in step S3055 above can be used: first, an auxiliary virtual camera is selected for each target virtual camera. Then, based on the target virtual camera parameters of each target virtual camera, the image to be processed from the auxiliary virtual camera corresponding to each target virtual camera, and the camera parameter information of the auxiliary virtual camera corresponding to each target virtual camera, a depth image rendering algorithm can be used to generate the target viewpoint image corresponding to each target virtual camera. For details, please refer to step S3055 above; it will not be repeated here.

[0219] Finally, a bullet-time video corresponding to the target progress time can be generated based on multiple target viewpoint images. For example, multiple target viewpoint images can be sequentially stitched together according to the arrangement order of multiple target virtual cameras to generate a bullet-time video corresponding to the target progress time.

[0220] By using virtual camera movement templates to generate bullet time videos, the efficiency and flexibility of bullet time video generation can be improved.

[0221] Figure 6 This is a block diagram illustrating a virtual camera movement template generation device according to an exemplary embodiment. (Refer to...) Figure 6 The device may include:

[0222] The image capture module 601 is configured to acquire multiple captured images of the target object simultaneously captured by a real camera array.

[0223] The parameter calibration module 603 is configured to perform parameter calibration processing on multiple real cameras in the real camera array to obtain the first camera parameters of each real camera.

[0224] The curve fitting module 605 is configured to perform curve fitting on the spatial positions of the plurality of real cameras based on the first camera parameters to obtain a spatial curve;

[0225] The virtual camera configuration module 607 is configured to perform virtual camera configuration processing on the spatial curve based on the multiple captured images and the first camera parameters, to obtain multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras;

[0226] The virtual camera movement template generation module 609 is configured to generate a virtual camera movement template based on the second camera parameters corresponding to the plurality of virtual cameras; the virtual camera movement template includes the second camera parameters corresponding to the plurality of virtual cameras, and the virtual camera movement template is used to generate bullet time video at any progress moment in the video to be processed.

[0227] In one possible implementation, the virtual camera configuration module 607 includes:

[0228] The key virtual camera adding unit is configured to add a key virtual camera as the current key virtual camera to be configured;

[0229] The numbering and location acquisition unit is configured to acquire the current video frame number of the current key virtual camera and the current location information of the current key virtual camera located on the spatial curve.

[0230] The virtual camera configuration unit is configured to perform operations based on the current location information, the current video frame number, and the first camera parameters of the plurality of real cameras to obtain the plurality of virtual cameras and the second camera parameters corresponding to the plurality of virtual cameras.

[0231] In one possible implementation, the virtual camera configuration unit includes:

[0232] The first auxiliary real camera selection unit is configured to select a first auxiliary real camera associated with the current key virtual camera from the plurality of real cameras;

[0233] The first virtual camera parameter generation unit is configured to generate the first virtual camera parameters of the current key virtual camera using the first camera parameters of the first auxiliary real camera, the image captured by the first auxiliary real camera, and the current location information.

[0234] The first-view image generation unit is configured to generate a first-view image corresponding to the current key virtual camera based on the first virtual camera parameters, the first camera parameters of the first reference real camera, and the captured image of the first reference real camera; the first reference real camera is a real camera among a plurality of real cameras whose distance from the current key virtual camera meets the distance condition.

[0235] The second camera parameter acquisition unit is configured to, when the first viewpoint image meets the image configuration conditions, obtain the plurality of virtual cameras and the second camera parameters corresponding to the plurality of virtual cameras based on the first virtual camera parameters corresponding to the image configuration conditions and the current video frame number.

[0236] In one possible implementation, the device may further include:

[0237] The position reconfiguration module is configured to, when the first view image does not meet the image configuration conditions, reconfigure the position information of the current key virtual camera on the spatial curve as the current position information; and return to the step of obtaining the current position information of the current key virtual camera on the spatial curve until the first view image meets the image configuration conditions.

[0238] In one possible implementation, the second camera parameter acquisition unit may include:

[0239] The adjacent historical key virtual camera acquisition subunit is configured to acquire the adjacent historical key virtual camera configured before the current key virtual camera when the current video frame number is not the preset initial number.

[0240] The virtual camera interpolation processing subunit is configured to perform virtual camera interpolation processing between the current key virtual camera and the historical key virtual camera to obtain at least one interpolated virtual camera and interpolation position information of each interpolated virtual camera. The number of the at least one interpolated virtual camera is the number of video frame numbers between the current video frame number and the historical video frame number corresponding to the adjacent historical key virtual camera.

[0241] The virtual camera configuration subunit is configured to perform operations based on the plurality of real cameras and the interpolated position information to obtain the plurality of virtual cameras and the second camera parameters corresponding to the plurality of virtual cameras.

[0242] In one possible implementation, the virtual camera configuration subunit includes:

[0243] The second auxiliary real camera selection subunit is configured to perform the selection of a second auxiliary real camera from the plurality of real cameras that is associated with the adjacent historical key virtual camera and the interpolated virtual camera, respectively.

[0244] The second virtual camera parameter generation subunit is configured to generate second virtual camera parameters for each of the interpolated virtual cameras using the first camera parameters of the second auxiliary real camera and the interpolation position information.

[0245] The second-view image generation subunit is configured to generate a second-view image for each of the at least one interpolated virtual camera based on the second virtual camera parameters, the first camera parameters of the second reference real camera, and the captured image of the second reference real camera; the second reference real camera is a real camera among a plurality of real cameras whose distance from the interpolated virtual camera satisfies a distance condition.

[0246] The first playback result acquisition subunit is configured to display the historical view image, the second view image, and the first view image to obtain the first playback result; the historical view image is the view image corresponding to any one of the historical key virtual cameras in the configured full set of historical key virtual cameras;

[0247] The second camera parameter acquisition subunit is configured to, when the first playback result satisfies the preset playback result, take the current key virtual camera, the at least one interpolated virtual camera, and the configured historical key virtual camera and historical interpolated virtual camera as the plurality of virtual cameras; and take the historical virtual camera parameters of each of the configured historical key virtual cameras, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the historical interpolated virtual camera as the second camera parameters corresponding to the plurality of virtual cameras.

[0248] In one possible implementation, the second camera parameter acquisition subunit may include:

[0249] The end indication information acquisition subunit is configured to acquire end indication information when the first playback result satisfies the preset playback result.

[0250] The second camera parameter determination subunit is configured to, when the end indication information indicates the end, take the current key virtual camera, the at least one interpolated virtual camera, and the full historical key virtual camera and the historical interpolated virtual camera as the plurality of virtual cameras; and take the historical virtual camera parameters of each of the full historical key virtual cameras, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the historical interpolated virtual camera as the second camera parameters corresponding to the plurality of virtual cameras.

[0251] In one possible implementation, the device may further include:

[0252] The first iteration module is configured to, if the end indication information indicates that the process has not ended, return to the step of adding a key virtual camera as the current key virtual camera to be configured.

[0253] In one possible implementation, the device may further include:

[0254] The second iteration module is configured to, if the first playback result does not meet the preset playback result, reset the video frame number of the current key virtual camera as the current video frame number configured for the current key virtual camera; and return to the step of obtaining the current video frame number configured for the current key virtual camera.

[0255] In one possible implementation, the device may further include:

[0256] The third iteration module is configured to, when the current video frame number is the preset initial number, return to the step of adding a key virtual camera as the current key virtual camera to be configured.

[0257] In one possible implementation, the virtual camera movement template generation module 509 may include:

[0258] The camera parameter sequence acquisition unit is configured to sort the second camera parameters corresponding to the plurality of virtual cameras according to the video frame numbers corresponding to each of the plurality of virtual cameras, thereby obtaining a camera parameter sequence.

[0259] The virtual camera movement template generation unit is configured to perform file generation processing on the camera parameter sequence to obtain the virtual camera movement template.

[0260] In one possible implementation, the virtual camera configuration module 607 may include:

[0261] The virtual camera and parameter setting unit is configured to configure a preset number of virtual cameras on the space curve and set the virtual camera parameters of the preset number of virtual cameras.

[0262] The third-view image generation unit is configured to generate a preset number of third-view images corresponding to each of the virtual cameras based on the plurality of captured images, the first camera parameters, and the virtual camera parameters.

[0263] The second playback result acquisition unit is configured to play the video stitched together from the third-view images according to the video frame numbers configured for each of the third-view images, and obtain the second playback result.

[0264] The second camera parameter determination unit is configured to, when the second playback result does not meet the preset playback result, reset the virtual camera parameters of the preset number of virtual cameras, return to the third-view image generation step, until the playback result meets the preset playback result, and use the virtual camera parameters corresponding to the preset playback result as the second camera parameters corresponding to the preset number of virtual cameras.

[0265] This specification also provides a video processing apparatus in its embodiments, the apparatus comprising:

[0266] The video acquisition module is configured to acquire multiple videos to be processed; these multiple videos are obtained by capturing images of the target object using multiple real cameras.

[0267] The bullet time determination module is configured to acquire the target progress time of the video to be processed, which is used to indicate the target progress of the video to be processed for bullet time video.

[0268] The image extraction module is configured to extract the image corresponding to the target progress time from the plurality of videos to be processed, thereby obtaining a plurality of images to be processed;

[0269] The template and camera parameter acquisition module is configured to acquire the target virtual camera movement template and the camera parameter information of each of the plurality of real cameras corresponding to the target progress; the target virtual camera movement template includes a plurality of target virtual cameras and target virtual camera parameters corresponding to each target virtual camera.

[0270] The camera movement template module is configured to generate target view images corresponding to each of the multiple target virtual cameras based on the multiple images to be processed, the multiple camera parameter information, and the target virtual camera parameters.

[0271] The bullet time video generation module is configured to generate a bullet time video corresponding to the target progress time based on multiple target viewpoint images.

[0272] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0273] Figure 7 This is a block diagram illustrating an electronic device for generating virtual camera movement templates according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for generating virtual motion templates.

[0274] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0275] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the virtual motion template generation method as described in the embodiments of this disclosure.

[0276] In an exemplary embodiment, a computer-readable storage medium is also provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the virtual camera movement template generation method of this disclosure. The computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0277] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the method for generating a virtual motion template in the embodiments of this disclosure.

[0278] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0279] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0280] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for generating a virtual camera movement template, characterized in that, include: Acquire multiple images of a target object captured by a real camera array at the same time. The parameters of the multiple real cameras in the real camera array are calibrated to obtain the first camera parameters of each real camera. Based on the first camera parameters, the spatial positions of the plurality of real cameras are curve-fitted to obtain a spatial curve; Based on the multiple captured images and the first camera parameters, virtual camera configuration processing is performed on the spatial curve to obtain multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras; A virtual camera movement template is generated based on the second camera parameters corresponding to the plurality of virtual cameras. The virtual camera movement template includes the second camera parameters corresponding to the plurality of virtual cameras and is used to generate bullet-time video at any progress point in the video to be processed.

2. The method according to claim 1, characterized in that, The step of configuring virtual cameras on the spatial curve based on the multiple captured images and the first camera parameters to obtain multiple virtual cameras and corresponding second camera parameters includes: Add a key virtual camera as the current key virtual camera to be configured; Obtain the current video frame number of the currently configured key virtual camera and the current position information of the currently configured key virtual camera on the spatial curve; Based on the current location information, the current video frame number, and the first camera parameters of the multiple real cameras, obtain the multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras.

3. The method according to claim 2, characterized in that, The step of obtaining the multiple virtual cameras and their corresponding second camera parameters based on the current location information, the current video frame number, and the first camera parameters of the multiple real cameras includes: Select a first auxiliary real camera associated with the current key virtual camera from among the plurality of real cameras; Using the first camera parameters of the first auxiliary real camera and the current position information, generate the first virtual camera parameters of the current key virtual camera; Based on the first virtual camera parameters, the first camera parameters of the first reference real camera, and the captured image of the first reference real camera, a first perspective image corresponding to the current key virtual camera is generated; the first reference real camera is one of the plurality of real cameras whose distance from the current key virtual camera meets the distance condition. When the first viewpoint image meets the image configuration conditions, the multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras are obtained based on the first virtual camera parameters corresponding to the image configuration conditions and the current video frame number.

4. The method according to claim 3, characterized in that, The method further includes: If the first viewpoint image does not meet the image configuration conditions, reconfigure the current position information of the current key virtual camera on the spatial curve; return to the step of obtaining the current position information of the current key virtual camera on the spatial curve until the first viewpoint image meets the image configuration conditions.

5. The method according to claim 3 or 4, characterized in that, The step of obtaining the plurality of virtual cameras and the corresponding second camera parameters based on the first virtual camera parameters corresponding to the image configuration conditions and the current video frame number includes: If the current video frame number is not the preset initial number, obtain the adjacent historical key virtual camera of the previous configuration of the current key virtual camera; Virtual camera interpolation is performed between the current key virtual camera and the historical key virtual camera to obtain at least one interpolated virtual camera and interpolation position information of each interpolated virtual camera. The number of the at least one interpolated virtual camera is the number of video frame numbers between the current video frame number and the historical video frame number corresponding to the adjacent historical key virtual camera. Based on the multiple real cameras and the interpolated position information, the multiple virtual cameras and the corresponding second camera parameters are obtained.

6. The method according to claim 5, characterized in that, The step of obtaining the multiple virtual cameras and their corresponding second camera parameters based on the multiple real cameras and the interpolated position information includes: Select a second auxiliary real camera associated with each of the interpolated virtual cameras from the plurality of real cameras; Using the first camera parameters of the second auxiliary real camera and the interpolated position information, the second virtual camera parameters of each of the interpolated virtual cameras are generated; Based on the second virtual camera parameters, the first camera parameters of the second reference real camera, and the captured images of the second reference real camera, a second perspective image of each of the at least one interpolated virtual camera is generated; the second reference real camera is the real camera among the plurality of real cameras whose distance from the interpolated virtual camera satisfies a distance condition. Displaying the historical perspective image, the second perspective image, and the first perspective image yields a first playback result; the historical perspective image is the perspective image corresponding to any one of the configured historical key virtual cameras. If the first playback result satisfies the preset playback result, the current key virtual camera, the at least one interpolated virtual camera, and the configured historical key virtual camera and historical interpolated virtual camera are used as the plurality of virtual cameras; and the historical virtual camera parameters of each of the configured historical key virtual cameras, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the historical interpolated virtual camera are used as the second camera parameters corresponding to the plurality of virtual cameras.

7. The method according to claim 6, characterized in that, When the first playback result satisfies the preset playback result, the current key virtual camera, the at least one interpolated virtual camera, and the configured historical key virtual camera and historical interpolated virtual camera are used as the plurality of virtual cameras; and the historical virtual camera parameters of each of the configured historical key virtual cameras, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the historical interpolated virtual camera are used as the second camera parameters corresponding to the plurality of virtual cameras, including: If the first playback result satisfies the preset playback result, obtain the end indication information; When the end indication information indicates the end, the current key virtual camera, the at least one interpolated virtual camera, and the configured historical key virtual camera and historical interpolated virtual camera are taken as the plurality of virtual cameras; and the historical virtual camera parameters, the first virtual camera parameters, at least one second virtual camera parameters, and the interpolated virtual camera parameters of the configured historical key virtual camera are taken as the second camera parameters corresponding to the plurality of virtual cameras.

8. The method according to claim 7, characterized in that, The method further includes: If the end indication message indicates that the process has not ended, return to the step of adding a key virtual camera as the current key virtual camera to be configured.

9. The method according to claim 6, characterized in that, The method further includes: If the first playback result does not meet the preset playback result, reset the video frame number of the current key virtual camera as the current video frame number configured for the current key virtual camera; return to the step of obtaining the current video frame number configured for the current key virtual camera.

10. The method according to claim 6, characterized in that, The method further includes: If the current video frame number is the preset initial number, return to the step of adding a key virtual camera as the current key virtual camera to be configured.

11. The method according to claim 1, characterized in that, The step of generating a virtual camera movement template based on the second camera parameters corresponding to the plurality of virtual cameras includes: Based on the video frame number corresponding to each of the multiple virtual cameras, the second camera parameters corresponding to the multiple virtual cameras are sorted to obtain a camera parameter sequence; The camera parameter sequence is processed to generate a file, thereby obtaining the virtual camera movement template.

12. The method according to claim 1, characterized in that, The step of configuring virtual cameras on the spatial curve based on the multiple captured images and the first camera parameters to obtain multiple virtual cameras and corresponding second camera parameters includes: A preset number of virtual cameras are configured on the spatial curve, and the virtual camera parameters of the preset number of virtual cameras are set; Based on the multiple captured images, the first camera parameters, and the virtual camera parameters, generate a preset number of third-view images corresponding to each of the virtual cameras; According to the video frame number configured for each of the third-view images, the video stitched together from the third-view images is played to obtain a second playback result; If the second playback result does not meet the preset playback result, the virtual camera parameters of the preset number of virtual cameras are reset, and the generation step of the third-view image is returned until the playback result meets the preset playback result. The virtual camera parameters corresponding to the preset playback result are then used as the second camera parameters corresponding to the preset number of virtual cameras.

13. A video processing method, characterized in that, include: Acquire multiple videos to be processed; The multiple videos to be processed were obtained by shooting the target object using multiple real cameras; Obtain the target progress time of the video to be processed, wherein the target progress time is used to indicate the target progress of the video to be processed for bullet time video; Extract the image corresponding to the target progress time from the plurality of videos to be processed to obtain a plurality of images to be processed; Obtain the target virtual camera movement template and the camera parameter information of each of the multiple real cameras corresponding to the target progress; The target virtual camera movement template includes multiple target virtual cameras and target virtual camera parameters corresponding to each target virtual camera; Based on the multiple images to be processed, the multiple camera parameter information, and the target virtual camera parameters, generate target view images corresponding to each of the multiple target virtual cameras; Based on multiple target perspective images, a bullet-time video corresponding to the target progress time is generated.

14. A virtual camera movement template generation device, characterized in that, include: The image capture module is configured to acquire multiple images of the target object captured by a real camera array at the same time. The parameter calibration module is configured to perform parameter calibration processing on multiple real cameras in the real camera array to obtain the first camera parameters of each real camera. The curve fitting module is configured to perform curve fitting on the spatial positions of the plurality of real cameras based on the first camera parameters to obtain a spatial curve; The virtual camera configuration module is configured to perform virtual camera configuration processing on the spatial curve based on the multiple captured images and the first camera parameters, to obtain multiple virtual cameras and the second camera parameters corresponding to the multiple virtual cameras; The virtual camera movement template generation module is configured to generate a virtual camera movement template based on the second camera parameters corresponding to the plurality of virtual cameras; the virtual camera movement template includes the second camera parameters corresponding to the plurality of virtual cameras, and the virtual camera movement template is used to generate bullet time video at any progress moment in the video to be processed.

15. A video processing apparatus, characterized in that, include: The video acquisition module is configured to acquire multiple videos to be processed. The multiple videos to be processed were obtained by shooting the target object using multiple real cameras; The bullet time determination module is configured to acquire the target progress time of the video to be processed, wherein the target progress time is used to indicate the target progress of the video to be processed for bullet time video. The image extraction module is configured to extract the image corresponding to the target progress time from the plurality of videos to be processed, thereby obtaining a plurality of images to be processed; The template and camera parameter acquisition module is configured to acquire the target virtual camera movement template and the camera parameter information of each of the multiple real cameras corresponding to the target progress. The target virtual camera movement template includes multiple target virtual cameras and target virtual camera parameters corresponding to each target virtual camera; The camera movement template module is configured to generate target view images corresponding to each of the multiple target virtual cameras based on the multiple images to be processed, the multiple camera parameter information, and the target virtual camera parameters. The bullet time video generation module is configured to generate a bullet time video corresponding to the target progress time based on multiple target viewpoint images.

16. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the virtual camera movement template generation method as described in any one of claims 1 to 12 or the video processing method as described in claim 13.

17. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the virtual camera movement template generation method as described in any one of claims 1 to 12 or the video processing method as described in claim 13.

18. A computer program product, characterized in that, The method includes computer instructions that, when executed by a processor, cause a computer to perform the virtual camera movement template generation method as described in any one of claims 1 to 12 or the video processing method as described in claim 13.

Images