Video capture method and apparatus, and computer-readable storage medium

Abstract

A video capture method, comprising: acquiring a first panoramic image by means of a panoramic capture device (S110); acquiring composition information, wherein the composition information indicates composition of a plurality of local views in the first panoramic image, and the composition information of the plurality of local views can be adjusted on the basis of an input of a user (S120); on the basis of the first panoramic image and the composition information, outputting a preview picture, wherein the preview picture contains a plurality of local images corresponding to the plurality of local views in the first panoramic image (S130); and in response to a video capture instruction of the user, outputting a target video having composition information identical to that corresponding to the preview picture, wherein the target video is obtained by means of a second panoramic image captured by the panoramic capture device (S140). The target video comprising the plurality of local views can be obtained. Also provided are an apparatus and a storage medium.

Description

Video shooting method, apparatus, and computer-readable storage medium Technical Field

[0001] This application relates to the field of video shooting, and more particularly to a video shooting method, apparatus, and computer-readable storage medium. Background Technology

[0002] Because there is a geometric mapping relationship between the three-dimensional world observed by panoramic cameras and the presentation of planar images, some related technologies stretch the overall image of panoramic images captured by panoramic cameras before outputting them to achieve a presentation effect similar to traditional planar images. However, the content of the stretched image is obviously stretched and compressed, and it is not suitable for direct viewing or dissemination.

[0003] In existing technologies, panoramic shooting devices often directly output raw panoramic videos, which users need to edit in post-production to obtain the desired target video. This process is relatively complex, has a high learning curve, and is difficult to meet the real-time requirements of applications such as live streaming.

[0004] Summary of the Invention

[0005] Based on this, this application provides a video shooting method, apparatus, and computer-readable storage medium, which can obtain target videos including multiple local perspectives.

[0006] In a first aspect, embodiments of this application provide a video shooting method, including:

[0007] The first panoramic image is acquired using a panoramic shooting device;

[0008] Obtain composition information, which indicates the composition of multiple local viewpoints in the first panoramic image, wherein the composition information of the multiple local viewpoints can be adjusted based on user input;

[0009] Based on the first panoramic image and the composition information, a preview screen is output, wherein the preview screen includes multiple local images corresponding to multiple local perspectives in the first panoramic image;

[0010] In response to the user's video shooting command, a target video with the same composition information as the preview screen is output, wherein the target video is obtained through a second panoramic image acquired by the panoramic shooting device.

[0011] Secondly, embodiments of this application provide a video shooting method, the method comprising:

[0012] In response to the user's video shooting command, a panoramic image is acquired through a panoramic shooting device;

[0013] A target video is obtained based on the panoramic image, wherein the target video is obtained by processing at least a portion of the panoramic image;

[0014] During the acquisition of the target video, composition information input by the user is acquired, the composition information indicating the composition of multiple local viewpoints in at least part of the panoramic image;

[0015] Based on the composition information, before obtaining the target video, at least a portion of the panoramic image is displayed with multiple local perspectives, wherein the changes in the multiple local perspectives in at least a portion of the panoramic image are associated with the composition information.

[0016] Thirdly, embodiments of this application provide a video recording device, which includes: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program and, when executing the computer program, to perform the following steps:

[0017] Acquire the first panoramic image;

[0018] Obtain composition information, which indicates the composition of multiple local viewpoints in the first panoramic image, wherein the composition information of the multiple local viewpoints can be adjusted based on user input;

[0019] Based on the first panoramic image and the composition information, a preview screen is output, wherein the preview screen includes multiple local images corresponding to multiple local perspectives in the first panoramic image;

[0020] In response to the user's video shooting command, a target video with the same composition information as the preview screen is output, wherein the target video is obtained by acquiring a second panoramic image after the video shooting command.

[0021] Fourthly, embodiments of this application provide a video recording device, the video recording device comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to execute the computer program and, when executing the computer program, to perform the following steps:

[0022] In response to the user's video recording command, acquire a panoramic image;

[0023] A target video is obtained based on the panoramic image, wherein the target video is obtained by processing at least a portion of the panoramic image;

[0024] During the acquisition of the target video, composition information input by the user is acquired, the composition information indicating the composition of multiple local viewpoints in at least part of the panoramic image;

[0025] Based on the composition information, before obtaining the target video, at least a portion of the panoramic image is displayed with multiple local perspectives, wherein the changes in the multiple local perspectives in at least a portion of the panoramic image are associated with the composition information.

[0026] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the above-described method.

[0027] This application provides a video shooting method, apparatus, and computer-readable storage medium. The video shooting apparatus can display a preview screen containing multiple local perspectives, adjust the composition according to user input and preview the composition effect, and directly generate a target video with the same composition as the preview screen according to the video shooting command. This can significantly reduce the complexity of obtaining the target video and enrich the forms of video output by the video shooting device.

[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the disclosure of the embodiments of this application. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 is a flowchart illustrating a video shooting method provided in an embodiment of this application;

[0031] Figure 2 is a schematic diagram of a panoramic shooting device provided in an embodiment of this application;

[0032] Figure 3a is a schematic diagram of a panoramic image in one embodiment of this application;

[0033] Figure 3b is a schematic diagram of the target object in the panoramic image;

[0034] Figures 3c and 3d are schematic diagrams of local perspectives of the target object;

[0035] Figure 3e is a schematic diagram of the target image;

[0036] Figure 4a is a schematic diagram of a panoramic image in another embodiment of this application;

[0037] Figures 4b to 7 are schematic diagrams of the layout in some implementation methods;

[0038] Figure 8 is a flowchart illustrating a video shooting method according to another embodiment of this application;

[0039] Figure 9 is a schematic block diagram of a video generation system provided in an embodiment of this application.

[0040] Figure 10 is a schematic block diagram of a video shooting device provided in an embodiment of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0043] When shooting video with traditional cameras, the field of view (FOV) needs to encompass the subject. However, when the subject or the camera itself is moving (e.g., walking, running, cycling, skiing), the subject often briefly falls out of the frame, or the composition is unsatisfactory, reducing the success rate of the shot. Furthermore, when the photographer is interested in more than one subject, the field of view of a traditional camera lens may not be able to capture multiple subjects simultaneously, necessitating separate shooting sessions or simultaneous shooting with multiple devices.

[0044] Compared to traditional cameras, the main advantage of panoramic cameras lies in their wider field of view. Because a panoramic camera offers a 360° panoramic view, it can capture images of a larger area of ​​the subject. Users of panoramic cameras can simultaneously record most of the information around the camera while shooting video, avoiding situations where elements exceed the frame or the composition is poor, leading to failed shots due to lens FOV limitations.

[0045] In such scenarios, users often want to capture images of subjects located in different positions simultaneously during video recording, in order to generate shooting effects that traditional cameras require setting up multiple camera positions to achieve, such as multi-screen images, picture-in-picture images, etc.

[0046] However, during the video shooting process of panoramic cameras, the relevant technologies only save the original panoramic video. Without complex post-processing, it is impossible to output multiple view files in the same video file. Currently, the editing of panoramic images requires users to edit them through post-editing software after shooting the panoramic images. The operation is relatively complicated, has a high learning curve, and is difficult to meet the needs of high real-time requirements such as live streaming.

[0047] Based on this, this application provides a video shooting method. Before outputting the video, the user can simultaneously preview and display multiple images from different local perspectives. The images from different local perspectives are generated from panoramic images. The multiple different perspectives can be adjusted according to the composition information input by the user before shooting, so that the user can adjust the composition and preview the composition effect before shooting, thereby reducing the complexity of post-editing for the target video with the preview effect obtained from the video shooting.

[0048] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0049] Please refer to Figure 1, which is a flowchart illustrating a video shooting method provided in an embodiment of this application.

[0050] As shown in Figure 1, the video shooting method of this application embodiment includes steps S110 to S140.

[0051] Step S110: Acquire the first panoramic image using a panoramic shooting device.

[0052] Step S120: Obtain composition information, wherein the composition information indicates the composition of multiple local perspectives in the first panoramic image, and the composition information of the multiple local perspectives can be adjusted based on user input.

[0053] Step S130: Based on the first panoramic image and the composition information, output a preview screen, wherein the preview screen includes multiple local images corresponding to multiple local perspectives in the first panoramic image.

[0054] Step S140: In response to the user's video shooting command, output a target video with the same composition information as the preview screen, wherein the target video is obtained through the second panoramic image acquired by the panoramic shooting device.

[0055] In this embodiment, before outputting the target video, the user can simultaneously preview multiple images from different local perspectives through a preview screen. These multiple perspectives can be adjusted based on the composition information input by the user before video shooting. The user can adjust the composition and preview the composition effect before shooting. For example, if the user is satisfied with the composition of the preview screen, they can trigger the video shooting command to trigger the video shooting device to output a target video with the same composition information as the preview screen. The composition of multiple local images in the target video is the same as the composition of the preview screen. Thus, compared to related technologies where panoramic shooting devices can only output raw panoramic videos, requiring post-editing to obtain the desired video, this embodiment allows the video shooting device to display a preview screen containing multiple local perspectives, adjust the composition based on user input, preview the composition effect, and directly generate a target video with the same composition as the preview screen according to the video shooting command. This significantly reduces the complexity of obtaining the target video, enriches the form of video output by the video shooting device, and reduces the complexity of post-editing.

[0056] The video shooting device can be a panoramic shooting device, or other terminal devices or server devices that communicate with the panoramic shooting device, and there is no limitation on it.

[0057] Step S110: Acquire the first panoramic image using a panoramic shooting device.

[0058] Acquiring a first panoramic image using a panoramic imaging device can be a process where the visual sensor of the panoramic imaging device converts light signals into electrical signals, and the first panoramic image can be generated based on the electrical signals output by the visual sensor. In some embodiments, the first panoramic image can be an image obtained after exposure by the panoramic imaging device; in some embodiments, the second panoramic image can be an image obtained after exposure and encoding by the panoramic imaging device.

[0059] In some implementations, the panoramic shooting device may be a panoramic camera or a mobile platform.

[0060] Optionally, the panoramic camera can be handheld by the user or mounted on a support, which can be held by the user or fixed to a mobile platform such as a vehicle or ship. For example, the support can include a pole-type support, such as a selfie stick, but it is not limited to this. For example, the panoramic camera can be connected to a helmet or bicycle handlebars via a bracket or other support.

[0061] Mobile platforms can be manned or unmanned. They can be land-based, such as land robots and cars; they can also be water-based or underwater; and they can be air-based, such as aircraft, including multi-rotor aircraft, fixed-wing aircraft, and helicopters. Mobile platforms can also be amphibious, such as flying cars.

[0062] Please refer to Figure 2, which is a schematic diagram of a panoramic camera. The panoramic camera 10 may include two back-to-back visual sensors 11. The two visual sensors 11 can be fisheye visual sensors. The FOV of a single fisheye visual sensor can be greater than 180°, thereby combining the two back-to-back visual sensors 11 to obtain a 360° panoramic field of view. The panoramic camera 10 may also include a display component, such as a display screen 12, to present a preview image or video image to the user, and / or to receive user input operations. The panoramic camera 10 may also include other user input components, such as joysticks, dials, or directional buttons 13, to input different control commands to the panoramic camera 10 through various user input components, adapting to different shooting needs. It should be understood that the arrangement of the visual sensors 11, display screen 12, and physical buttons 13 of the panoramic camera 10 can be set according to actual needs. For example, the display screen 12 and the physical buttons 13 can be set on the same side of the panoramic camera 10, which is not limited here.

[0063] For ease of explanation, the following explanation will primarily use a panoramic camera as the panoramic shooting device to illustrate the implementation of this application.

[0064] In some implementations, the panoramic image includes a horizontal panoramic image of at least 180 degrees. For example, the panoramic image includes a horizontal panoramic image of 360 degrees. "Horizontal" can refer to a horizontal plane along the perimeter of the panoramic shooting device, and the panoramic image includes a panoramic image of at least 180 degrees around the perimeter of the panoramic shooting device. For example, the panoramic image includes at least 180 degrees of the image in front of the panoramic shooting device, or at least 180 degrees of the image behind the panoramic shooting device, or at least 180 degrees of the image to the left of the panoramic shooting device, or at least 180 degrees of the image to the right of the panoramic shooting device; when the panoramic image includes a horizontal panoramic image of 360 degrees, the panoramic image includes the image surrounding the panoramic shooting device.

[0065] In some embodiments, the panoramic image includes a panoramic image with a vertical range of at least 180 degrees. For example, the panoramic image includes a panoramic image with a vertical range of 360 degrees. "Vertical" can refer to being perpendicular to the horizontal plane of the panoramic shooting device. For instance, the panoramic image includes a panoramic image with a vertical range of at least 180 degrees above the panoramic shooting device, or a panoramic image with a vertical range of at least 180 degrees below the panoramic shooting device, or a panoramic image extending from the top side of the panoramic shooting device through the front, left, rear, or right side to the bottom side by at least 180 degrees.

[0066] In some embodiments, the panoramic image includes a panoramic image with at least 180 degrees horizontally and at least 180 degrees vertically. For example, the panoramic image includes a panoramic image with 360 degrees horizontally and 360 degrees vertically. For instance, the panoramic image may include at least a hemispherical panoramic image; when the panoramic image includes a panoramic image with 360 degrees horizontally and 360 degrees vertically, the panoramic image may be a spherical panoramic image.

[0067] It should be understood that the field of view (FOV) of a single visual sensor can be greater than 180°, and therefore a single visual sensor can acquire panoramic images with an FOV greater than 180°. In some embodiments, acquiring the first panoramic image and / or the second panoramic image via a panoramic shooting device includes: acquiring the panoramic image via a single visual sensor. For example, when the panoramic shooting device is a panoramic camera, the first panoramic image and / or the second panoramic image can be acquired by a single visual sensor of the panoramic camera.

[0068] For example, the visual sensor is the main visual sensor of the panoramic shooting device. For instance, the main visual sensor may include a fisheye camera and / or a wide-angle camera; the main visual sensor may also be one of multiple visual sensors included in the panoramic shooting device, used to perform the shooting function. Optionally, the main visual sensor has a field of view greater than or equal to 180 degrees, enabling it to capture images within a wider field of view.

[0069] In some embodiments, acquiring the first panoramic image and / or the second panoramic image via a panoramic shooting device includes: obtaining the panoramic image through image processing obtained from at least two visual sensors. For example, when the panoramic shooting device is a panoramic camera, the panoramic image can be acquired by two visual sensors of the panoramic camera positioned opposite each other.

[0070] For example, a panoramic imaging device may include at least two visual sensors, and the shooting directions of the at least two visual sensors may be different; or, the imaging axis directions of the at least two visual sensors may be different. Specifically, when the visual sensors are optical sensors, the imaging axis may be the optical axis of the visual sensors. The images acquired by the at least two visual sensors can be used to obtain the aforementioned panoramic image through image stitching, image synthesis, or other methods. The at least two visual sensors may include a fisheye camera and / or a wide-angle camera, or may also include cameras with multiple viewpoints. The aforementioned panoramic image can also be obtained through image stitching, image synthesis, or other methods.

[0071] For example, the at least two visual sensors include two visual sensors arranged back-to-back. Optionally, the two visual sensors are distributed vertically along the vertical direction of the movable platform, or distributed horizontally or backward along the horizontal direction of the movable platform. For instance, if each of the two visual sensors has a hemispherical viewing angle, a panoramic image with a spherical viewing angle can be obtained based on image stitching, image synthesis, or other methods.

[0072] In some embodiments, the panoramic shooting device may be equipped with a perceptual vision sensor, which is used to detect obstacles in the environment surrounding the panoramic shooting device, such as a movable platform.

[0073] For example, the visual sensor is disposed on at least one side of the movable platform. For example, the at least one side includes at least one of the front, rear, left, right, upper, and lower sides of the movable platform. For instance, a visual sensor is disposed on the front side of the movable platform. In response to detecting an obstacle by the visual sensor on the front side of the movable platform, the movable platform stops moving forward and may also move backward to avoid colliding with the obstacle on the front side. For instance, the visual sensor may be a black-and-white visual sensor or a color visual sensor, and / or the resolution of the visual sensor may be lower than that of a visual sensor used to capture panoramic images, thereby improving the efficiency of obstacle detection. Optionally, the visual sensor includes a monocular visual sensor and / or a binocular visual sensor; wherein the binocular visual sensor can acquire depth information of the environment, which can improve the accuracy of obstacle avoidance.

[0074] For example, when the panoramic shooting device is a mobile platform, the first panoramic image and / or the second panoramic image can be acquired by a vision sensor that is not a perception sensor in the mobile platform. Specifically, when the panoramic shooting device is equipped with both a main vision sensor and a perception vision sensor, the panoramic image can be acquired by the main vision sensor, which is not a perception vision sensor.

[0075] Step S120: Obtain composition information, wherein the composition information indicates the composition of multiple local perspectives in the first panoramic image, and the composition information of the multiple local perspectives can be adjusted based on user input.

[0076] Step S130: Based on the first panoramic image and the composition information, output a preview screen, wherein the preview screen includes multiple local images corresponding to multiple local perspectives in the first panoramic image.

[0077] In some implementations, the mapping information is used to indicate at least one of the following:

[0078] The number of the multiple local perspectives;

[0079] The FOVs corresponding to the multiple local perspectives;

[0080] The FOV size corresponding to the multiple local viewpoints;

[0081] The display positions of the multiple local images corresponding to the multiple local viewpoints;

[0082] The display size of the multiple local images corresponding to the multiple local viewpoints.

[0083] For example, please refer to Figures 3a to 3e. The composition information indicates two local viewpoints, and the corresponding preview screen includes two local images corresponding to the two local viewpoints. One local viewpoint indicated by the composition information is the FOV where the elephant is located, and the size of this FOV corresponds to the size of one of the rectangles in Figure 3c. The other local viewpoint is the FOV where the person is located, and the size of this FOV corresponds to the size of one of the rectangles in Figure 3d. As shown in Figure 3e, the composition information also indicates that the display positions of the two local images are such that the local image corresponding to the elephant occupies the entire screen, and the local image corresponding to the person occupies the upper right corner of the screen. Furthermore, the display size of the local image corresponding to the elephant is larger than the display size of the local image corresponding to the person.

[0084] In some implementations, the step of outputting a preview image based on the first panoramic image and the composition information includes: cropping the first panoramic image according to the composition information to obtain multiple local images corresponding to the multiple local viewpoints; and outputting the preview image according to the composition indicated by the composition information and the multiple local images corresponding to the multiple local viewpoints.

[0085] It should be understood that the partial image can be obtained by cropping from the panoramic image, which can improve the real-time performance of the preview. However, this is not the only possibility; for example, the partial image can be obtained by scaling or adjusting the resolution of a portion of the panoramic image. The video recording device can also perform perspective transformation processing on the first panoramic image or partial image to make the image displayed in the preview a more intuitive planar image.

[0086] In some implementations, multiple partial images in the preview screen can be displayed based on the display positions indicated by the composition information. The video recording device can also combine multiple partial images into a single preview image according to the display positions indicated by the composition information before outputting it.

[0087] For example, outputting the preview screen according to the composition indicated by the composition information and the multiple local images corresponding to the multiple local viewpoints includes: synthesizing the multiple local images corresponding to the multiple local viewpoints according to the composition information to obtain the preview screen. For example, multiple local images are stitched together according to the display position and display size of the multiple local images corresponding to the multiple local viewpoints indicated by the composition information to obtain the preview screen.

[0088] The preview image can include multiple local perspectives from the panoramic image. Specifically, the preview image can be generated by combining multiple local perspectives from the panoramic image using a different compositional method than the panoramic image itself. Alternatively, the preview image can be generated by reprocessing and combining multiple local perspectives from the panoramic image. In particular, the display positions of the multiple images corresponding to the multiple local perspectives in the preview image can differ from their display positions in the original panoramic image.

[0089] For example, Figure 3a shows a schematic diagram of the first panoramic image, Figure 3b shows that the subjects in the first panoramic image include an elephant and a person, Figure 3c shows a schematic diagram of some local perspectives corresponding to the elephant, and Figure 3d shows a schematic diagram of some local perspectives corresponding to the person.

[0090] Figure 3e shows a schematic diagram of a preview screen in one embodiment. This preview screen includes a partial image corresponding to a local viewpoint of the elephant in Figure 3c, and a partial image corresponding to a local viewpoint of the person in Figure 3d. Figure 3e includes viewpoints from both Figures 3d and 3b, but the display positions of Figures 3d and 3b in the original panoramic image differ from their positions in the preview screen. Therefore, during the preview process, multiple local viewpoints of interest can be displayed by showing Figure 3e instead of the entire panoramic image, reducing the difficulty in distinguishing individual local viewpoints caused by displaying the entire panoramic image.

[0091] In this way, before the target video is captured, the perspective change information of multiple local images from the panoramic video is determined based on the composition information set by the user. The user can simultaneously preview multiple local images from different perspectives, and can also preview the positional relationship and display size of different local images, i.e., preview the composition effect. The images from different perspectives are generated from the panoramic image, and the multiple different perspectives can be adjusted according to the composition information input by the user before shooting, so that the user can adjust the composition before shooting.

[0092] Step S140: In response to the user's video shooting command, output a target video with the same composition information as the preview screen, wherein the target video is obtained through the second panoramic image acquired by the panoramic shooting device.

[0093] The target video may contain multiple frames of target images, and each target image contains multiple local images corresponding to multiple local viewpoints indicated by composition information. In some embodiments, the composition information corresponding to the preview screen may be the same as the composition information corresponding to the target image. Specifically, the number, display position, and display size of the multiple local viewpoints in the preview screen may be the same as those in the target image. In some embodiments, after a video recording command is given, the preview screen may display the target image in real time.

[0094] The difference between the first panoramic image and the second panoramic image is that the first panoramic image is the panoramic image acquired by the panoramic shooting device before the user issues a video shooting command, while the second panoramic image is the panoramic image acquired by the panoramic shooting device in response to the user's video shooting command. Both can be acquired in the same way by the panoramic shooting device.

[0095] In some implementations, before receiving a user's video recording command, the video recording device may cache only the first panoramic image to generate a preview image based on the first panoramic image, reducing the resource consumption caused by storing the first panoramic image. Upon receiving the user's video recording command, the video recording device may also output a video formed from the second panoramic image simultaneously with the target video, so as to create a comparison with the target video, and / or to perform post-editing processing on the generated target video based on the video formed from the second panoramic image.

[0096] In some embodiments, the video capturing device can respond to a video capturing command by first obtaining a second panoramic image through exposure and / or encoding processing, and then performing processing such as cropping and perspective transformation on the second panoramic image to obtain multiple target images in the target video. In some embodiments, the video capturing device can respond to a video capturing command by dividing the second panoramic image into two paths after exposure for processing. One path is stored after encoding, and the other path is processed through one or more operations such as cropping, perspective transformation, stitching, and encoding to obtain multiple target images in the target video. This can improve the efficiency of target video acquisition.

[0097] In some embodiments, the video recording device may be equipped with physical controls, which are used to generate the video recording command in response to a user's preset operation. The physical controls include at least one of a display interface, a joystick, a button, or a dial. The user can generate the video recording command by operating the physical controls, which is not limited here.

[0098] In some implementations, the step of outputting a target video with the same composition information as the preview screen in response to the user's video shooting command includes: in response to the user's video shooting command, storing multiple target images corresponding to the preview screen in chronological order to obtain the target video.

[0099] Optionally, the frame rate of the target video can be the same as or different from the frame rate of the panoramic image acquired by the panoramic shooting device. For example, frame extraction can make the frame rate of the target video lower than the frame rate of the panoramic image, or frame interpolation can make the frame rate of the target video higher than the frame rate of the panoramic image.

[0100] In some embodiments, the method further includes: terminating the output of the target video in response to a user's command to end recording. Alternatively, upon ending recording, a video export control may be displayed, and the target video may be stored in response to a user's operation on the video export control. Or, upon ending recording, a video upload control may be displayed, and the target video may be uploaded to a specified location in response to a user's operation on the video upload control.

[0101] For example, during video recording, such as after receiving a video recording instruction and before receiving a recording stop instruction, multiple target images corresponding to the target video are acquired based on multiple local viewpoints in the panoramic image indicated by the composition information. The target images include multiple local images corresponding to the multiple local viewpoints in the second panoramic image. This ensures that the composition of the target video corresponds to the composition of the preview screen, resulting in a target video with the same compositional effect.

[0102] For example, the video shooting method of this application embodiment can be applied to a panoramic shooting device. The panoramic shooting device responds to the user's video shooting command, generates a target image corresponding to the preview screen based on the currently acquired panoramic image and the composition information, and stores multiple frames of target images in chronological order to obtain the target video; the panoramic shooting device can also display the target video in real time.

[0103] For example, the video shooting method of this application embodiment is applied to a terminal device that can communicate with a panoramic shooting device. The terminal device acquires a panoramic image through the panoramic shooting device, and in response to the user's video shooting command, generates a target image corresponding to the preview screen based on the currently acquired panoramic image and the composition information, and stores multiple frames of target images in chronological order to obtain the target video; the terminal device can also display the target video in real time.

[0104] In some implementations, step S140 can be replaced by: responding to a user's video shooting command by outputting a second panoramic image and composition information corresponding to the preview image. For example, the panoramic shooting device can output the acquired second panoramic image and the composition information corresponding to the preview image to a terminal device, so that the terminal device can process the second panoramic image according to the composition information to generate a target video, thereby reducing the computing power requirements of the panoramic shooting device.

[0105] In some embodiments, the method further includes: sending a second panoramic image acquired by the panoramic shooting device and the composition information to a target device, so that the target device generates a target video based on the second panoramic image and the composition information. For example, the target device includes a terminal device and / or a server device. For instance, the terminal device may include at least one of a mobile phone, tablet computer, laptop computer, desktop computer, personal digital assistant, wearable device, remote control, etc. The server device may include a standalone server or a server cluster.

[0106] For example, at least some steps of the video shooting method in this application embodiment are applied to a terminal device and / or a server device capable of communicating with a panoramic shooting device. For instance, the server device can acquire the panoramic image currently captured by the panoramic shooting device and acquire the composition information, and generate a target image corresponding to the preview screen based on the currently acquired panoramic image and the composition information, storing multiple frames of the target image in chronological order to obtain the target video. The composition information and / or the video shooting instructions can be determined based on user operations on the panoramic shooting device or output to a terminal device capable of communicating with the panoramic shooting device, or can be determined by the server device itself. Optionally, the server device can also output the target video to the panoramic shooting device or to a terminal device capable of communicating with the panoramic shooting device. Optionally, the server device can also store and / or distribute the target video, wherein distributing the target video from the server device to terminal devices such as mobile phones and tablets can include live streaming or on-demand playback.

[0107] In some implementations, the panoramic shooting device and / or the terminal device and / or server device capable of communicating with the panoramic shooting device may be referred to as a video shooting device.

[0108] In some implementations, the user's video recording command can be triggered by the user pressing the video recording button or by the user pressing the live video streaming button. When the video recording button is pressed, outputting the target video may include storing the target video in the local memory of the panoramic shooting device. When the live video streaming button is pressed, outputting the target video may include sending the target video to the server in real time. Specifically, the live video streaming button or the video recording button can be a virtual button on the display screen of the video shooting device or a physical button on the video shooting device, and is not limited thereto.

[0109] In some implementations, users can also end the streaming of the target video or complete the storage of the target video by ending the shooting command.

[0110] In some implementations, the step of outputting a target video with the same composition information as the preview screen in response to a user's video shooting command includes: in response to a user's video shooting command, sending multiple target images corresponding to the preview screen to a target device in chronological order, so that the target device obtains the target video.

[0111] For example, the video shooting method of this application embodiment is applied to a panoramic shooting device or a terminal device capable of communicating with the panoramic shooting device. In response to a user's video shooting command, the panoramic shooting device or the terminal device generates a target image corresponding to the preview screen based on the panoramic image currently acquired by the panoramic shooting device and the composition information. Multiple frames of the target image corresponding to the preview screen are sent to the target device in chronological order, so that the target device obtains the target video. Optionally, for example, the panoramic shooting device or the terminal device can send the multiple frames of the target image corresponding to the preview screen to the target device via real-time network streaming. For example, the panoramic shooting device can upload the multiple frames of the target image corresponding to the preview screen to a server device, which stores and / or distributes the target video. The server device distributing the target video to terminal devices such as mobile phones and tablets can include live streaming or on-demand playback.

[0112] It should be understood that the embodiments of this application can output target videos containing multiple local perspectives in real time, so that in live streaming scenarios, users can intuitively view multiple local perspectives of interest.

[0113] It should be noted that both the first panoramic image and the second panoramic image are panoramic images acquired through a panoramic shooting device. For ease of explanation, the panoramic image acquired before the video shooting command is received is referred to as the first panoramic image, and the panoramic image acquired after the video shooting command is received is referred to as the second panoramic image.

[0114] In some implementations, acquiring composition information includes at least one of the following: automatically determining the composition information by a video shooting device such as a panoramic shooting device, terminal device, or server device before outputting the preview screen; determining the composition information based on user input before or during outputting the preview screen; and adjusting the composition information based on user input when outputting a target video with the same composition information as the preview screen. The steps for acquiring composition information will be described below.

[0115] In some implementations, the field of view (FOV) indicated in the composition information may correspond to a target object. The target object may be the main subject in the panoramic image. In some implementations, the target object is determined by the panoramic shooting device, automatically by the target device, or by the user. The user-determined target object is a specific content or point of interest that the user wishes to capture.

[0116] For example, the field of view (FOV) corresponding to at least one of the multiple local viewpoints in the composition information is automatically determined by the panoramic shooting device or a target device capable of communicating with the panoramic shooting device. For instance, the panoramic shooting device can automatically determine the FOV corresponding to at least one local viewpoint in the panoramic image acquired by the panoramic shooting device. This can be achieved by performing semantic segmentation on the panoramic image and determining the FOV corresponding to at least one local viewpoint based on the semantic segmentation results. This can improve the intelligence level of the video shooting device and reduce the difficulty and workload for users.

[0117] For example, the FOV corresponding to the at least one local viewpoint is determined based on the identification results of target objects in the first panoramic image and / or the second panoramic image. Target objects may include movable targets or stationary targets. Movable targets include people, vehicles, ships, aircraft, etc., while stationary targets include buildings, geographical landscapes, etc.

[0118] Figure 4a shows a schematic diagram of the global image, and Figure 4b shows a schematic diagram of the preview screen or the target image corresponding to the preview screen. The target objects included in the global image can be two people, so the number of local viewpoints in the composition information is two; the FOV corresponding to the two local viewpoints are the areas where the two people are located, and the size of the FOV can be determined based on the upper body of the person or based on the user's operation.

[0119] For example, the panoramic shooting device can automatically identify target objects in the first panoramic image and / or the second panoramic image. This includes identifying specific types of objects as target objects, such as target users or vehicles with preset characteristics. Alternatively, the panoramic shooting device can identify preset objects at preset locations as target objects. Identifying target objects in a panoramic image, compared to identifying them in a localized image from a limited perspective, increases the probability of target object detection and prevents the target video from missing information about the target object when it is removed from the localized image.

[0120] For example, the method further includes: obtaining scaling information of the local viewpoint, the scaling information being determined based on the user's scaling operation on the target image in the preview screen and / or the target video; and adjusting the FOV size corresponding to the local viewpoint according to the scaling information.

[0121] Optionally, by adjusting the FOV size of the local viewpoint corresponding to one of the local images based on the user's zoom operation on that local image, the focal length of that local viewpoint can be adjusted; alternatively, the FOV size of at least one other local image corresponding to a local viewpoint can be adjusted to adjust the aspect ratio of different local viewpoints. Optionally, the zoom operation may include at least one of the following: two-finger operation, double-tap operation, or operation on a preset control.

[0122] For example, when displaying the preview screen, the user can zoom in and out on a local image to adjust the FOV (Field of View) of that local image as indicated by the composition information. During video recording, the user can also zoom in and out on a local image within the target video to adjust the FOV of that local image as indicated by the composition information, thereby adjusting the composition of the target video in real time.

[0123] For example, the method further includes: acquiring orientation information of the support component of the panoramic shooting device; and determining the target object in the first panoramic image and / or the second panoramic image based on the orientation information, wherein the orientation of the target object in the first panoramic image and / or the second panoramic image is determined based on the orientation information. For instance, the support component is a selfie stick. Referring to Figure 3d in conjunction with Figure 3b, the user of the panoramic shooting device holds a selfie stick, and the panoramic image acquired by the panoramic shooting device includes the selfie stick. Based on the orientation information of the selfie stick in the panoramic image, the orientation of the target object in the panoramic image can be determined, thereby determining the FOV corresponding to the local viewpoint where the target object is located.

[0124] Optionally, the local viewpoint corresponding to the target object changes with the movement of the target object, so that the target object is always located within the local viewpoint. For example, the target object can be the target being followed by the panoramic shooting device. In response to the change in the position of the target being followed in the panoramic image, the FOV corresponding to the local viewpoint is adjusted, so that the target being followed remains in the local image corresponding to the local viewpoint, thereby achieving the following display of the target being followed.

[0125] In some implementations, acquiring the composition information includes: acquiring the composition information in response to a user's composition input operation on the panoramic shooting device or on a device communicatively connected to the panoramic shooting device. This means the composition information can be manually determined by the user, offering high flexibility and meeting diverse compositional needs.

[0126] For example, the composition input operation includes: a range selection operation on at least a portion of the first panoramic image, and / or a target object selection operation on at least a portion of the first panoramic image.

[0127] For example, a panoramic shooting device or a device connected to a panoramic shooting device can display at least a portion of the panoramic image. While displaying a portion of the panoramic image, the user can switch the displayed area to select a local viewpoint and / or target object within the entire panoramic image. As shown in Figure 3b, the user's target object selection operation on the panoramic image determines elephants and human figures as target objects; or as shown in Figure 3c or 3d, the user's range selection operation on a portion of the panoramic image determines the size of the local viewpoint range. Different sizes of the local viewpoint range can correspond to different focal lengths for that local viewpoint, achieving virtual focal length settings.

[0128] Optionally, the multiple rectangles in Figure 3c or Figure 3d represent multiple ranges of local viewpoints automatically determined by the panoramic shooting device. The range of the local viewpoint can be determined based on the user's selection operation on the rectangles. However, this is not the only possibility; for example, the range selection operation can include the user's selection operation on at least a portion of the panoramic image, and the range of the local viewpoint can be determined by this selection operation.

[0129] In some embodiments, the method further includes: displaying a current composition layout, the current composition layout including multiple image positions.

[0130] For example, the current composition layout is selected by the user from a set of preset layouts, including split-screen and / or picture-in-picture layouts. However, this is not the only possibility. For instance, the current composition layout can be automatically determined by the panoramic shooting device or a target device capable of communicating with it. Optionally, the current composition layout can be determined based on the scene being captured in the video and / or objects identified in the first panoramic image.

[0131] Optionally, the composition layout shown in Figure 3e includes two image positions, one of which is the entire screen and the other is the upper right corner; the composition layout shown in Figure 4b includes the image positions on the left and right; the composition layout shown in Figure 5 includes the entire screen as well as the upper left and upper right corners; the composition layout shown in Figure 6 includes the entire screen as well as the four positions on the left; and the composition layout shown in Figure 7 includes the upper left, upper right, lower left, and lower right corners.

[0132] The composition layouts shown in Figures 3e, 5, and 6 can be called picture-in-picture compositions. The partial image displayed in the entire screen can be called the main screen, the partial viewpoint corresponding to the main screen can be called the main viewpoint, and other partial images can be called sub-screens. The main screen is used to present the core viewpoint, and the sub-screens are used to present secondary viewpoints. Optionally, picture-in-picture compositions can be applied to live streaming, such as using the partial image corresponding to the broadcaster as the main screen and the partial image corresponding to the subject as a sub-screen. Alternatively, picture-in-picture compositions can be applied to scenes such as cycling and skiing that require first-person and second-person views. For example, the first-person view can be used as the main screen, and the second-person view can be used as a sub-screen.

[0133] The compositions shown in Figures 4b and 7 can be called split-screen compositions. Figure 4b can be called a two-screen split-screen composition, and Figure 7 can be called a four-screen split-screen composition. Of course, split-screen compositions can also include three-screen and five-screen split-screen compositions. The different local images in a split-screen composition can be the same or different sizes. The local images in a split-screen composition can be arranged horizontally and / or vertically, or along a diagonal. For example, a two-screen split-screen composition can be used in interview scenarios, while a three-screen or four-screen split-screen composition can be used in recording videos, presenting multiple angles of the scene.

[0134] For example, obtaining composition information includes: determining the positional relationship between the plurality of local images indicated by the composition information based on the local images at each of the image positions in the current composition layout.

[0135] For example, a panoramic shooting device or a device corresponding to a panoramic shooting device can display the current composition layout. The user can move a local image or target object determined in the first panoramic image to the image position in the current composition layout to display a local image with the corresponding local perspective at that image position; or the user can select an image position in the current image layout and specify a local image or target object at that image position in the first panoramic image, or after selecting an image position, specify the type, orientation, and other information of the target object at that image position.

[0136] Optionally, the method further includes: determining the display positions of the multiple local images corresponding to the multiple local viewpoints based on the user's position adjustment operation on at least one of the local images. For example, when displaying the preview image, the user can adjust the position of at least one of the local images, such as dragging the local image displayed on the left side of the preview image to the right side, and adjusting the local image displayed on the right side to the left side. This allows the composition of the target video to more accurately meet the user's composition needs.

[0137] For example, in response to a user's video recording command, a target video with the same composition information as the preview screen is displayed, wherein the layout of local images in the target video is determined according to the composition information.

[0138] In some implementations, the positional relationships between the plurality of local images indicated by the composition information are automatically determined by the panoramic shooting device or a target device capable of communicating with the panoramic shooting device. This can improve the intelligence level of the video shooting device and reduce the difficulty and workload for users.

[0139] For example, the positional relationship between the plurality of local images indicated by the composition information is determined based on the positional relationship of the plurality of local viewpoints in the first panoramic image. As shown in Figures 4a and 4b, if the positional relationship of two people in the first panoramic image is one on the left and one on the right, then the positional relationship of the two local images in the preview image and the corresponding target image is also one on the left and one on the right. This makes the positional relationship of objects in the preview image and the target image the same as the positional relationship in the actual scene, making the target video more consistent with the actual situation.

[0140] For example, the positional relationship between the multiple local images indicated by the composition information is determined based on the type of target object corresponding to the local viewpoint. As shown in Figure 3e, the image position of the local viewpoint corresponding to a wild animal target object is the entire screen, while the image position of the local viewpoint corresponding to a human is the upper right corner of the screen. The local viewpoint corresponding to the target object of greater interest can be presented as the core viewpoint. As shown in Figure 4b, when the target objects corresponding to multiple local viewpoints are of the same type, the positional relationship between the multiple local images can be determined by split-screen composition.

[0141] In some embodiments, the method further includes: acquiring preset orientation information in the first panoramic image; wherein, multiple local viewpoints in the preview screen are arranged according to the orientation indicated by the preset orientation information.

[0142] For example, the preset directional information indicates: East, South, West, North, or indicates front, back, left, right, up, down. Of course, it is not limited to these; for example, the preset directional information indicates Southeast, Northeast, Southwest, Northwest, or indicates Left Front, Right Front, Left Back, Right Back. As shown in Figure 6, the four local viewpoints of the four image positions on the left are the viewpoints of the four preset directional positions: East, South, West, and North. As shown in Figure 7, the four local viewpoints of the four image positions—the upper left corner, the upper right corner, the lower left corner, and the lower right corner—are the viewpoints of the four preset directional positions: front, back, left, and right of the panoramic shooting device.

[0143] Optionally, the preset orientation information is determined based on the panoramic shooting device as the center. For example, the preset orientations of east, south, west, and north are centered on the panoramic shooting device, and the orientations of front, back, left, right, up, and down are also based on the panoramic shooting device. This allows users to have an immersive experience when viewing the preview and target video.

[0144] For example, the preset orientation information is determined based on user input, and / or, the preset orientation information is acquired based on one or more sensors of the panoramic shooting device. For example, based on the north orientation input by the user, the preset orientations of east, south, west, and north are automatically determined, or based on the forward orientation set by the user, the preset orientations of front, back, left, right, up, and down are automatically determined. For example, the panoramic shooting device is equipped with an attitude sensor, which can determine the preset orientations of front, back, left, right, up, and down based on the data from the attitude sensor; or the panoramic shooting device is equipped with an electronic compass, which can determine the preset orientations of east, south, west, and north based on the data from the electronic compass.

[0145] In some embodiments, the method further includes: acquiring composition information input by a user during the acquisition of the target video; and, based on the composition information, displaying at least a plurality of local viewpoints in at least a portion of the panoramic image before obtaining the target video, wherein changes in the plurality of local viewpoints in at least a portion of the panoramic image are associated with the composition information. For example, the composition information can be adjusted before the shooting ends, and the local images corresponding to the local viewpoints in the target image of the target video can be adjusted in response to changes in the composition information.

[0146] For example, during video recording, users can adjust the composition of the target video in real time. During the acquisition of the target video, composition information can be adjusted based on user input. For instance, when displaying the target image in real time, the number of local viewpoints can be adjusted based on user input, or the FOV and / or FOV size corresponding to at least one of the local viewpoints can be adjusted, or the display position and / or display size of at least one of the local images can be adjusted.

[0147] It should be understood that the user input mentioned above can be input operations to the panoramic shooting device, such as input operations to the display screen 12 or physical buttons 13 shown in Figure 2; the user input can also be input operations to a terminal device that is communicatively connected to the panoramic shooting device. Input operations include, but are not limited to, touch operations (long press, short press, drag, etc.) and non-touch operations (gestures, voice, etc.), which will not be listed here.

[0148] During video shooting, multiple partial views of the panoramic image can be displayed based on the composition information input by the user, and these partial views can be adjusted according to the user input, reducing the complexity of post-editing the original panoramic video.

[0149] For example, during the process of outputting a target video with the same composition information as the preview image, the composition information input by the user and the time information corresponding to the composition information can be obtained; and based on the composition information and the time information, the composition of the multiple local viewpoints in the target image corresponding to the time information can be adjusted according to the composition information. The composition layout of the target video can be adjusted according to the user's operation, and the subsequently acquired second panoramic image can be processed according to the adjusted layout to update the composition layout of the subsequently obtained target image.

[0150] In some implementations, when the panoramic image is obtained by processing images acquired by at least two visual sensors, among the multiple local images corresponding to the multiple local viewpoints, at least one local image corresponding to a local viewpoint is acquired by a single visual sensor, and at least another local image corresponding to a local viewpoint is obtained by cropping images acquired by the two visual sensors.

[0151] In some embodiments, when the panoramic image is obtained by image processing from at least two visual sensors, the plurality of local perspectives in the first panoramic image include a first perspective and a second perspective, wherein the first perspective is a local perspective of one of the visual sensors, and the second perspective includes local perspectives from at least two of the visual sensors.

[0152] For example, there is at least one local viewpoint where the position of the target object relative to the panoramic shooting device is relatively fixed. The target object is always within the viewpoint of one of the visual sensors of the panoramic shooting device. For example, if the panoramic shooting device is fixed on a bicycle, the cyclist is always within the viewpoint of one of the visual sensors. In this case, the local image corresponding to the target object can be directly obtained from the image acquired by the visual sensor. It is not necessary to synthesize the images acquired by multiple visual sensors of the panoramic shooting device into a panoramic image and then obtain the local image corresponding to the target object from the panoramic image, which can improve the real-time performance of the local image display.

[0153] At least one other local viewpoint corresponds to a change in the positional relationship of the target object relative to the panoramic shooting device, or the target object is located in the edge region of the viewpoints of the two visual sensors. By cropping the images obtained by the two visual sensors to obtain a local image of the target object, the target object can be prevented from being lost, and / or the quality of the local image can be improved, for example, the distortion of the local image can be reduced.

[0154] The video shooting method provided in this application includes: acquiring a first panoramic image using a panoramic shooting device; acquiring composition information, wherein the composition information indicates the composition of multiple local perspectives in the first panoramic image, and the composition information of the multiple local perspectives can be adjusted based on user input; outputting a preview screen based on the first panoramic image and the composition information, wherein the preview screen includes multiple local images corresponding to the multiple local perspectives in the first panoramic image; and outputting a target video with the same composition information as the preview screen in response to a user's video shooting command, wherein the target video is obtained through a second panoramic image acquired by the panoramic shooting device. Before outputting the video, the user can simultaneously preview and display images with multiple different local perspectives. The images with different local perspectives are generated from the panoramic image, and the multiple different perspectives can be adjusted according to the composition information input by the user before shooting, so that the user can adjust the composition and preview the composition effect before shooting, thereby enabling the target video with the preview effect obtained from the video shooting to be output in real time without post-editing.

[0155] In some implementations, embodiments of this application can enable a video shooting device to directly output a target video including multiple partial images. For example, through a single video capture by a panoramic camera, a virtual multi-camera target video can be directly output, achieving the output effect of multiple cameras in the same frame. This meets the user's expectations for using a panoramic camera and eliminates the need for additional post-processing of the panoramic image to obtain the virtual multi-camera target video. Optionally, the virtual multi-camera supports adjustment of partial viewpoints or the definition of virtual lenses. The target object of the virtual lens can be assigned by the user or identified and recommended by the video shooting device. It can also support adjustment of the virtual focal length, adjusting the distance and / or size of the partial viewpoint. Optionally, the virtual multi-camera supports adjustment of the current composition layout, such as split-screen composition and / or picture-in-picture composition.

[0156] In some implementations, by sending the second panoramic image acquired by the panoramic shooting device and the composition information to the target device, the target device can generate a target video based on the second panoramic image and the composition information. Optionally, the target device can also adjust the composition information; that is, after obtaining the target video through virtual multi-camera shooting, the target device can also analyze and edit the target video.

[0157] In some implementations, in response to a user's video shooting command, multiple target images corresponding to the preview screen are sent to the target device in chronological order, which can realize the function support of virtual multi-camera live streaming link.

[0158] Panoramic shooting equipment has the advantage of wide-angle and / or multiple lenses, which can simultaneously capture multiple perspectives. After the image content is reconstructed and merged, it can output a single image that meets the content, aesthetics, composition, and user requirements, achieving multi-view simultaneous / picture-in-picture effects. Panoramic shooting equipment can also achieve simultaneous multi-camera effects, meeting various needs such as interviews and recording.

[0159] When multiple target objects that need to be presented simultaneously appear in a panoramic image, and the distance between different target objects is large, the target video output by this application embodiment can also include partial images of multiple target objects at the same time, without discarding some screen elements; when there is a need to record multiple targets in video shooting such as multi-person interviews, this application embodiment can maintain a stable screen composition, with better and more continuous viewing experience. In terms of output presentation to users, it can achieve small distortion and natural output from multiple perspectives.

[0160] Compared to traditional multi-camera shooting, processing panoramic images acquired through panoramic shooting equipment to output the target video allows for the achievement of multiple camera positions in a single shot, significantly reducing the number of hardware components and the complexity of the shooting process. Compared to traditional post-processing methods, this embodiment significantly reduces operational complexity, eliminating the need for complex post-production workflows and skills, allowing users to directly output multi-camera target videos, greatly simplifying the operation. Compared to traditional multi-camera and control panel solutions for live streaming, this embodiment significantly reduces the complexity of multi-camera live streaming, enabling users to easily and conveniently conduct multi-camera live streams, greatly improving the user experience.

[0161] Please refer to Figure 8 in conjunction with the foregoing embodiments. Figure 8 is a flowchart illustrating a video shooting method provided in another embodiment of this application.

[0162] As shown in Figure 8, the video shooting method includes steps S210 to S240.

[0163] Step S210: In response to the user's video shooting command, acquire a panoramic image through the panoramic shooting device;

[0164] Step S220: Obtain a target video based on the panoramic image, wherein the target video is obtained by processing at least a portion of the panoramic image;

[0165] Step S230: During the process of acquiring the target video, the composition information input by the user is acquired, wherein the composition information indicates the composition of multiple local viewpoints in at least part of the panoramic image;

[0166] Step S240: Based on the composition information, before obtaining the target video, display at least a portion of multiple local perspectives in the panoramic image, wherein the changes in the multiple local perspectives in at least a portion of the panoramic image are associated with the composition information.

[0167] The difference between the embodiments corresponding to steps S210 to S240 and those corresponding to steps S110 to S140 is that steps S210 to S240 are all executed by the panoramic shooting device in response to the user's video shooting command. The specific implementation methods can be found in the above method embodiments, and will not be repeated here to avoid repetition. The panoramic image in step S210 can be the second panoramic image in step S140.

[0168] In some implementations, the above method may further include: in response to a user's command to end recording, ending the recording of the target video and obtaining the target video.

[0169] In some implementations, at least a portion of the panoramic image may be a portion of a multi-frame panoramic image.

[0170] In this embodiment of the application, during video shooting, the video shooting device can display multiple partial perspectives of the panoramic image based on the composition information input by the user. These multiple partial perspectives can be adjusted according to the user input, and the adjusted shooting effect can be displayed, enriching the video output format of the video shooting device. Therefore, the user can adjust the composition of the target video in real time during shooting, reducing the complexity of post-editing the original panoramic video and improving the efficiency of film production.

[0171] In some implementations, the step of acquiring a target video based on the panoramic image in response to a user's video shooting command includes: during video shooting, acquiring multiple target images corresponding to the target video based on multiple local viewpoints in the panoramic image indicated by the composition information, wherein the target images include multiple local images corresponding to the multiple local viewpoints in the panoramic image. For example, in response to a user's video shooting command, the multiple target images are stored in chronological order to obtain the target video.

[0172] For example, in the process of acquiring the target video, acquiring the composition information input by the user includes: acquiring the composition information input by the user and the time information corresponding to the composition information during the acquisition of the target video; displaying multiple local viewpoints in the panoramic image based on the composition information includes: adjusting the composition of the multiple local viewpoints in the target image corresponding to the time information according to the composition information based on the composition information and the time information. The composition layout of the target video can be adjusted according to user operations, and the subsequently acquired panoramic images can be processed according to the adjusted layout to update the composition layout of the subsequently obtained target images.

[0173] In some embodiments, acquiring a panoramic image via a panoramic imaging device includes: acquiring the panoramic image via a single visual sensor, or obtaining the panoramic image through image processing obtained from at least two visual sensors.

[0174] For example, the at least two vision sensors include two vision sensors arranged back to back.

[0175] For example, when the panoramic image is obtained by processing images obtained by at least two visual sensors, among the multiple local images corresponding to the multiple local viewpoints, at least one local image corresponding to a local viewpoint is obtained by a single visual sensor, and at least another local image corresponding to a local viewpoint is obtained by cropping images obtained by the two visual sensors.

[0176] For example, when the panoramic image is obtained by image processing from at least two visual sensors, the plurality of local viewpoints in the panoramic image include a first viewpoint and a second viewpoint, wherein the first viewpoint is a local viewpoint of one of the visual sensors, and the second viewpoint includes local viewpoints from at least two of the visual sensors.

[0177] For example, the visual sensor may be the main visual sensor of the panoramic shooting device, or the visual sensor may not be the perceptual visual sensor of the panoramic shooting device.

[0178] In some implementations, the mapping information is used to indicate at least one of the following:

[0179] The number of the multiple local perspectives;

[0180] The FOVs corresponding to the multiple local perspectives;

[0181] The FOV size corresponding to the multiple local viewpoints;

[0182] The display positions of the multiple local images corresponding to the multiple local viewpoints;

[0183] The display size of the multiple local images corresponding to the multiple local viewpoints.

[0184] For example, the FOV corresponding to at least one of the multiple local perspectives is automatically determined by the panoramic shooting device.

[0185] For example, the FOV corresponding to the at least one local viewpoint is determined based on the recognition result of the target object in the panoramic image.

[0186] For example, the method further includes:

[0187] Obtain the orientation information of the support components of the panoramic shooting device;

[0188] Based on the orientation information, the target object in the panoramic image is determined, wherein the orientation of the target object in the panoramic image is determined based on the orientation information.

[0189] For example, the local viewpoint corresponding to the target object changes with the movement of the target object, so that the target object is always located in the local viewpoint.

[0190] For example, obtaining the composition information includes:

[0191] The composition information is acquired in response to a user's composition input operation on the panoramic shooting device or on a device communicatively connected to the panoramic shooting device.

[0192] For example, the composition input operation includes:

[0193] Range selection operation on at least a portion of the panoramic image, and / or target object selection operation on at least a portion of the panoramic image.

[0194] For example, the method further includes:

[0195] Obtain the scaling information of the local viewpoint, the scaling information being determined based on the user's scaling operation on the target image in the preview screen and / or the target video;

[0196] Based on the scaling information, adjust the FOV size corresponding to the local viewpoint.

[0197] For example, the method further includes:

[0198] The display positions of the multiple local images corresponding to the multiple local viewpoints are determined based on the user's position adjustment operation on at least one of the local images.

[0199] For example, the method further includes: displaying a current composition layout, the current composition layout including multiple image positions;

[0200] The step of obtaining composition information includes: determining the positional relationship between the plurality of local images indicated by the composition information based on the local images at each of the image positions in the current composition layout.

[0201] For example, the current composition layout is selected by the user from a plurality of preset composition layouts, including split-screen composition and / or picture-in-picture composition.

[0202] For example, the positional relationship between the plurality of local images indicated by the composition information is automatically determined by the panoramic shooting device or a target device capable of communicating with the panoramic shooting device.

[0203] For example, the positional relationship between the plurality of local images indicated by the composition information is determined based on the positional relationship of the plurality of local viewpoints in the panoramic image.

[0204] For example, the positional relationship between the plurality of local images indicated by the composition information is determined according to the type of target object corresponding to the local viewpoint.

[0205] For example, the method further includes: acquiring preset orientation information in the panoramic image;

[0206] In the preview screen, multiple local perspectives are arranged according to the directions indicated by the preset orientation information.

[0207] For example, the preset directional information indicates: east, south, west, north, or indicates front, back, left, right, up, down.

[0208] For example, the preset orientation information is determined based on the panoramic shooting device as the center.

[0209] For example, the preset orientation information is determined based on user input, and / or the preset orientation information is acquired based on one or more sensors of the panoramic shooting device.

[0210] For example, the method further includes:

[0211] The second panoramic image acquired by the panoramic shooting device and the composition information are sent to the target device, so that the target device can generate a target video based on the second panoramic image and the composition information.

[0212] For example, the panoramic image includes a panoramic image with a horizontal range of at least 180 degrees.

[0213] For example, the panoramic image includes a horizontal 360-degree panoramic image.

[0214] For example, the panoramic image includes a panoramic image with a vertical angle of at least 180 degrees.

[0215] For example, the panoramic image includes a vertical 360-degree panoramic image.

[0216] For example, the panoramic image includes a panoramic image with a horizontal degree of at least 180 degrees and a vertical degree of at least 180 degrees.

[0217] For example, the panoramic image includes a panoramic image with 360 degrees horizontally and 360 degrees vertically.

[0218] For example, the panoramic shooting device is a panoramic camera or a mobile platform.

[0219] For example, when the panoramic shooting device is a panoramic camera, the panoramic image is acquired by a single visual sensor or two visual sensors arranged in opposite directions of the panoramic camera.

[0220] When the panoramic shooting device is a mobile platform, the panoramic image is acquired by a visual sensor that is not a perception sensor in the mobile platform.

[0221] For example, the step of displaying at least a portion of multiple local perspectives in the panoramic image before obtaining the target video based on the composition information includes: cropping the panoramic image according to the composition information to obtain multiple local images corresponding to the multiple local perspectives; and displaying the multiple local images corresponding to the multiple local perspectives according to the composition indicated by the composition information.

[0222] For example, displaying multiple local images corresponding to the multiple local viewpoints according to the composition indicated by the composition information includes:

[0223] The multiple local images corresponding to the multiple local viewpoints are synthesized according to the composition information, and the synthesized target image is displayed.

[0224] For example, acquiring the target video based on the panoramic image includes:

[0225] The synthesized target images are stored in chronological order to obtain the target video;

[0226] or,

[0227] The synthesized multiple frames of the target image are sent to the target device in chronological order so that the target device can obtain the target video.

[0228] For example, the target device includes a terminal device and / or a server device.

[0229] The specific principles and some implementation methods of the video shooting method provided in this application embodiment are similar to those of the video shooting method in the foregoing embodiment, and will not be repeated here.

[0230] Referring to Figure 9, this application also provides a video generation system. In some embodiments, the video generation system may include one or more of a panoramic shooting device 10, a terminal device 20, and a server device 30. Any of the panoramic shooting device 10, the terminal device 20, and the server device 30 can independently execute the method embodiment described in steps 110-140 above or the method embodiment described in steps 210-240 above.

[0231] In some implementations, the video generation system may include a panoramic shooting device 10 and a terminal device 20, which cooperate to implement the method embodiments described in steps 110-140 above or the method embodiments described in steps 210-240 above. For example, the terminal device 20 may display a preview screen before video shooting, and / or multiple partial perspectives during video shooting; the terminal device 20 may also generate composition information in response to user input and send the composition information to the panoramic shooting device 10.

[0232] In some embodiments, the video generation system may also include a panoramic shooting device 10 and a server device 30. The server device 30 can be used to receive the target video output by the panoramic shooting device 10 and send it to other terminal devices connected to the server device 30 for display. Of course, in some embodiments, the video generation system may also include a panoramic shooting device 10, a terminal device 20, and a server device 30 simultaneously, depending on the actual scenario.

[0233] Please refer to FIG10 in conjunction with the above embodiments. FIG10 is a schematic block diagram of the video shooting device 600 provided in the embodiments of this application.

[0234] The video shooting device 600 may include at least one of the following: a panoramic shooting device, a terminal device, or a server device. The terminal device or server device is capable of communicating with the panoramic shooting device to acquire panoramic images, at least through the panoramic shooting device.

[0235] The video recording device 600 includes one or more processors 601, which operate individually or collectively to implement the steps of the aforementioned video recording method.

[0236] For example, the video recording device 600 may also include a memory 602 for storing computer programs.

[0237] For example, processor 601 and memory 602 are connected via bus 603, such as an I2C (Inter-integrated Circuit) bus.

[0238] Specifically, the processor 601 can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0239] Specifically, the memory 602 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0240] The processor 601 is used to execute a computer program stored in the memory 602, and to implement the steps of the aforementioned video shooting method when executing the computer program.

[0241] In some embodiments, the processor 601 is configured to execute a computer program stored in the memory 602, and, when executing the computer program, perform the following steps:

[0242] Acquire the first panoramic image;

[0243] Obtain composition information, which indicates the composition of multiple local viewpoints in the first panoramic image, wherein the composition information of the multiple local viewpoints can be adjusted based on user input;

[0244] Based on the first panoramic image and the composition information, a preview screen is output, wherein the preview screen includes multiple local images corresponding to multiple local perspectives in the first panoramic image;

[0245] In response to the user's video shooting command, a target video with the same composition information as the preview screen is output, wherein the target video is obtained by acquiring a second panoramic image after the video shooting command.

[0246] In some embodiments, the processor 601 is configured to execute a computer program stored in the memory 602, and, when executing the computer program, perform the following steps:

[0247] In response to the user's video recording command, acquire a panoramic image;

[0248] A target video is obtained based on the panoramic image, wherein the target video is obtained by processing at least a portion of the panoramic image;

[0249] During the acquisition of the target video, composition information input by the user is acquired, the composition information indicating the composition of multiple local viewpoints in at least part of the panoramic image;

[0250] Based on the composition information, before obtaining the target video, at least a portion of the panoramic image is displayed with multiple local perspectives, wherein the changes in the multiple local perspectives in at least a portion of the panoramic image are associated with the composition information.

[0251] The specific principle and implementation of the video shooting device provided in this application embodiment are similar to the video shooting method in the foregoing embodiment, and will not be repeated here.

[0252] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps of the video shooting method provided in the above embodiments.

[0253] The computer-readable storage medium can be an internal storage unit of the video shooting device described in any of the foregoing embodiments, such as the hard disk or memory of the video shooting device. The computer-readable storage medium can also be an external storage device of the video shooting device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the video shooting device.

[0254] It should be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application.

[0255] It should also be understood that the term “and / or” as used in this application and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0256] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video shooting method, characterized in that, include: The first panoramic image is acquired using a panoramic shooting device; Obtain composition information, which indicates the composition of multiple local viewpoints in the first panoramic image, wherein the composition information of the multiple local viewpoints can be adjusted based on user input; Based on the first panoramic image and the composition information, a preview screen is output, wherein the preview screen includes multiple local images corresponding to multiple local perspectives in the first panoramic image; In response to the user's video shooting command, a target video with the same composition information as the preview screen is output, wherein the target video is obtained through a second panoramic image acquired by the panoramic shooting device.

2. The video shooting method according to claim 1, characterized in that, The acquisition of the first panoramic image through the panoramic shooting device includes: acquiring the panoramic image through a single visual sensor, or obtaining the panoramic image through image processing of at least two visual sensors.

3. The video shooting method according to claim 2, characterized in that, The at least two vision sensors include two vision sensors arranged back-to-back.

4. The video shooting method according to claim 2, characterized in that, In the case where the panoramic image is obtained by image processing of images obtained by at least two visual sensors, among the multiple local images corresponding to the multiple local viewpoints, at least one local image corresponding to a local viewpoint is obtained by a single visual sensor, and at least another local image corresponding to a local viewpoint is obtained by cropping the images obtained by the two visual sensors.

5. The video shooting method according to claim 2, characterized in that, In the case where the panoramic image is obtained by image processing from at least two visual sensors, the plurality of local perspectives in the first panoramic image include a first perspective and a second perspective, wherein the first perspective is a local perspective of one of the visual sensors, and the second perspective includes local perspectives from at least two of the visual sensors.

6. The video shooting method according to claim 2, characterized in that, The visual sensor is either the main visual sensor of the panoramic shooting device, or it is not the perceptual visual sensor of the panoramic shooting device.

7. The video shooting method according to claim 1, characterized in that, The composition information is used to indicate at least one of the following: The number of the multiple local perspectives; The FOVs corresponding to the multiple local perspectives; The FOV size corresponding to the multiple local viewpoints; The display positions of the multiple local images corresponding to the multiple local viewpoints; The display size of the multiple local images corresponding to the multiple local viewpoints.

8. The video shooting method according to claim 7, characterized in that, The field of view (FOV) corresponding to at least one of the multiple local perspectives is automatically determined by the panoramic shooting device.

9. The video shooting method according to claim 8, characterized in that, The FOV corresponding to the at least one local viewpoint is determined based on the recognition results of the target object in the first panoramic image and / or the second panoramic image.

10. The video shooting method according to claim 9, characterized in that, The method further includes: Obtain the orientation information of the support components of the panoramic shooting device; Based on the orientation information, the target object in the first panoramic image and / or the second panoramic image is determined, wherein the orientation of the target object in the first panoramic image and / or the second panoramic image is determined based on the orientation information.

11. The video shooting method according to claim 9, characterized in that, The local viewpoint corresponding to the target object changes as the target object moves, so that the target object is always within the local viewpoint.

12. The video shooting method according to claim 7, characterized in that, The acquisition of composition information includes: The composition information is acquired in response to a user's composition input operation on the panoramic shooting device or on a device communicatively connected to the panoramic shooting device.

13. The video shooting method according to claim 12, characterized in that, The composition input operation includes: Range selection operation on at least a portion of the first panoramic image, and / or target object selection operation on at least a portion of the first panoramic image.

14. The video shooting method according to claim 7, characterized in that, The method further includes: Obtain the zoom information of the local viewpoint, the zoom information being based on the user's view in the preview screen and / or the target. The scaling operation on the target image in the video is determined; Based on the scaling information, adjust the FOV size corresponding to the local viewpoint.

15. The video shooting method according to claim 7, characterized in that, The method further includes: The display positions of the multiple local images corresponding to the multiple local viewpoints are determined based on the user's position adjustment operation on at least one of the local images.

16. The video shooting method according to claim 15, characterized in that, The method further includes: displaying the current composition layout, wherein the current composition layout includes multiple image positions; The step of obtaining composition information includes: determining the positional relationship between the plurality of local images indicated by the composition information based on the local images at each of the image positions in the current composition layout.

17. The video shooting method according to claim 16, characterized in that, The current composition layout is selected by the user from a plurality of preset composition layouts, including split-screen composition and / or picture-in-picture composition.

18. The video shooting method according to claim 7, characterized in that, The positional relationship between the plurality of local images indicated by the composition information is automatically determined by the panoramic shooting device or a target device capable of communicating with the panoramic shooting device.

19. The video shooting method according to claim 18, characterized in that, The composition information indicates the positional relationship between the plurality of local images, which is determined based on the positional relationship of the plurality of local viewpoints in the first panoramic image.

20. The video shooting method according to claim 18, characterized in that, The positional relationship between the plurality of local images indicated by the composition information is determined according to the type of target object corresponding to the local viewpoint.

21. The video shooting method according to claim 7, characterized in that, The method further includes: acquiring preset orientation information in the first panoramic image; In the preview screen, multiple local perspectives are arranged according to the directions indicated by the preset orientation information.

22. The video shooting method according to claim 21, characterized in that, The preset directional information indicates: east, south, west, north, or indicates front, back, left, right, up, down.

23. The video shooting method according to claim 21, characterized in that, The preset orientation information is determined based on the panoramic shooting device as the center.

24. The video shooting method according to claim 21, characterized in that, The preset orientation information is determined based on user input, and / or the preset orientation information is acquired based on one or more sensors of the panoramic shooting device.

25. The video shooting method according to any one of claims 1-24, characterized in that, The method further includes: The second panoramic image acquired by the panoramic shooting device and the composition information are sent to the target device, so that the target device can generate a target video based on the second panoramic image and the composition information.

26. The video shooting method according to any one of claims 1-24, characterized in that, The first panoramic image and / or the second panoramic image include panoramic images with a horizontal range of at least 180 degrees.

27. The video shooting method according to claim 26, characterized in that, The first panoramic image and / or the second panoramic image include a horizontal 360-degree panoramic image.

28. The video shooting method according to any one of claims 1-24, characterized in that, The first panoramic image and / or the second panoramic image include panoramic images with a vertical range of at least 180 degrees.

29. The video shooting method according to claim 28, characterized in that, The first panoramic image and / or the second panoramic image include a vertical 360-degree panoramic image.

30. The video shooting method according to any one of claims 1-24, characterized in that, The first panoramic image and / or the second panoramic image comprises a panoramic image with a horizontal degree of at least 180 degrees and a vertical degree of at least 180 degrees.

31. The video shooting method according to claim 30, characterized in that, The first panoramic image and / or the second panoramic image include panoramic images with a horizontal 360-degree angle and a vertical 360-degree angle.

32. The video shooting method according to any one of claims 1-24, characterized in that, The panoramic shooting device is either a panoramic camera or a mobile platform.

33. The video shooting method according to claim 32, characterized in that, When the panoramic shooting device is a panoramic camera, the first panoramic image and / or the second panoramic image are acquired by a single visual sensor or two visual sensors arranged in opposite directions by the panoramic camera. When the panoramic shooting device is a mobile platform, the first panoramic image and / or the second panoramic image are acquired by a visual sensor that is not a perception sensor in the mobile platform.

34. The video shooting method according to any one of claims 1-24, characterized in that, The step of outputting a preview image based on the first panoramic image and the composition information includes: The first panoramic image is cropped according to the composition information to obtain multiple local images corresponding to the multiple local viewpoints; The preview screen is output according to the composition indicated by the composition information and the multiple local images corresponding to the multiple local perspectives.

35. The video shooting method according to claim 34, characterized in that, The step of outputting the preview screen according to the composition indicated by the composition information and the multiple local images corresponding to the multiple local perspectives includes: The preview image is obtained by synthesizing multiple local images corresponding to the multiple local viewpoints according to the composition information.

36. The video shooting method according to any one of claims 1-24, characterized in that, The step of responding to the user's video shooting command and outputting a target video with the same composition information as the preview screen includes: In response to the user's video recording command, multiple target images corresponding to the preview screen are stored in chronological order to obtain the target video; or, In response to the user's video recording command, multiple target images corresponding to the preview screen are sent to the target device in chronological order so that the target device can obtain the target video.

37. The video shooting method according to claim 36, characterized in that, The target device includes terminal devices and / or server devices.

38. The video shooting method according to claim 1, characterized in that, The action of responding to the user's video shooting command and outputting a target video with the same composition information as the preview screen can be replaced with: In response to the user's video shooting command, the system outputs the second panoramic image and the composition information corresponding to the preview screen. The second panoramic image and the composition information corresponding to the preview screen can be processed to obtain the target video.

39. A video shooting method, characterized in that, The method includes: In response to the user's video shooting command, a panoramic image is acquired through a panoramic shooting device; A target video is obtained based on the panoramic image, wherein the target video is obtained by processing at least a portion of the panoramic image; During the acquisition of the target video, composition information input by the user is acquired, the composition information indicating the composition of multiple local viewpoints in at least part of the panoramic image; Based on the composition information, before obtaining the target video, at least a portion of the panoramic image is displayed with multiple local perspectives, wherein the changes in the multiple local perspectives in at least a portion of the panoramic image are associated with the composition information.

40. The video shooting method according to claim 39, characterized in that, The process of acquiring a target video based on the panoramic image in response to a user's video capture command includes: During video shooting, based on the multiple local perspectives in the panoramic image indicated by the composition information, multiple target images corresponding to the target video are obtained, and the target images include multiple local images corresponding to the multiple local perspectives in the panoramic image.

41. The video shooting method according to claim 40, characterized in that, The process of acquiring the target video, including acquiring the composition information input by the user, includes: During the process of acquiring the target video, the composition information input by the user and the time information corresponding to the composition information are acquired. The step of displaying multiple local perspectives in the panoramic image based on the composition information includes: Based on the composition information and the time information, the composition of the multiple local viewpoints is adjusted in the target image corresponding to the time information according to the composition information.

42. A video recording device, characterized in that, The video recording device includes a memory and a processor, wherein the memory stores a computer program; and the processor executes the computer program and, when executing the computer program, performs the following steps: Acquire the first panoramic image; Obtain composition information, which indicates the composition of multiple local viewpoints in the first panoramic image, wherein the composition information of the multiple local viewpoints can be adjusted based on user input; Based on the first panoramic image and the composition information, a preview screen is output, wherein the preview screen includes multiple local images corresponding to multiple local perspectives in the first panoramic image; In response to the user's video shooting command, a target video with the same composition information as the preview screen is output, wherein the target video is obtained by acquiring a second panoramic image after the video shooting command.

43. A video recording device, characterized in that, The video recording device includes a memory and a processor, wherein the memory stores a computer program; and the processor executes the computer program and, when executing the computer program, performs the following steps: In response to the user's video recording command, acquire a panoramic image; A target video is obtained based on the panoramic image, wherein the target video is obtained by processing at least a portion of the panoramic image; During the acquisition of the target video, composition information input by the user is acquired, the composition information indicating the composition of multiple local viewpoints in at least part of the panoramic image; Based on the composition information, before obtaining the target video, at least a portion of the panoramic image is displayed with multiple local perspectives, wherein the changes in the multiple local perspectives in at least a portion of the panoramic image are associated with the composition information.

44. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the video capture method according to any one of claims 1-41.

Images