Method, device, equipment and storage medium for panoramic video recording

Abstract

The present application provides a method, apparatus, device and storage medium for panoramic video recording. The method includes: in response to a panoramic recording instruction of the current user, determining a panoramic camera model in a virtual space, wherein the panoramic camera model is composed of multiple virtual cameras constructed around the current user; and recording a panoramic picture of the current user in the virtual space through the panoramic camera model to generate a corresponding panoramic video stream. The embodiment of the present application uses a panoramic camera model to record a panoramic video stream in a virtual space, so that the panoramic video stream does not need to change with the perspective of the current user, thereby avoiding shaking during recording of the panoramic video stream in the virtual space and ensuring stable recording of the panoramic video stream in the virtual space. Moreover, when any viewing user watches the panoramic video stream, multi-camera switching of the panoramic video stream is supported, thereby enhancing the user's immersive experience of watching the panoramic video stream.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video processing technology, and in particular to a method, apparatus, device, and storage medium for panoramic video recording. Background Art

[0002] Currently, VR technology is being applied in increasingly diverse scenarios, providing users with a wide variety of virtual scenes, such as various VR games and virtual live streaming applications. To support users in successfully sharing their interactions in various virtual scenes with other users, allowing them to experience different virtual scenes, after a user enters any virtual scene, the user's interactive screen in that virtual scene can be recorded in real time according to the user's recording needs and shared with other users.

[0003] Typically, the primary user's first-person perspective (PPO) in the virtual scene can be determined in real time based on their real-time motion data, such as their six degrees of freedom (6DoF) data. The interactive footage is then recorded based on the changes in the primary user's PPO, ensuring that the recorded interactive footage accurately reflects the primary user's field of view.

[0004] However, when the main user makes large movements in the virtual scene, the recorded interactive image will be shaken to a certain extent, which will cause a certain sense of dizziness to the viewing users, thereby reducing the viewing experience of the viewing users. Summary of the Invention

[0005] The present application provides a method, apparatus, device and storage medium for panoramic video recording, which realizes stable recording of panoramic video streams in a virtual space, supports viewers to switch between multiple cameras under the panoramic video stream, and enhances the immersive experience of watching the panoramic video stream.

[0006] In a first aspect, an embodiment of the present application provides a method for panoramic video recording, the method comprising:

[0007] In response to a panoramic recording instruction of a current user, determining a panoramic camera model in a virtual space, the panoramic camera model being composed of a plurality of virtual cameras constructed around the current user;

[0008] The panoramic picture of the current user in the virtual space is recorded by the panoramic camera model to generate a corresponding panoramic video stream.

[0009] In a second aspect, an embodiment of the present application provides a device for panoramic video recording, the device comprising:

[0010] a panoramic camera model determination module, configured to determine a panoramic camera model in a virtual space in response to a panoramic recording instruction of a current user, wherein the panoramic camera model is composed of a plurality of virtual cameras constructed around the current user;

[0011] The panoramic video recording module is used to record the panoramic picture of the current user in the virtual space through the panoramic camera model to generate a corresponding panoramic video stream.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0013] A processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to execute the panoramic video recording method provided in the first aspect of the present application.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, which enables a computer to execute the panoramic video recording method provided in the first aspect of the present application.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, which enable a computer to execute the panoramic video recording method provided in the first aspect of the present application.

[0016] Through the technical solution of the present application, after entering any virtual space, in response to the current user's panoramic recording instruction, a panoramic camera model composed of multiple virtual cameras built around the current user in the virtual space will first be determined. Then, the panoramic camera model is used to record the panoramic picture of the current user in the virtual space to generate a corresponding panoramic video stream. Thus, the panoramic video stream in the virtual space is recorded by the panoramic camera model, so that the panoramic video stream does not need to change with the current user's perspective, thereby avoiding shaking when recording the panoramic video stream in the virtual space and ensuring stable recording of the panoramic video stream in the virtual space. Moreover, when any viewing user watches the panoramic video stream, multi-camera switching of the panoramic video stream is supported, thereby enhancing the user's immersive experience of watching the panoramic video stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1aA diagram illustrating a scene architecture for the panoramic video recording method provided in an embodiment of the present application;

[0019] Figure 1b A schematic diagram of a spherical model rendered by a panoramic video stream provided in an embodiment of the present application;

[0020] Figure 2 A flowchart of a panoramic video recording method provided in an embodiment of the present application;

[0021] Figure 3 An exemplary schematic diagram of a panoramic camera model provided in an embodiment of the present application;

[0022] Figure 4 A flowchart of another panoramic video recording method provided in an embodiment of the present application;

[0023] Figure 5a Five exemplary schematic diagrams of the surrounding interactive areas of the current user in the virtual space provided in the embodiment of the present application;

[0024] Figure 5b Another exemplary schematic diagram of the surrounding interactive area of ​​the current user in the virtual space provided in an embodiment of the present application;

[0025] Figure 6 Another exemplary schematic diagram of a panoramic camera model provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of a panoramic video recording device provided in an embodiment of the present application;

[0027] Figure 8 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0029] Clearly and completely described, it is obvious that the embodiments described are only part of the embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, ordinary technicians in this field will not make any mistakes.

[0030] All other embodiments obtained on the premise of creative work shall fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence.

[0032] It should be understood that the terms so used are interchangeable under appropriate circumstances so that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein.

[0033] The words "include" and "have" and any variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0034] 5 In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or

[0035] Note that any embodiment or solution described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete way.

[0036] 0 Considering that each user is watching the interactive screen recorded by a main user in a virtual scene,

[0037] The recorded interactive screen usually changes with the perspective of the main user, resulting in a certain sense of shaking in the interactive screen. Therefore, in order to avoid the shaking feeling when the user watches the recorded interactive screen, the inventive concept of this application is: after detecting the panoramic recording instruction of the current user in any virtual space, the panoramic camera model composed of multiple virtual cameras built around the current user in the virtual space will be determined first, so as to record the panoramic screen of the current user in the virtual space through the panoramic camera model, thereby generating a corresponding panoramic video stream, so that the panoramic video stream does not need to change with the perspective of the current user, thereby avoiding shaking during the recording of the panoramic video stream in the virtual space and ensuring stable recording of the panoramic video stream in the virtual space.

[0038] Before introducing the specific technical solutions of this application, we first explain the overall scenario system in which a user records a panoramic video stream in a virtual space to support other users to watch it:

[0039] Figure 1a This is a diagram of a scenario architecture for the panoramic video recording method provided in an embodiment of the present application. The application scenario may include a current user terminal 110, a cloud service terminal 120, and at least one viewing user terminal 130.

[0040] Considering that this application is primarily aimed at recording the real-time interaction between the current user and various virtual objects in a virtual space, the current user terminal 110 can be a head-mounted display (HMD) of any virtual space product. The virtual space product can be a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, an extended reality (XR) device, etc.

[0041] When a viewing user watches a user's recorded interactive screen in a virtual space, the recorded interactive screen can be played through various types of devices. Therefore, the viewing user terminal 110 can be any electronic device that supports video playback, including but not limited to tablet computers, mobile phones (such as foldable screen phones, large screen phones, etc.), wearable devices, car devices, VR / AR / MR devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart TVs, smart screens, HDTVs, 4K TVs, smart speakers, smart projectors, etc.

[0042] After putting on the HMD and turning it on, the user enters a virtual space, which presents them with a variety of virtual applications, such as virtual gaming and live streaming. The user can then log into a virtual application within the virtual space to create a corresponding 3D virtual scene, allowing them to interact with the various virtual objects within the 3D virtual scene.

[0043] Furthermore, after entering the virtual space, if the current user wants to record his or her interactive scenes in the virtual space to share with other users, the current user can perform the corresponding panoramic recording operation so that the panoramic scenes of the current user in the virtual space can be recorded in real time through a panoramic camera model composed of multiple virtual cameras built around the current user in the virtual space, thereby generating a corresponding panoramic video stream.

[0044] At this time, for the panoramic video stream of the current user performing various interactions in the virtual space, the current user terminal 110 can generate a corresponding panoramic video stream based on the current user's various motion data in the virtual space and upload it to the cloud server 120. Alternatively, the current user terminal 110 can report the current user's various motion data in the virtual space to the cloud server 120, so that the cloud server 120 can generate a corresponding panoramic video stream based on the current user's various motion data in the virtual space.

[0045] Among them, the various motion data of the current user in the virtual space may include but are not limited to motion sensing parameters collected by inertial sensors, positioning sensors, etc. installed on the HMD, as well as touch data performed by handles, body trackers, etc. adapted to the HMD.

[0046] Then, the current user terminal 110 or the cloud service terminal 120 can determine the 6DoF data of the current user in the virtual space by analyzing the above motion data, thereby determining the position of each virtual camera built around the current user in the virtual space, and obtaining the corresponding panoramic camera model to record the panoramic picture of the current user in the virtual space in real time, thereby generating a corresponding panoramic video stream.

[0047] When any viewing user wants to watch the panoramic video stream recorded by the current user in the virtual space, he or she can directly initiate a corresponding panoramic viewing request to the cloud server 120. Then, after receiving the panoramic viewing request of any viewing user for the current user, the cloud server 120 can directly perform corresponding panoramic rendering of the panoramic video stream of the current user in the virtual space through the VR spherical model. Then, according to the camera position selected by the viewing user terminal 130, the viewport position in the VR spherical model is switched, so as to send the interactive image under the corresponding viewport to the viewing user terminal 130, and decode and display it on the viewing user terminal 130, so as to support the viewing user's multi-camera switching of the panoramic video stream and enhance the user's immersive experience of watching the panoramic video stream.

[0048] Typically, in a VR spherical model, the user stands at the center of the sphere and looks outward. However, due to the limited field of view of the human eye, at any given moment, the user typically only sees the portion of the VR spherical model that is within their field of view. Only when the user rotates their viewing angle, causing their field of view to change, can they see the rest of the VR spherical model.

[0049] Considering that in each panoramic image rendered in the VR spherical model, the top and bottom of the panoramic image will have a certain barrel distortion due to stretching. If the top and bottom of the panoramic image are compressed into one side of a cube, and then the images from the front, back, left and right of the panoramic image are combined, a cubic box can be formed. This can not only ensure no pixel loss, but also reduce the rendering volume of the panoramic image. So, if Figure 1b As shown, the present application can place the panoramic picture pixel ball rendered at each moment into a cubic box that is larger than the pixel ball, and then continuously enlarge the panoramic picture pixel ball until it fills the entire cubic box, thereby obtaining a VR spherical model presented to the user, thereby avoiding barrel distortion during panoramic picture rendering and improving the user's viewing experience of the panoramic video stream.

[0050] Next, the specific recording solution for the panoramic video stream provided by this application is described in detail:

[0051] Figure 2 A flowchart of a method for panoramic video recording provided in an embodiment of the present application. The method can be performed by the panoramic video recording device provided in the present application, wherein the panoramic video recording device can be implemented by any software and / or hardware. Exemplarily, the panoramic video recording device can be applied to any electronic device, which may include but is not limited to various devices with data processing capabilities such as wearable devices, VR / AR / MR devices, and servers. The present application does not impose any restrictions on the specific type of electronic device.

[0052] Specifically, such as Figure 2 As shown, the method may include the following steps:

[0053] S210 , in response to a panoramic recording instruction of the current user, determining a panoramic camera model in a virtual space, where the panoramic camera model is composed of a plurality of virtual cameras constructed around the current user.

[0054] Considering that when each user watches the interactive screen recorded by a main user in a virtual scene, the recorded interactive screen usually changes with the main user's perspective, resulting in a certain sense of shaking in the interactive screen. Therefore, in order to avoid shaking when recording videos in the virtual space, the present application can record the interactive screen when the current user interacts with each virtual object in the virtual space from a third-party perspective. That is, the present application can construct a virtual camera from a third-party perspective in the virtual space, and not obtain the corresponding interactive screen according to the current perspective indicated by the current user's 6DoF data in the virtual space.

[0055] Moreover, in order to ensure that the viewing users can fully view the interactive scenes of the current user in the virtual space from multiple angles, the present application can set a panoramic recording control in the virtual space so that the current user can interact with various virtual objects in the virtual space, and can trigger the panoramic recording control to record the corresponding panoramic scenes in real time, thereby providing each viewing user with a corresponding panoramic video stream to support the viewing users to switch to multiple angles for viewing the panoramic video stream.

[0056] In this application, after the current user enters the virtual space, if there is a need to record their own interactive scenes in the virtual space, the panoramic recording control in the virtual space will be triggered, thereby generating the corresponding panoramic recording instruction. Then, after receiving the panoramic recording instruction of the current user, it is indicated that this application needs to record the interactive scenes of the current user in the virtual space when interacting with various virtual objects in the virtual space from multiple angles to determine the panoramic picture of the current user in the virtual space. Therefore, the panoramic camera model used to record the panoramic picture will be determined in the virtual space first.

[0057] In order to ensure the full recording of the panoramic picture of the current user in the virtual space, the present application can determine the current user's position in the virtual space based on the real-time motion data of the current user in the virtual space, such as 6DoF data. Figure 3 As shown, based on the current user's position in the virtual space, multiple virtual cameras can be constructed around the current user, creating the effect of a circle of virtual cameras surrounding the current user. In this case, each virtual camera constructed around the current user is set with a different camera position and shooting angle facing the current user, thereby forming the panoramic camera model in this application, which can record a panoramic image of the current user in the virtual space from multiple angles.

[0058] S220: Record a panoramic picture of the current user in the virtual space through a panoramic camera model to generate a corresponding panoramic video stream.

[0059] After determining the panoramic camera model within the virtual space, each virtual camera within the model can be used to record the user's interactions with various virtual objects in the virtual space in real time from different camera positions and shooting angles, thus forming a complete panoramic image. The real-time recorded panoramic images are then integrated according to their respective shooting times to generate a corresponding panoramic video stream, providing each viewer with corresponding panoramic image support and enabling users to switch between multiple angles of the panoramic video stream.

[0060] As an optional implementation scheme in this application, for recording of panoramic video streams, this application can be implemented through the following steps:

[0061] The first step is to use each virtual camera in the panoramic camera model to record the current user's interactive screen in the virtual space from the perspective of the virtual camera's corresponding camera position.

[0062] Because each virtual camera within the panoramic camera model is located at a different position around the current user, the camera angles set by different virtual cameras when recording the current user will also be different. Therefore, based on the camera position and camera angle of each virtual space within the panoramic camera model, the relative position relationship of the current user and various virtual objects in the pre-constructed virtual space relative to each virtual camera can be analyzed, thereby determining the virtual image within each virtual camera's viewport. In this way, each virtual camera can record the current user's interactive image within the virtual space from the corresponding camera angle.

[0063] It should be understood that in a live broadcast scenario, since the current user's panoramic video stream is pushed to the viewing user in real time, in order to ensure that the viewing user can adaptively switch between multiple perspectives of the panoramic video stream, the viewing user's real-time motion data in the virtual space, such as 6DoF data, is forwarded to the cloud server. Then, in this application, the camera angle of each virtual camera in the panoramic camera model can be adjusted based on the viewing user's real-time motion data.

[0064] That is to say, by analyzing the real-time motion data of the viewing user on the cloud server, the viewing perspective of the viewing user for the panoramic video stream can be determined. The viewing perspective can include the horizontal perspective of a specific viewing position represented by turning the head left and right in the horizontal direction and the vertical perspective of the virtual camera under each position represented by turning the head up and down in the vertical direction when facing the current user.

[0065] During the recording of the current user's panoramic video stream, while ensuring that each virtual camera in the panoramic camera model is facing the current user, the vertical orientation of each virtual camera is adjusted based on the viewing user's vertical viewing angle, thereby adjusting the camera's viewing angle. For example, if the viewing user looks up, each virtual camera can be controlled to face the current user and then uniformly adjust its viewing angle upward, thereby updating the camera's viewing angle to record the panoramic video stream that matches the viewing user's perspective.

[0066] In the second step, the interactive images recorded by each virtual camera at the same timestamp are stitched together to obtain a panoramic image at that timestamp.

[0067] For the interactive screens recorded in real time by each virtual camera in the panoramic camera model, this application can set a corresponding timestamp for each interactive screen based on the recording time of each interactive screen. Then, the interactive screens recorded by each virtual camera at the same timestamp are obtained, and the panoramic screens at the same timestamp are stitched together according to the distribution between the virtual cameras to obtain the panoramic screen at that timestamp. According to the above steps, a panoramic screen at each timestamp can be obtained during the real-time recording process.

[0068] The third step is to integrate the panoramic images at each timestamp to obtain the corresponding panoramic video stream.

[0069] According to the order of timestamps, the panoramic images at each timestamp can be uniformly coded and integrated to obtain the corresponding panoramic video stream.

[0070] At this time, when the cloud server receives a viewing request from any viewing user for the current user's panoramic video stream, it can determine the viewing angle of the viewing user for the panoramic video stream based on the viewing user's real-time motion data, so as to determine the portion of the image within the viewing field of the viewing user in the VR spherical model based on the viewing angle, and then push the portion of the image within the user's field of view to the viewing user. At this time, because the viewing angle of the viewing user for the panoramic video stream will change smoothly according to the viewing user's real-time motion data, the viewing field of the viewing user in the VR spherical model will also change smoothly. Therefore, when the viewing user switches the viewing angle to watch the corresponding panoramic video stream, it can ensure that the panoramic video stream switches smoothly on the viewing user side, thereby improving the viewing experience of the viewing user when switching to watch the panoramic video stream from multiple perspectives.

[0071] The technical solution provided by the embodiment of the present application, after entering any virtual space, responds to the current user's panoramic recording instruction and first determines a panoramic camera model composed of multiple virtual cameras built around the current user in the virtual space. Then, the panoramic camera model is used to record the panoramic picture of the current user in the virtual space to generate a corresponding panoramic video stream. Thus, the panoramic video stream in the virtual space is recorded by the panoramic camera model, so that the panoramic video stream does not need to change with the current user's perspective, thereby avoiding shaking when recording the panoramic video stream in the virtual space and ensuring stable recording of the panoramic video stream in the virtual space. Moreover, when any viewing user watches the panoramic video stream, multi-camera switching of the panoramic video stream is supported, thereby enhancing the user's immersive experience of watching the panoramic video stream.

[0072] As an optional implementation scheme in the present application, in order to fully record the panoramic picture of the current user in the virtual space, the present application describes in detail the specific process of determining the panoramic camera model in the virtual space.

[0073] Figure 4 This is a flowchart of another method for panoramic video recording provided in an embodiment of the present application. The method may specifically include the following steps:

[0074] S410 , in response to a panoramic recording instruction of the current user, determining a peripheral interactive area of ​​the current user in the virtual space.

[0075] After receiving the panoramic recording instruction of the current user in the virtual space, in order to record the interactive screen of the current user in the virtual space from a third-party perspective, this application can set up multiple virtual cameras in all directions around the current user, so as to shoot the current user from multiple angles.

[0076] Therefore, in order to realize the all-round setting of the panoramic camera model around the current user, this application will first define a surrounding interactive area for the current user in the virtual space according to the current user's position in the virtual space, so as to set up multiple virtual cameras on the boundary of the area to ensure that the panoramic camera model is set all-round around the current user.

[0077] It should be understood that the current user's surrounding interactive area in this application can be a three-dimensional spatial area delineated within the virtual space and including the current user, or a two-dimensional planar area delineated on the ground within the virtual space and including the current user. Furthermore, the current user's surrounding interactive area can be a regular area, such as a cylindrical space area, a circular area, a hexahedral area, a quadrilateral area, etc., or it can be an arbitrarily delineated irregular area, etc., and this application does not limit this.

[0078] In some implementations, when the current user interacts with various virtual objects in the virtual space, the current user will perform various movements in the virtual space. If the current user's range of movement in the virtual space is too large, it may exceed the panoramic shooting range of the panoramic camera model. Therefore, in order to prevent the panoramic camera model from missing the panoramic picture of the current user, the present application can first determine the current user's movable area in the virtual space; when the movable area is less than or equal to the preset upper limit of the panoramic coverage area, the movable area is used as the current user's peripheral interaction area; and when the movable area is greater than the preset upper limit of the panoramic coverage area, the current user's peripheral interaction area is determined based on the current user's movement position in the virtual space, thereby ensuring that the panoramic camera model constructed according to the area boundary of the peripheral interaction area can follow the current user's random movement to fully record the current user's panoramic picture in the virtual space.

[0079] That is, considering the limited coverage of each virtual camera, in order to ensure that the panoramic camera model fully records the panoramic image of the current user in the virtual space, this application will set an upper limit for the panoramic camera model's panoramic coverage area. When the panoramic camera model is constructed within this upper limit to record the panoramic image of the current user in the virtual space, the current user in the panoramic image can achieve a certain clarity. However, when the panoramic camera model is constructed outside this upper limit to record the panoramic image of the current user in the virtual space, the current user in the panoramic image may have lower clarity, and it cannot be guaranteed that the current user's interactive content in the virtual space can be clearly viewed.

[0080] Therefore, when constructing a panoramic camera model within a virtual space, the user's active area within the virtual space is first determined. This active area can be a limited area, such as the virtual stage area for performers, or the area surrounding the ping-pong table in a virtual table tennis game. Alternatively, the active area can encompass the entire virtual space. For example, in a shooting game, the user is constantly running, dodging, and so on. The game map is very large, so the user's active area is also very large, covering the entire virtual space of the game map.

[0081] If the user's active area within the virtual space is less than or equal to the preset upper limit of the panoramic coverage area, the user is moving within a limited area, and even if a panoramic camera model is fixed outside the active area, the panoramic image captured by the panoramic camera model can still maintain clarity and visibility of the user. Therefore, in this case, the active area can be directly used as the user's surrounding interactive area, allowing the panoramic camera model to be constructed at the boundary of the active area to fully record the user's panoramic image within the virtual space.

[0082] If the current user's movable area in the virtual space is larger than the preset upper limit of the panoramic coverage area, it means that the current user's range of movement in the virtual space is very wide. If a fixed panoramic camera model is set in the virtual space, the current user may exceed the panoramic coverage range of the panoramic camera model, and the panoramic picture of the current user in the virtual space cannot be recorded. Therefore, in this case, the surrounding interactive area used to construct the panoramic camera model also needs to move with the movement of the current user. Then, the present application can first determine the movement position of the current user in the virtual space to analyze the movement of the current user in the virtual space. Then, based on the movement of the current user in the virtual space and the upper limit of the panoramic coverage area set by the panoramic camera model on the basis of ensuring that the current user in the panoramic picture reaches a certain clarity, a limited spatial area is delineated around the current user as the surrounding interactive area of ​​the current user.

[0083] As an optional implementation scheme in this application, in order to ensure the diversity of the surrounding interactive area of ​​the current user, this application determines the surrounding interactive area of ​​the current user based on the movement position of the current user in the virtual space. This can be achieved in the following two ways:

[0084] 1) Divide the virtual space into multiple virtual subspaces; determine the target virtual subspace where the current user is located according to the movement position of the current user in the virtual space, as the surrounding interaction area of ​​the current user.

[0085] That is, if Figure 5a As shown, according to the upper limit of the panoramic coverage area set by the panoramic camera model for the current user's recording clarity requirements, this application can evenly divide the virtual space into multiple virtual subspaces. The area of ​​each virtual subspace can be smaller than the upper limit of the panoramic coverage area to ensure the clarity of the current user in the panoramic image recorded by the panoramic camera model.

[0086] Then, based on the user's movement position within the virtual space, the user is determined to be in which of the various virtual subspaces divided within the virtual space, thereby determining the target virtual subspace the user is currently in. This target virtual subspace is then used as the user's surrounding interactive area, allowing the corresponding panoramic camera model to be constructed at the boundary of the target virtual subspace.

[0087] 2) Taking the current user's movement position in the virtual space as the center point, determine the current user's surrounding interactive area according to the pre-set surrounding area size.

[0088] That is, during the current user's movement in the virtual space, the current user's movement position in the virtual space can be determined in real time. Furthermore, by presetting the shape of the current user's surrounding interactive area, such as a circle, a rectangle, or an irregular shape, the size of the pre-set surrounding area can be determined. Then, during the current user's real-time movement in the virtual space, the current user's movement position in the virtual space can be used as the center point of the surrounding interactive area, and a surrounding interactive area can be delineated around the current user according to the pre-set surrounding area size.

[0089] In this case, if Figure 5b As shown, during the current user's movement in the virtual space, the surrounding interactive area can change in real time following the current user's movement. In this case, the current user can be at the center of the surrounding interactive area in real time, so that the panoramic camera model constructed at the area boundary of the surrounding interactive area can also move with the current user's movement.

[0090] S420: Construct multiple virtual cameras according to the area boundaries of the surrounding interactive area to obtain corresponding panoramic camera models.

[0091] After determining the surrounding interactive area of ​​the current user in the virtual space, multiple virtual cameras can be constructed in sequence on the area boundary of the surrounding interactive area according to the preset adjacent camera intervals, so that the multiple virtual cameras are constructed around the current user, thereby forming a corresponding panoramic camera model.

[0092] In some possible implementations, in order to ensure that the panoramic camera model stably records the panoramic picture of the current user in the virtual space, the present application can set multiple virtual cameras in the panoramic camera model to be at the same horizontal height, and when following the movement of the current user, the horizontal height of the multiple virtual cameras in the panoramic camera model will remain unchanged to avoid the panoramic camera model shaking in the horizontal direction when recording the panoramic picture.

[0093] Therefore, for the panoramic camera model, the present application can set a fixed horizontal height, which can be higher than the height of the current user, so that the panoramic camera model constructed at this horizontal height can record the panoramic picture of the current user in the virtual space in real time from a third-party perspective higher than the current user.

[0094] Furthermore, if Figure 6As shown, based on the set horizontal height, the positions for each virtual camera are determined above the boundary of the surrounding interactive area. A virtual camera is constructed at each position, ensuring that each virtual camera is at a horizontal height above the ground, rather than being directly placed on the ground, ensuring the diversity of panoramic recording. Then, for each virtual camera, the camera position and perspective can be set according to the direction of the current user, thereby generating a corresponding panoramic camera model to accurately record the current user's panoramic view in the virtual space.

[0095] S430: Record the panoramic image of the current user in the virtual space through the panoramic camera model to generate a corresponding panoramic video stream.

[0096] The technical solution provided by the embodiment of the present application, after entering any virtual space, responds to the current user's panoramic recording instruction and first determines a panoramic camera model composed of multiple virtual cameras built around the current user in the virtual space. Then, the panoramic camera model is used to record the panoramic picture of the current user in the virtual space to generate a corresponding panoramic video stream. Thus, the panoramic video stream in the virtual space is recorded by the panoramic camera model, so that the panoramic video stream does not need to change with the current user's perspective, thereby avoiding shaking when recording the panoramic video stream in the virtual space and ensuring stable recording of the panoramic video stream in the virtual space. Moreover, when any viewing user watches the panoramic video stream, multi-camera switching of the panoramic video stream is supported, thereby enhancing the user's immersive experience of watching the panoramic video stream.

[0097] Figure 7 This is a schematic diagram of a panoramic video recording apparatus provided in an embodiment of the present application. The panoramic video recording apparatus 700 may include:

[0098] A panoramic camera model determination module 710 is configured to determine a panoramic camera model in a virtual space in response to a panoramic recording instruction of a current user, wherein the panoramic camera model is composed of multiple virtual cameras constructed around the current user;

[0099] The panoramic video recording module 720 is configured to record the panoramic image of the current user in the virtual space through the panoramic camera model to generate a corresponding panoramic video stream.

[0100] In some implementations, the panoramic camera model determination module 710 may include:

[0101] a surrounding area determining unit, configured to determine a surrounding interactive area of ​​the current user in the virtual space;

[0102] The panoramic camera model construction unit is used to construct multiple virtual cameras according to the area boundaries of the surrounding interactive area to obtain corresponding panoramic camera models.

[0103] In some implementations, the surrounding area determination unit may be specifically configured to:

[0104] Determining an active area of ​​the current user in the virtual space;

[0105] If the movable area is smaller than or equal to a preset upper limit of the panoramic coverage area, the movable area is used as the surrounding interactive area of ​​the current user;

[0106] If the movable area is larger than a preset upper limit of the panoramic coverable area, a peripheral interactive area of ​​the current user is determined according to the movement position of the current user in the virtual space.

[0107] In some implementations, the surrounding area determination unit may be specifically configured to:

[0108] dividing the virtual space into a plurality of virtual subspaces;

[0109] According to the movement position of the current user in the virtual space, a target virtual subspace where the current user is located is determined as a peripheral interactive area of ​​the current user.

[0110] In some implementations, the surrounding area determination unit may further be specifically configured to:

[0111] The surrounding interactive area of ​​the current user is determined based on the movement position of the current user in the virtual space as the center point and according to a preset size of the surrounding area.

[0112] In some implementations, the panoramic camera model construction unit may be specifically configured to:

[0113] Determining the position of each virtual camera above the boundary of the peripheral interactive area according to a preset horizontal height;

[0114] A corresponding virtual camera is constructed at each camera position, and the camera position and viewing angle of the virtual camera are set to obtain a corresponding panoramic camera model.

[0115] In some implementations, the panoramic video recording module 720 may be specifically configured to:

[0116] Recording the interactive scene of the current user in the virtual space from the camera perspective corresponding to each virtual camera in the panoramic camera model;

[0117] Stitching the interactive images recorded by each virtual camera at the same time stamp to obtain a panoramic image at the time stamp;

[0118] Integrate the panoramic images at each timestamp to obtain the corresponding panoramic video stream.

[0119] In some implementations, the panoramic video recording apparatus 700 may further include:

[0120] The camera angle adjustment module is used to adjust the camera angle of each virtual camera in the panoramic camera model according to the real-time motion data of the viewing user.

[0121] In an embodiment of the present application, after entering any virtual space, in response to the current user's panoramic recording instruction, a panoramic camera model composed of multiple virtual cameras constructed around the current user in the virtual space is first determined. Then, the panoramic camera model is used to record the current user's panoramic image in the virtual space to generate a corresponding panoramic video stream. Therefore, the panoramic video stream in the virtual space is recorded by the panoramic camera model, so that the panoramic video stream does not need to change with the current user's perspective, thereby avoiding shaking when recording the panoramic video stream in the virtual space and ensuring stable recording of the panoramic video stream in the virtual space. Moreover, when any viewing user watches the panoramic video stream, multi-camera switching of the panoramic video stream is supported, thereby enhancing the user's immersive experience of watching the panoramic video stream.

[0122] It should be understood that the device embodiment and the method embodiment in the present application may correspond to each other, and similar descriptions may refer to the method embodiment in the present application. To avoid repetition, they will not be described here.

[0123] Specifically, Figure 7 The device 700 shown can execute any method embodiment provided in this application, and Figure 7 The aforementioned and other operations and / or functions of the various modules in the illustrated apparatus 700 are for implementing the corresponding processes of the aforementioned method embodiments, and for the sake of brevity, they are not described again herein.

[0124] The above-mentioned method embodiment of the embodiment of the present application is described from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in the form of hardware, can be implemented by instructions in the form of software, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above-mentioned method embodiment in conjunction with its hardware.

[0125] Figure 8 It is a schematic block diagram of an electronic device provided in an embodiment of the present application.

[0126] like Figure 8 As shown, the electronic device 800 may include:

[0127] The memory 810 and the processor 820 are configured to store computer programs and transmit the program code to the processor 820. In other words, the processor 820 can call and run the computer program from the memory 810 to implement the method in the embodiment of the present application.

[0128] For example, the processor 820 may be configured to execute the above method embodiments according to instructions in the computer program.

[0129] In some embodiments of the present application, the processor 820 may include but is not limited to:

[0130] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0131] In some embodiments of the present application, the memory 810 includes but is not limited to:

[0132] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0133] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 810 and executed by the processor 820 to implement the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program on the electronic device 800.

[0134] like Figure 8 As shown, the electronic device may further include:

[0135] The transceiver 830 may be connected to the processor 820 or the memory 810 .

[0136] The processor 820 may control the transceiver 830 to communicate with other devices. Specifically, the processor 820 may send information or data to other devices or receive information or data sent by other devices. The transceiver 830 may include a transmitter and a receiver. The transceiver 830 may further include one or more antennas.

[0137] It should be understood that the various components in the electronic device 800 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0138] The present application also provides a computer storage medium having a computer program stored thereon, which enables the computer to perform the method of the above method embodiment when the computer program is executed by the computer.

[0139] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to perform the method of the above method embodiment.

[0140] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0141] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for panoramic video recording, characterized in that: The method comprises: In response to a panoramic recording instruction of a current user, determining a panoramic camera model in a virtual space, the panoramic camera model being composed of a plurality of virtual cameras constructed around the current user; Recording a panoramic picture of the current user in the virtual space by using the panoramic camera model to generate a corresponding panoramic video stream; Among them, if the panoramic recording instruction is a live recording instruction, the camera angle corresponding to each virtual camera in the panoramic camera model is determined by the horizontal angle of view when the virtual camera is facing the current user and the vertical angle of view of the audience user in the vertical direction.

2. The method according to claim 1, characterized in that Determining the panoramic camera model in the virtual space includes: Determining a peripheral interactive area of ​​the current user in the virtual space; According to the area boundary of the peripheral interactive area, multiple virtual cameras are constructed to obtain corresponding panoramic camera models.

3. The method according to claim 2, characterized in that Determining the surrounding interactive area of ​​the current user in the virtual space includes: Determining an active area of ​​the current user in the virtual space; If the movable area is smaller than or equal to a preset upper limit of the panoramic coverage area, the movable area is used as the surrounding interactive area of ​​the current user; If the movable area is larger than a preset upper limit of the panoramic coverable area, a peripheral interactive area of ​​the current user is determined according to the movement position of the current user in the virtual space.

4. The method according to claim 3, characterized in that The determining of the surrounding interactive area of ​​the current user according to the movement position of the current user in the virtual space includes: dividing the virtual space into a plurality of virtual subspaces; According to the movement position of the current user in the virtual space, a target virtual subspace where the current user is located is determined as a peripheral interactive area of ​​the current user.

5. The method according to claim 3, characterized in that The determining of the surrounding interactive area of ​​the current user according to the movement position of the current user in the virtual space further includes: The surrounding interactive area of ​​the current user is determined based on the movement position of the current user in the virtual space as the center point and according to a preset size of the surrounding area.

6. The method according to claim 2, characterized in that The step of constructing a plurality of virtual cameras according to the area boundaries of the surrounding interactive area to obtain corresponding panoramic camera models includes: Determining the position of each virtual camera above the boundary of the peripheral interactive area according to a preset horizontal height; A corresponding virtual camera is constructed at each camera position, and the camera position and viewing angle of the virtual camera are set to obtain a corresponding panoramic camera model.

7. The method according to claim 1, characterized in that The step of recording a panoramic picture of the current user in the virtual space by using the panoramic camera model to generate a corresponding panoramic video stream includes: Recording the interactive scene of the current user in the virtual space from the camera perspective corresponding to each virtual camera in the panoramic camera model; Stitching the interactive images recorded by each virtual camera at the same time stamp to obtain a panoramic image at the time stamp; Integrate the panoramic images at each timestamp to obtain the corresponding panoramic video stream.

8. The method according to claim 7, characterized in that The method further comprises: The camera angle of each virtual camera in the panoramic camera model is adjusted according to the real-time motion data of the viewing user.

9. A device for panoramic video recording, characterized in that: The device comprises: a panoramic camera model determination module, configured to determine a panoramic camera model in a virtual space in response to a panoramic recording instruction of a current user, wherein the panoramic camera model is composed of a plurality of virtual cameras constructed around the current user; A panoramic video recording module, configured to record a panoramic image of the current user in the virtual space through the panoramic camera model to generate a corresponding panoramic video stream; Among them, if the panoramic recording instruction is a live recording instruction, the camera angle corresponding to each virtual camera in the panoramic camera model is determined by the horizontal angle of view when the virtual camera is facing the current user and the vertical angle of view of the audience user in the vertical direction.

10. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the panoramic video recording method according to any one of claims 1 to 8 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for panoramic video recording according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising instructions, characterized in that When the computer program product is run on an electronic device, the electronic device is enabled to execute the panoramic video recording method according to any one of claims 1 to 8.

Images