Video generation methods, devices and electronic equipment

Abstract

This application provides a video generation method, apparatus, and electronic device. The method includes: acquiring a panoramic browsing animation obtained by panoramic shooting of a target space based on a spatial point browsing strategy, wherein the spatial point browsing strategy indicates the movement path of the observation point, the optimal rotation angle corresponding to the shooting of the observation point, and the shooting field of view; rendering the panoramic browsing animation on an adapted panoramic canvas, and extracting frame images from the panoramic canvas; synthesizing the extracted batch of frame images into a video at a target frame rate to obtain a playback video corresponding to the target frame rate; and playing the playback video on a playback device whose refresh rate matches the target frame rate. This application improves shooting efficiency while ensuring shooting quality by shooting according to a preset planned path and shooting at each observation point in a specified manner; and by synthesizing a video at a specified frame rate and playing it on an adapted device, it avoids screen tearing and flickering, ensuring the smoothness of the video.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video generation method, apparatus, and electronic device. Background Technology

[0002] With the rapid development of VR (Virtual Reality), AR (Augmented Reality), and 3D modeling technologies, the demand for digital representation and visualization of space is increasing. In many fields such as architecture, interior design, and game development, accurate spatial imaging and high-quality rendering have become particularly important.

[0003] Currently, traditional spatial shooting and video processing methods mainly involve the following operations: the shooting personnel use shooting equipment to shoot the space from multiple angles to obtain the raw data of the spatial shooting; and the raw data of the shooting is displayed on the playback terminal.

[0004] Despite some progress in space photography and video processing, the following problems still exist:

[0005] 1. Video shooting is affected by the shooting equipment. Different devices produce videos with varying resolutions and frame rates, which may cause stuttering, tearing, or shaking during video playback, affecting the user experience.

[0006] 2. Shooting in space is highly random, often without a planned shooting path, or the shooting path is planned too simply, resulting in low shooting efficiency, poor data integrity, and difficulty in obtaining comprehensive and coherent high-quality data.

[0007] Therefore, there is an urgent need to improve spatial photography and video processing based on existing technologies in order to provide high-quality spatial display videos. Summary of the Invention

[0008] In view of the above problems, embodiments of this application provide a video generation method, apparatus, and electronic device that overcomes or at least partially solves the above problems.

[0009] In a first aspect, embodiments of this application provide a video generation method, including:

[0010] A panoramic browsing animation is obtained by taking panoramic photos of a target space based on a spatial point browsing strategy, wherein the spatial point browsing strategy indicates the observation point movement path, the optimal rotation angle corresponding to the observation point shooting, and the shooting field of view corresponding to the observation point shooting.

[0011] The panoramic browsing animation is rendered on an adapted panoramic canvas, and frame images are extracted from the panoramic canvas.

[0012] The extracted batch of frame images are combined into a video at the target frame rate to obtain a playback video corresponding to the target frame rate; wherein, the playback video is played on a playback device whose device refresh rate matches the target frame rate.

[0013] Secondly, embodiments of this application provide a video generation apparatus, including:

[0014] The acquisition module is used to acquire panoramic browsing animations obtained by panoramic shooting of the target space based on a spatial point browsing strategy. The spatial point browsing strategy indicates the movement path of the observation point, the optimal rotation angle corresponding to the shooting of the observation point, and the shooting field of view corresponding to the shooting of the observation point.

[0015] The rendering and extraction module is used to render the panoramic browsing animation on an adapted panoramic canvas and to extract frame images from the panoramic canvas.

[0016] The processing module is used to synthesize the extracted batch of frame images into a video at a target frame rate, and obtain a playback video corresponding to the target frame rate; wherein, the playback video is played on a playback device whose device refresh rate matches the target frame rate.

[0017] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the video generation method as described in the first aspect above.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the video generation method described in the first aspect above.

[0019] The technical solution of this application embodiment, by performing panoramic shooting of the target space according to a pre-set spatial point browsing strategy, can shoot according to a reasonable planned path and shoot at different observation points in a specified manner, thereby improving shooting efficiency while obtaining complete shooting data and ensuring shooting quality; after obtaining panoramic browsing animation based on panoramic shooting, the panoramic browsing animation is rendered on an adapted panoramic canvas, and frame images are extracted from the panoramic canvas. The extracted frame images are synthesized into a video at a target frame rate, and the video is played on a playback device whose refresh rate matches the target frame rate, which can avoid screen tearing, reduce jitter and flicker, ensure the smoothness of the video picture, and accurately display the video content. Attached Figure Description

[0020] Figure 1A schematic diagram illustrating the video generation method provided in an embodiment of this application;

[0021] Figure 2 This diagram illustrates an overall implementation flowchart of the video generation method provided in this application.

[0022] Figure 3 This is a schematic diagram illustrating the video generation apparatus provided in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the electronic device structure provided in the embodiments of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Multiple embodiments in this application may include two or more.

[0026] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0027] This application provides a video generation method, which is applied to a terminal device, such as a PC (Personal Computer) device. Figure 1 As shown, it includes:

[0028] Step 101: Obtain the panoramic browsing animation obtained by taking panoramic photos of the target space based on the spatial point browsing strategy. The spatial point browsing strategy indicates the movement path of the observation point, the optimal rotation angle corresponding to the observation point shooting, and the shooting field of view corresponding to the observation point shooting.

[0029] Based on a pre-generated spatial point browsing strategy for the target space, a panoramic view of the target space is captured using a camera to obtain a corresponding panoramic browsing animation. Furthermore, when acquiring the panoramic browsing animation, a virtual VR scene of the target space needs to be displayed in the front-end runtime environment, captured using a camera (virtual camera). Specifically, a virtual VR scene can be generated in the front-end runtime environment based on the panoramic image. Through digital technology, the internal structure, layout, and decoration style of the house are presented virtually using the panoramic image, constructing a realistic virtual space; then, a virtual camera is used to capture the presented virtual scene to obtain the panoramic browsing animation of the target space.

[0030] The spatial point-based browsing strategy created for the target space can indicate the movement path of the observation points in the target space, the optimal rotation angle for shooting at the corresponding observation points, and the corresponding field of view for shooting at the corresponding observation points. When using a virtual camera to perform panoramic shooting of the target space (the virtual VR scene of the target space) based on the spatial point-based browsing strategy, shooting is performed at different observation points according to the observation point movement path. At each observation point, the camera rotates to shoot according to the optimal rotation angle of that observation point, and the field of view is adjusted based on the field of view indicated by the shooting strategy before shooting to ensure that the shooting is performed at a suitable field of view. Furthermore, during the movement of the virtual camera based on the observation point movement path, the virtual camera moves and shoots to capture the image content corresponding to the areas it passes through during the movement.

[0031] It should be noted that during the process of moving from one observation point to another, the rotation angle of the camera does not change. Therefore, when reaching the next observation point, the angle at which the previous observation point stopped rotating should be maintained. Since the size of the space captured by different observation points is different, the field of view needs to be adjusted. For example, when transitioning from a larger room to a smaller room, increasing the field of view can increase the field of view accordingly.

[0032] By taking panoramic photos of the target space according to a preset spatial point browsing strategy, it is possible to ensure that the shooting is carried out along a reasonable planned path, so as to obtain complete shooting data while ensuring shooting efficiency; and by taking photos at different observation points according to the specified strategy during the shooting process, the shooting quality can be guaranteed.

[0033] Step 102: Render the panoramic browsing animation on the adapted panoramic canvas, and extract frame images from the panoramic canvas.

[0034] After capturing a panoramic view of the target space according to the spatial point browsing strategy and obtaining a panoramic browsing animation of the target space, the panoramic browsing animation is rendered on an adapted panoramic canvas so that it can be displayed on the panoramic canvas through animation rendering.

[0035] In this stage, based on the captured panoramic browsing animation (raw shooting data), a dynamic canvas adaptation mechanism is used to select an appropriate panoramic canvas, and the panoramic browsing animation is rendered on the appropriate panoramic canvas to display high-quality virtual scene video.

[0036] After rendering the captured panoramic animation on a panoramic canvas, frame images are extracted from the canvas. Frame image extraction can be achieved using the canvas's built-in API (Application Programming Interface). APIs such as drawImage can be used to add other textures and annotation information. After each frame, the texture extraction is converted into a base64 format frame image using a method such as toDataURL, thus enabling the extraction of frame images from the panoramic canvas.

[0037] Step 103: Combine the extracted batch of frame images into a video at the target frame rate to obtain a playback video corresponding to the target frame rate; wherein, the playback video is played on a playback device whose device refresh rate matches the target frame rate.

[0038] After extracting frame images from the panoramic canvas, the extracted batch of frame images are composited at the target frame rate to obtain a playback video corresponding to the target frame rate. For video compositing, FFmpeg, an open-source multimedia processing framework, is used. FFmpeg can composite batches of images into video at a specified frame rate.

[0039] While limitations of shooting equipment prevent an animation from executing a specific number of frames within a specified time, it's possible to execute a specified number of frames and then place those frames within a specified timeframe to obtain a video with a specified frame rate. For example, if a 1-second 60-frame animation is needed, instead of binding the animation's execution progress to time, it can be bound to the number of frames executed. After the animation executes 60 frames, the video is composited. Even if the actual execution time is two seconds, the 60 frames from those two seconds can be combined into a 1-second video, thus achieving control over the video's frame rate. Based on this idea, a batch of extracted frame images can be composited into a video at the target frame rate to obtain a playback video corresponding to the target frame rate. The resulting playback video corresponding to the target frame rate is played on a playback device with a refresh rate matching the target frame rate, ensuring video playback quality, avoiding stuttering, tearing, and reducing jitter and flickering, thereby improving the user's visual experience.

[0040] By determining the target frame rate for video playback based on the device refresh rate, and then performing video synthesis based on the target frame rate, video playback can be performed in an optimal state where the device refresh rate and the video frame rate match, thus bringing a smooth and complete visual experience.

[0041] It should be noted that in this embodiment, acquiring the panoramic browsing animation, rendering the panoramic browsing animation on an adapted panoramic canvas, and extracting frame images from the panoramic canvas are all performed in the front-end runtime environment; while storing the extracted batch of frame images and performing video synthesis at the target frame rate are performed in the back-end runtime environment. For example, the front-end runtime environment is a PC browser, and the back-end runtime environment is a PC host. The front-end uses the browser's rendering capabilities to render the acquired panoramic browsing animation and extract frame images, while the back-end is mainly responsible for data storage and processing (storing frame images and video synthesis), utilizing powerful computing and storage capabilities to complete complex data processing tasks; this division of labor can fully leverage the advantages of the front-end and back-end, improving overall performance and user experience.

[0042] The above-described implementation scheme of this application, by performing panoramic shooting of the target space according to a pre-set spatial point browsing strategy, can shoot according to a reasonable planned path and shoot at different observation points in a specified manner, thereby improving shooting efficiency while obtaining complete shooting data and ensuring shooting quality. After obtaining panoramic browsing animation based on panoramic shooting, the panoramic browsing animation is rendered on an adapted panoramic canvas, and frame images are extracted from the panoramic canvas. The extracted frame images are synthesized into a video at the target frame rate, and the video is played on a playback device whose refresh rate matches the target frame rate. This can avoid screen tearing, reduce jitter and flicker, ensure the smoothness of the video picture, and accurately display the video content.

[0043] The process of generating a spatial point browsing strategy for the target space is described below. Optionally, the method further includes:

[0044] In response to determining the shooting starting point of the target space, the optimal browsing route is determined from the N browsing routes corresponding to the target space, where N is an integer greater than or equal to 1;

[0045] Based on the optimal browsing route, determine the jump observation point between the previous subspace and the next subspace that connects continuous shooting. The target space includes at least one subspace.

[0046] In each subspace of the target space, the optimal observation point is determined based on the field of view of the observation point, the rotation angle corresponding to the shooting at the observation point, and the adjustment method of the field of view angle corresponding to the shooting at the observation point.

[0047] A spatial point browsing strategy is generated based on the optimal browsing route, the determined jump observation points, the determined optimal observation points for each subspace, and the shooting strategy corresponding to the optimal observation points.

[0048] After determining the shooting starting point of the target space, such as the starting point of the entrance space, all browsing routes are traversed through the spatial mapping table of the target space to determine the optimal browsing route. In this embodiment, the spatial mapping table is a data structure used to describe the connection relationships and path information between various subspaces within the target space. The number of all browsing routes (e.g., N browsing routes) corresponding to the target space is greater than or equal to 1. By determining the optimal browsing route among the N browsing routes, the most reasonable route through the target space can be planned to ensure shooting efficiency and the integrity of the shooting data.

[0049] After determining the optimal browsing route, jump observation points are identified based on this route to connect the previous and next subspaces in continuous shooting. The path relationships between subspaces are already determined based on the spatial mapping table. However, each subspace will have one or more observation points. When moving from the current subspace to the next, the observation point closest to the connection point between the current and next subspaces is selected as the jump point. If the next subspace to be visited is not connected to the current subspace, an observation point in the transition subspace is used. Subspaces that have already been observed are marked as observed and cannot be observed again; they can only be used as transition subspaces.

[0050] Within each subspace of the target space, the optimal observation point needs to be determined based on its field of view, the required rotation angle for shooting at that point, and the corresponding field-of-view adjustment method. Simultaneously, the shooting strategy for that observation point can be determined based on its corresponding shooting rotation angle and field-of-view adjustment method.

[0051] After determining the optimal browsing route, jump observation points, and the optimal observation points (carrying the shooting strategies for each observation point) for each subspace, a spatial point browsing strategy for the target space can be generated. When shooting the virtual scene of the target space, the virtual camera will shoot according to this spatial point browsing strategy, improving shooting efficiency while obtaining complete shooting data and ensuring shooting quality. The generation of the spatial point browsing strategy for the target space is completed on the backend. The backend can pre-generate scripts, and the virtual camera will shoot according to the pre-generated spatial point browsing strategy.

[0052] Optionally, when determining the optimal browsing route among the N browsing routes corresponding to the target space, the following steps are included:

[0053] After determining the shooting starting point of the target space, N browsing routes are determined based on the spatial relationship of the subspaces included in the target space;

[0054] The route weight of each browsing route is determined based on route length, subspace importance level, and additional factors for related route selection.

[0055] The optimal browsing route is determined based on the route weights corresponding to the N browsing routes.

[0056] After determining the shooting starting point of the target space, when traversing the browsing route, the route weight is calculated for each browsing route. The route length, subspace importance, and other logic of expanding related route selection are comprehensively considered to determine the route weight of each browsing route. Based on the route weights corresponding to the N browsing routes, the highest route weight is determined, and then the best browsing route is selected.

[0057] When calculating weights based on route length, shorter routes receive higher weights to ensure that shorter shooting routes are prioritized and to reduce movement time during shooting. When calculating weights based on subspace importance, weights are assigned according to the function and importance of the subspace. For example, important subspaces must be visited; if an important subspace is missed, an additional negative weight is applied. Furthermore, important subspaces have lower weights if they are not prioritized for access, and higher weights if they are prioritized for access, ensuring that important subspaces appear first in the browsing route and are prioritized for shooting.

[0058] In route selection, in addition to considering route length and subspace importance, additional factors related to route selection, such as shooting comfort, also need to be considered. These additional factors can be quantified by introducing corresponding weight coefficients and ultimately affect route selection.

[0059] As an example, when considering shooting comfort, a comfort factor C can be introduced. The higher the comfort factor, the higher the weight. The comfort factor C can be determined based on factors such as the smoothness of the route or other factors.

[0060] After comprehensively considering route length, subspace importance, and other additional factors related to route selection, the weights of each dimension are summed to determine the route weight (total weight), and then the route with the highest route weight is selected as the best browsing route.

[0061] The above method for determining the optimal browsing route comprehensively considers route length, subspace importance, and additional factors related to route selection, ensuring that the selected browsing route is both efficient and meets actual needs.

[0062] Optionally, in each subspace of the target space, when determining the optimal observation point based on the field of view of the observation point, the rotation angle corresponding to the image taken at the observation point, and the adjustment method of the field of view angle corresponding to the image taken at the observation point, the following steps are included:

[0063] In the observation points corresponding to the subspace, for each observation point, a first weight is determined based on the field of view of the observation point, a second weight is determined based on the rotation angle required for shooting at the observation point, and a third weight is determined based on the field of view angle adjustment required for shooting at the observation point.

[0064] Based on the first weight and first coefficient, the second weight and second coefficient, and the third weight and third coefficient corresponding to the observation point, the comprehensive weight of the observation point is determined. The first coefficient, the second coefficient, and the third coefficient are the weight coefficients corresponding to the field of view range, rotation angle, and field of view adjustment method, respectively, and the first coefficient is greater than the second coefficient, and the second coefficient is greater than the third coefficient.

[0065] The optimal observation point in the subspace is determined based on the comprehensive weight of the observation points.

[0066] The process of entering a subspace for shooting can be seen as a subspace browsing process. During this process, you will look at the diagonal of the subspace to ensure that you see the largest area of ​​the subspace. After entering the subspace, you need to look at the connection port of the next subspace to be visited. The virtual camera rotates and shoots during this stage. Then, switch to the jump point to enter the next subspace. When switching subspaces, the field of view may need to be adjusted due to the change in the size of the subspace to ensure that more content is observed.

[0067] Based on the above logic, when determining the optimal observation point for a subspace, it is necessary to consider the field of view of the observation point, the rotation angle required for shooting at the observation point, and the adjustment method of the field of view angle required for shooting at the observation point. For any subspace, which corresponds to one or more observation points, for each observation point, the above three factors are considered: the first weight is determined based on the field of view of the observation point, the second weight is determined based on the rotation angle required for shooting at the observation point, and the third weight is determined based on the field of view angle adjustment required for shooting at the observation point.

[0068] The larger the field of view of the observation point, such as the longer the diagonal length that can be seen, the greater the first weight. When shooting from the viewpoint in the current subspace, the virtual camera needs to rotate to see the connection point of the next subspace. If the rotation angle is within a set range, the second weight is larger. Furthermore, the closer the rotation angle required for shooting at the observation point is to the optimal rotation angle, the greater the second weight. For example, if the optimal rotation angle is 90 degrees, the second weight gradually decreases as it deviates from the optimal rotation angle. When shooting in the current subspace, if the size of the current subspace differs significantly from the previous subspace, the field of view may be adjusted to ensure more content can be observed. For example, increasing the field of view when observing a small room to see a wider area. When the field of view does not need adjustment, the third weight is the largest, and the simpler the field of view adjustment, the greater the third weight.

[0069] After considering the three factors mentioned above, the overall weight of the observation point needs to be determined based on the respective weights of these three factors. This determination requires calculation according to the weight ratios of the field of view, rotation angle, and field-of-view adjustment method. Since the field of view of the observation point determines the framing range of the subspace, it has the largest weight ratio; the rotation angle has the second largest weight ratio, and the field-of-view adjustment method has the smallest weight ratio. The overall weight is calculated by assigning a weight coefficient to each factor.

[0070] The weighting coefficient corresponding to the field of view (first coefficient) is α, the weighting coefficient corresponding to the rotation angle (second coefficient) is β, and the weighting coefficient corresponding to the field of view adjustment method (third coefficient) is γ. Assuming the values ​​of α, β, and γ are 0.5, 0.3, and 0.2 respectively, these values ​​represent that the field of view accounts for 50% of the total weight, the rotation angle accounts for 30%, and the field of view adjustment method accounts for 20%. When calculating the overall weight of the observation point, the product of the first weight and the first coefficient, the product of the second weight and the second coefficient, and the product of the third weight and the third coefficient are calculated. The overall weight of the observation point is determined based on the sum of these products.

[0071] After determining the comprehensive weight of each observation point, the largest comprehensive weight is determined based on the comprehensive weight of each observation point, and the observation point corresponding to the comprehensive weight is determined as the best observation point. This process selects the best observation point after considering the weights of multiple dimensions and the weight ratio of each dimension.

[0072] Optionally, after generating a spatial point browsing strategy for the target space, when acquiring the panoramic browsing animation obtained by taking panoramic photos of the target space based on the spatial point browsing strategy, the process includes:

[0073] When the shooting device shoots the target space based on the optimal browsing route, the determined jump observation point, the determined optimal observation point of each subspace, and the shooting strategy corresponding to the optimal observation point, in response to the completion of the panoramic shooting of the target space, the panoramic browsing animation corresponding to the target space is obtained.

[0074] In each subspace, the camera at the optimal observation point captures images based on the field of view and rotation angle indicated by the shooting strategy.

[0075] When performing panoramic photography of a target space based on a spatial point-based browsing strategy, the shooting device (such as a virtual camera) starts shooting from the starting point along the optimal browsing route and switches between adjacent subspaces according to jump observation points to continuously shoot multiple subspaces along the planned path. During shooting in each subspace, at the optimal observation point of the subspace, the field of view is adjusted based on the corresponding adjustment method, and then rotated based on the corresponding rotation angle to turn the virtual camera toward the spatial connection point to the next subspace, thus completing the shooting of the current subspace.

[0076] In brief, after shooting from the starting point, the camera moves and shoots along a planned route. When switching subspaces, it moves to the next subspace based on a jump observation point, and then rotates to shoot from the optimal observation point in the next subspace. After completing the rotational shooting, it continues moving and shooting, switching subspaces again based on the jump observation point, and so on, until a panoramic shot of the target space is completed based on the spatial point-based browsing strategy. After the camera completes the panoramic shooting of the target space, a panoramic browsing animation corresponding to the target space can be obtained.

[0077] By shooting along a well-planned route and at different observation points in a specified manner, complete shooting data can be obtained while ensuring shooting efficiency and shooting quality.

[0078] The following describes a method for rendering captured panoramic browsing animations and extracting frame images from the rendered animations to synthesize a video at a specified frame rate. The process of rendering the panoramic browsing animation on a compatible panoramic canvas includes:

[0079] Adjust the size of the panoramic canvas based on the image size of the images captured during the panoramic shooting process;

[0080] Render a panoramic browsing animation on a panoramic canvas for animated presentation.

[0081] After obtaining a panoramic browsing animation based on panoramic photography, rendering software is used to render the panoramic browsing animation, and the rendered animation is output to a canvas for display, thereby realizing the visualization of space.

[0082] The canvas here is a panoramic canvas. Before displaying the animation based on the panoramic canvas, the canvas size is adjusted according to the image size of the images captured during the panoramic shooting process. For high-resolution images, a large canvas size is used to reduce the loss caused by extracting frame images. For low-resolution images, a small canvas size is used to reduce storage time and space while ensuring image quality. By reasonably adjusting the canvas size, a balance is struck between rendering quality and storage efficiency.

[0083] The process of synthesizing the extracted batch of frame images into a video at a target frame rate and obtaining the playback video corresponding to the target frame rate includes:

[0084] After extracting frame images from the panoramic canvas, the extracted frame images are saved based on time sequence.

[0085] The number of saved frame images is counted. In response to the number of saved frame images meeting the preset condition, the saved frame images are combined into a video to generate a sub-video corresponding to the target frame rate.

[0086] The playback video generated based on batch frame images includes multiple sub-videos corresponding to the target frame rate.

[0087] After extracting frame images from the panoramic canvas, the extracted frame images are stored in chronological order on the backend, such as by saving them to a specified folder. The number of extracted frame images is then counted. When the count reaches a preset number, the count is reset to zero, and the count of extracted frame images is restarted. This process is repeated until the preset number of frame images is reached again, and so on, to process the extracted frame images.

[0088] When the number of frame images reaches a preset number, the preset number of frame images are combined into a video to generate a sub-video corresponding to the target frame rate. For example, when the number of frame images reaches 60, a 1-second sub-video is synthesized based on 60 frame images to obtain a sub-video corresponding to the target frame rate.

[0089] It is important to note that, under normal circumstances, a preset number of frames are used to synthesize a sub-video. In special cases where the remaining frames are less than the preset number, a shorter sub-video corresponding to the target frame rate is synthesized. For example, a 1-second sub-video corresponding to the target frame rate is synthesized based on 60 frames, and a 1 / 2-second sub-video corresponding to the target frame rate is synthesized based on 30 frames, ensuring that each sub-video obtained corresponds to the target frame rate.

[0090] Based on the above video synthesis strategy, video synthesis can be performed on the extracted batch of frame images to obtain a playback video with a specified frame rate. This allows the video to be played on a playback device with a compatible refresh rate, fully utilizing the performance of the playback device and ensuring the best possible visual effect.

[0091] In this embodiment of the application, the target frame rate is not only adapted to the refresh rate of the playback device, but also belongs to the appropriate frame rate for video playback. By using the specified appropriate frame rate to play the video, the continuity of the picture can conform to the visual habits of the human eye, ensuring the smoothness of the picture and ensuring the integrity of details.

[0092] Optionally, each sub-video corresponds to a synthesis task, and multiple synthesis tasks are executed sequentially based on queue management. During the execution of the synthesis task, memory usage is identified; in response to the identification that memory usage exceeds a preset threshold, the execution of the synthesis task is paused.

[0093] When performing video synthesis based on extracted frame images, queue management is required. Each sub-video corresponds to a synthesis task, and multiple synthesis tasks are executed sequentially based on queue management. That is, after the synthesis of one sub-video is completed, the next synthesis task is pushed in to ensure that the synthesis of sub-videos is not processed in parallel, so as to avoid excessive consumption of CPU cores.

[0094] Furthermore, during the execution of the synthesis task, the system load can be detected in real time. When the memory usage exceeds a preset threshold (such as 80%), the queue will be automatically paused and the synthesis task will be stopped.

[0095] The above implementation scheme, by synthesizing video based on a specified frame rate and playing the video on a playback device whose refresh rate matches the specified frame rate, can avoid screen tearing, reduce jitter and flicker, ensure the smoothness of the video picture, accurately display the video content, and thus ensure the user's visual experience.

[0096] The video generation method provided in this application embodiment is described below through an overall implementation process, such as... Figure 2 As shown, it includes the following steps:

[0097] Step 201: Determine the optimal browsing route from among the N browsing routes corresponding to the target space.

[0098] Step 202: Based on the optimal browsing route, determine the jump observation point between the previous subspace and the next subspace for continuous shooting.

[0099] Step 203: In each subspace of the target space, determine the optimal observation point based on the field of view of the observation point, the rotation angle corresponding to the observation point, and the field of view adjustment method corresponding to the observation point.

[0100] Step 204: Generate a spatial point browsing strategy based on the optimal browsing route, the determined jump observation points, the determined optimal observation points for each subspace, and the shooting strategy corresponding to the optimal observation points.

[0101] Step 205: Obtain the panoramic browsing animation obtained by taking panoramic photos of the target space based on the spatial point browsing strategy.

[0102] Step 206: Render the panoramic browsing animation on the adapted panoramic canvas for animation display, and extract frame images from the panoramic canvas.

[0103] Step 207: Combine the extracted batch of frame images into a video at the target frame rate, obtain the playback video corresponding to the target frame rate, and play it on the adapted playback device.

[0104] In the above implementation process, after considering multiple factors to determine the optimal browsing route, a spatial point browsing strategy is further determined based on the optimal browsing route. This can provide high-quality browsing animation while ensuring shooting efficiency. By performing video synthesis based on a specified frame rate and playing the video on a playback device with a refresh rate that matches the specified frame rate, the smoothness of the video can be guaranteed and the user's visual experience can be improved.

[0105] This application provides a video generation device, such as... Figure 3 As shown, it includes:

[0106] The acquisition module 301 is used to acquire a panoramic browsing animation obtained by panoramic shooting of the target space based on a spatial point browsing strategy. The spatial point browsing strategy indicates the movement path of the observation point, the optimal rotation angle corresponding to the shooting of the observation point, and the shooting field of view corresponding to the shooting of the observation point.

[0107] The rendering extraction module 302 is used to render the panoramic browsing animation on an adapted panoramic canvas and to extract frame images from the panoramic canvas.

[0108] The processing module 303 is used to synthesize the extracted batch of frame images into a video at a target frame rate, and obtain a playback video corresponding to the target frame rate; wherein the playback video is played on a playback device whose device refresh rate matches the target frame rate.

[0109] Optionally, the device further includes:

[0110] The first determining module is used to determine the optimal browsing route from among the N browsing routes corresponding to the target space in response to determining the shooting starting point of the target space, where N is an integer greater than or equal to 1;

[0111] The second determining module is used to determine, based on the optimal browsing route, the jump observation point connecting the previous subspace and the next subspace for continuous shooting, wherein the target space includes at least one subspace.

[0112] The third determining module is used to determine the optimal observation point in each subspace of the target space based on the field of view of the observation point, the rotation angle corresponding to the observation point, and the field of view adjustment method corresponding to the observation point.

[0113] The generation module is used to generate the spatial point browsing strategy based on the optimal browsing route, the determined jump observation points, the determined optimal observation points of each subspace, and the shooting strategy corresponding to the optimal observation points.

[0114] Optionally, the first determining module includes:

[0115] The first determining submodule is used to determine N browsing routes based on the spatial positional relationship of the subspaces included in the target space after determining the shooting starting point of the target space;

[0116] The second determination submodule is used to determine the route weight of each browsing route based on route length, importance level of subspace, and additional factors related to route selection.

[0117] The third determining submodule is used to determine the optimal browsing route based on the route weights corresponding to the N browsing routes.

[0118] Optionally, the third determining module includes:

[0119] The fourth determination submodule is used to determine the first weight based on the field of view of the observation point, the second weight based on the rotation angle required for shooting at the observation point, and the third weight based on the field of view adjustment required for shooting at the observation point in the corresponding observation point in the subspace.

[0120] The fifth determination submodule is used to determine the comprehensive weight of the observation point based on the first weight and first coefficient, the second weight and second coefficient, and the third weight and third coefficient corresponding to the observation point. The first coefficient, the second coefficient, and the third coefficient are the weight coefficients corresponding to the field of view, rotation angle, and field of view adjustment method, respectively, and the first coefficient is greater than the second coefficient and the second coefficient is greater than the third coefficient.

[0121] The sixth determination submodule is used to determine the best observation point in the subspace based on the comprehensive weight of the observation points.

[0122] Optionally, the rendering extraction module is further configured to:

[0123] Adjust the size of the panoramic canvas based on the image size of the images captured during the panoramic shooting process;

[0124] The panoramic browsing animation is rendered on the panoramic canvas for animation display.

[0125] Optionally, the acquisition module is further configured to:

[0126] When the shooting device shoots the target space based on the optimal browsing route, the determined jump observation point, the determined optimal observation point of each subspace, and the shooting strategy corresponding to the optimal observation point, in response to the completion of the panoramic shooting of the target space, the panoramic browsing animation corresponding to the target space is obtained.

[0127] In each subspace, at the optimal observation point, the imaging device captures images based on the field of view and rotation angle indicated by the imaging strategy.

[0128] Optionally, the processing module includes:

[0129] The first processing submodule is used to extract frame images from the panoramic canvas and then save the extracted frame images in chronological order.

[0130] The second processing submodule is used to count the saved frame images. In response to the number of saved frame images meeting the preset conditions, the saved frame images are used to synthesize video and generate a sub-video corresponding to the target frame rate.

[0131] The playback video generated based on batch frame images includes multiple sub-videos corresponding to the target frame rate.

[0132] Optionally, each sub-video corresponds to a compositing task, and multiple compositing tasks are executed sequentially based on queue management. The apparatus further includes:

[0133] The identification module is used to identify memory usage during the execution of the synthesis task;

[0134] The control module is used to suspend the execution of the synthesis task in response to the detection that memory usage exceeds a preset threshold.

[0135] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0136] This application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described video generation method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0137] For example, Figure 4 A schematic diagram of the physical structure of an electronic device is shown. (For example...) Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions stored in the memory 430. The processor 410 is used to execute various processes of the video generation method according to the embodiments of this application, which will not be further elaborated here.

[0138] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0139] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described video generation method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0140] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0142] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0143] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0145] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0146] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0148] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0149] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video generation method, characterized in that, include: When the shooting device captures a target space based on a spatial point browsing strategy, in response to the completion of panoramic shooting of the target space, a panoramic browsing animation corresponding to the target space is acquired. The target space includes at least one subspace. The spatial point browsing strategy is generated based on the optimal browsing route, the determined jump observation points of adjacent subspaces, the determined optimal observation points of each subspace, and the shooting strategy corresponding to the optimal observation points. The spatial point browsing strategy indicates the observation point movement path, the optimal rotation angle corresponding to the observation point shooting, and the shooting field of view corresponding to the observation point shooting. During panoramic shooting, shooting is performed at different observation points according to the observation point movement path. At the optimal observation point of each subspace, the shooting device captures images based on the field of view and rotation angle indicated by the shooting strategy. The panoramic browsing animation is rendered on an adapted panoramic canvas, and frame images are extracted from the panoramic canvas. The extracted batch of frame images are combined into a video at the target frame rate to obtain a playback video corresponding to the target frame rate; wherein, the playback video is played on a playback device whose device refresh rate matches the target frame rate.

2. The method according to claim 1, characterized in that, The method further includes: In response to determining the shooting starting point of the target space, the optimal browsing route is determined from the N browsing routes corresponding to the target space, where N is an integer greater than or equal to 1; Based on the optimal browsing route, determine the jump observation point between the previous subspace and the next subspace that connects continuous shooting; In each subspace of the target space, the optimal observation point is determined based on the field of view of the observation point, the rotation angle corresponding to the shooting at the observation point, and the field of view adjustment method corresponding to the shooting at the observation point.

3. The method according to claim 2, characterized in that, The step of determining the optimal browsing route from the N browsing routes corresponding to the target space in response to determining the shooting starting point of the target space includes: After determining the shooting starting point of the target space, N browsing routes are determined based on the spatial relationship of the subspaces included in the target space; The route weight of each browsing route is determined based on route length, subspace importance level, and additional factors for related route selection. The optimal browsing route is determined based on the route weights corresponding to the N browsing routes.

4. The method according to claim 2, characterized in that, In each subspace of the target space, the optimal observation point is determined based on the field of view of the observation point, the rotation angle corresponding to the image taken at the observation point, and the adjustment method of the field of view angle corresponding to the image taken at the observation point, including: In the observation points corresponding to the subspace, for each observation point, a first weight is determined based on the field of view of the observation point, a second weight is determined based on the rotation angle required for shooting at the observation point, and a third weight is determined based on the field of view angle adjustment required for shooting at the observation point. Based on the first weight and first coefficient, the second weight and second coefficient, and the third weight and third coefficient corresponding to the observation point, the comprehensive weight of the observation point is determined. The first coefficient, the second coefficient, and the third coefficient are the weight coefficients corresponding to the field of view range, rotation angle, and field of view adjustment method, respectively, and the first coefficient is greater than the second coefficient, and the second coefficient is greater than the third coefficient. The optimal observation point in the subspace is determined based on the comprehensive weight of the observation points.

5. The method according to claim 1, characterized in that, The step of rendering the panoramic browsing animation on an adapted panoramic canvas includes: Adjust the size of the panoramic canvas based on the image size of the images captured during the panoramic shooting process; The panoramic browsing animation is rendered on the panoramic canvas for animation display.

6. The method according to any one of claims 1 to 5, characterized in that, The step of synthesizing the extracted batch of frame images into a video at a target frame rate, and obtaining a playback video corresponding to the target frame rate, includes: After extracting frame images from the panoramic canvas, the extracted frame images are saved in chronological order. The number of saved frame images is counted. In response to the number of saved frame images meeting the preset condition, the saved frame images are combined into a video to generate a sub-video corresponding to the target frame rate. The playback video generated based on batch frame images includes multiple sub-videos corresponding to the target frame rate.

7. The method according to claim 6, characterized in that, Each sub-video corresponds to a compositing task, and multiple compositing tasks are executed sequentially based on queue management. The method further includes: During the synthesis task, memory usage is identified; In response to the detection that memory usage exceeds a preset threshold, the synthesis task is suspended.

8. A video generation apparatus, characterized in that, include: The acquisition module is used to acquire a panoramic browsing animation corresponding to the target space when the shooting device is shooting the target space based on the spatial point browsing strategy. The target space includes at least one subspace. The spatial point browsing strategy is generated based on the optimal browsing route, the determined jump observation points of adjacent subspaces, the determined optimal observation points of each subspace, and the shooting strategy corresponding to the optimal observation points. The spatial point browsing strategy indicates the observation point movement path, the optimal rotation angle corresponding to the observation point shooting, and the shooting field of view corresponding to the observation point shooting. During panoramic shooting, shooting is performed at different observation points according to the observation point movement path. At the optimal observation point of each subspace, the shooting device captures the image based on the field of view and rotation angle indicated by the shooting strategy. The rendering and extraction module is used to render the panoramic browsing animation on an adapted panoramic canvas and to extract frame images from the panoramic canvas. The processing module is used to synthesize the extracted batch of frame images into a video at a target frame rate, and obtain a playback video corresponding to the target frame rate; wherein, the playback video is played on a playback device whose device refresh rate matches the target frame rate.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the video generation method as described in any one of claims 1 to 7.

Images