Methods for camera movement in panoramic video, and electronic devices

Abstract

Disclosed in the present invention are methods for camera movement in a panoramic video, and electronic devices. A method for camera movement in a panoramic video comprises: acquiring, on the basis of interaction, at least one viewpoint of interest selected by a user within a panoramic video segment; and on the basis of the at least one viewpoint of interest, recommending at least one camera movement template to the user, wherein a camera movement video obtained on the basis of the at least one camera movement template has at least one camera movement effect and comprises the at least one viewpoint of interest.

Description

A panoramic video camera movement method and electronic device Technical Field

[0001] This application relates to the field of computer technology, and in particular to a panoramic video camera movement method and electronic device. Background Technology

[0002] In related technologies, panoramic video editing is usually done manually by the user. Keyframe editing mainly refers to adding keyframes on the timeline of the panoramic video and setting the playback angle of each keyframe. The operation steps are complicated and time-consuming, and the editing threshold is high, making it unsuitable for novice users. Summary of the Invention

[0003] This application provides a panoramic video camera movement method and an electronic device.

[0004] The technical solution of this application is implemented as follows:

[0005] This application provides a panoramic video camera movement method, the method comprising:

[0006] Based on interaction, obtain at least one point of interest selected by the user in the panoramic video clip;

[0007] Based on the at least one perspective of interest, at least one camera movement template is recommended to the user. The camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one perspective of interest.

[0008] This application provides another panoramic video camera movement method, which includes:

[0009] Identify at least one salient viewpoint in a panoramic video clip;

[0010] Based on the at least one salient viewpoint, at least one camera movement template is recommended to the user, and the camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one salient viewpoint.

[0011] This application provides another panoramic video camera movement method, which includes:

[0012] At least one camera movement editing point is displayed in the panoramic camera movement video, which is obtained by moving panoramic video segments based on a target camera movement template. The camera movement editing point is used to allow users to modify the camera movement angle of the corresponding panoramic video frame.

[0013] In response to the user's modification of the camera angle at the camera movement editing point, a modified panoramic camera movement video is generated based on the target camera movement template. The modified panoramic camera movement video includes the modified camera angle.

[0014] At least one embodiment of this application provides an electronic device, including:

[0015] The system includes a processor and a memory interconnected thereto. The memory is used to store a computer program. When the computer program is executed by the processor, the processor is configured to interactively acquire at least one viewpoint of interest selected by the user in a panoramic video clip; recommend at least one camera movement template to the user based on the at least one viewpoint of interest; and the camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one viewpoint of interest.

[0016] At least one embodiment of this application provides another electronic device, comprising:

[0017] The system includes a processor and a memory interconnected thereto, wherein the memory is used to store a computer program that, when executed by the processor, is configured to determine at least one salient viewpoint in a panoramic video clip; recommend at least one camera movement template to a user based on the at least one salient viewpoint; and generate a camera movement video based on the at least one camera movement template that has at least one camera movement effect and includes the at least one salient viewpoint.

[0018] At least one embodiment of this application provides another electronic device, comprising:

[0019] The system includes a processor and a memory interconnected thereto. The memory is used to store a computer program. When the computer program is executed by the processor, the processor is configured to display at least one camera movement editing point in a panoramic video. The panoramic video is obtained by moving panoramic video segments based on a target camera movement template. The camera movement editing point is used to allow users to modify the camera movement angle of the corresponding panoramic video frame.

[0020] In response to the user's modification of the camera angle at the camera movement editing point, a modified panoramic camera movement video is generated based on the target camera movement template. The modified panoramic camera movement video includes the modified camera angle.

[0021] This invention provides a computer-readable storage medium, comprising: the computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the panoramic video camera movement method provided in the first aspect of this invention.

[0022] This application embodiment obtains at least one viewpoint of interest selected by the user in a panoramic video clip through interactive means. Based on this viewpoint, at least one camera movement template is recommended to the user. The resulting video with camera movement based on the at least one camera movement template has at least one camera movement effect and includes at least one viewpoint of interest. In this solution, the camera movement template recommended based on the user's viewpoint of interest can produce a video with camera movement effects and at least one viewpoint of interest, increasing the richness of camera movements. This not only lowers the barrier for users to achieve complex camera movement effects but also better reflects the user's editing intentions by recommending camera movement templates based on the user's viewpoint of interest. Camera movement templates can help users achieve richer camera movements. Attached Figure Description

[0023] Figure 1 is a schematic diagram of the implementation process of a panoramic video camera movement method provided in an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of the region of interest in a panoramic video provided by an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of a camera movement editing point provided in an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of a panoramic video time period provided by an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of the implementation process of another panoramic video camera movement method provided in an embodiment of the present invention;

[0028] Figure 6 is a schematic diagram illustrating the implementation process of another panoramic video camera movement method provided in an embodiment of the present invention.

[0029] Figure 7 is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] Figure 1 is a schematic flowchart illustrating the implementation of a panoramic video camera movement method according to an embodiment of the present invention. The executing entity of the panoramic video camera movement method is an electronic device. Referring to Figure 1, the panoramic video camera movement method includes:

[0032] S101, based on interaction, obtain at least one viewpoint of interest selected by the user in the panoramic video clip.

[0033] The panoramic video clip can be a segment of the original panoramic video, and its specific duration can be defined by the user. The panoramic video data can be obtained by a panoramic camera, which consists of two or more fisheye lenses, and the resulting panoramic video is the original spherical video.

[0034] The viewpoint of interest is the perspective corresponding to the target or area of ​​interest selected by the user, which is also the user's preferred viewpoint. Users can specify one or more points of interest or areas of interest at different times in the panoramic video. Specifically, they can specify the viewpoint of interest by sliding the player or by selecting the viewpoint of interest by placing a box in the player.

[0035] Based on the user's interaction with the electronic device, at least one viewpoint of interest for the user within a panoramic video clip can be obtained. For example, if a user is interested in a particular scene within a panoramic video clip, the user can select a panoramic video frame containing that scene, and determine the corresponding viewpoint of interest based on the shooting angle of that scene within the panoramic video frame.

[0036] For example, a panoramic video frame may include multiple targets (including people, main characters, buildings, sculptures, landscapes, vehicles, etc.). Users can select targets of interest and determine the corresponding viewpoint of interest based on the shooting angle of the target of interest in the panoramic video frame.

[0037] S102, recommend at least one camera movement template to the user based on the at least one perspective of interest, and the camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one perspective of interest.

[0038] In some embodiments, multiple camera movement templates can be preset, and a camera movement template that matches at least one point of interest can be determined from among the preset multiple camera movement templates.

[0039] In some embodiments, a camera movement recommendation model can be pre-trained, with at least one point of interest as input to the camera movement recommendation model, and the camera movement template output by the camera movement recommendation model can be obtained.

[0040] After receiving at least one recommended camera movement template, the user can select the target camera movement template to move the panoramic video clip and obtain the video movement.

[0041] During camera movement, a camera movement trajectory matching at least one point of interest is first generated based on a camera movement template. Then, the panoramic video clip is moved according to the camera movement trajectory to obtain a panoramic video containing at least one point of interest.

[0042] A camera movement trajectory can represent the movement path of a virtual camera lens in three-dimensional space. The camera movement trajectory can be composed of multiple virtual camera poses (camera pose refers to the camera's position and orientation in space). The camera movement trajectory of a video can be understood as the trajectory of the lens movement during virtual camera shooting, i.e., the camera movement trajectory of the virtual camera in panoramic video. The camera movement trajectory consists of a series of trajectory points, each corresponding to a playback viewpoint. In this embodiment, the camera movement trajectory includes trajectory points corresponding to at least one viewpoint of interest.

[0043] The video footage generated based on the recommended camera movement templates has at least one camera movement effect and includes at least one point of interest. Each camera movement template is used to implement at least one camera movement effect, and different camera movement templates correspond to different camera movement effects.

[0044] In one embodiment, a camera movement template corresponds to a camera movement effect. When the camera movement template is applied to a panoramic video, the resulting panoramic camera movement video has the corresponding camera movement effect and also includes one or more user-specified viewpoints of interest.

[0045] This application embodiment obtains at least one viewpoint of interest selected by the user in a panoramic video clip through interactive means. Based on this viewpoint, at least one camera movement template is recommended to the user. The resulting video with camera movement based on the at least one camera movement template has at least one camera movement effect and includes at least one viewpoint of interest. In this solution, the camera movement template recommended based on the user's viewpoint of interest can produce a video with camera movement effects and at least one viewpoint of interest, increasing the richness of camera movements. This not only lowers the barrier for users to achieve complex camera movement effects but also better reflects the user's editing intentions by recommending camera movement templates based on the user's viewpoint of interest. Camera movement templates can help users achieve richer camera movements.

[0046] In one embodiment, the step of interactively acquiring at least one viewpoint of interest selected by the user in a panoramic video clip includes:

[0047] Obtain at least one point of interest or region of interest selected by the user at any point in time within the panoramic video clip;

[0048] At least one interest perspective is determined based on the selected point of interest or region of interest at the at least one time point.

[0049] In this context, a single point in time can correspond to a panoramic video frame within a panoramic video segment. By interacting with electronic devices, the system can retrieve the panoramic video frame selected by the user, and further retrieve the points of interest or regions of interest chosen within that frame. For example, a user can specify a panoramic video frame by sliding their finger across the player, and then select points of interest or regions of interest using methods such as box selection within the player.

[0050] As shown in Figure 2, users can select a region of interest in the panoramic video. The dashed rectangular area in Figure 2 is the region of interest selected by the user, which includes target 1.

[0051] Based on the location of the user-selected point of interest or region of interest within the panoramic video frame, the corresponding viewpoint can be determined.

[0052] In one embodiment, recommending at least one camera movement template to the user based on the at least one point of interest includes:

[0053] Use the at least one point of interest as input to the camera movement recommendation model;

[0054] Alternatively, the at least one point of interest and several panoramic video frames can be used as input to the camera movement recommendation model, wherein the several panoramic video frames are extracted from the panoramic video clips.

[0055] Obtain at least one camera movement template output by the camera movement recommendation model.

[0056] The camera movement recommendation model can be implemented and trained using methods such as Convolutional Neural Network (CNN) or Transformer. After training to obtain a usable camera movement recommendation model, at least one viewpoint of interest is input into the camera movement recommendation model, or at least one viewpoint of interest and several panoramic video frames are input into the camera movement recommendation model to obtain the camera movement template output by the camera movement recommendation model.

[0057] Here, several extracted panoramic video frames, along with at least one point of interest, can be used as input to the camera movement recommendation model. The camera movement recommendation model outputs a camera movement template that matches at least one point of interest, based on the panoramic video frames and at least one point of interest.

[0058] The number of panoramic video frames extracted from the panoramic video can be a pre-set number, such as 10 frames, which means 10 panoramic video frames can be randomly extracted from the panoramic video.

[0059] Using the extracted panoramic video frames and at least one point of interest as input to the camera movement recommendation model can make the output of the camera movement recommendation model more consistent with the current panoramic video clip.

[0060] In one embodiment, the camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one point of interest viewpoint and the object attributes it contains; or

[0061] The camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one point of interest viewpoint, the object attributes it contains, and the scene information contained in the several panoramic video frames.

[0062] For example, a viewpoint of interest is defined as pitch = 45°, yaw = 180°, where yaw refers to the horizontal angle of the lens and pitch refers to the vertical angle. This means that in a panoramic video frame, the viewpoint of interest is oriented towards the position where pitch = 45°, yaw = 180°.

[0063] For example, the camera movement trajectory corresponding to the camera movement template includes multiple trajectory points, and each trajectory point corresponds to the position of a target playback viewpoint. Therefore, when the camera movement recommendation model outputs the camera movement template, it can match the position of at least one viewpoint of interest with the trajectory points on the camera movement trajectory of the camera movement template. If the position of at least one viewpoint of interest can find a corresponding trajectory point on the camera movement trajectory of the camera movement template, then the two are considered to be matched.

[0064] Object attributes refer to the attributes of the target object corresponding to the viewpoint of interest in a panoramic video frame. For example, object attributes can include target type (target type can be a person, main character, building, sculpture, landscape, vehicle, etc.). The keyframes of the camera movement template can include attribute information, which can be the target type (target type can be a person, main character, building, sculpture, landscape, vehicle, etc.). Therefore, when outputting the camera movement recommendation model, it can refer to the object attributes contained in at least one viewpoint of interest to output a matching camera movement template.

[0065] In one embodiment, the method further includes:

[0066] Determine the target camera movement template from the at least one camera movement template;

[0067] The panoramic video clip is processed based on the target camera movement template to obtain a corresponding target camera movement video. The target camera movement video includes camera movement effects corresponding to the target camera movement template and includes at least one point of interest.

[0068] Users can choose a target camera movement template from at least one template, or they can be presented with at least one recommended template sorted by recommendation confidence level, or the template with the highest confidence level can be directly selected as the target camera movement template. This saves users time in selecting camera movement templates and helps them quickly obtain the best panoramic video camera movement effect.

[0069] In one embodiment, the camera movement template includes: multiple keyframes, each keyframe corresponding to position information, viewpoint information, and velocity information between adjacent keyframes;

[0070] The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes:

[0071] The panoramic video keyframes corresponding to the panoramic video segment are determined based on the position information corresponding to each keyframe in the target camera movement template.

[0072] The target playback angle corresponding to each panoramic video keyframe is determined based on the viewpoint information of each keyframe.

[0073] Based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one viewpoint of interest, the target playback viewpoint corresponding to each panoramic video frame is obtained.

[0074] The panoramic video is obtained based on the target playback viewpoint corresponding to each panoramic video frame and the speed information between adjacent keyframes.

[0075] The keyframe position information indicates the location of keyframes within a panoramic video clip. This position information can be represented by the frame number or a time point. For example, assuming a panoramic video clip comprises 100 frames, the position information of keyframe 1 in the camera movement template indicates that the 5th frame of the panoramic video clip is keyframe 1, and the position information of keyframe 2 in the camera movement template indicates that the 50th frame of the panoramic video clip is keyframe 2, and so on. If represented by time points, for example, the first panoramic video keyframe is located at time point 00:01:10, the second panoramic video keyframe is located at time point 00:20:00, and the third panoramic video keyframe is located at time point 01:40:00.

[0076] The viewpoint information of keyframes is used to indicate the target playback viewpoint of panoramic video keyframes in a panoramic video clip. For example, the viewpoint information of a keyframe includes pitch = 45° and yaw = 180°, indicating that the target playback viewpoint of the panoramic video keyframe is towards the position where pitch = 45° and yaw = 180°.

[0077] In addition, keyframes can also include attribute information, which can include the target type corresponding to the keyframe (target type can be a person, protagonist, building, sculpture, landscape, vehicle, etc.).

[0078] If a panoramic video keyframe happens to be the same panoramic video frame that corresponds to the viewpoint of interest, then the target playback viewpoint of the panoramic video keyframe is the viewpoint of interest.

[0079] In one embodiment, obtaining the target playback viewpoint corresponding to each panoramic video frame based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one viewpoint of interest includes:

[0080] Interpolation processing is performed on other panoramic video frames based on the target playback viewpoint corresponding to each panoramic video keyframe and the at least one viewpoint of interest to obtain the target playback viewpoint corresponding to each panoramic video frame.

[0081] Interpolation processing includes two methods: linear interpolation and spherical interpolation. For example, the target playback angles of panoramic video keyframe 1 and panoramic video keyframe 2 are known. To achieve a smooth transition between panoramic video keyframe 1 and panoramic video keyframe 2, the target playback angle corresponding to each panoramic video frame between keyframe 1 and panoramic video keyframe 2 can be calculated using linear or spherical functions based on the target playback angles of panoramic video keyframe 1 and panoramic video keyframe 2.

[0082] In one embodiment, the step of interpolating other panoramic video frames based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one viewpoint of interest to obtain the target playback viewpoint corresponding to each panoramic video frame includes:

[0083] Obtain the position information corresponding to the target playback viewpoint of each panoramic video keyframe.

[0084] Based on the position information corresponding to the target playback viewpoint of two adjacent panoramic video keyframes, interpolation processing is performed on the panoramic video frame located between the two panoramic video keyframes to obtain the target playback viewpoint corresponding to each intermediate panoramic video frame.

[0085] The positional information corresponding to the target playback viewpoint can include parameters such as horizontal angle (yaw), vertical angle (pitch), rotation angle (roll), field of view (Fov), and projection distance. The field of view refers to the angle formed by the two edges of the maximum range of the image after the panoramic video is projected; the projection distance refers to the distance from the center of the panoramic video projection to the projection surface.

[0086] For example, the target playback viewpoint of panoramic video keyframe 1 corresponds to pitch = 45° and yaw = 180°, while the target playback viewpoint of panoramic video keyframe 2 corresponds to pitch = -45° and yaw = 180°. Based on the position information of the target playback viewpoints of panoramic video keyframes 1 and 2, interpolation processing is performed on the panoramic video frames between them to obtain the target playback viewpoint for each intermediate panoramic video frame, thus achieving a smooth transition between panoramic video keyframes 1 and 2.

[0087] Based on the target playback angle corresponding to the panoramic video frame and the speed information between adjacent keyframes, the panoramic video clip is played to obtain a panoramic camera movement video. The speed between adjacent keyframes can be set using a Bezier curve, which can be set to a constant speed or a continuously changing speed.

[0088] In one embodiment, the method further includes:

[0089] Obtain a user-defined camera movement template and store the camera movement template in the user camera movement library. The camera movement template includes: the positions of multiple keyframes, the viewpoint corresponding to each keyframe, and the speed between adjacent keyframes.

[0090] By allowing users to customize camera movement templates, users can set up camera movement templates according to their own preferences. In addition, when users see a good camera movement pattern, they can save it as their own template for later use.

[0091] When recommending camera movement templates, a camera movement template that matches at least one point of interest can be determined from the user's camera movement library.

[0092] In one embodiment, the step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes:

[0093] The camera movement trajectory of the panoramic video clip is generated based on the target camera movement template. The camera movement trajectory includes trajectory points corresponding to the at least one viewpoint of interest. The panoramic video clip is played based on the camera movement trajectory to obtain the corresponding target camera movement video.

[0094] A panoramic video frame includes footage from multiple shooting angles. The playback angle of each panoramic video frame can be determined based on the camera movement trajectory. Playing panoramic video segments based on the camera movement trajectory yields the corresponding panoramic video. In this embodiment, the camera movement trajectory includes at least one trajectory point corresponding to a viewpoint of interest. The resulting panoramic video includes at least one viewpoint of interest, thus highlighting the content within that viewpoint and making the video more aligned with user preferences and expectations.

[0095] In one embodiment, the method further includes:

[0096] At least one camera movement editing point is displayed in the obtained target camera movement video; the camera movement editing point is used to allow the user to modify the camera movement angle of the corresponding panoramic video frame;

[0097] In response to the user's modification of the camera angle at the camera movement editing point, the camera movement trajectory is regenerated based on the target camera movement template, so that the panoramic video segment is moved according to the regenerated camera movement trajectory to obtain the modified target camera movement video; the modified target camera movement video includes: the modified camera angle.

[0098] For example, a user can select a camera movement editing point and modify the camera angle at that point. In response to the user's modification of the camera angle at the editing point, a new camera movement trajectory is generated based on the camera movement template. This regenerated trajectory is then used to move the panoramic video clip, resulting in a modified panoramic video. The modified panoramic video includes the modified camera angle. By reserving camera movement editing points, users can fine-tune the video movement, ensuring the adjusted effect better meets their expectations.

[0099] As shown in Figure 3, there are multiple camera movement editing points. Users can select one of these camera movement editing points and modify the target playback angle of the corresponding panoramic video frame at the camera movement editing point. Then, the camera movement video is regenerated based on the modified target playback angle.

[0100] In one embodiment, the camera movement template includes: standard camera movement duration and standard camera movement speed;

[0101] The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes:

[0102] Determine the actual duration corresponding to the panoramic video segment;

[0103] The actual camera movement speed is determined based on the standard camera movement duration and the actual duration.

[0104] The panoramic video clip is processed according to the actual camera movement speed to obtain the target camera movement video.

[0105] For example, the standard camera movement template has a standard camera movement duration of 1 minute and a standard camera movement speed of 1. If the actual duration of the panoramic video clip is 2 minutes, then the actual camera movement speed is 2. Based on the actual duration of the panoramic video clip, the actual camera movement speed is adaptively adjusted to match the actual duration, thus improving the camera movement effect.

[0106] In one embodiment, the method further includes:

[0107] In response to user adjustments to the panoramic video clip, a camera movement trajectory is automatically generated for the adjusted panoramic video clip based on the target camera movement template. The adjustments to the panoramic video clip include at least one of adding video clips or removing video clips.

[0108] After the camera movement is complete, users can adjust the panoramic video clip. For example, they can move the entire camera movement range forward or backward, such as shifting the selected panoramic video clip from 10s-20s to 12s-22s. This is equivalent to adding or removing a portion of the video clip. Alternatively, users can add or remove only video clips. Based on the target camera movement template, the system automatically generates a camera movement trajectory for the adjusted panoramic video clip, and plays the adjusted panoramic video according to the trajectory to obtain the adjusted panoramic video.

[0109] In one embodiment, the method further includes:

[0110] Obtain the time period selected by the user from the panoramic video, and determine the panoramic video segment based on the time period; or

[0111] Obtain the start or end time point selected by the user from the panoramic video, and determine the panoramic video segment based on the duration of the selected camera movement template.

[0112] For example, as shown in Figure 4, users can select a time period from 00:58 to 01:21 from the panoramic video to obtain panoramic video clips within that time period.

[0113] Alternatively, users can select a start time point or an end time point, and determine the panoramic video segment based on the duration of the target camera movement template. For example, if the user selects a start time point of 00:58 and the target camera movement template duration is 50 seconds, then the end time point is determined to be 01:48, and a panoramic video segment from 00:58 to 01:48 is obtained.

[0114] Figure 5 is a schematic flowchart illustrating the implementation of another panoramic video camera movement method provided in an embodiment of the present invention. The executing entity of the panoramic video camera movement method is an electronic device. Referring to Figure 5, the panoramic video camera movement method includes:

[0115] S501, Identify at least one salient viewpoint in the panoramic video clip.

[0116] A salient viewpoint is a visually striking and eye-catching perspective in a panoramic video clip, and is the editing perspective that users are most likely to select.

[0117] For example, in a user's skiing scenario, prominent perspectives include the skier, ski highlights, ski resort scenery, fellow skiers, and the direction of travel. In a diving scenario, prominent perspectives include the diver, underwater creatures, underwater scenery, diving highlights, and diving buddies. In a family-friendly scenario, prominent perspectives include cute babies, parent-child interactions, and family selfies.

[0118] For example, salient targets in panoramic video clips can be automatically identified, and the shooting angle corresponding to the identified salient target can be determined as the salient angle.

[0119] For example, in a skiing scenario, the system can identify the user's skiing highlights and use the camera angle corresponding to the highlights as the salient angle.

[0120] Alternatively, the user's chosen salient perspective can be obtained through interaction.

[0121] S502, recommend at least one camera movement template to the user based on the at least one salient viewpoint, and the camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one salient viewpoint.

[0122] In some embodiments, multiple camera movement templates can be preset, and a camera movement template that matches at least one salient viewpoint can be determined from among the preset multiple camera movement templates.

[0123] In some embodiments, a camera movement recommendation model can be pre-trained, and at least one salient viewpoint can be used as input to the camera movement recommendation model to obtain the camera movement template output by the camera movement recommendation model.

[0124] After receiving at least one recommended camera movement template, the user can select the target camera movement template to move the panoramic video clip and obtain the video movement.

[0125] During camera movement, a camera movement trajectory matching at least one salient viewpoint is first generated based on a camera movement template. Then, the panoramic video clip is moved according to the camera movement trajectory to obtain a panoramic video containing at least one salient viewpoint.

[0126] A camera movement trajectory can represent the movement path of a virtual camera lens in three-dimensional space. The camera movement trajectory can be composed of multiple virtual camera poses (camera pose being the camera's position and orientation in space). The camera movement trajectory of a video can be understood as the trajectory of the lens movement during virtual camera shooting, i.e., the camera movement trajectory of the virtual camera in panoramic video. The camera movement trajectory consists of a series of trajectory points, each corresponding to a playback viewpoint. In this embodiment, the camera movement trajectory includes trajectory points corresponding to at least one salient viewpoint.

[0127] The camera movement videos generated based on the recommended camera movement templates have at least one camera movement effect and include at least one salient viewpoint. Each camera movement template is used to achieve at least one camera movement effect, and different camera movement templates correspond to different camera movement effects.

[0128] In one embodiment, a camera movement template corresponds to a camera movement effect. When the camera movement template is applied to a panoramic video, the resulting panoramic camera movement video has the corresponding camera movement effect and also includes one or more salient viewpoints specified by the user.

[0129] This application embodiment determines at least one salient viewpoint in a panoramic video clip and recommends at least one camera movement template to the user based on that salient viewpoint. The resulting camera movement video, based on the at least one camera movement template, has at least one camera movement effect and includes at least one salient viewpoint. In this solution, the camera movement template recommended based on the salient viewpoint in the panoramic video clip can produce a camera movement video with camera movement effects and including at least one salient viewpoint, increasing the richness of camera movements. This not only lowers the barrier for users to achieve complex camera movement effects but also, by recommending camera movement templates based on salient viewpoints, better expresses the user's editing intentions. Camera movement templates can help users achieve richer camera movements.

[0130] In one embodiment, the method further includes:

[0131] Obtain the time period selected by the user from the panoramic video, and determine the panoramic video segment based on the time period; or

[0132] Obtain the start or end time point selected by the user from the panoramic video, and determine the panoramic video segment based on the duration of the selected camera movement template.

[0133] For example, as shown in Figure 4, users can select a time period from 00:58 to 01:21 from the panoramic video to obtain panoramic video clips within that time period.

[0134] Alternatively, users can select a start time point or an end time point, and determine the panoramic video segment based on the duration of the target camera movement template. For example, if the user selects a start time point of 00:58 and the target camera movement template duration is 50 seconds, then the end time point is determined to be 01:48, and a panoramic video segment from 00:58 to 01:48 is obtained.

[0135] In one embodiment, determining at least one salient viewpoint in a panoramic video segment includes:

[0136] Identify at least one salient target in the panoramic video segment;

[0137] Determine at least one salient perspective corresponding to the at least one salient objective.

[0138] For example, a salient target analysis can be performed on a panoramic video clip to obtain at least one salient target in the panoramic video clip, and the camera angle corresponding to the salient target can be taken as the salient angle.

[0139] For example, a salient viewpoint is pitch = 45°, yaw = 180°, where yaw refers to the horizontal angle of the lens and pitch refers to the vertical angle of the lens, indicating that the lens is facing a position where pitch = -45°, yaw = 180°.

[0140] In one embodiment, determining at least one salient viewpoint corresponding to the at least one salient target includes:

[0141] Determine the panoramic video frame of the panoramic video segment corresponding to the at least one salient target;

[0142] At least one salient viewpoint is determined based on the position of the at least one salient target in the panoramic video frame.

[0143] Identify panoramic video frames containing salient targets, and then determine the salient viewpoint based on the target's position within the panoramic video frames. Here, the target's position within the panoramic video frames refers to the camera's viewpoint.

[0144] In one embodiment, recommending at least one camera movement template to the user based on the at least one salient viewpoint includes:

[0145] Use the at least one salient viewpoint as input to the camera movement recommendation model;

[0146] Alternatively, the at least one salient viewpoint and several panoramic video frames can be used as input to the camera movement recommendation model, wherein the several panoramic video frames are extracted from the panoramic video clip.

[0147] Obtain at least one camera movement template output by the camera movement recommendation model.

[0148] The camera movement recommendation model can be implemented and trained using methods such as Convolutional Neural Network (CNN) or Transformer. After training to obtain a usable camera movement recommendation model, at least one salient viewpoint is input into the camera movement recommendation model, or at least one salient viewpoint and several panoramic video frames are input into the camera movement recommendation model to obtain the camera movement template output by the camera movement recommendation model.

[0149] Here, several extracted panoramic video frames and at least one salient viewpoint can be used as input to the camera movement recommendation model. The camera movement recommendation model outputs a camera movement template that matches the at least one salient viewpoint based on the panoramic video frames and the at least one salient viewpoint.

[0150] The number of panoramic video frames extracted from the panoramic video can be a pre-set number, such as 10 frames, which means 10 panoramic video frames can be randomly extracted from the panoramic video.

[0151] Using the extracted panoramic video frames and at least one salient viewpoint as input to the camera movement recommendation model can make the output of the camera movement recommendation model more consistent with the current panoramic video clip.

[0152] In one embodiment, the camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one salient viewpoint and the object attributes it contains; or

[0153] The camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one salient viewpoint, the object attributes it contains, and the scene information contained in the several panoramic video frames.

[0154] For example, a salient viewpoint is located at pitch = 45° and yaw = 180°.

[0155] For example, the camera movement trajectory corresponding to the camera movement template includes multiple trajectory points, and each trajectory point corresponds to the position of a target playback viewpoint. Therefore, when the camera movement recommendation model outputs the camera movement template, it can match the position of at least one viewpoint of interest with the trajectory points on the camera movement trajectory of the camera movement template. If the position of at least one viewpoint of interest can find a corresponding trajectory point on the camera movement trajectory of the camera movement template, then the two are considered to be matched.

[0156] Object attributes refer to the attributes of the target object corresponding to the salient viewpoint in a panoramic video frame. For example, object attributes can include target type (target type can be a person, main character, building, sculpture, landscape, vehicle, etc.). The keyframes of the camera movement template can include attribute information, which can be the target type (target type can be a person, main character, building, sculpture, landscape, vehicle, etc.). Therefore, when outputting a camera movement template, the camera movement recommendation model can refer to the object attributes contained in at least one salient viewpoint to output a matching camera movement template.

[0157] In one embodiment, the method further includes:

[0158] Determine the target camera movement template from the at least one camera movement template;

[0159] The panoramic video clip is processed based on the target camera movement template to obtain the corresponding target camera movement video, which includes the at least one salient viewpoint.

[0160] Users can choose a target camera movement template from at least one template, or they can be presented with at least one recommended template sorted by recommendation confidence level, or the template with the highest confidence level can be directly selected as the target camera movement template. This saves users time in selecting camera movement templates and helps them quickly obtain the best panoramic video camera movement effect.

[0161] In one embodiment, the camera movement template includes: multiple keyframes, each keyframe corresponding to position information, viewpoint information, and velocity information between adjacent keyframes;

[0162] The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes:

[0163] The panoramic video keyframes corresponding to the panoramic video segment are determined based on the position information corresponding to each keyframe in the target camera movement template.

[0164] The target playback angle corresponding to each panoramic video keyframe is determined based on the viewpoint information of each keyframe.

[0165] Based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one salient viewpoint, the target playback viewpoint corresponding to each panoramic video frame is obtained.

[0166] The panoramic video is obtained based on the target playback viewpoint corresponding to each panoramic video frame and the speed information between adjacent keyframes.

[0167] The keyframe position information indicates the location of keyframes within a panoramic video clip. This position information can be represented by the frame number or a time point. For example, assuming a panoramic video clip comprises 100 frames, the position information of keyframe 1 in the camera movement template indicates that the 5th frame of the panoramic video clip is keyframe 1, and the position information of keyframe 2 in the camera movement template indicates that the 50th frame of the panoramic video clip is keyframe 2, and so on. If represented by time points, for example, the first panoramic video keyframe is located at time point 00:01:10, the second panoramic video keyframe is located at time point 00:20:00, and the third panoramic video keyframe is located at time point 01:40:00.

[0168] The viewpoint information of keyframes is used to indicate the target playback viewpoint of panoramic video keyframes in a panoramic video clip. For example, the viewpoint information of a keyframe includes pitch = 45° and yaw = 180°, indicating that the target playback viewpoint of the panoramic video keyframe is towards the position where pitch = 45° and yaw = 180°.

[0169] If a panoramic video keyframe is also a panoramic video frame corresponding to a salient viewpoint, then the target playback viewpoint of the panoramic video keyframe is the salient viewpoint.

[0170] In addition, keyframes can also include attribute information, which can include the target type corresponding to the keyframe (target type can be a person, protagonist, building, sculpture, landscape, vehicle, etc.).

[0171] In one embodiment, obtaining the target playback viewpoint corresponding to each panoramic video frame based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one salient viewpoint includes:

[0172] Interpolation processing is performed on other panoramic video frames based on the target playback viewpoint corresponding to each panoramic video keyframe and the at least one salient viewpoint to obtain the target playback viewpoint corresponding to each panoramic video frame.

[0173] Interpolation processing includes two methods: linear interpolation and spherical interpolation. For example, the target playback angles of panoramic video keyframe 1 and panoramic video keyframe 2 are known. To achieve a smooth transition between panoramic video keyframe 1 and panoramic video keyframe 2, the target playback angle corresponding to each panoramic video frame between keyframe 1 and panoramic video keyframe 2 can be calculated using linear or spherical functions based on the target playback angles of panoramic video keyframe 1 and panoramic video keyframe 2.

[0174] Based on the target playback angle corresponding to the panoramic video frame and the speed information between adjacent keyframes, the panoramic video clip is played to obtain a panoramic camera movement video. The speed between adjacent keyframes can be set using a Bezier curve, which can be set to a constant speed or a continuously changing speed.

[0175] In one embodiment, the step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes:

[0176] The camera movement trajectory of the panoramic video clip is generated based on the target camera movement template. The camera movement trajectory includes trajectory points corresponding to the at least one salient viewpoint. The panoramic video clip is played based on the camera movement trajectory to obtain the corresponding target camera movement video.

[0177] Based on the target camera movement template, the target playback viewpoint for each panoramic video frame of the panoramic video clip is determined. According to the target playback viewpoint of each panoramic video frame, a camera movement trajectory for the panoramic video clip can be generated. Playing the panoramic video clip based on the camera movement trajectory yields the corresponding panoramic camera movement video. In this embodiment, the camera movement trajectory includes trajectory points corresponding to at least one salient viewpoint, and the resulting target camera movement video includes at least one salient viewpoint, highlighting the content within that salient viewpoint, thereby making the camera movement video more aligned with user preferences and expectations.

[0178] In one embodiment, the method further includes:

[0179] At least one camera movement editing point is displayed in the obtained target camera movement video; the camera movement editing point is used to allow the user to modify the camera movement angle of the corresponding panoramic video frame;

[0180] In response to the user's modification of the camera angle at the camera movement editing point, the camera movement trajectory is regenerated based on the target camera movement template, so that the panoramic video segment is moved according to the regenerated camera movement trajectory to obtain the modified target camera movement video; the modified target camera movement video includes: the modified camera angle.

[0181] The modified camera angle is the target playback angle of the corresponding panoramic video frame.

[0182] For example, a user can select a camera movement editing point and modify the camera angle at that point. In response to the user's modification of the camera angle at the editing point, a new camera movement trajectory is generated based on the target camera movement template. This regenerated trajectory is then used to move the panoramic video clip, resulting in a modified panoramic video. The modified panoramic video includes the modified camera angle. By reserving camera movement editing points, users can fine-tune the video movement, ensuring the adjusted effect better meets their expectations.

[0183] As shown in Figure 3, there are multiple camera movement editing points. Users can select one of these camera movement editing points and modify the target playback angle of the corresponding panoramic video frame at the camera movement editing point. Then, the camera movement video is regenerated based on the modified target playback angle.

[0184] In one embodiment, the camera movement template includes: standard camera movement duration and standard camera movement speed;

[0185] The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes:

[0186] Determine the actual duration corresponding to the panoramic video segment;

[0187] The actual camera movement speed is determined based on the standard camera movement duration and the actual duration.

[0188] The panoramic video clip is processed according to the actual camera movement speed to obtain the target camera movement video.

[0189] For example, the standard camera movement template has a standard camera movement duration of 1 minute and a standard camera movement speed of 1. If the actual duration of the panoramic video clip is 2 minutes, then the actual camera movement speed is 2. Based on the actual duration of the panoramic video clip, the actual camera movement speed is adaptively adjusted to match the actual duration, thus improving the camera movement effect.

[0190] In one embodiment, the method further includes:

[0191] In response to user adjustments to the panoramic video clip, a camera movement trajectory is automatically generated for the adjusted panoramic video clip based on the target camera movement template. The adjustments to the panoramic video clip include either adding or removing video clips.

[0192] After the camera movement is complete, users can adjust the panoramic video clip. For example, they can move the entire camera movement range forward or backward, such as shifting the selected panoramic video clip from 10s-20s to 12s-22s. This is equivalent to adding or removing a portion of the video clip. Alternatively, users can add or remove only video clips. Based on the target camera movement template, the system automatically generates a camera movement trajectory for the adjusted panoramic video clip, and plays the adjusted panoramic video according to the trajectory to obtain the adjusted panoramic video.

[0193] The method further includes:

[0194] Obtain a user-defined camera movement template and store the camera movement template in the user camera movement library. The camera movement template includes: the positions of multiple keyframes, the viewpoint corresponding to each keyframe, and the speed between adjacent keyframes.

[0195] By allowing users to customize camera movement templates, users can set up camera movement templates according to their own preferences. In addition, when users see a good camera movement pattern, they can save it as their own template for later use.

[0196] When recommending camera movement templates, a camera movement template that matches at least one salient viewpoint can be determined from the user's camera movement library.

[0197] Figure 6 is a schematic flowchart illustrating another panoramic video camera movement method provided in an embodiment of the present invention, wherein the executing entity of the panoramic video camera movement method is an electronic device. Referring to Figure 6, the panoramic video camera movement method includes:

[0198] S601, Display at least one camera movement editing point in the panoramic camera movement video, wherein the panoramic camera movement video is obtained by moving panoramic video segments based on a target camera movement template, and the camera movement editing point is used to allow users to modify the camera movement angle of the corresponding panoramic video frame.

[0199] After performing camera movement on the panoramic video clip based on the target camera movement template to obtain a panoramic camera movement video, at least one camera movement editing point is displayed in the panoramic camera movement video. Each camera movement editing point corresponds to a panoramic video frame, and users are allowed to modify the camera movement angle of the corresponding panoramic video frame at the camera movement editing point.

[0200] In some embodiments, the number and location of camera movement editing points are predefined. For example, three camera movement editing points are predefined and evenly distributed in the panoramic camera movement video.

[0201] In some embodiments, the number and location of camera movement editing points can be defined by the target camera movement template.

[0202] S602, in response to the user's modification of the camera angle at the camera movement editing point, a modified panoramic camera movement video is generated based on the target camera movement template, the modified panoramic camera movement video including: the modified camera angle.

[0203] For example, a user can select a camera movement editing point, specifying the camera angle of the panoramic video frame at that point. In response to the user's modification of the camera angle at the editing point, a new camera movement trajectory is generated based on the target camera movement template. This regenerated trajectory is then used to move the panoramic video clip, resulting in a modified panoramic video, which includes the modified camera angle. By reserving camera movement editing points, users can fine-tune the video, ensuring the adjusted effect better meets their expectations.

[0204] As shown in Figure 3, there are multiple camera movement editing points. Users can select one of these camera movement editing points and modify the camera movement angle of the corresponding panoramic video frame at the camera movement editing point. Then, the camera movement video is regenerated based on the modified camera movement angle.

[0205] The modified camera angle is the target playback angle of the corresponding panoramic video frame.

[0206] In this embodiment, camera movement editing points are provided for users to modify the camera movement angle, improving the convenience of camera movement operations. If users are not satisfied with the camera movement effect of the panoramic video, they can modify the camera movement angle of the corresponding panoramic video frame through the camera movement editing points to fine-tune the camera movement effect of the panoramic video and help users achieve a more satisfactory camera movement.

[0207] In one embodiment, the step of generating a modified panoramic video based on the camera movement template in response to a user's modification of the camera movement angle at the camera movement editing point includes:

[0208] Get the camera angle that the user reselects at the camera editing point;

[0209] The camera movement trajectory is adjusted based on the reselected camera angle so that the generated panoramic video includes the modified camera angle.

[0210] Each camera movement editing point corresponds to a panoramic video frame. The camera movement angle that the user reselects at the camera movement editing point is the target playback angle of the corresponding panoramic video frame. The camera movement trajectory is adjusted according to the modified camera movement angle so that the generated panoramic video movement includes the modified camera movement angle.

[0211] If the camera movement editing point happens to correspond to a keyframe in the panoramic video, then the modified camera movement angle is the target playback angle of that panoramic video keyframe. Based on the modified camera movement angle of that panoramic video keyframe and the target playback angle of adjacent panoramic video keyframes, interpolation processing needs to be performed on other panoramic video frames in these two panoramic video keyframes to obtain the target playback angle corresponding to each panoramic video frame. Then, the camera movement trajectory is adjusted according to the target playback angle corresponding to each panoramic video frame.

[0212] In one embodiment, the method further includes:

[0213] Based on the position and number of camera movement editing points defined by the target camera movement template, or the position and number of salient targets in the panoramic camera movement video, determine at least one camera movement editing point in the panoramic camera movement video.

[0214] The position of the camera movement edit points defined by the target camera movement template can be represented by a frame number or a time point. For example, if the target camera movement template defines 3 camera movement edit points, their positions indicate frames 5, 50, and 80 of the panoramic camera movement video. If represented by time points, for example, the first camera movement edit point is at time 00:01:10, the second at time 00:20:00, and the third at time 01:40:00.

[0215] Alternatively, at least one camera edit point in the panoramic video can be determined based on the location and number of salient targets in the video. For example, by performing salient target detection on the panoramic video and determining that the salient targets are located at frames 10, 30, and 50, then the camera edit points are determined to be located at frames 10, 30, and 50, respectively.

[0216] In one embodiment, the step of generating a modified panoramic video based on the target camera movement template in response to a user's modification of the camera angle at the camera movement editing point includes:

[0217] The camera movement trajectory is regenerated based on the modified information and the target camera movement template;

[0218] The panoramic video clip is then moved using the regenerated camera movement trajectory to obtain the modified target camera movement video.

[0219] If the camera movement editing point selected by the user does not correspond to the keyframe of the panoramic video, then the camera movement trajectory can be generated first based on the target camera movement template, and then the corresponding trajectory points in the camera movement trajectory can be adjusted according to the modified camera movement perspective.

[0220] If the camera movement editing point selected by the user corresponds to a keyframe in the panoramic video, then the modified camera movement angle becomes the target playback angle for that keyframe. The target playback angles for other keyframes in the panoramic video clip are determined by the target camera movement template. Then, based on the target playback angles of all the panoramic video keyframes, the target playback angle for each panoramic video frame is determined through interpolation, and a camera movement trajectory is generated based on the target playback angle for each panoramic video frame.

[0221] In one embodiment, the method further includes:

[0222] In response to user adjustments to the panoramic video clip, a camera movement trajectory is automatically generated for the adjusted panoramic video clip based on the target camera movement template. The adjustments to the panoramic video clip include at least one of adding video clips or removing video clips.

[0223] After the camera movement is complete, users can adjust the panoramic video clip. For example, they can move the entire camera movement range forward or backward, such as shifting the selected panoramic video clip from 10s-20s to 12s-22s. This is equivalent to adding or removing a portion of the video clip. Alternatively, users can add or remove only video clips. Based on the target camera movement template, the system automatically generates a camera movement trajectory for the adjusted panoramic video clip, and plays the adjusted panoramic video according to the trajectory to obtain the adjusted panoramic video.

[0224] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0225] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0226] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0227] In addition, in the embodiments of the present invention, "first," "second," etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0228] To implement the method of the embodiments of this application, an electronic device is also provided. Figure 7 is a schematic diagram of the hardware composition structure of the electronic device according to an embodiment of this application. As shown in Figure 7, the electronic device includes:

[0229] A communication interface enables information exchange with other devices, such as network devices.

[0230] The processor is connected to the communication interface to enable information interaction with other devices;

[0231] The memory stores a computer program that, when executed by the processor, is configured to interactively acquire at least one viewpoint of interest selected by the user in a panoramic video clip; recommend at least one camera movement template to the user based on the at least one viewpoint of interest; and the camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one viewpoint of interest.

[0232] In one embodiment, the processor is configured to acquire a point of interest or region of interest selected by a user at least at one time point in the panoramic video segment; and to determine at least one corresponding viewpoint of interest based on the point of interest or region of interest selected at the at least one time point.

[0233] In one embodiment, the processor is configured to take the at least one viewpoint of interest as input to the camera movement recommendation model; or, take the at least one viewpoint of interest and a plurality of panoramic video frames as input to the camera movement recommendation model, wherein the plurality of panoramic video frames are extracted from the panoramic video clip; and obtain at least one camera movement template output by the camera movement recommendation model.

[0234] In one embodiment, the camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one point of interest viewpoint and the object attributes contained therein; or the camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one point of interest viewpoint and the object attributes contained therein, and the scene information contained in the plurality of panoramic video frames.

[0235] In one embodiment, the processor is configured to determine a target camera movement template from the at least one camera movement template; perform camera movement processing on the panoramic video segment based on the target camera movement template to obtain a corresponding target camera movement video, wherein the target camera movement video includes camera movement effects corresponding to the target camera movement template and includes the at least one point of interest.

[0236] In one embodiment, the camera movement template includes: multiple keyframes, each keyframe corresponding to position information, viewpoint information, and velocity information between adjacent keyframes; the processor is configured to determine the panoramic video keyframe corresponding to the panoramic video segment based on the position information corresponding to each keyframe in the target camera movement template; determine the target playback viewpoint corresponding to each panoramic video keyframe based on the viewpoint information of each keyframe; obtain the target playback viewpoint corresponding to each panoramic video frame based on the target playback viewpoint corresponding to each panoramic video keyframe and the at least one viewpoint of interest; and obtain the panoramic camera movement video based on the target playback viewpoint corresponding to each panoramic video frame and the velocity information between adjacent keyframes.

[0237] In one embodiment, the processor is configured to interpolate other panoramic video frames based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one viewpoint of interest, to obtain the target playback viewpoint corresponding to each panoramic video frame.

[0238] In one embodiment, the processor is configured to generate a camera movement trajectory for the panoramic video clip based on the target camera movement template, the camera movement trajectory including trajectory points corresponding to the at least one viewpoint of interest, and play the panoramic video clip based on the camera movement trajectory to obtain a corresponding target camera movement video.

[0239] In one embodiment, the processor is configured to display at least one camera movement edit point in the obtained target camera movement video; the camera movement edit point is used to allow a user to modify the camera movement angle of the corresponding panoramic video frame; in response to the user's modification of the camera movement angle at the camera movement edit point, a camera movement trajectory is regenerated based on the target camera movement template, so that the panoramic video segment is cameraed according to the regenerated camera movement trajectory to obtain a modified target camera movement video; the modified target camera movement video includes: the modified camera movement angle.

[0240] In one embodiment, the camera movement template includes a standard camera movement duration and a standard camera movement speed. The processor is configured to determine the actual duration corresponding to the panoramic video segment; determine the actual camera movement speed based on the standard camera movement duration and the actual duration; and perform camera movement processing on the panoramic video segment based on the actual camera movement speed to obtain the target camera movement video.

[0241] In one embodiment, the processor is configured to automatically generate a camera movement trajectory for the adjusted panoramic video segment based on the target camera movement template in response to user adjustments to the panoramic video segment. The adjustments to the panoramic video segment include at least one of adding video segments and removing video segments.

[0242] In one embodiment, the processor is configured to acquire a time period selected by the user from the panoramic video and determine the panoramic video segment based on the time period; or acquire a start time point or end time point selected by the user from the panoramic video and determine the panoramic video segment based on the duration of the selected camera movement template.

[0243] In one embodiment, the processor is configured to acquire a user-defined camera movement template and store the camera movement template in a user camera movement library. The camera movement template includes: the positions of multiple keyframes, the viewpoint corresponding to each keyframe, and the speed between adjacent keyframes.

[0244] This application also provides another electronic device, as shown in FIG7, which includes:

[0245] A communication interface enables information exchange with other devices, such as network devices.

[0246] The processor is connected to the communication interface to enable information interaction with other devices;

[0247] The memory stores a computer program that, when executed by the processor, configures the processor to determine at least one salient viewpoint in a panoramic video clip; recommend at least one camera movement template to a user based on the at least one salient viewpoint; and generate a camera movement video based on the at least one camera movement template that has at least one camera movement effect and includes the at least one salient viewpoint.

[0248] In one embodiment, the processor is configured to acquire a time period selected by the user from the panoramic video and determine the panoramic video segment based on the time period; or acquire a start time point or end time point selected by the user from the panoramic video and determine the panoramic video segment based on the duration of the selected camera movement template.

[0249] In one embodiment, the processor is configured to determine at least one salient target in the panoramic video segment; and to determine at least one salient viewpoint corresponding to the at least one salient target.

[0250] In one embodiment, the processor is configured to determine a panoramic video frame of the panoramic video segment corresponding to the at least one salient target; and to determine at least one salient viewpoint based on the position of the at least one salient target in the panoramic video frame.

[0251] In one embodiment, the processor is configured to take the at least one salient viewpoint as input to a camera movement recommendation model; or, take the at least one salient viewpoint and a plurality of panoramic video frames as input to a camera movement recommendation model, wherein the plurality of panoramic video frames are extracted from the panoramic video clip; and obtain at least one camera movement template output by the camera movement recommendation model.

[0252] In one embodiment, the camera movement recommendation model is used to recommend camera movement templates based on object attributes contained in the at least one salient viewpoint and / or scene information contained in the plurality of panoramic video frames.

[0253] In one embodiment, the processor is configured to determine a target camera movement template from the at least one camera movement template; perform camera movement processing on the panoramic video segment based on the target camera movement template to obtain a corresponding target camera movement video, the target camera movement video including the at least one salient viewpoint.

[0254] In one embodiment, the camera movement template includes: multiple keyframes, each keyframe corresponding to position information, viewpoint information, and velocity information between adjacent keyframes; the processor is configured to determine the panoramic video keyframe corresponding to the panoramic video segment based on the position information corresponding to each keyframe in the target camera movement template; determine the target playback viewpoint corresponding to each panoramic video keyframe based on the viewpoint information of each keyframe; obtain the target playback viewpoint corresponding to each panoramic video frame based on the target playback viewpoint corresponding to each panoramic video keyframe and the at least one salient viewpoint; and obtain the panoramic camera movement video based on the target playback viewpoint corresponding to each panoramic video frame and the velocity information between adjacent keyframes.

[0255] In one embodiment, the processor is configured to interpolate other panoramic video frames based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one salient viewpoint, to obtain the target playback viewpoint corresponding to each panoramic video frame.

[0256] In one embodiment, the processor is configured to generate a camera movement trajectory for the panoramic video clip based on the target camera movement template, the camera movement trajectory including trajectory points corresponding to the at least one salient viewpoint, and play the panoramic video clip based on the camera movement trajectory to obtain the corresponding target camera movement video.

[0257] In one embodiment, the processor is configured to display at least one camera movement edit point in the obtained target camera movement video; the camera movement edit point is used to allow a user to modify the camera movement angle of the corresponding panoramic video frame;

[0258] In response to the user's modification of the camera angle at the camera movement editing point, the camera movement trajectory is regenerated based on the target camera movement template, so that the panoramic video segment is moved according to the regenerated camera movement trajectory to obtain the modified target camera movement video; the modified target camera movement video includes: the modified camera angle.

[0259] In one embodiment, the camera movement template includes: a standard camera movement duration and a standard camera movement speed; the processor is configured to determine the actual duration corresponding to the panoramic video segment; determine the actual camera movement speed based on the standard camera movement duration and the actual duration; and perform camera movement processing on the panoramic video segment based on the actual camera movement speed to obtain the target camera movement video.

[0260] In one embodiment, the processor is configured to automatically generate a camera movement trajectory for the adjusted panoramic video segment based on the target camera movement template in response to user adjustments to the panoramic video segment. The adjustments to the panoramic video segment include either adding or removing video segments.

[0261] In one embodiment, the processor is configured to acquire a user-defined camera movement template and store the camera movement template in a user camera movement library. The camera movement template includes: the positions of multiple keyframes, the viewpoint corresponding to each keyframe, and the speed between adjacent keyframes.

[0262] This application also provides another electronic device. As shown in FIG7, the electronic device includes:

[0263] A communication interface enables information exchange with other devices, such as network devices.

[0264] The processor is connected to the communication interface to enable information interaction with other devices;

[0265] The memory stores a computer program that, when executed by the processor, configures the processor to display at least one camera movement editing point in a panoramic video, the panoramic video being obtained by moving panoramic video segments based on a target camera movement template, the camera movement editing point being used to allow a user to modify the camera movement angle of the corresponding panoramic video frame; in response to the user's modification of the camera movement angle at the camera movement editing point, a modified panoramic video is generated based on the target camera movement template, the modified panoramic video including: the modified camera movement angle.

[0266] In one embodiment, the processor is configured to acquire the camera angle reselected by the user at the camera movement editing point; and to adjust the camera trajectory based on the reselected camera angle so that the generated panoramic video includes the modified camera angle.

[0267] In one embodiment, the processor is configured to determine at least one camera edit point in the panoramic camera video based on the position and number of camera edit points defined by the target camera movement template, or the position and number of salient targets in the panoramic camera video.

[0268] In one embodiment, the processor is configured to automatically generate a camera movement trajectory for the adjusted panoramic video segment based on the target camera movement template in response to user adjustments to the panoramic video segment. The adjustments to the panoramic video segment include at least one of adding video segments and removing video segments.

[0269] In one embodiment, the processor is configured to regenerate the camera movement trajectory based on the modification information and the target camera movement template; and to perform camera movement on the panoramic video segment based on the regenerated camera movement trajectory to obtain the modified target camera movement video.

[0270] Of course, in practical applications, the various components in an electronic device are coupled together through a bus system. It can be understood that the bus system is used to achieve communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus systems in Figure 7.

[0271] The memory in this application embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0272] It is understood that memory can be volatile or non-volatile, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0273] The methods disclosed in the embodiments of this application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads the program from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0274] Optionally, when the processor executes the program, it implements the corresponding processes implemented by the electronic device in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0275] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor of an electronic device to perform the steps described in the method of this application.

[0276] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory storing a computer program, which can be executed by a processor of an electronic device to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0277] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, electronic devices, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0278] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0279] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0280] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0281] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0282] It should be noted that the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0283] In addition, in this application example, terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0284] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A panoramic video camera movement method, comprising: Based on interaction, obtain at least one point of interest selected by the user in the panoramic video clip; Based on the at least one perspective of interest, at least one camera movement template is recommended to the user. The camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one perspective of interest.

2. The method according to claim 1, wherein, The method of obtaining at least one viewpoint of interest selected by the user in a panoramic video clip based on interaction includes: Obtain at least one point of interest or region of interest selected by the user at any point in time within the panoramic video clip; At least one interest perspective is determined based on the selected point of interest or region of interest at the at least one time point.

3. The method according to claim 1, wherein, The method of recommending at least one camera movement template to the user based on the at least one perspective of interest includes: The at least one point of interest viewpoint is used as the input to the camera movement recommendation model; or, the at least one point of interest viewpoint and several panoramic video frames are used as the input to the camera movement recommendation model, wherein the several panoramic video frames are extracted from the panoramic video clip. Obtain at least one camera movement template output by the camera movement recommendation model.

4. The method according to claim 3, wherein, The camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one point of interest viewpoint and the object attributes it contains; or The camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one point of interest viewpoint, the object attributes it contains, and the scene information contained in the several panoramic video frames.

5. The method according to claim 1, wherein, The method further includes: Determine the target camera movement template from the at least one camera movement template; The panoramic video clip is processed based on the target camera movement template to obtain a corresponding target camera movement video. The target camera movement video includes camera movement effects corresponding to the target camera movement template and includes at least one point of interest.

6. The method according to claim 5, wherein, The camera movement template includes: multiple keyframes, each keyframe corresponding to position information, viewpoint information and velocity information between adjacent keyframes; The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes: The panoramic video keyframes corresponding to the panoramic video segment are determined based on the position information corresponding to each keyframe in the target camera movement template. The target playback angle corresponding to each panoramic video keyframe is determined based on the viewpoint information of each keyframe. Based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one viewpoint of interest, the target playback viewpoint corresponding to each panoramic video frame is obtained. The panoramic video is obtained based on the target playback viewpoint corresponding to each panoramic video frame and the speed information between adjacent keyframes.

7. The method according to claim 6, wherein, The step of obtaining the target playback viewpoint corresponding to each panoramic video frame based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one viewpoint of interest includes: Interpolation processing is performed on other panoramic video frames based on the target playback viewpoint corresponding to each panoramic video keyframe and the at least one viewpoint of interest to obtain the target playback viewpoint corresponding to each panoramic video frame.

8. The method according to claim 5, wherein, The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes: The camera movement trajectory of the panoramic video clip is generated based on the target camera movement template. The camera movement trajectory includes trajectory points corresponding to the at least one viewpoint of interest. The panoramic video clip is played based on the camera movement trajectory to obtain the corresponding target camera movement video.

9. The method according to claim 8, wherein, The method further includes: At least one camera movement editing point is displayed in the obtained target camera movement video; the camera movement editing point is used to allow the user to modify the camera movement angle of the corresponding panoramic video frame; In response to the user's modification of the camera angle at the camera movement editing point, the camera movement trajectory is regenerated based on the target camera movement template, so that the panoramic video segment is moved according to the regenerated camera movement trajectory to obtain the modified target camera movement video; the modified target camera movement video includes: the modified camera angle.

10. The method according to claim 5, wherein, The camera movement template includes: standard camera movement duration and standard camera movement speed; The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes: Determine the actual duration corresponding to the panoramic video segment; The actual camera movement speed is determined based on the standard camera movement duration and the actual duration. The panoramic video clip is processed according to the actual camera movement speed to obtain the target camera movement video.

11. The method according to claim 5, wherein, The method further includes: In response to user adjustments to the panoramic video clip, a camera movement trajectory is automatically generated for the adjusted panoramic video clip based on the target camera movement template. The adjustments to the panoramic video clip include at least one of adding video clips or removing video clips.

12. The method according to claim 1, wherein, The method further includes: Obtain the time period selected by the user from the panoramic video, and determine the panoramic video segment based on the time period; or Obtain the start or end time point selected by the user from the panoramic video, and determine the panoramic video segment based on the duration of the selected camera movement template.

13. The method according to claim 1, wherein, The method further includes: Obtain a user-defined camera movement template and store the camera movement template in the user camera movement library. The camera movement template includes: the positions of multiple keyframes, the viewpoint corresponding to each keyframe, and the speed between adjacent keyframes.

14. A panoramic video camera movement method, comprising: Identify at least one salient viewpoint in a panoramic video clip; Based on the at least one salient viewpoint, at least one camera movement template is recommended to the user, and the camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one salient viewpoint.

15. The method according to claim 14, wherein, The method further includes: Obtain the time period selected by the user from the panoramic video, and determine the panoramic video segment based on the time period; or Obtain the start or end time point selected by the user from the panoramic video, and determine the panoramic video segment based on the duration of the selected camera movement template.

16. The method of claim 14, wherein, Determining at least one salient viewpoint in a panoramic video segment includes: Identify at least one salient target in the panoramic video segment; Determine at least one salient perspective corresponding to the at least one salient objective.

17. The method according to claim 14, wherein, Determining at least one salient perspective corresponding to the at least one salient objective includes: Determine the panoramic video frame of the panoramic video segment corresponding to the at least one salient target; At least one salient viewpoint is determined based on the position of the at least one salient target in the panoramic video frame.

18. The method according to claim 14, wherein, The method of recommending at least one camera movement template to the user based on the at least one salient viewpoint includes: Use the at least one salient viewpoint as input to the camera movement recommendation model; Alternatively, the at least one salient viewpoint and several panoramic video frames can be used as input to the camera movement recommendation model, wherein the several panoramic video frames are extracted from the panoramic video clip. Obtain at least one camera movement template output by the camera movement recommendation model.

19. The method according to claim 18, wherein, The camera movement recommendation model is used to recommend camera movement templates based on object attributes contained in the at least one salient viewpoint and / or scene information contained in the plurality of panoramic video frames.

20. The method of claim 14, wherein, The method further includes: Determine the target camera movement template from the at least one camera movement template; The panoramic video clip is processed based on the target camera movement template to obtain the corresponding target camera movement video, which includes the at least one salient viewpoint.

21. The method according to claim 20, wherein, The camera movement template includes: multiple keyframes, each keyframe corresponding to position information, viewpoint information and velocity information between adjacent keyframes; The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes: The panoramic video keyframes corresponding to the panoramic video segment are determined based on the position information corresponding to each keyframe in the target camera movement template. The target playback angle corresponding to each panoramic video keyframe is determined based on the viewpoint information of each keyframe. Based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one salient viewpoint, the target playback viewpoint corresponding to each panoramic video frame is obtained. The panoramic video is obtained based on the target playback viewpoint corresponding to each panoramic video frame and the speed information between adjacent keyframes.

22. The method according to claim 21, wherein, The step of obtaining the target playback viewpoint corresponding to each panoramic video frame based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one salient viewpoint includes: Interpolation processing is performed on other panoramic video frames based on the target playback viewpoint corresponding to each panoramic video keyframe and the at least one salient viewpoint to obtain the target playback viewpoint corresponding to each panoramic video frame.

23. The method of claim 20, wherein, The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes: The camera movement trajectory of the panoramic video clip is generated based on the target camera movement template. The camera movement trajectory includes trajectory points corresponding to the at least one salient viewpoint. The panoramic video clip is played based on the camera movement trajectory to obtain the corresponding target camera movement video.

24. The method according to claim 23, wherein, The method further includes: At least one camera movement editing point is displayed in the obtained target camera movement video; the camera movement editing point is used to allow the user to modify the camera movement angle of the corresponding panoramic video frame; In response to the user's modification of the camera angle at the camera movement editing point, the camera movement trajectory is regenerated based on the target camera movement template, so that the panoramic video segment is moved according to the regenerated camera movement trajectory to obtain the modified target camera movement video; the modified target camera movement video includes: the modified camera angle.

25. The method according to claim 20, wherein, The camera movement template includes: standard camera movement duration and standard camera movement speed; The step of performing camera movement processing on the panoramic video clip based on the target camera movement template to obtain the corresponding target camera movement video includes: Determine the actual duration corresponding to the panoramic video segment; The actual camera movement speed is determined based on the standard camera movement duration and the actual duration. The panoramic video clip is processed according to the actual camera movement speed to obtain the target camera movement video.

26. The method of claim 20, wherein, The method further includes: In response to user adjustments to the panoramic video clip, a camera movement trajectory is automatically generated for the adjusted panoramic video clip based on the target camera movement template. The adjustments to the panoramic video clip include either adding or removing video clips.

27. The method according to claim 14, wherein, The method further includes: Obtain a user-defined camera movement template and store the camera movement template in the user camera movement library. The camera movement template includes: the positions of multiple keyframes, the viewpoint corresponding to each keyframe, and the speed between adjacent keyframes.

28. A panoramic video camera movement method, comprising: At least one camera movement editing point is displayed in the panoramic camera movement video, which is obtained by moving panoramic video segments based on a target camera movement template. The camera movement editing point is used to allow users to modify the camera movement angle of the corresponding panoramic video frame. In response to the user's modification of the camera angle at the camera movement editing point, a modified panoramic camera movement video is generated based on the target camera movement template. The modified panoramic camera movement video includes the modified camera angle.

29. The method according to claim 28, wherein, The step of responding to the user's modification of the camera angle at the camera editing point and generating a modified panoramic camera movement video based on the camera movement template includes: Get the camera angle that the user reselects at the camera editing point; The camera movement trajectory is adjusted based on the reselected camera angle so that the generated panoramic video includes the modified camera angle.

30. The method according to claim 28, wherein, The method further includes: Based on the position and number of camera movement editing points defined by the target camera movement template, or the position and number of salient targets in the panoramic camera movement video, determine at least one camera movement editing point in the panoramic camera movement video.

31. The method according to claim 28, wherein, The method further includes: In response to user adjustments to the panoramic video clip, a camera movement trajectory is automatically generated for the adjusted panoramic video clip based on the target camera movement template. The adjustments to the panoramic video clip include at least one of adding video clips or removing video clips.

32. The method according to claim 28, wherein, The step of responding to the user's modification of the camera angle at the camera editing point and generating a modified panoramic camera movement video based on the target camera movement template includes: The camera movement trajectory is regenerated based on the modified information and the target camera movement template; The panoramic video clip is then moved using the regenerated camera movement trajectory to obtain the modified target camera movement video.

33. An electronic device, comprising a memory and a processor, the memory storing a computer program, wherein when the computer program is executed by the processor, the processor is configured to interactively acquire at least one viewpoint of interest selected by a user in a panoramic video clip; recommend at least one camera movement template to the user based on the at least one viewpoint of interest, wherein a camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one viewpoint of interest.

34. The electronic device according to claim 33, wherein, The processor is configured to acquire at least one point of interest or region of interest selected by the user at at least one time point in the panoramic video segment; and to determine at least one corresponding viewpoint of interest based on the point of interest or region of interest selected at the at least one time point.

35. The electronic device according to claim 33, wherein, The processor is configured to take the at least one viewpoint of interest as input to the camera movement recommendation model; or, take the at least one viewpoint of interest and several panoramic video frames as input to the camera movement recommendation model, wherein the several panoramic video frames are extracted from the panoramic video clip; and obtain at least one camera movement template output by the camera movement recommendation model.

36. The electronic device according to claim 35, wherein, The camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one point of interest viewpoint and the object attributes contained therein; or the camera movement recommendation model is used to recommend camera movement templates based on the position of the at least one point of interest viewpoint and the object attributes contained therein, and the scene information contained in the plurality of panoramic video frames.

37. The electronic device according to claim 33, wherein, The processor is configured to determine a target camera movement template from the at least one camera movement template; perform camera movement processing on the panoramic video segment based on the target camera movement template to obtain a corresponding target camera movement video, wherein the target camera movement video includes camera movement effects corresponding to the target camera movement template and includes the at least one point of interest.

38. The electronic device according to claim 37, wherein, The camera movement template includes multiple keyframes, each keyframe corresponding to position information, viewpoint information, and velocity information between adjacent keyframes; the processor is configured to determine the panoramic video keyframe corresponding to the panoramic video segment based on the position information corresponding to each keyframe in the target camera movement template; determine the target playback viewpoint corresponding to each panoramic video keyframe based on the viewpoint information of each keyframe; obtain the target playback viewpoint corresponding to each panoramic video frame based on the target playback viewpoint corresponding to each panoramic video keyframe and the at least one viewpoint of interest; and obtain the panoramic camera movement video based on the target playback viewpoint corresponding to each panoramic video frame and the velocity information between adjacent keyframes.

39. The electronic device according to claim 38, wherein, The processor is configured to interpolate other panoramic video frames based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one viewpoint of interest, to obtain the target playback viewpoint corresponding to each panoramic video frame.

40. The electronic device according to claim 37, wherein, The processor is configured to generate a camera movement trajectory for the panoramic video clip based on the target camera movement template. The camera movement trajectory includes trajectory points corresponding to the at least one viewpoint of interest. The panoramic video clip is then played based on the camera movement trajectory to obtain the corresponding target camera movement video.

41. The electronic device according to claim 40, wherein, The processor is configured to display at least one camera movement editing point in the obtained target camera movement video; the camera movement editing point is used to allow a user to modify the camera movement angle of the corresponding panoramic video frame; in response to the user's modification of the camera movement angle at the camera movement editing point, the camera movement trajectory is regenerated based on the target camera movement template, so that the panoramic video segment is cameraed according to the regenerated camera movement trajectory to obtain the modified target camera movement video. The modified target camera movement video includes: the modified camera movement angle.

42. The electronic device according to claim 37, wherein, The camera movement template includes a standard camera movement duration and a standard camera movement speed. The processor is configured to determine the actual duration corresponding to the panoramic video segment; determine the actual camera movement speed based on the standard camera movement duration and the actual duration; and perform camera movement processing on the panoramic video segment based on the actual camera movement speed to obtain the target camera movement video.

43. The electronic device according to claim 37, wherein, The processor is configured to automatically generate a camera movement trajectory for the adjusted panoramic video segment based on the target camera movement template in response to user adjustments to the panoramic video segment. The adjustments to the panoramic video segment include at least one of adding video segments or removing video segments.

44. The electronic device according to claim 33, wherein, The processor is configured to acquire a time period selected by the user from the panoramic video and determine the panoramic video segment based on the time period; or acquire a start time point or end time point selected by the user from the panoramic video and determine the panoramic video segment based on the duration of the selected camera movement template.

45. The electronic device according to claim 33, wherein, The processor is configured to acquire a user-defined camera movement template and store the camera movement template in the user camera movement library. The camera movement template includes: the positions of multiple keyframes, the viewpoint corresponding to each keyframe, and the speed between adjacent keyframes.

46. ​​An electronic device comprising a memory and a processor, the memory storing a computer program, wherein when the computer program is executed by the processor, the processor is configured to determine at least one salient viewpoint in a panoramic video segment; recommend at least one camera movement template to a user based on the at least one salient viewpoint, wherein a camera movement video obtained based on the at least one camera movement template has at least one camera movement effect and includes the at least one salient viewpoint.

47. The electronic device according to claim 46, wherein, The processor is configured to acquire a time period selected by the user from the panoramic video and determine the panoramic video segment based on the time period; or acquire a start time point or end time point selected by the user from the panoramic video and determine the panoramic video segment based on the duration of the selected camera movement template.

48. The electronic device according to claim 46, wherein, The processor is configured to determine at least one salient target in the panoramic video segment; and to determine at least one salient viewpoint corresponding to the at least one salient target.

49. The electronic device according to claim 46, wherein, The processor is configured to determine a panoramic video frame of the panoramic video segment corresponding to the at least one salient target; and to determine at least one salient viewpoint based on the position of the at least one salient target in the panoramic video frame.

50. The electronic device according to claim 46, wherein, The processor is configured to take the at least one salient viewpoint as input to a camera movement recommendation model; or, take the at least one salient viewpoint and a plurality of panoramic video frames as input to a camera movement recommendation model, wherein the plurality of panoramic video frames are extracted from the panoramic video clip; and obtain at least one camera movement template output by the camera movement recommendation model.

51. The electronic device according to claim 50, wherein, The camera movement recommendation model is used to recommend camera movement templates based on object attributes contained in the at least one salient viewpoint and / or scene information contained in the plurality of panoramic video frames.

52. The electronic device according to claim 46, wherein, The processor is configured to determine a target camera movement template from the at least one camera movement template; perform camera movement processing on the panoramic video segment based on the target camera movement template to obtain a corresponding target camera movement video, wherein the target camera movement video includes the at least one salient viewpoint.

53. The electronic device according to claim 52, wherein, The camera movement template includes multiple keyframes, each keyframe corresponding to position information, viewpoint information, and velocity information between adjacent keyframes; the processor is configured to determine the panoramic video keyframe corresponding to the panoramic video segment based on the position information corresponding to each keyframe in the target camera movement template; determine the target playback viewpoint corresponding to each panoramic video keyframe based on the viewpoint information of each keyframe; obtain the target playback viewpoint corresponding to each panoramic video frame based on the target playback viewpoint corresponding to each panoramic video keyframe and the at least one salient viewpoint; and obtain the panoramic camera movement video based on the target playback viewpoint corresponding to each panoramic video frame and the velocity information between adjacent keyframes.

54. The electronic device according to claim 53, wherein, The processor is configured to interpolate other panoramic video frames based on the target playback viewpoint corresponding to each of the panoramic video keyframes and the at least one salient viewpoint, to obtain the target playback viewpoint corresponding to each panoramic video frame.

55. The electronic device according to claim 52, wherein, The processor is configured to generate a camera movement trajectory for the panoramic video clip based on the target camera movement template. The camera movement trajectory includes trajectory points corresponding to the at least one salient viewpoint. The panoramic video clip is then played based on the camera movement trajectory to obtain the corresponding target camera movement video.

56. The electronic device according to claim 55, wherein, The processor is configured to display at least one camera movement editing point in the obtained target camera movement video; the camera movement editing point is used to allow a user to modify the camera movement angle of the corresponding panoramic video frame; In response to the user's modification of the camera angle at the camera movement editing point, the camera movement trajectory is regenerated based on the target camera movement template, so that the panoramic video segment is moved according to the regenerated camera movement trajectory to obtain the modified target camera movement video. The modified target camera movement video includes: the modified camera movement angle.

57. The electronic device according to claim 52, wherein, The camera movement template includes: a standard camera movement duration and a standard camera movement speed; the processor is configured to determine the actual duration corresponding to the panoramic video segment; determine the actual camera movement speed based on the standard camera movement duration and the actual duration; and perform camera movement processing on the panoramic video segment based on the actual camera movement speed to obtain the target camera movement video.

58. The electronic device according to claim 52, wherein, The processor is configured to automatically generate a camera movement trajectory for the adjusted panoramic video segment based on the target camera movement template in response to user adjustments to the panoramic video segment. The adjustments to the panoramic video segment include either adding or removing video segments.

59. The electronic device according to claim 46, wherein, The processor is configured to acquire a user-defined camera movement template and store the camera movement template in the user camera movement library. The camera movement template includes: the positions of multiple keyframes, the viewpoint corresponding to each keyframe, and the speed between adjacent keyframes.

60. An electronic device comprising a memory and a processor, the memory storing a computer program, wherein when the computer program is executed by the processor, the processor is configured to display at least one camera movement edit point in a panoramic video, the panoramic video being obtained by moving a panoramic video segment based on a target camera movement template, the camera movement edit point being configured to allow a user to modify the camera movement angle of a corresponding panoramic video frame; and in response to a user modifying the camera movement angle at the camera movement edit point, generating a modified panoramic video based on the target camera movement template, the modified panoramic video comprising: The revised camera angle.

61. The electronic device according to claim 60, wherein, The processor is configured to acquire the camera angle reselected by the user at the camera movement editing point; and to adjust the camera trajectory based on the reselected camera angle so that the generated panoramic camera movement video includes the modified camera angle.

62. The electronic device according to claim 60, wherein, The processor is configured to determine at least one camera edit point in the panoramic video based on the position and number of camera edit points defined by the target camera movement template, or the position and number of salient targets in the panoramic video.

63. The electronic device according to claim 60, wherein, The processor is configured to automatically generate a camera movement trajectory for the adjusted panoramic video segment based on the target camera movement template in response to user adjustments to the panoramic video segment. The adjustments to the panoramic video segment include at least one of adding video segments or removing video segments.

64. The electronic device according to claim 60, wherein, The processor is configured to regenerate the camera movement trajectory based on the modified information and the target camera movement template; and to perform camera movement on the panoramic video clip based on the regenerated camera movement trajectory to obtain the modified target camera movement video.

Images