Video generation method and device and storage medium

Abstract

The invention relates to a video generation method and device and a storage medium. The video generation method comprises the following steps: shooting a video picture, and monitoring the moving speed of a target object in the video picture in the video picture shooting process; determining a first time point when the moving speed reaches a speed threshold value; continuing to shoot the video picture after the first time point, aligning the first time point and the second time point, adding the media file into the shot video, and generating a target video; wherein the target video meets the condition that when the shot picture is played to the first time node, the media file is synchronously played to the second time node. According to the method and the device, a video with background music matched with the video picture can be generated immediately after the video picture is shot.

Description

Technical Field

[0001] The present disclosure relates to the field of multimedia technology, and in particular to a video generation method, device and storage medium. Background Art

[0002] With the continuous advancement of multimedia technology, shooting videos has become an indispensable form of entertainment in people's daily lives.

[0003] In order to make the video more vivid, video producers often add background music (BGM) and other media files. At the same time, when there are fast actions in the video, in order to improve the audience's viewing experience, video producers usually slow down these fast actions. If the added background music can enter the climax when the video starts to play in slow motion, the audio-visual impact of the video will be enhanced.

[0004] In the related art, in order to ensure that the background music can enter the climax part when the video starts to play in slow motion, the video producer needs to manually adjust the soundtrack so that the climax part of the first audio can accurately match the start time of the slow motion in the video. However, it is very difficult to manually adjust the soundtrack to ensure that the climax part of the first audio and the slow motion of the video match perfectly, and it is not easy to achieve the ideal effect even after repeated attempts. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a video generation method, device and readable storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, a video generation method is provided, comprising: shooting a video frame, and during the shooting of the video frame, monitoring the moving speed of a target object in the video frame; determining a first time point at which the moving speed reaches a speed threshold; continuing to shoot the video frame after the first time point, and aligning the first time point and a second time point, adding a media file to the shot video, and generating a target video; wherein the target video satisfies that when the video frame is played to the second time node, the media file is synchronously played to the first time node.

[0007] In one embodiment, determining the first time point when the moving speed reaches the speed threshold includes: taking the time point when the moving speed of the target object changes from a first speed to a second speed and the second speed is greater than the speed threshold as the first time point; wherein the first speed is a speed less than the speed threshold.

[0008] In one embodiment, continuing to capture video images after a first time point includes: after the first time point, continuing to capture video images at a first frame rate, the first frame rate being less than a frame rate threshold and less than a second frame rate; the second frame rate being the frame rate of the video images captured before the first time point.

[0009] In one implementation manner, the second time point is determined in the following manner: a media file is acquired, and a time point when a predefined media content is played in the media file is used as the second time point.

[0010] In one implementation, the media file is background music, and the predefined media content is a climax part of the background music.

[0011] In one embodiment, the monitoring of the moving speed of the target object in the video screen includes: monitoring the pixel address where the light intensity of the target object changes in the video screen based on a dynamic visual sensor; generating an asynchronous address event based on the pixel address to determine the moving speed of the target object.

[0012] According to a second aspect of an embodiment of the present disclosure, a video generating device is provided, comprising: a shooting unit, configured to shoot a video screen, and continue shooting the video screen after a first time point; a speed measuring unit, configured to monitor the moving speed of a target object in the video screen during the shooting of the video screen; a generating unit, configured to determine a first time point at which the moving speed reaches a speed threshold, and align the first time point with a second time point, add a media file to the shot video, and generate a target video; wherein the target video satisfies that when the video screen is played to the second time node, the media file is synchronously played to the first time node.

[0013] In one embodiment, the generation unit determines the first time point when the moving speed reaches the speed threshold in the following manner: the time point when the moving speed of the target object changes from the first speed to the second speed and the second speed is greater than the speed threshold is taken as the first time point; wherein the first speed is a speed less than the speed threshold.

[0014] In one embodiment, the shooting unit continues to shoot video pictures after a first time point in the following manner: after the first time point, the video pictures are continued to be shot at a first frame rate, the first frame rate is less than a frame rate threshold and less than a second frame rate; the second frame rate is the frame rate of the video pictures shot before the first time point.

[0015] In one implementation, the generating unit determines the second time point in the following manner: acquiring a media file, and setting a time point when the predefined media content is played in the media file as the second time point.

[0016] In one implementation, the media file is background music, and the predefined media content is a climax part of the background music.

[0017] In one embodiment, the speed measurement unit monitors the moving speed of the target object in the video screen in the following manner: based on a dynamic visual sensor, the pixel address where the light intensity of the target object changes in the video screen is monitored; based on the pixel address, an asynchronous address event is generated to determine the moving speed of the target object.

[0018] According to a third aspect of an embodiment of the present disclosure, a video generating device is provided, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to: execute the method described in any one of the first aspect of the embodiment of the present disclosure.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute any one of the methods described in the first aspect of the embodiment of the present disclosure.

[0020] The technical solution provided by the embodiment of the present disclosure may include the following beneficial effects: real-time capture of video images and monitoring of the motion state of the target object therein, including parameters such as its moving speed. By setting a speed threshold, a time point is determined, which is defined as the first time point at which the moving speed reaches the threshold. After determining the first time point, continue to capture the video image and align it with the second time point. This process involves synchronization processing of the video image and the timeline. While recording the video, the addition of background music is synchronously realized, that is, the production of the target video is realized without the need for separate editing. Add the media file to the captured video image to generate the target video. The target video has a characteristic that when the video image is played to the first time point, the media file will be played synchronously to the second time point. The technical solution provided by the embodiment of the present disclosure realizes accurate synchronization of the video image and the media file by monitoring the motion state of the target object in the video image in real time and setting a speed threshold to determine the time point. This improves the effect of video production, enables the added media file to be perfectly integrated with the video image, and improves the user's viewing experience.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] Figure 1The figure is a flowchart of a video generating method according to an exemplary embodiment.

[0024] Figure 2 The figure is a flowchart of another video generating method according to an exemplary embodiment.

[0025] Figure 3 The figure is a flow chart of a method for shooting a video image according to an exemplary embodiment.

[0026] Figure 4 The figure is a flow chart showing a method for determining a second time point according to an exemplary embodiment.

[0027] Figure 5 is a flowchart showing yet another method for determining a second time point according to an exemplary embodiment.

[0028] Figure 6 The figure is a flow chart showing a method for monitoring the moving speed of a target object according to an exemplary embodiment.

[0029] Figure 7 The present invention is a flowchart showing a method for composing music for a video based on a dynamic visual sensor according to an exemplary embodiment.

[0030] Figure 8 The present invention is a flowchart of a method for producing a video including a badminton playing scene and matching it with background music according to an exemplary embodiment.

[0031] Fig. 9 It is a block diagram of a video generating device according to an exemplary embodiment.

[0032] Fig.10 It is a block diagram of a video generating device according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0034] In the accompanying drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limitations on the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure. The embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings.

[0035] The video generation method provided by the embodiments of the present disclosure can be applied to relevant scenarios of video production, for example, when adding media files to a video, such as configuring background music.

[0036] In the related art, the processing method of video soundtrack is usually placed in the post-processing stage. That is to say, after shooting the video, choose the appropriate soundtrack, and complete the video editing and soundtrack by extracting music clips and inserting them into the video. In order to make a slow-motion video, you need to shoot a complete video, and adjust the video playback speed so that part of the picture remains at normal speed and the other part is presented in slow motion. When the object moves quickly, start high-frame rate slow-motion shooting, and at the same time, accompany the climax of the music. By adding background music and adjusting the soundtrack, ensure that the background music reaches the climax or chorus when the video switches to slow motion playback. In this way, you can produce a video that is shot normally when the object is slow, and presented in slow motion when the object is moving quickly, and at the same time, it is accompanied by the chorus / climax of the music, which improves the viewing and artistic effect of the video.

[0037] However, there are some problems with the above video generation method, such as: the editing time is too long, the sound track needs to be adjusted, it is necessary to know how to slow down the video, and the background music is difficult to accurately match the start time of the slow motion in the video.

[0038] In view of this, an embodiment of the present disclosure provides a video generation method. In the video generation method provided by the present disclosure, during the video shooting process, the moving speed of the target object in the video screen is monitored in real time. Once the moving speed of the target object exceeds a preset threshold, the time point is immediately recorded as the first time point when the speed reaches the threshold. Then, after the first time point, the video screen continues to be shot, and the first time point and the second time point are aligned. In this process, a pre-prepared media file is added to the shot video to generate a target video. In the target video, when the media file is played to the second time node, it will be played synchronously to the first time node. Through this method, after the video screen shooting is completed, a video with background music and the background music matching the video screen can be immediately generated without the user manually adjusting the sound track.

[0039] Figure 1 is a flow chart of a video generation method according to an exemplary embodiment. Figure 1 As shown, the steps include the following steps S11 to S14.

[0040] In step S11, a video frame is captured, and during the process of capturing the video frame, a moving speed of a target object in the video frame is monitored.

[0041] In the disclosed embodiment, the process of monitoring the moving speed of the target object in the video picture can be performed by a device with a speed measurement function.

[0042] In the disclosed embodiment, the target object is not limited to the entire individual, such as a person, but may also include a specific part of the individual, such as a person's right hand. Before shooting a video, the user may select the target object according to their needs.

[0043] In step S12, the first time point at which the moving speed reaches the speed threshold is determined.

[0044] In the embodiment of the present disclosure, the speed threshold may be set by the user before shooting begins. The user may set the speed threshold according to his or her own needs, and the present disclosure does not limit this.

[0045] In step S13, video frames continue to be captured after the first time point, and the first time point and the second time point are aligned.

[0046] In step S14, the media file is added to the captured video to generate a target video. The target video satisfies that when the captured image is played to the first time node, the media file is synchronously played to the second time node.

[0047] In the embodiment of the present disclosure, the second time point refers to a time point in the media file. The time point can be any time point on the time axis of the media file, and the present disclosure does not limit this.

[0048] In one embodiment, the video images captured before the first time point are imported into a video editing software on a terminal. A media file is imported into the video editing software. By adjusting the timeline of the media file, it is ensured that the video images before the first time point have been played when the video images are played to the second time point. The video editing software is used to edit the video images after the first time point to the video images before the first time point to generate a target video.

[0049] The method for aligning the first time point and the second time point provided in the above implementation can complete video editing during the process of video frame shooting. When the video frame shooting is completed, the terminal can immediately generate the video frame.

[0050] It can be understood that the above implementation provided by the embodiment of the present disclosure is only one of multiple feasible implementations, and the embodiment of the present disclosure does not limit how to align the first time point and the second time point.

[0051] In the embodiments of the present disclosure, the media files may be acquired in advance or in real time from the present device or other electronic devices, and the present disclosure does not limit this.

[0052] In the video generation method provided by the embodiment of the present disclosure, by real-time monitoring the moving speed of the target object in the video screen, when the moving speed of the target object reaches the preset threshold speed, the terminal device will immediately record the current time point as the first time point. After the first time point, the terminal device continues to capture the video screen and aligns the second time point preset in the media file to generate a target video. After completing the video screen recording, the terminal device can quickly generate a target video with background music. In the target video, when the media file is played to the second time point, the captured video screen is synchronously playing the video screen at the first time point. At the same time, this embodiment also realizes the addition of background music when recording the video screen, that is, the background music is embedded in the video while shooting the video, without the need for editing and synthesis in the later stage, which greatly simplifies the process of generating videos and improves the efficiency of generating videos.

[0053] In a scenario where a slow motion video needs to be produced, the user usually wants the first half to show the action at normal speed, while the second half is presented in slow motion. At the same time, it is necessary to ensure that when the video switches to slow motion playback, the background music just reaches the chorus part. In view of this, the embodiment of the present disclosure provides another video generation method.

[0054] Figure 2 is a flow chart of another video generation method according to an exemplary embodiment. Figure 2 As shown, the steps include steps S21 to S24.

[0055] In step S21, a video frame is captured, and during the process of capturing the video frame, a moving speed of a target object in the video frame is monitored.

[0056] The above steps have been Figure 1 The corresponding embodiments are described in detail and will not be repeated here.

[0057] In step S22, the time point when the moving speed of the target object changes from the first speed to the second speed and the second speed is greater than the speed threshold is taken as the first time point, wherein the first speed is a speed less than the speed threshold.

[0058] In the disclosed embodiment, the set speed threshold is a reference value, which can be set by the user before the shooting starts, and the user can set the speed threshold according to his own needs. In the disclosed embodiment, as long as the moving speed of the target object is less than this reference value, then this speed can be regarded as the first speed. In other words, the first speed is not fixed, but can be adjusted according to the actual situation. Similarly, as long as the moving speed of the target object is greater than this reference value, then this speed can be regarded as the second speed, and the second speed is also not fixed. For example, the user sets the threshold speed to A (m / s) (unit: meters per second). After starting to shoot the video screen, the target object moves at B (m / s), B is less than A, then B (m / s) is the first speed. Starting from a certain time point t1, the target object moves at C (m / s), C is greater than B and C is less than A, then C (m / s) is also the first speed. Since B (m / s) is the first speed and C (m / s) is also the first speed, the moving speed of the target object does not meet the condition of changing from the first speed to the second speed, so t1 is not the first time point. Starting from a certain time point t2 after time point t1, the target object moves at D (m / s), where D is greater than A. Since D is greater than A, D (m / s) is the second speed. The target object's moving speed changes from the first speed to the second speed at time point t2, so the target object's moving speed satisfies the condition of changing from the first speed to the second speed, so t2 is the first time point. A, B, C, and D are all non-negative real numbers.

[0059] In step S23, video frames continue to be captured after the first time point, and the first time point and the second time point are aligned.

[0060] In step S24, the media file is added to the captured video to generate a target video. The target video satisfies that when the captured image is played to the first time node, the media file is synchronously played to the second time node.

[0061] The above steps have been Figure 1 The corresponding embodiments are described in detail and will not be repeated here.

[0062] In the video generation method provided by the embodiment of the present disclosure, the terminal implements real-time monitoring of the moving speed of the target object in the video screen. When the moving speed of the target object changes from the first speed to the second speed, the terminal immediately records this time point as the first time point. After the first time point, the terminal continues to shoot the video screen. There is a preset second time point in the media file. The terminal adds the media file to the video being recorded, and ensures that the first time point is aligned with the second time point, thereby generating the target video. In this way, after completing the video screen recording, the terminal can quickly generate a video. The first half of the video shows the action at normal speed, while the second half is presented in slow motion. When the video switches to slow motion playback, the background music just reaches the chorus part.

[0063] It is well known to those skilled in the art that the clarity of the generated video will be affected by the frame rate and the moving speed of the target object selected by the terminal when shooting the video. During the video shooting process, if the moving speed of the target object is high and the frame rate selected by the terminal is low, the clarity of the captured video will be relatively poor. In order to improve the clarity of the generated video, an embodiment of the present disclosure provides a video shooting method.

[0064] Figure 3 FIG. 1 is a flow chart of a method for capturing a video image according to an exemplary embodiment. Figure 3 As shown, the process includes the following steps S31 to S32.

[0065] In step S31, before a first time point, a video frame is captured at a second frame rate.

[0066] In the disclosed embodiment, the video frame rate shot at the second frame rate is low. Before the first time point, the moving speed of the target object does not reach the speed threshold, so the second frame rate is selected to shoot the video to obtain a sufficiently clear video.

[0067] In the embodiment of the present disclosure, the second frame rate can be determined by the user before the shooting starts. The user can determine a suitable frame rate as the second frame rate according to his / her own needs.

[0068] In step S32, after the first time point, the video frames continue to be captured at a first frame rate, where the first frame rate is greater than the second frame rate.

[0069] In the disclosed embodiment, after the first time point, the moving speed of the target object has reached the speed threshold, and at this time the second frame rate can no longer meet the requirement of capturing a clear video image, so the video image continues to be captured at the first frame rate. Since the first frame rate is greater than the second frame rate, even if the moving speed of the target object has reached the speed threshold, the captured video image is still clear.

[0070] In the disclosed embodiment, the first frame rate can be determined by the user before shooting, or by the terminal according to the moving speed of the target object. The user can choose to determine a suitable frame rate as the first frame rate according to his own needs, or choose to let the terminal select a suitable frame rate as the first frame rate according to the moving speed of the target object.

[0071] The video image shooting method provided by the embodiment of the present disclosure adjusts the frame rate at a first time point. When the moving speed of the target object reaches a threshold speed, the video image is shot at a higher frame rate than when the moving speed of the target object does not reach the threshold speed, thereby obtaining a clear video image.

[0072] In the above embodiment, any point in the media file can be the second time point. In actual video shooting scenarios, users usually expect to define the second time point according to their own needs. In view of this, the embodiment of the present disclosure provides a method for determining the second time point.

[0073] Figure 4 is a flow chart showing a method for determining a second time point according to an exemplary embodiment. Figure 4 As shown, the steps include the following steps S41 to S42.

[0074] In step S41, a media file is obtained.

[0075] In the disclosed embodiment, the media file may be acquired in advance from the device or other electronic devices.

[0076] In step S42, the time point when the predefined media content is played in the media file is set as the second time point.

[0077] In the embodiment of the present disclosure, the time point when the predefined media content is played in the media file can be preset by the user according to his / her own needs. The process of presetting the time point when the predefined media content is played in the media file needs to be before the video screen shooting starts.

[0078] The method for determining the second time point provided by the embodiment of the present disclosure allows the user to define the second time point according to his / her own needs.

[0079] In actual video shooting scenes, commonly used media files may be background music. Usually, users expect to set the time point at which the climax of the background music begins as the second time point. In view of this, the embodiment of the present disclosure provides another method for determining the second time point.

[0080] Figure 5 is a flow chart showing a method for determining a second time point according to an exemplary embodiment. Figure 5As shown, the steps include the following steps S51 to S52.

[0081] In step S51, background music is obtained.

[0082] In the disclosed embodiment, the background music may be acquired in advance or in real time from the device or other electronic devices.

[0083] In step S52, the time point when the climax part of the background music is played is set as the second time point.

[0084] In the embodiment of the present disclosure, the time point when the climax part of the background music is played can be the time point when the faster-paced part of the background music starts to play, or it can be the time point when the chorus part of the background music is played, or it can be a time point selected by the user according to his or her own needs, and the present disclosure does not limit this.

[0085] The method for determining the second time point provided by the embodiment of the present disclosure can quickly determine the second time point when the media file is background music, eliminating the step of the user predefining the second time point of the media file, and optimizing the video generation process.

[0086] In the above embodiment, the process of monitoring the moving speed of the target object in the video image can be performed by a device with a speed measurement function. Among them, the device with a speed measurement function is not limited to devices such as speed sensors or acceleration sensors that directly have the function of detecting the moving speed of the target object. Some devices that do not directly have a speed measurement function but can determine the moving speed of the target object through some methods can also be regarded as devices with a speed measurement function, such as: dynamic vision sensors. In the embodiment of the present disclosure, a method for detecting the moving speed of a target object based on a dynamic vision device is provided.

[0087] Figure 6 FIG. 1 is a flow chart showing a method for monitoring the moving speed of a target object according to an exemplary embodiment. Figure 6 As shown, the steps include the following steps S61 to S62.

[0088] In step S61, the pixel addresses where the light intensity of the target object changes in the video picture are monitored based on the dynamic vision sensor.

[0089] Dynamic Vision Sensor (DVS) is an asynchronous image sensor using complementary metal oxide semiconductor technology, which is based on Address-Event Representation (AER). Unlike traditional image sensors that output pixel information in frames, DVS draws on the working principle of biological retinal nerves and uses asynchronous address event representation, outputting pixel addresses and information only when the light intensity changes. Therefore, DVS has the characteristics of real-time dynamic response to scene changes, ultra-sparse image representation, and asynchronous output of time domain change events. These characteristics enable DVS to overcome the problems of high redundancy, high latency, high noise, low dynamic range, and transmission bottlenecks brought about by traditional image sensor frame sampling methods, making it suitable for high-speed vision fields with high real-time requirements.

[0090] In the disclosed embodiment, when the DVS monitors the video screen, it detects the change in the light intensity of the target object. When the light intensity of the target object changes, the DVS records the address information of these changed pixels. In this way, the pixels whose light intensity changes in the video screen can be tracked in real time, and the dynamic information of the target object can be captured. Compared with traditional image sensors, this monitoring method has lower redundancy, latency and noise, as well as a higher dynamic range, and can better meet the needs of high-speed and real-time applications.

[0091] In step S62, an asynchronous address event is generated based on the pixel address to determine the moving speed of the target object.

[0092] In the disclosed embodiment, the pixel address of the light intensity change recorded by the DVS can generate asynchronous address events. These events reflect the movement of the target object in the video screen. By analyzing these asynchronous address events, the moving speed of the target object in each time period can be calculated. This method can track the movement state of the target object in real time and provide accurate dynamic information for high-speed, real-time applications.

[0093] In the disclosed embodiment, by analyzing these asynchronous address events, the terminal can calculate the movement speed of the object with the help of a motion detection algorithm, thereby achieving accurate judgment of the local movement speed.

[0094] There are many different motion detection algorithms. Among the many algorithms, Optical Flow (OF) is a relatively common algorithm. The Optical Flow method is based on the movement pattern of pixels and is particularly suitable for processing event stream data generated by DVS sensors. This method calculates the movement direction and speed of pixels by analyzing the position changes between events. The Optical Flow method processes each event, calculates its motion vector relative to the previous frame, and then captures the motion state of objects in the scene in real time. It should be clear that the Optical Flow method is only one of many motion detection algorithms, and the embodiments of the present disclosure do not make specific restrictions on which motion detection algorithm to use.

[0095] The method for monitoring the moving speed of a target object provided by the embodiment of the present disclosure accurately detects the moving speed of the target object based on a DVS sensor. By adopting the DVS sensor, the terminal can record and analyze the motion state of the target object in real time. When the moving speed of the target object reaches a preset threshold speed, the method can ensure that the terminal can accurately identify the position of the time point on the time axis at the first time point.

[0096] The video generation method mentioned above is described below with reference to examples in the embodiments of the present disclosure.

[0097] Figure 7 is a flow chart showing a method for composing music for a video based on a dynamic visual sensor according to an exemplary embodiment. Figure 7 As shown, the steps include the following steps S71 to S75.

[0098] In step S71, a piece of background music is selected, and a time node is determined before the climax of the music as the triggering time point of the dynamic visual sensor, that is, the access time point when the slow motion is triggered.

[0099] In the embodiments of the present disclosure, the sensors used include dynamic vision sensors and various conventional sensors. The video music matching method based on dynamic vision sensors shown in the embodiments of the present disclosure can be used on all devices equipped with dynamic vision sensors. In addition, a hybrid dynamic vision sensor (hybrid-dvs) that combines the functions of a dynamic vision sensor with those of a conventional sensor can also be selected.

[0100] In the disclosed embodiment, background music is first selected, and then a specific time node is determined before the climax of the music. This time node is set as the triggering time point of the dynamic visual sensor, that is, the access time point when the slow motion is triggered. Thus, in the final video generated, when the video screen plays to the beginning of the slow motion segment, the background music plays just to the climax. Such a setting is intended to enhance the audience's viewing experience, while also making the expression of the video content more vivid and rich. The climax of the background music may refer to the part of the background music with the most intense emotional expression, the richest musical elements, and the most moving melody. The position of the climax in the song is usually very obvious, and often appears in the chorus or ending of the song, so that the audience can recognize and remember it.

[0101] In step S72, an object motion speed value is determined as a threshold for triggering slow motion shooting.

[0102] In the disclosed embodiment, the threshold for triggering slow motion shooting can be set by the user according to personal needs before shooting, and can also be adjusted in real time by the shooting device according to the moving speed of the target object during the shooting process.

[0103] In step S73, a shooting frame rate value of a slow motion video is determined.

[0104] In the disclosed embodiment, the shooting frame rate value of the slow-motion video can be set by the user according to his or her own needs before shooting, or it can be adjusted in real time by the shooting device during the shooting process according to the moving speed of the target object and the preset frame rate value of the sensor.

[0105] In step S74, video capture is started. The first part is normal speed video. When the motion of the object is accelerated and reaches the speed threshold, slow motion capture is started and the time node of slow motion capture is determined.

[0106] In the disclosed embodiment, normal speed video refers to a video picture recorded using an initially set shooting frame rate when the moving speed of the target object does not exceed a preset threshold. When the moving speed of the object being filmed increases and reaches the speed threshold, the terminal device will start the slow motion shooting mode and use the slow motion video shooting frame rate value to capture the video picture. The time node of slow motion shooting refers to the specific moment in the video picture at which the terminal device will switch to the slow motion video shooting frame rate value for recording. The purpose of such an operation is to ensure that when the video plays to the beginning of the slow motion segment, the background music plays just at the climax.

[0107] In step S75, after the shooting is completed, the climax time node of the background music is matched with the time node of starting slow motion to form a complete video.

[0108] In the video soundtrack method based on dynamic visual sensors provided by the embodiment of the present disclosure, during the video shooting process, the terminal monitors the moving speed of the target object in real time. Once the moving speed of the target object reaches a preset threshold, the terminal records the time point as the first time point. After the first time point, the terminal continues to shoot the video screen and aligns the first time point with the second time point. In this process, the terminal adds the pre-prepared media file to the shot video to generate a target video. In the target video, when playing to the second time node, the media file will be played synchronously to the first time node. Through this method, the terminal can immediately generate a video with background music after completing the video screen shooting, and the background music matches the video screen.

[0109] Compared with the video generation method in the related art, the video soundtrack method based on dynamic visual sensors provided by the embodiment of the present disclosure has the following advantages: the editing time is shortened, and there is no need to adjust the slow motion of the sound track and the video; the background music is accurately matched with the start time of the slow motion in the video; there is no need to shoot a whole section of slow motion high frame rate video, thereby saving resource usage. By accurately matching the climax time node of the background music with the start time node of the slow motion, the terminal ensures the time coordination between the slow motion shooting and the climax part of the background music. After the shooting is completed, this matching process can make the switching between the climax part of the background music and the slow motion shooting clip smoother and more natural when watching the video. In this way, the terminal not only improves the viewing and artistic effect of the video, but also brings a better audio-visual experience to the user. In order to better illustrate the video generation method disclosed in the present disclosure, the following will take the production of a video screen containing a badminton scene and matching it with background music as an example for detailed description.

[0110] Figure 8 The flowchart of a method for producing a video containing a badminton scene and matching it with background music is shown according to an exemplary embodiment. Figure 8 As shown, the steps include the following steps S81 to S85.

[0111] In step S81, a piece of background music is selected, and a time point is determined before the climax.

[0112] The above steps have been Figure 7 The corresponding embodiments are described in detail and will not be repeated here.

[0113] In step S82, the badminton racket of the player is selected as the target object, and the target object speed threshold and the shooting frame rate value of the slow motion video are selected.

[0114] In step S83, video shooting starts. Before the athlete's badminton racket hits the badminton, the movement speed is slow and the slow motion shooting mode is not triggered.

[0115] In step S84, when the athlete's badminton racket hits the badminton, the speed increases and the slow motion shooting mode begins.

[0116] In the disclosed embodiment, the target object is a badminton racket of an athlete. The speed at which the athlete hits the badminton is set to a threshold speed. Before the athlete hits the badminton, the movement speed is slow and the slow-motion shooting mode is not triggered. At this time, the terminal uses the initially set shooting frame rate to shoot the athlete. When the athlete hits the badminton, the movement speed becomes faster, triggering the slow-motion shooting mode of the terminal. In the slow-motion shooting mode, the terminal adjusts the shooting frame rate to the shooting frame rate value of the slow-motion video.

[0117] In step S85, after the shooting is finished, the time node of the selected background music is matched with the node where the video starts slow motion to generate a complete video.

[0118] The disclosed embodiment provides a method for making a video containing a badminton scene and matching it with background music. The method can divide the video into a non-climax part played at normal speed and a climax part played in slow motion. By determining the time node before the climax of the background music, the moment when the athlete hits the badminton in the video is perfectly combined with the climax of the music, thereby improving the viewing and attractiveness of the video. In addition, by selecting appropriate object movement speed values ​​and slow-motion video shooting frame rate values, the video presents a slow-motion effect when the athlete hits the badminton, which is conducive to the audience to better appreciate and analyze the technical movements of the athlete. At the same time, the time node of the selected background music is matched with the node where the video starts slow motion, so that the rhythm of the video and the music is coordinated, bringing a better audio-visual experience to the audience.

[0119] Based on the same concept, an embodiment of the present disclosure also provides a video generating device.

[0120] It is understandable that the video generation device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0121] Fig. 9 is a block diagram of a video generation device according to an exemplary embodiment. Fig. 9 The device includes a shooting unit 101, a speed measuring unit 102 and a generating unit 103.

[0122] The shooting unit 101 is used to shoot video images.

[0123] The speed measuring unit 102 is used to monitor the moving speed of the target object in the video picture during the process of shooting the video picture.

[0124] The shooting unit 101 is further used to continue shooting video images after the first time point.

[0125] The generating unit 103 is used to determine the first time point when the moving speed reaches the speed threshold, align the first time point with the second time point, add the media file to the captured video, and generate a target video. The target video satisfies that when the video screen is played to the second time point, the media file is synchronously played to the first time point.

[0126] In one embodiment, the generation unit 103 determines the first time point when the moving speed reaches the speed threshold in the following manner: the time point when the moving speed of the target object changes from the first speed to the second speed and the second speed is greater than the speed threshold is taken as the first time point. The first speed is a speed less than the speed threshold.

[0127] In one embodiment, the shooting unit 101 continues to shoot the video picture after the first time point in the following manner: after the first time point, the video picture is continued to be shot at a first frame rate, the first frame rate is greater than or less than a frame rate threshold, and greater than a second frame rate. The second frame rate is the frame rate of the video picture shot before the first time point.

[0128] In one embodiment, the generating unit 103 determines the second time point in the following manner: acquiring a media file, and setting the time point when the predefined media content is played in the media file as the second time point.

[0129] In one embodiment, the media file is background music, and the predefined media content is a climax part of the background music.

[0130] In one embodiment, the speed measuring unit monitors the moving speed of the target object in the video screen in the following manner: based on the dynamic vision sensor, the pixel address where the light intensity of the target object changes in the video screen is monitored, and based on the pixel address, an asynchronous address event is generated to determine the moving speed of the target object.

[0131] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0132] Fig.102 is a block diagram of a device 200 for video generation according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0133] Reference Fig.10 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0134] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0135] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0136] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0137] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0138] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0139] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0140] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0141] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0142] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0143] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0144] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0145] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0146] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0147] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0148] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0149] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A video generation method, characterized in that: include: Shooting a video screen, and during the shooting of the video screen, monitoring the moving speed of the target object in the video screen; Determine a first time point at which the moving speed reaches a speed threshold; Continue to shoot video images after the first time point, align the first time point with the second time point, add the media file to the shot video, and generate a target video; The target video satisfies that when the video screen is played to the first time node, the media file is synchronously played to the second time node.

2. The video generation method according to claim 1, characterized in that: The determining the first time point at which the moving speed reaches the speed threshold comprises: The time point at which the moving speed of the target object changes from the first speed to the second speed and the second speed is greater than the speed threshold is taken as the first time point; The first speed is a speed less than a speed threshold.

3. The video generation method according to claim 1 or 2, characterized in that: Continue to capture video footage after the first time point, including: After the first time point, continue to capture video images at a first frame rate, where the first frame rate is greater than the second frame rate; The second frame rate is a frame rate of video images captured before the first time point.

4. The video generation method according to claim 1, characterized in that: The second time point is determined in the following manner: A media file is acquired, and a time point when the predefined media content in the media file is played is used as a second time point.

5. The video generation method according to claim 4, characterized in that: The media file is background music, and the predefined media content is a climax part of the background music.

6. The video generation method according to claim 1, characterized in that: The monitoring of the moving speed of the target object in the video screen includes: Based on the dynamic vision sensor, the pixel address where the light intensity of the target object in the video screen changes is monitored; An asynchronous address event is generated based on the pixel address to determine a moving speed of the target object.

7. A video generating device, characterized in that: include: A shooting unit, used for shooting video images, and continuing to shoot video images after the first time point; The speed measuring unit is used to monitor the moving speed of the target object in the video picture during the process of shooting the video picture; A generating unit, configured to determine a first time point at which the moving speed reaches a speed threshold, align the first time point with a second time point, add a media file to the captured video, and generate a target video; The target video satisfies that when the video screen is played to the second time node, the media file is synchronously played to the first time node.

8. The video generating device according to claim 7, characterized in that: The speed measuring unit monitors the moving speed of the target object in the video image in the following manner: Based on the dynamic vision sensor, the pixel address where the light intensity of the target object in the video screen changes is monitored; An asynchronous address event is generated based on the pixel address to determine a moving speed of the target object.

9. A video generating device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the method described in any one of claims 1 to 6.

10. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor of the terminal, the terminal is enabled to execute the method according to any one of claims 1 to 6.

Images