Video processing method, device, electronic device and storage medium

Abstract

The present invention provides a video processing method, device, electronic device and storage medium, the method comprising: receiving a recorded video sent by an audio game device, the recorded video being obtained after the audio game device records the beat action made by a user based on the preset rhythm class video when playing the preset rhythm class video; in the recorded video, intercepting a sub-video corresponding to a target time period, and determining the preset beat data corresponding to the target time period in the preset rhythm class video; in the case of determining that the current beat data in the sub-video does not match the preset beat data, determining the average beat time difference between the current beat data and the preset beat data; according to the average beat time difference, adjusting the sub-video in the recorded video to obtain a target video, the beat data in the target video matches the beat data in the preset rhythm class video. The whole process can save a lot of video processing time to improve video processing efficiency, and at the same time, can optimize the problem of different synchronization between the recorded video and the audio beat, so that the quality of the target video is better.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and in particular to a video processing method, device, electronic equipment and storage medium. Background Art

[0002] In an entertainment scenario, music game equipment (such as a dance machine, etc.) will play rhythm videos for users to watch and make corresponding rhythmic movements according to a preset rhythm or beat. At this time, the electronic device can record the rhythmic movements made by the user.

[0003] However, due to the user's skill level, reaction speed, concentration or other external factors (such as device delay, network delay, etc.), the user's beat may not always be accurate. Therefore, some beats may lag behind or be ahead of schedule, resulting in poor video recording effect.

[0004] At present, the existing video processing methods often edit the recorded video by replacing the background music. However, the whole editing process is relatively complicated, resulting in poor quality of the video processing results. Summary of the invention

[0005] The present invention provides a video processing method, device, electronic device and storage medium, which are used to solve the defect that the existing video processing method is relatively complex, resulting in poor quality of video processing results. By combining a recorded video and a preset rhythm video, according to the average beat time difference corresponding to the two videos in the same target time period, the sub-video in the recorded video is adjusted to obtain a target video. The whole process is relatively simple, which can save a lot of video processing time to improve video processing efficiency. At the same time, it can optimize the problem of different synchronization between video and audio beats, so that the quality of the final target video is better.

[0006] The invention provides a video processing method, which comprises the following steps.

[0007] A recorded video is received from a music game device, where the recorded video is obtained by recording a beat movement made by a user based on a preset rhythm video while the music game device is playing the preset rhythm video.

[0008] In the recorded video, a sub-video corresponding to a target time period is captured, and preset beat data corresponding to the target time period in the preset rhythm video is determined.

[0009] When it is determined that the current beat data in the sub-video does not match the preset beat data, an average beat time difference between the current beat data and the preset beat data is determined.

[0010] According to the average beat time difference, the sub-video is adjusted in the recorded video to obtain a target video, and the beat data in the target video matches the beat data in the preset rhythm video.

[0011] According to a video processing method provided by the present invention, the sub-video is adjusted in the recorded video according to the average beat time difference to obtain a target video, including: extracting the audio time axis corresponding to the target time period in the preset rhythm type video; according to the average beat time difference, in the recorded video, adjusting the position of the sub-video on the audio time axis to obtain a first video; and determining the target video according to the video blank area and / or video overlapping area in the first video.

[0012] According to a video processing method provided by the present invention, the determination that the current beat data in the sub-video does not match the preset beat data includes: when the current beat data includes at least one first beat and the preset beat data includes at least one second beat, determining the first moment corresponding to each of the at least one first beat according to the time axis of the sub-video, and determining the second moment corresponding to each of the at least one second beat according to the audio time axis; when all first moments are not exactly the same as all second moments, determining that the current beat data does not match the preset beat data; and / or, when the action score corresponding to the sub-video is less than a preset threshold, determining that the current beat data in the sub-video does not match the preset beat data.

[0013] According to a video processing method provided by the present invention, the preset beat action corresponding to the target time period in the preset rhythm video includes at least one beat action, and the at least one beat action corresponds one-to-one to the at least one second beat; the determination of the average beat time difference between the current beat data and the preset beat data includes: traversing the at least one beat action, and for the current beat action, determining the first target beat corresponding to the current beat action from the at least one first beat, and determining the second target beat corresponding to the current beat action from the at least one second beat; determining the beat time difference corresponding to the current beat action according to the first moment corresponding to the first target beat and the second moment corresponding to the second target beat; and determining the average beat time difference according to the beat time differences corresponding to each of the at least one beat action.

[0014] According to a video processing method provided by the present invention, the target video is determined according to the video blank area in the first video, including: in the first video, determining the first video interval and the second video interval forming the video blank area, the playback time of the first video interval is earlier than the playback time of the second video interval; slowing down the preset tail width of the first video interval, and / or slowing down the preset head width of the second video interval to obtain the target video; or, in the first video, determining the first video interval and the second video interval forming the video blank area; inserting frames in the first video interval, and / or inserting frames in the second video interval to obtain the target video.

[0015] According to a video processing method provided by the present invention, the target video is determined according to the video overlapping area in the first video, including: in the first video, determining a third video interval and a fourth video interval forming the video overlapping area, the playback time of the third video interval being earlier than the playback time of the fourth video interval; fast-forwarding the third video interval, and / or fast-forwarding the fourth video interval to obtain the target video; or, in the first video, extracting two overlapping video intervals according to the overlapping width to obtain the video overlapping area; extracting frames in the video overlapping area to obtain the target video.

[0016] According to a video processing method provided by the present invention, the target video is determined based on the video blank area and / or video overlapping area in the first video, including: determining the second video based on the video blank area and / or video overlapping area in the first video; replacing the audio corresponding to the second video with the audio corresponding to the preset rhythm type video to obtain the target video.

[0017] The present invention also provides a video processing device, comprising the following modules.

[0018] The data transceiver module is used to receive the recorded video sent by the audio game device. The recorded video is obtained by recording the beat movements made by the user based on the preset rhythm video when the audio game device plays the preset rhythm video.

[0019] A video processing module is used to capture a sub-video corresponding to a target time period in the recorded video, and determine preset beat data corresponding to the target time period in the preset rhythm class video; when it is determined that the current beat data in the sub-video does not match the preset beat data, determine the average beat time difference between the current beat data and the preset beat data; based on the average beat time difference, adjust the sub-video in the recorded video to obtain a target video, wherein the beat data in the target video matches the beat data in the preset rhythm class video.

[0020] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned video processing methods is implemented.

[0021] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the video processing method described above is implemented.

[0022] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned video processing methods.

[0023] The video processing method, device, electronic device and storage medium provided by the present invention combine the recorded video and the preset rhythm type video, and adjust the sub-video in the recorded video according to the average beat time difference corresponding to the two videos in the same target time period to obtain the target video. The whole process is relatively simple, which can save a lot of video processing time to improve the video processing efficiency. At the same time, it can optimize the problem of different synchronization between the video and audio beats, so that the quality of the target video finally determined is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a flow chart of the video processing method provided by the present invention.

[0026] Figure 2 It is a structural schematic diagram of the video blank area and the video overlapping area provided by the present invention.

[0027] Figure 3 It is a structural schematic diagram of the video processing device provided by the present invention.

[0028] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be noted that the execution subject involved in the embodiments of the present invention can be a video processing device or an electronic device. Optionally, the electronic device can include: a service device (ie, a server), a computer, a mobile terminal, a wearable device, etc.

[0031] The embodiment of the present invention is further described below by taking an electronic device as an example.

[0032] Figure 1 It is a flow chart of the video processing method provided by the present invention, such as Figure 1 As shown, the method includes the following steps 101 and 104.

[0033] Step 101: Receive a recorded video sent by an audio game device, where the recorded video is obtained by recording the beat movements made by the user based on a preset rhythm video when the audio game device is playing the preset rhythm video.

[0034] Among them, music game equipment (abbreviated as: music game equipment) is a hardware device or software platform specially used for music games, which usually has functions such as responding to user operations, playing music and displaying game screens.

[0035] Recorded video is a dynamic image recorded by the camera on the audio game device. In audio games, recorded video may include the screen of the user's game operation, background music, user actions, etc.

[0036] Preset rhythm videos are pre-made video content with a specific rhythm or beat, usually containing a clear music rhythm or beat to guide users to play the game for reference or following. The playback time of the preset rhythm video is usually the same as the recording time of the recorded video.

[0037] The beat action is the action that the user makes in the music game according to the rhythm or beat of the music. Optionally, the beat action may include: key operation, gesture, dance action, etc.

[0038] When the audio-game device plays a preset rhythm video, the user makes corresponding rhythmic movements based on the preset rhythm video. At this time, the audio-game device records the entire process through a camera device to obtain a recorded video to record the user's gaming process and performance, and then sends the recorded video to an electronic device; the electronic device receives the recorded video for subsequent video processing.

[0039] Optionally, the preset rhythm video is obtained based on the following steps: the audio and video device displays a video list; the user inputs a selection operation to the audio and video device based on the video list; and the audio and video device determines the preset rhythm video corresponding to the selection operation.

[0040] The video list is a list or a collection of multiple rhythm-type video entries.

[0041] Optionally, multiple rhythm video items are displayed on a graphical user interface (GUI) of the music game device in a certain order or category.

[0042] Optionally, the plurality of rhythm video entries may be provided by the music game device itself, or may be uploaded or downloaded by the user himself, which is not specifically limited here.

[0043] The video list is usually displayed as a vertical or horizontal scrolling list on the GUI. Users can browse different rhythm video entries by sliding or clicking. Each rhythm video entry may contain information such as the video thumbnail, title, author, difficulty level, etc., so that users can quickly understand the content of the video. When the user sees a rhythm video entry of interest, the user can send a selection operation to the audio and video device by clicking or touching the rhythm video entry; the audio and video device will receive the selection operation and retrieve the corresponding preset rhythm video from the stored video database for subsequent playback. The entire process can effectively improve the interaction between the audio and video device and the user, and enhance the user's audio and video experience.

[0044] Step 102: In the recorded video, a sub-video corresponding to the target time period is captured, and preset beat data corresponding to the target time period in the preset rhythm video is determined.

[0045] The target time period refers to a specific time period that the user wants to extract from the original video (such as a recorded video or a preset rhythm video). That is, the duration of the target time period is less than or equal to the duration of the original video (such as the recording duration of the recorded video, or the playback duration of the preset rhythm video).

[0046] A sub-video is a partial video clip corresponding to a target time period captured from a recorded video.

[0047] Optionally, the target time period is greater than a preset time period.

[0048] Optionally, the preset time length may be preset before the electronic device leaves the factory, or may be user-defined, which is not specifically limited here. Exemplarily, the preset time length is 10 seconds (s).

[0049] The electronic device can capture a partial video clip corresponding to the target time period from the recorded video to obtain a sub-video, and determine the video clip corresponding to the target time period and the preset beat data corresponding to the video clip in the preset rhythm video for subsequent video processing.

[0050] It should be noted that there is no limitation on the timing between the electronic device capturing the sub-video and the electronic device determining the preset beat data.

[0051] Step 103: When it is determined that the current beat data in the sub-video does not match the preset beat data, determine the average beat time difference between the current beat data and the preset beat data.

[0052] The average beat time difference refers to the average value of the difference between the current beat data and the preset beat data.

[0053] After determining the sub-video, the electronic device may first analyze and detect the sub-video to obtain the beat points or rhythm data corresponding to the user's beat movements, that is, to obtain at least one first beat to determine the current beat data; and in the preset rhythm video, determine the beat points or rhythm data preset in the target time period, that is, to obtain at least one second beat to determine the preset beat data; then, the electronic device matches the current beat data with the preset beat data: if they match, it means that there is no difference between the user's beat movements and the beat points or rhythm data corresponding to the music; if they do not match, it means that there is a difference between the user's beat movements and the beat points or rhythm data corresponding to the music. At this time, the electronic device can calculate the current beat data and the preset beat data to obtain the average beat time difference to improve data support for subsequent video processing.

[0054] It should be noted that the electronic device has no limitation on the timing of determining the current beat data and the preset beat data.

[0055] In some embodiments, the electronic device determines that the current beat data in the sub-video does not match the preset beat data, which may include at least one of the following: implementation method 1 and implementation method 2.

[0056] Implementation method 1: When the current beat data includes at least one first beat and the preset beat data includes at least one second beat, the electronic device determines the first moment corresponding to each of the at least one first beat according to the time axis of the sub-video, and determines the second moment corresponding to each of the at least one second beat according to the audio time axis; when all the first moments are not exactly the same as all the second moments, the electronic device determines that the current beat data and the preset beat data do not match.

[0057] In the case where the current beat data includes at least one first beat, the electronic device determines the specific time point corresponding to each of the at least one first beat based on the time axis of the sub-video, that is, the first moment, and determines the specific time point corresponding to each of the at least one second beat based on the audio time axis, that is, the second moment; then, the electronic device determines whether all the first moments are the same as all the second moments: if they are the same, it is determined that the current beat data and the preset beat data match; if they are not the same, it means that there is a difference between the current beat data and the preset beat data, and it can be determined that the current beat data and the preset beat data do not match.

[0058] The entire process can accurately compare two beat datasets by determining the specific time point of each beat on the respective video timeline, ensuring that even slightly different numbers or distributions of beats can be accurately identified.

[0059] It should be noted that the number of the first beat and the number of the second beat should be basically the same and there should be no obvious difference. The two numbers can be subtracted to obtain a difference. If the difference is less than or equal to a preset difference threshold, it can be said that there is no obvious difference between the first beat and the second beat.

[0060] Exemplarily, the value of the preset difference threshold is 1.

[0061] Implementation method 2: When the action score corresponding to the sub-video is less than a preset threshold, the electronic device determines that the current beat data in the sub-video does not match the preset beat data.

[0062] The action score is usually a quantitative evaluation of the quality, accuracy or synchronization of the user's action in the sub-video. The action score may be based on multiple factors, such as the smoothness, accuracy, and synchronization with the music of the action.

[0063] The preset threshold is a standard used by the electronic device to determine whether the action score is qualified. If the action score is less than this threshold, then the electronic device will consider that the quality of the user's action is not up to standard. For example, the preset threshold is 99 points.

[0064] When the user finishes the music game, the music game device will give a corresponding total action score based on the user's actions. At this time, the electronic device can extract the action score corresponding to the sub-video from the music game device, and compare the action score with the preset threshold: if the action score is greater than or equal to the preset threshold, it is determined that the current beat data and the preset beat data match; if the action score is less than the preset threshold, it means that there is a difference between the current beat data and the preset beat data, and it can be determined that the current beat data and the preset beat data do not match.

[0065] The whole process introduces a scoring mechanism to provide a more comprehensive evaluation of the matching degree between the sub-video and the video clip corresponding to the target time period in the preset rhythm class video. This can not only detect whether the beat data is aligned, but also evaluate the quality of the user's action, thereby providing a more accurate matching result.

[0066] In some embodiments, the preset beat action corresponding to the target time period in the preset rhythm video may include at least one beat action, and at least one beat action corresponds one-to-one to at least one second beat; the electronic device determines the average beat time difference between the current beat data and the preset beat data, which may include: traversing at least one beat action, for the current beat action, the electronic device determines the first target beat corresponding to the current beat action from at least one first beat, and determines the second target beat corresponding to the current beat action from at least one second beat; determines the beat time difference corresponding to the current beat action according to the first moment corresponding to the first target beat and the second moment corresponding to the second target beat; the electronic device determines the average beat time difference according to the beat time difference corresponding to each of the at least one beat actions.

[0067] Since the average beat time difference refers to the average value of the difference between the current beat data and the preset beat data, the electronic device can traverse at least one beat action when determining the average beat time difference. For the current beat action, the electronic device first determines the first target beat corresponding to the current beat action from at least one first beat, and determines the second target beat corresponding to the current beat action from at least one second beat; then determines the first moment corresponding to the first target beat from at least one first moment, and determines the second moment corresponding to the second target beat from at least one second moment, and then calculates these two moments to obtain the beat time difference corresponding to the current beat action, which is a time difference. After traversing at least one beat action, the electronic device can determine the beat time difference corresponding to each of the at least one beat action, and then average all the beat time differences to obtain the average beat time difference.

[0068] This process of going through all the beats ensures that all relevant data is taken into account. Whether it is fast or slow beats, whether it is high or low volume beats, they are all included in the calculation, providing a comprehensive assessment, based on which the final average beat time difference is more accurate, which can accurately reflect the overall difference in beat data.

[0069] It should be noted that the timing of the electronic device determining the first target beat and the electronic device determining the second target beat is not limited.

[0070] Step 104: According to the average beat time difference, the sub-video is adjusted in the recorded video to obtain a target video, and the beat data in the target video matches the beat data in the preset rhythm video.

[0071] In order to process the recorded video so that the beat data in the processed recorded video matches the beat data in the preset rhythm video, the electronic device can edit and adjust the sub-video in the recorded video according to the average beat time difference to obtain the processed recorded video, that is, the target video. The whole process is relatively simple, which can save a lot of video processing time to improve the video processing efficiency. At the same time, it can optimize the problem of different synchronization between the video and audio beats, so that the quality of the final target video is better.

[0072] In some embodiments, the electronic device adjusts a sub-video in a recorded video according to an average beat time difference to obtain a target video, which may include: the electronic device extracts an audio timeline corresponding to a target time period in a preset rhythm type video; the electronic device adjusts the position of the sub-video on the audio timeline in the recorded video according to the average beat time difference to obtain a first video; the electronic device determines the target video based on a video blank area and / or a video overlapping area in the first video.

[0073] The audio timeline is a sequence of timestamps of audio events (such as beats, notes, etc.) corresponding to video clips corresponding to target time periods in preset rhythm-type videos, and is used to represent the rhythm and structure of audio content.

[0074] Video blank areas are areas without video content.

[0075] The video overlap area is the area where two video clips are displayed simultaneously.

[0076] When processing a recorded video, the electronic device may first analyze the video clip corresponding to the target time period in the preset rhythm video, and extract the audio timeline from the video clip; then determine the degree to which the sub-video needs to be adjusted based on the average beat time difference, and adjust the position of the sub-video on the audio timeline based on the average beat time difference to obtain a first video. Specifically, this can be achieved by moving the starting point or the ending point of the sub-video so that the beat points in the sub-video are aligned with the beat points on the audio timeline, ensuring that the sub-video is synchronized with the audio content of the above-mentioned video clip, which improves the viewing and professionalism of the video; after adjusting the position of the sub-video, video blank areas and / or video overlapping areas may appear in the first video. At this time, the electronic device processes these areas to obtain a more coherent target video.

[0077] In some embodiments, the electronic device determines the target video according to the video blank area in the first video, which may include one of the following: implementation method 1 and implementation method 2.

[0078] Implementation method 1: The electronic device determines, in a first video, a first video interval and a second video interval that form a video blank area, and the playback time of the first video interval is earlier than the playback time of the second video interval; the electronic device slows down the preset tail width of the first video interval, and / or slows down the preset head width of the second video interval to obtain a target video.

[0079] The preset tail width refers to a specific time period before the first video interval ends, for example, the last 3 seconds of the first video interval.

[0080] The preset header width and the preset tail width refer to a specific time period before the start and end of the first video interval. For example: the first 4 seconds of the second video interval.

[0081] When processing a video blank area, the electronic device can process the video blank area by slowing down the segments corresponding to the several beats of the two adjacent video intervals. Specifically, the first video is first parsed to obtain the first video interval and the second video interval that form the video blank area, that is, the first video interval and the second video interval are incoherent, and there is an area without video content between the two video intervals; in order to fill the video blank area and avoid abrupt transitions, the electronic device can slow down specific parts of the two video intervals, that is, slow down the preset tail width of the first video interval, and / or slow down the preset head width of the second video interval, so as to achieve a smooth transition between the two video intervals and obtain a target video with better quality.

[0082] It should be noted that slow motion processing usually means reducing the playback speed of the video interval, so that the content of the video interval is displayed more slowly visually without increasing the total length of the video. This processing method can effectively extend the playback time of the video segment, thereby filling the blank area of ​​the video, while maintaining the fluency and continuity of the target video.

[0083] Implementation method 2: The electronic device determines a first video interval and a second video interval forming a video blank area in a first video; the electronic device inserts frames in the first video interval and / or inserts frames in the second video interval to obtain a target video.

[0084] When processing a video blank area, the electronic device may first parse the first video to obtain a first video interval and a second video interval that form the video blank area; in order to fill the video blank area and avoid abrupt transitions, the electronic device may interpolate frames in the first video interval based on the content of adjacent frames of the first video interval, and / or interpolate frames in the second video interval based on the content of adjacent frames of the second video interval, to obtain a target video with better quality.

[0085] In some embodiments, the electronic device determines the target video according to the video overlapping area in the first video, which may include one of the following: implementation method 1 and implementation method 2.

[0086] Implementation method 1: The electronic device determines, in the first video, a third video interval and a fourth video interval that form a video overlapping area, and the playback time of the third video interval is earlier than the playback time of the fourth video interval; the electronic device fast-forwards the third video interval and / or fast-forwards the fourth video interval to obtain the target video.

[0087] When processing the video overlap area, the electronic device can process the segments corresponding to the several beats of the two adjacent video intervals forming the video overlap area by fast-playing. Specifically, the first video is first parsed to obtain the third video interval and the fourth video interval forming the video overlap area, that is, there is an area with the same video content between the first video interval and the second video interval; in order to delete the video overlap area, the electronic device can fast-play the two video intervals to achieve a smooth connection between the two video intervals, obtain a target video with better quality, and maintain the fluency and continuity of the target video.

[0088] Implementation method 2: In the first video, the electronic device extracts two overlapping video intervals according to the overlapping width to obtain a video overlapping area; the electronic device extracts frames in the video overlapping area to obtain a target video.

[0089] In the process of processing the video overlapping area, the electronic device can first identify two overlapping video intervals (i.e., the third video area and the fourth video area) in the first video, and extract them according to the overlapping width to obtain the video overlapping area; then, in order to avoid the appearance of the video overlapping area, the electronic device can extract frames from the two video areas to achieve the deletion of repeated content. Specifically, according to a preset frame extraction algorithm, specific frames or frame sequences are extracted from the video overlapping area to obtain a target video with higher accuracy.

[0090] Optionally, the preset frame extraction algorithm may include: uniform frame extraction based on time intervals, key frame extraction based on image content, etc., aiming to extract the required frames or frame sequences from the video overlap area.

[0091] For example, Figure 2 As shown, it is a structural schematic diagram of the video blank area and the video overlap area provided by the present invention. In the process of processing the video blank area, the electronic device assumes that the width of the video blank area is 0.2s, and can process the video blank area in a slow-motion manner, or extract a certain length of video frames at the connection edge of the video blank area and the adjacent video interval for slow-motion, so as to achieve video filling. In the process of processing the video overlap area, the electronic device assumes that the width of the video overlap area is 0.2s, and can double the speed of 0.2s at the connection edge of the video overlap area and the adjacent video interval, so that the original 0.4s video is doubled. In order to make the connection smoother, it is also possible to expand a larger video interval. Of course, the beat action between the fast-beat video interval and the normal video interval can also be accelerated.

[0092] In some embodiments, the electronic device determines the target video based on the video blank area and / or the video overlap area in the first video, which may include at least one of the following: implementation method 1 and implementation method 2.

[0093] Implementation method 1: The electronic device determines the second video according to the video blank area and / or the video overlapping area in the first video; the electronic device replaces the audio corresponding to the second video with the audio corresponding to the preset rhythm type video to obtain the target video.

[0094] Since there may be background noise in the recorded video, the quality of the second video is poor. Therefore, in order to improve the quality of the second video, the audio corresponding to the second video can be replaced with the audio corresponding to the preset rhythm video, which effectively avoids the appearance of background noise and makes the rhythm of the user's actions in the second video fit the audio as closely as possible, so as to obtain a target video with better quality.

[0095] Implementation method 2: The electronic device determines the second video based on the video blank area and / or video overlapping area in the first video; when the second video has a blank or overlap due to translation, the electronic device translates the segment before the start and / or after the end of the second video to obtain the target video.

[0096] After determining the second video, the electronic device will check whether there are blanks or overlaps in the second video due to translation. This usually occurs when an object or scene in the second video moves. Due to changes in movement speed or direction, there is a mismatch between the previous and next frames. In order to correct these problems, the electronic device will translate the segment before the start and / or after the end of the second video, perform frame-by-frame analysis of the video frames, and adjust the content and position of the frames according to the movement trajectory and speed of the object to ensure the smoothness and continuity of the target video.

[0097] In the embodiment of the present invention, by combining the recorded video and the preset rhythm video, the sub-video in the recorded video is adjusted according to the average beat time difference of the two videos in the same target time period to obtain the target video. The whole process is relatively simple, which can save a lot of video processing time to improve the video processing efficiency. At the same time, it can optimize the problem of different synchronization between the video and audio beats, so that the quality of the final target video is better.

[0098] The video processing device provided by the present invention is described below. The video processing device described below and the video processing method described above can be referred to each other.

[0099] Figure 3 is a schematic diagram of the structure of the video processing device provided by the present invention, such as Figure 3 As shown, the device includes a data transceiver module 301 and a video processing module 302 .

[0100] The data transceiver module 301 is used to receive the recorded video sent by the audio game device. The recorded video is obtained by recording the beat movements made by the user based on the preset rhythm video when the audio game device plays the preset rhythm video.

[0101] The video processing module 302 is used to capture a sub-video corresponding to a target time period in the recorded video, and determine the preset beat data corresponding to the target time period in the preset rhythm class video; when it is determined that the current beat data in the sub-video does not match the preset beat data, determine the average beat time difference between the current beat data and the preset beat data; based on the average beat time difference, adjust the sub-video in the recorded video to obtain a target video, in which the beat data in the target video matches the beat data in the preset rhythm class video.

[0102] Optionally, the video processing module 302 is specifically used to extract the audio timeline corresponding to the target time period in the preset rhythm type video; according to the average beat time difference, in the recorded video, adjust the position of the sub-video on the audio timeline to obtain a first video; and determine the target video based on the video blank area and / or video overlapping area in the first video.

[0103] Optionally, the video processing module 302 is specifically used to determine, when the current beat data includes at least one first beat and the preset beat data includes at least one second beat, the first moment corresponding to each of the at least one first beat according to the time axis of the sub-video, and the second moment corresponding to each of the at least one second beat according to the audio time axis; when all the first moments are not exactly the same as all the second moments, determine that the current beat data and the preset beat data do not match; and / or, when the action score corresponding to the sub-video is less than a preset threshold, determine that the current beat data in the sub-video and the preset beat data do not match.

[0104] Optionally, the preset beat action corresponding to the target time period in the preset rhythm video includes at least one beat action, and the at least one beat action corresponds one-to-one to the at least one second beat; the video processing module 302 is specifically used to traverse the at least one beat action, and for the current beat action, determine the first target beat corresponding to the current beat action from the at least one first beat, and determine the second target beat corresponding to the current beat action from the at least one second beat; determine the beat time difference corresponding to the current beat action according to the first moment corresponding to the first target beat and the second moment corresponding to the second target beat; determine the average beat time difference according to the beat time difference corresponding to each of the at least one beat actions.

[0105] Optionally, the video processing module 302 is specifically used to determine, in the first video, a first video interval and a second video interval that form the video blank area, the playback time of the first video interval being earlier than the playback time of the second video interval; slow down the preset tail width of the first video interval, and / or slow down the preset head width of the second video interval to obtain the target video; or, in the first video, determine the first video interval and the second video interval that form the video blank area; insert frames in the first video interval, and / or insert frames in the second video interval to obtain the target video.

[0106] Optionally, the video processing module 302 is specifically used to determine, in the first video, a third video interval and a fourth video interval that form the video overlapping area, the playback time of the third video interval being earlier than the playback time of the fourth video interval; fast-forwarding the third video interval, and / or fast-forwarding the fourth video interval to obtain the target video; or, in the first video, extracting the two overlapping video intervals according to the overlapping width to obtain the video overlapping area; and extracting frames in the video overlapping area to obtain the target video.

[0107] Optionally, the video processing module 302 is specifically used to determine the second video according to the video blank area and / or the video overlapping area in the first video; replace the audio corresponding to the second video with the audio corresponding to the preset rhythm type video to obtain the target video.

[0108] like Figure 4 As shown, it is a schematic diagram of the structure of the electronic device provided by the present invention, and the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute the video processing method, which includes: receiving a recorded video sent by an audio and video device, wherein the recorded video is obtained by recording the beat movements made by the user based on the preset rhythm video when the audio and video device is playing the preset rhythm video; in the recorded video, intercepting a sub-video corresponding to a target time period, and determining the preset beat data corresponding to the target time period in the preset rhythm video; when it is determined that the current beat data in the sub-video does not match the preset beat data, determining the average beat time difference between the current beat data and the preset beat data; according to the average beat time difference, adjusting the sub-video in the recorded video to obtain a target video, wherein the beat data in the target video matches the beat data in the preset rhythm video.

[0109] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0110] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video processing method provided by the above-mentioned methods, which includes: receiving a recorded video sent by an audio and video device, wherein the recorded video is obtained by recording the beat movements made by the user based on the preset rhythm type video when the audio and video device plays the preset rhythm type video; in the recorded video, intercepting a sub-video corresponding to a target time period, and determining the preset beat data corresponding to the target time period in the preset rhythm type video; when it is determined that the current beat data in the sub-video does not match the preset beat data, determining the average beat time difference between the current beat data and the preset beat data; according to the average beat time difference, adjusting the sub-video in the recorded video to obtain a target video, and the beat data in the target video matches the beat data in the preset rhythm type video.

[0111] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the video processing method provided by the above-mentioned methods, the method comprising: receiving a recorded video sent by an audio and video device, wherein the recorded video is obtained by recording the beat movements made by the user based on the preset rhythm type video when the audio and video device is playing the preset rhythm type video; in the recorded video, intercepting a sub-video corresponding to a target time period, and determining the preset beat data corresponding to the target time period in the preset rhythm type video; when it is determined that the current beat data in the sub-video does not match the preset beat data, determining the average beat time difference between the current beat data and the preset beat data; according to the average beat time difference, adjusting the sub-video in the recorded video to obtain a target video, wherein the beat data in the target video matches the beat data in the preset rhythm type video.

[0112] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video processing method, characterized in that: include: Receiving a recorded video sent by a music game device, wherein the recorded video is obtained by recording a beat action performed by a user based on a preset rhythm video while the music game device is playing the preset rhythm video; In the recorded video, a sub-video corresponding to a target time period is intercepted, and preset beat data corresponding to the target time period in the preset rhythm video is determined; In the case where it is determined that the current beat data in the sub-video does not match the preset beat data, determining an average beat time difference between the current beat data and the preset beat data; According to the average beat time difference, the sub-video is adjusted in the recorded video to obtain a target video, and the beat data in the target video matches the beat data in the preset rhythm class video; wherein the adjusting the sub-video comprises: adjusting the position of the sub-video on the audio time axis corresponding to the target time period in the preset rhythm class video; The current beat data includes at least one first beat, the preset beat data includes at least one second beat, the preset beat action corresponding to the target time period in the preset rhythm video includes at least one beat action, and the at least one beat action corresponds to the at least one second beat in a one-to-one manner; determining the average beat time difference between the current beat data and the preset beat data includes: Traversing the at least one beat action, for the current beat action, determining a first target beat corresponding to the current beat action from the at least one first beat, and determining a second target beat corresponding to the current beat action from the at least one second beat; determining the beat time difference corresponding to the current beat action according to a first moment corresponding to the first target beat and a second moment corresponding to the second target beat; The average beat time difference is determined according to the beat time difference corresponding to each of the at least one beat movements.

2. The method according to claim 1, characterized in that The step of adjusting the sub-video in the recorded video according to the average beat time difference to obtain the target video includes: Extracting the audio timeline corresponding to the target time period in the preset rhythm video; According to the average beat time difference, in the recorded video, adjusting the position of the sub-video on the audio time axis to obtain a first video; The target video is determined according to a video blank area and / or a video overlap area in the first video.

3. The method according to claim 2, characterized in that The determining that the current beat data in the sub-video does not match the preset beat data includes: In the case where the current beat data includes at least one first beat and the preset beat data includes at least one second beat, determining the first moment corresponding to each of the at least one first beat according to the time axis of the sub-video, and determining the second moment corresponding to each of the at least one second beat according to the audio time axis; in the case where all first moments are not completely the same as all second moments, determining that the current beat data and the preset beat data do not match; and / or, When the action score corresponding to the sub-video is less than a preset threshold, it is determined that the current beat data in the sub-video does not match the preset beat data.

4. The method according to claim 2, characterized in that: Determining the target video according to the video blank area in the first video includes: In the first video, a first video interval and a second video interval forming the video blank area are determined, and the playing time of the first video interval is earlier than the playing time of the second video interval; slow-playing a preset tail width of the first video interval, and / or slow-playing a preset head width of the second video interval to obtain the target video; or, In the first video, a first video interval and a second video interval forming the video blank area are determined; frames are inserted into the first video interval, and / or frames are inserted into the second video interval to obtain the target video.

5. The method according to claim 2, characterized in that: Determining the target video according to the video overlap area in the first video includes: In the first video, a third video interval and a fourth video interval forming the video overlap area are determined, wherein the playback time of the third video interval is earlier than the playback time of the fourth video interval; the third video interval is fast-forwarded, and / or the fourth video interval is fast-forwarded to obtain the target video; or, In the first video, two overlapping video intervals are extracted according to the overlapping width to obtain the video overlapping area; and frames are extracted from the video overlapping area to obtain the target video.

6. The method according to claim 2, 4 or 5, characterized in that: After determining the target video, the method further includes: The audio corresponding to the target video is replaced with the audio corresponding to the preset rhythm video.

7. A video processing device, characterized in that: include: A data transceiver module, used for receiving a recorded video sent by a music game device, wherein the recorded video is obtained by recording a beat action performed by a user based on a preset rhythm video when the music game device is playing the preset rhythm video; A video processing module, used to intercept a sub-video corresponding to a target time period in the recorded video, and determine preset beat data corresponding to the target time period in the preset rhythm video; when it is determined that the current beat data in the sub-video does not match the preset beat data, determine the average beat time difference between the current beat data and the preset beat data; According to the average beat time difference, the sub-video is adjusted in the recorded video to obtain a target video, the beat data in the target video matches the beat data in the preset rhythm class video, and the video processing module is specifically used to adjust the position of the sub-video on the audio time axis corresponding to the target time period in the preset rhythm class video; The current beat data includes at least one first beat, the preset beat data includes at least one second beat, the preset beat action corresponding to the target time period in the preset rhythm video includes at least one beat action, and the at least one beat action has a one-to-one correspondence with the at least one second beat, and the video processing module is further used to traverse the at least one beat action, and for the current beat action, determine a first target beat corresponding to the current beat action from the at least one first beat, and determine a second target beat corresponding to the current beat action from the at least one second beat; The beat time difference corresponding to the current beat action is determined according to the first moment corresponding to the first target beat and the second moment corresponding to the second target beat; and the average beat time difference is determined according to the beat time difference corresponding to each of the at least one beat actions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the video processing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video processing method according to any one of claims 1 to 6 is implemented.

Images