A video processing method, apparatus, device and medium

Abstract

Embodiments of the present disclosure relate to a video processing method, device, equipment and medium, wherein the method comprises: obtaining a target video and a segment duration threshold, wherein the target video comprises a plurality of video frames; inputting the plurality of video frames into a cut point prediction model to output cut point probabilities of the video frames as video cut points; based on the plurality of cut point probabilities, the segment duration threshold and the total number of the plurality of video frames, extracting one of the plurality of cut point probabilities as a cut point threshold; determining a video frame with a cut point probability greater than or equal to the cut point threshold in the plurality of video frames as a video cut point; and cutting the target video into a plurality of video segments based on the video cut points. The present disclosure adaptively determines the corresponding cut point threshold according to the distribution of the video, overcomes the tediousness and inaccuracy of manually setting the corresponding threshold for different videos, effectively improves the threshold determination efficiency on the basis of ensuring the accuracy of the cut point threshold, and further improves the video cutting efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and in particular to a video processing method, apparatus, device and medium. Background Technology

[0002] Video is a crucial means of showcasing an item. A key step in video editing is segmenting the uploaded video according to scenes or transitions for subsequent secondary editing. Related technologies typically employ two main methods for video segmentation: traditional shot segmentation algorithms and deep learning-based shot segmentation algorithms. Traditional algorithms suffer from drawbacks such as susceptibility to misjudgments and inaccurate shot boundary detection. While deep learning-based algorithms can segment videos by predicting transition points, setting a universal threshold for the predicted probability leads to unsatisfactory segmentation results. Deep learning-based shot segmentation algorithms allow for manually setting appropriate thresholds for different videos to accurately determine segmentation points; however, this method is labor-intensive, cumbersome, and potentially less accurate, requiring further improvement. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a video processing method, apparatus, device, and medium.

[0004] This disclosure provides a video processing method, the method comprising:

[0005] Obtain the target video and the segment duration threshold, wherein the target video includes multiple video frames;

[0006] The multiple video frames are input into the segmentation point prediction model, and the output is the probability that each video frame is a video segmentation point.

[0007] Based on the multiple segmentation point probabilities, the segment duration threshold, and the total number of frames of the multiple video frames, one of the multiple segmentation point probabilities is extracted and determined as the segmentation point threshold;

[0008] Video frames in the plurality of video frames whose segmentation point probability is greater than or equal to the segmentation point threshold are determined as video segmentation points;

[0009] The target video is divided into multiple video segments based on the video segmentation points.

[0010] This disclosure also provides a video processing apparatus, the apparatus comprising:

[0011] An acquisition module is used to acquire a target video and a segment duration threshold, wherein the target video includes multiple video frames;

[0012] The probability module is used to input the multiple video frames into the segmentation point prediction model and output the probability that each video frame is a segmentation point of the video segmentation point.

[0013] The threshold module is used to extract one of the multiple segmentation point probabilities as the segmentation point threshold based on multiple segmentation point probabilities, the segment duration threshold, and the total number of frames of the multiple video frames;

[0014] The segmentation point module is used to identify video frames in the plurality of video frames whose segmentation point probability is greater than or equal to the segmentation point threshold as video segmentation points;

[0015] The segmentation module is used to segment the target video into multiple video segments based on the video segmentation points.

[0016] This disclosure also provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the video processing method provided in this disclosure.

[0017] This disclosure also provides a computer-readable storage medium storing a computer program for performing the video processing method provided in this disclosure.

[0018] Compared with the prior art, the technical solution provided in this disclosure has the following advantages: The video processing solution provided in this disclosure obtains a target video and a segment duration threshold, wherein the target video includes multiple video frames; the multiple video frames are input into a segmentation point prediction model, and the output is the segmentation point probability of each video frame as a video segmentation point; based on the multiple segmentation point probabilities, the segment duration threshold, and the total number of frames of the multiple video frames, one of the multiple segmentation point probabilities is extracted and determined as the segmentation point threshold; the video frames in the multiple video frames whose segmentation point probability is greater than or equal to the segmentation point threshold are determined as video segmentation points; the target video is segmented into multiple video segments based on the video segmentation points. By adopting the above technical solution, after determining the probability of segmentation points for each video frame, the segmentation point threshold can be dynamically determined based on the segment duration threshold, the probability of segmentation points for each video frame, and the total number of frames. The video segmentation points are then determined based on this segmentation point threshold to segment the video. This achieves adaptive determination of the corresponding segmentation point threshold based on the distribution of the video, overcoming the tediousness and inaccuracy of manually setting corresponding thresholds for different videos. It also avoids the problem of being unable to segment videos that are taken in one continuous shot. While ensuring the accuracy of the segmentation point threshold, it effectively improves the efficiency of threshold determination, thereby improving the efficiency of video segmentation. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A flowchart illustrating a video processing method provided in an embodiment of this disclosure;

[0021] Figure 2 A schematic diagram illustrating the probability of a segmentation point provided in an embodiment of this disclosure;

[0022] Figure 3 A flowchart illustrating another video processing method provided in this embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram illustrating a video processing procedure provided in an embodiment of this disclosure;

[0024] Figure 5 This is a schematic diagram of the structure of a video processing apparatus provided in an embodiment of the present disclosure;

[0025] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0026] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0027] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0028] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0029] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0030] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0032] The quality of video segmentation during production significantly impacts the final image quality. Video segmentation can typically be achieved through two main categories: traditional shot segmentation algorithms and deep learning-based shot segmentation algorithms. Traditional shot segmentation algorithms include pixel comparison, histogram comparison, and edge detection methods. These methods determine shot boundaries by comparing differences in pixels, color histograms, or image edges between adjacent frames. However, these methods are sensitive to changes in lighting and subtle object movements, prone to misjudgments, and inaccurately detect frames with similar colors but different content. They also perform poorly in scenes with subtle edge changes. Deep learning-based algorithms, on the other hand, typically construct their training data by stitching a single video together with various transition effects (such as hard cuts and fade-in). Through learning and training on a large amount of data, the model can automatically learn features to predict video transition points and segment shots based on all predicted transition points. This type of method often performs poorly with one-shot videos. Unlike traditional methods that infer based on scene changes, deep learning-based methods are trained based on transition labels. One-shot videos have no transitions, and setting a general threshold usually fails to segment any shots.

[0033] The shortcomings of the two video segmentation methods described above can be summarized as inflexible threshold settings. Because different videos are shot in different environments and using different techniques, a suitable threshold needs to be set individually for each video to accurately determine the segmentation point. If the threshold is set improperly, it will affect the result of the segmentation. A uniform threshold setting cannot meet the requirements of all videos. Related technologies allow for manually setting appropriate thresholds for different videos to accurately determine the segmentation point; however, this method is labor-intensive, cumbersome, and may have lower accuracy, thus requiring improvement.

[0034] To address the aforementioned problems, this disclosure provides a video processing method, which will be described below with reference to specific embodiments.

[0035] Figure 1This is a flowchart illustrating a video processing method provided in an embodiment of the present disclosure. The method can be executed by a video processing device, which can be implemented using software and / or hardware, and is generally integrated into an electronic device. Figure 1 As shown, the method includes:

[0036] Step 101: Obtain the target video and the segment duration threshold, wherein the target video includes multiple video frames.

[0037] The target video can be any video that needs to be segmented, regardless of its source. For example, it can be a live-action, one-shot video or a video downloaded from the internet. A video frame is the smallest unit that constitutes a video; a video frame can be understood as an image. The target video can include multiple video frames, with no limit on the specific number. The segment duration threshold represents the set duration range for the video segments after segmentation from the target video. When the duration of a video segment meets the threshold, the segment is usable. The segment duration threshold can include an upper limit and a lower limit, with the lower limit being less than the upper limit. For example, the segment duration threshold can be expressed as [2s, 5s], with a lower limit of 2s and an upper limit of 5s, meaning the duration of the segmented video segments is limited to between 2 and 5 seconds.

[0038] Specifically, the video processing device can acquire the target video and a pre-set segment duration threshold, which can be set according to actual business needs.

[0039] Step 102: Input multiple video frames into the segmentation point prediction model and output the probability of each video frame being a video segmentation point.

[0040] The segmentation point probability can be the probability of predicting a video frame as a video segmentation point, and the video segmentation point can be the specific segmentation point when a video is divided into multiple video segments.

[0041] A segmentation point prediction model can be a deep learning model used to predict the probability that each video frame is a video segmentation point, and the specific type of the segmentation point prediction model is not limited. For example, taking video segmentation points as transition points, the training data for the segmentation point prediction model can be constructed by adding various transition effects to sample videos and labeling the transition points, and then training a deep convolutional neural network with the training data.

[0042] Specifically, the video processing device can input multiple video frames of the target video into the segmentation point prediction model, analyze the probability of each video frame being a video segmentation point through the segmentation point prediction model, output the segmentation point probability corresponding to each video frame, and thus determine the multiple segmentation point probabilities of multiple video frames.

[0043] For example, Figure 2 This is a schematic diagram illustrating the probability of a segmentation point provided in an embodiment of this disclosure, such as... Figure 2 As shown in the figure, the target video is a one-shot video as an example. The horizontal axis of the figure is the frame index, and the vertical axis is the probability of the segmentation point corresponding to each video frame. The horizontal line in the figure represents the segmentation point threshold. The figure only shows the probability distribution of segmentation points corresponding to each video frame of a video. The probability distribution of segmentation points corresponding to different videos is completely different. Setting a uniform threshold is not applicable. It is necessary to dynamically set the corresponding segmentation point threshold.

[0044] Step 103: Based on multiple segmentation point probabilities, segment duration thresholds, and the total number of frames in multiple video frames, extract one of the multiple segmentation point probabilities and determine it as the segmentation point threshold.

[0045] The total number of frames can be the specific number of video frames included in the target video. The segmentation threshold can be a minimum value set for the segmentation probability of a video frame. When the segmentation probability of a video frame is greater than the segmentation threshold, the video frame is considered a video segmentation point.

[0046] Specifically, after determining the probabilities of multiple segmentation points for multiple video frames of the target video, the video processing device can calculate and extract one of the multiple segmentation point probabilities as the segmentation point threshold based on the multiple segmentation point probabilities, the segment duration threshold, and the total number of video frames, considering the minimum duration difference of the segmented video segments.

[0047] For example, Figure 3 This is a flowchart illustrating another video processing method provided in an embodiment of the present disclosure. In one feasible implementation, based on multiple segmentation point probabilities, a segment duration threshold, and the total number of frames in multiple video frames, one of the multiple segmentation point probabilities is extracted and determined as the segmentation point threshold. This may include the following steps:

[0048] Step 301: Based on the segment duration threshold and the total number of frames, extract multiple candidate thresholds from the probabilities of multiple segmentation points.

[0049] The candidate threshold can be a preliminarily determined segmentation point threshold, or it can include segmentation point probabilities that are initially extracted from multiple segmentation point probabilities, and there can be multiple candidate thresholds. The segment duration threshold can represent the duration range set for video segments after segmenting the target video. The segment duration threshold can include an upper limit value and a lower limit value for the segment duration, and the lower limit value for the segment duration is less than the upper limit value for the segment duration.

[0050] In some embodiments, extracting multiple candidate thresholds from multiple segmentation point probabilities based on segment duration thresholds and total number of frames may include: determining a segmentation point number threshold based on an upper limit of segment duration, a lower limit of segment duration, total number of frames, and the frame rate of the target video; determining a candidate frame range based on an upper limit of the number of segmentation points, a lower limit of the number of segmentation points, and total number of frames; and determining multiple segmentation point probabilities corresponding to the candidate frame range from the multiple segmentation point probabilities as multiple candidate thresholds.

[0051] The threshold for the number of video segmentation points can be a range of video segmentation points determined based on a segment duration threshold. This threshold can include an upper limit and a lower limit. The upper limit can be the maximum number of video segmentation points, and the lower limit can be the minimum number of video segmentation points. The upper limit is greater than the lower limit; for example, the upper limit could be 10 and the lower limit 5 (this is just an example). The candidate frame range can be a range of frames extracted from multiple video frames after sorting them in ascending order of segmentation point probabilities, based on the threshold for the number of video segmentation points. Selecting a segmentation point probability as the threshold within this candidate frame range ensures that the number of video segmentation points remains within the range specified by the threshold. This candidate frame range is used to extract the segmentation point probabilities corresponding to each video frame from multiple video frames as preliminary candidate thresholds.

[0052] Specifically, when determining multiple candidate thresholds, the video processing device can input the upper and lower limits of the segment duration threshold, as well as the total number of frames and the frame rate of the target duration, into the formula. and In the calculation, the upper limit and lower limit of the number of cutting points are obtained, where n min n represents the lower limit of the number of split points. max This represents the upper limit of the number of segmentation points, where n represents the total number of frames in the target video, and t represents the maximum number of segments. high t represents the upper limit of the segment duration. lowHere, represents the lower limit of the segment duration, and fps represents the frame rate of the target video. Then, the first difference between the total number of frames and the upper limit of the number of segmentation points, and the second difference between the total number of frames and the lower limit of the number of segmentation points, are calculated. Multiple video frames are then sorted in ascending order of segmentation point probability. Based on the sorting results, the first video frame ranked at the first difference is determined as the first frame in the candidate frame range, and the second video frame ranked at the second difference is determined as the last frame in the candidate frame range. The candidate frame range can be represented as j∈[nn]. max’ nn min ],nn max Let nn represent the first difference. min This represents the second difference. After determining the candidate frame range, multiple video frames between the first and second video frames can be extracted from multiple video frames, and the probabilities of multiple segmentation points corresponding to the extracted video frames can be determined as multiple candidate thresholds.

[0053] Step 302: Determine the segment duration difference value corresponding to each candidate threshold.

[0054] The segment duration difference can be the duration difference between adjacent video segments after the target video is segmented according to a segmentation threshold. The variance of adjacent video segments can be used to represent the segment duration difference; the smaller the variance, the smaller the segment duration difference.

[0055] In some embodiments, determining the segment duration difference value corresponding to each candidate threshold includes: for each candidate threshold, determining the duration of candidate segment segments of the target video based on the candidate threshold, and determining the variance of the duration of the candidate segment segments, and determining the variance as the segment duration difference value corresponding to the candidate threshold.

[0056] The candidate segment duration can be the duration of the video segment after the target video is segmented according to the candidate threshold. The corresponding candidate segment duration is determined for each candidate threshold.

[0057] Specifically, after determining multiple candidate thresholds, the video processing device determines a corresponding segment duration difference value for each candidate threshold. For example, multiple candidate thresholds can be represented as follows: With a candidate threshold y j For example, determine if y is greater than or equal to y. j The probability of the split point includes y j ,y j+1 ,...,y n y j ,y j+1 ,...,y n The corresponding video frame at the video segmentation point of the target video is represented as follows: This indicates the number of video segmentation points at this point. The difference between adjacent video segmentation points on the time axis is calculated, which represents the duration of the candidate segment. The duration of this candidate segment is expressed as:

[0058]

[0059] Indicates the video segmentation point. The variance of the candidate segment duration is calculated using the following formula:

[0060]

[0061] μ j The mean of the candidate segment duration can be expressed as:

[0062]

[0063] Calculate the variance of the candidate segment duration. The difference in segment duration was determined.

[0064] Step 303: Determine the candidate threshold with the smallest difference in segment duration among multiple candidate thresholds as the segmentation threshold.

[0065] Specifically, after determining the duration difference values ​​of multiple segments among multiple candidate thresholds, the video processing device can extract the candidate threshold with the smallest duration difference value as the segment threshold, and the segment threshold y th =y q y th Represents the threshold for the split point, y q The candidate threshold representing the minimum difference in segment duration satisfies...

[0066] In the above scheme, by determining the probability of the segmentation point with the smallest difference in segment duration as the segmentation point threshold, the time difference between each shot of the segmented segment can be minimized, making the duration distribution of the segmented segment more uniform.

[0067] Step 104: Determine the video frames in which the probability of the segmentation point is greater than or equal to the segmentation point threshold as video segmentation points.

[0068] A video segmentation point can be a specific point where a video is divided into multiple video segments. In this embodiment of the disclosure, a video segmentation point can be a transition point or a shot segmentation point. A transition point can be the position where two shots or scenes in a video switch. Clever transition techniques can make the video smoother and more coherent. A shot segmentation point can be the position where a video is divided according to different shots. Shot segmentation points are determined based on factors such as the continuity of video content, changes in action, changes in composition, or emotional expression.

[0069] Specifically, after determining the segmentation point threshold, the video processing device can compare the segmentation point probability of each video in multiple videos with the segmentation point threshold. If the segmentation point probability is greater than or equal to the segmentation point threshold, the video frame corresponding to the segmentation point probability is a video segmentation point, thus obtaining multiple video segmentation points.

[0070] Step 105: Divide the target video into multiple video segments based on video segmentation points.

[0071] Among them, video clips can be short segments obtained by dividing the target video. A video clip can be the smallest segment that is continuous and visually continuous over a period of time. Dividing the target video into multiple video clips helps with subsequent video editing.

[0072] After determining the video segmentation point, the video processing device can segment the target video at the segmentation point to obtain multiple video segments.

[0073] The video processing scheme provided in this disclosure obtains a target video and a segment duration threshold. The target video includes multiple video frames. The multiple video frames are input into a segmentation point prediction model, which outputs the probability of each video frame being a segmentation point. Based on the multiple segmentation point probabilities, the segment duration threshold, and the total number of frames in the multiple video frames, one of the multiple segmentation point probabilities is extracted and determined as the segmentation point threshold. Video frames with a segmentation point probability greater than or equal to the segmentation point threshold are determined as video segmentation points. The target video is then segmented into multiple video segments based on the video segmentation points. Using this technical solution, after determining the segmentation point probabilities of each video frame, the segmentation point threshold can be dynamically determined based on the segment duration threshold, the segmentation point probabilities of each video frame, and the total number of frames. The video segmentation points are then determined based on this segmentation point threshold to segment the video. This achieves adaptive determination of the corresponding segmentation point threshold according to the distribution of the video, overcoming the tediousness and uncertainty of manually setting corresponding thresholds for different videos. While ensuring the accuracy of the segmentation point threshold, it effectively improves the threshold determination efficiency, thereby improving the video segmentation efficiency.

[0074] In some embodiments, after the target video is divided into multiple video segments based on video segmentation points, the video processing method of this disclosure may further include: extracting video segments that do not meet the segment duration threshold from the multiple video segments as video segments to be processed; and performing post-processing on the video segments to be processed.

[0075] The video segments to be processed can be those whose duration does not meet the segment duration threshold and require further processing to meet it. The number of video segments to be processed can be one or more, depending on the specific circumstances. Post-processing can involve deleting or further segmenting the video segments to be processed, ensuring that the post-processed video segments meet the segment duration threshold.

[0076] After determining multiple video segments, the video processing device can determine whether each video segment meets the segment duration threshold, that is, whether the duration of each video segment is greater than or equal to the lower limit of the segment duration threshold and less than or equal to the upper limit of the segment duration threshold. If so, the video segment meets the segment duration threshold; otherwise, the video segment is a video segment to be processed, and further post-processing is performed on the video segment to be processed.

[0077] Optionally, post-processing of the video segment to be processed may include: if the duration of the video segment to be processed is less than the lower limit of the segment duration threshold, then the video segment to be processed is deleted; if the duration of the video segment to be processed is greater than the upper limit of the segment duration threshold, then the video segment to be processed is segmented using a binary search method until the segmented video segments meet the segment duration threshold.

[0078] When post-processing video segments, the video processing device first compares the duration of the video segment with a segment duration threshold. If the duration of the video segment is less than the lower limit, it means that the video segment is too short to be used, and the video segment is deleted. If the duration of the video segment is greater than the upper limit, the video segment is further segmented. For example, a binary search method is used, where the video frame with the highest probability of being segmented is used as the segmentation point to obtain two video segments. These two video segments are then checked to see if they meet the segment duration threshold. If they do, they are retained. Video segments that do not meet the segment duration threshold are extracted as new video segments to be processed and returned for further post-processing until all segmented video segments meet the segment duration threshold. By recursively segmenting using the binary search method, all video segments meet the specified duration requirement and are usable.

[0079] The relevant technologies do not take into account the duration of the segmented video segments when processing video segmentation, and do not take into account the actual requirements for the duration of the resulting video. The video segments cut in this way have poor usability and may be too short or too long, making them unusable for subsequent video editing processes.

[0080] In the above scheme, the video segmentation also incorporates the duration of the segmented video segments into the segmentation strategy to ensure that the segmented videos meet the predetermined duration requirements. This effectively avoids the situation where the segmented segments are unusable due to being too long or too short, solves the problem of poor usability caused by the segmented segments not meeting the duration requirements, improves the usability of video segments, and thus helps with subsequent video editing.

[0081] In some embodiments, after determining video frames with a segmentation point probability greater than or equal to a segmentation point threshold as video segmentation points, the video processing method of this disclosure embodiment may further include: if there are at least two consecutive video frames that are video segmentation points, then deleting other video segmentation points from the at least two video segmentation points corresponding to the at least two consecutive video frames except for the one with the highest segmentation point probability.

[0082] Since there are multiple video segmentation points, after determining multiple video segmentation points, the video processing device can determine whether there are at least two consecutive video frames that are video segmentation points, that is, there is continuous high-frequency oscillation. In this case, only the video segmentation point corresponding to the highest segmentation point probability among the at least two video segmentation points can be retained, and other video segmentation points can be deleted. That is, the video segmentation point that takes the peak point at the continuous point suppresses other video segmentation points, effectively avoiding the segmented video segments being too short and improving the usability of the video segments.

[0083] The video processing scheme of this disclosure embodiment will be further illustrated by a specific example below. For example, Figure 4 This is a schematic diagram of a video processing process provided in an embodiment of the present disclosure, such as... Figure 4 As shown in the figure, the segmentation process of the target video can be divided into the following steps: starting; inputting a video list and a segment duration threshold, wherein the video list includes multiple video frames of the target video; using a segmentation point prediction model to predict the probability of the corresponding segmentation point frame by frame; determining the segmentation point threshold, the specific implementation process of which is described in the above embodiment and will not be repeated here; after dividing the target video into multiple video segments using the segmentation point threshold, post-processing is performed on the video segments that do not meet the segment duration threshold; and ending the segmentation.

[0084] This invention proposes an adaptive threshold adjustment video segmentation scheme that can adaptively set the segmentation point threshold according to the distribution of each video, eliminating the complexity and inaccuracy of manually setting the segmentation threshold for each video in traditional video segmentation schemes, and avoiding the problem that a one-shot video cannot be segmented. Furthermore, in the actual video segmentation process, the requirements of the actual application for the length of the resulting video are taken into account, resolving the problem of poor usability caused by the segmentation result video not meeting the length requirements.

[0085] Figure 5This is a schematic diagram of a video processing apparatus provided in an embodiment of the present disclosure. The apparatus can be implemented by software and / or hardware and is generally integrated into an electronic device. Figure 5 As shown, the device includes:

[0086] The acquisition module 501 is used to acquire the target video and the segment duration threshold, wherein the target video includes multiple video frames;

[0087] The probability module 502 is used to input the plurality of video frames into the segmentation point prediction model and output the probability that each video frame is a segmentation point of the video segmentation point.

[0088] Threshold module 503 is used to extract one of the multiple segmentation point probabilities as the segmentation point threshold based on multiple segmentation point probabilities, the segment duration threshold, and the total number of frames of the multiple video frames;

[0089] The segmentation point module 504 is used to determine video frames in the plurality of video frames whose segmentation point probability is greater than or equal to the segmentation point threshold as video segmentation points;

[0090] The segmentation module 505 is used to segment the target video into multiple video segments based on the video segmentation points.

[0091] Optionally, the threshold module 503 includes:

[0092] The extraction unit is used to extract multiple candidate thresholds from multiple segmentation point probabilities based on the segment duration threshold and the total number of frames;

[0093] The difference unit is used to determine the segment duration difference value corresponding to each of the candidate thresholds;

[0094] The determining unit is used to determine the candidate threshold with the smallest difference in segment duration among the multiple candidate thresholds as the segmentation point threshold.

[0095] Optionally, the segment duration threshold includes an upper limit value and a lower limit value for segment duration, and the extraction unit is used for:

[0096] Based on the upper limit of the segment duration, the lower limit of the segment duration, the total number of frames, and the frame rate of the target video, a threshold for the number of segmentation points is determined, wherein the threshold for the number of segmentation points includes an upper limit and a lower limit.

[0097] The candidate frame range is determined based on the upper limit of the number of segmentation points, the lower limit of the number of segmentation points, and the total number of frames;

[0098] Multiple segmentation point probabilities corresponding to the candidate frame range among the multiple segmentation point probabilities are determined as multiple candidate thresholds. Optionally, the difference unit is used for:

[0099] For each candidate threshold, the duration of a candidate segment of the target video is determined based on the candidate threshold, and the variance of the duration of the candidate segment is determined. This variance is then used as the segment duration difference value corresponding to the candidate threshold.

[0100] Optionally, the device further includes a post-processing module for: dividing the target video into multiple video segments based on the video segmentation points,

[0101] Extract video segments from the plurality of video segments that do not meet the segment duration threshold as video segments to be processed;

[0102] Post-processing is performed on the video segment to be processed.

[0103] Optionally, the post-processing module is specifically used for:

[0104] If the duration of the video segment to be processed is less than the lower limit of the segment duration threshold, then the video segment to be processed is deleted.

[0105] If the duration of the video segment to be processed is greater than the upper limit of the segment duration threshold, then the video segment to be processed is divided using a binary search method until the divided video segments meet the segment duration threshold.

[0106] Optionally, the device further includes a deletion module, configured to: after determining video frames in the plurality of video frames whose segmentation point probability is greater than or equal to the segmentation point threshold as video segmentation points,

[0107] If there are at least two consecutive video frames that are video segmentation points, then delete the other video segmentation points from the at least two video segmentation points corresponding to the at least two consecutive video frames, except for the one with the highest probability of segmentation.

[0108] The video processing apparatus provided in this disclosure can execute the video processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0109] This disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the video processing method provided in any embodiment of this disclosure.

[0110] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present disclosure. See below for details. Figure 6The diagram illustrates a structural schematic suitable for implementing the electronic device 600 in the embodiments of this disclosure. The electronic device 600 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0111] like Figure 6 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0112] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0113] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the video processing method of embodiments of this disclosure.

[0114] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0115] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0116] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0117] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire a target video and a segment duration threshold, wherein the target video comprises multiple video frames; input the multiple video frames into a segmentation point prediction model, and output the segmentation point probability of each video frame as a video segmentation point; based on the multiple segmentation point probabilities, the segment duration threshold, and the total number of frames of the multiple video frames, extract one of the multiple segmentation point probabilities and determine it as a segmentation point threshold; determine the video frames among the multiple video frames whose segmentation point probability is greater than or equal to the segmentation point threshold as video segmentation points; and segment the target video into multiple video segments based on the video segmentation points.

[0118] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0120] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0121] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0122] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0123] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0124] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0125] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0126] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method of video processing, the method comprising: The method comprises: obtaining a target video and a segment duration threshold, wherein the target video comprises a plurality of video frames; inputting the plurality of video frames into a cut point prediction model to output cut point probabilities of each of the video frames as video cut points; based on a plurality of cut point probabilities, the segment duration threshold and the total number of the plurality of video frames, extracting one of the plurality of cut point probabilities as a cut point threshold; determining, as video cut points, video frames in the plurality of video frames whose cut point probabilities are greater than or equal to the cut point threshold; based on the video cut points, cutting the target video into a plurality of video segments.

2. The method of claim 1, wherein, Based on a plurality of cut point probabilities, the segment duration threshold and the total number of the plurality of video frames, extracting one of the plurality of cut point probabilities as a cut point threshold, comprising: based on the segment duration threshold and the total number of frames, extracting a plurality of candidate thresholds from the plurality of cut point probabilities; determining a segment duration difference value corresponding to each of the candidate thresholds; determining, as the cut point threshold, the candidate threshold with the smallest segment duration difference value among the plurality of candidate thresholds.

3. The method of claim 2, wherein, The segment duration threshold comprises an upper limit value of segment duration and a lower limit value of segment duration, and based on the segment duration threshold and the total number of frames, extracting a plurality of candidate thresholds from the plurality of cut point probabilities, comprising: based on the upper limit value of segment duration, the lower limit value of segment duration, the total number of frames and the frame rate of the target video, determining a cut point number threshold, wherein the cut point number threshold comprises an upper limit value of cut point number and a lower limit value of cut point number; based on the upper limit value of cut point number, the lower limit value of cut point number and the total number of frames, determining a candidate frame range; determining, as a plurality of candidate thresholds, a plurality of cut point probabilities corresponding to the candidate frame range among the plurality of cut point probabilities.

4. The method of claim 2, wherein, Determining a segment duration difference value corresponding to each of the candidate thresholds, comprising: for each of the candidate thresholds, determining a candidate cut segment duration of the target video based on the candidate threshold, and determining a variance of the candidate cut segment duration, which is determined as the segment duration difference value corresponding to the candidate threshold.

5. The method of claim 1, wherein, After cutting the target video into a plurality of video segments based on the video cut points, the method further comprises: extracting a video segment that does not satisfy the segment duration threshold from the plurality of video segments as a to-be-processed video segment; post-processing the to-be-processed video segment.

6. The method of claim 5, wherein, Post-processing the to-be-processed video segment, comprising: if the duration of the to-be-processed video segment is less than the lower limit value of the segment duration threshold, deleting the to-be-processed video segment; if the duration of the to-be-processed video segment is greater than the upper limit value of the segment duration threshold, cutting the to-be-processed video segment using a dichotomy method until the cut video segment satisfies the segment duration threshold.

7. The method of claim 1, wherein, After determining, as video cut points, video frames in the plurality of video frames whose cut point probabilities are greater than or equal to the cut point threshold, the method further comprises: If there are at least two continuous video frames which are all video split points, other video split points in the at least two continuous video frames corresponding to the at least two video split points are deleted except for a maximum split point probability.

8. A video processing apparatus, comprising: Comprise: An acquisition module is configured to acquire a target video and a segment duration threshold, wherein the target video comprises a plurality of video frames; A probability module is configured to input the plurality of video frames into a split point prediction model, and output a split point probability that each of the video frames is a video split point; A threshold module is configured to extract one of the plurality of split point probabilities as a split point threshold based on the plurality of split point probabilities, the segment duration threshold, and a total number of the plurality of video frames; A split point module is configured to determine a video frame in the plurality of video frames as a video split point if a split point probability of the video frame is greater than or equal to the split point threshold; A split module is configured to split the target video into a plurality of video segments based on the video split points.

9. An electronic device, comprising: The electronic device comprises: A processor; A memory for storing executable instructions of the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the video processing method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the video processing method of any one of claims 1-7.

Images