Automatic video editing method based on AI split mirror generation

Abstract

The invention relates to the technical field of video editing, in particular to an automatic video editing method based on AI split mirror generation, which comprises the following steps: analyzing each video clip in a preliminary video edited based on a template to determine whether the editing of each video clip is qualified or not, detecting the material matching accuracy of the video, and judging whether the video clip is qualified or not; when it is determined that editing of a single video clip is abnormal, it is determined that corresponding material matching is abnormal, and processing parameters for the single video clip are adjusted to re-match materials, including correcting coordinates of the video clip, correcting a time axis of the single video clip or calibrating the single video clip as an abnormal clip; according to the method and the device, when whether the editing of each video clip in the preliminary video is qualified is determined subsequently, whether the generation of the preliminary video is qualified is determined based on the calibrated statistical number of each video clip, and corresponding notification information is further sent according to the abnormal clip, so that the editing precision is improved, and the editing efficiency of the video is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video editing, and particularly to an automated video editing method based on AI scene generation. Background Art

[0002] Early video editing mainly relied on linear editing devices, which physically cut and spliced film or tape, with complex operations and difficult modification. With the development of computer technology, non-linear editing software emerged, such as Adobe Premiere Pro, Final Cut Pro, etc. Editors can edit digital video materials on a computer, greatly improving the flexibility and efficiency of editing. However, these software still require manual operation and have high requirements for the professional skills of editors.

[0003] In recent years, artificial intelligence technology has developed rapidly. Especially, deep learning algorithms have achieved remarkable results in fields such as image recognition, speech recognition, and natural language processing. Image recognition technology can accurately identify objects, people, and scenes in images, providing a basis for video content analysis; machine learning algorithms can learn the patterns and rules of video editing through a large amount of data, thus realizing automatic editing decisions.

[0004] Driven by artificial intelligence technology, some research institutions and enterprises have begun to explore automated video editing technology. Early attempts mainly focused on simple video splicing and basic special effect addition, with limited video quality and flexibility. With the continuous progress of technology, template-based automated video editing systems have gradually emerged.

[0005] Chinese Patent Publication No.: CN118660115A discloses a video data scene generation method and system for intelligent video generation, including: obtaining structured video data; extracting the copywriting and scenes from the video data, calculating the matching index between the copywriting and the scenes, and creating a mapping relationship between the copywriting and the scenes based on the matching index to generate a training data set; performing the first fine-tuning on the large language model based on the training data set and the prediction result scoring formula to enable it to have the ability to generate scenes; performing the second fine-tuning on the large language model to optimize the scene generation ability of the model; using the fine-tuned large language model to intelligently generate a copywriting and scene sequence according to the input keywords and storing the generated copywriting and scene sequence in the database. It can be seen that the above technical solution has the following problems: it is impossible to monitor the accuracy of material matching for videos edited based on templates, which affects the editing efficiency of videos. Summary of the Invention

[0006] To this end, the present invention provides an automated video editing method based on AI storyboard generation to overcome the problem in the prior art that the accuracy of material matching for videos edited based on templates cannot be monitored, which affects the editing efficiency of videos.

[0007] To achieve the above object, the present invention provides an automated video editing method based on AI storyboard generation, including:

[0008] S1, automatically generate a storyboard script, and sequentially construct corresponding storyboard information for each storyboard, including the preset contour information of features in each frame of the video and the storyboard duration;

[0009] S2, obtain the materials shot by the user according to the storyboard script;

[0010] S3, material matching, perform matching processing on each video segment included in the materials with the corresponding storyboard information;

[0011] S4, cut each video segment to remove the redundant pictures at the beginning and end of the video segment;

[0012] S5, splice each video segment according to the order of each storyboard of the storyboard script to complete the generation of the preliminary video;

[0013] S6, based on the feature contour evaluation parameter and the storyboard duration, determine one by one whether the editing of each video segment in the preliminary video is qualified, including,

[0014] Determine that the editing of a single video segment in the preliminary video is qualified, and label the video segment as a qualified segment;

[0015] Or, the editing of a single video segment in the preliminary video is abnormal, and adjust the processing parameters of the single video segment based on the proportion difference amount, where the processing parameters include correcting the coordinates of the video segment, correcting the time axis of the single video segment, or labeling the single video segment as an abnormal segment;

[0016] S7, when determining whether the editing of each video segment in the preliminary video is qualified, determine whether the generation of the preliminary video is qualified based on the statistical quantity of the labeled video segments, including,

[0017] Send a notice message for unqualified materials, send a notice message for regenerating the storyboard script, or determine that the generation of the preliminary video is qualified and output the generated video.

[0018] Further, in the S6, determining whether the editing of each video segment in the preliminary video is qualified based on the feature contour evaluation parameter includes:

[0019] Obtain several time nodes of a single video segment;

[0020] Compare the contour features in the frame at each time node with the preset contour information of the features in the corresponding storyboard shot;

[0021] Calculate the area ratio of the overlapping area to the area of the preset contour information of the features in the corresponding storyboard shot for a single time node;

[0022] Solve the average value of each area ratio for a single video clip to obtain the feature contour evaluation parameter for the single video clip;

[0023] If the feature contour evaluation parameter is less than or equal to the first preset evaluation parameter, it is determined that the editing of the single video clip in the preliminary video is abnormal, and the processing parameters for the single video clip are adjusted based on the proportion difference amount;

[0024] If the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, determine whether the editing of the single video clip in the preliminary video is qualified based on the shot duration of the storyboard shot corresponding to the single video clip;

[0025] If the feature contour evaluation parameter is greater than the second preset evaluation parameter, it is determined that the editing of the single video clip in the preliminary video is qualified, and the video clip is marked as a qualified clip.

[0026] Further, determining whether the editing of the single video clip in the preliminary video is qualified based on the shot duration of the storyboard shot corresponding to the single video clip includes:

[0027] If the shot duration is less than or equal to the preset shot duration, adjust the first preset evaluation parameter and the second preset evaluation parameter to the corresponding values based on the shot duration;

[0028] If the shot duration is greater than the preset shot duration, it is determined that the editing of the single video clip in the preliminary video is abnormal, and the processing parameters for the single video clip are adjusted based on the proportion difference amount.

[0029] Further, adjusting the first preset evaluation parameter and the second preset evaluation parameter to the corresponding values based on the shot duration, where

[0030] The reduction amplitudes of the first preset evaluation parameter and the second preset evaluation parameter are inversely proportional to the shot duration.

[0031] Further, adjusting the processing parameters for the single video clip based on the proportion difference amount includes:

[0032] Record the variance of the calculated area ratios at each time node for the single video clip as the proportion difference amount;

[0033] If the proportion difference amount is less than or equal to the first preset difference amount, correct the coordinates of the video clip;

[0034] If the difference in proportion is less than or equal to the second preset difference and greater than the first preset difference, the timeline of a single video clip is corrected based on the area ratio of adjacent time nodes;

[0035] If the difference in proportion is greater than the second preset difference, the single video clip is marked as an abnormal clip.

[0036] Further, the process of correcting the timeline of a single video clip based on the area ratio of adjacent time nodes includes:

[0037] In a single storyboard, select the feature preset contour information of a single video frame, calculate the overlapping area between the feature preset contour information and the contour feature of the video clip at the corresponding time node, and record it as the current overlapping area; calculate the overlapping area between the preset contour feature and the contour feature of the video clip in the previous video frame, and record it as the historical overlapping area, and record the ratio of the calculated current overlapping area to the historical overlapping area as the time difference ratio;

[0038] If the time difference ratio is greater than one, the timeline of the video clip is delayed, and the corrected duration of the delay is determined based on the time difference ratio;

[0039] If the time difference ratio is equal to one, reselect a single video frame in the single storyboard, and correct the timeline of the single video clip based on the newly determined time difference ratio;

[0040] If the time difference ratio is less than one, the timeline of the video clip is advanced, and the corrected duration of the advance is determined based on the time difference ratio.

[0041] Further, the corrected duration of the advance is determined based on the time difference ratio, where

[0042] The absolute value of the difference between the calculated time difference ratio and one is recorded as the correction reference value;

[0043] The increase amplitude of the corrected duration is proportional to the correction reference value.

[0044] Further, in S7, the process of determining whether the generation of the preliminary video is qualified based on the statistical quantity of each calibrated video clip includes:

[0045] Calculate the difference between the number of clips marked as qualified and the number of abnormal clips, and record the ratio of the solved difference to the number of clips marked as qualified as the compliance ratio;

[0046] If the compliance ratio is equal to one, it is determined that the generation of the preliminary video is qualified, and the generated video is output;

[0047] If the compliance ratio is less than one and greater than the preset compliance parameter, a notice message for unqualified material is sent;

[0048] If the compliance ratio is less than or equal to the preset compliance parameter, a notification message for regenerating the storyboard script is sent.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows: each video segment in the preliminary video based on template editing is analyzed to determine whether the editing of each video segment is qualified, the accuracy of material matching of the video is detected, when it is determined that the editing of a single video segment is abnormal, it is determined that the corresponding material matching is abnormal, and the processing parameters for the single video segment are adjusted to rematch the material, including correcting the coordinates of the video segment, correcting the time axis of the single video segment, or marking the single video segment as an abnormal segment. Subsequently, when it is determined whether the editing of each video segment in the preliminary video is qualified, it is determined whether the generation of the preliminary video is qualified based on the statistical quantity of the calibrated video segments, and corresponding notification information is further sent according to the abnormal segments, improving the editing accuracy and further improving the editing efficiency of the video.

[0050] Further, for a single video segment, several time nodes are randomly selected or selected according to a certain rule. At each time node, the overlapping area between the contour feature in the picture and the preset contour information of the feature in the corresponding storyboard is calculated, and the feature contour evaluation parameter is determined. The feature contour evaluation parameter characterizes the material matching situation of the contour feature. When the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, it is determined whether the editing of the single video segment is qualified by comprehensively considering the storyboard duration of the single video segment. When the storyboard duration is less than or equal to the preset storyboard duration, the storyboard duration is short, and the determination may be deviated due to too few selected node numbers. At this time, the determination criterion is adjusted according to the difference in the storyboard duration, and the determination criterion is comprehensively determined according to the actual situation of the storyboard in specific cases, further improving the detection accuracy of the single video segment while improving the data processing efficiency.

[0051] Furthermore, based on the proportion difference amount, the processing parameters for a single video segment are adjusted to analyze and process the situation where a single video segment is unqualified. The proportion difference amount characterizes the stability of material matching in a single video segment. When the proportion difference amount is less than or equal to the first preset difference amount, the contour features in a single video segment generally have a stable deviation from the feature preset contour information. In this case, there is a deviation between the picture of the video segment and the expected picture in the corresponding storyboard. The picture is skewed, and the scaling ratio is inconsistent, resulting in a deviation between the position of the actual key elements and the expected position. At this time, the coordinates of the video segment are corrected through an image correction algorithm, and perspective transformation, rotation, and translation operations are used to adjust the picture to a position that better matches the feature preset contour information; when the proportion difference amount is less than or equal to the second preset difference amount and greater than the first preset difference amount, at this time, the contour features in a single video segment generally show a relatively discrete deviation from the position of the feature preset contour information. At this time, due to problems with the time axis, the offset of the contour features occurs irregularly. In this case, the overall time axis of a single video segment is moved forward and backward in the time dimension to perform matching in the time dimension, improve the adaptability of video editing, and perform reasonable editing and matching for materials with different shooting qualities. While improving the accuracy of video editing, the editing efficiency of the video is further improved.

[0052] Furthermore, for the correction of the time axis, a single video frame is selected in a single storyboard for analysis, and the video frame of the video segment corresponding to the video frame in the storyboard at the corresponding time node is obtained to determine the overlapping area; by comparing the overlapping area with the historical overlapping area, the advance and delay of the time axis are determined. When the time difference ratio is greater than one, in this case, the feature preset contour information of a single video frame in a single storyboard better matches the contour features of the video segment at the corresponding time node. Considering the premise that the time axis is abnormal, this situation is a case where the picture is ahead, and the time axis of the video segment is delayed; when the time difference ratio is equal to one, the picture of the video segment is stationary during this period. At this time, the node is changed, and the advance or delay of the time axis is determined according to the newly determined time difference ratio; when the time difference ratio is less than one, the current picture is slower than expected, and the time axis of the video segment is advanced; accurately identify the abnormal situation of the time axis and make adjustments, while ensuring the accuracy of video editing, further improving the editing efficiency of the video.

[0053] Further, after the determination of each video clip is completed, an overall determination is made based on the compliance ratio, which represents the proportion of abnormal clips. When the compliance ratio is less than one and greater than the preset compliance parameter, the proportion of abnormal clips is small at this time, and a notice of unqualified material is issued to inform the user which video clips have problems, so that the user can make targeted modifications. When the compliance ratio is less than or equal to the preset compliance parameter, the proportion of abnormal clips is large, and a notice of reconstructing the storyboard is issued to advise the user to re-examine the video theme and content, use AI to regenerate a more reasonable storyboard script, and re-shoot and upload the material, improving the quality and efficiency of video production while achieving efficient and accurate video editing and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 FIG. is a flowchart of the steps of an automated video editing method based on AI storyboard generation according to an embodiment of the present invention;

[0055] Figure 2 FIG. is a logical decision diagram for determining whether the editing of video clips in a preliminary video is qualified based on a feature profile evaluation parameter according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] In order to make the objectives and advantages of the present invention clearer, the present invention will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only for explaining the present invention and are not used to limit the present invention.

[0057] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and do not limit the protection scope of the present invention.

[0058] It should be noted that in the description of the present invention, the terms indicating directions or positional relationships such as "upper", "lower", "left", "right", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings. This is only for convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0059] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0060] Please refer toFigure 1 and Figure 2 As shown respectively in Figure 1 and Figure 2 , they are the step flow chart of the automated video editing method based on AI storyboarding generation and the logical decision chart for determining whether the video clips in the preliminary video are qualified based on the feature contour evaluation parameter in the embodiments of the present invention. An automated video editing method based on AI storyboarding generation in the embodiments of the present invention includes:

[0061] S1, automatically generate a storyboard script, and sequentially construct corresponding storyboard information for each storyboard, including the preset feature contour information in each frame of the video and the storyboard duration;

[0062] S2, obtain the materials shot by the user according to the storyboard script;

[0063] S3, material matching, perform matching processing on each video clip included in the materials and the corresponding storyboard information;

[0064] S4, cut each video clip to remove the redundant pictures at the start and end of the video clip;

[0065] S5, splice each video clip according to the order of each storyboard in the storyboard script to complete the generation of the preliminary video;

[0066] S6, based on the feature contour evaluation parameter and the storyboard duration, determine one by one whether the editing of each video clip in the preliminary video is qualified, including,

[0067] determine that the editing of a single video clip in the preliminary video is qualified, and label the video clip as a qualified clip;

[0068] or, the editing of a single video clip in the preliminary video is abnormal, and adjust the processing parameters for the single video clip based on the proportion difference amount, where the processing parameters include correcting the coordinates of the video clip, correcting the time axis of the single video clip, or labeling the single video clip as an abnormal clip;

[0069] S7, when the determination of whether the editing of each video clip in the preliminary video is qualified is completed, determine whether the generation of the preliminary video is qualified based on the statistical quantity of the labeled video clips, including,

[0070] send a notice message for unqualified materials, send a notice message for regenerating the storyboard script, or determine that the generation of the preliminary video is qualified and output the generated video.

[0071] Specifically, each video segment in the preliminary video clipped based on the template is analyzed to determine whether the clipping of each video segment is qualified, and the accuracy of material matching of the video is detected. When it is determined that the clipping of a single video segment is abnormal, it is determined that the corresponding material matching is abnormal, and the processing parameters for the single video segment are adjusted to rematch the material, including correcting the coordinates of the video segment, correcting the timeline of the single video segment, or marking the single video segment as an abnormal segment. Subsequently, when it is determined whether the clipping of each video segment in the preliminary video is qualified, it is determined whether the generation of the preliminary video is qualified based on the statistical quantity of the calibrated video segments, and corresponding notification information is further sent according to the abnormal segments, improving the clipping accuracy and thus improving the clipping efficiency of the video.

[0072] Specifically, there is no limitation on the specific method of automatically generating a storyboard script and constructing corresponding storyboard information for each storyboard in turn. The storyboard script can be automatically generated through AI technology according to the video theme, style, and target audience, or several initial templates can be preset; the preset contour information of the features in each frame of the video can be the feature contours of the main body and key elements in each frame of the video, and can include the external contour of a person and the layout contour of a specific scene, and is accurately described in a digital manner to determine the set of coordinate points or the contour shape parameters, which will not be elaborated here.

[0073] Specifically, the storyboard duration is the duration set for each storyboard, which is accurately recorded in seconds, and will not be elaborated here.

[0074] Specifically, the material is a video segment shot according to the requirements of each storyboard information; the shooting personnel should shoot the corresponding video segment according to the requirements of each shot. During the shooting process, the instructions in the storyboard information in the storyboard script should be strictly followed. After shooting, these video segments are uploaded, which will not be elaborated here.

[0075] Specifically, there is no limitation on the specific method of material matching. The specific storyboard corresponding to each video segment can be determined by comparing the key frame features and timestamps of the video segment with the feature contours and time ranges in the storyboard information through an intelligent matching algorithm, which will not be elaborated here.

[0076] Specifically, there is no limitation on the specific method of cutting each video segment. The cutting operation on the matched video segment can be performed by analyzing the content of the video segment and combining the start and end time points in the storyboard information to automatically remove the redundant pictures at the beginning and end of the video segment, ensuring that the duration of each video segment exactly matches the storyboard requirements, which will not be elaborated here.

[0077] Specifically, the specific method for splicing to generate the preliminary video is not limited. The video segments after the cropping process are spliced in the order of the storyboards. During the splicing process, the transition effects between the segments can be automatically processed, including fades and transitions, to ensure the smoothness and visual effects of the video, thereby completing the preliminary generation of the video, which will not be elaborated here.

[0078] Specifically, in S6, it is determined whether the editing of each video segment in the preliminary video is qualified based on the feature contour evaluation parameter, including:

[0079] Obtain several time nodes of a single video segment;

[0080] Compare the contour features in the picture at each time node with the feature preset contour information in the corresponding storyboard;

[0081] Calculate the area ratio of the overlapping area to the area of the feature preset contour information in the corresponding storyboard for a single time node;

[0082] Solve the average value of each area ratio for a single video segment to obtain the feature contour evaluation parameter for a single video segment;

[0083] If the feature contour evaluation parameter is less than or equal to the first preset evaluation parameter, it is determined that the editing of a single video segment in the preliminary video is abnormal, and the processing parameter for a single video segment is adjusted based on the proportion difference;

[0084] If the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, it is determined whether the editing of a single video segment in the preliminary video is qualified based on the storyboard duration corresponding to the single video segment;

[0085] If the feature contour evaluation parameter is greater than the second preset evaluation parameter, it is determined that the editing of a single video segment in the preliminary video is qualified, and the video segment is marked as a qualified segment.

[0086] Specifically, the first preset evaluation parameter is selected within the range of [0.56, 0.69], and the second preset evaluation parameter is selected within the range of [0.8, 0.87].

[0087] Specifically, the method of selecting several time nodes of a single video segment can be to periodically select several time nodes at a predetermined selection time interval, or to determine the number of selected time nodes based on the duration of a single video segment, where the duration is proportional to the number of selected time nodes. On the premise of determining the number of selected time nodes, several time nodes are randomly selected in a single video segment, which will not be elaborated here.

[0088] Specifically, the method for obtaining the area ratio of the overlapping area calculated for a single time node to the area of the feature preset contour information in the corresponding storyboard can be as follows: For contour extraction, at a single time node, the video clip's frame is processed. First, the frame is converted into a grayscale image to simplify subsequent calculations. Then, an edge detection algorithm, such as the Canny edge detection algorithm, is used to extract the edge information of the main body or key elements in the frame. Next, through a contour discovery algorithm, such as the findContours function in OpenCV, the edge information is converted into contour data to obtain the contour features of the video clip at this time node; the feature preset contour information in the storyboard already exists in digital form during the early construction of the storyboard, represented by a set of coordinate points or a specific contour description model, and it is directly converted into the same representation form as the video clip's contour for subsequent comparison; to facilitate the calculation of the overlapping area, polygon approximation processing is performed on the extracted contour features of the video clip and the preset contour. The Douglas-Peucker algorithm or the like is used to simplify the contour and convert it into an approximate polygon. Then, through a matching algorithm, the corresponding relationship between the two polygons is found, and the Sutherland-Hodgman polygon clipping algorithm is used to clip one polygon with the other polygon. The resulting clipped result is the polygon of the overlapping part. Then, the area of this overlapping polygon is calculated to obtain the overlapping area; the area of the feature preset contour information in the corresponding storyboard is obtained, and this area has been determined and recorded during the storyboard construction stage; the calculated overlapping area is divided by the area of the preset contour information to obtain the area ratio for a single time node, which will not be elaborated here.

[0089] Specifically, to determine whether the editing of a single video clip in the preliminary video is qualified based on the storyboard duration corresponding to the single video clip, it includes:

[0090] If the storyboard duration is less than or equal to the preset storyboard duration, the first preset evaluation parameter and the second preset evaluation parameter are adjusted to the corresponding values based on the storyboard duration;

[0091] If the storyboard duration is greater than the preset storyboard duration, it is determined that the editing of the single video clip in the preliminary video is abnormal, and the processing parameters for the single video clip are adjusted based on the proportion difference.

[0092] Specifically, the preset storyboard duration F0 is selected within the range of [7s, 15s].

[0093] Specifically, for a single video clip, several time nodes are randomly selected or selected according to a certain rule. At each time node, the overlapping area between the contour features in the picture and the preset contour information of the features in the corresponding storyboard is calculated, and the feature contour evaluation parameter is determined accordingly. The feature contour evaluation parameter characterizes the material matching situation of the contour features. When the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, the editing of the single video clip is determined to be qualified by comprehensively considering the storyboard duration of the corresponding storyboard of the single video clip. When the storyboard duration is less than or equal to the preset storyboard duration, the storyboard duration is short, and the determination is deviated due to too few selected nodes. At this time, the determination benchmark is adjusted according to the difference in the storyboard duration, and the determination benchmark is comprehensively determined according to the actual situation of the storyboard in specific cases, which improves the data processing efficiency and further improves the detection accuracy of a single video clip.

[0094] Specifically, based on the storyboard duration, the first preset evaluation parameter and the second preset evaluation parameter are adjusted to corresponding values, where

[0095] The reduction amplitudes of the first preset evaluation parameter and the second preset evaluation parameter are inversely proportional to the storyboard duration.

[0096] In this embodiment, optionally,

[0097] The storyboard duration is compared with the first preset duration comparison threshold and the second preset duration comparison threshold;

[0098] If the storyboard duration is less than or equal to the first preset duration comparison threshold, the first preset duration comparison threshold is adjusted to 0.72 times the initial first preset duration comparison threshold, and the second preset duration comparison threshold is adjusted to 0.72 times the initial second preset duration comparison threshold;

[0099] If the storyboard duration is less than or equal to the second preset duration comparison threshold and greater than the first preset duration comparison threshold, the first preset duration comparison threshold is adjusted to 0.81 times the initial first preset duration comparison threshold, and the second preset duration comparison threshold is adjusted to 0.82 times the initial second preset duration comparison threshold;

[0100] If the storyboard duration is greater than the second preset duration comparison threshold, the first preset duration comparison threshold is adjusted to 0.93 times the initial first preset duration comparison threshold, and the second preset duration comparison threshold is adjusted to 0.91 times the initial second preset duration comparison threshold;

[0101] The first preset duration comparison threshold is 0.3F0. The second preset duration comparison threshold is 0.8F0.

[0102] Specifically, based on the re-determined first preset evaluation parameter and the second preset evaluation parameter, it is determined whether the editing of each video segment in the preliminary video is qualified. When the feature contour evaluation parameter is still less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, it is determined that the editing of a single video segment in the preliminary video is abnormal, and the processing parameter for the single video segment is adjusted based on the proportion difference amount.

[0103] Specifically, adjusting the processing parameter for a single video segment based on the proportion difference amount includes:

[0104] For a single video segment, the variance of the area ratios at each calculated time node is recorded as the proportion difference amount;

[0105] If the proportion difference amount is less than or equal to the first preset difference amount, the coordinates of the video segment are corrected;

[0106] If the proportion difference amount is less than or equal to the second preset difference amount and greater than the first preset difference amount, the time axis of the single video segment is corrected based on the area ratios of adjacent time nodes;

[0107] If the proportion difference amount is greater than the second preset difference amount, the single video segment is marked as an abnormal segment.

[0108] Specifically, the first preset difference amount is selected within the range of [0.05, 0.09], and the second preset difference amount is selected within the range of [0.2, 0.3].

[0109] Specifically, the method for correcting the coordinates of the video segment can be to correct the coordinates of the video segment through an image correction algorithm, and use perspective transformation, rotation, and translation operations to adjust the picture to a position matching the feature preset contour information.

[0110] Specifically, based on the proportion difference amount, the processing parameters for a single video segment are adjusted to analyze and process the situation where a single video segment is unqualified. The proportion difference amount characterizes the stability of material matching in a single video segment. When the proportion difference amount is less than or equal to the first preset difference amount, the contour features in a single video segment generally have a stable deviation from the preset contour information of the features. In this case, there is a deviation between the picture of the video segment and the expected picture in the corresponding storyboard. The picture is skewed, and the scaling ratio is inconsistent, resulting in a deviation between the position of the actual key elements and the expected position. At this time, the coordinates of the video segment are corrected through an image correction algorithm, and through perspective transformation, rotation, and translation operations, the picture is adjusted to a position that better matches the preset contour information of the features; when the proportion difference amount is less than or equal to the second preset difference amount and greater than the first preset difference amount, at this time, the contour features in a single video segment generally have a relatively discrete deviation from the position of the preset contour information of the features. At this time, due to problems in the time axis, the deviation of the contour features occurs irregularly. In this case, the overall time axis of a single video segment is moved forward and backward in the time dimension to perform matching in the time dimension, improve the adaptability of video editing, and perform reasonable editing and matching for materials with different shooting qualities. While improving the accuracy of video editing, the editing efficiency of the video is further improved.

[0111] Specifically, the process of correcting the time axis of a single video segment based on the area ratio of adjacent time nodes includes:

[0112] In a single storyboard, select the preset contour information of a single video frame, calculate the overlapping area between the preset contour information of the features and the contour features of the video segment at the corresponding time node, and record it as the current overlapping area; calculate the overlapping area between the preset contour features and the contour features of the video segment in the previous video frame, and record it as the historical overlapping area. Record the ratio of the calculated current overlapping area to the historical overlapping area as the time difference ratio;

[0113] If the time difference ratio is greater than one, the time axis of the video segment is processed for delay, and the correction duration for delay is determined based on the time difference ratio;

[0114] If the time difference ratio is equal to one, re-select a single video frame in a single storyboard, and correct the time axis of a single video segment based on the newly determined time difference ratio;

[0115] If the time difference ratio is less than one, the time axis of the video segment is processed for advance, and the correction duration for advance is determined based on the time difference ratio.

[0116] Specifically, for the correction of the timeline, a single video frame is selected in a single shot for analysis, and the video frames corresponding to the video clip corresponding to the video frame in the shot are obtained at the corresponding time nodes to determine the overlapping area; by comparing the overlapping area with the historical overlapping area, the advance and delay of the timeline are determined. When the time difference ratio is greater than one, in this case, the preset contour information of the single video frame in the single shot matches the contour features of the video clip at the corresponding time node better. Considering the premise that the timeline is abnormal, this is the case of the picture being ahead, and the timeline of the video clip is delayed; when the time difference ratio is equal to one, the picture of the video clip is static during this period. At this time, the node is changed, and the advance or delay of the timeline is determined according to the newly determined time difference ratio; when the time difference ratio is less than one, the current picture is slower than expected, and the timeline of the video clip is advanced; accurately identify the abnormal situation of the timeline and make adjustments, which improves the editing efficiency of the video while ensuring the accuracy of video editing.

[0117] Specifically, based on the time difference ratio, the correction duration for advance is determined, where

[0118] the absolute value of the difference between the calculated time difference ratio and one is denoted as the correction reference value;

[0119] the increase amplitude of the correction duration is proportional to the correction reference value.

[0120] In this embodiment, optionally,

[0121] the correction reference value is compared with the first preset reference comparison threshold and the second preset reference comparison threshold;

[0122] If the correction reference value is less than or equal to the first preset reference comparison threshold, the correction duration is adjusted to 1.11 times the initial correction duration;

[0123] If the correction reference value is less than or equal to the second preset reference comparison threshold and greater than the first preset reference comparison threshold, the correction duration is adjusted to 1.18 times the initial correction duration;

[0124] If the correction reference value is greater than the second preset reference comparison threshold, the correction duration is adjusted to 1.27 times the initial correction duration;

[0125] The first preset reference comparison threshold is taken as 0.2, and the second preset reference comparison threshold is taken as 0.3.

[0126] Specifically, in the step S7, the process of determining whether the generation of the preliminary video is qualified based on the statistical quantity of each calibrated video clip includes:

[0127] Calculate the difference between the number of calibrated qualified segments and the number of abnormal segments, and record the ratio of the solved difference to the number of calibrated qualified segments as the compliance ratio;

[0128] If the compliance ratio is equal to one, it is determined that the generation of the preliminary video is qualified, and the generated video is output;

[0129] If the compliance ratio is less than one and greater than the preset compliance parameter, a notice message for unqualified material is sent;

[0130] If the compliance ratio is less than or equal to the preset compliance parameter, a notice message for regenerating the storyboard script is sent.

[0131] The preset compliance parameter is selected within the range of [-0.3, -0.2].

[0132] Specifically, after the determination of each video segment is completed, the overall determination is made according to the compliance ratio. The compliance ratio represents the proportion of abnormal segments. When the compliance ratio is less than one and greater than the preset compliance parameter, the proportion of abnormal segments is small at this time, and a notice of unqualified material is sent to inform the user which video segments have problems for the user to make targeted modifications. When the compliance ratio is less than or equal to the preset compliance parameter, the proportion of abnormal segments is large, and a notice of reconstructing the storyboard is sent to suggest that the user re-examine the video theme and content, use AI to regenerate a more reasonable storyboard script, and re-shoot and upload the material, while achieving efficient and accurate video editing processing, improving the quality and efficiency of video production.

[0133] So far, the technical solution of the present invention has been described in conjunction with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

[0134] The above are only the preferred embodiments of the present invention and are not used to limit the present invention; for those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automated video editing method based on AI scene generation, characterized in that, Including: S1, automatically generate a storyboard script, and sequentially construct corresponding storyboard information for each storyboard, including the preset contour information of features in each frame of the video and the storyboard duration; S2, obtain the materials shot by the user according to the storyboard script; S3, material matching, perform matching processing on each video segment included in the material and the corresponding storyboard information; S4, cut each video segment to remove the redundant pictures at the beginning and end of the video segment; S5, splice each video segment according to the order of each storyboard in the storyboard script to complete the generation of the preliminary video; S6, based on the feature contour evaluation parameter and the storyboard duration, determine one by one whether the editing of each video segment in the preliminary video is qualified, including, Determine that the editing of a single video segment in the preliminary video is qualified, and label the video segment as a qualified segment; Or, the editing of a single video segment in the preliminary video is abnormal, and adjust the processing parameters for the single video segment based on the proportion difference amount, where the processing parameters include correcting the coordinates of the video segment, correcting the time axis of a single video segment, or labeling a single video segment as an abnormal segment; S7, when determining whether the editing of each video segment in the preliminary video is qualified, determine whether the generation of the preliminary video is qualified based on the statistical quantity of the labeled video segments, including, Send a notice message for unqualified materials, send a notice message for regenerating the storyboard script, or determine that the generation of the preliminary video is qualified and output the generated video.

2. The automated video editing method based on AI storyboard generation according to claim 1, wherein In the said S6, based on the feature contour evaluation parameter, determine whether the editing of each video segment in the preliminary video is qualified, including: Obtain several time nodes of a single video segment; Perform overlapping comparison between the contour features in the picture at each time node and the preset contour information of features in the corresponding storyboard; Calculate the area ratio of the overlapping area to the area of the preset contour information of features in the corresponding storyboard for a single time node; Solve the average value of each area ratio for a single video segment to obtain the feature contour evaluation parameter for the single video segment; If the feature contour evaluation parameter is less than or equal to the first preset evaluation parameter, determine that the editing of a single video segment in the preliminary video is abnormal, and adjust the processing parameters for the single video segment based on the proportion difference amount; If the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, determine whether the editing of a single video segment in the preliminary video is qualified based on the storyboard duration of the single video segment corresponding to the storyboard; If the feature contour evaluation parameter is greater than the second preset evaluation parameter, determine that the editing of a single video segment in the preliminary video is qualified, and label the video segment as a qualified segment.

3. The automated video editing method based on AI shot segmentation generation according to claim 2, characterized in that, Based on the storyboard duration of a single video segment corresponding to the storyboard, determine whether the editing of a single video segment in the preliminary video is qualified, including: If the storyboard duration is less than or equal to the preset storyboard duration, adjust the first preset evaluation parameter and the second preset evaluation parameter to the corresponding values based on the storyboard duration; If the storyboard duration is greater than the preset storyboard duration, determine that the editing of a single video segment in the preliminary video is abnormal, and adjust the processing parameters for the single video segment based on the proportion difference amount.

4. The automated video editing method based on AI storyboard generation according to claim 3, wherein Adjust the first preset evaluation parameter and the second preset evaluation parameter to corresponding values based on the duration of the storyboard shot, where The reduction amplitudes of the first preset evaluation parameter and the second preset evaluation parameter are inversely proportional to the duration of the storyboard shot.

5. The automated video editing method based on AI storyboard generation according to claim 4, wherein Adjust the processing parameters for a single video segment based on the proportion difference amount, including: For a single video segment, record the variance of the area ratios at each time node calculated as the proportion difference amount; If the proportion difference amount is less than or equal to the first preset difference amount, correct the coordinates of the video segment; If the proportion difference amount is less than or equal to the second preset difference amount and greater than the first preset difference amount, correct the time axis of a single video segment based on the area ratios of adjacent time nodes; If the proportion difference amount is greater than the second preset difference amount, label the single video segment as an abnormal segment.

6. The automated video editing method based on AI storyboard generation according to claim 5, wherein The process of correcting the time axis of a single video segment based on the area ratios of adjacent time nodes includes: In a single storyboard shot, select the feature preset contour information of a single video frame, calculate the overlapping area between the feature preset contour information and the contour feature of the video segment at the corresponding time node, and record it as the current overlapping area; calculate the overlapping area between the preset contour feature and the contour feature of the video segment in the previous video frame, and record it as the historical overlapping area, and record the ratio of the calculated current overlapping area to the historical overlapping area as the time difference ratio; If the time difference ratio is greater than one, perform a delay process on the time axis of the video segment, and determine the correction duration of the delay based on the time difference ratio; If the time difference ratio is equal to one, reselect a single video frame in the single storyboard shot, and correct the time axis of the single video segment based on the re-determined time difference ratio; If the time difference ratio is less than one, perform an advance process on the time axis of the video segment, and determine the correction duration of the advance based on the time difference ratio.

7. The automated video editing method based on AI scene division generation according to claim 6, wherein, Determine the correction duration of the advance based on the time difference ratio, where Record the absolute value of the difference between the calculated time difference ratio and one as the correction reference value; The increase amplitude of the correction duration is proportional to the correction reference value.

8. The automated video editing method based on AI scene segmentation generation according to claim 7, characterized in that, In S7, the process of determining whether the generation of the preliminary video is qualified based on the statistical quantity of each calibrated video segment includes: Calculate the difference between the number of calibrated qualified segments and the number of abnormal segments, and record the ratio of the solved difference to the number of calibrated qualified segments as the compliance ratio; If the compliance ratio is equal to one, determine that the generation of the preliminary video is qualified, and output the generated video; If the compliance ratio is less than one and greater than the preset compliance parameter, send a notice message for unqualified material; If the compliance ratio is less than or equal to the preset compliance parameter, send a notice message for regenerating the storyboard script.

Images