Automatic video clip generation method based on AI split-screen generation

Abstract

The application relates to the technical field of video clip, in particular to an automatic video clip method based on AI split-screen generation, which comprises the following steps: analyzing each video segment in a preliminary video based on template clip to determine whether the clip of each video segment is qualified, detecting the material matching accuracy of the video, determining that corresponding material matching is abnormal when the clip of a single video segment is abnormal, adjusting the processing parameters for the single video segment to rematch the material, including correcting the coordinates of the video segment, correcting the time axis of the single video segment or marking the single video segment as an abnormal segment, and determining whether the generation of the preliminary video is qualified based on the statistical quantity of the marked video segments when the determination of whether the clip of each video segment in the preliminary video is qualified is completed, and further issuing corresponding notification information according to the abnormal segment, so that the clip precision is improved, and the video clip efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of video editing technology, and in particular to an automated video editing method based on AI storyboard generation. Background Technology

[0002] Early video editing relied primarily on linear editing equipment, physically cutting and splicing film or tape, a process that was complex and difficult to modify. With the development of computer technology, non-linear editing software emerged, such as Adobe Premiere Pro and Final Cut Pro. Editors could edit digitized video footage on computers, greatly improving the flexibility and efficiency of editing. However, these software programs still require manual operation and demand a high level of professional skill from the editors.

[0003] In recent years, artificial intelligence technology has developed rapidly, especially deep learning algorithms, which have achieved remarkable results in fields such as image recognition, speech recognition, and natural language processing. Image recognition technology can accurately identify objects, people, and scenes in images, providing a foundation for video content analysis; machine learning algorithms can learn patterns and rules in video editing through large amounts of data, thereby enabling automatic editing decisions.

[0004] Driven by artificial intelligence technology, some research institutions and companies have begun to explore automated video editing techniques. Early attempts mainly focused on simple video splicing and basic special effects addition, resulting in videos with limited quality and flexibility. As the technology continues to advance, template-based automated video editing systems have gradually emerged.

[0005] Chinese Patent Publication No. CN118660115A discloses a video data storyboard method and system for intelligent video generation, including: acquiring structured video data; extracting scripts and storyboards from the video data, calculating the matching index between the scripts and storyboards, and creating a mapping relationship between the scripts and storyboards based on the matching index to generate a training dataset; performing a first fine-tuning on a large language model based on the training dataset and a prediction result scoring formula to enable it to generate storyboards; performing a second fine-tuning on the large language model to optimize the model's storyboard generation capability; using the fine-tuned large language model to intelligently generate scripts and storyboard sequences based on input keywords, and storing the generated scripts and storyboard sequences in a database. It is evident that the above technical solution has the following problems: it cannot monitor the accuracy of material matching for template-based video editing, affecting the efficiency of video editing. Summary of the Invention

[0006] To address this issue, the present invention provides an automated video editing method based on AI storyboard generation, which overcomes the problem in existing technologies that cannot monitor the accuracy of material matching for template-based video editing, thus affecting the efficiency of video editing.

[0007] To achieve the above objectives, this invention provides an automated video editing method based on AI-generated storyboards, comprising:

[0008] S1 automatically generates storyboard scripts and constructs corresponding storyboard information for each storyboard in sequence, including feature preset contour information in each frame of video and storyboard duration;

[0009] S2, acquire footage shot by the user based on the storyboard;

[0010] S3, material matching, matches each video clip in the material with the corresponding storyboard information;

[0011] S4, cut each video segment to remove redundant images at the beginning and end of the video segment;

[0012] S5: The video clips are spliced ​​together according to the order of the scenes in the storyboard to generate the initial video.

[0013] S6, based on feature contour evaluation parameters and shot duration, determines the quality of editing for each video segment in the preliminary video, including...

[0014] Determine that the editing of a single video segment in the preliminary video is qualified, and mark that video segment as a qualified segment;

[0015] Alternatively, if the editing of a single video segment in the initial video is abnormal, the processing parameters for the single video segment are adjusted based on the percentage difference. These processing parameters include correcting the coordinates of the video segment, correcting the timeline of the single video segment, or marking the single video segment as an abnormal segment.

[0016] S7, when determining whether the editing of each video segment in the preliminary video is qualified, the qualification of the generated preliminary video is determined based on the statistical count of each calibrated video segment, including,

[0017] Issue notifications for substandard materials, issue notifications for regenerating storyboards, or determine that the initial video generation is qualified and output the generated video.

[0018] Further, in step S6, determining whether the editing of each video segment in the preliminary video is qualified based on the feature contour evaluation parameters includes:

[0019] Obtain several time points from a single video segment;

[0020] The contour features in the scene at each time point are compared with the preset contour information of the corresponding storyboard.

[0021] For a single time point, calculate the area ratio between the overlapping area and the area of ​​the pre-defined contour information of the corresponding scene.

[0022] The average value of each area ratio is calculated for a single video segment to obtain the feature contour evaluation parameters for that single video segment.

[0023] If the feature contour evaluation parameter is less than or equal to the first preset evaluation parameter, then the editing of a single video segment in the preliminary video is determined to be abnormal, and the processing parameters for the single video segment are adjusted based on the proportion difference.

[0024] If the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, then the quality of editing of a single video segment in the preliminary video is determined based on the segment duration of the segment corresponding to the single video segment.

[0025] If the feature contour evaluation parameter is greater than the second preset evaluation parameter, then the editing of a single video segment in the preliminary video is deemed qualified, and the video segment is marked as a qualified segment.

[0026] Furthermore, the quality of editing of individual video segments in the preliminary video is determined based on the storyboard duration corresponding to each individual video segment, including:

[0027] If the storyboard duration is less than or equal to the preset storyboard duration, the first preset evaluation parameter and the second preset evaluation parameter will be adjusted to the corresponding values ​​based on the storyboard duration.

[0028] If the storyboard duration exceeds the preset storyboard duration, the editing of a single video segment in the initial video is determined to be abnormal, and the processing parameters for the single video segment are adjusted based on the percentage difference.

[0029] Furthermore, based on the storyboard duration, the first and second preset evaluation parameters are adjusted to corresponding values, wherein,

[0030] The reduction in the first and second preset evaluation parameters is inversely proportional to the storyboard duration.

[0031] Furthermore, the processing parameters for a single video segment are adjusted based on the percentage difference, including:

[0032] For a single video segment, the variance of the area ratio at each time point is recorded as the percentage difference.

[0033] If the percentage difference is less than or equal to the first preset difference, the coordinates of the video segment are corrected.

[0034] If the percentage difference is less than or equal to the second preset difference and greater than the first preset difference, then the timeline of a single video segment is corrected based on the area ratio of adjacent time nodes.

[0035] If the percentage difference is greater than the second preset difference, then the individual video segment will be marked as an abnormal segment.

[0036] Furthermore, the process of correcting the timeline of a single video segment based on the area ratio of adjacent time nodes includes:

[0037] In a single scene, the feature preset contour information of a single video frame is selected, and the overlap area between the feature preset contour information and the contour feature of the video segment at the corresponding time node is calculated and recorded as the current overlap area; the overlap area between the preset contour feature and the contour feature of the video segment at the previous video frame is calculated and recorded as the historical overlap area, and the ratio of the calculated current overlap area to the historical overlap area is recorded as the time difference ratio.

[0038] If the time difference ratio is greater than one, the timeline of the video segment is delayed, and the correction duration of the delay is determined based on the time difference ratio.

[0039] If the time difference ratio is equal to one, then a single video frame in a single shot is reselected, and the timeline of the single video segment is corrected based on the re-determined time difference ratio.

[0040] If the time difference ratio is less than one, the timeline of the video segment is preprocessed, and the correction duration is determined based on the time difference ratio.

[0041] Furthermore, the correction duration is determined based on the time difference ratio, where,

[0042] The absolute value of the difference between the calculated time difference ratio and one is recorded as the correction reference value;

[0043] The increase in correction duration is proportional to the correction reference value.

[0044] Furthermore, in S7, the process of determining whether the initial video generation is qualified based on the statistical count of each calibrated video segment includes:

[0045] Calculate the difference between the number of qualified segments and the number of abnormal segments, and record the ratio of this difference to the number of qualified segments as the compliance ratio;

[0046] If the pass rate is equal to one, the initial video generation is deemed qualified, and the generated video is output.

[0047] If the compliance rate is less than one but greater than the preset compliance parameter, a notification message for unqualified materials will be issued.

[0048] If the pass rate is less than or equal to the preset pass rate parameter, a notification message will be sent to regenerate the storyboard.

[0049] Compared with existing technologies, the beneficial effects of this invention are as follows: It analyzes each video segment in the initial video based on template editing to determine whether the editing of each video segment is qualified; it detects the accuracy of the video material matching; when an editing anomaly is determined for a single video segment, it determines that the corresponding material matching is abnormal; it adjusts the processing parameters for the single video segment to re-match the material, including correcting the coordinates of the video segment, correcting the timeline of the single video segment, or marking the single video segment as an abnormal segment; subsequently, when determining whether the editing of each video segment in the initial video is qualified, it determines whether the generation of the initial video is qualified based on the statistical number of marked video segments; and it further issues corresponding notification information based on abnormal segments, thereby improving editing accuracy and thus improving video editing efficiency.

[0050] Furthermore, for a single video segment, several time nodes are randomly or systematically selected. At each time node, the overlap area between the contour features in the frame and the preset contour information of the corresponding scene is calculated. Based on the determined feature contour evaluation parameters, which characterize the material matching of the contour features, when the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter but greater than the first preset evaluation parameter, the editing of the single video segment is determined by considering the scene duration of the corresponding scene. When the scene duration is less than or equal to the preset scene duration, the short scene duration may lead to a deviation in the judgment due to too few selected nodes. In this case, the judgment benchmark is adjusted according to the difference in scene duration. Based on the actual situation of the scene, the judgment benchmark is comprehensively determined under specific circumstances. This improves data processing efficiency and further enhances the detection accuracy of a single video segment.

[0051] Furthermore, based on the proportion difference, the processing parameters for individual video segments are adjusted to analyze and handle cases where individual video segments are unqualified. The proportion difference represents the stability of material matching in a single video segment. When the proportion difference is less than or equal to the first preset difference, the contour features in the individual video segment generally have a stable deviation from the preset contour information. In this case, the image of the video segment deviates from the expected image in the corresponding storyboard. The image is distorted, and inconsistent scaling ratios cause the actual position of key elements to deviate from the expected position. At this time, the coordinates of the video segment are corrected through an image correction algorithm. Perspective transformation, rotation, and translation operations are used to adjust the image. The adjustment is made to a position that better matches the preset contour information of the features. When the proportion difference is less than or equal to the second preset difference and greater than the first preset difference, the contour features in a single video segment generally show a relatively discrete deviation from the preset contour information. At this time, due to the problem of the time axis, the contour features will shift intermittently. In this case, the time axis of the entire single video segment is moved forward and backward in the time dimension to match in the time dimension, improve the adaptability of video editing, and perform reasonable editing and matching for materials of different shooting quality. While improving the accuracy of video editing, it further improves the efficiency of video editing.

[0052] Furthermore, regarding timeline correction, a single video frame within a single shot is selected for analysis. The video frame corresponding to that frame in the shot is then compared to the corresponding time node to determine the overlap area. The timeline is adjusted by comparing the overlap area with historical overlap areas. When the time difference ratio is greater than one, the preset contour information of the single video frame in the single shot better matches the contour features of the video segment at the corresponding time node. Given the timeline anomaly (indicating the image is ahead), the timeline of the video segment is delayed. When the time difference ratio is equal to one, the video segment's image is static within that time period. The node is changed, and the timeline is adjusted based on the newly determined time difference ratio. When the time difference ratio is less than one, the current image is slower than expected, and the timeline of the video segment is advanced. This precise identification and adjustment of timeline anomalies ensures the accuracy of video editing while further improving editing efficiency.

[0053] Furthermore, after evaluating each video segment, an overall assessment is made based on the pass rate. The pass rate represents the proportion of abnormal segments. When the pass rate is less than one but greater than the preset pass parameter, the proportion of abnormal segments is low, and a material failure notification is issued, informing the user which video segments have problems so that the user can make targeted modifications. When the pass rate is less than or equal to the preset pass parameter, the proportion of abnormal segments is high, and a storyboard reconstruction notification is issued, suggesting that the user re-examine the video theme and content, use AI to regenerate a more reasonable storyboard, and reshoot and upload the material. This achieves efficient and accurate video editing while improving the quality and efficiency of video production. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the steps of an automated video editing method based on AI-generated storyboards, as described in an embodiment of the present invention.

[0055] Figure 2 This is a logic diagram for determining whether the editing of video segments in the preliminary video is qualified based on feature contour evaluation parameters in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0057] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0058] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0059] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0060] Please see Figure 1 as well as Figure 2 The diagrams shown are, respectively, a flowchart of the automated video editing method based on AI storyboard generation according to an embodiment of the present invention, and a logic judgment diagram for determining whether the editing of video segments in the preliminary video is qualified based on feature contour evaluation parameters; an automated video editing method based on AI storyboard generation according to an embodiment of the present invention includes:

[0061] S1 automatically generates storyboard scripts and constructs corresponding storyboard information for each storyboard in sequence, including feature preset contour information in each frame of video and storyboard duration;

[0062] S2, acquire footage shot by the user based on the storyboard;

[0063] S3, material matching, matches each video clip in the material with the corresponding storyboard information;

[0064] S4, cut each video segment to remove redundant images at the beginning and end of the video segment;

[0065] S5: The video clips are spliced ​​together according to the order of the scenes in the storyboard to generate the initial video.

[0066] S6, based on feature contour evaluation parameters and shot duration, determines the quality of editing for each video segment in the preliminary video, including...

[0067] Determine that the editing of a single video segment in the preliminary video is qualified, and mark that video segment as a qualified segment;

[0068] Alternatively, if the editing of a single video segment in the initial video is abnormal, the processing parameters for the single video segment are adjusted based on the percentage difference. These processing parameters include correcting the coordinates of the video segment, correcting the timeline of the single video segment, or marking the single video segment as an abnormal segment.

[0069] S7, when determining whether the editing of each video segment in the preliminary video is qualified, the qualification of the generated preliminary video is determined based on the statistical count of each calibrated video segment, including,

[0070] Issue notifications for substandard materials, issue notifications for regenerating storyboards, or determine that the initial video generation is qualified and output the generated video.

[0071] Specifically, the process analyzes each video segment in the initial video edited using templates to determine if the editing of each segment is up to standard. It also checks the accuracy of the video footage matching. When an editing anomaly is identified in a single video segment, the corresponding footage matching is also found to be abnormal. The processing parameters for that single video segment are adjusted to re-match the footage, including correcting the coordinates of the video segment, correcting the timeline of the single video segment, or marking the single video segment as an abnormal segment. Subsequently, when determining the quality of the editing of each video segment in the initial video, the statistical count of the marked video segments is used to determine whether the initial video generation is up to standard. Furthermore, corresponding notification information is issued based on abnormal segments, improving editing accuracy and thus increasing video editing efficiency.

[0072] Specifically, there are no restrictions on the specific methods for automatically generating storyboards and constructing corresponding storyboard information for each scene. AI technology can be used to automatically generate storyboards based on the video theme, style, and target audience, or several initial templates can be preset. The feature preset contour information in each frame of the video can be the feature contours of the main body and key elements in each frame of the video, which can include the outline of the character and the layout contour of a specific scene. These can be accurately described in a digital way, and the set of coordinate points or contour shape parameters can be determined. This will not be elaborated further.

[0073] Specifically, the storyboard duration is the set duration for each storyboard, recorded precisely in seconds, which will not be elaborated further.

[0074] Specifically, the footage consists of video clips shot according to the requirements of each storyboard; the filming crew should shoot the corresponding video clips according to the requirements of each shot. During the shooting process, the instructions in the storyboard should be strictly followed. After shooting, these video clips should be uploaded, which will not be elaborated further.

[0075] Specifically, there are no restrictions on the specific method of material matching. Intelligent matching algorithms can be used to compare the key frame features and timestamps of video clips with the feature contours and time ranges in the storyboard information to determine the specific storyboard corresponding to each video clip. This will not be elaborated further.

[0076] Specifically, there are no restrictions on the specific methods for cutting each video segment. The cutting operation of the matched video segments can be performed by analyzing the content of the video segments and combining the start and end time points in the storyboard information to automatically remove the redundant images at the beginning and end of the video segments, ensuring that the duration of each video segment is precisely matched with the storyboard requirements. This will not be elaborated further.

[0077] Specifically, there are no restrictions on the specific method for splicing to generate the initial video. The video segments that have been cropped are spliced ​​together in the order of the storyboard. During the splicing process, the transition effects between segments can be automatically processed, including fade-in and fade-out and transitions, to ensure the smoothness and visual effect of the video, thereby completing the initial generation of the video. This will not be elaborated further.

[0078] Specifically, in S6, determining whether the editing of each video segment in the preliminary video is qualified based on the feature contour evaluation parameters includes:

[0079] Obtain several time points from a single video segment;

[0080] The contour features in the scene at each time point are compared with the preset contour information of the corresponding storyboard.

[0081] For a single time point, calculate the area ratio between the overlapping area and the area of ​​the pre-defined contour information of the corresponding scene.

[0082] The average value of each area ratio is calculated for a single video segment to obtain the feature contour evaluation parameters for that single video segment.

[0083] If the feature contour evaluation parameter is less than or equal to the first preset evaluation parameter, then the editing of a single video segment in the preliminary video is determined to be abnormal, and the processing parameters for the single video segment are adjusted based on the proportion difference.

[0084] If the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, then the quality of editing of a single video segment in the preliminary video is determined based on the segment duration of the segment corresponding to the single video segment.

[0085] If the feature contour evaluation parameter is greater than the second preset evaluation parameter, then the editing of a single video segment in the preliminary video is deemed qualified, and the video segment is marked as a qualified segment.

[0086] Specifically, the first preset evaluation parameter is selected within the range [0.56, 0.69], and the second preset evaluation parameter is selected within the range [0.8, 0.87].

[0087] Specifically, the methods for selecting several time points from a single video segment can be as follows: periodically selecting several time points with a predetermined selection time period; or determining the number of time points to select based on the duration of a single video segment, with the duration being proportional to the number of selections. Given a determined number of selections, several time points can be randomly selected from a single video segment. This will not be elaborated further.

[0088] Specifically, the method for obtaining the ratio of the overlapping area calculated for a single time node to the area of ​​the pre-defined contour information in the corresponding storyboard can be as follows: Contour extraction at a single time node involves processing the video clip's image. First, the image is converted to grayscale to simplify subsequent calculations. Then, edge detection algorithms, such as the Canny edge detection algorithm, are used to extract the edge information of the main subject or key elements in the image. Next, contour discovery algorithms, such as the findContours function in OpenCV, are used to convert the edge information into contour data, obtaining the contour features of the video clip at that time node. The pre-defined contour information in the storyboard already exists in digital form during the initial storyboard construction, represented by a set of coordinate points or a specific contour description model. It is directly converted into the same representation as the video clip contour for subsequent comparison. To facilitate the calculation of the overlapping area, polygon approximation processing is performed on the extracted video clip contour features and the pre-defined contour. The contours are simplified using algorithms such as the Douglas-Peucker algorithm, transforming them into approximate polygons. Then, a matching algorithm is used to find the correspondence between two polygons. The Sutherland-Hodgman polygon clipping algorithm is used to clip one polygon to another, and the clipped result is the overlapping polygon. The overlapping area is then obtained by calculating the area of ​​the overlapping polygon. The area of ​​the feature preset contour information in the corresponding storyboard is obtained. This area has been determined and recorded during the storyboard construction stage. The calculated overlapping area is divided by the area of ​​the preset contour information to obtain the area ratio for a single time point, which will not be elaborated further.

[0089] Specifically, the quality of editing of individual video segments in the initial video is determined based on the segment length corresponding to the storyboard, including:

[0090] If the storyboard duration is less than or equal to the preset storyboard duration, the first preset evaluation parameter and the second preset evaluation parameter will be adjusted to the corresponding values ​​based on the storyboard duration.

[0091] If the storyboard duration exceeds the preset storyboard duration, the editing of a single video segment in the initial video is determined to be abnormal, and the processing parameters for the single video segment are adjusted based on the percentage difference.

[0092] Specifically, the preset storyboard duration F0 is selected within the range [7s, 15s].

[0093] Specifically, for a single video segment, several time points are randomly or systematically selected. At each time point, the overlap area between the contour features in the frame and the preset contour information of the corresponding scene is calculated. Based on the determined feature contour evaluation parameters, which characterize the material matching of the contour features, when the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter but greater than the first preset evaluation parameter, the editing of the single video segment is determined by considering the scene duration of the corresponding scene. When the scene duration is less than or equal to the preset scene duration, the short scene duration may lead to a deviation in the judgment due to too few selected time points. In this case, the judgment benchmark is adjusted according to the difference in scene duration. Based on the actual situation of the scene, the judgment benchmark is comprehensively determined under specific circumstances. This improves data processing efficiency and further enhances the detection accuracy of single video segments.

[0094] Specifically, based on the storyboard duration, the first and second preset evaluation parameters are adjusted to corresponding values, wherein,

[0095] The reduction in the first and second preset evaluation parameters is inversely proportional to the storyboard duration.

[0096] In this embodiment, optionally,

[0097] The storyboard duration is compared with the first preset duration comparison threshold and the second preset duration comparison threshold;

[0098] If the storyboard duration is less than or equal to the first preset duration comparison threshold, then the first preset duration comparison threshold is adjusted to 0.72 times the initial first preset duration comparison threshold, and the second preset duration comparison threshold is adjusted to 0.72 times the initial second preset duration comparison threshold.

[0099] If the storyboard duration is less than or equal to the second preset duration comparison threshold and greater than the first preset duration comparison threshold, then the first preset duration comparison threshold is adjusted to 0.81 times the initial first preset duration comparison threshold, and the second preset duration comparison threshold is adjusted to 0.82 times the initial second preset duration comparison threshold.

[0100] If the storyboard duration exceeds the second preset duration comparison threshold, the first preset duration comparison threshold is adjusted to 0.93 times the initial first preset duration comparison threshold, and the second preset duration comparison threshold is adjusted to 0.91 times the initial second preset duration comparison threshold.

[0101] The first preset duration comparison threshold is set to 0.3F0. The second preset duration comparison threshold is set to 0.8F0.

[0102] Specifically, based on the newly determined first and second preset evaluation parameters, it is determined whether the editing of each video segment in the preliminary video is qualified. When the feature contour evaluation parameter is still less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, it is determined that the editing of a single video segment in the preliminary video is abnormal, and the processing parameters of the single video segment are adjusted based on the percentage difference.

[0103] Specifically, the processing parameters for a single video segment are adjusted based on the percentage difference, including:

[0104] For a single video segment, the variance of the area ratio at each time point is recorded as the percentage difference.

[0105] If the percentage difference is less than or equal to the first preset difference, the coordinates of the video segment are corrected.

[0106] If the percentage difference is less than or equal to the second preset difference and greater than the first preset difference, then the timeline of a single video segment is corrected based on the area ratio of adjacent time nodes.

[0107] If the percentage difference is greater than the second preset difference, then the individual video segment will be marked as an abnormal segment.

[0108] Specifically, the first preset difference is selected within the range [0.05, 0.09], and the second preset difference is selected within the range [0.2, 0.3].

[0109] Specifically, the method for correcting the coordinates of a video segment can be to correct the coordinates of the video segment using an image correction algorithm, and to adjust the image to a position that matches the preset contour information of the features using perspective transformation, rotation and translation operations.

[0110] Specifically, based on the proportion difference, the processing parameters for individual video segments are adjusted to analyze and handle cases where individual video segments are unqualified. The proportion difference represents the stability of material matching in a single video segment. When the proportion difference is less than or equal to the first preset difference, the contour features in the single video segment generally have a stable deviation from the preset contour information. In this case, the image of the video segment deviates from the expected image in the corresponding storyboard. The image is distorted, and inconsistent scaling ratios cause the actual position of key elements to deviate from the expected position. At this time, the coordinates of the video segment are corrected through an image correction algorithm. Perspective transformation, rotation, and translation operations are used to adjust the image. The adjustment is made to a position that better matches the preset contour information of the features. When the proportion difference is less than or equal to the second preset difference and greater than the first preset difference, the contour features in a single video segment generally show a relatively discrete deviation from the preset contour information. At this time, due to the problem of the time axis, the contour features will shift intermittently. In this case, the time axis of the entire single video segment is moved forward and backward in the time dimension to match in the time dimension, improve the adaptability of video editing, and perform reasonable editing and matching for materials of different shooting quality. While improving the accuracy of video editing, it further improves the efficiency of video editing.

[0111] Specifically, the process of correcting the timeline of a single video segment based on the area ratio of adjacent time points includes:

[0112] In a single scene, the feature preset contour information of a single video frame is selected, and the overlap area between the feature preset contour information and the contour feature of the video segment at the corresponding time node is calculated and recorded as the current overlap area; the overlap area between the preset contour feature and the contour feature of the video segment at the previous video frame is calculated and recorded as the historical overlap area, and the ratio of the calculated current overlap area to the historical overlap area is recorded as the time difference ratio.

[0113] If the time difference ratio is greater than one, the timeline of the video segment is delayed, and the correction duration of the delay is determined based on the time difference ratio.

[0114] If the time difference ratio is equal to one, then a single video frame in a single shot is reselected, and the timeline of the single video segment is corrected based on the re-determined time difference ratio.

[0115] If the time difference ratio is less than one, the timeline of the video segment is preprocessed, and the correction duration is determined based on the time difference ratio.

[0116] Specifically, for timeline correction, a single video frame is selected within a single shot for analysis. The video frame corresponding to that frame in the shot is then compared to the corresponding time node to determine the overlap area. The timeline is adjusted by comparing the overlap area with historical overlap areas. When the time difference ratio is greater than one, the preset contour information of the single video frame in the single shot better matches the contour features of the video segment at the corresponding time node. Given the timeline anomaly (indicating the image is ahead), the timeline of the video segment is delayed. When the time difference ratio is equal to one, the video segment's image is static within that time period. The node is changed, and the timeline is adjusted based on the newly determined time difference ratio. When the time difference ratio is less than one, the current image is slower than expected, and the timeline of the video segment is advanced. This precise identification and adjustment of timeline anomalies ensures the accuracy of video editing while further improving editing efficiency.

[0117] Specifically, the correction duration is determined based on the time difference ratio, where,

[0118] The absolute value of the difference between the calculated time difference ratio and one is recorded as the correction reference value;

[0119] The increase in correction duration is proportional to the correction reference value.

[0120] In this embodiment, optionally,

[0121] The corrected reference value is compared with the first preset reference comparison threshold and the second preset reference comparison threshold;

[0122] If the corrected reference value is less than or equal to the first preset reference comparison threshold, the correction duration will be adjusted to 1.11 times the initial correction duration.

[0123] If the corrected reference value is less than or equal to the second preset reference comparison threshold and greater than the first preset reference comparison threshold, the correction duration will be adjusted to 1.18 times the initial correction duration.

[0124] If the corrected reference value is greater than the second preset reference comparison threshold, the correction duration will be adjusted to 1.27 times the initial correction duration.

[0125] The first preset reference comparison threshold is set to 0.2, and the second preset reference comparison threshold is set to 0.3.

[0126] Specifically, in S7, the process of determining whether the initial video generation is qualified based on the statistical count of each calibrated video segment includes:

[0127] Calculate the difference between the number of qualified segments and the number of abnormal segments, and record the ratio of this difference to the number of qualified segments as the compliance ratio;

[0128] If the pass rate is equal to one, the initial video generation is deemed qualified, and the generated video is output.

[0129] If the compliance rate is less than one but greater than the preset compliance parameter, a notification message for unqualified materials will be issued.

[0130] If the pass rate is less than or equal to the preset pass rate parameter, a notification message will be sent to regenerate the storyboard.

[0131] The preset target parameters are selected within the range [-0.3, -0.2].

[0132] Specifically, after evaluating each video segment, an overall assessment is made based on the pass rate. The pass rate represents the proportion of abnormal segments. When the pass rate is less than one but greater than the preset pass parameter, the proportion of abnormal segments is low, and a material failure notification is issued, informing the user which video segments have problems so that the user can make targeted modifications. When the pass rate is less than or equal to the preset pass parameter, the proportion of abnormal segments is high, and a storyboard reconstruction notification is issued, suggesting that the user re-examine the video theme and content, use AI to regenerate a more reasonable storyboard, and reshoot and upload the material. This achieves efficient and accurate video editing while improving the quality and efficiency of video production.

[0133] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An automated video editing method based on AI-generated storyboard generation, characterized in that, include: S1 automatically generates storyboard scripts and constructs corresponding storyboard information for each storyboard in sequence, including feature preset contour information in each frame of video and storyboard duration; S2, acquire footage shot by the user based on the storyboard; S3, material matching, matches each video clip in the material with the corresponding storyboard information; S4, cut each video segment to remove redundant images at the beginning and end of the video segment; S5: The video clips are spliced ​​together according to the order of the scenes in the storyboard to generate the initial video. S6, based on the feature contour evaluation parameters and the storyboard duration, determines whether the editing of each video segment in the preliminary video is qualified, including, Obtain several time points from a single video segment; The contour features in the scene at each time point are compared with the preset contour information of the corresponding storyboard. For a single time point, calculate the area ratio between the overlapping area and the area of ​​the pre-defined contour information of the corresponding scene. The average value of each area ratio is calculated for a single video segment to obtain the feature contour evaluation parameters for that single video segment. If the feature contour evaluation parameter is less than or equal to the first preset evaluation parameter, then the editing of a single video segment in the preliminary video is determined to be abnormal, and the processing parameters for the single video segment are adjusted based on the proportion difference. The processing parameters include correcting the coordinates of the video segment, correcting the time axis of the single video segment, or marking the single video segment as an abnormal segment. If the feature contour evaluation parameter is less than or equal to the second preset evaluation parameter and greater than the first preset evaluation parameter, then the quality of editing of a single video segment in the preliminary video is determined based on the segment duration of the segment corresponding to the single video segment. If the feature contour evaluation parameter is greater than the second preset evaluation parameter, then the editing of a single video segment in the preliminary video is deemed qualified, and the video segment is marked as a qualified segment. The first preset evaluation parameter is selected within the interval [0.56, 0.69], and the second preset evaluation parameter is selected within the interval [0.8, 0.87]. The methods for selecting several time points in a single video segment are as follows: periodically selecting several time points with a predetermined selection time period, or determining the number of time points to be selected based on the duration of a single video segment, with the duration being proportional to the number of selections. Under the premise of determining the number of selections, several time points are randomly selected from a single video segment. The quality of editing of individual video segments in the initial video is determined based on the segment length corresponding to the storyboard, including: If the storyboard duration is less than or equal to the preset storyboard duration, the first preset evaluation parameter and the second preset evaluation parameter will be adjusted to the corresponding values ​​based on the storyboard duration. If the storyboard duration exceeds the preset storyboard duration, the editing of a single video segment in the preliminary video is determined to be abnormal, and the processing parameters for the single video segment are adjusted based on the percentage difference. The preset storyboard duration F0 is selected within the range [7s, 15s]. Adjusting processing parameters for a single video segment based on the percentage difference includes: For a single video segment, the variance of the area ratio at each time point is recorded as the percentage difference. If the percentage difference is less than or equal to the first preset difference, the coordinates of the video segment are corrected. If the percentage difference is less than or equal to the second preset difference and greater than the first preset difference, then the timeline of a single video segment is corrected based on the area ratio of adjacent time nodes. If the percentage difference is greater than the second preset difference, then the individual video segment will be marked as an abnormal segment. The first preset difference value is selected within the range [0.05, 0.09], and the second preset difference value is selected within the range [0.2, 0.3]. The method for correcting the coordinates of video segments is to correct the coordinates of video segments through image correction algorithms, and to adjust the image to a position that matches the preset feature contour information by using perspective transformation, rotation and translation operations. S7, when determining whether the editing of each video segment in the preliminary video is qualified, the qualification of the generated preliminary video is determined based on the statistical count of each calibrated video segment, including, Issue notifications for substandard materials, issue notifications for regenerating storyboards, or determine that the initial video generation is qualified and output the generated video.

2. The automated video editing method based on AI storyboard generation according to claim 1, characterized in that, Based on the storyboard duration, the first and second preset evaluation parameters are adjusted to their corresponding values, wherein, The reduction in the first and second preset evaluation parameters is inversely proportional to the storyboard duration.

3. The automated video editing method based on AI storyboard generation according to claim 2, characterized in that, The process of correcting the timeline of a single video segment based on the area ratio of adjacent time points includes: In a single scene, the feature preset contour information of a single video frame is selected, and the overlap area between the feature preset contour information and the contour feature of the video segment at the corresponding time node is calculated and recorded as the current overlap area; the overlap area between the feature preset contour feature and the contour feature of the video segment at the previous video frame is calculated and recorded as the historical overlap area, and the ratio of the calculated current overlap area to the historical overlap area is recorded as the time difference ratio. If the time difference ratio is greater than one, the timeline of the video segment is delayed, and the correction duration of the delay is determined based on the time difference ratio. If the time difference ratio is equal to one, then a single video frame in a single shot is reselected, and the timeline of the single video segment is corrected based on the re-determined time difference ratio. If the time difference ratio is less than one, the timeline of the video segment is preprocessed, and the correction duration is determined based on the time difference ratio.

4. The automated video editing method based on AI storyboard generation according to claim 3, characterized in that, The correction duration is determined based on the time difference ratio, where, The absolute value of the difference between the calculated time difference ratio and one is recorded as the correction reference value; The increase in correction duration is proportional to the correction reference value.

5. The automated video editing method based on AI storyboard generation according to claim 4, characterized in that, In S7, the process of determining whether the initial video generation is qualified based on the statistical count of each calibrated video segment includes: Calculate the difference between the number of qualified segments and the number of abnormal segments, and record the ratio of this difference to the number of qualified segments as the compliance ratio; If the pass rate is equal to one, the initial video generation is deemed qualified, and the generated video is output. If the compliance rate is less than one but greater than the preset compliance parameter, a notification message for unqualified materials will be issued. If the pass rate is less than or equal to the preset pass rate parameter, a notification message will be sent to regenerate the storyboard.

Images