Video editing method and system based on AI intelligence

Abstract

The invention is suitable for the technical field of video editing, and provides a video editing method and system based on AI intelligence, and the method comprises the steps: obtaining a template video which is in a video editing expected mode, and obtaining editing features through the template video; acquiring a to-be-processed video, preprocessing the to-be-processed video to obtain a first video, and editing the first video through the AI according to the editing feature; obtaining the edited video, obtaining the subjective score and a first weight corresponding to the objective score, obtaining a total score of the objective score according to the second weight, and obtaining a total score according to the total score of the objective score and the subjective score in combination with the first weight; the method comprises the following steps: acquiring a scoring and grading rule, acquiring optimization modes corresponding to different grades, sending the corresponding optimization modes to a terminal, re-scoring an optimized video, and determining whether the optimization modes are qualified or not according to a second score, thereby solving the problem that in the existing video editing process, editing cannot be carried out according to expected finished products; and the consistency of the edited video and the sample wafer cannot be ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of video editing, and particularly relates to a video editing method and system based on AI intelligence. Background Technique

[0002] With the wide application of video content, the requirements for video editing quality in industries such as short videos, film and television editing, and advertising promotion are constantly increasing. Traditional video editing mainly relies on manual operations. Editors need to manually perform editing, adjust colors, optimize audio, etc. This not only takes a lot of time and effort, but is also easily affected by subjective factors, resulting in unstable editing quality. In addition, there are still many challenges in ensuring the smoothness of rhythm, visual consistency, audio-video synchronization, etc. in manual editing.

[0003] In recent years, with the development of artificial intelligence technology, AI-based intelligent editing methods have gradually become an important technical direction in video editing. AI editing can automatically identify the editing style of the template video, and analyze, process, and optimize the video to be processed to make it more in line with the expected editing quality.

[0004] However, in the existing video editing process, there is a problem that it is impossible to edit according to the expected finished product, and it is impossible to ensure that the edited video is consistent with the sample video. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a video editing method based on AI intelligence, aiming to solve the problems proposed in the third part of the background technique.

[0006] The embodiments of the present invention are implemented as follows. A video editing method based on AI intelligence, the method includes:

[0007] Obtain a template video, where the template video is the expected mode of video editing, and obtain editing features through the template video;

[0008] Obtain the video to be processed, preprocess the video to be processed to obtain a first video, and edit the first video by AI according to the editing features;

[0009] Obtain the edited video, obtain the subjective score and the objective score corresponding to the first weight, obtain the total objective score according to the second weight, and obtain the total score by combining the objective score total and the subjective score with the first weight;

[0010] Obtain the scoring classification rules, obtain the optimization methods corresponding to different levels, send the corresponding optimization methods to the terminal, re-score the optimized video, and determine whether the optimization method is qualified according to the second score.

[0011] Preferably, the steps of obtaining the video to be processed, preprocessing the video to be processed to obtain the first video, and performing video editing on the first video by AI according to the editing features specifically include:

[0012] Obtain the video to be processed, and preprocess the video to be processed. The preprocessing includes video decoding, color and frame rate standardization, and audio sampling.

[0013] Obtain the preprocessed video to obtain the first video, and perform video editing on the first video by AI according to the editing features. The editing includes scene segmentation, segment combination, and subsequent processing.

[0014] Separate the first video into key frames similar to the template video according to the template video through scene segmentation, make the first video conform to the sound effect style of the template video through segment combination, and optimize the generated video through subsequent processing.

[0015] Preferably, the steps of obtaining the edited video, obtaining the corresponding first weight of the subjective score and the objective score, obtaining the total objective score according to the second weight, and obtaining the total score according to the combination of the total objective score and the subjective score with the first weight specifically include:

[0016] Obtain the edited video to obtain the second video, and obtain the video scoring criteria. The scoring criteria include subjective scores and objective scores.

[0017] Obtain the corresponding first weight of the subjective score and the objective score. The objective score includes evaluation of editing rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation.

[0018] Obtain the corresponding second weight of the subcategories within the objective score, obtain the total objective score according to the second weight, and obtain the total score according to the combination of the total objective score and the subjective score with the first weight. The total score is the final score of the second video.

[0019] Preferably, the steps of obtaining the scoring grading rules, obtaining the corresponding optimization methods for different levels, sending the corresponding optimization methods to the terminal, re-scoring the optimized video, and determining whether the optimization method is qualified according to the second score specifically include:

[0020] Obtain the scoring grading rules. The scoring grading rules are used to measure the editing quality of the second video, and the grading rules are divided into five levels.

[0021] Obtain the corresponding optimization methods for different levels. The optimization methods are used to improve the viewing quality of the second video, and send the corresponding optimization methods to the terminal. The optimization methods include editing rhythm optimization, visual optimization, audio-video alignment, and lens connection optimization.

[0022] Re - score the optimized video to obtain a second score, and determine whether the optimization method is qualified according to the second score.

[0023] Preferably, the editing features include shot transition mode, rhythm, color style, and composition.

[0024] Another object of the embodiments of the present invention is to provide an AI - based intelligent video editing system, which includes:

[0025] A template video module, which obtains a template video. The template video is the expected mode of video editing, and editing features are obtained through the template video.

[0026] A video processing module, which obtains a video to be processed, pre - processes the video to be processed to obtain a first video, and edits the first video by AI according to the editing features.

[0027] A scoring module, which obtains the edited video, obtains the subjective score and the corresponding first weight of the objective score, obtains the total objective score according to the second weight, and obtains the total score according to the combination of the total objective score and the subjective score with the first weight.

[0028] An optimization module, which obtains the scoring classification rules, obtains the corresponding optimization methods for different levels, sends the corresponding optimization methods to the terminal, re - scores the optimized video, and determines whether the optimization method is qualified according to the second score.

[0029] Preferably, the video processing module includes:

[0030] A pre - processing unit, which obtains the video to be processed and pre - processes the video to be processed. The pre - processing includes video decoding, color and frame rate standardization, and audio sampling.

[0031] A first video unit, which obtains the pre - processed video to obtain a first video, and edits the first video by AI according to the editing features. The editing includes scene segmentation, segment combination, and subsequent processing.

[0032] An editing unit, which divides the first video into key frames similar to the template video according to the template video through scene separation, makes the first video conform to the sound effect style of the template video through segment combination, and optimizes the generated video through subsequent processing.

[0033] Preferably, the scoring module includes:

[0034] A second video unit, which obtains the edited video to obtain a second video, and obtains the video scoring criteria. The scoring criteria include subjective score and objective score.

[0035] A scoring unit that obtains the first weights corresponding to the subjective score and the objective score. The objective score includes evaluations of clip rhythm matching degree, visual quality, audio synchronization, and video coherence;

[0036] A total score unit that obtains the second weights corresponding to the subcategories within the objective score, obtains the total objective score based on the second weights, and obtains the total score by combining the total objective score and the subjective score with the first weights. The total score is the final score of the second video.

[0037] Preferably, the optimization module includes:

[0038] A grading unit that obtains the scoring grading rules, which are used to measure the clip quality of the second video, and the grading rules are divided into five levels;

[0039] An optimization unit that obtains the corresponding optimization methods for different levels. The optimization methods are used to improve the viewing quality of the second video and send the corresponding optimization methods to the terminal. The optimization methods include clip rhythm optimization, visual optimization, audio-video alignment, and shot transition optimization;

[0040] A re-measurement unit that re-scores the optimized video to obtain a second score and determines whether the optimization method is qualified according to the second score.

[0041] Preferably, the clip features include shot transition mode, rhythm, color style, and composition.

[0042] A video clip method based on AI intelligence provided by an embodiment of the present invention. It obtains a template video, obtains clip features through the template video, obtains a video to be processed, preprocesses the video to be processed, obtains the preprocessed video to get the first video, and clips the first video by AI according to the clip features. The clipping includes scene segmentation, segment combination, and subsequent processing. The first video is segmented into key frames similar to the template video according to the template video through scene segmentation, the first video is made to conform to the audio style of the template video through segment combination, and the generated video is optimized through subsequent processing to obtain the clipped video to get the second video. It obtains the video scoring standard, obtains the first weights corresponding to the subjective score and the objective score, obtains the second weights corresponding to the subcategories within the objective score, obtains the total objective score based on the second weights, obtains the total score by combining the total objective score and the subjective score with the first weights, obtains the scoring grading rules, obtains the corresponding optimization methods for different levels, sends the corresponding optimization methods to the terminal, re-scores the optimized video to obtain a second score, and determines whether the optimization method is qualified according to the second score, solving the problem that in the existing video clipping process, it is impossible to clip according to the expected finished product and it is impossible to ensure that the clipped video is consistent with the sample video. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1Flowchart of a video editing method based on AI intelligence provided by an embodiment of the present invention;

[0044] Figure 2 Flowchart of the step of preprocessing a video to be processed and editing a first video according to editing features by AI provided by an embodiment of the present invention;

[0045] Figure 3 Flowchart of the step of obtaining a total score by combining an objective score total and a subjective score with a first weight provided by an embodiment of the present invention;

[0046] Figure 4 Flowchart of the step of obtaining a scoring classification rule, obtaining corresponding optimization methods for different levels, and determining whether the optimization method is qualified according to a second score provided by an embodiment of the present invention;

[0047] Figure 5 Architecture diagram of a video editing system based on AI intelligence provided by an embodiment of the present invention;

[0048] Figure 6 Architecture diagram of a video processing module provided by an embodiment of the present invention;

[0049] Figure 7 Architecture diagram of a scoring module provided by an embodiment of the present invention;

[0050] Figure 8 Architecture diagram of an optimization module provided by an embodiment of the present invention. Detailed implementation manners

[0051] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] It can be understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of this application, the first xx script may be referred to as the second xx script, and similarly, the second xx script may be referred to as the first xx script.

[0053] As Figure 1 shown, a video editing method based on AI intelligence provided by an embodiment of the present invention, the method includes:

[0054] S100, obtaining a template video, the template video being an expected mode for video editing, and obtaining editing features through the template video.

[0055] In this step, a sample video is obtained. In an intelligent video editing system, the sample video is used to set the expected mode of editing. By analyzing the sample video, a series of editing features can be extracted to guide the editing process of the target video, making it conform to the style and quality requirements of the sample video.

[0056] Editing features are obtained through the sample video. The editing features include the shot transition mode, rhythm, color style, and composition. The shot transition mode refers to the transition method between different shots in the video, including the way of switching, frequency, and the combination mode of different types of shots; the editing rhythm refers to the time interval between shot transitions in the video and its changing mode, which affects the viewing experience, and the rhythm can be fast or slow.

[0057] The color style refers to the overall hue, contrast, saturation, etc. of the video, which affects the visual perception of the audience; the composition refers to the arrangement of visual elements within the video frame, including the position of the protagonist, background composition, camera angle, etc.; when applied to the target video, it keeps the editing style consistent.

[0058] S200, Obtain the video to be processed, preprocess the video to be processed to obtain the first video, and use AI to edit the first video according to the editing features.

[0059] In this step, the video to be processed is obtained. The video to be processed is the original video material that has not been edited, which may contain unnecessary segments, noise, and color mismatch problems. The video to be processed is preprocessed. The preprocessing includes video decoding, color and frame rate standardization, and audio sampling. After preprocessing, it meets the editing requirements and provides high-quality input data for AI editing to obtain the first video.

[0060] Use AI to edit the first video according to the editing features. By using AI to match the editing style, adjust the rhythm, optimize the color, and composition, a high-quality video can be automatically generated.

[0061] S300, Obtain the edited video, obtain the subjective score and the corresponding first weight for the objective score, obtain the total objective score according to the second weight, and obtain the total score according to the combination of the total objective score and the subjective score with the first weight.

[0062] In this step, the edited video is obtained. The edited video has passed AI scene segmentation, segment combination, and subsequent optimization and conforms to the style of the sample video. Obtain the subjective score and the corresponding first weight for the objective score. The subjective score is provided by human reviewers and mainly measures factors such as the visual perception, mood, and visual impact of the video. The objective score includes the evaluation of editing rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation. The evaluation of editing rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation correspond to different second weights.

[0063] The total score is obtained by combining the objective score total and the subjective score with the first weight. The final score is obtained by combining the subjective score and the objective score with the first weight. The video quality is determined according to the total score, and it is decided whether optimization is required.

[0064] S400, obtain the scoring and grading rules, obtain the corresponding optimization methods for different levels, send the corresponding optimization methods to the terminal, re-score the optimized video, and determine whether the optimization method is qualified according to the second score.

[0065] In this step, obtain the scoring and grading rules. The scoring and grading are divided into five levels. The scoring levels include S level, A level, B level, C level, and D level. Obtain the corresponding optimization methods for different levels. The optimization methods are automatically assigned according to the scoring levels. Send the corresponding optimization methods to the terminal;

[0066] Re-score the optimized video. The second video optimized version is scored again. According to the previous subjective score + objective score calculation method, generate the second score. Determine whether the optimization method is qualified according to the second score. If the score after optimization is improved to A level or S level, it is considered qualified.

[0067] As Figure 2 shown, as a preferred embodiment of the present invention, the steps of obtaining the video to be processed, preprocessing the video to be processed to obtain the first video, and performing video editing on the first video by AI according to the editing features specifically include:

[0068] S201, obtain the video to be processed, and preprocess the video to be processed. The preprocessing includes video decoding, color and frame rate standardization, and audio sampling.

[0069] In this step, obtain the video to be processed. The video to be processed is the original video material that has not been edited and may contain unnecessary segments, noise, and color mismatch problems. Preprocess the video to be processed. The preprocessing includes video decoding, color and frame rate standardization, and audio sampling;

[0070] Video decoding refers to converting videos in different coding formats into a standard format for subsequent processing. Different videos may use different coding methods (such as H.264, HEVC, VP9), and decoding can convert them into a common format (such as MP4 + H.264);

[0071] Color and frame rate standardization means that the color spaces and frame rates of different videos may vary. To ensure the visual consistency of the final edit, color and frame rate standardization processes are required. For color standardization, LUT (Look-Up Table) conversion is used for color matching to unify the video style, and Histogram Matching is used for histogram matching to make the tones of all videos consistent. Frame rate standardization determines the smoothness of the video. Videos with different frame rates (such as 24fps, 30fps, 60fps) may result in inconsistent rhythms during playback or editing. Therefore, frame rate standardization is needed. Frame rate adjustment includes dropping frames and inserting frames. Optical Flow can be used for smooth frame insertion, such as from 24fps to 30fps, and DeepFrameInterpolation (deep frame interpolation model) is used to improve the quality of frame insertion;

[0072] Audio sampling means that different videos may use different audio sampling rates (such as 44.1kHz, 48kHz, 96kHz), which affects the synchronization during editing and playback. Therefore, audio sampling standardization is required to unify the sampling rate. For example, all audio is converted to 48kHz, and high-quality resampling algorithms (SoX, FFmpeg) are used for conversion.

[0073] S202. Obtain the preprocessed video to get the first video, and perform editing on the first video by AI according to the editing features. The editing includes scene segmentation, segment combination, and subsequent processing.

[0074] In this step, after obtaining the preprocessed video and completing video preprocessing (decoding, color standardization, frame rate unification, audio sampling), the first video, which is the standardized video, is obtained. Editing is performed on the first video by AI according to the editing features. The editing includes scene segmentation, segment combination, and subsequent processing;

[0075] Scene segmentation refers to dividing the video into different scenes or shots for subsequent screening and editing. Usually, each scene contains logically related shots, such as the same dialogue scene or a complete action shot. The frame difference method can be used to calculate the pixel changes between adjacent frames to identify the shot transition points;

[0076] Segment combination refers to selecting the optimal segments from the segmented scenes and recombining them according to the editing features of the template video. Segment combination includes editing rhythm matching, emotion matching, speech text analysis, and visual matching;

[0077] The post-processing includes clip fine-tuning, transition effects, audio-visual synchronization, subtitle effects, etc., to make the final video smoother and meet the user's needs. The post-processing is for transition effects, visual effects, background music matching, and subtitle generation. For example, hard cuts are used in action scenes, fades in and out are used in narrative scenes, the dialogue is delayed by 0.2 seconds, automatic correction is performed, the dialogue content is automatically converted into subtitles, and an appearance animation is added.

[0078] S203. The first video is separated into key frames similar to the template video according to the template video through scene separation, the first video is made to conform to the audio style of the template video through clip combination, and the generated video is optimized through post-processing.

[0079] In this step, the first video is separated into key frames similar to the template video according to the template video through scene separation. Scene separation is to divide the frame structure of the first video so that it is consistent with the scene cutting method and key frames of the template video. The AI analyzes the lens switching mode, scene duration, and composition layout of the template video, cuts the first video into similar paragraphs, the AI calculates the lens rhythm of the template video, and re-cuts the first video to match the key scenes and editing style of the template video;

[0080] Clip combination means selecting appropriate clips from the first video to make it conform to the audio style of the template video. The AI analyzes the background music, dialogue rhythm, and sound effect distribution of the template video, adjusts the audio structure of the first video, the AI calculates the rhythm points of the template video, and automatically adjusts the editing points of the first video to align with the music rhythm, extracts the sound effect distribution of the template video, and adds similar environmental sound effects to the first video;

[0081] Post-processing is to optimize the edited video, including transition effects, color style matching, subtitle and animation enhancement, to ensure that the video quality reaches a high standard. The AI performs Lip-sync adjustment to correct the problem of out-of-sync lip movements, and uses dynamic range compression (DRC) to balance the volume of the background music and dialogue.

[0082] As Figure 3 shown, as a preferred embodiment of the present invention, the steps of obtaining the edited video, obtaining the subjective score and the objective score corresponding to the first weight, obtaining the total objective score according to the second weight, and obtaining the total score according to the total objective score and the subjective score combined with the first weight specifically include:

[0083] S301. Obtain the edited video to get the second video, and obtain the video scoring criteria, where the scoring criteria include subjective score and objective score.

[0084] In this step, obtain the clipped video to get the second video. The second video is a video that meets the clipping expectations after scene segmentation, segment combination, and subsequent optimization, and needs to be further scored and adjusted to ensure that the quality meets the standards.

[0085] Obtain the video scoring criteria. The video scoring criteria are divided into subjective scoring (based on the audience experience) and objective scoring (based on technical indicators). The subjective scoring is provided by the audience and expert reviewers, measuring aspects such as the viewing experience, information transmission, and creative performance of the video, directly scoring, and taking the average score as the subjective score.

[0086] The objective scoring is based on the technical indicators of the video, including the evaluation of the clipping rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation.

[0087] S302, obtain the first weights corresponding to the subjective score and the objective score. The objective score includes the evaluation of the clipping rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation.

[0088] In this step, obtain the first weights corresponding to the subjective score and the objective score. The subjective score and the objective score are integrated through the first weights to finally determine the video quality rating. The objective score includes the evaluation of the clipping rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation.

[0089] The evaluation of the clipping rhythm matching degree is used to measure the matching degree between the clipping rhythm and the template video, ensure that the video rhythm is smooth and conforms to the expected style, count the average shot length of the video, and compare the time series rhythm of the template video and the clipped video.

[0090] The visual quality evaluation measures the clarity, color consistency, and stability of the video to ensure that the visual quality meets the standards. The visual quality evaluation includes clarity detection, color consistency, and video stability. Calculate the SSIM (Structural Similarity Index) to evaluate the sharpness of the picture, use ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) to correct the blurred video, calculate the color histogram matching degree between the target video and the template video using Color Histogram Matching, and calculate the optical flow offset (Optical Flow) to ensure that the picture does not shake too much.

[0091] The audio synchronization evaluation ensures that the audio (dialogue, background music, ambient sound) is synchronized with the picture without delay or misalignment. Use Dlib and OpenPose for lip movement analysis, calculate the synchronization error between the audio and the lip movement, calculate the beat points of the background music (Beat Tracking), and ensure that the clipping points match the beat points.

[0092] Video coherence assessment evaluates the logical coherence of the clipped video to ensure natural scene transitions. It uses YOLO / Faster R-CNN to detect the consistency of the main subjects (people, objects), calculates the similarity between the front and rear shots (Cosine Similarity), and ensures visual coherence.

[0093] S303. Obtain the corresponding second weight for the subcategory of the objective score, obtain the total objective score based on the second weight, and obtain the total score by combining the total objective score and the subjective score with the first weight. The total score is the final score of the second video.

[0094] In this step, obtain the corresponding second weight for the subcategory of the objective score. The second weight is used to calculate the total objective score. The weight for the rhythm matching degree assessment is 30%, the weight for the visual quality assessment is 25%, the weight for the audio synchronization assessment is 25%, and the weight for the video coherence assessment is 20%. Obtain the total objective score based on the second weight. The total objective score = rhythm matching degree score × 30% + visual quality score × 25% + audio synchronization score × 25% + video coherence score × 20%.

[0095] Obtain the total score by combining the total objective score and the subjective score with the first weight. The corresponding first weight for the total objective score is 60%, and the corresponding first weight for the subjective score is 40%. The total score = total objective score × 60% + subjective score × 40% to obtain the final score of the second video.

[0096] As Figure 4 shown, as a preferred embodiment of the present invention, the steps of obtaining the scoring grading rules, obtaining the corresponding optimization methods for different levels, sending the corresponding optimization methods to the terminal, re-scoring the optimized video, and determining whether the optimization method is qualified according to the second score specifically include:

[0097] S401. Obtain the scoring grading rules, where the scoring grading rules are used to measure the editing quality of the second video, and the grading rules are divided into five levels.

[0098] In this step, obtain the scoring grading rules. The scoring grading is divided into five levels. The scoring levels include S level, A level, B level, C level, and D level. The scoring grading rules are used to measure the editing quality of the second video.

[0099] Table 1: Scoring grading rules

[0100]

[0101] S402. Obtain the corresponding optimization methods for different levels. The optimization methods are used to improve the viewing quality of the second video, and send the corresponding optimization methods to the terminal. The optimization methods include editing rhythm optimization, visual optimization, audio-video alignment, and shot connection optimization.

[0102] In this step, obtain the optimization methods corresponding to different levels. The optimization methods are used to improve the viewing quality of the second video. For S-level, directly release it or enhance it with advanced special effects; for A-level, perform minor optimizations such as color matching and audio adjustment; for B-level, focus on optimizing the editing rhythm, color matching, and audio balance; for C-level, overall optimization is required, such as lens stabilization, color correction, and audio-video synchronization; for D-level, re-edit or even replace the materials.

[0103] Send the corresponding optimization method to the terminal, and an optimization report can be generated, enabling the terminal to view more directly.

[0104] S403, re-rate the optimized video to obtain a second rating, and determine whether the optimization method is qualified according to the second rating.

[0105] In this step, re-rate the optimized video. The second video optimized version of the optimized video needs to be recalculated according to the previous rating system to obtain a new subjective rating and objective rating, and combine the first weight to obtain the final second rating.

[0106] Determine whether the optimization method is qualified according to the second rating. If the rating after optimization is improved to A-level or S-level, it is qualified. That is, if the score ≤ 79, it means the optimization fails to meet the standard. Only when the optimization is qualified can the video be released. This process ensures that the video editing quality meets industry standards and improves the viewing experience of the audience.

[0107] Such as Figure 5 shown, a video editing system based on AI intelligence provided by an embodiment of the present invention, the system includes:

[0108] A template video module 100, used to obtain a template video, and the template video is an expected mode for video editing, and editing features are obtained through the template video.

[0109] In this system, the template video module 100 obtains a template video. In the intelligent video editing system, the template video is used to set the expected mode of editing. By analyzing the template video, a series of editing features can be extracted to guide the editing process of the target video to make it meet the style and quality requirements of the template video.

[0110] Obtain editing features through the template video. The editing features include the lens switching mode, rhythm, color style, and composition. The lens switching mode refers to the transition method between different lenses in the video, including the switching method, frequency, and combination mode of different types of lenses; the editing rhythm refers to the time interval between lens switches in the video and its change mode, which affects the viewing experience, and the rhythm can be a fast rhythm or a slow rhythm.

[0111] The color style refers to the overall tone, contrast, saturation, etc. of the video, which affects the visual perception of the audience; the composition refers to the arrangement of visual elements within the video frame, including the position of the protagonist, background composition, camera angle, etc.; it is applied to the target video to keep the editing style consistent.

[0112] The video processing module 200 is used to obtain the video to be processed, preprocess the video to be processed to obtain the first video, and edit the first video according to the editing features by AI.

[0113] In this system, the video processing module 200 obtains the video to be processed. The video to be processed is the original video material that has not been edited, which may contain unnecessary segments, noise, and color mismatch problems. Preprocess the video to be processed. The preprocessing includes video decoding, color and frame rate standardization, and audio sampling. After preprocessing, it meets the editing requirements and provides high-quality input data for AI editing to obtain the first video;

[0114] Edit the first video according to the editing features by AI. By using AI to match the editing style, adjust the rhythm, optimize the color and composition, high-quality videos can be automatically generated.

[0115] The scoring module 300 is used to obtain the edited video, obtain the subjective score and the corresponding first weight of the objective score, obtain the total objective score according to the second weight, and obtain the total score according to the combination of the total objective score and the subjective score with the first weight.

[0116] In this system, the scoring module 300 obtains the edited video. The edited video has passed AI scene segmentation, segment combination, and subsequent optimization and conforms to the style of the sample video. Obtain the subjective score and the corresponding first weight of the objective score. The subjective score is provided by human reviewers and mainly measures factors such as the visual perception, emotion, and visual impact of the video. The objective score includes the evaluation of the editing rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation. The evaluation of the editing rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation correspond to different second weights;

[0117] Obtain the total score according to the combination of the total objective score and the subjective score with the first weight. Through the first weight, combine the subjective score and the objective score to obtain the final score. Determine the video quality according to the total score and decide whether optimization is needed.

[0118] The optimization module 400 is used to obtain the scoring grading rules, obtain the corresponding optimization methods for different levels, send the corresponding optimization methods to the terminal, re-score the optimized video, and determine whether the optimization method is qualified according to the second score.

[0119] In this system, the optimization module 400 obtains the scoring and grading rules. The scoring and grading are divided into five levels, and the scoring levels include S level, A level, B level, C level, and D level. It obtains the corresponding optimization methods for different levels, and the optimization methods are automatically assigned according to the scoring levels, and sends the corresponding optimization methods to the terminal;

[0120] Re-score the optimized video. The second optimized version of the video is scored again. According to the previous subjective scoring + objective scoring calculation method, a second score is generated. Determine whether the optimization method is qualified according to the second score. If the score after optimization is improved to level A or S level, it is considered qualified.

[0121] As Figure 6 shown, as a preferred embodiment of the present invention, the video processing module 200 includes:

[0122] A preprocessing unit 201, which is used to obtain the video to be processed and preprocess the video to be processed. The preprocessing includes video decoding, color and frame rate standardization, and audio sampling.

[0123] In this module, the preprocessing unit 201 obtains the video to be processed. The video to be processed is the original video material that has not been edited, which may contain unnecessary segments, noise, and color mismatch problems. Preprocess the video to be processed. The preprocessing includes video decoding, color and frame rate standardization, and audio sampling;

[0124] Video decoding refers to converting videos in different coding formats into a standard format for subsequent processing. Different videos may adopt different coding methods (such as H.264, HEVC, VP9), and decoding can convert them into a common format (such as MP4+H.264);

[0125] Color and frame rate standardization: The color spaces and frame rates of different videos may be different. In order to ensure the visual consistency of the final clip, color and frame rate standardization processing is required. Color standardization uses LUT (lookup table) conversion for color matching to unify the video style, and uses Histogram Matching for histogram matching to make all video tones consistent; Frame rate standardization determines the smoothness of the video. Videos with different frame rates (such as 24fps, 30fps, 60fps) may cause inconsistent rhythms during playback or editing. Therefore, frame rate standardization is required. Frame rate adjustment includes frame dropping and frame interpolation. Frame interpolation can use Optical Flow for smooth frame interpolation. For example, from 24fps to 30fps, DeepFrameInterpolation (deep frame interpolation model) is used to improve the quality of frame interpolation;

[0126] Audio sampling means that different videos may use different audio sampling rates (such as 44.1 kHz, 48 kHz, 96 kHz), which affects the synchronization during editing and playback. Therefore, audio sampling standardization is required to unify the sampling rate. For example, all audio is converted to 48 kHz, and high-quality resampling algorithms (SoX, FFmpeg) are used for conversion.

[0127] The first video unit 202 is used to obtain the preprocessed video to get the first video, and the first video is edited by AI according to the editing features. The editing includes scene segmentation, segment combination, and subsequent processing.

[0128] In this module, the first video unit 202 obtains the preprocessed video. After completing video preprocessing (decoding, color standardization, frame rate unification, audio sampling), the first video, that is, the standardized video, is obtained. The first video is edited by AI according to the editing features. The editing includes scene segmentation, segment combination, and subsequent processing;

[0129] Scene segmentation refers to dividing the video into different scenes or shots for subsequent screening and editing. Usually, each scene contains logically related shots, such as the same dialogue scene or a complete action shot. The frame difference method can be used to calculate the pixel changes between adjacent frames to identify the shot transition points;

[0130] Segment combination refers to selecting the optimal segments from the segmented scenes and recombining them according to the editing features of the template video. Segment combination includes editing rhythm matching, emotion matching, speech text analysis, and visual matching;

[0131] Subsequent processing includes editing fine-tuning, transition special effects, audio-visual synchronization, subtitle special effects, etc., to make the final video smoother and meet the user's needs. The subsequent processing is transition effects, visual effects, background music matching, and subtitle generation. For example, hard cuts are used in action scenes, fades are used in narrative scenes, the dialogue is delayed by 0.2 seconds, automatically corrected, the dialogue content is automatically converted into subtitles, and an appearance animation is added.

[0132] The editing unit 203 is used to divide the first video into key frames similar to the template video according to the template video through scene separation, make the first video conform to the sound effect style of the template video through segment combination, and optimize the generated video through subsequent processing.

[0133] In this module, the editing unit 203 divides the first video into key frames similar to the template video through scene segmentation. Scene segmentation is to divide the frame structure of the first video so that it is consistent with the scene cutting method and key frames of the template video. The AI analyzes the shot transition mode, scene duration, and composition layout of the template video, cuts the first video into similar segments, calculates the shot rhythm of the template video, and recuts the first video to match the key scenes and editing style of the template video;

[0134] Segment combination means selecting appropriate segments from the first video to conform to the sound effect style of the template video. The AI analyzes the background music, dialogue rhythm, and sound effect distribution of the template video, adjusts the audio structure of the first video, calculates the rhythm points of the template video, and automatically adjusts the editing points of the first video to align with the music rhythm, extracts the sound effect distribution of the template video, and adds similar ambient sound effects to the first video;

[0135] Subsequent processing is to optimize the edited video, including transition effects, color style matching, subtitles, and animation enhancement, to ensure that the video quality reaches a high standard. The AI performs Lip-sync adjustment to correct the problem of out-of-sync lip movements and uses dynamic range compression (DRC) to balance the volume of background music and dialogue.

[0136] As Figure 7 shown, as a preferred embodiment of the present invention, the scoring module 300 includes:

[0137] A second video unit 301 for obtaining the edited video to get a second video and obtaining video scoring criteria, where the scoring criteria include subjective scoring and objective scoring.

[0138] In this module, the second video unit 301 obtains the edited video to get a second video. The second video is a video that meets the editing expectations generated after scene segmentation, segment combination, and subsequent optimization, and needs to be further scored and adjusted to ensure that the quality meets the standards;

[0139] Obtain video scoring criteria. The video scoring criteria are divided into subjective scoring (based on the audience experience) and objective scoring (based on technical indicators). The subjective scoring is provided by the audience and expert reviewers, measures aspects such as the viewing experience, information transmission, and creative performance of the video, and directly gives scores. The average score is taken as the subjective score;

[0140] The objective scoring is based on the technical indicators of the video, including the evaluation of the editing rhythm matching degree, visual quality evaluation, audio synchronization evaluation, and video coherence evaluation.

[0141] The scoring unit 302 is used to obtain the first weights corresponding to the subjective score and the objective score. The objective score includes the evaluation of the editing rhythm matching degree, the visual quality evaluation, the audio synchronization evaluation, and the video coherence evaluation.

[0142] In this module, the scoring unit 302 obtains the first weights corresponding to the subjective score and the objective score. The subjective score and the objective score are integrated through the first weights to finally determine the video quality rating. The objective score includes the evaluation of the editing rhythm matching degree, the visual quality evaluation, the audio synchronization evaluation, and the video coherence evaluation.

[0143] The evaluation of the editing rhythm matching degree is used to measure the matching degree between the editing rhythm and the template video, ensure that the video rhythm is smooth and conforms to the expected style, count the average shot length of the video, and compare the time series rhythm of the template video and the edited video.

[0144] The visual quality evaluation measures the clarity, color consistency, and stability of the video to ensure that the visual quality meets the standards. The visual quality evaluation includes clarity detection, color consistency, and video stability. Calculate the SSIM (Structural Similarity Index) to evaluate the sharpness of the picture, use ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) to correct the blurred video, calculate the color histogram matching degree between the target video and the template video using Color Histogram Matching, and calculate the optical flow offset (Optical Flow) to ensure that the picture does not shake too much.

[0145] The audio synchronization evaluation ensures that the audio (dialogue, background music, ambient sound) is synchronized with the picture without delay or misalignment. Use Dlib and OpenPose for lip movement analysis, calculate the synchronization error between the audio and the lip movement, calculate the beat points of the background music (Beat Tracking), and ensure that the editing points match the beat points.

[0146] The video coherence evaluation evaluates the logical coherence of the edited video to ensure natural scene transitions. Use YOLO / Faster R-CNN to detect the consistency of the main subjects (people, objects), calculate the similarity between the front and rear shots (Cosine Similarity), and ensure visual coherence.

[0147] The total score unit 303 is used to obtain the second weights corresponding to the sub-categories within the objective score, obtain the total objective score according to the second weights, and obtain the total score according to the total objective score and the subjective score combined with the first weights. The total score is the final score of the second video.

[0148] In this module, the total score unit 303 obtains the corresponding second weight for the subcategories of the objective score. The second weight is used to calculate the total objective score. The weight for the rhythm matching degree evaluation is 30%, the weight for the visual quality evaluation is 25%, the weight for the audio synchronization evaluation is 25%, and the weight for the video coherence evaluation is 20%. The total objective score is obtained according to the second weight. The total objective score = rhythm matching degree score × 30% + visual quality score × 25% + audio synchronization score × 25% + video coherence score × 20%;

[0149] The total score is obtained by combining the total objective score and the subjective score with the first weight. The first weight corresponding to the total objective score is 60%, and the first weight corresponding to the subjective score is 40%. The total score = total objective score × 60% + subjective score × 40%, and the final score of the second video is obtained.

[0150] As Figure 8 shown, as a preferred embodiment of the present invention, the optimization module 400 includes:

[0151] A grading unit 401 for obtaining a scoring grading rule, which is used to measure the quality of the second video clip. The grading rule is divided into five levels.

[0152] In this module, the grading unit 401 obtains a scoring grading rule. The scoring grading is divided into five levels. The scoring levels include S level, A level, B level, C level, and D level. The scoring grading rule is used to measure the quality of the second video clip.

[0153] An optimization unit 402 for obtaining corresponding optimization methods for different levels. The optimization methods are used to improve the viewing quality of the second video, and send the corresponding optimization methods to the terminal. The optimization methods include clip rhythm optimization, visual optimization, audio-visual alignment, and shot connection optimization.

[0154] In this module, the optimization unit 402 obtains corresponding optimization methods for different levels. The optimization methods are used to improve the viewing quality of the second video. The S level is directly released or enhanced with advanced special effects; the A level is slightly optimized, such as color matching and audio adjustment; the B level focuses on optimizing the clip rhythm, color matching, and audio balance; the C level requires overall optimization, such as lens stabilization, color correction, and audio-visual synchronization; the D level requires re-editing or even replacement of materials;

[0155] Sending the corresponding optimization methods to the terminal can generate an optimization report, which can enable the terminal to view more directly.

[0156] A retest unit 403 for re-scoring the optimized video to obtain a second score, and determining whether the optimization method is qualified according to the second score.

[0157] In this module, the remeasurement unit 403 re-evaluates the optimized video. The second optimized version of the video needs to be recalculated according to the previous scoring system to obtain the new subjective score and objective score, and the final second score is obtained by combining the first weight.

[0158] Determine whether the optimization method is qualified according to the second score. If the optimized score is improved to level A or S level, it is considered qualified. That is, the score ≤ 79 indicates that the optimization fails to meet the standard. Only when the optimization is qualified can the video be released. This process ensures that the video editing quality meets the industry standards and improves the audience experience.

[0159] In one embodiment, a computer device is proposed. The computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0160] Obtain a sample video, where the sample video is the expected mode of video editing, and obtain editing features through the sample video;

[0161] Obtain the video to be processed, preprocess the video to be processed to obtain the first video, and edit the first video by AI according to the editing features;

[0162] Obtain the edited video, obtain the subjective score and the first weight corresponding to the objective score, obtain the total objective score according to the second weight, and obtain the total score by combining the total objective score and the subjective score with the first weight;

[0163] Obtain the scoring classification rules, obtain the optimization methods corresponding to different levels, send the corresponding optimization methods to the terminal, re-evaluate the optimized video, and determine whether the optimization method is qualified according to the second score.

[0164] In one embodiment, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, the processor is caused to execute the following steps:

[0165] Obtain a sample video, where the sample video is the expected mode of video editing, and obtain editing features through the sample video;

[0166] Obtain the video to be processed, preprocess the video to be processed to obtain the first video, and edit the first video by AI according to the editing features;

[0167] Obtain the edited video, obtain the subjective score and the first weight corresponding to the objective score, obtain the total objective score according to the second weight, and obtain the total score by combining the total objective score and the subjective score with the first weight;

[0168] Obtain the scoring and grading rules, obtain the corresponding optimization methods for different levels, send the corresponding optimization methods to the terminal, re-score the optimized video, and determine whether the optimization method is qualified according to the second score.

[0169] It should be understood that although the steps in the flowcharts of the embodiments of the present invention are shown sequentially according to the indications of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0170] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0171] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0172] The above-described embodiments merely represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent for the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several variations and improvements can still be made, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention shall be subject to the appended claims.

[0173] The foregoing is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A video editing method based on AI intelligence, characterized in that: The method comprises: Obtaining a sample video, where the sample video is an expected mode of video editing, and obtaining editing features through the sample video; Obtaining a video to be processed, preprocessing the video to be processed to obtain a first video, and editing the first video according to editing features by using AI; Obtain the edited video, obtain the subjective score and the objective score corresponding to the first weight, obtain the total objective score according to the second weight, and obtain the total score according to the total objective score and the subjective score combined with the first weight; Obtain scoring and grading rules, obtain optimization methods corresponding to different levels, send the corresponding optimization methods to the terminal, re-score the optimized video, and determine whether the optimization method is qualified based on the second score.

2. The AI-based video editing method according to claim 1, characterized in that: The steps of obtaining a video to be processed, preprocessing the video to be processed, obtaining a first video, and editing the first video according to the editing features by AI specifically include: Obtaining a video to be processed and preprocessing the video to be processed, wherein the preprocessing includes video decoding, color and frame rate standardization, and audio sampling; The pre-processed video is obtained to obtain a first video, and the first video is edited by AI according to the editing features, wherein the editing includes scene segmentation, segment combination, and subsequent processing; The first video is divided into key pictures similar to the model video through scene segmentation, the first video is made to conform to the sound effect style of the model video through segment combination, and the generated video is optimized through subsequent processing.

3. The AI-based video editing method according to claim 2, characterized in that: The steps of obtaining the edited video, obtaining the subjective score and the objective score corresponding to the first weight, obtaining the objective score total score according to the second weight, and obtaining the total score according to the objective score total score and the subjective score combined with the first weight specifically include: Obtaining the edited video to obtain a second video, and obtaining a video scoring standard, wherein the scoring standard includes a subjective score and an objective score; Obtaining the subjective score and the objective score corresponding to the first weight, the objective score includes editing rhythm matching evaluation, visual quality evaluation, audio synchronization evaluation and video coherence evaluation; Obtain the corresponding second weight of the subcategory in the objective score, obtain the total objective score according to the second weight, and obtain the total score according to the total objective score and the subjective score combined with the first weight, and the total score is the final score of the second video.

4. The AI-based video editing method according to claim 3, characterized in that: The steps of obtaining the scoring and grading rules, obtaining the optimization methods corresponding to different grades, sending the corresponding optimization methods to the terminal, re-scoring the optimized video, and determining whether the optimization method is qualified according to the second score specifically include: Obtaining a rating and grading rule, wherein the rating and grading rule is used to measure the quality of the second video clip, and the rating rule is divided into five levels; Obtaining optimization methods corresponding to different levels, where the optimization methods are used to improve the viewing quality of the second video, and sending the corresponding optimization methods to the terminal, where the optimization methods include editing rhythm optimization, visual optimization, audio and video alignment, and shot connection optimization; The optimized video is re-scored to obtain a second score, and whether the optimization method is qualified is determined based on the second score.

5. The AI-based video editing method according to claim 1, characterized in that: The editing features include shot switching patterns, rhythm, color style and composition.

6. A video editing system based on AI intelligence, characterized in that: The system comprises: A sample video module obtains a sample video, wherein the sample video is an expected mode of video editing, and the editing features are obtained through the sample video; The video processing module obtains the video to be processed, pre-processes the video to be processed, obtains the first video, and edits the first video according to the editing features through AI; A scoring module obtains the edited video, obtains the subjective score and the objective score corresponding to the first weight, obtains the total objective score according to the second weight, and obtains the total score according to the total objective score and the subjective score combined with the first weight; The optimization module obtains the scoring and grading rules, obtains the optimization methods corresponding to different levels, sends the corresponding optimization methods to the terminal, re-scores the optimized video, and determines whether the optimization method is qualified according to the second score.

7. The AI-based video editing system according to claim 6, characterized in that: The video processing module comprises: A preprocessing unit, which obtains a video to be processed and preprocesses the video to be processed, wherein the preprocessing includes video decoding, color and frame rate standardization, and audio sampling; The first video unit obtains the pre-processed video to obtain the first video, and edits the first video according to the editing features through AI, wherein the editing includes scene segmentation, segment combination and subsequent processing; The editing unit divides the first video into key pictures similar to the model video through scene separation, makes the first video conform to the sound effect style of the model video through segment combination, and optimizes the generated video through subsequent processing.

8. The AI-based video editing system according to claim 7, characterized in that: The scoring module includes: The second video unit obtains the edited video to obtain the second video, and obtains the video scoring standard, wherein the scoring standard includes a subjective score and an objective score; A scoring unit, which obtains a subjective score and an objective score corresponding to a first weight, wherein the objective score includes an editing rhythm matching evaluation, a visual quality evaluation, an audio synchronization evaluation, and a video continuity evaluation; The total score unit obtains the corresponding second weight of the subcategory in the objective score, obtains the total score of the objective score according to the second weight, and obtains the total score according to the total score of the objective score and the subjective score combined with the first weight, and the total score is the final score of the second video.

9. The AI-based video editing system according to claim 8, characterized in that: The optimization module includes: A grading unit, obtaining a grading rule, wherein the grading rule is used to measure the quality of the second video clip, and the grading rule is divided into five levels; An optimization unit obtains optimization methods corresponding to different levels, the optimization methods are used to improve the viewing quality of the second video, and sends the corresponding optimization methods to the terminal, the optimization methods include editing rhythm optimization, visual optimization, audio and video alignment, and shot connection optimization; The retest unit re-scores the optimized video to obtain a second score, and determines whether the optimization method is qualified according to the second score.

10. The AI-based video editing system according to claim 9, characterized in that: The editing features include shot switching patterns, rhythm, color style and composition.

Images