Video editing method and device, equipment and storage medium

By performing video editing on the server side and using deep learning and computer vision technology for intelligent identification and marking, the problem of traditional video editing software requiring professional knowledge and high hardware configuration is solved, and convenient and efficient video editing and diversified processing is achieved.

CN119011908APending Publication Date: 2024-11-22金霄
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411235544.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Traditional video editing software requires professional knowledge and high hardware configuration, lacks intelligent processing capabilities, and is difficult to meet users' convenient editing needs.

Method used

Implement video editing methods on the server side, providing intelligent editing functions through key information extraction, content description data recognition, dynamic compensation and marking processing, supporting user interaction and streaming processing, combining deep learning and computer vision technology for video content recognition and marking .

Benefits of technology

It improves the convenience and efficiency of video editing, reduces the workload of manual film review, realizes automatic search and association of video content, meets diverse video processing needs, improves creativity and effect, and frees up hardware configuration restrictions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011908A_ABST
    Figure CN119011908A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of video editing, and discloses a video editing method, device and equipment and a storage medium, and the method comprises the steps: obtaining a video file of a user side, carrying out the key information extraction of the video file, and obtaining video metadata; in response to a first operation instruction of a user side, identifying the video metadata to obtain content description data; performing dynamic compensation on the video content description data based on the video file to obtain transformation description data; in response to a second operation instruction of the user side, marking the video file based on the content description data and the transformation description data to obtain a plurality of marks, and generating mark description data according to the plurality of marks; and in response to a third operation instruction of the user side, editing the video file based on the mark description data to obtain an edited new video. According to the method, the video editing function is not limited by local hardware configuration any more, and the usability and flexibility are greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of video editing, and in particular relates to a video editing method, device, equipment and storage medium. Background Art

[0002] Traditional video editing software usually requires the installation of a dedicated software system on the local device, and has high requirements for computer hardware configuration. With the development of mobile Internet, various video media application scenarios have emerged, and more and more users want to try to record their work and life in the form of video and share them with colleagues, relatives and friends. However, video shooting skills are more complicated than traditional photography skills, and subsequent video editing requires more professional knowledge and related software usage skills.

[0003] Traditional video editing software usually requires users to have certain professional knowledge to operate, and lacks intelligent processing capabilities. Summary of the invention

[0004] The purpose of the present invention is to provide a video editing method, device, equipment and storage medium to solve the problem that the existing video editing lacks intelligent processing capability.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a video editing method, applied to a server, the method comprising: Obtain the video file on the user side, extract key information from the video file, and obtain video metadata; In response to a first operation instruction from the user terminal, the video metadata is identified to obtain content description data; Dynamically compensating the video content description data based on the video file to obtain transformation description data, wherein the transformation description data is used to characterize dynamic changes of the content description data; In response to a second operation instruction from the user terminal, the video file is marked based on the content description data and the transformation description data to obtain a plurality of marks, and the mark description data is generated according to the plurality of marks; In response to the third operation instruction of the user terminal, the video file is edited based on the tag description data to obtain an edited new video.

[0006] Preferably, the method further comprises: sending the edited new video to the user end.

[0007] Preferably, key information is extracted from the video file to obtain video metadata, including: Extract key information from a video file, the key information including at least: video content, length, frame rate, resolution, video frame and IBP frame data; Generate video metadata based on key information.

[0008] Preferably, the video metadata is identified to obtain content description data, including: Extracting video frames in key information according to frame rate to obtain several image key frames; Recognize a number of image key frames based on a preset recognition algorithm to obtain a number of image frame recognition results; The recognition results of several image frames are analyzed and processed to obtain content description data; wherein the analysis and processing at least includes: classification processing, clustering processing and merging processing.

[0009] Preferably, the video content description data is dynamically compensated based on the video file to obtain the transformed description data, including: Decode the video file to obtain a complete decoded file; Collecting dynamic values ​​between consecutive frames in the complete coded file, and identifying the transformation event of the video content description data according to the dynamic values ​​between consecutive frames in the complete coded file; The transformation description data is generated according to the transformation event of the video content description data.

[0010] Preferably, the editing process is user interactive processing or streaming processing, wherein the user interactive processing at least includes: frame clipping, color processing, special effect processing and advanced content processing.

[0011] Preferably, when the editing process is streaming process, the video file is edited based on the tag description data to obtain an edited new video, including: Automatic processing rules are constructed, the tag description data is referred to, and the video file is processed using the automatic processing rules to obtain an edited new video.

[0012] In a second aspect, the present invention provides a video editing device, the device is used to implement the above-mentioned video editing method, the device comprises: The information extraction module is used to obtain the video file of the user end, extract key information from the video file, and obtain video metadata; A data identification module, used to respond to a first operation instruction from a user terminal, identify video metadata, and obtain content description data; A transformation compensation module, used to dynamically compensate the video content description data based on the video file to obtain transformation description data, wherein the transformation description data is used to characterize the dynamic change of the content description data; A data marking module is used to respond to a second operation instruction of the user terminal, mark the video file based on the content description data and the transformation description data, obtain a plurality of marks, and generate mark description data according to the plurality of marks; The video editing module is used to respond to the third operation instruction of the user terminal, edit the video file based on the tag description data, and obtain an edited new video.

[0013] In a third aspect, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned video editing method when executing the computer program.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned video editing method when executed by a processor.

[0015] Beneficial effects: 1. The present invention performs video file editing operations on the server, so that the video editing function is no longer limited by the local hardware configuration, which greatly improves the usability and flexibility; 2. The present invention marks video files based on content description data and transformation description data, so as to facilitate automatic search and association of video content; 3. The present invention obtains content description data by identifying video metadata, and transforms the description data to characterize the dynamic changes of the content description data, so as to facilitate the convenience and efficiency of users in editing video files; 4. The present invention can conduct compliance review of video content through content description data and tag description data, and can also greatly reduce the workload of manual film review. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings: Figure 1 is a flow chart of a video editing method provided by one embodiment of the present invention; Figure 2 It is a block diagram of a video editing device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0018] Embodiment 1 Figure 1 This is a flowchart of a video editing method provided by an embodiment of the present invention. The application scenario of the video editing method of this embodiment is composed of a server and a user end. The video editing method is mainly run on the server, which is a cloud server. The user end can be a PC, a mobile phone, a tablet, a smart camera or other smart device. The user end uploads the video file to the server. The upload method includes the following three methods: 1. The user end connects to the server through a private network or the Internet (via the browser domain name), and accesses the server through the browser. After the upload action is triggered on the user end, the video file is selected on the smart device and uploaded through the browser; 2. Install client software on PCs, mobile phones, tablets and other devices. After the user establishes a communication connection with the server, the user can upload local video files to the server by triggering the client software, and the server will perform the editing operation of the video files; 3. Receive the video stream collected by the terminal device: This method is only for the scenario where the client software has been installed and the video image acquisition device, such as a digital camera, digital camera, etc., has been connected. Select the readable image acquisition device in the client software to trigger the acquisition function. The software will continuously read the real-time audio and video data stream from the image acquisition device and cache the data locally. After reaching the set data volume, the audio and video data will be packaged and uploaded to the server. This operation will continue until the user manually terminates it. The client software will notify the server to stop the acquisition and process and merge the uploaded audio and video data to generate a video file.

[0019] Based on the above application scenario, this embodiment provides a video editing method, such as Figure 1 As shown, the method includes: Step S10: The server obtains the video file from the user, extracts key information from the video file, and obtains video metadata.

[0020] In this embodiment, after receiving the video file uploaded by the user, the server saves the video file.

[0021] Specifically, key information is extracted from the video file to obtain video metadata, including: Step S101: extract key information from the video file, the key information at least including: video content, length, frame rate, resolution, video frame and IBP frame data; wherein the IBP frame data is I frame (Intra-coded frame, independent frame) data, B frame (Bidirectionally-predicted frame, bidirectionally predicted frame) data and P frame (Predicted frame, predicted frame) data.

[0022] Step S102: Generate video metadata according to key information.

[0023] In this embodiment, the generated video metadata is stored in the server, and the video metadata also includes data such as video content, length, frame rate, resolution, video frame and IBP frame data for subsequent processing.

[0024] Step S20: the server responds to the first operation instruction of the user terminal, identifies the video metadata, and obtains content description data.

[0025] In this embodiment, the first operation instruction is generated at the user end, and the first operation instruction is an identification operation for video metadata. After receiving the first operation instruction, the server identifies the video metadata.

[0026] Specifically, the video metadata is identified to obtain content description data, including: Step S201: extracting video frames in key information according to the frame rate to obtain a number of image key frames; Step S202: recognizing a number of image key frames based on a preset recognition algorithm to obtain a number of image frame recognition results; Step S203: analyzing and processing the recognition results of several image frames to obtain content description data; wherein the analysis and processing at least includes: classification processing, clustering processing and merging processing.

[0027] In step S201, the user can extract the image key frames in the video data stream according to the frame rate of the video and save them to the cloud frame buffer of the server. According to the user's selection operation, this process can be for the entire video file or part of the video content (according to the timeline range).

[0028] In step S202, the preset recognition algorithm may use a deep learning algorithm (Deep-Learning) and computer vision technology (Computer-Vision), and the preset recognition algorithm performs content recognition processing on the image frame data stored in the cloud frame buffer of the server, for example, the content of a certain frame of image is scenery or character, environment, brightness, character profile, subtitle, scene object list, etc. This processing continues until all image frames are recognized, and the system classifies, aggregates, and merges the recognition results to form content description data, which is stored in the data processing cache of the server for subsequent use.

[0029] Step S30: The server performs dynamic compensation on the video content description data based on the video file to obtain transformation description data, where the transformation description data is used to characterize dynamic changes of the content description data.

[0030] Specifically, the video content description data is dynamically compensated based on the video file to obtain the transformed description data, including: Step S301: Decode the video file to obtain a complete decoded file; Step S302: collecting dynamic values ​​between consecutive frames in the complete coded file, and identifying the transformation events of the video content description data according to the dynamic values ​​between consecutive frames in the complete coded file; Step S304: Generate transformation description data according to the transformation event of the video content description data, and superimpose the transformation description data on the corresponding image key frame.

[0031] In this embodiment, after completing the image recognition stage processing, the server will completely re-decode the video file. This step observes, collects and processes the dynamic numerical changes between consecutive frames, and can identify transformation events such as scene transitions, luminosity transformations, object displacements, human position transformations, subtitle transformations, etc., and generates the recognition results of the transformation events as transformation description data, which is stored in the server's data processing cache for subsequent use.

[0032] In this embodiment, the server may also organize the content description data and the transformation description data, convert them into a form that is easy for users to read, and display them on the cloud interface of the server for subsequent operations by users.

[0033] Step S40: the server responds to the second operation instruction of the user terminal, marks the video file based on the content description data and the transformation description data, obtains a plurality of marks, and generates mark description data according to the plurality of marks.

[0034] In this embodiment, the second operation instruction is also generated at the user end. The second operation instruction is a marking instruction. After receiving the second operation instruction, the server marks the video file.

[0035] In this embodiment, based on the above-mentioned content description data and transformation description data, the video file is intelligently marked, and the intelligent marking includes but is not limited to: character recognition, scene marking, object marking, spatial position, size dimension, color range and other markings, and the marking results are generated as objects of the video content to obtain marking description data.

[0036] In this embodiment, the video image can be displayed on the system interface of the server, and the tag description data can be superimposed on the video image. At this time, a visible tag can be displayed on the video image (i.e., the tag description data is visualized); this can facilitate the compliance review of the video content and greatly reduce the workload of manual film review; it can also facilitate automatic search and association of video content.

[0037] Step S50: the server responds to the third operation instruction of the user terminal, and edits the video file based on the tag description data to obtain an edited new video.

[0038] In this embodiment, the third operation instruction is also generated on the user side. The third operation instruction is a clipping instruction. After the server receives the third operation instruction, the video file is clipped.

[0039] In this embodiment, the editing process is user interaction processing or streaming processing, wherein the user interaction processing at least includes: frame clipping, color processing, special effect processing and advanced content processing.

[0040] Specifically, the user can refer to the visual tag description data to perform editing operations on the video file, for example: a) Frame editing: segmentation, sorting, scaling, rotation, etc.; b) Color processing: brightness adjustment, hue adjustment, image filter, etc.; c) Special effects processing: various transition effects, such as fade in and fade out, interlacing, dynamic in and out, etc.; d) Advanced content processing: This is a special feature and key function of the embodiment. Users can perform operations such as color adjustment, shadow processing, elimination, superposition, deformation, displacement, and replacement on the content that has been identified in the video (content description data and transformation description data); that is, independent processing of the target object in the two-dimensional video image.

[0041] As a further optimization of this embodiment, when the editing process is streaming process, the video file is edited based on the tag description data to obtain an edited new video, including: Automatic processing rules are constructed, the tag description data is referred to, and the video file is processed using the automatic processing rules to obtain an edited new video.

[0042] In this embodiment, the user can create automatic processing rules on the server, such as: erasing specified objects in the video content within a specified timeline range; after the user triggers automatic processing, the video content of the video file is processed according to the rules configured by the user. This operation can simplify the user's repetitive operations and greatly improve processing efficiency.

[0043] As a further optimization of this embodiment, the method further includes: sending the edited new video to the user end.

[0044] In this embodiment, the edited new video can be output to a designated location according to the user's selection, including but not limited to local storage, cloud storage, sharing link, etc.

[0045] The present invention has the following advantages: 1) Based on cloud computing technology, the video editing function is no longer limited by local hardware configuration, greatly improving usability and flexibility; 2) Combining deep learning and computer vision technology, it realizes intelligent recognition and tagging of video content, improving the convenience and efficiency of user operations; 3) Innovative intelligent tagging engine that can automatically search for video content and recommend related content; 4) It provides a wealth of video processing functions to meet users' diverse needs for video editing and improve the creativity and effects of video editing; 5) Based on content recognition and tagging, video content can be reviewed for compliance, greatly reducing the workload of manual film review.

[0046] Embodiment 2 Figure 2 is a block diagram of a video editing device provided by an embodiment of the present invention, such as Figure 2 As shown, this embodiment provides a video editing device, which is used to implement the video editing method of embodiment 1, and includes: The information extraction module is used to obtain the video file of the user end, extract key information from the video file, and obtain video metadata; A data identification module, used to respond to a first operation instruction from a user terminal, identify video metadata, and obtain content description data; A transformation compensation module, used to dynamically compensate the video content description data based on the video file to obtain transformation description data, wherein the transformation description data is used to characterize the dynamic change of the content description data; A data marking module is used to respond to a second operation instruction of the user terminal, mark the video file based on the content description data and the transformation description data, obtain a plurality of marks, and generate mark description data according to the plurality of marks; The video editing module is used to respond to the third operation instruction of the user terminal, edit the video file based on the tag description data, and obtain an edited new video.

[0047] This embodiment further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the video editing method of the first embodiment when executing the computer program.

[0048] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the video editing method of the first embodiment is implemented.

[0049] The present invention has the following advantages: 1) Based on cloud computing technology, the video editing function is no longer limited by local hardware configuration, greatly improving usability and flexibility; 2) Combining deep learning and computer vision technology, it realizes intelligent recognition and tagging of video content, improving the convenience and efficiency of user operations; 3) Innovative intelligent tagging engine that can automatically search for video content and recommend related content; 4) It provides a wealth of video processing functions to meet users' diverse needs for video editing and improve the creativity and effects of video editing; 5) Based on content recognition and tagging, video content can be reviewed for compliance, greatly reducing the workload of manual film review.

[0050] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0051] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0052] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A video editing method, applied to a server, characterized in that: The method comprises: Obtain the video file on the user side, extract key information from the video file, and obtain video metadata; In response to a first operation instruction from the user terminal, the video metadata is identified to obtain content description data; Dynamically compensating the video content description data based on the video file to obtain transformation description data, wherein the transformation description data is used to characterize dynamic changes of the content description data; In response to a second operation instruction from the user terminal, the video file is marked based on the content description data and the transformation description data to obtain a plurality of marks, and the mark description data is generated according to the plurality of marks; In response to the third operation instruction of the user terminal, the video file is edited based on the tag description data to obtain an edited new video.

2. The video editing method according to claim 1, characterized in that: The method also includes: sending the edited new video to the user end.

3. The video editing method according to claim 1, characterized in that: Extract key information from video files to obtain video metadata, including: Extract key information from a video file, the key information including at least: video content, length, frame rate, resolution, video frame and IBP frame data; Generate video metadata based on key information.

4. The video editing method according to claim 3, characterized in that: Identify video metadata to obtain content description data, including: Extracting video frames in key information according to frame rate to obtain several image key frames; Recognize a number of image key frames based on a preset recognition algorithm to obtain a number of image frame recognition results; The recognition results of several image frames are analyzed and processed to obtain content description data; wherein the analysis and processing at least includes: classification processing, clustering processing and merging processing.

5. The video editing method according to claim 1, characterized in that: The video content description data is dynamically compensated based on the video file to obtain the transformed description data, including: Decode the video file to obtain a complete decoded file; Collecting dynamic values ​​between consecutive frames in the complete coded file, and identifying the transformation event of the video content description data according to the dynamic values ​​between consecutive frames in the complete coded file; The transformation description data is generated according to the transformation event of the video content description data.

6. The video editing method according to claim 1, characterized in that: The editing process is user interaction processing or streaming processing, wherein the user interaction processing at least includes: frame clipping, color processing, special effect processing and advanced content processing.

7. The video editing method according to claim 6, characterized in that: When the editing process is a streaming process, the video file is edited based on the tag description data to obtain an edited new video, including: Automatic processing rules are constructed, the tag description data is referred to, and the video file is processed using the automatic processing rules to obtain an edited new video.

8. A video editing device, the device being used to implement the video editing method according to any one of claims 1 to 7, characterized in that: The device comprises: The information extraction module is used to obtain the video file of the user end, extract key information from the video file, and obtain video metadata; A data identification module, used to respond to a first operation instruction from a user terminal, identify video metadata, and obtain content description data; A transformation compensation module, used to dynamically compensate the video content description data based on the video file to obtain transformation description data, wherein the transformation description data is used to characterize the dynamic change of the content description data; A data marking module is used to respond to a second operation instruction of the user terminal, mark the video file based on the content description data and the transformation description data, obtain a plurality of marks, and generate mark description data according to the plurality of marks; The video editing module is used to respond to the third operation instruction of the user terminal, edit the video file based on the tag description data, and obtain an edited new video.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the video editing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the video editing method described in any one of claims 1 to 7 is implemented.