Video clip method, apparatus, device, storage medium, computer program product

Abstract

This application discloses a video editing method, apparatus, device, storage medium, and computer program product to address the problems of existing video editing solutions that rely heavily on manual operation, resulting in low editing efficiency, inability to handle the real-time editing needs of massive live videos, and highly subjective and inconsistent editing output. The method includes: sequentially extracting semantic features from each candidate video segment in an acquired set of candidate video segments to obtain a first semantic feature corresponding to each candidate video segment; determining a second semantic feature corresponding to the acquired editing requirements; selecting an editable video segment from the set of candidate video segments based on the feature similarity between the second semantic feature and the first semantic feature; and performing video editing based on the editable video segment.

Description

Technical Field

[0001] This application relates to the field of wireless communication technology, and in particular to a video editing method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] With the rapid development of the online live streaming industry, live video content has exploded, covering multiple fields such as gaming, education, and entertainment. To make live content easier to watch, share, and create derivative works, editing live videos to extract highlights or segments that meet specific needs has become a common market demand.

[0003] Currently, live video editing primarily relies on manual operation. Specifically, editors need to watch lengthy video streams, manually identify, select, and edit the required segments. This approach has significant limitations: First, manual processing is inefficient and cannot handle the real-time editing needs of massive amounts of live video. Second, editing quality is highly dependent on the editor's personal experience, judgment, and preferences, resulting in highly subjective outputs and a lack of objective consistency. Furthermore, entirely manual operation struggles to accurately understand and match the complex and ever-changing personalized editing needs expressed by users through natural language.

[0004] Therefore, how to accurately understand users' personalized editing needs while achieving efficient and automated video editing has become a pressing technical problem to be solved in this field. Summary of the Invention

[0005] This application provides a video editing method to address the problems of existing video editing solutions that mainly rely on manual operation, have low editing efficiency, cannot meet the real-time editing needs of massive live videos, and have highly subjective and inconsistent editing output results.

[0006] This application also provides a video editing device to solve the problems of existing video editing solutions that mainly rely on manual operation, have low editing efficiency, cannot meet the real-time editing needs of massive live videos, and have highly subjective and inconsistent editing output results.

[0007] This application also provides a video editing device to solve the problems of existing video editing solutions that mainly rely on manual operation, have low editing efficiency, cannot meet the real-time editing needs of massive live videos, and have highly subjective and inconsistent editing output results.

[0008] This application also provides a computer-readable storage medium to address the problems of existing video editing solutions that mainly rely on manual operation, have low editing efficiency, cannot meet the real-time editing needs of massive live videos, and have highly subjective and inconsistent editing output results.

[0009] A computer program product is provided to address the problems of existing video editing solutions that rely mainly on manual operation, have low editing efficiency, cannot meet the real-time editing needs of massive live videos, and have highly subjective and inconsistent editing output results.

[0010] The embodiments of this application adopt the following technical solutions: A video editing method includes: sequentially extracting semantic features from each candidate video segment in an acquired set of candidate video segments to obtain a first semantic feature corresponding to each candidate video segment; determining a second semantic feature corresponding to the acquired editing requirements; selecting an editable video segment from the set of candidate video segments based on the feature similarity between the second semantic feature and the first semantic feature; and performing video editing based on the editable video segment.

[0011] A video editing apparatus includes: a video analysis unit, configured to sequentially extract semantic features from each candidate video segment in an acquired set of candidate video segments to obtain a first semantic feature corresponding to each candidate video segment; a demand analysis unit, configured to determine a second semantic feature corresponding to the acquired editing demand; and a demand matching unit, configured to select an editable video segment from the set of candidate video segments based on the feature similarity between the second semantic feature and the first semantic feature, and to perform video editing based on the editable video segment.

[0012] A video editing device, comprising: The processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the following operations: sequentially extracting semantic features from each candidate video segment in an acquired set of candidate video segments to obtain a first semantic feature corresponding to each candidate video segment; determining a second semantic feature corresponding to the acquired editing requirements; selecting an editable video segment from the set of candidate video segments based on the feature similarity between the second semantic feature and the first semantic feature, and performing video editing based on the editable video segment.

[0013] A computer-readable storage medium stores one or more programs that, when executed by an electronic device including multiple applications, cause the electronic device to perform the following operations: sequentially extracting semantic features from each candidate video segment in an acquired set of candidate video segments to obtain a first semantic feature corresponding to each candidate video segment; determining a second semantic feature corresponding to an acquired editing requirement; selecting an editable video segment from the set of candidate video segments based on the feature similarity between the second semantic feature and the first semantic feature; and performing video editing based on the editable video segment.

[0014] A computer program product includes a computer program that, when executed by a processor, performs the following: sequentially extracting semantic features from each candidate video segment in an acquired set of candidate video segments to obtain a first semantic feature corresponding to each candidate video segment; determining a second semantic feature corresponding to the acquired editing requirements; selecting an editable video segment from the set of candidate video segments based on the feature similarity between the second semantic feature and the first semantic feature; and performing video editing based on the editable video segment.

[0015] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: Using the video editing method provided in this application, when video editing is required, candidate video segments can first be obtained from the video stream by randomly sampling frames. Semantic features are extracted from each candidate video segment in the obtained candidate video segment set to obtain the first semantic feature corresponding to each candidate video segment. At the same time, the second semantic feature corresponding to the editing requirements can be determined according to the obtained editing requirements. Then, based on the feature similarity between the second semantic feature and the first semantic feature, the video segment to be edited is selected from the candidate video segment set, and the video editing is completed according to the video segment to be edited. The video editing method provided in this application has several advantages. First, by extracting semantic features from candidate video segments and matching their similarity with the semantic features of the editing requirements, the selection and editing of video segments can be completed quickly, significantly reducing the time and effort required for manual browsing, filtering, and editing of video content, and greatly improving the overall efficiency of video editing. Second, the method provided in this application can abstract editing requirements into second semantic features and video content into first semantic features. By measuring similarity within a unified semantic space, it can deeply understand the underlying semantics of user needs and accurately match the core meaning of video content, thereby more accurately selecting video segments that truly meet user intent. Finally, since the method provided in this application is based on objective semantic feature similarity calculation for video filtering and editing, the editing results are not affected by the personal subjectivity of the editor, improving the reliability and standardization of the editing results. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram illustrating a specific process of a video editing method provided in an embodiment of this application; Figure 2 A schematic diagram of the specific structure of a video editing device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the specific structure of a video editing device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] This application provides a video editing method to address the problems of existing video editing solutions that mainly rely on manual operation, have low editing efficiency, cannot meet the real-time editing needs of massive live videos, and have highly subjective and inconsistent editing output results.

[0019] The execution subject of the video editing method provided in this application embodiment may be, but is not limited to, at least one of a video server, a live streaming server, and an editing server; in addition, the execution subject of the method may also be the system or application (APP) itself running on these servers.

[0020] For ease of description, the following description uses a video editing system as the execution subject to illustrate the implementation of this method. It should be understood that using a video editing system as the execution subject is merely an illustrative example and should not be construed as a limitation of the method.

[0021] The specific implementation flowchart of the video editing method provided in this application is shown in the figure below. Figure 1 As shown, the main steps include the following: Step 11: Semantic features are extracted sequentially from each candidate video segment in the obtained candidate video segment set to obtain the first semantic feature corresponding to each candidate video segment; In one implementation, the video editing system can acquire real-time or offline live video streams from a live streaming platform or video streaming server, and perform frame extraction on the acquired live video streams to construct various candidate video segments.

[0022] It should be noted that, in order to improve processing efficiency while ensuring content coverage, in one implementation, the video editing system can employ a random frame extraction strategy to acquire candidate video segments. For example, the video editing system can extract a series of image frames from the video stream at random time intervals, and combine several consecutive or adjacent frames into a candidate video segment, thereby reducing video segment redundancy and increasing content diversity.

[0023] After obtaining the set of candidate video clips through the above method, the video editing system can sequentially obtain the first semantic features corresponding to each candidate video clip according to the following sub-steps, which may include: Sub-step 1101: Perform image content recognition on the image frames in each candidate video segment to determine the text description sequence corresponding to the candidate video segment; Specifically, for each candidate video segment, the image frames contained in each candidate video segment can be sequentially input into a pre-built image-to-text converter. This image-to-text converter can employ a deep learning model based on an encoder-decoder structure. For example, the encoder can use a convolutional neural network (CNN) to extract the visual features of the image frames, and the decoder can use a long short-term memory network (LSTM) or Transformer model architecture to generate a natural language text describing the content of the image frame based on the encoded visual features. Through the above operations, a candidate video segment can be converted into a text description sequence, where each element corresponds to the text description of an image frame.

[0024] Sub-step 1102 involves performing contextual semantic encoding on the text description sequence obtained by executing the above sub-step 1101 to obtain the first semantic feature used to characterize the semantic content corresponding to the candidate video segment.

[0025] Because videos are sequential and the content between frames is semantically related, in order to capture the contextual information between video frames, in this embodiment of the application, the video editing system can encode the acquired text description sequence according to the following process: Process 1, Semantic Embedding: Using a pre-trained text semantic embedding matrix (e.g., embedding layers of Word2Vec, GloVe, or BERT), each sentence of the text description sequence is vectorized, mapping each word or subword to a high-dimensional vector. The semantic embedding vector for each sentence is then obtained through pooling or a sequence model. Through these operations, the entire candidate video segment can be represented as a sequence of semantic embedding vectors.

[0026] Process 2, Context Encoding: The semantic embedding vector sequence obtained through the above process is input into the context semantic encoder. In one implementation, the context semantic encoder can be a bidirectional long short-term memory (Bi-LSTM) network.

[0027] Bi-LSTM can scan the semantic embedding vector sequence in both forward and backward directions, fully capturing the long-distance dependencies between each descriptive text and its surrounding context. Finally, the hidden state of the last time step of Bi-LSTM, or the vector obtained by pooling the hidden states of all time steps, can be used as the first semantic feature representing the overall semantic content of the candidate video segment.

[0028] Step 12: Based on the obtained editing requirements, determine the second semantic features corresponding to the editing requirements; In this embodiment, users can submit editing requests to the video editing system through graphical interface input boxes, voice input to text, or other methods. These requests are typically expressed in natural language text, such as "find all goal moments" or "edit out the segments where the lecturer explains the core concepts."

[0029] It should be noted that since the first semantic feature corresponding to the candidate video segment and the editing request input by the user belong to different modalities, this modal difference may lead to significant differences in their respective feature representations and feature spaces, resulting in a "semantic gap." This makes it difficult to directly match and analyze video features with text features. Therefore, in order to achieve cross-modal content understanding and retrieval, in this embodiment, the obtained editing request can be encoded as a feature vector that resides in the same semantic space as the first semantic feature.

[0030] In one implementation, the video editing system can input editing requirements into a semantic encoder. It should be noted that, in order to fully understand the complex semantics and long-range dependencies in the text, one implementation employs a Transformer-based semantic encoder to encode the editing requirements. Utilizing the Transformer's self-attention mechanism can effectively capture global relationships between words in the text and uncover deeper semantics. In this embodiment, the encoder's output can be used as a second semantic feature representing the core semantics of the user's editing requirements.

[0031] Through the above encoding process, the first semantic features corresponding to the candidate video segments and the editing requirements can be in the same modality, so as to more accurately understand the correspondence between the editing requirements expressed by the user through text and the video content.

[0032] Step 13: Based on the feature similarity between the second semantic feature and the first semantic feature obtained by performing the above steps, select a video clip from the candidate video clip set and perform video editing based on the video clip.

[0033] In this embodiment of the application, the video editing system can interactively match the second semantic feature with the first semantic feature to determine the feature similarity between the second semantic feature and the first semantic feature, and then select the video clip to be edited from the candidate video clip set based on the feature similarity.

[0034] Specifically, in one implementation, the video editing system may interactively match the second semantic feature with the first semantic feature according to the following sub-steps: Sub-step 1301: Input the second semantic feature and the first semantic feature into the feature-decoupled interactive matching network; The second semantic feature and the first semantic feature are input into an interaction matching network based on feature decoupling.

[0035] In this embodiment, the interactive matching network is designed as a deep learning sub-network for calculating the deep similarity between two high-dimensional semantic feature vectors. Its overall architecture adopts an encoding-interaction-aggregation paradigm, mainly comprising the following sequentially connected logical modules: Feature decoupling module: used to decompose the input global feature vector into a series of representative local feature vectors.

[0036] Local interaction calculation module: used to calculate fine-grained associations between corresponding local feature pairs from two features.

[0037] Attention weight generation and adjustment module: used to evaluate the importance of each local interaction result to the final matching decision, and to regularize it through random deactivation technique.

[0038] Weighted aggregation module: used to integrate all local interaction information and generate a global matching degree representation vector.

[0039] The interactive matching network takes the first and second semantic features output from steps 11 and 12 above as input, and finally outputs a global matching degree representation vector, which is then used by the subsequent classifier for editing decisions.

[0040] Sub-step 1302: The second semantic feature and the first semantic feature are decoupled by the interactive matching network to obtain the demand local feature sequence corresponding to the second semantic feature and the video local feature sequence corresponding to the first semantic feature. The interactive matching network first decouples the two feature vectors, decomposing them into a sequence of multiple local feature vectors.

[0041] For example, the first and second semantic features are fed into two independent but structurally identical feature decoupling layers. Each decoupling layer is typically implemented by a set of one-dimensional convolutional kernels or a fully connected layer. Its function is to "cut" a high-dimensional feature vector into multiple lower-dimensional but more targeted local feature vectors, thereby obtaining the desired local feature sequence {d1, d2, ..., d...} after decoupling. N} and the video local feature sequence {v1, v2, ..., v N}, where N is the number of local features.

[0042] Sub-step 1303: Based on the required local feature sequence and the video local feature sequence, determine the local interaction vector sequence of the second semantic feature and the first semantic feature; Specifically, for each group of local features (d) in the sequence i v i The local semantic association interaction vector between them is calculated using the following formula [1]: [1] Where Concat represents vector concatenation, +, -, and ⊙ represent point-by-point addition, point-by-point subtraction, and point-by-point multiplication, respectively; Conv1D represents one-dimensional convolution operation; and Pooling represents pooling operation. By traversing all local features, a local interaction vector sequence {L1, L2, ..., L...} is obtained. N}

[0043] Sub-step 1304: Perform a weighted summation on the local interaction vector sequence obtained by executing sub-step 1303 to obtain the feature similarity between the second semantic feature and the first semantic feature; In this embodiment, the video editing system can calculate the attention weight of each local interaction vector in the local interaction vector sequence according to the following formula [2]: [2] in, W and b represent the learnable weight matrix and bias vector, tanh is the hyperbolic tangent activation function, and u represents the learnable attention scoring reference vector. To enhance the model's generalization ability and prevent overfitting, the calculated original attention weight sequence {a1, a2, ..., a...} can be... N} Perform random deactivation to obtain the processed weights {a} ， 1, a ， 2, ..., a ， N The local interaction vector sequences are weighted and summed to obtain a comprehensive representation of the feature similarity between the second semantic feature and the first semantic feature.

[0044] After determining the feature similarity between the second semantic feature and the first semantic feature through the above steps, the video editing system can select candidate video segments from the candidate video segment set whose feature similarity with the second semantic feature is higher than a preset threshold, and use them as video segments for editing, and then perform video editing based on the selected video segments.

[0045] Specifically, in one implementation, the video editing system can input the feature similarity obtained in the previous step into a cutting result generator, which can be composed of fully connected layers and a Softmax function. The cutting result generator can determine whether the candidate video segment meets the editing requirements based on the feature similarity and output a decision result, such as a probability value or a "yes / no" binary classification label. If the decision is "yes", the candidate video segment is selected as the edited video segment.

[0046] Finally, perform video editing operations based on one or more selected video clips. This can include directly outputting these clips or splicing them together chronologically into a new highlight video file.

[0047] Using the video editing method provided in this application, when video editing is required, candidate video segments can first be obtained from the video stream by randomly sampling frames. Semantic features are extracted from each candidate video segment in the obtained candidate video segment set to obtain the first semantic feature corresponding to each candidate video segment. At the same time, the second semantic feature corresponding to the editing requirements can be determined according to the obtained editing requirements. Then, based on the feature similarity between the second semantic feature and the first semantic feature, the video segment to be edited is selected from the candidate video segment set, and the video editing is completed according to the video segment to be edited. The video editing method provided in this application has several advantages. First, by extracting semantic features from candidate video segments and matching their similarity with the semantic features of the editing requirements, the selection and editing of video segments can be completed quickly, significantly reducing the time and effort required for manual browsing, filtering, and editing of video content, and greatly improving the overall efficiency of video editing. Second, the method provided in this application can abstract editing requirements into second semantic features and video content into first semantic features. By measuring similarity within a unified semantic space, it can deeply understand the underlying semantics of user needs and accurately match the core meaning of video content, thereby more accurately selecting video segments that truly meet user intent. Finally, since the method provided in this application is based on objective semantic feature similarity calculation for video filtering and editing, the editing results are not affected by the personal subjectivity of the editor, improving the reliability and standardization of the editing results.

[0048] In one embodiment, this application also provides a video editing device to address the problems of existing video editing solutions relying primarily on manual operation, resulting in low editing efficiency, inability to meet the real-time editing needs of massive amounts of live video, and highly subjective and inconsistent editing output. A schematic diagram of the specific structure of this video editing device is shown below. Figure 2 As shown, it includes: video analysis unit 21, demand analysis unit 22 and demand matching unit 23.

[0049] The video analysis unit 21 is used to extract semantic features from each candidate video segment in the acquired set of candidate video segments in turn, so as to obtain the first semantic features corresponding to each candidate video segment. The requirement analysis unit 22 is used to determine the second semantic feature corresponding to the obtained editing requirements; The demand matching unit 23 is used to select a video clip from the candidate video clip set based on the feature similarity between the second semantic feature and the first semantic feature, and to perform video editing based on the video clip.

[0050] In one embodiment, the video analysis unit 21 is specifically configured to: perform image content recognition on image frames in each of the candidate video segments to determine the text description sequence corresponding to the candidate video segments; and perform contextual semantic encoding on the text description sequence to obtain the first semantic feature used to characterize the semantic content corresponding to the candidate video segments.

[0051] In one embodiment, the video analysis unit 21 is specifically used to: perform semantic embedding encoding on each descriptive text in the text description sequence to obtain a semantic embedding vector sequence corresponding to the text description sequence; and encode the semantic embedding vector sequence based on a bidirectional long short-term memory network to obtain the first semantic feature.

[0052] In one implementation, the requirement analysis unit 22 is specifically used to: encode the text format clipping requirement description through a semantic encoder to obtain the second semantic feature.

[0053] In one implementation, the demand matching unit 23 is specifically used to: interactively match the second semantic feature with the first semantic feature to determine the feature similarity between the second semantic feature and the first semantic feature; and select edited video segments from the candidate video segment set based on the feature similarity.

[0054] In one implementation, the demand matching unit 23 is specifically configured to: input the second semantic feature and the first semantic feature into an interaction matching network based on feature decoupling; decouple the second semantic feature and the first semantic feature through the interaction matching network to obtain a demand local feature sequence corresponding to the second semantic feature and a video local feature sequence corresponding to the first semantic feature; determine a local interaction vector sequence between the second semantic feature and the first semantic feature based on the demand local feature sequence and the video local feature sequence; and perform a weighted summation on the local interaction vector sequence to obtain the feature similarity between the second semantic feature and the first semantic feature.

[0055] Using the video editing apparatus provided in this application embodiment, when video editing is required, candidate video segments can first be obtained from the video stream by randomly sampling frames. Semantic features are extracted from each candidate video segment in the obtained candidate video segment set to obtain the first semantic feature corresponding to each candidate video segment. At the same time, the second semantic feature corresponding to the editing requirements can be determined according to the obtained editing requirements. Then, based on the feature similarity between the second semantic feature and the first semantic feature, the video segment to be edited is selected from the candidate video segment set, and the video editing is completed according to the video segment to be edited. The video editing apparatus provided in this application embodiment, on the one hand, extracts semantic features from candidate video segments and matches them with the semantic features of editing requirements based on similarity, thereby quickly completing the selection and editing of segments. This significantly reduces the time and effort required for manual browsing, filtering, and editing of video content, greatly improving the overall efficiency of video editing. On the other hand, the method provided in this application embodiment can abstract editing requirements into second semantic features and video content into first semantic features. By measuring similarity within a unified semantic space, it can deeply understand the underlying semantics of user needs and accurately match the core meaning of video content, thus more accurately selecting video segments that truly meet user intent. Finally, since the method provided in this application embodiment is based on objective semantic feature similarity calculation for video filtering and editing, the editing results are not affected by the personal subjectivity of the editor, improving the reliability and standardization of the editing results.

[0056] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0057] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0058] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0059] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a video editing device at the logical level. The processor executes the program stored in memory and specifically performs the following operations: Semantic features are extracted sequentially from each candidate video segment in the acquired set of candidate video segments to obtain a first semantic feature corresponding to each candidate video segment; a second semantic feature corresponding to the acquired editing requirements is determined; an editable video segment is selected from the set of candidate video segments based on the feature similarity between the second semantic feature and the first semantic feature, and video editing is performed based on the editable video segment.

[0060] The above is as stated in this application. Figure 3The video editing electronic device method disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0061] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0062] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The video editing method shown in the embodiment is specifically used to perform the following operations: Semantic features are extracted sequentially from each candidate video segment in the acquired set of candidate video segments to obtain a first semantic feature corresponding to each candidate video segment; a second semantic feature corresponding to the acquired editing requirements is determined; an editable video segment is selected from the set of candidate video segments based on the feature similarity between the second semantic feature and the first semantic feature, and video editing is performed based on the editable video segment.

[0063] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0068] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0069] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0070] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0071] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A video editing method, characterized in that, include: Semantic features are extracted sequentially from each candidate video segment in the acquired set of candidate video segments to obtain the first semantic feature corresponding to each candidate video segment; Based on the obtained editing requirements, determine the second semantic feature corresponding to the editing requirements; Based on the feature similarity between the second semantic feature and the first semantic feature, a video clip is selected from the candidate video clip set, and video editing is performed based on the video clip.

2. The method according to claim 1, characterized in that, The step of sequentially extracting semantic features from each candidate video segment in the acquired set of candidate video segments to obtain the first semantic feature corresponding to each candidate video segment specifically includes: Image content recognition is performed on the image frames in each of the candidate video segments to determine the text description sequence corresponding to the candidate video segments; The text description sequence is subjected to contextual semantic encoding to obtain the first semantic feature used to characterize the semantic content corresponding to the candidate video segment.

3. The method according to claim 2, characterized in that, The context semantic encoding of the text description sequence specifically includes: Semantic embedding encoding is performed on each descriptive text in the text description sequence to obtain a semantic embedding vector sequence corresponding to the text description sequence; The semantic embedding vector sequence is encoded using a bidirectional long short-term memory network to obtain the first semantic feature.

4. The method according to claim 1, characterized in that, The obtained editing requirements include a text-formatted description of the editing requirements; The step of determining the second semantic feature corresponding to the obtained editing requirements specifically includes: The second semantic feature is obtained by encoding the text format editing requirement description through a semantic encoder.

5. The method according to claim 1, characterized in that, The step of selecting editable video segments from the candidate video segment set based on the feature similarity between the second semantic feature and the first semantic feature includes: Interactive matching is performed between the second semantic feature and the first semantic feature to determine the feature similarity between the second semantic feature and the first semantic feature; Based on the feature similarity, a video clip is selected from the set of candidate video clips for editing.

6. The method according to claim 5, characterized in that, The interactive matching of the second semantic feature and the first semantic feature to determine the feature similarity between the second semantic feature and the first semantic feature specifically includes: The second semantic feature and the first semantic feature are input into an interaction matching network based on feature decoupling; The interactive matching network is used to decouple the second semantic feature and the first semantic feature respectively to obtain the demand local feature sequence corresponding to the second semantic feature and the video local feature sequence corresponding to the first semantic feature. Based on the required local feature sequence and the video local feature sequence, determine the local interaction vector sequence of the second semantic feature and the first semantic feature; The local interaction vector sequence is weighted and summed to obtain the feature similarity between the second semantic feature and the first semantic feature.

7. A video editing device, characterized in that, include: The video analysis unit is used to extract semantic features from each candidate video segment in the acquired set of candidate video segments in turn, so as to obtain the first semantic feature corresponding to each candidate video segment. The requirement analysis unit is used to determine the second semantic feature corresponding to the obtained editing requirements; The demand matching unit is used to select editable video segments from the candidate video segment set based on the feature similarity between the second semantic feature and the first semantic feature, and to perform video editing based on the editable video segments.

8. A video editing device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the following operations: Semantic features are extracted sequentially from each candidate video segment in the acquired set of candidate video segments to obtain the first semantic feature corresponding to each candidate video segment; Based on the obtained editing requirements, determine the second semantic feature corresponding to the editing requirements; Based on the feature similarity between the second semantic feature and the first semantic feature, a video clip is selected from the candidate video clip set, and video editing is performed based on the video clip.

9. A computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the video editing method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the video editing method as described in any one of claims 1-6.

Images