Video processing method and apparatus

Abstract

The application provides a video processing method, system and device, wherein the video processing method comprises: acquiring a to-be-processed video and description information corresponding to the to-be-processed video; inputting the to-be-processed video into a recognition model for processing to obtain a target video containing a timestamp; processing the target video based on the timestamp to obtain at least one video segment and determining a video label set according to the description information; constructing a video feature corresponding to each video segment and selecting a target video label corresponding to each video segment in the video label set according to the video feature, so that the to-be-processed video is processed in segments, the waste of video processing resources is reduced, the system computing pressure is reduced, the video is automatically processed, the labor cost is reduced, and the video processing is efficiently and accurately completed.

Description

Technical Field

[0001] This application relates to the field of video editing technology, and in particular to a video processing method. This application also relates to a video processing apparatus, a video processing system, a computing device, and a computer-readable storage medium. Background Technology

[0002] With the development of internet technology, video files carry increasingly larger amounts of information, and video quality is constantly improving. Consequently, processing these massive video files consumes more and more computing resources and slows down the processing speed. Current video processing methods still largely rely on manual processing. With the increasing proportion of long videos, the cost of manual labor becomes unavoidable. Therefore, automated video processing has emerged. However, existing automated video processing methods still cannot solve the problem of the large amount of storage and computing resources consumed during the processing of long videos. Furthermore, current video editing methods primarily process video frames, neglecting the information carried in audio files. Therefore, a technological solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this application provide a video processing method to address the technical deficiencies existing in the prior art. Embodiments of this application also provide a video processing apparatus, a video processing system, a computing device, and a computer-readable storage medium.

[0004] According to a first aspect of the embodiments of this application, a video processing method is provided, comprising:

[0005] Obtain the video to be processed and its corresponding description information;

[0006] The video to be processed is input into the recognition model for processing to obtain the target video containing timestamps;

[0007] The target video is processed based on the timestamp to obtain at least one video segment, and a video tag set is determined based on the description information;

[0008] Construct video features corresponding to each video segment, and select target video tags corresponding to each video segment from the video tag set based on the video features.

[0009] Optionally, processing the target video based on the timestamp to obtain at least one video segment includes:

[0010] The target video is split into a sequence of image frames and an audio file;

[0011] Based on the timestamps, the image frame sequence and the audio file are segmented to obtain target image frame sequence segments and target audio segments;

[0012] The target image frame sequence fragments and the target audio fragments are combined according to the time correspondence to obtain at least one video fragment.

[0013] Optionally, the step of segmenting the image frame sequence and the audio file based on the timestamp to obtain the target image frame sequence segment and the target audio segment includes:

[0014] Determine the timestamp position on the timeline of the audio file;

[0015] Based on the timestamp position, the audio file is divided, and the audio segment containing audio information in the division result is selected as the target audio segment;

[0016] Image frame sequence segments are extracted from the image frame sequence according to the time interval of the target audio segment.

[0017] Optionally, determining the video tag set based on the description information includes:

[0018] The descriptive information is subjected to text recognition to obtain multiple word units corresponding to the descriptive information;

[0019] Based on preset filtering rules, at least one keyword is determined from the multiple word units, and a video tag set is created based on the at least one keyword.

[0020] Optionally, it also includes:

[0021] Receive user instructions and determine scene parameters based on the user instructions;

[0022] Filter out the video segments that correspond to the scene parameters from the at least one video segment, and create a video segment set;

[0023] Accordingly, video features corresponding to each video segment are constructed, including:

[0024] Construct video features corresponding to each video segment in the video segment set.

[0025] Optionally, constructing video features corresponding to each video segment and selecting target video tags corresponding to each video segment from the video tag set based on the video features includes:

[0026] Each video segment is split into sub-image frame sequences and sub-audio segments;

[0027] Determine the audio features corresponding to the sub-audio segment and / or the image features corresponding to the sub-image frame sequence;

[0028] Based on the audio features and / or the image features, target video tags corresponding to each video segment are selected from the video tag set.

[0029] Optionally, it also includes:

[0030] Iterate through the target video tags corresponding to each video segment, and delete the video segments with empty target video tags based on the traversal results to obtain the target video segment set.

[0031] The video segments in the target video segment set are classified according to the target video tags to obtain the target video set corresponding to the target video tags;

[0032] The video segments contained in the target video set are spliced ​​together to obtain the spliced ​​video corresponding to the target video tag.

[0033] Optionally, the training of the recognition model is determined in the following manner:

[0034] Obtain video samples and target video samples containing sample timestamps corresponding to the video samples;

[0035] The video sample is input into the initial recognition model for processing to obtain the predicted video containing the prediction timestamp output by the initial recognition model;

[0036] The initial recognition model is tuned based on the predicted video and the target video sample until the recognition model that meets the training stopping condition is obtained.

[0037] Optionally, obtaining the target video sample containing the sample timestamp corresponding to the video sample includes:

[0038] Extract the sample image frame information and sample audio information of the video sample;

[0039] The sample timestamp position is determined in the sample image frame information or the sample audio information according to the preset recognition feature list;

[0040] The sample timestamp position is mapped to the timeline of the video sample to obtain the target video sample containing the sample timestamp.

[0041] According to a second aspect of the embodiments of this application, a video processing system is provided, comprising:

[0042] Client and business side;

[0043] The client is configured to: acquire a video to be processed and a description of the video to be processed; input the video to be processed into a recognition model for processing to obtain a target video containing a timestamp; process the target video based on the timestamp to obtain at least one video segment; and upload the at least one video segment and the description information to the service server.

[0044] The service server is configured to determine a video tag set based on the description information; construct video features corresponding to each video segment; and select target video tags corresponding to each video segment in the video tag set based on the video features.

[0045] Optionally, the client is further configured to determine the video duration of each video segment and the interval between each video segment; determine an upload strategy based on the video duration and the interval; and upload the video segments to the business server based on the upload strategy.

[0046] Optionally, the client is further configured to determine a duration threshold and an interval threshold; upload a video segment if the video duration is greater than the duration threshold; and upload a video segment after splicing it together if the video duration is less than the duration threshold and the interval time is less than the interval threshold.

[0047] According to a third aspect of the embodiments of this application, a video processing apparatus is provided, comprising:

[0048] The acquisition module is configured to acquire the video to be processed and the description information corresponding to the video to be processed.

[0049] The processing module is configured to input the video to be processed into the recognition model for processing to obtain the target video containing the timestamp;

[0050] The determination module is configured to process the target video based on the timestamp to obtain at least one video segment, and to determine a video tag set based on the description information;

[0051] A selection module is configured to construct video features corresponding to each video segment and select target video tags corresponding to each video segment from the video tag set based on the video features. According to a fourth aspect of the embodiments of this application, a computing device is provided, comprising:

[0052] Memory and processor;

[0053] The memory is used to store computer-executable instructions, and the processor executes the computer-executable instructions to implement the steps of the video processing method.

[0054] According to a fifth aspect of the present application, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the video processing method.

[0055] The video processing method provided in this application obtains the video to be processed and its corresponding descriptive information; determines a video tag set based on the descriptive information, inputs the video to be processed into a recognition model for recognition, timestamps the positions in the video to be processed that meet the recognition conditions, and obtains the target video; based on the timestamp of the target video, at least one video segment is extracted from the target video; finally, according to the attribute information of each video segment, a corresponding target video tag is configured for the video segment from the video tag set. This enables segmented processing of the video to be processed, reduces waste of video processing resources, alleviates the computational pressure on the system, automates video processing, reduces manual costs, and can complete video processing efficiently and accurately. Attached Figure Description

[0056] Figure 1 This is a flowchart of a video processing method provided in an embodiment of this application;

[0057] Figure 2 This is a schematic diagram of a video processing system provided in an embodiment of this application;

[0058] Figure 3 This is a flowchart illustrating a video processing method applied to open course video processing, provided in one embodiment of this application.

[0059] Figure 4 This is a schematic diagram of an image frame of a target video provided in an embodiment of this application;

[0060] Figure 5 This is a flowchart illustrating a method for selecting video clips according to an embodiment of this application;

[0061] Figure 6 This is a schematic diagram illustrating a spliced ​​video display according to an embodiment of this application;

[0062] Figure 7 This is a schematic diagram of the structure of a video processing device provided in one embodiment of this application;

[0063] Figure 8 This is a structural block diagram of a computing device provided in one embodiment of this application. Detailed Implementation

[0064] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0065] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a,” “the,” and “the” used in one or more embodiments of this application and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0066] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first.

[0067] First, the terminology used in one or more embodiments of the present invention will be explained.

[0068] Human pose recognition is a research area in the field of computer vision, which includes static and dynamic recognition. It can capture and recognize human movements offline or in real time.

[0069] Voice Activity Detection (VAD): Based on the spectral range of human voice, the input spectrum is divided into several sub-bands, and the energy of each sub-band is calculated. Then, the probability density function of a Gaussian model is used to calculate a log-likelihood ratio function. The log-likelihood ratio is divided into global and local. The global value is the weighted sum of the sub-bands, while the local value refers to any single sub-band. Therefore, the speech decision first judges the sub-bands. If a sub-band fails the judgment, the global value is judged. As long as either the global or local value passes, speech activity is detected.

[0070] This application provides a video processing method. This application also relates to a video processing apparatus, a video processing system, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0071] In practical applications, video processing, especially video editing, is mostly done manually. The editing method involves splicing together video and graphics, then adding background music to create highlights, focusing on the visual aspects of the original video while neglecting the audio content. This results in poor sound matching. However, this method becomes unsatisfactory when dealing with long videos in the education industry where audio is crucial. The increased video length significantly complicates manual processing, and important information in the audio is lost. Existing automatic video editing methods struggle to guarantee accuracy and efficiency. Furthermore, the large video length leads to a sharp increase in the system's computational resource requirements and processing time, placing significant pressure on hardware such as storage and data processing components.

[0072] The video processing method provided in this application obtains the video to be processed and its corresponding descriptive information; determines a video tag set based on the descriptive information, inputs the video to be processed into a recognition model for recognition, timestamps the positions in the video to be processed that meet the recognition conditions, and obtains the target video; based on the timestamp of the target video, at least one video segment is extracted from the target video; finally, according to the attribute information of each video segment, a corresponding target video tag is configured for the video segment from the video tag set. This enables segmented processing of the video to be processed, reduces waste of video processing resources, alleviates the computational pressure on the system, automates video processing, reduces manual costs, and can complete video processing efficiently and accurately.

[0073] Figure 1 A flowchart of a video processing method according to an embodiment of this application is shown, which specifically includes the following steps:

[0074] Step S102: Obtain the video to be processed and the description information corresponding to the video to be processed.

[0075] The description information corresponding to the video to be processed is a text segment provided by the user for the video to be processed. Tag content can be extracted from the text provided in the description information. The processing result obtained after the video to be processed will be matched with the tags.

[0076] Step S104: Input the video to be processed into the recognition model for processing to obtain the target video containing the timestamp.

[0077] Specifically, after obtaining the video to be processed, it is necessary to identify the part of the video that needs to be processed and determine its location.

[0078] The recognition model identifies specific frames or audio content within the video to be processed. For each identified frame or audio segment, a timestamp is added to mark its location, facilitating subsequent extraction of that specific content from the video. The recognition model can be modeled using an LSTM (Long Short-Term Memory) architecture and trained to meet the specific requirements of the chosen scenario.

[0079] Based on this, the video processing program inputs the acquired video to be processed into the recognition model, and marks specific frames or audio parts in the video with timestamps to obtain the target video containing timestamps.

[0080] Furthermore, the recognition model can accurately and correctly identify the video in the input model and mark the parts of the input video containing specific information. Model training is required. In this embodiment, the specific implementation method is as follows:

[0081] Acquire video samples and corresponding target video samples containing sample timestamps; input the video samples into an initial recognition model for processing to obtain a predicted video containing a prediction timestamp output by the initial recognition model; tune the parameters of the initial recognition model based on the predicted video and the target video samples until a recognition model that meets the training stopping condition is obtained.

[0082] Among them, the video sample is a known video, the target video sample is a video corresponding to the video sample, and a video whose specific images or audio content in its own video content has been timestamped, the sample timestamp is the timestamp in the target video sample, used to mark the specific images or audio content in the video sample, the predicted video is the video output after the video sample is input to the initial recognition model, and the predicted timestamp is the timestamp in the predicted video, used to mark the position of the specific images or audio content in the video sample recognized by the initial recognition model.

[0083] Based on this, the video processing program acquires video samples and corresponding target video samples, which are time-stamped with specific images or audio content in the video sample content. Then, the video samples are input into the initial recognition model, and the output is a predicted video. The predicted video contains the timestamps of specific images or audio content in the video samples identified by the initial recognition model. These timestamps are called predicted timestamps. Finally, based on the difference in timestamp positions between the predicted video and the target video samples, the parameters in the initial recognition model are adjusted to obtain a recognition model that can accurately and correctly identify specific images or audio content in the video.

[0084] For example, during the training of the recognition model, teacher lecture videos are selected as video samples. The initial recognition model needs to identify the question-and-answer segments in the lecture videos. The target sample video is a teacher lecture video with the question-and-answer segments timestamped. After obtaining the teacher lecture videos and target video samples, the teacher lecture videos are input into the initial recognition model. The output prediction video uses predicted timestamps to mark the question-and-answer segments identified by the initial recognition model in the teacher lecture videos. Based on the prediction video and target video samples, the parameters of the initial recognition model are tuned to obtain a recognition model that can identify student hand-raising scenes in videos.

[0085] In summary, after training the initial recognition model, a recognition model is obtained that can accurately identify specific images or audio content in the input video.

[0086] Furthermore, when the initial recognition model identifies video samples, it needs to acquire both the video samples and the target video samples. The target video samples also need to be determined. In this embodiment, the specific implementation method is as follows:

[0087] Extract the sample image frame information and sample audio information of the video sample; determine the sample timestamp position in the sample image frame information or the sample audio information according to the preset recognition feature list; map the sample timestamp position to the time axis of the video sample to obtain the target video sample containing the sample timestamp.

[0088] The identification feature list contains specific images or audio content that need to be identified.

[0089] Based on this, the video processing program extracts the sample image frame information and sample audio information from the video sample, compares the extracted sample image frame information and sample audio information with the specific image or specific audio recorded in the recognition feature list, then timestamps the positions of the content recorded in the recognition feature list in the sample image frame information and sample audio information, and maps the timestamped positions to the timeline of the video sample, so that the corresponding positions of the video sample are also marked, thus obtaining the target video sample.

[0090] Following the previous example, after the video processing program obtains the teacher's lecture video, it timestamps the scenes of students raising their hands in the video. Then, it identifies the audio content following the timestamp of the hand-raising scene at the corresponding time. This audio content represents the students' answers. A timestamp is added again at the end of this audio segment. The content between these two timestamps represents the question-and-answer session. These two timestamps are mapped onto the timeline of the teacher's lecture video. At this point, the teacher's lecture video with both timestamps is used as the target video sample.

[0091] In summary, by identifying the target video sample in the above manner, it is possible to accurately locate specific images or audio content in the sample video, and the accurately located target sample video ensures the accuracy of the initial recognition model training.

[0092] Step S106: Process the target video based on the timestamp to obtain at least one video segment, and determine a video tag set based on the description information.

[0093] Specifically, after the video to be processed is processed by the recognition model, the target video containing the timestamp is obtained. In this case, the target video can be divided based on the timestamp to obtain individual video segments. Then, only the video segments containing the required information are processed. The descriptive information can also be split to extract video tags, which can be used to classify the video segments.

[0094] Among them, video clips are portions extracted from target videos based on requirements, while video tag sets are classification criteria for video clips determined through descriptive information.

[0095] Based on this, the video processing program processes the target video based on the timestamp in the target video, extracts video segments, and also processes the text content of the descriptive information to obtain a video tag set containing video tags.

[0096] Furthermore, the process of segmenting the target video actually involves separating the image content and audio content of the target video. In this embodiment, the specific implementation method is as follows:

[0097] The target video is split into an image frame sequence and an audio file; the image frame sequence and the audio file are segmented based on the timestamp to obtain target image frame sequence segments and target audio segments; the target image frame sequence segments and the target audio segments are combined according to the time correspondence to obtain at least one video segment.

[0098] The image frame sequence is a set of video frames, and the video frames in the set are arranged in a time-order relationship.

[0099] Based on this, the video processing program separates the audio and video parts of the target video to obtain image frame sequences and audio files. Then, based on the timestamp positions on the target video timeline, it segments the image frame sequences and audio files at the corresponding time points to obtain target image frame sequence segments and target audio segments. By combining the obtained target image frame sequences with the target audio segments at the corresponding times, a video segment can be obtained.

[0100] For example, when processing a teacher's lecture video, the recognition model identifies the question-and-answer segments within the video, timestamps the identified segments, and obtains the target video. The video processing system then needs to extract these question-and-answer segments. First, the system divides the target video into image frame sequences and audio files. Then, based on the timestamps on the video's timeline, it locates the corresponding positions in the image frame sequences and audio files. Based on these positions, it segments the image frame sequences and audio files into image frame sequence fragments and audio file fragments. Finally, it combines these fragments to obtain the video segment.

[0101] In summary, dividing the target video into image and audio parts and then combining the division results to obtain video segments ensures that the audio and video content of the resulting video segments is synchronized after the target video is divided.

[0102] Furthermore, dividing the target video based on timestamps transforms it into multiple video segments. However, not every video segment contains useful information. Therefore, it is necessary to filter the resulting video segments. In this embodiment, the specific implementation method is as follows:

[0103] Determine the timestamp position on the timeline of the audio file; based on the timestamp position, divide the audio file and select the audio segment containing audio information in the division result as the target audio segment; extract image frame sequence segments from the image frame sequence according to the time interval of the target audio segment.

[0104] Specifically, when the video processing program segments the target video to obtain video segments, it filters the video segments based on whether the audio segments contain audio content in order to ensure that the obtained video segments contain video content that is meaningful for editing. Based on the filtered audio segments, the time interval of the audio file is mapped to the time axis of the corresponding image frame sequence. Image frame sequence segments with processing significance are selected, and video segments with processing significance are formed based on the filtered audio segments and image frame sequences, which are then used as elements for subsequent video segment processing.

[0105] Continuing with the previous example, the target video contains four timestamps, corresponding to time points t1, t2, t3, and t4 on the target video's timeline. Based on these timestamps, the video processing program can extract three video segments from the target video. These three video segments correspond to time intervals t1-t2, t2-t3, and t3-t4 on the target video's timeline. After filtering the audio segments within these three video segments, it is found that the audio segments in the t2-t3 time interval contain no video content. Therefore, these audio segments are removed. The remaining two audio segments are then mapped to the image frame sequences of the target video, resulting in two image frame sequence segments corresponding to the time intervals t1-t2 and t3-t4. It should be noted that the practical application of the above example is to ensure that the audio file segments extracted from a teacher's lecture video contain audio information.

[0106] In summary, by querying audio information within audio file segments, video processing programs can process only that specific video segment, thus speeding up processing and reducing the hardware load on the system.

[0107] Furthermore, the video tag set can be paired with the obtained video segments, and the video segments can be classified based on the video tags in the video tag set. The process of obtaining the video tag set based on descriptive information is specifically implemented as follows in this embodiment:

[0108] The descriptive information is subjected to text recognition to obtain multiple word units corresponding to the descriptive information; at least one keyword is determined from the multiple word units based on preset filtering rules, and a video tag set is created based on the at least one keyword.

[0109] Among them, word units are text phrases obtained from text recognition in the description information, and keywords are text phrases in the obtained word units that are related to the video to be processed.

[0110] Based on this, the video processing program performs text recognition on the descriptive information, extracts the text content from the descriptive information, obtains multiple text phrases, identifies the text phrases related to the video to be processed among these text phrases as keywords, creates video tags based on the obtained keywords, and places all the video tags in a set to obtain a video tag set.

[0111] For example, the video processing program identifies the descriptive information "students actively participated in the teacher's Q&A session" from a teacher's lecture video using the VSE++ recognition method. The resulting word units are "students", "actively", "participate", "teacher", "question", and "session". Based on these word units, the program selects a keyword, which is "question". Then, it creates a video tag based on the keyword "question" and a video tag set based on this video tag.

[0112] In summary, by creating video tags based on the input description information, users can control the video tags to be displayed and show the parts of the video clips extracted from the video to be edited, based on the video tags. This simplifies video processing and reduces the professional requirements for users.

[0113] Step S108: Construct video features corresponding to each video segment, and select target video tags corresponding to each video segment in the video tag set according to the video features.

[0114] Specifically, after obtaining the video clips and video tag set, it is necessary to match the video clips with the tags in the video tag set so that the video clips can be categorized and displayed according to the video tags.

[0115] Among them, video features are vector representations of the content of the corresponding video segments.

[0116] Based on this, the video processing program constructs corresponding video features from the content of the obtained video segments, and selects appropriate video tags from the video tag set based on the constructed video features.

[0117] Furthermore, before constructing the video features corresponding to each video segment, due to the large number of video segments or user settings based on their own needs, it is necessary to filter the obtained video segments. In this embodiment, the specific implementation method is as follows:

[0118] Receive user instructions and determine scene parameters based on the user instructions; filter out video segments corresponding to the scene parameters from the at least one video segment and create a video segment set; accordingly, construct video features corresponding to each video segment, including constructing video features corresponding to each video segment in the video segment set.

[0119] Among them, scene parameters are the feature representations in the video content of a video clip, used to indicate specific attribute information in the video clip.

[0120] Based on this, the video processing program receives user instructions, determines the scene parameters specified by the user based on the user instructions, filters video segments that meet the scene parameters from the obtained video segments, and constructs a video segment set from the filtered video segments. When constructing the video features corresponding to the video segments in the future, only the video segments in the video segment set can be selected for construction.

[0121] For example, a video processing program divides a teacher's lecture video into segments. After obtaining video segments, the program receives user instructions and obtains scene parameters based on these instructions, specifying a single person. The constraint for a single person is that the segment must have a frontal view accounting for 80% of the total. Based on these scene parameter constraints, the program filters the video segments, resulting in a video segment set. Subsequently, the program extracts video features from the video segments and selects video tags based on these features. The selection range for video segments is the video segments within this video segment set.

[0122] In summary, by constructing scene parameters based on user commands and then filtering video clips using these parameters, the selection of video clips aligns with user needs, reducing the number of video clips requiring operation. This reduces the processing load on the device, speeds up the processing, and minimizes resource waste caused by processing useless video clips.

[0123] Furthermore, the process of constructing video features for a video segment and selecting appropriate video tags from the video tag set based on these features is essentially the process of constructing features for sub-image frame sequences and sub-audio segments within the video segment, and selecting corresponding target video tags based on these features. In this embodiment, the specific implementation is as follows:

[0124] Each video segment is split into sub-image frame sequences and sub-audio segments; the audio features corresponding to the sub-audio segments and / or the image features corresponding to the sub-image frame sequences are determined; and the target video tags corresponding to each video segment are selected from the video tag set based on the audio features and / or the image features.

[0125] Among them, the sub-image frame sequence is the video image part of the video segment, and the sub-audio segment is the audio part of the video segment.

[0126] Based on this, the video processing program segments the video image portion and the audio portion of the video clip to obtain sub-image frame sequences and sub-audio segments. Based on the obtained sub-image frame sequences and sub-audio segments, corresponding image features and audio features are created. Then, appropriate video tags are selected from the video tag set according to the obtained image features and audio features.

[0127] For example, a video processing program segments a video clip from a teacher's lecture to obtain sub-image frame sequences and sub-audio segments. Then, it creates corresponding image features and audio features based on the sub-image frame sequences and sub-audio segments. Finally, it selects the video tag "Question" from the video tag set based on the obtained image features and audio features.

[0128] In summary, segmenting the image and audio parts of a video clip and establishing their respective feature vectors, then selecting the corresponding video tags from the video tag set based on the obtained feature vectors, can make the selection of video tags more accurate and reasonable from multiple perspectives.

[0129] Furthermore, in the process of selecting corresponding video tags for video segments, it cannot be guaranteed that all video segments can match suitable video tags. When this happens, the concept of empty tags is introduced to extract the parts of the video segment that cannot match suitable video tags. In this embodiment, the specific implementation method is as follows:

[0130] The target video tags corresponding to each video segment are traversed, and video segments with empty target video tags are deleted according to the traversal results to obtain a set of target video segments; the video segments in the target video segment set are classified according to the target video tags to obtain a set of target videos corresponding to the target video tags; the video segments contained in the target video sets are spliced ​​together to obtain a spliced ​​video corresponding to the target video tags.

[0131] An empty tag is a special tag selected when a video clip does not have a corresponding video tag in the video tag set.

[0132] Based on this, the video processing program traverses the video segments, selects video segments that do not correspond to video tags in the video tag set but match empty tags, deletes these video segments, and the remaining video segments form the target video segment set. Then, the video segments are classified according to video tags, video segments with the same video tags are grouped into the same category, and video segments in the same category are spliced ​​together to obtain a spliced ​​video for display.

[0133] For example, in the selected video clips, video clips A and B are tagged with "question", while video clip C is tagged with an empty tag. Video clip C is deleted, and video clips A and B are classified into the same category. Then, video clips A and B are spliced ​​together to obtain a spliced ​​video.

[0134] In summary, deleting video segments that do not correspond to video tags in the video tag set is a further screening of video segments, which further reduces the number of video segments. This reduces the workload of video segment splicing and makes the process more user-friendly, thus improving processing efficiency.

[0135] The video processing method provided in this application obtains the video to be processed and its corresponding descriptive information; determines a video tag set based on the descriptive information, inputs the video to be processed into a recognition model for recognition, timestamps the positions in the video to be processed that meet the recognition conditions, and obtains the target video; based on the timestamp of the target video, at least one video segment is extracted from the target video; finally, according to the attribute information of each video segment, a corresponding target video tag is configured for the video segment from the video tag set. This enables segmented processing of the video to be processed, reduces waste of video processing resources, alleviates the computational pressure on the system, automates video processing, reduces manual costs, and can complete video processing efficiently and accurately.

[0136] In video processing scenarios, segmenting the video to be processed and processing the segmented video segments are not limited to the client side. After the video to be processed is segmented, subsequent video segment processing tasks can be performed on the business side.

[0137] Figure 2 A schematic diagram of a video processing system according to an embodiment of this application is shown. The video processing system 200 includes a client 210 and a service terminal 220.

[0138] The client 210 is configured to acquire a video to be processed and a description of the video; input the video to be processed into a recognition model for processing to obtain a target video containing a timestamp; process the target video based on the timestamp to obtain at least one video segment; upload the at least one video segment and the description information to the service server 220; the service server 220 is configured to determine a video tag set based on the description information; construct video features corresponding to each video segment, and select target video tags corresponding to each video segment from the video tag set based on the video features.

[0139] Specifically, client 210 is a user's mobile device, personal computer, or other electronic device that can store and segment video clips, while business terminal 220 is a server, internet-connected video processing program, or other device or program that can segment the video clips obtained from the video clips stored on client 210.

[0140] Based on this, client 210 acquires the video to be processed and receives the corresponding descriptive information. It then identifies the video based on a recognition model, obtaining a target video with timestamps marking specific images or audio content within it. The target video can be further segmented by client 210 to obtain video segments. The segmentation process is described in the above embodiments and will not be elaborated further here. The video segments and descriptive information obtained by client 210 can be transmitted to service terminal 220. Furthermore, after client 210 segments the target video to obtain video segments, these segments need to undergo a filtering process. The processing methods for these video segments can also be applied to client 210. Client 210 can filter the segmented video segments using a preset method, and the filtered video segments are then transmitted to service terminal 220. This reduces the number of transmitted video segments and the amount of transmitted information, lowers transmission time, and speeds up video processing.

[0141] The business unit 220 parses the description information to obtain a set of video tags containing video tags, matches video segments with their corresponding video tags, and finally processes the video segments that match the video tags to obtain a spliced ​​video that can be displayed. The business unit 220 matches the received video segments with the video tags generated from the description information. During the process of matching video segments with video features, further filtering of the video segments is required to ensure that the obtained video segments meet the user's needs and fit the actual application scenario. Therefore, the matching process between video segments and video tags performed by the business unit 220 also needs to follow this process.

[0142] Furthermore, in the step of extracting video segments from the target video, the video segments may contain some meaningless video fragments that are too short, or the target video may be segmented too finely, which is not conducive to subsequent processing. Therefore, before the video segments are uploaded by the client 210 to the service terminal 220, they need to be processed based on their duration. In this embodiment, the client 210 is also configured to determine the video duration of each video segment and the interval time between each video segment; determine an upload strategy based on the video duration and the interval time; and upload the video segments to the service terminal based on the upload strategy.

[0143] The interval time is the time distance between two adjacent video segments on the timeline of the target video.

[0144] Based on this, the video segment on client 210 is first determined to have its own duration, and then the time interval between two consecutive video segments on the timeline of the target video is determined. Then, based on the user's actual needs, an upload strategy is defined. This upload strategy depends on the video duration and the time interval. Video segments with different video durations and video intervals will choose different upload methods during the upload process.

[0145] In the process of selecting the upload method for video segments based on the video duration and time interval, the upload strategy can be configured as follows: the client 210 is further configured to determine a duration threshold and an interval threshold; upload video segments when the video duration is greater than the duration threshold; and upload video segments after splicing them together when the video duration is less than the duration threshold and the interval time is less than the interval threshold.

[0146] The interval threshold and duration threshold are both known values ​​preset based on the usage scenario.

[0147] Based on this, video length determines whether a video segment is fragmented too much. When the video length of a segment exceeds a preset length threshold, it indicates that the segment is not too short and fragmented; it is relatively complete and contains sufficient video information, thus meeting the conditions for direct upload. Conversely, when the video length is less than the length threshold, it indicates that the segment is too fragmented. If the time interval between two consecutive videos is less than the interval threshold, it means that client 210, during the process of segmenting the target video, has fragmented the originally continuous content... The video clip is divided into two parts that are approximately continuous in time and whose content can be considered coherent. In this case, the two parts can be spliced ​​together, and the spliced ​​video can be uploaded. Finally, in addition to these two cases, there are also cases where the duration of a video clip is less than the duration threshold, and the time interval between the two parts is less than the interval threshold. This indicates that the video clip itself contains relatively little video content and is not related to other video clips. Therefore, it can be determined that this video clip has no practical use value, and it can be chosen not to upload it.

[0148] For example, the client determines that four video segments are cut from the teacher's lecture video, namely video segments A, B, C, and D. At this time, the video durations Ta, Tb, Tc, and Td of video segments A, B, C, and D are determined respectively, and their time intervals are Tab, Tbc, and Tcd. Then, based on the scenario requirements, a duration threshold T and an interval threshold Tx are determined. After comparison, it is obtained that Ta>T, Tb<T, Tc<T, Td<T, while Tab>Tx, Tbc>Tx, Tcd<Tx. Then, at this time, video segment A is uploaded, and video segments C and D are spliced and then uploaded, while video segment B does not meet all the preset conditions and is selected not to be uploaded.

[0149] In summary, using this method to screen the video segments to be uploaded can make the total duration of the uploaded video segments positively correlated with the duration of the target video. That is, the longer the duration of the target video, the longer the total duration of the uploaded video segments will be.

[0150] The video processing system provided by this application, through the configuration of the client 210 and the service end 220, enables the client 210 to obtain the video to be processed and the corresponding description information. Then, the video to be processed is input into the recognition model for recognition to obtain a target video with specific images or specific audio content marked by time stamps. Then, the target video is divided into video segments according to the time stamps, and the video segments and the description information are uploaded to the service end 220. The service end 220 is configured to generate video tags according to the description information and then match the corresponding video tags for the video segments. This enables the processing of videos to be carried out across programs and tasks, making the video processing more flexible and closer to real-world use, reducing the waste of video processing resources, reducing the system computing pressure, enabling automatic video processing, reducing labor costs, and efficiently and accurately completing video processing.

[0151] The following combines the attached Figure 3 , taking the application of the video processing method provided by this application to the processing of open class videos as an example, to further illustrate the video processing method. Among them, Figure 3 FIG. shows the processing flow chart of a video processing method applied to the processing of open class videos provided by an embodiment of this application, which specifically includes the following steps:

[0152] Step S302: The client obtains the video to be processed and the description information corresponding to the video to be processed.

[0153] Specifically, the client obtains the open class video to be processed, as well as the description information corresponding to the open class video: "The students actively spoke up, rushed to answer questions, several students performed wonderful programs on stage, and finally there was a speech and summary."

[0154] Step S304: Input the video to be processed into the recognition model for processing to obtain the target video containing the timestamp.

[0155] Specifically, the training process of the recognition model is as follows: Video samples are acquired, and sample image frame information and sample audio information are extracted. Human pose recognition and speech activity detection are used to recognize the sample image frame information and sample audio information. The recognized image content and audio content are compared with specific images or specific audio content in the recognition feature list. Timestamps are added to the timeline of the sample videos corresponding to the parts of the recognized image content and audio content that match the recognition feature list, thus obtaining the target video sample. The video sample is then input into the initial recognition model. The initial recognition model outputs a predicted video containing predicted timestamps. Based on the predicted video and the target sample video, the parameters of the initial recognition model are tuned until a recognition model that meets the training stopping condition is obtained.

[0156] After obtaining the recognition model, the open course video is input into the recognition model for processing, resulting in the target video containing a timestamp. For example... Figure 4 The target video image frame diagram is shown below. Actions 1, 3, and 5 are idle frames, while actions 2, 4, and 6 are video frames representing waving, speaking, and clapping, respectively. After inputting the video to be processed into the recognition model, the target video is obtained. If the specific images in the video to be processed that need to be calibrated are waving and clapping actions, then... Figure 4 In the target video, the timestamp is located in the video frame containing actions 2 and 6.

[0157] Step S306: Process the target video based on the timestamp to obtain at least one video segment.

[0158] Specifically, the target video is split into image and audio segments to obtain image frame sequences and audio files. Then, the timestamp position of the audio file on the timeline is determined based on the timestamp on the target video timeline. The audio file is then divided according to the obtained position information. The audio segments containing audio information in the division results are selected as target audio segments. The time interval of the target audio segments is determined. Target image frame sequence segments are extracted from the same time interval as the image frame sequences. The target image frame sequence segments and target audio segments are combined according to the correspondence to obtain at least video segments A, B, C, D, E, F, and G.

[0159] Step S308: Filter the video clips and upload the filtered video clips and description information to the business terminal.

[0160] Specifically, a duration threshold of 2 seconds and an interval threshold of 1 second are selected. The video durations of video segments A, B, C, D, E, F, and G are determined to be 0.9, 1, 1.1, 2, 3, 4, and 5 seconds, respectively, with intervals of 3, 0.5, 1, 1.5, 2, and 2.5 seconds, respectively. Video segments with durations greater than the duration threshold are uploaded; video segments with durations less than the duration threshold and intervals less than the interval threshold are spliced ​​together and then uploaded. Ultimately, video segment A is not uploaded, video segments B and C are spliced ​​together and uploaded, and video segments D, E, F, and G are uploaded. For example... Figure 5 The flowchart shown illustrates the process of filtering video clips. Video clips obtained based on timestamps are filtered, and video clips with a duration greater than 2 seconds are uploaded. Video clips with a duration less than 2 seconds and a time interval greater than 1 second are discarded from uploading. Video clips with a duration less than 2 seconds and a time interval less than 1 second are spliced ​​together and then uploaded. After determining the video clips to be uploaded, the description information also needs to be uploaded to the business end at the same time.

[0161] Step S310: Determine the video tag set based on the description information.

[0162] Specifically, in the business side, natural language processing technology is used. In this embodiment, the method adopted is to use VSE++ to perform text recognition on the description information "Students actively spoke, answered questions, several students performed wonderful programs on stage, and finally gave a speech and summary." to obtain multiple word units corresponding to the description information, such as "students", "actively", "speak", "answer", ..., "summary". Based on preset filtering rules, the keywords are determined to be "answer", "performance", and "speech" among the multiple word units, and a video tag set is created based on these keywords.

[0163] Step S312: Filter video clips based on scene parameters.

[0164] Specifically, the user provides user instructions, and scene parameters are determined based on the user instructions. Scene parameters include the number of people, which can be selected as single or multiple people. In this embodiment, the focus of single people is on close-ups, and a segment with a frontal view accounting for 80% is selected; the focus of multiple people is on the full number of people, and a segment with more than 60% of the people is selected.

[0165] Location options include home / classroom.

[0166] The format can be selected as speech / performance / lecture. In this embodiment, the above formats affect the proportion of sound and action after overall editing. The speech sound proportion is set to be above 80% > lecture 60-80% > performance below 60%.

[0167] It should be noted that multiple scene parameters can be selected when determining the scene parameters.

[0168] Based on user instructions, the scene parameters are determined as single or two people, classroom location, and format as lecture, performance, or listening. Based on the selected scene parameters, the spliced ​​video clips B and C, as well as video clips D, E, F, and G, are filtered to ensure that all spliced ​​video clips B and C, as well as video clips D, E, F, and G, conform to the scene parameters.

[0169] Step S314: Construct video features corresponding to each video segment, and select target video tags corresponding to each video segment in the video tag set according to the video features.

[0170] Specifically, video features are constructed for each video segment, and target video tags corresponding to each video segment are selected from the video tag set based on the video features. This results in the video segments B, C and D with the video tag "quiz," video segments E and F with the video tag "performance," and video segment G with the video tag "speech."

[0171] Step S316: Cluster and stitch together video clips with the same video tags, and then display the processed videos according to the video tags.

[0172] Specifically, video clips B, C, and D are clustered and then stitched together; video clips E and F are clustered and then stitched together; the video stitched from B, C, and D is displayed with the tag "Quiz"; the video stitched from E and F is displayed with the tag "Performance"; and video clip G is displayed with the tag "Speech". For example... Figure 6 The diagram shown illustrates the spliced ​​video display. In the final display format, there is an area for displaying descriptive information. When the user selects the "Quick Answer" tag, the spliced ​​video clips B, C, and D will be expanded and played. When the user selects the "Performance" tag, the spliced ​​video clips E and F will be expanded and played. When the user selects the "Speech" tag, video clip G will be expanded and played.

[0173] The embodiments provided in this application describe content that enables video processing to be performed across programs and tasks, making video processing more flexible and closer to real-world use, reducing waste of video processing resources, alleviating system computing pressure, automating video processing, reducing labor costs, and enabling efficient and accurate video processing.

[0174] Corresponding to the above method embodiments, this application also provides video processing apparatus embodiments. Figure 7 A schematic diagram of the structure of a video processing apparatus according to an embodiment of this application is shown. Figure 7 As shown, the device includes:

[0175] The acquisition module 702 is configured to acquire the video to be processed and the description information corresponding to the video to be processed;

[0176] Processing module 704 is configured to input the video to be processed into a recognition model for processing to obtain a target video containing a timestamp;

[0177] The determining module 706 is configured to process the target video based on the timestamp to obtain at least one video segment, and to determine a video tag set based on the description information;

[0178] The selection module 708 is configured to construct video features corresponding to each video segment, and select target video tags corresponding to each video segment in the video tag set according to the video features.

[0179] In an optional embodiment, the determining module 706 is further configured to:

[0180] The target video is split into an image frame sequence and an audio file; the image frame sequence and the audio file are segmented based on the timestamp to obtain target image frame sequence segments and target audio segments; the target image frame sequence segments and the target audio segments are combined according to the time correspondence to obtain at least one video segment.

[0181] In an optional embodiment, the determining module 706 is further configured to:

[0182] Determine the timestamp position on the timeline of the audio file; based on the timestamp position, divide the audio file and select the audio segment containing audio information in the division result as the target audio segment; extract image frame sequence segments from the image frame sequence according to the time interval of the target audio segment.

[0183] In an optional embodiment, the determining module 706 is further configured to:

[0184] The descriptive information is subjected to text recognition to obtain multiple word units corresponding to the descriptive information; at least one keyword is determined from the multiple word units based on preset filtering rules, and a video tag set is created based on the at least one keyword.

[0185] In an optional embodiment, the video processing apparatus further includes:

[0186] The parameter module is configured to receive user instructions, determine scene parameters based on the user instructions, filter out video segments corresponding to the scene parameters from the at least one video segment, and create a video segment set; accordingly, construct video features corresponding to each video segment, including constructing video features corresponding to each video segment in the video segment set.

[0187] In an optional embodiment, the selection module 708 is further configured to:

[0188] Each video segment is split into sub-image frame sequences and sub-audio segments; the audio features corresponding to the sub-audio segments and / or the image features corresponding to the sub-image frame sequences are determined; and the target video tags corresponding to each video segment are selected from the video tag set based on the audio features and / or the image features.

[0189] In an optional embodiment, the video processing apparatus further includes:

[0190] The empty tag filtering module is configured to traverse the target video tags corresponding to each video segment, delete video segments with empty target video tags based on the traversal results, and obtain a set of target video segments; classify the video segments in the target video segment set according to the target video tags to obtain a set of target videos corresponding to the target video tags; and splice the video segments contained in the target video set to obtain a spliced ​​video corresponding to the target video tags.

[0191] In an optional embodiment, the video processing apparatus further includes:

[0192] The recognition model training module is configured to acquire video samples and target video samples containing sample timestamps corresponding to the video samples; input the video samples into an initial recognition model for processing to obtain a predicted video containing a prediction timestamp output by the initial recognition model; and tune the parameters of the initial recognition model based on the predicted video and the target sample video until the recognition model that meets the training stopping condition is obtained.

[0193] In an optional embodiment, the recognition model training module is further configured to:

[0194] Extract the sample image frame information and sample audio information of the video sample; determine the sample timestamp position in the sample image frame information or the sample audio information according to the preset recognition feature list; map the sample timestamp position to the time axis of the video sample to obtain the target video sample containing the sample timestamp.

[0195] The video processing apparatus provided in this application acquires the video to be processed and its corresponding descriptive information; determines a video tag set based on the descriptive information; inputs the video to be processed into a recognition model for recognition; timestamps the positions in the video to be processed that meet the recognition conditions to obtain the target video; extracts at least one video segment from the target video based on the timestamp of the target video; and finally, configures corresponding target video tags for each video segment from the video tag set according to the attribute information of each video segment. This enables segmented processing of the video to be processed, reduces waste of video processing resources, alleviates system computing pressure, automates video processing, reduces labor costs, and can complete video processing efficiently and accurately.

[0196] The above is an illustrative scheme of a video processing apparatus according to this embodiment. It should be noted that the technical solution of this video processing apparatus and the technical solution of the aforementioned video processing method belong to the same concept. Details not described in detail in the technical solution of the video processing apparatus can be found in the description of the technical solution of the aforementioned video processing method. Furthermore, the components in the apparatus embodiment should be understood as functional modules necessary to implement each step of the program flow or each step of the method; these functional modules are not actual functional divisions or separations. An apparatus claim defined by such a set of functional modules should be understood as a functional module architecture that primarily implements the solution through the computer program described in the specification, and not as a physical device that primarily implements the solution through hardware.

[0197] Figure 8 A structural block diagram of a computing device 800 according to an embodiment of this application is shown. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0198] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0199] In one embodiment of this application, the aforementioned components of the computing device 800 and Figure 8 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 8 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0200] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 800 can also be a mobile or stationary server.

[0201] The processor 820 is used to execute the following computer-executable instructions:

[0202] Obtain the video to be processed and its corresponding description information;

[0203] The video to be processed is input into the recognition model for processing to obtain the target video containing timestamps;

[0204] The target video is processed based on the timestamp to obtain at least one video segment, and a video tag set is determined based on the description information;

[0205] Construct video features corresponding to each video segment, and select target video tags corresponding to each video segment from the video tag set based on the video features.

[0206] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the video processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the video processing method described above.

[0207] An embodiment of this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, are used to: acquire a video to be processed and description information corresponding to the video to be processed;

[0208] The video to be processed is input into the recognition model for processing to obtain the target video containing timestamps;

[0209] The target video is processed based on the timestamp to obtain at least one video segment, and a video tag set is determined based on the description information;

[0210] Construct video features corresponding to each video segment, and select target video tags corresponding to each video segment from the video tag set based on the video features.

[0211] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the video processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the video processing method described above.

[0212] One embodiment of this application also provides a chip that stores a computer program, which, when executed by the chip, implements the steps of the video processing method.

[0213] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0214] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0215] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0216] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0217] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A video processing method, characterized in that, include: Obtain the video to be processed and its corresponding description information; The description information is provided by the user regarding the video to be processed. supply The text; The video to be processed is input into the recognition model for processing. Specific frames or specific audio parts in the video to be processed are marked with timestamps to obtain the target video containing timestamps. The target video is processed based on the timestamp to obtain at least one video segment, and a video tag set is determined based on the description information. The method of processing the target video based on the timestamp includes splitting the target video into an image frame sequence and an audio file, dividing the audio file based on the timestamp position of the timestamp on the time axis of the audio file, and selecting the audio segment containing audio information in the division result as the target audio segment; and extracting the target image frame sequence segment from the image frame sequence according to the time interval of the target audio segment. The method of determining the video tag set based on the description information includes performing text recognition on the description information to obtain multiple word units corresponding to the description information; determining at least one keyword from the multiple word units based on preset filtering rules, and creating a video tag set based on the at least one keyword. Construct video features corresponding to each video segment, and select target video tags corresponding to each video segment in the video tag set according to the video features. The video segment includes a sub-image frame sequence and a sub-audio segment. The video features include audio features corresponding to the sub-audio segment and / or image features corresponding to the sub-image frame sequence.

2. The method according to claim 1, characterized in that, The process of processing the target video based on the timestamp to obtain at least one video segment includes: The target image frame sequence fragments and the target audio fragments are combined according to the time correspondence to obtain at least one video fragment.

3. The method according to claim 1, characterized in that, Before constructing the video features corresponding to each video segment, the process also includes: Receive user instructions and determine scene parameters based on the user instructions; Filter out the video segments that correspond to the scene parameters from the at least one video segment, and create a video segment set; Accordingly, video features are constructed for each video segment, including: Construct video features corresponding to each video segment in the video segment set.

4. The method according to claim 1, characterized in that, The step of constructing video features corresponding to each video segment and selecting target video tags corresponding to each video segment in the video tag set based on the video features includes: Each video segment is split into sub-image frame sequences and sub-audio segments; Determine the audio features corresponding to the sub-audio segment and / or the image features corresponding to the sub-image frame sequence; Based on the audio features and / or the image features, target video tags corresponding to each video segment are selected from the video tag set.

5. The method according to claim 1, characterized in that, After selecting the target video tags corresponding to each video segment from the video tag set based on the video features, the method further includes: Iterate through the target video tags corresponding to each video segment, and delete the video segments with empty target video tags based on the traversal results to obtain the target video segment set. The video segments in the target video segment set are classified according to the target video tags to obtain the target video set corresponding to the target video tags; The video segments contained in the target video set are spliced ​​together to obtain the spliced ​​video corresponding to the target video tag.

6. The method according to claim 1, characterized in that, The training of the recognition model is determined in the following way: Obtain video samples and target video samples containing sample timestamps corresponding to the video samples; The video sample is input into the initial recognition model for processing to obtain the predicted video containing the prediction timestamp output by the initial recognition model; The initial recognition model is tuned based on the predicted video and the target video sample until the recognition model that meets the training stopping condition is obtained.

7. The method according to claim 6, characterized in that, The step of obtaining the target video sample containing the sample timestamp corresponding to the video sample includes: Extract the sample image frame information and sample audio information of the video sample; The sample timestamp position is determined in the sample image frame information or the sample audio information according to the preset recognition feature list; The sample timestamp position is mapped to the timeline of the video sample to obtain the target video sample containing the sample timestamp.

8. A video processing system, characterized in that, include: Client and business side; The client is configured to obtain the video to be processed and the description information corresponding to the video to be processed; The description information is text provided by the user for the video to be processed; The video to be processed is input into the recognition model for processing. Specific frames or specific audio parts in the video to be processed are marked with timestamps to obtain the target video containing timestamps. The target video is processed based on the timestamp to obtain at least one video segment, and a video tag set is determined based on the description information. The processing of the target video based on the timestamp includes splitting the target video into an image frame sequence and an audio file; dividing the audio file based on the timestamp position of the timestamp on the timeline of the audio file; selecting audio segments containing audio information as target audio segments; and extracting target image frame sequence segments from the image frame sequence according to the time interval of the target audio segments. The method for determining the video tag set based on the description information includes performing text recognition on the description information to obtain multiple word units corresponding to the description information; determining at least one keyword from the multiple word units based on preset filtering rules; and creating a video tag set based on the at least one keyword; and uploading the at least one video segment and the description information to the service terminal. The service terminal is configured to determine a video tag set based on the description information; construct video features corresponding to each video segment; and select target video tags corresponding to each video segment in the video tag set based on the video features. The video segment includes a sub-image frame sequence and a sub-audio segment, and the video features include audio features corresponding to the sub-audio segment and / or image features corresponding to the sub-image frame sequence.

9. The system according to claim 8, characterized in that, The client is also configured to determine the video duration of each video segment and the interval between each video segment; determine an upload strategy based on the video duration and the interval; and upload the video segments to the business server based on the upload strategy.

10. The system according to claim 9, characterized in that, The client is also configured to determine a duration threshold and an interval threshold; upload a video segment if the video duration is greater than the duration threshold; and upload a video segment after splicing it together if the video duration is less than the duration threshold and the interval is less than the interval threshold.

11. A video processing apparatus, characterized in that, include: The acquisition module is configured to acquire the video to be processed and the description information corresponding to the video to be processed. The description information is text provided by the user for the video to be processed; The processing module is configured to input the video to be processed into the recognition model for processing, and to mark specific frames or specific audio parts in the video to be processed with timestamps to obtain a target video containing timestamps. The determination module is configured to process the target video based on the timestamp to obtain at least one video segment, and to determine a video tag set based on the description information. The processing of the target video based on the timestamp includes splitting the target video into an image frame sequence and an audio file; dividing the audio file based on the timestamp position of the timestamp on the timeline of the audio file; selecting audio segments containing audio information in the division results as target audio segments; and extracting target image frame sequence segments from the image frame sequence according to the time interval of the target audio segments. The method for determining the video tag set based on the description information includes performing text recognition on the description information to obtain multiple word units corresponding to the description information; determining at least one keyword from the multiple word units based on preset filtering rules; and creating a video tag set based on the at least one keyword. The selection module is configured to construct video features corresponding to each video segment, and select target video tags corresponding to each video segment in the video tag set according to the video features. The video segment includes a sub-image frame sequence and a sub-audio segment, and the video features include audio features corresponding to the sub-audio segment and / or image features corresponding to the sub-image frame sequence.

12. A computing device, characterized in that, It includes a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the method according to any one of claims 1-7.

13. A computer-readable storage medium storing computer instructions, characterized in that, When executed by the processor, this instruction implements the steps of the method according to any one of claims 1-7.

Images