Video processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a video processing method and device, and relates to the technical field of computers, in particular to the technical fields of large models, deep learning, image processing, voice recognition and the like. According to the specific implementation scheme, at least two video content types are acquired; based on the video content type and the original video set, obtaining a to-be-processed video set comprising at least one to-be-processed video, each to-be-processed video corresponding to one video content type; based on the to-be-processed video set, obtaining video knowledge points of each to-be-processed video; and the to-be-processed video set is processed based on the video content type and the video knowledge point to generate the target video set, so that the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, specifically to technical fields such as large models, deep learning, image processing, and speech recognition, and in particular to a video processing method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] When constructing a video knowledge base, the video is generally segmented or classified based on the subtitles or video content of the video. The structural units in the knowledge base constructed in this way only summarize the surface information of the video and cannot fully and deeply express the video information. This is especially true for courseware explanation videos, as this method cannot deeply represent the courseware information. Summary of the invention

[0003] The present disclosure provides a video processing method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] According to a first aspect, a video processing method is provided, the method comprising: obtaining at least two video content types; based on the video content type and the original video set, obtaining a set of videos to be processed including at least one video to be processed, each video to be processed corresponding to a video content type; based on the set of videos to be processed, obtaining video knowledge points of each video to be processed; based on the video content type and the video knowledge points, processing the set of videos to be processed to generate a target video set.

[0005] According to a second aspect, a video processing device is provided, which includes: an acquisition unit, configured to acquire at least two video content types; a video acquisition unit, configured to obtain a set of videos to be processed including at least one video to be processed based on the video content type and the original video set, each video to be processed corresponds to a video content type; a knowledge point acquisition unit, configured to obtain video knowledge points of each video to be processed based on the set of videos to be processed; and a generation unit, configured to process the set of videos to be processed based on the video content type and the video knowledge points to generate a target video set.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner of the first aspect.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described in any implementation of the first aspect.

[0008] According to a fifth aspect, a computer program product is provided, comprising a computer program, which implements the method described in any implementation manner of the first aspect when executed by a processor.

[0009] The video processing method and device provided by the embodiments of the present disclosure first obtain at least two video content types; secondly, based on the video content type and the original video set, obtain a to-be-processed video set including at least one to-be-processed video, each to-be-processed video corresponds to a video content type; thirdly, based on the to-be-processed video set, obtain the video knowledge points of each to-be-processed video; finally, based on the video content type and the video knowledge points, process the to-be-processed video set to generate a target video set. Thus, the video type and the video knowledge points are used as the association points of the to-be-processed videos in the to-be-processed video set, and target videos of different types with the same video knowledge points are generated, which pays more attention to the internal logic and association of the knowledge in the video, and realizes the structured processing of the video; when the target video is recommended to the user, the user experience can be improved.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a flow chart of an embodiment of a video processing method according to the present disclosure;

[0013] Figure 2 It is a structural schematic diagram of the target video set generation process disclosed in the present invention;

[0014] Figure 3 It is a structural schematic diagram of the process of generating a video set in the present disclosure;

[0015] Figure 4 is another structural schematic diagram of the target video set generation process disclosed in the present invention;

[0016] Figure 5 is a flowchart of another embodiment of the video processing method according to the present disclosure;

[0017] Figure 6 is a structural schematic diagram of an embodiment of a video processing device according to the present disclosure;

[0018] Figure 7 The block diagram is a block diagram of an electronic device for implementing the video processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] In view of the defects in the traditional technology, the present invention proposes a video processing method. Figure 1 A process 100 according to an embodiment of a video processing method of the present disclosure is shown. The video processing method comprises the following steps:

[0021] Step 101: Obtain at least two video content types.

[0022] In this embodiment, the execution entity on which the video processing method runs can be a video on demand system, which enables users to watch, like, comment and upload videos. The video on demand system can also summarize all videos, recommend related videos to users, and build and maintain a video knowledge base.

[0023] In this embodiment, the video content type is used to characterize the specific type of the video content in the video, wherein the video content type may correspond to the subject type of the video content. If a video has at least two video content types, then the video has at least two subject types.

[0024] In this embodiment, at least two video content types may be pre-set types, for example, an information point introduction video and an information point application video.

[0025] Step 102: Based on the video content type and the original video set, a to-be-processed video set including at least one to-be-processed video is obtained.

[0026] In this embodiment, the original video set includes at least one original video, and the original video is an unprocessed video having at least one video content type. By processing the original videos, a to-be-processed video corresponding to each original video can be obtained.

[0027] In this embodiment, the set of videos to be processed includes at least one video to be processed, and the video to be processed is a video obtained after processing the original video (for example, information labeling, video segmentation), and the video to be processed is also a video with only one video content type. Specifically, the video to be processed can be a video obtained after the original video in the original video set is labeled with the video content type.

[0028] In this embodiment, each video to be processed in the set of videos to be processed corresponds to a video content type. For courseware explanation videos, the videos to be processed can be knowledge concept explanation videos, question explanation videos, etc., among which knowledge concept explanation videos belong to a video content type, and question explanation videos belong to a video content type.

[0029] In the technical solution disclosed in the present invention, the collection, storage, use, processing, transmission, provision and disclosure of the video collection to be processed are carried out after authorization and in compliance with relevant laws and regulations. The information related to the user in the video collection to be processed is information obtained after the user's permission, and the video processing related to the user is processed under confidentiality conditions.

[0030] In this embodiment, the above step 102 includes: extracting the video text of each original video in the original video set through subtitles or voice recognition; determining the video to be processed included in each original video based on the video text; determining the video content type of the video to be processed based on the video text of each video to be processed, for example, for courseware explanation videos, the video content type is a concept type or a question type.

[0031] Step 103, based on the set of videos to be processed, obtaining video knowledge points of each video to be processed.

[0032] In this embodiment, the video knowledge point is the basic unit of knowledge that constitutes the video to be processed, and the video knowledge point is determined by the video content of the video to be processed. For example, if the video to be processed is a video explaining the monotonicity of quadratic equations, then the video knowledge point of the video to be processed is: the monotonicity of quadratic equations.

[0033] In this embodiment, knowledge point analysis can be performed on the video information of each video to be processed in the video set to obtain the video knowledge points of each video to be processed. Specifically, the video information of the video to be processed is extracted, and the video information can be video text or video voice; the video information is matched with a preset knowledge point set, and in response to a successful match between the video information and a knowledge point in the preset knowledge point set, the successfully matched knowledge point is used as the video knowledge point.

[0034] In this embodiment, for the videos to be processed that fail to match the preset knowledge point set in the video set to be processed, the video information of the videos to be processed can be input into a pre-trained knowledge point extraction model to obtain the video knowledge points output by the knowledge point extraction model.

[0035] Step 104: Process the to-be-processed video set based on the video content type and the video knowledge points to generate a target video set.

[0036] In this embodiment, for each video to be processed in the video set to be processed, the video knowledge point and video type of the video to be processed are determined; and the videos to be processed with the same knowledge point and different video types in the video set to be processed are spliced ​​together as the target video.

[0037] In this embodiment, the target video set includes at least one target video, each target video has a corresponding video type and video knowledge points. The target video in the target video set can be a video to be processed with the video type and video knowledge points added. The target video in the target video set can also be a video after the video type and video knowledge points are added and the video to be processed is processed.

[0038] In this embodiment, the video set to be processed may be a sub-video set including at least two different video content types, and the videos in the sub-video set of each video content type have their own corresponding video knowledge points. By associating or merging the videos in different types of video sets according to the video knowledge points, at least one target video set may be obtained, such as Figure 2 As shown, two sub-video sets of different video content types are: a first type video set and a second type video set. In the first type video set, there are n (n>1) videos with a first video knowledge point D1, in the first type video set, there are n videos with a second video knowledge point D2, ... in the first type video set, there are n videos with an nth video knowledge point Dn, wherein the first video knowledge point D1, the second video knowledge point D2, ... and the mth video knowledge point Dm (m>1) are all video knowledge points in the video knowledge points; in the second type video set, there are n (n>1) videos with a first video knowledge point D1, in the second type video set, there are n videos with a second video knowledge point D2, ... in the second type video set, there are n videos with an nth video knowledge point Dm. The first type video set with the first video knowledge point D1 and the second type video set with the first video knowledge point D1 in the video set to be processed are merged or associated together to obtain a target video set. There is at least one target video in the target video set, and the at least one target video forms a video group corresponding to different video knowledge points, such as Figure 2 Multiple groups of video 1 to video n in .

[0039] The video processing method provided by the embodiment of the present disclosure first obtains at least two video content types; secondly, based on the video content type and the original video set, a to-be-processed video set including at least one to-be-processed video is obtained, and each to-be-processed video corresponds to a video content type; thirdly, based on the to-be-processed video set, the video knowledge points of each to-be-processed video are obtained; finally, based on the video content type and the video knowledge points, the to-be-processed video set is processed to generate a target video set. Thus, the video type and the video knowledge points are used as the association points of the to-be-processed videos in the to-be-processed video set, and target videos of different types with the same video knowledge points are generated, which pays more attention to the internal logic and association of the knowledge in the video, and realizes the structured processing of the video; when the target video is recommended to the user, the user experience can be improved.

[0040] In some optional implementations of the present disclosure, the above-mentioned method of obtaining a set of videos to be processed including at least one video to be processed based on the video content type and the original video set includes: clustering the original videos in the original video set to obtain clustering information; based on the clustering information, marking the video content type of the original videos belonging to only one clustering type; based on the clustering information, detecting whether there are original videos belonging to different clustering types; in response to detecting original videos belonging to different clustering types, segmenting the original video according to the clustering type to obtain at least one segmented video segment; marking the video content type of each segmented video segment to obtain a set of videos to be processed including at least one video to be processed.

[0041] In this optional implementation, the clustering information is information obtained by subject-clustering the video content of the original video, and the clustering information includes: video content theme, clustering type, wherein the video content theme is the theme of the video content of the original video, and the clustering type is the theme type to which the video content belongs.

[0042] In this optional implementation, the video content of an original video may have one video content theme or multiple video content themes. When the original video has at least one video content theme and the video content theme belongs to one clustering type, it is determined that the original video belongs to only one clustering type; when the original video has multiple content themes and the video content themes belong to multiple clustering types, it is determined that the original video belongs to different clustering types.

[0043] In this optional implementation, segmenting the original video according to the clustering type to obtain at least one segmented video segment means: segmenting the video segments according to the clustering types corresponding to different video segments in the original video to obtain at least one segmented video segment.

[0044] In this optional implementation, the to-be-processed videos in the to-be-processed video set may include segmented video segments with video content types marked, and may also include: original videos with video content types marked.

[0045] The method for obtaining a set of videos to be processed provided by this optional implementation manner clusters original videos in the original video set to obtain clustering information; based on the clustering information, video content type annotation is performed on original videos belonging to only one clustering type; based on the clustering information, whether there are original videos belonging to different clustering types is detected; in response to detecting original videos belonging to different clustering types, the original video is segmented according to the clustering type to obtain at least one segmented video segment; video content type annotation is performed on each segmented video segment to obtain a set of videos to be processed including at least one video to be processed, thereby improving the reliability and accuracy of obtaining the target video set.

[0046] Optionally, when each original video in the original video set has only one video content type, the above-mentioned method of obtaining a set of videos to be processed including at least one video to be processed based on the video content type and the original video set includes: clustering the original videos in the original video set to obtain clustering information; and marking the original videos with corresponding video content types based on the clustering information.

[0047] Optionally, when each original video in the original video set has only one video content type, the above-mentioned step of obtaining the video set to be processed including at least one video to be processed based on the video content type and the original video set includes: determining the video content type of each original video in the original video set; and marking the original videos with corresponding video content types.

[0048] Optionally, when each original video in the original video set has more than two video content types, the above-mentioned method of obtaining the video set to be processed including at least one video to be processed based on the video content type and the original video set includes: clustering the original videos in the original video set to obtain clustering information; segmenting each original video according to the clustering type to obtain at least one segmented video segment; and marking the video content type of each segmented video segment to obtain the video set to be processed including at least one video to be processed.

[0049] In some optional implementations of the present disclosure, the above-mentioned obtaining a set of videos to be processed including at least one video to be processed based on the video content type and the original video set includes: obtaining a pre-trained big model; based on the big model, obtaining the video content type of each original video in the original video set; based on the original video set and the video content type of each original video, obtaining the set of videos to be processed.

[0050] In this optional implementation, the pre-trained large model is a multimodal large model, which is used to analyze the original videos in the original video set to obtain the video content type of each original video in the original video set.

[0051] In this optional implementation, the above-mentioned obtaining of the video set to be processed based on the original video set and the video content type of each original video includes: in response to the original video of the original video set having at least two video content types, the original video is segmented according to the video content type to obtain the video set to be processed including at least one video to be processed.

[0052] The method for obtaining a set of videos to be processed provided by this optional implementation method obtains a pre-trained large model; based on the large model, the video content type of each original video in the original video set is obtained; based on the original video set and the video content type of each original video, the set of videos to be processed is obtained, thereby distinguishing the video content type of each original video through the large model, thereby improving the reliability of obtaining the video content type; based on the obtained video content type and the original video set, the set of videos to be processed is obtained, thereby improving the reliability of obtaining the set of videos to be processed.

[0053] In some optional implementations of the present disclosure, obtaining the video set to be processed based on the original video set and the video content type of each original video includes: acquiring at least one video feature of the original video set; and processing the original videos in the original video set based on the video feature and the video content type of each original video to obtain the video set to be processed.

[0054] In this optional implementation, video features are features that reflect the original video's own information, wherein video features include: video picture features, video audio features, and video text features. Video text features include: video subtitle text features and video content text features. Video subtitle features are features obtained by extracting subtitles from the original video; video picture features are features obtained by extracting image features from the original video; and video audio features are features obtained by performing voice analysis on the original video.

[0055] In this optional implementation, the original videos in the original video set are processed based on the video features and the video content types of each original video to obtain the video set to be processed, including: segmenting the original video based on the video features of each original video; identifying and labeling the video content types of the segmented video segments to obtain the video set to be processed.

[0056] The method for obtaining the video set to be processed provided by this optional implementation obtains at least one video feature of the original video set; based on the video feature and the video content type of each original video, the original videos in the original video set are processed to obtain the video set to be processed. Thus, the original videos in the original video set are processed in combination with the video feature and the video content type to obtain the video set to be processed, thereby improving the reliability and accuracy of obtaining the video set to be processed.

[0057] In some optional implementations of the present disclosure, the above-mentioned video features include video picture features and video audio features. Based on the video features and the video content type, the original videos in the original video set are processed to obtain the video set to be processed, including: for the original videos in the original video set, based on the video picture features, the original videos are sliced ​​to obtain at least one picture video segment; based on the video audio features, the original videos are sliced ​​to obtain at least one audio video segment; at least one video segment and at least one audio video segment are aligned and merged to obtain at least one merged video; based on the video content type of the original video, the video content type of each merged video in at least one merged video is identified and labeled to obtain a processed video set.

[0058] The method for obtaining a processed video set provided by this optional implementation slices the original video in the original video set based on the video picture features to obtain at least one picture video segment; slices the original video based on the video audio features to obtain at least one audio and video segment; aligns and merges at least one video segment and at least one audio and video segment to obtain at least one merged video; and identifies and labels the video content type of each merged video in at least one merged video based on the video content type of the original video to obtain a processed video set, which provides a reliable implementation method for obtaining the processed video set and improves the reliability and accuracy of obtaining the processed video set.

[0059] Optionally, the above-mentioned video features include video picture features and video audio features. Based on the video features and video content types, the original videos in the original video set are processed to obtain the video set to be processed, including: for the original videos in the original video set, based on video subtitle features, the original video is sliced ​​to obtain at least one subtitle video segment; based on video audio features, the original video is sliced ​​to obtain at least one audio video segment; at least one video segment and at least one subtitle video segment are aligned and merged to obtain at least one combined video; based on the video content type of the original video, the video content type of each combined video in at least one combined video is identified and labeled to obtain a processed video set.

[0060] In this optional implementation, speech recognition may be performed on the original videos in the original video set to generate video and audio features of the original videos; and text information of the original videos may be extracted to generate video subtitle features of the original videos.

[0061] In this optional implementation, if Figure 3 As shown, the system for generating a target video set includes: a data input layer, a processing layer, a fusion layer, and an output layer, wherein the data input layer is used to receive the video files of the original video set and input the video text as the original data. The processing layer includes an ASR module, an OCR module, and a content analysis module. The ASR module is used to extract the video and audio features of the original video, the OCR module is used to extract the video subtitle features of the original video, and the content analysis module is used to analyze the video audio features and the video subtitle features. The fusion layer slices and fuses each original video based on the video subtitle features and video audio features generated by the processing layer. The output layer is used to mark the video of the fusion layer with video knowledge points and video types, and obtain a target video set including at least one target video.

[0062] In this optional implementation, the speech recognition of the original video in the original video set may be performed using ASR (Automatic Speech Recognition) technology to obtain the subtitle text.

[0063] In this optional implementation, video subtitle features of the original video may be extracted through OCR (Optical Character Recognition).

[0064] In this optional implementation, the content video segment and the subtitle video segment can be aligned based on the time of the time axis. The processed video is a video that integrates the subtitle video segment and the content video segment in the same time period and is annotated with the video type and video knowledge points.

[0065] In some optional implementations of the present disclosure, obtaining at least two video content types includes: obtaining video content types of courseware explanation videos, and the video content types include at least: concept types and question types.

[0066] In this optional implementation, for the video content type of the courseware explanation video, the concept type is the video type that explains different concepts in the courseware, and the question type is the video type that explains different questions in the courseware.

[0067] The method for determining the video type provided by this optional implementation method obtains at least two video content types, including: obtaining the video content type of the courseware explanation video, and the video content type includes at least: concept type and question type, thereby improving the accuracy of the obtained video type.

[0068] For an original video set including a concept type and a question type in terms of video content type, a set of videos to be processed is obtained. Based on the set of videos to be processed, video knowledge points of each video to be processed are obtained. Based on the video content type and the video knowledge points, the set of videos to be processed is processed to generate a target video set. For the set of videos to be processed including a concept type and a question type, the process of obtaining the target video set is as follows: Figure 4 shown.

[0069] like Figure 4 As shown, the video set to be processed includes a concept video set and a question video set, wherein the concept video set is a set of videos to be processed corresponding to the concept type, the question video set is a set of videos to be processed corresponding to the question type, the concept video set includes m groups of concept videos, each group of concept videos has n concept videos, each group of concept videos corresponds to a knowledge point, Figure 4 Each set of concept videos includes concept videos 1 to concept videos n; the set of topic videos includes m topic videos, such as Figure 4 Questions 1 to m are all question-type videos, and each question-type video corresponds to a video knowledge point; the target video is a video composed of a concept-type video and a question-type video, such as Figure 4 Among them, there is a target video composed of concept video 1 and the video of question 1, another target video composed of concept video 1 and the video of question 2, and another target video composed of concept video 1 and the video of question m.

[0070] Optionally, the obtaining of at least two video content types includes: inputting the original video set and the type identification prompt word into the multimodal large model to obtain at least two video content types output by the multimodal large model. The type identification prompt word is a prompt word that prompts the multimodal large model, and the type identification prompt word is used to prompt the multimodal large model to identify the video content type of each original video in the original video set.

[0071] For an original video set whose video content types include concept types and topic types, a to-be-processed video set including at least one to-be-processed video is obtained based on the video content types and the original video set, including: in response to the original videos in the original video set being concept explanation videos including at least two concepts, identifying the concept name and the number of concepts in each concept explanation video; based on the number of concepts, segmenting the concept explanation video to obtain at least one first segmented video segment; based on the concept name, clustering the first segmented video segments; based on the clustering results of the first segmented video segments, identifying the corresponding video content type for each clustered segment video, to obtain a processed video set including at least one processed video.

[0072] In this optional implementation, the concept explanation video is a video that explains different concepts in the video knowledge points. Generally, one video knowledge point can correspond to at least one concept. Therefore, the concept explanation video including at least two concepts can be a video corresponding to one video knowledge point or a video corresponding to multiple video knowledge points.

[0073] Optionally, the above-mentioned process of obtaining a set of videos to be processed including at least one video to be processed based on the video content type and the original video set also includes: in response to the original videos in the original video set being mixed explanation videos including at least one concept explanation video and a question explanation video, obtaining a concept segmentation video and a question segmentation video based on the mixed explanation video; identifying the concept name and the number of concepts in the concept segmentation video; in response to the number of concepts being multiple, segmenting the concept segmentation video based on the number of concepts to obtain at least one second segmentation video segment; clustering the second segmentation video segments based on the concept name; obtaining at least one clustered video based on the clustering results of the second segmentation video segments; identifying the corresponding video content type for each clustered video; in response to the number of questions in the question segmentation video being one, identifying the title video content type for the question segmentation video; using the identified clustered video and the identified question segmentation video as processing videos respectively, to obtain a processing video set including at least one processing video.

[0074] In this optional implementation, the mixed explanation video is a video that has both a concept explanation part and a question explanation part. After the mixed explanation video is split, a concept explanation video and a question explanation video can be obtained.

[0075] Optionally, the above-mentioned process of obtaining a set of videos to be processed including at least one video to be processed based on the video content type and the original video set also includes: obtaining a concept segmentation video and a question segmentation video based on the mixed explanation video; identifying the concept name and the number of concepts in the concept segmentation video; in response to the number of concepts being one, identifying the corresponding video content type for the concept segmentation video; in response to the number of questions in the question segmentation video being one, identifying the question video content type for the question segmentation video; and using the annotated concept segmentation video and the annotated question segmentation video as processing videos, respectively, to obtain a set of processed videos including at least one processed video.

[0076] In some optional implementations of the present disclosure, the above-mentioned obtaining video knowledge points of each video to be processed based on the video set to be processed includes: inputting the video set to be processed and knowledge point prompt words into the knowledge point extraction big model, and obtaining the video knowledge points of each video to be processed output by the knowledge point extraction big model.

[0077] In this optional implementation, the knowledge point prompt word is a prompt word for prompting the multimodal large model, and the knowledge point prompt word is used to prompt the multimodal large model to identify the video knowledge points of each video to be processed in the set of videos to be processed.

[0078] The method for determining video knowledge points provided by this optional implementation method analyzes the video set to be processed through a large knowledge point extraction model to obtain the video knowledge points of each video to be processed, thereby improving the accuracy of obtaining the video knowledge points.

[0079] In some optional implementations of the present disclosure, the above-mentioned processing of the video set to be processed based on the video content type and the video knowledge point to generate the target video set includes: splicing the videos to be processed of different video content types with the same video knowledge points in the video set to be processed to obtain a target video set including at least one target video.

[0080] In this optional implementation, the videos are combined based on the video types and video knowledge points of each video to be processed to obtain a target video.

[0081] The method for generating a target video set provided by this optional implementation mode splices videos of different video content types with the same video knowledge points in the to-be-processed video set to obtain a target video set including at least one target video, and provides a reliable implementation mode for obtaining the target video set. Since the target video is obtained by directly combining the videos to be processed, the target video can include video information of multiple content types, thereby improving the reliability and accuracy of obtaining the target video set.

[0082] Optionally, based on the video content type and video knowledge points, the video set to be processed is processed to generate a target video set, including: clipping blank parts of each video to be processed in the video set to obtain a clipped video set, splicing clipped videos in the clipped video set with the same video knowledge points and different video types, to obtain a target video set including at least one target video, in which case the target video is a clipped video with video knowledge points and video types.

[0083] Optionally, for courseware explanation videos, the video set to be processed is processed based on the video content type and the video knowledge points to generate a target video set including: splicing the videos to be processed of the concept type and the videos to be processed of the question type that have the same video knowledge points in the video set to be processed to obtain a target video set including at least one target video, in which the target video has video segments of the concept type and video segments of the question type.

[0084] In some embodiments of the present disclosure, the above method further includes: sending the target video in the target video set to the user terminal.

[0085] In this embodiment, the user terminal is a device terminal used by the user, and there may be multiple user terminals. Since each user has video knowledge points of interest, corresponding video knowledge points may be set for each user terminal.

[0086] In this embodiment, after the target video is obtained, the target video corresponding to the video knowledge point can be sent to the user terminal based on the video knowledge point corresponding to the user terminal.

[0087] In this embodiment, the target video in the target video set is sent to the user terminal, so that the user of the user terminal can watch the target video, thereby improving the user's understanding of the video knowledge points in the target video.

[0088] Figure 5 A process 500 according to an embodiment of the information processing method of the present disclosure is shown. The information processing method includes the following steps:

[0089] Step 501: Obtain at least two video content types.

[0090] Step 502: Based on the video content type and the original video set, a to-be-processed video set including at least one to-be-processed video is obtained.

[0091] In this embodiment, each to-be-processed video corresponds to a video content type.

[0092] Step 503: based on the set of videos to be processed, obtain the video knowledge points of each video to be processed.

[0093] Step 504: based on the video content type and the video knowledge points, the to-be-processed video set is processed to generate a target video set.

[0094] In this embodiment, it should be understood that the operations and features in the above steps 501 to 504 correspond to the operations and features in steps 101 to 104, respectively. Therefore, the above descriptions of the operations and features in steps 101 to 104 are also applicable to steps 501 to 504 and will not be repeated here.

[0095] Step 505: Generate a knowledge question set based on the target video set.

[0096] In this embodiment, the video content of each target video in the target video set can be analyzed to determine the knowledge questions matching the video content of each target video, and the knowledge questions of all target videos can be collected to obtain a knowledge question set, where the knowledge questions can be units in a question library.

[0097] In this embodiment, the target video set and question generation prompt words can be input into the multimodal big model to obtain a knowledge question set output by the multimodal big model, wherein the multimodal big model is pre-trained and is used to generate questions for the video content of the target video.

[0098] Step 506: Generate a knowledge base based on the knowledge question set and the target video set.

[0099] In obtaining the knowledge question set, target videos in the target video set and knowledge questions in the knowledge question set are stored correspondingly according to the library storage rule to obtain a knowledge base.

[0100] The video processing method provided in this embodiment generates a knowledge question set based on the target video set after obtaining the target video set; generates a knowledge base and a target video under each knowledge point based on the knowledge question set and the target video set, thereby ensuring the systematicness and logic of the knowledge structure in the knowledge base; and determines each target video through knowledge questions, thereby improving the use value of the knowledge base and user experience.

[0101] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a video processing device, which is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically implemented in various electronic devices.

[0102] like Figure 6 As shown, the video processing device 600 provided in this embodiment includes: an acquisition unit 601, a video acquisition unit 602, a knowledge point acquisition unit 603, and a generation unit 604. The acquisition unit 601 can be configured to acquire at least two video content types. The video acquisition unit 602 can be configured to obtain a set of videos to be processed including at least one video to be processed based on the video content type and the original video set, and each video to be processed corresponds to a video content type. The knowledge point acquisition unit 603 can be configured to obtain video knowledge points of each video to be processed based on the set of videos to be processed. The generation unit 604 can be configured to process the set of videos to be processed based on the video content type and the video knowledge points to generate a target video set.

[0103] In this embodiment, the specific processing of the acquisition unit 601, the video acquisition unit 602, the knowledge point acquisition unit 603, and the generation unit 604 and the technical effects thereof can be referred to in detail. Figure 1 The relevant descriptions of step 101, step 102, step 103, and step 104 in the corresponding embodiment are not repeated here.

[0104] In some optional implementations of the present disclosure, the apparatus 600 further includes: a sending unit (not shown in the figure), and the sending unit is configured to send the target video in the target video set to the user terminal.

[0105] In some optional implementations of the present disclosure, the above-mentioned device 600 also includes: a library unit (not shown in the figure), and the above-mentioned library unit is configured to generate a knowledge question set based on the target video set; and generate a knowledge base based on the knowledge question set and the target video set.

[0106] In some optional implementations of the present disclosure, the video obtaining unit 602 is configured to: cluster the original videos in the original video set to obtain clustering information; based on the clustering information, label the video content type of the original videos that belong to only one clustering type; based on the clustering information, detect whether there are original videos belonging to different clustering types; in response to detecting original videos belonging to different clustering types, segment the original video according to the clustering type to obtain at least one segmented video segment; label each segmented video segment with a video content type to obtain at least one video to be processed.

[0107] In some optional implementations of the present disclosure, the video obtaining unit 602 is configured to: obtain a pre-trained large model; based on the large model, obtain the video content type of each original video in the original video set; based on the original video set and the video content type of each original video, obtain a video set to be processed including at least one video to be processed.

[0108] In some optional implementations of the present disclosure, the video obtaining unit 602 is configured to: obtain at least one video feature of the original video set; based on the video feature and the video content type of each original video, process the original videos in the original video set to obtain a video set to be processed.

[0109] In some optional implementations of the present disclosure, the above-mentioned video features include video picture features and video audio features, and the above-mentioned video obtaining unit 602 is configured to: for the original video in the original video set, based on the video picture features, slice the original video to obtain at least one picture video segment; based on the video audio features, slice the original video to obtain at least one audio and video segment; align and merge at least one video segment and at least one audio and video segment to obtain at least one merged video; based on the video content type of the original video, identify and label the video content type of each merged video in at least one merged video to obtain a processed video set.

[0110] In some optional implementations of the present disclosure, the acquisition unit 601 is configured to: acquire the video content type of the courseware explanation video, and the video content type at least includes: concept type and question type.

[0111] In some optional implementations of the present disclosure, the above-mentioned knowledge point obtaining unit 603 is configured to: input the video set to be processed and the knowledge point prompt words into the knowledge point extraction model to obtain the video knowledge points of each video to be processed output by the knowledge point extraction model.

[0112] In some optional implementations of the present disclosure, the generation unit 604 is configured to: piece together the to-be-processed videos of different video content types having the same video knowledge points in the to-be-processed video set to obtain a target video set including at least one target video.

[0113] The video processing device provided by the embodiment of the present disclosure, first, the acquisition unit 601 acquires at least two video content types; secondly, the video acquisition unit 602 obtains a to-be-processed video set including at least one to-be-processed video based on the video content type and the original video set, and each to-be-processed video corresponds to a video content type; thirdly, the knowledge point acquisition unit 603 obtains the video knowledge points of each to-be-processed video based on the to-be-processed video set, and finally, the generation unit 604 processes the to-be-processed video set based on the video content type and the video knowledge points to generate a target video set. Thus, the video type and the video knowledge points are used as the association points of the to-be-processed videos in the to-be-processed video set, and target videos of different types with the same video knowledge points are generated, which pays more attention to the internal logic and association of the knowledge in the video, and realizes the structured processing of the video; when the target video is recommended to the user, the user experience can be improved.

[0114] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0115] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0116] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0117] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0118] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as video processing methods. For example, in some embodiments, the video processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the video processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the video processing method in any other appropriate manner (e.g., by means of firmware).

[0120] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable video processing device, so that the program code, when executed by the processor or controller, causes the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0122] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an information server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0125] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0126] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0127] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A video processing method, wherein: The method comprises: Obtain at least two video content types; Based on the video content type and the original video set, a to-be-processed video set is obtained, which includes at least one to-be-processed video, each to-be-processed video corresponding to a video content type; Based on the set of videos to be processed, obtaining video knowledge points of each video to be processed; Based on the video content type and the video knowledge points, the to-be-processed video set is processed to generate a target video set.

2. The method according to claim 1, further comprising: The target video in the target video set is sent to the user terminal.

3. The method according to claim 1, further comprising: Based on the target video set, generating a knowledge question set; A knowledge base is generated based on the knowledge question set and the target video set.

4. The method according to any one of claims 1 to 3, wherein: The step of obtaining a to-be-processed video set including at least one to-be-processed video based on the video content type and the original video set comprises: Clustering the original videos in the original video set to obtain clustering information; Based on the clustering information, labeling the video content type of the original video that belongs to only one clustering type; Based on the clustering information, detecting whether there are original videos belonging to different clustering types; In response to detecting an original video belonging to a different cluster type, segmenting the original video according to the cluster type to obtain at least one segmented video segment; The video content type is labeled for each segmented video segment to obtain a to-be-processed video set including at least one to-be-processed video.

5. The method according to any one of claims 1 to 3, wherein obtaining a to-be-processed video set including at least one to-be-processed video based on the video content type and the original video set comprises: Get a pre-trained large model; Based on the large model, obtaining the video content type of each original video in the original video set; Based on the original video set and the video content type of each original video, a video set to be processed is obtained.

6. The method according to claim 5, wherein obtaining the to-be-processed video set based on the original video set and the video content type of each original video comprises: Acquire at least one video feature of the original video set; Based on the video features and the video content type of each original video, the original videos in the original video set are processed to obtain a video set to be processed.

7. The method according to claim 6, wherein: The video features include video picture features and video audio features, and the processing of the original videos in the original video set based on the video features and the video content type to obtain the to-be-processed video set includes: for the original videos in the original video set, based on the video picture features, slicing the original videos to obtain at least one picture video segment; Based on the video and audio features, slicing the original video to obtain at least one audio and video segment; aligning and merging the at least one video segment and the at least one audio and video segment to obtain at least one merged video; Based on the video content type of the original video, the video content type of each merged video in the at least one merged video is identified and labeled to obtain a processed video set.

8. According to the method according to any one of claims 1 to 3, obtaining at least two video content types comprises: The video content type of the courseware explanation video is obtained, and the video content type at least includes: concept type and question type.

9. The method according to any one of claims 1 to 3, wherein: The step of obtaining the video knowledge points of each video to be processed based on the video set to be processed includes: The video set to be processed and the knowledge point prompt words are input into the knowledge point extraction model to obtain the video knowledge points of each video to be processed output by the knowledge point extraction model.

10. The method according to any one of claims 1 to 3, wherein: The processing of the to-be-processed video set based on the video content type and the video knowledge points to generate a target video set includes: The videos to be processed of different video content types having the same video knowledge points in the set of videos to be processed are spliced ​​to obtain a target video set including at least one target video.

11. A video processing device, comprising: an acquisition unit, configured to acquire at least two video content types; A video obtaining unit is configured to obtain a to-be-processed video set including at least one to-be-processed video based on the video content type and the original video set, wherein each to-be-processed video corresponds to a video content type; A knowledge point obtaining unit is configured to obtain video knowledge points of each video to be processed based on the set of videos to be processed; The generating unit is configured to process the to-be-processed video set based on the video content type and the video knowledge points to generate a target video set.

12. The device according to claim 11, further comprising: The sending unit is configured to send the target video in the target video set to the user terminal.

13. The device according to claim 11, further comprising: A library unit is configured to generate a knowledge question set based on the target video set; A knowledge base is generated based on the knowledge question set and the target video set.

14. The device according to any one of claims 11 to 13, wherein: The video obtaining unit is configured to: cluster the original videos in the original video set to obtain clustering information; and based on the clustering information, mark the video content type of the original videos belonging to only one clustering type; Based on the clustering information, detect whether there are original videos belonging to different clustering types; in response to detecting the original videos belonging to different clustering types, segment the original video according to the clustering type to obtain at least one segmented video segment; and label each segmented video segment with a video content type to obtain at least one video to be processed.

15. The device according to any one of claims 11 to 13, wherein: The video acquisition unit is configured to: acquire a pre-trained large model; based on the large model, acquire the video content type of each original video in the original video set; based on the original video set and the video content type of each original video, acquire a to-be-processed video set including at least one to-be-processed video.

16. The device according to claim 15, wherein: The video obtaining unit is configured to: obtain at least one video feature of the original video set; Based on the video features and the video content type of each original video, the original videos in the original video set are processed to obtain a video set to be processed.

17. The device according to claim 16, wherein: The video features include video picture features and video audio features, and the video obtaining unit is configured to: for an original video in the original video set, based on the video picture features, perform video slicing on the original video to obtain at least one picture video segment; Based on the video and audio features, slicing the original video to obtain at least one audio and video segment; Align and merge the at least one video segment and the at least one audio and video segment to obtain at least one merged video; based on the video content type of the original video, identify and label the video content type of each merged video in the at least one merged video to obtain a processed video set.

18. The device according to any one of claims 11 to 13, wherein: The acquisition unit is configured to: acquire the video content type of the courseware explanation video, and the video content type at least includes: concept type and question type.

19. The device according to any one of claims 11 to 13, wherein: The knowledge point obtaining unit is configured to: input the video set to be processed and the knowledge point prompt words into the knowledge point extraction model to obtain the video knowledge points of each video to be processed output by the knowledge point extraction model.

20. The device according to any one of claims 11 to 13, wherein: The generating unit is configured to: splice the to-be-processed videos of different video content types having the same video knowledge points in the to-be-processed video set to obtain a target video set including at least one target video.

21. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

23. A computer program product, comprising a computer program, which, when executed by a processor, implements the method of any one of claims 1 to 10.

Images