Video processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a video processing method and device, electronic equipment and a storage medium, video text associated information and video editing guidance information of a target video can be acquired, and the video text associated information at least comprises a video text; inputting the video text association information and the video editing guidance information into a large language model; determining a target text required by an edited video in the video text association information based on the video editing guidance information through the large language model, and obtaining video positioning information of the target text; and clipping an edited video matched with the video positioning information from the target video, thereby improving the editing efficiency of the target video.

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to a video processing method, device, electronic device and storage medium. Background Art

[0002] The existing method of editing existing videos according to different requirements is mainly to edit the existing videos frame by frame through editing software, which is inefficient. Summary of the invention

[0003] The embodiments of the present application provide a video processing method, device, electronic device and storage medium, which can improve the editing efficiency of a target video.

[0004] In a first aspect, an embodiment of the present application provides a video processing method, including:

[0005] Acquire video-text association information and video editing guidance information of a target video, wherein the video-text association information at least includes the video text;

[0006] Inputting the video text association information and the video editing guidance information into a large language model;

[0007] Determine the target text required for the video after editing in the video-text association information through the large language model based on the video editing guidance information, and obtain video positioning information of the target text;

[0008] A clipped video matching the video positioning information is cropped from the target video.

[0009] In a second aspect, an embodiment of the present application further provides a video processing device, including:

[0010] A target video information acquisition module, used to acquire video text association information and video editing guidance information of a target video, wherein the video text association information at least includes video text;

[0011] An input module, used for inputting the video text association information and the video editing guidance information into a large language model;

[0012] A video positioning information determination module, configured to determine the target text required for the video after editing in the video-text association information through the large language model based on the video editing guidance information, and obtain the video positioning information of the target text;

[0013] The editing module is used to cut out an edited video matching the video positioning information from the target video.

[0014] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the video processing methods provided in the embodiments of the present application.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, comprising a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the video processing methods provided in the embodiments of the present application.

[0016] In a fifth aspect, an embodiment of the present application also provides a computer program product, comprising a computer program, wherein the computer program is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of any one of the video processing methods provided in the embodiments of the present application.

[0017] By adopting the scheme of the embodiment of the application, the video-text association information and video editing guidance information of the target video can be obtained, wherein the video-text association information at least includes the video text; the video-text association information and the video editing guidance information are input into a large language model; the target text required for the edited video in the video-text association information is determined by the large language model based on the video editing guidance information, and the video positioning information of the target text is obtained; and the edited video matching the video positioning information is cropped from the target video, thereby improving the editing efficiency of the target video. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a schematic diagram of a system architecture of the application of the video processing method provided in the embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of the data interaction flow between a terminal and a server in a video processing method provided in an embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of an embodiment of the video processing method provided in the embodiments of the present application;

[0022] Figure 4 It is a schematic flow chart of a specific embodiment of the video processing method provided in the embodiments of the present application;

[0023] Figure 5 is a schematic diagram of the structure of a video processing device provided in an embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0026] At the same time, in the description of the embodiments of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0027] Embodiments of the present application provide a video processing method, device, electronic device, and computer-readable storage medium.

[0028] The video processing method provided in the embodiment of the present application can be applied to the system architecture 100. Figure 1 ,like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0029] Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages, etc.

[0030] The terminal devices 101, 102, 103 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Eperts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Eperts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) players, laptop computers and desktop computers, etc. The terminal may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. The terminal and the server may perform two-way communication via a network.

[0031] The server 105 may be a server that provides various services, such as a background server that supports the pages displayed on the terminal devices 101, 102, and 103. The server may be an independent server, or a server network or server cluster composed of servers, including but not limited to a computer, a network host, a single network server, a plurality of network server sets, or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0032] It should be noted that the video processing method provided in the embodiment of the present application is executed by a server, and accordingly, the video processing device is arranged in the server.

[0033] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. The terminal devices 101, 102, 103 in the embodiment of the present application may specifically correspond to application systems in actual production.

[0034] like Figure 2 As shown, Figure 2The schematic diagram of data interaction between the terminal and the server. The terminal can receive the target video and video editing guidance information, and send the target video to the server. The server determines the video text association information of the target video through automatic speech recognition technology, sends the video text association information and video editing guidance information to the large language model, determines the target text required for the edited video in the video text association information based on the video editing guidance information through the large language model, obtains the video positioning information of the target text, returns the video positioning information to the server, and the server inputs the video positioning information and the target video into the video clipper, so that the video clipper cuts out the edited video matching the video positioning information from the target video, and returns the edited video to the server, and the server returns the edited video to the terminal.

[0035] The following is a detailed description with reference to the accompanying drawings. In this embodiment, the execution subject is an electronic device that can obtain the target video. It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments. Although the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different from that shown in the accompanying drawings.

[0036] Please refer to Figure 3 The specific process of the video processing method can be as follows: Step 201 to Step 204, wherein:

[0037] Step 201: Obtain video-text association information and video editing guidance information of a target video, where the video-text association information at least includes the video text.

[0038] In the embodiment of the present application, the target video represents a to-be-processed video that needs to be edited. The target video can be captured by a video capture device or obtained from a storage space that has pre-stored videos.

[0039] In the embodiment of the present application, the target video may include audio data and video frame data. The target video may also include only video frame data, where the video frame represents an image at a certain time point of the target video, and the image where the video frame is located includes text information.

[0040] In the embodiment of the present application, the video text associated information refers to the information associated with the video text of the target video. The video text associated information includes but is not limited to the video text and the timestamp information of the video text. Among them, the timestamp information of the video text refers to the time information used to record the appearance and disappearance of the video text. It can be understood that the video text and the timestamp information of the video text are in a corresponding relationship. That is, if the video text is known, the timestamp information of the video text can be determined. If the timestamp information of the video text is known, the video text corresponding to the timestamp information can be determined.

[0041] In an embodiment of the present application, the process of obtaining the video text of the target video may be: obtaining a video frame of the target video, performing text recognition on the area where the text of each video frame is located, and using the obtained text information as the video text of the target video. The process of obtaining the video text of the target video may also be: based on automatic speech recognition technology (ASR, Automatic Speech Recognition), converting the audio data in the target video into video text. The process of obtaining the video text of the target video can be adjusted according to actual conditions, and the embodiment of the present application does not limit it.

[0042] In an embodiment of the present application, the process of obtaining the timestamp information of the video text of the target video may be to obtain the timestamp information of the video frame where the video text is located as the timestamp information of the video text. The process of obtaining the timestamp information of the video text of the target video may also be to use the timestamp information of the audio converted into the video text as the timestamp information of the video text based on the automatic speech recognition technology.

[0043] Exemplarily, the video text and the video stamp information corresponding to the video text are obtained through automatic speech recognition technology. For example, in the target video, an audio data segment of "If you have acne" appears, the start time parameter of the audio data segment is "390", and the end time parameter of the video text is "1120". Through the automatic speech recognition technology, the audio data segment of "If you have acne" is recognized, and a video text "If you have acne" and the timestamp information corresponding to the video text (390, 1120) are determined, where "390" is the start time parameter of the video text, and "1120" is the end time parameter of the video text.

[0044] In an embodiment of the present application, the video editing guidance information can be used to indicate an editing scheme for a target video. The video editing guidance information includes at least one video text editing rule, which can be a rule for editing based on the video text of the target video, or a rule for restricting the text of the video after editing.

[0045] Exemplarily, the video text editing rule may be an adding rule. For example, there is an adding rule, and the adding rule is "opening remarks, summarized in one sentence, such as: keep this XXX strategy", then an adding positioning information may be generated in the video positioning information, and the adding positioning information is the first frame of the video after editing, and the content of the first frame is "keep this XXX strategy", wherein XXX may replace XXX by identifying the theme content of the target video.

[0046] Exemplarily, the video text clipping rule may also be a deletion rule. The deletion rule may be to delete the video text identified as specific content. For example, to delete the video text identified as advertising information. For another example, to delete the video text identified as redundant information. The deletion rule may also be to delete the video text including specific content. For example, to delete the video text including XX product.

[0047] Exemplarily, the video text editing rule may also be a limiting rule, for example, limiting the target text to retain only one strategy.

[0048] In an embodiment of the present application, the video clipping guidance information can also be used for the output format and content requirements of the video positioning information. For example, the output format of the video positioning information includes only the target text. For another example, the output format of the video positioning information includes only the timestamp information of the target text. For another example, the output format of the video positioning information includes only the timestamp information of the target text, and the timestamp information is output in the format of a timeline.

[0049] It is understandable that the specific content of the video text clipping rules, the output format and content requirements of the video positioning information can be adjusted according to actual conditions, and the embodiments of the present application do not limit this.

[0050] Step 202: Input the video text association information and the video editing guidance information into the large language model.

[0051] In the embodiments of the present application, a large language model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. The large language model can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc.

[0052] In an embodiment of the present application, specifically, video text and video editing guidance information are input into a large language model, or video text, timestamp information corresponding to the video text, and video editing guidance information are input into a large language model.

[0053] In the embodiment of the present application, the large language model understands the content of the target video through the video text association information. Different video editing guidance information can be designed for different editing requirements. The video editing guidance information is a structured input for guiding the large language model to input an editing scheme for the target video.

[0054] Step 203: Determine the target text required for the edited video in the video-text association information through the large language model based on the video editing guidance information, and obtain the video positioning information of the target text.

[0055] In the embodiment of the present application, if the video text association information only includes the video text, in the large language model, the target text required for the video after editing in the video text is determined according to the video editing guidance information. The video positioning information of the target text is the target text.

[0056] In an embodiment of the present application, if the video text association information includes the video text and the timestamp information corresponding to the video text, in the large language model, according to the video editing guidance information, the target text required for the video after editing in the video text is determined, and the video positioning information of the target text may be only the target text. The video positioning information of the target text may be only the target timestamp information of the target text. The video positioning information of the target text may also include the target text and the target timestamp information of the target text at the same time.

[0057] In an embodiment of the present application, if the video editing guidance information includes at least one video text editing rule and the output format of the video positioning information, the target text required for the video after editing in the video text association information is determined by the large language model based on the video editing guidance information, and the video positioning information of the target text is obtained, including:

[0058] Based on the video text clipping rules, the large language model is used to determine the target text in the video text association information that meets the video text clipping rules, and obtain the target timestamp information corresponding to the target text, or the target text and the target timestamp information corresponding to the target text.

[0059] Video positioning information that conforms to the output format is selected from the target timestamp information corresponding to the target text, or from the target text and the target timestamp information corresponding to the target text.

[0060] Step 204: Crop the edited video that matches the video positioning information from the target video.

[0061] In an embodiment of the present application, if the video text association information only includes the video text, the video positioning information is the target text itself. The process of cutting out the edited video matching the video positioning information from the target video is as follows: cutting out the video segment containing the video text matching the target text from the target video; and fusing the video segments to obtain the edited video.

[0062] In this embodiment, the acquired video clips can be merged according to the order of timestamp information of the video texts matched by the target text in the target video to obtain the edited video. The video clips can also be merged according to a custom fusion order to obtain the edited video.

[0063] In an embodiment of the present application, if the video text association information includes the video text and the timestamp information corresponding to the video text, the video positioning information includes the target timestamp information, or the video positioning information includes the target text and the target timestamp information corresponding to the target text.

[0064] A clipped video matching the video positioning information is cropped from the target video, including:

[0065] cropping at least one video segment corresponding to the target timestamp information from the target video;

[0066] Based on the order of the target timestamp information, all video clips are merged to obtain the edited video.

[0067] In this embodiment, the target timestamp information includes a target start time and a target end time. The process of cutting the video segment corresponding to the target timestamp information from the target video may be to determine the video segment where the target start time and the target end time are located in the target video, and cut the video segment out of the target video. Alternatively, determine the video segment where the target start time and the target end time are located in the target video, and copy the video segment to achieve the effect of cutting out the video segment corresponding to the target timestamp information from the target video.

[0068] In an embodiment of the present application, the video processing method further includes:

[0069] Perform segment integrity detection on video segments to determine complete segments and incomplete segments;

[0070] If there are incomplete fragments, the incomplete fragments are processed to obtain processed fragments.

[0071] In this embodiment, integrity detection refers to the process of detecting whether a video segment is a complete segment.

[0072] In this embodiment, the integrity detection may be to detect whether there is shot split in the video clip. If there is shot split in the video clip, the video clip is determined to be an incomplete clip. If there is no shot split in the video clip, the video clip is determined to be a complete clip.

[0073] In this embodiment, the integrity check may also be performed by selecting at least one video frame from the first frame of the video segment, the sub-segment where the first frame is located, the last frame, and the sub-segment where the last frame is located, and calculating the similarity between the selected video frame and the target video frame, and if the similarity is higher than a preset similarity, determining that the video segment is an incomplete segment. The target video frame is a video frame that is adjacent to the selected video frame and does not belong to the video segment.

[0074] It is understandable that the integrity check of the video clips is because the large model directly outputs the target text, or the target timestamp information of the target text, or the target text and the target timestamp information of the target text. When the target video is directly edited and spliced ​​based on the above information, abnormal frames such as flash frames and black frames may appear. For example, the video clip is only a partial clip, the shot where the video clip is located has not ended, and the shot where the video clip is located has ended. That is, the video clip is an incomplete clip.

[0075] In this embodiment, the number of incomplete segments may be zero, and the number of incomplete segments may be the same as the number of video segments processed by cropping. That is, the number of incomplete segments is at most the same as the number of video segments processed by cropping. The number of complete segments may be zero, and the number of complete segments may be the same as the number of video segments processed by cropping. That is, the number of complete segments is at most the same as the number of video segments processed by cropping. It can be understood that the sum of the number of complete segments and incomplete segments is equal to the number of video segments processed by cropping.

[0076] In the embodiment of the present application, all video clips are merged based on the order of target timestamp information, and the edited video may also include:

[0077] Based on the order of the target timestamp information, the processed segments and the complete segments are merged to obtain the edited video.

[0078] In this embodiment, if there are incomplete fragments, the number of processed fragments is consistent with the number of incomplete fragments. If there are no incomplete fragments, the number of processed fragments is zero.

[0079] In the embodiment of the present application, the incomplete fragment is processed to obtain the processed fragment, including:

[0080] Get the shot cut position of the incomplete clip;

[0081] Determine the target sub-segment with a longer duration in the two video sub-segments before and after the shot split position of the incomplete segment;

[0082] Cut out the target sub-segment from the incomplete segment;

[0083] The duration of the target sub-segment is processed so that the duration of the target sub-segment is consistent with the duration of the incomplete segment, and the processed target sub-segment is used as the processed segment.

[0084] In this embodiment, the shot segmentation position refers to a segmentation position at which the video content can be further analyzed and indexed using the detected shot as a basic unit.

[0085] It is understandable that if there is no shot splitting position in a video clip, it means that the video clip is a complete clip, and there are no flash frames and black frames in the video clip. If there is a shot splitting position in a video clip, it means that the video clip is an incomplete clip, and the flash frames and black frames divide the complete video clip into two parts. It is understandable that the duration of the video sub-segment corresponding to the flash frames and black frames is very short, and in the video clip, the duration of the flash frames and black frames is usually just a flash.

[0086] In this embodiment, the method of making the duration of the target sub-segment consistent with the duration of the incomplete segment includes but is not limited to magnification adjustment and supplementary video frames. The specific method of making the duration of the target sub-segment consistent with the duration of the incomplete segment can be adjusted according to actual conditions, and the embodiment of the present application does not limit it.

[0087] Exemplarily, the process of processing the duration of the target sub-segment so that the duration of the target sub-segment is consistent with the duration of the incomplete segment, and using the processed target sub-segment as the processed segment may be: determining the duration of the incomplete segment and the duration of the target sub-segment; adjusting the duration of the target sub-segment by a magnification so that the duration of the target sub-segment is consistent with the duration of the incomplete segment. For example, if the duration of the target sub-segment is 4 seconds and the duration of the incomplete segment is 5 seconds, the playback magnification of the target sub-segment is adjusted to 1.25 times, that is, the target sub-segment is played slowly at a magnification of 1.25.

[0088] Exemplarily, the duration of the target sub-segment is processed to make the duration of the target sub-segment consistent with the duration of the incomplete segment, and the process of using the processed target sub-segment as the processed segment can be: determining the difference between the duration of the incomplete segment and the duration of the target sub-segment, determining the number of video frames required for the difference, and if the number of video frames is lower than a preset threshold, selecting the first frame or the last frame of the target sub-segment for repetition, so as to achieve the consistency of the duration of the target sub-segment with the duration of the incomplete segment in effect.

[0089] In the embodiment of the present application, the video editing guidance information includes at least one video text editing rule, and the output format of the video positioning information is determined by a large language model based on the video editing guidance information to determine the target text required for the video after editing in the video text association information, and obtain the video positioning information of the target text, including:

[0090] Determine the target text that meets the video text clipping rule in the video text association information based on the video text clipping rule through the large language model, and obtain the target timestamp information corresponding to the target text, or the target text and the target timestamp information corresponding to the target text;

[0091] Video positioning information that conforms to the output format is selected from the target timestamp information corresponding to the target text, or from the target text and the target timestamp information corresponding to the target text.

[0092] In this embodiment, the output format of the video positioning information is the target timestamp information, and cutting at least one video segment corresponding to the target timestamp information from the target video includes:

[0093] inputting target timestamp information into a predetermined video editor;

[0094] The target video is edited on the time axis by a predetermined video editor based on a video editing method corresponding to the output format to obtain video clips corresponding to each target time stamp.

[0095] In this embodiment, the predetermined video editor is a customized video editor. The predetermined video editor may be a video editor configured with a video editing script, and the video editing script may be used to receive video positioning information in a predetermined output format, that is, target timestamp information.

[0096] In this embodiment, the video editing method corresponding to the output format can be achieved by determining the timeline of the target video and the edited timeline corresponding to the target timestamp information, aligning the timeline of the target video with the edited timeline corresponding to the target timestamp information, and editing the overlapping parts of the two timelines to obtain video clips corresponding to each target timestamp.

[0097] In this embodiment, the video editing method corresponding to the output format can be achieved by determining the editing timeline corresponding to the target timestamp information, and editing the video segments in the target video that meet the editing timeline to obtain video segments corresponding to each target timestamp.

[0098] In this embodiment, the output format of the video positioning information is the target text, and cutting at least one video segment corresponding to the target timestamp information from the target video includes:

[0099] Input the target text into the predetermined video editor;

[0100] The video text of the target video is edited by a predetermined video editor based on a video editing method corresponding to the output format to obtain video clips corresponding to each target text.

[0101] In this embodiment, the predetermined video editor may be a video editor configured with a video editing script, and the video editing script may be used to receive video positioning information in a predetermined output format, ie, target text.

[0102] In this embodiment, the video clipping method corresponding to the output format can be performed by determining the target timestamp information of the target text in the target video, and based on the target timestamp information, clipping the video segments corresponding to each target text from the target video.

[0103] In this embodiment, the video editing method corresponding to the output format can determine the timeline of the target text and the timeline of the target video, align the timeline of the target text and the timeline of the target video, and edit the overlapping parts of the two timelines to obtain video clips corresponding to each target timestamp.

[0104] The present application is explained below with a specific embodiment. Figure 4 , Figure 4 The present invention is a flowchart of a specific embodiment of the video processing method provided in the embodiments of the present application. Figure 4 For advertising video editing application scenarios. Figure 4 As shown, video materials that meet the requirements are pulled from the database. The requirements can be adjusted according to actual conditions, and the number of video materials is not limited. The video material is the target video in this application. Through ASR technology, the video text and the corresponding timestamp corresponding to each video material are obtained, and the video text and the corresponding timestamp are converted into structured text, which are input into the large language model. Through the output information of the large language model, the corresponding video editing information is obtained, and each video material is edited into the advertising video corresponding to the video material through the editing script and editing information.

[0105] By adopting the scheme of the embodiment of the application, the video-text association information and video editing guidance information of the target video can be obtained, wherein the video-text association information at least includes the video text; the video-text association information and the video editing guidance information are input into a large language model; the target text required for the edited video in the video-text association information is determined by the large language model based on the video editing guidance information, and the video positioning information of the target text is obtained; and the edited video matching the video positioning information is cropped from the target video, thereby improving the editing efficiency of the target video.

[0106] Furthermore, by adopting the solution of the embodiment of the present application, the video content can be streamlined, the core information can be highlighted while the advertising information and redundant information can be deleted, making it suitable for production as an advertisement.

[0107] Furthermore, by adopting the solution of the embodiment of the present application, batches of target videos can be processed, the production efficiency of edited videos can be improved, and labor costs can be reduced.

[0108] This embodiment further provides a video processing device, which can be integrated into an electronic device, such as a computer device. The computer device can be a terminal, a server, or other device, and this embodiment has no limitation on this.

[0109] For example, Figure 5 As shown, the video processing device may include:

[0110] The target video information acquisition module 301 is used to acquire the video-text association information and video editing guidance information of the target video, where the video-text association information at least includes the video text.

[0111] The input module 302 is used to input the video text association information and the video editing guidance information into the large language model.

[0112] The video positioning information determination module 303 is used to determine the target text required for the edited video in the video text association information based on the video editing guidance information through a large language model, and obtain the video positioning information of the target text.

[0113] The clipping module 304 is used to clip the target video to obtain a clipped video that matches the video positioning information.

[0114] Optionally, in the device of the embodiment of the present application, the video-text association information includes the video text and the timestamp information corresponding to the video text, the video positioning information includes the target timestamp information, or the video positioning information includes the target text and the target timestamp information corresponding to the target text.

[0115] In the clipping module 304, a clipped video matching the video positioning information is clipped from the target video, including:

[0116] The first video segment acquisition subunit is used to cut at least one video segment corresponding to the target timestamp information from the target video.

[0117] The first fusion unit is used to fuse all video clips based on the sequence of target timestamp information to obtain a clipped video.

[0118] Optionally, in the device of the embodiment of the present application, the video processing device further includes:

[0119] The integrity detection module is used to perform segment integrity detection on video segments to determine complete segments and incomplete segments.

[0120] The processed segment determination module is used to process the incomplete segment if there is an incomplete segment to obtain a processed segment.

[0121] In the first fusion unit, all video clips are fused based on the order of target timestamp information to obtain a clipped video, including:

[0122] The first fusion subunit is used to fuse the processed segments and the complete segments based on the sequence of target timestamp information to obtain a clipped video.

[0123] Optionally, in the device of the embodiment of the present application, in the processed segment determination module, processing the incomplete segment to obtain the processed segment includes:

[0124] The shot cutting position obtaining unit is used to obtain the shot cutting position of the incomplete segment.

[0125] The target sub-segment determining unit is used to determine the target sub-segment with a longer duration in the two video sub-segments before and after the shot cutting position of the incomplete segment.

[0126] The cropping unit is used to crop a target sub-segment from an incomplete segment.

[0127] The processing unit is used to process the duration of the target sub-segment so that the duration of the target sub-segment is consistent with the duration of the incomplete segment, and use the processed target sub-segment as the processed segment.

[0128] Optionally, in the device of the embodiment of the present application, the video-text association information only includes the video text; and the video positioning information is the target text itself.

[0129] In the clipping module 304, a clipped video matching the video positioning information is clipped from the target video, including:

[0130] The second video segment acquisition subunit is used to cut out the video segment containing the video text matching the target text from the target video.

[0131] The second fusion unit is used to fuse the video clips to obtain an edited video.

[0132] Optionally, in the device of the embodiment of the present application, the video editing guidance information includes at least one video text editing rule and an output format of the video positioning information.

[0133] In the video positioning information determination module 303, the target text required for the video after editing in the video text association information is determined based on the video editing guidance information through the large language model, and the video positioning information of the target text is obtained, including:

[0134] The target information determination unit is used to determine the target text that meets the video text clipping rules in the video text association information based on the video text clipping rules through a large language model, and obtain the target timestamp information corresponding to the target text, or the target text and the target timestamp information corresponding to the target text.

[0135] The video positioning information determining unit is used to select the video positioning information that conforms to the output format from the target timestamp information corresponding to the target text, or the target text and the target timestamp information corresponding to the target text.

[0136] Optionally, in the device of the embodiment of the present application, the output format of the video positioning information is the target timestamp information, and cutting at least one video segment corresponding to the target timestamp information from the target video includes:

[0137] The input unit is used to input the target time stamp information into a predetermined video clipper.

[0138] The editing unit is used to edit the target video on the timeline through a predetermined video editor based on a video editing method corresponding to the output format to obtain video clips corresponding to each target time stamp.

[0139] By adopting the scheme of the embodiment of the application, the video-text association information and video editing guidance information of the target video can be obtained, wherein the video-text association information at least includes the video text; the video-text association information and the video editing guidance information are input into a large language model; the target text required for the edited video in the video-text association information is determined by the large language model based on the video editing guidance information, and the video positioning information of the target text is obtained; and the edited video matching the video positioning information is cropped from the target video, thereby improving the editing efficiency of the target video.

[0140] Furthermore, by adopting the solution of the embodiment of the present application, the video content can be streamlined, the core information can be highlighted while the advertising information and redundant information can be deleted, making it suitable for production as an advertisement.

[0141] Furthermore, by adopting the solution of the embodiment of the present application, batches of target videos can be processed, the production efficiency of edited videos can be improved, and labor costs can be reduced.

[0142] Accordingly, an embodiment of the present application further provides an electronic device, which may be a terminal, and the terminal may be a smart phone, a tablet computer, a laptop computer, a touch screen, a game console, a personal computer (PC, Personal Computer), a personal digital assistant (Personal Digital Assistant, PDA) and other terminal devices. Alternatively, the electronic device may be a server.

[0143] like Figure 6 As shown, Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 400 includes a processor 401 having one or more processing cores, a memory 402 having one or more computer-readable storage media, and a computer program stored in the memory 402 and executable on the processor. The processor 401 is electrically connected to the memory 402. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or arrange components differently.

[0144] The processor 401 is the control center of the electronic device 400, and uses various interfaces and lines to connect various parts of the entire electronic device 400, and executes various functions and processes data of the electronic device 400 by running or loading software programs and / or units stored in the memory 402, and calling data stored in the memory 402. The processor 401 can be a processor CPU, a graphics processor GPU, a network processor (Network Processor, NP), etc., and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0145] In the embodiment of the present application, the processor 401 in the electronic device 400 will load instructions corresponding to the processes of one or more application programs into the memory 402 according to the following steps, and the processor 401 will run the application programs stored in the memory 402 to implement various functions, such as:

[0146] Acquire video-text association information and video editing guidance information of a target video, wherein the video-text association information at least includes the video text;

[0147] Inputting the video text association information and the video editing guidance information into a large language model;

[0148] Determine the target text required for the video after editing in the video-text association information through the large language model based on the video editing guidance information, and obtain video positioning information of the target text;

[0149] A clipped video matching the video positioning information is cropped from the target video.

[0150] Furthermore, various functions implemented by running the application stored in the memory 402 can also be found in the description of the aforementioned embodiments, which will not be repeated here.

[0151] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0152] Optional, such as Figure 6 As shown, the electronic device 400 further includes: a touch screen 403, a radio frequency circuit 404, an audio circuit 405, an input unit 406, and a power supply 407. The processor 401 is electrically connected to the touch screen 403, the radio frequency circuit 404, the audio circuit 405, the input unit 406, and the power supply 407, respectively. Those skilled in the art can understand that Figure 6 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0153] The touch display screen 403 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch display screen 403 may include a display panel and a touch panel. Among them, the display panel may be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device, and these graphical user interfaces may be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel may be configured in the form of a liquid crystal display (LCD, Liquid Crystal Display), an organic light-emitting diode (OLED, Organic Light-Emitting Diode) and the like. The touch panel may be used to collect the user's touch operation on or near it (such as the user using any suitable object or attachment such as a finger, a stylus, etc. on the touch panel or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Optionally, the touch panel may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 401, and can receive the command sent by the processor 401 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 401 to determine the type of touch event, and then the processor 401 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 403 to realize the input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize the input and output functions. That is, the touch display screen 403 can also be used as a part of the input unit 406 to realize the input function.

[0154] The radio frequency circuit 404 may be used to send and receive radio frequency signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and to send and receive signals with the network device or other electronic devices.

[0155] The audio circuit 405 can be used to provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 405 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 405 and converted into audio data, and then the audio data is output to the processor 401 for processing, and then sent to another electronic device through the radio frequency circuit 404, or the audio data is output to the memory 402 for further processing. The audio circuit 405 may also include an earplug jack to provide communication between an external headset and an electronic device.

[0156] The input unit 406 may be used to receive input target video and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0157] The power supply 407 is used to supply power to various components of the electronic device 400. Optionally, the power supply 407 can be logically connected to the processor 401 through a power management system, so that the power management system can manage charging, discharging, and power consumption. The power supply 407 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0158] although Figure 6 Not shown, the electronic device 400 may also include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0159] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0160] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0161] To this end, an embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to perform any video processing method provided in the embodiment of the present application. For example, the computer program can perform the following steps of the video processing method:

[0162] Acquire video-text association information and video editing guidance information of a target video, wherein the video-text association information at least includes the video text;

[0163] Inputting the video text association information and the video editing guidance information into a large language model;

[0164] Determine the target text required for the video after editing in the video-text association information through the large language model based on the video editing guidance information, and obtain video positioning information of the target text;

[0165] A clipped video matching the video positioning information is cropped from the target video.

[0166] Furthermore, for the detailed steps of the above method steps, reference can be made to the description in the above embodiments, which will not be repeated here.

[0167] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0168] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0169] Since the computer program stored in the computer-readable storage medium can execute any one of the video processing methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the video processing methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0170] According to one aspect of the present application, a computer program product is also provided, including a computer program, which is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the methods provided in various optional implementations of the above embodiments.

[0171] In the above-mentioned video processing device, computer-readable storage medium, electronic device, and computer program product embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-mentioned video processing device, computer-readable storage medium, computer program product, electronic device and its corresponding units can refer to the description of the video processing method in the above embodiment, and will not be repeated here.

[0172] The above is a detailed introduction to a video processing method, device, electronic device, computer-readable storage medium and computer program product provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A video processing method, characterized in that: include: Acquire video-text association information and video editing guidance information of a target video, wherein the video-text association information at least includes the video text; Inputting the video text association information and the video editing guidance information into a large language model; Determine the target text required for the video after editing in the video-text association information through the large language model based on the video editing guidance information, and obtain video positioning information of the target text; A clipped video matching the video positioning information is cropped from the target video.

2. The video processing method according to claim 1, characterized in that: The video text association information includes the video text and the timestamp information corresponding to the video text, the video positioning information includes the target timestamp information, or the video positioning information includes the target text and the target timestamp information corresponding to the target text; The step of cutting out a clipped video matching the video positioning information from the target video includes: cropping at least one video segment corresponding to the target timestamp information from the target video; Based on the sequence of the target timestamp information, all the video clips are merged to obtain an edited video.

3. The video processing method according to claim 2, characterized in that: The method further comprises: Performing segment integrity detection on the video segment to determine complete segments and incomplete segments; If the incomplete fragment exists, processing the incomplete fragment to obtain a processed fragment; The step of fusing all the video clips based on the sequence of the target timestamp information to obtain an edited video includes: Based on the sequence of the target timestamp information, the processed segments and the complete segments are merged to obtain an edited video.

4. The video processing method according to claim 3, characterized in that: The step of processing the incomplete fragment to obtain a processed fragment includes: Obtaining the shot cut position of the incomplete segment; Determine a target sub-segment with a longer duration in the two video sub-segments before and after the shot segmentation position of the incomplete segment; Cutting out the target sub-segment from the incomplete segment; The duration of the target sub-segment is processed so that the duration of the target sub-segment is consistent with the duration of the incomplete segment, and the processed target sub-segment is used as a processed segment.

5. The video processing method according to claim 1, characterized in that: The video text association information only includes the video text; the video positioning information is the target text itself; The step of cutting out a clipped video matching the video positioning information from the target video includes: Cut out a video segment containing a video text matching the target text from the target video; The video clips are merged to obtain an edited video.

6. The video processing method according to claim 2, characterized in that: The video editing guidance information includes at least one video text editing rule, and the output format of the video positioning information; The step of determining the target text required for the video after editing in the video-text association information based on the video editing guidance information by using the large language model to obtain the video positioning information of the target text includes: Determine, by means of the large language model, a target text in the video text association information that complies with the video text clipping rule based on the video text clipping rule, and obtain target timestamp information corresponding to the target text, or the target text and the target timestamp information corresponding to the target text; The video positioning information conforming to the output format is selected from the target timestamp information corresponding to the target text, or from the target text and the target timestamp information corresponding to the target text.

7. The video processing method according to claim 6, characterized in that: The output format of the video positioning information is target timestamp information, and the step of cutting at least one video segment corresponding to the target timestamp information from the target video includes: inputting the target timestamp information into a predetermined video clipper; The target video is edited on the time axis by the predetermined video editor based on the video editing method corresponding to the output format to obtain video clips corresponding to each target timestamp.

8. A video processing device, characterized in that: include: A target video information acquisition module, used to acquire video text association information and video editing guidance information of a target video, wherein the video text association information at least includes video text; An input module, used for inputting the video text association information and the video editing guidance information into a large language model; A video positioning information determination module, configured to determine the target text required for the video after editing in the video-text association information through the large language model based on the video editing guidance information, and obtain the video positioning information of the target text; The editing module is used to cut out an edited video matching the video positioning information from the target video.

9. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the video processing method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The invention comprises a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the video processing method according to any one of claims 1 to 7.

Images