Short video production method, electronic device and computer program product

Abstract

The invention relates to a short video production method, electronic equipment and a computer program product, and the method comprises the steps: obtaining a plurality of target short videos, the target short videos comprise short videos of which the user attention degree is greater than a specified threshold value; matching each target short video in the plurality of target short videos with the feature video to obtain a matching result of each target short video; for any target short video in the plurality of target short videos, under the condition that the target short video is matched with the feature video, determining a first imitation short video of the target short video according to clip information of a video clip matched with the target short video in the feature video; and generating a plurality of second imitation short videos according to the plurality of first imitation short videos of the plurality of target short videos matched with the feature video. Therefore, high-quality short videos can be automatically produced in batches at low cost and high efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a short video production method, electronic equipment, and computer program product. Background Art

[0002] With the rapid development of the short video industry, various types of short videos have greatly occupied people's fragmented time. Therefore, it has become a popular trend to use long videos such as TV dramas and variety shows to produce short videos, such as for the promotion of long videos.

[0003] The current method of producing short videos is usually to manually produce them based on the highlight moments in the main content of long videos such as TV series and variety shows. This has high labor costs, low efficiency, and small output, which makes it difficult to meet the needs of video platforms for short video production. Summary of the invention

[0004] In view of this, the present disclosure proposes a short video production method, electronic device and computer program product, which can realize automatic batch production of high-quality short videos at low cost and high efficiency.

[0005] According to one aspect of the present disclosure, a short video production method is provided, comprising: obtaining a plurality of target short videos, the target short videos comprising short videos of which the user attention degree is greater than a specified threshold; matching each of the plurality of target short videos with a main video to obtain a matching result for each target short video; wherein the matching result comprises: whether the target short video matches the main video, and, when the target short video matches the main video, segment information of a video segment in the main video that matches the target short video; for any target short video among the plurality of target short videos, when the target short video matches the main video, determining a first imitation short video of the target short video according to the segment information of a video segment in the main video that matches the target short video; and generating a plurality of second imitation short videos according to a plurality of first imitation short videos of the plurality of target short videos that match the main video.

[0006] In a possible implementation, the clip information includes the main film lines in the video clip, and the start time and the end time of the video clip; the matching of each target short video in the multiple target short videos with the main film video to obtain the matching result of each target short video includes: for any target short video in the multiple target short videos, extracting short video lines from the target short video to obtain a first line set corresponding to the target short video, wherein the first line set includes multiple short video lines appearing in the target short video; matching each short video line in the first line set with the main film lines in the main film line library The character string is matched to obtain the line matching result of each short video line; wherein the main film line library includes all the main film lines in the main film video and the starting time and ending time of the video segment where each main film line in all the main film lines is located; the line matching result includes: whether it matches the main film lines in the main film line library, and, when matching the main film lines in the main film line library, the starting time and ending time of the video segment where the matched main film lines are located; according to the line matching result of each short video line in the first line set, the matching result of the target short video is determined.

[0007] In a possible implementation, the segment information includes a main video frame in the video segment, and a start time and an end time of the video segment where the main video frame is located; the matching of each target short video in the multiple target short videos with the main video to obtain a matching result of each target short video includes: for any target short video in the multiple target short videos, when no short video lines are extracted from the target short video, or when the number of short video lines extracted from the target short video is less than a specified number of lines, similarity matching is performed between the short video frames in the target short video and the main video frames in the main frame library to obtain a frame matching result of the short video frames in the target short video; wherein the main frame library includes all main video frames extracted at intervals from the main video; the frame matching result includes: whether the main video frames in the main frame library are matched, and, when the main video frames in the main frame library are matched, the frame time of the matched main video frames; and determining the matching result of the target short video according to the frame matching result of the short video frames in the target short video.

[0008] In a possible implementation, the feature video includes multiple video episodes, and performing string matching on each short video line in the first line set with the feature film lines in the feature film line library to obtain a line matching result of each short video line includes: for any short video line in the first line set, performing string matching on the short video lines with the feature film lines in the feature film line library to obtain multiple feature film lines matching the short video lines, and selecting feature film lines appearing in the target set video from the multiple feature film lines matching the short video lines according to the target set video where the context lines of the short video lines are located; and determining the line matching result of the short video lines according to the start time and end time of the video clip where the selected feature film lines appearing in the target set video are located.

[0009] In a possible implementation, performing string matching on each short video line in the first line set with the main film lines in the main film line library to obtain a line matching result of each short video line includes: for any short video line in the first line set, performing string matching on the short video lines with the main film lines in the main film line library to obtain a plurality of main film lines matching the short video lines, performing similarity matching on the main film video frames where the plurality of main film lines matching the short video lines are located with the short video frames where the short video lines are located to obtain a main film video frame with the highest similarity to the short video frame where the short video lines are located; and determining the line matching result of the short video lines according to the main film lines corresponding to the main film video frame with the highest similarity to the short video frame where the short video lines are located and the start time and end time of the video clip where the short video lines are located.

[0010] In a possible implementation, the extracting short video lines from the target short video to obtain a first line set corresponding to the target short video includes: extracting short video frames from the target short video at n frames per second to obtain multiple extracted short video frames, and performing character recognition on each short video frame extracted from the target short video to obtain recognized character strings in the multiple short video frames and positions of the character strings, where n is a positive integer; clustering and grouping the recognized character strings in the multiple short video frames according to the positions of the recognized character strings in the multiple short video frames to obtain multiple character string groups; and determining a first character string group containing the largest number of character strings among the multiple character string groups as the first line set.

[0011] In a possible implementation, the first imitation short video of the target short video is determined based on the segment information of the video segments that match the target short video in the main video, including: when there are multiple video segments that match the target short video in the main video, according to the segment information of the multiple video segments that match the target short video in the main video, the multiple video segments that match the target short video in the main video are spliced ​​to obtain an initial imitation short video corresponding to the target short video; according to the top title set and / or fancy word set of the target short video, a top title and / or fancy word set is added to the initial imitation short video corresponding to the target short video to obtain the first imitation short video of the target short video.

[0012] In a possible implementation, the method further includes: determining a second string group among the multiple string groups, whose string positions are at the top of the short video frame and whose string repetition times exceed a specified threshold, as a top title set of the target short video; and / or determining string groups among the multiple string groups other than the first string group and the second string group as a flowery character set in the target short video.

[0013] In a possible implementation, the first imitation short video of the target short video is determined based on the segment information of the video segment that matches the target short video in the main video, including: when there are multiple video segments that match the target short video in the main video, according to the segment information of the multiple video segments that match the target short video in the main video, the multiple video segments that match the target short video in the main video are spliced ​​to obtain an initial imitation short video corresponding to the target short video; using a generative artificial intelligence model to predict the emotion expressed by the initial imitation short video according to the lines corresponding to the initial imitation short video and the auxiliary information of the main video; the auxiliary information includes a plot introduction and / or a plot commentary; determining a first background music that matches the emotion expressed by the initial imitation short video from a preset background music library, and adding the first background music to the initial imitation short video to obtain the first imitation short video of the target short video.

[0014] In a possible implementation, the generating of multiple second imitation short videos based on multiple first imitation short videos of multiple target short videos matching the main video includes: when more than a specified number of multiple first imitation short videos of multiple target short videos matching the main video are obtained, generating multiple second line combinations based on the first line combinations in the multiple first imitation short videos and auxiliary information related to the main video using a generative artificial intelligence model; the auxiliary information includes a plot introduction and / or a plot comment; determining the video segment in which each line in each of the multiple second line combinations is located in the main video, and determining multiple second imitation short videos corresponding to the multiple second line combinations based on the video segment in which each line in each of the multiple second line combinations is located in the main video.

[0015] In a possible implementation, determining the video segment in which each line of each second line combination in the plurality of second line combinations is located in the feature video includes: for any second line combination in the plurality of second line combinations, performing string matching on each line of the second line combination with feature lines in a feature line library to obtain a line matching result for each line in the second line combination, the line matching result including: whether it matches the feature lines in the feature line library, and, when matching the feature lines in the feature line library, the start time and the end time of the matched feature lines and the video segment in which the matched feature lines are located; and determining the video segment in which each line of the second line combination is located in the feature video according to the line matching result for each line in the second line combination.

[0016] In a possible implementation, the determining of a plurality of second imitation short videos corresponding to the plurality of second line combinations based on the video segments in which the respective lines in each second line combination in the plurality of second line combinations are located in the feature video includes: for any second line combination in the plurality of second line combinations, when the second line combination includes multiple lines, splicing the video segments in which the respective lines in the second line combination are located in the feature video to obtain an initial imitation short video corresponding to the second line combination; and adding a top title and / or an imitation short video to the initial imitation short video corresponding to the second line combination based on a top title set and / or an imitation character set of a plurality of target short videos matching the feature video to obtain a second imitation short video corresponding to the second line combination.

[0017] In a possible implementation, the determining of a plurality of second imitation short videos corresponding to the plurality of second line combinations according to the video segments in the feature video where the lines in each second line combination in the plurality of second line combinations are located includes: for any second line combination in the plurality of second line combinations, when the second line combination includes multiple lines, splicing the video segments in the feature video where the lines in the second line combination are located to obtain an initial imitation short video corresponding to the second line combination; predicting the emotion expressed by the initial imitation short video corresponding to the second line combination according to the line content of the second line combination and the auxiliary information using a generative artificial intelligence model; determining from a preset background music library second background music that matches the emotion expressed by the initial imitation short video corresponding to the second line combination, and adding the second background music to the initial imitation short video corresponding to the second line combination to obtain a second imitation short video corresponding to the second line combination.

[0018] According to another aspect of the present disclosure, a short video production device is provided, including: an acquisition module, used to acquire multiple target short videos, wherein the target short videos include short videos whose user attention degree is greater than a specified threshold; a matching module, used to match each of the multiple target short videos with a main video to obtain a matching result for each target short video; wherein the matching result includes: whether the target short video matches the main video, and, when the target short video matches the main video, the segment information of the video segment in the main video that matches the target short video; a first imitation module, used to determine, for any target short video among the multiple target short videos, a first imitation short video of the target short video according to the segment information of the video segment in the main video that matches the target short video when the target short video matches the main video; a second imitation module, used to generate multiple second imitation short videos according to multiple first imitation short videos of the multiple target short videos that match the main video; the multiple first imitation short videos and the multiple second imitation short videos are used to promote the main video.

[0019] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0020] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0021] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0022] According to various aspects of the present disclosure, by utilizing a target short video whose user attention level is higher than a specified threshold and which matches the main video, a high-quality first imitation short video is generated which is similar in logic to the target short video, has a strong sense of rhythm, conforms to user preferences, etc., and matches the content of the main video. Then, multiple first imitation short videos that match the main video are utilized to generate more new second imitation short videos which are similar in logic to the target short video, have a strong sense of rhythm, conform to user preferences, etc., and match the content of the main video. This achieves low-cost and high-efficiency automatic batch production of high-quality imitation short videos. In an exemplary application scenario, the promotional effect of utilizing the first imitation short video and the second imitation short video to promote the main video can be improved, thereby meeting the promotional needs of the video platform for the main video.

[0023] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0025] Figure 1 A flow chart of a short video production method according to an embodiment of the present disclosure is shown.

[0026] Figure 2 A schematic diagram of a screen of a target short video according to an embodiment of the present disclosure is shown.

[0027] Figure 3 A schematic diagram showing a process of stripping and OCR for a feature video according to an embodiment of the present disclosure is shown.

[0028] Figure 4 A schematic diagram showing a process of a short video production method according to an embodiment of the present disclosure.

[0029] Figure 5 A block diagram of a short video production device according to an embodiment of the present disclosure is shown.

[0030] Figure 6 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0031] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0032] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0033] The term "and / or" herein is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent any one or more elements selected from the set consisting of A, B, and C. In the description of the present disclosure, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0034] It should be understood that the terms "first", "second", etc. in the claims, specifications and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0035] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0036] The short video production method of the embodiment of the present disclosure can be deployed on various terminal devices through software or hardware modification. The terminal device involved in the embodiment of the present disclosure may refer to a device with a wireless connection function and / or a wired connection function. The wireless connection function refers to the ability to connect to other devices through wireless connection methods such as wifi and Bluetooth. The terminal device involved in the embodiment of the present disclosure may also communicate with other devices through a wired connection function. The terminal device involved in the embodiment of the present disclosure may be a touch screen, a non-touch screen, or a screenless terminal device. The touch screen device can be controlled by clicking and sliding on the display screen with a finger or a stylus. The non-touch screen device can be connected to an input device such as a mouse, keyboard, touch panel, etc., and the terminal device can be controlled through the input device. For example, the device without a screen may be a Bluetooth speaker without a screen. For example, the terminal device of the present application may include but is not limited to user equipment (UE), mobile device, user terminal, terminal, handheld device, tablet computer, laptop computer, PDA, computing device, etc.

[0037] The short video production method of the embodiment of the present disclosure can also be deployed on a server, which can be located in the cloud or locally, and can be a physical device or a virtual device, such as a virtual machine, a container, etc., with a wireless communication function, wherein the wireless communication function can be set in the chip (system) or other parts or components of the server. It can refer to a device with a wireless connection function, and the function of wireless connection refers to the ability to connect to other servers or terminal devices through wireless connection methods such as Wi-Fi and Bluetooth. The server involved in the embodiment of the present disclosure may also have a function of communicating through a wired connection. For example, the server of the embodiment of the present disclosure can be located in the cloud, communicate with the terminal device, receive the target short video sent by the terminal device, and generate multiple first imitation short videos and multiple second imitation short videos based on the target short video using the production method deployed on the server, and return them to the terminal device, so as to display the generated multiple first imitation short videos and multiple second imitation short videos to the user in the terminal device, and also facilitate the user to use the generated multiple first imitation short videos and multiple second imitation short videos for promotion of feature videos and other application scenarios.

[0038] The feature video (or long video) in this application includes a video with a long length (for example, more than 10 minutes) and continuous and complete content, including but not limited to film and television dramas, variety shows, performance recordings, etc. The short video in this application includes a video with a short length (for example, less than 15 minutes) and the content is an excerpt or clip of the feature video. This application does not limit the specific length and content of short videos and feature videos.

[0039] The lines in this application may include dialogues, monologues, narrations and other forms, may be presented in the video in the form of subtitles or audio, and may be identified through relevant recognition technologies.

[0040] Figure 1 A flowchart of a short video production method according to an embodiment of the present disclosure is shown. The method can be applied to electronic devices such as the above-mentioned terminal device or server, such as Figure 1 As shown, the method includes: step S11 to step S14.

[0041] In step S11, a plurality of target short videos are obtained, wherein the target short videos include short videos of which the user attention level is greater than a specified threshold.

[0042] The user's attention can be measured individually or comprehensively by the number of likes, the number of favorites, the number of reposts, the number of views, etc., which is not limited in the embodiments of the present disclosure. It should be understood that those skilled in the art can set the specific value of the specified threshold according to actual needs. For example, if the number of likes is used to measure the user's attention, the specified threshold can be set to 100,000. In other words, short videos with more than 100,000 likes can be obtained as target short videos. Such short videos can be understood as popular short videos with high popularity.

[0043] In practical applications, short videos with a user attention level greater than a specified threshold can be downloaded from various social platforms, short video platforms and other network platforms as target short videos. The disclosed embodiment does not limit the method for obtaining the target short video.

[0044] Optionally, the target short videos may be screened to exclude target short videos that are irrelevant to the content of the feature video, target short videos of poor quality, and other unnecessary target short videos, and the screened target short videos are used as the multiple target short videos acquired in step S11, thereby reducing the amount of data processing in the subsequent step S12. The screening method includes but is not limited to manual selection, screening based on a trained artificial intelligence model, and the like.

[0045] It should be understood that the target short videos that attract a high degree of user attention and are related to the content of the main video usually include the highlights, or wonderful moments, in the main video. Since such target short videos have been verified by the market and have multiple factors such as smooth logic, a strong sense of rhythm, and compliance with the preferences of users of various platforms, the embodiment of the present disclosure uses a method for generating imitation short videos by analyzing target short videos. This can make the imitation short videos also include highlight moments and also have multiple factors such as smooth logic, a strong sense of rhythm, and compliance with the preferences of users of various platforms, which is conducive to improving the production efficiency, content quality and publicity effect of imitation short videos.

[0046] In step S12, each target short video among the multiple target short videos is matched with the feature video to obtain a matching result of each target short video.

[0047] The matching result includes: whether the target short video matches the main video, and, when the target short video matches the main video, the segment information of multiple video segments in the main video that match the target short video. The matching of the target short video with the main video can be understood as there is at least one video segment in the main video that matches the target short video, and the mismatch of the target short video with the main video can be understood as there is no video segment in the main video that matches the target short video.

[0048] It is understandable that the feature video may contain lines, so the target short video can be matched with the feature video based on the feature lines, and the above-mentioned segment information can include the feature lines in the video segment, as well as the start time and end time of the video segment; in a possible implementation, the above-mentioned step S12, matching each target short video in the multiple target short videos with the feature video, and obtaining the matching result of each target short video, can include:

[0049] Step S121, for any target short video among the multiple target short videos, extract short video lines from the target short video to obtain a first line set corresponding to the target short video, where the first line set includes multiple short video lines appearing in the target short video;

[0050] Step S122, character string matching is performed on each short video line in the first line set with the main film lines in the main film line library to obtain a line matching result of each short video line; wherein the main film line library includes all the main film lines in the main film video and the start time and end time of the video segment where each main film line in all the main film lines is located; the line matching result includes: whether it matches the main film lines in the main film line library, and, when matching the main film lines in the main film line library, the start time and end time of the video segment where the matched main film lines are located;

[0051] Step S123, determining the matching result of the target short video according to the dialogue matching result of each short video dialogue in the first dialogue set.

[0052] In step S121, a text recognition technology known in the art, such as optical character recognition (OCR) technology, can be used to extract short video lines from the target short video. For example, frames can be extracted from the target short video and OCR technology can be used to perform string recognition on the extracted short video frames, and the short video lines appearing in the target short video can be obtained by recognizing the subtitles in the short video frames. Those skilled in the art should understand that short video lines can also be extracted by other means, such as by using speech recognition technology to recognize the speech content in the audio of the target short video to extract short video lines.

[0053] The extracted short video lines can be in the form of strings, and each video frame containing lines can correspond to one or more strings, and multiple video frames can correspond to the same string, that is, multiple video frames can correspond to the same lines. Each sentence in a natural language can be identified as one or more strings; wherein, the strings located in the same row in the video frame can be identified as a line, or the multiple lines of strings adjacent to the vertical coordinate can be identified as a line, or the same line or multiple lines of strings adjacent to each other can be divided into at least one line according to the punctuation marks (such as commas, periods, question marks) identified in the same line or multiple lines of strings adjacent to each other.

[0054] Considering that the subtitles of the lines in the short video may appear anywhere in the short video, and there may also be top titles and fancy words in the short video, fancy words can be understood as text content in the short video with any color, font, font size, etc. except the lines and top titles, for example, Figure 2 A schematic diagram of a target short video is shown, such as Figure 2 As shown, the target short video has a top title "AA variety show's funny scene", a short video line "Do you believe in light?" and a fancy word "Hahaha". Therefore, all the texts identified from the target short video can be grouped by vertical coordinate aggregation, and the group with the largest number of text contents (such as the character string below) is taken as the short video lines, the top of the short video and the number of repetitions exceeding a specified threshold (such as 5 times) is taken as the top title of the target short video, and the other groups are taken as fancy words. Specifically, the above step S121 extracts the short video lines from the target short video to obtain the first line set corresponding to the target short video, including:

[0055] Step S1211, extracting short video frames from the target short video at n frames per second to obtain multiple extracted short video frames, and performing character recognition on each short video frame extracted from the target short video to obtain recognized character strings and positions of the character strings in the multiple short video frames, where n is a positive integer;

[0056] Step S1212, clustering and grouping the character strings identified in the multiple short video frames according to the positions of the character strings identified in the multiple short video frames to obtain multiple character string groups;

[0057] Step S1213, determining a first character string group containing the largest number of character strings among the multiple character string groups as a first line set;

[0058] And, the method may further include:

[0059] Step S1214, determining a second string group in which the position of the string is at the top of the short video frame and the number of repetitions of the string exceeds a specified number threshold as the top title set of the target short video; and / or,

[0060] Step S1215 : determining the character string groups other than the first character string group and the second character string group in the multiple character string groups as the flower character set in the target short video.

[0061] In step S1211, those skilled in the art can set a specific value of n based on historical experience. For example, n can be set to 2, that is, short video frames can be extracted from the target short video at a rate of two frames per second, and 20 short video frames can be extracted from a 10-second target short video; then, OCR technology can be used to perform character recognition on each short video frame extracted from the target short video to obtain the character strings recognized in each short video frame and the position of the character strings, and then the character strings are clustered and grouped based on the positions of the character strings (that is, all the character strings in the target short video are grouped in a vertical coordinate aggregation manner). It should be understood that the positions of the various character strings in the same character string group in the vertical coordinate direction are the same or similar, and the positions of the character strings in different character string groups are different.

[0062] Since the target short video has the largest number of lines, the strings in the first string group containing the largest number of strings can be determined as the short video lines in the target short video; since the top title of a short video is usually at the top of the short video and is repeated multiple times (i.e., appears in multiple short video frames), the strings in the second string group whose positions are at the top of the short video frame and whose repetition times exceed a specified threshold can be determined as the top title in the target short video; since the position and number of flower characters in the target short video are not fixed, the strings in the string groups other than the first string group and the second string group can be determined as flower characters in the target short video. In this way, the short video lines, top titles, and flower characters in the target short video can be accurately identified.

[0063] It should be understood that if there is no top title in the target short video, the character string groups other than the first character string group can be determined as fancy words. If there is no fancy words in the target short video, the short video lines in the first character group and the top title in the second character string group can be obtained. If there is no top title and fancy words in the target short video, a character string group may be clustered, and the clustered character string group is also the first line set.

[0064] In step S122, the feature film dialogue library can pre-identify and split all the feature film dialogues in the feature film video through video stripping or OCR technology to obtain all the feature film dialogues and the start and end times of the video segments where each feature film dialogue is located, wherein a video segment contains the same feature film dialogue. A video frame displaying the same feature film dialogue subtitle may be split into multiple video segments. For example, when "I haven't eaten yet, let's eat together" is displayed as a subtitle on multiple consecutive video frames, these multiple consecutive video frames may be split into two video segments based on voice features, the first video segment corresponding to the voice "I haven't eaten yet", and the second video segment corresponding to the voice "Let's eat together". The way of splitting the video segments is related to the specific way of video stripping, and the present disclosure does not limit this.

[0065] Since the main film lines obtained by the video stripping technology have a certain error rate, the OCR technology can be used to correct the main film lines obtained by the video stripping technology to ensure the accuracy of the recognized main film lines, thereby improving the success rate of subsequent matching of target short videos. Figure 3 A process of stripping and OCR for a feature video is shown, Figure 3 As shown, the episode list of the feature video can be obtained and stored in the local device, and then multiple episodes of the feature video can be downloaded to the local device in batches and the videos being downloaded and downloaded successfully are marked (that is, the videos being downloaded and downloaded successfully are marked) and stored in the local device. At the same time, the multiple episodes of the video downloaded to the local device are stripped or OCR recognition is performed, and the video segments obtained by stripping the video are stored in a storage server (such as a server using an object storage service (Object Storage Service, OSS)), and the OCR recognition result is used to correct the lines of the video stripping result, and the corrected feature film lines are stored in the local database; wherein, multiple devices can be used to concurrently perform video stripping and OCR recognition, and data can be synchronized between multiple devices to avoid duplication or omission.

[0066] In actual applications, each feature line and the corresponding start and end time (i.e., the start and end times of the video clip where the feature lines are located) identified by video stripping or OCR technology can be written into the database in the form of a JSON array to obtain a feature line library. For example, the feature lines and the corresponding start and end times stored in the form of a JSON array can be expressed as: [{"text":"Have you eaten?","startTime":102400,"endTime":105400},{"text":"I haven't eaten yet,","startTime":105400,"endTime":108880},{"text":"Where do you want to eat?","startTime":108880,"endTime":110280}], where text represents the feature lines, startTime represents the start time, and endTime represents the end time.

[0067] In practical applications, after obtaining the JSON arrays corresponding to all the main lines in the main video, the JSON arrays corresponding to the main lines can be combined, and the video processing technology known in the art, such as FFmpeg (an open source computer program for processing video and audio) software, can be used to divide the main video into video segments, and the start and end time of each video segment is the above-mentioned startTime and endTime, and each segmented video segment can be stored in a storage server (such as a server using an object storage service (OSS)), so as to facilitate the subsequent splicing of video segments to obtain the simulated short video.

[0068] Based on the above-mentioned feature film dialogue library, in step S122, any string matching technology known in the art can be used to achieve string matching of each short video dialogue in the first dialogue set with the feature film dialogue in the feature film dialogue library to obtain the dialogue matching results of each short video dialogue. For example, by calculating the similarity between each short video dialogue in the first dialogue set and the feature film dialogue in the feature film dialogue library, the feature film dialogue with a similarity exceeding a specified similarity threshold (such as 80%) is determined as the feature film dialogue that matches the short video dialogue. That is, if the similarity between the short video dialogue and a feature film dialogue exceeds 80%, it is considered that the short video dialogue matches the feature film dialogue (that is, the match is successful), otherwise, it is considered that the short video dialogue does not match the feature film dialogue. The dialogue corresponding to a video clip of the feature film video can be considered as "a line of dialogue". The division of "one sentence" in the short video dialogue can be matched with the principle of video stripping of the main film video, so that the "one sentence" of the short video dialogue corresponds to the "one sentence" in a video clip of the main film as much as possible, so as to improve the accuracy of the match. For example, a short video dialogue and a main film dialogue can be divided into sentences divided by punctuation marks in natural language. For example, using the subtitles in the short video, when extracting short video dialogues based on OCR and other technical means, a string ending with a specific punctuation mark can be identified as "a sentence" (such as a comma, a greeting, a period), etc. When the main film video is stripped based on voice, these punctuation marks form a sentence break, so they are also divided into video clips, and each video clip corresponds to a sentence of the main film. For example, the above "Have you eaten?", "I haven't eaten yet," and "Where do you want to eat?" are three main film lines respectively. If a short video line is identified in the short video as "Have you eaten?", "Have you eaten?" is matched with "Have you eaten?", "I haven't eaten yet," and "Where do you want to eat?" respectively. If the subtitle of a short video is "I haven't eaten yet, where do you want to go for dinner?", it can be identified as "I haven't eaten yet" and "Where do you want to go for dinner?" and matched with "Have you eaten?", "I haven't eaten yet," and "Where do you want to go for dinner?" respectively.

[0069] Considering that the feature video is usually long and may include multiple episodes, the same feature line may appear multiple times in different episodes of the feature video, so that the same short video line may match multiple feature lines. Therefore, in a possible implementation, in the above step S122, string matching is performed on each short video line in the first line set with the feature lines in the feature line library to obtain the line matching results of each short video line, which may include:

[0070] For any short video line in the first line set, when character string matching is performed between the short video line and the feature film lines in the feature film line library to obtain multiple feature film lines matching the short video line, the feature film lines appearing in the target set video are selected from the multiple feature film lines matching the short video lines according to the target set video where the context lines of the short video line are located; the line matching result of the short video line is determined according to the start time and end time of the video clip where the selected feature film lines appearing in the target set video are located. In this way, feature film lines matching the overall content in the target short video can be selected from the multiple feature film lines matching the short video lines.

[0071] Among them, the context lines of the short video lines can be understood as the adjacent lines that appear in the target short video above and below the short video lines. It can be understood that the short video lines and the context lines of the short video lines usually appear in the same video episode or the same video segment. Therefore, the target set video where the context lines appear can be used to select the main film lines that appear in the target set from multiple main film lines that match the short video lines, so that the main film lines that match the short video lines and the starting time and ending time of the video segment where the main film lines are located can be obtained.

[0072] Among them, the video clip where the context lines are located can be obtained by performing string matching between the context lines and the main film lines, and then the target set video to which the video clip where the context lines are located belongs can be obtained. It should be understood that if the context lines only appear in a certain episode of video, then the episode where the context lines appear is also the target set video. If the context lines also appear in multiple episodes of video, then the episode where the context lines appear the most times can be used as the target set video.

[0073] Considering that the same short video lines may match multiple lines of the main film and the multiple lines of the main film may also be in the same video episode or the same video segment, this means that the same line of the main film may appear in the same episode or the same video segment. In order to more accurately select the main film lines that match the target short video content from the multiple lines of the main film that match the short video lines, in a possible implementation, the above step S122, performing string matching on each short video line in the first line set and the main film lines in the main film line library to obtain the line matching results of each short video line, may include:

[0074] For any short video line in the first line set, when the short video line is matched with the main film lines in the main film line library by string matching, and multiple main film lines matching the short video line are obtained, the main film video frames where the multiple main film lines matching the short video line are located are matched with the short video frames where the short video lines are located respectively for similarity, and the main film video frames with the highest similarity to the short video frames where the short video lines are located are obtained; the line matching result of the short video line is determined according to the main film lines corresponding to the main film video frames with the highest similarity to the short video frames where the short video lines are located and the start time and end time of the video clip where the lines are located. In this way, the main film lines matching the overall content in the target short video can be accurately selected from the multiple main film lines matching the short video lines.

[0075] It is understandable that although the same short video lines may match multiple lines of the main film, the pictures of the main film video frames when the main film lines appear are usually different. Therefore, by calculating the similarity between the main film video frames where the multiple main film lines matching the short video lines are located and the short video frames where the short video lines are located, the main film lines corresponding to the main film video frames with the highest similarity from the matched multiple main film lines are selected as the main film lines that correctly match the short video lines, and then based on the main film lines corresponding to the main film video frames with the highest similarity and the start and end times of the video clips where the main film lines are located, the lines matching results of the short video lines are obtained. Among them, those skilled in the art can use any similarity calculation method known in the art to calculate the similarity between the main film video frames and the short video frames, which is not limited by the embodiments of the present disclosure.

[0076] For example, if a line in a short video matches the lines A1, A2 and A3 of the main film, and the lines A1, A2 and A3 of the main film correspond to three segments B1, B2 and B3, respectively, each containing 20 frames of the main film video, and the short video corresponding to the short video line contains 20 short video frames, the similarity between the first short video frame and the 20 short video frames of the three segments B1, B2 and B3 can be calculated, and the three segments B1, B2 and B3 are selected respectively. The positive video frame with the highest similarity with the first short video frame among the 20 positive video frames of 1, B2, and B3 (such as the first positive video frame in B1, the second positive video frame in B2, and the third positive video frame in B3) is used as the positive video frame matching the first short video frame in the three positive segments B1, B2, and B3, and then the similarity between the second short video frame and the 20 positive video frames of the three positive segments B1, B2, and B3 is calculated. The similarity of the frames is determined by selecting the positive video frames with the highest similarity to the first short video frame from the 20 positive video frames of the three positive film segments B1, B2, and B3 (such as the second positive video frame in B1, the third positive video frame in B2, and the fourth positive video frame in B3) as the positive video frames that match the second short video frame in the three positive film segments B1, B2, and B3. By analogy, the positive video frames that match each short video frame in the three positive film segments B1, B2, and B3 can be obtained. Then, the average similarity corresponding to the positive video frames that match each short video frame in the three positive film segments B1, B2, and B3 can be calculated, and the positive lines (positive lines A1) corresponding to the positive film segment with the largest average similarity (such as the positive film segment B1) are taken as the positive lines corresponding to the positive video frame with the highest similarity, that is, as the positive lines that correctly match the short video lines.

[0077] It should be understood that for each short video line in the first line set, the line matching result of each short video line can be obtained according to the implementation method of the above step S122, so that in step S123, the matching result of the target short video can be obtained according to the line matching result of each short video line in the first line set. And for each target short video obtained in step S11, the matching result of each target short video can be obtained according to the above steps S121 to S122.

[0078] It is understandable that there may be some lines in the target short video that do not belong to the main film. Therefore, there may be some short video lines in the first line set that do not match the main film lines in the main film line library. In this case, the short video lines that do not match the main film lines in the main film line library can be directly filtered out; and, since the multiple target short videos in step S11 are obtained based on the user's attention level, the video content of the target short video may not match the main film at all, so that all the short video lines in the first line set do not match the main film lines in the main film line library. In this case, the target short video that does not match the main film at all can be directly filtered out, that is, the target short video that does not match the main film at all is not subsequently processed, so that the target short video that matches the main film video can be screened out from the multiple target short videos obtained in step S11.

[0079] Considering that some of the multiple target short videos obtained in step S11 may have no lines or very few lines, in these cases, video clips matching the highlight moments in the target short videos can be found by comparing video frames. For example, each short video frame in the target short video can be compared with all the feature video frames extracted from the feature video, and the feature video frames with a similarity higher than 80% are used as the condition for successful matching, so that the first imitation short video can be produced without lines. Specifically, in the above step S12, the clip information can also include the feature video frame in the video clip, and the start time and end time of the video clip where the feature video frame is located. The above matching of each target short video in the multiple target short videos with the feature video to obtain the matching result of each target short video can include:

[0080] Step S124, for any target short video among the multiple target short videos, when short video lines are not extracted from the target short video, or the number of short video lines extracted from the target short video is less than the specified number of lines, the short video frames in the target short video are matched with the main video frames in the main film frame library for similarity, to obtain a frame matching result of the short video frames in the target short video; wherein the main film frame library includes all main film video frames extracted at intervals from the main film video; the frame matching result includes: whether it matches the main film video frames in the main film frame library, and, when matching the main film video frames in the main film frame library, the frame time of the matched main film video frames;

[0081] Step S125, determining the matching result of the target short video according to the frame matching result of the short video frame in the target short video.

[0082] Among them, failure to extract short video lines from the target short video can be understood as short video lines not being identified from the target short video, and the number of short video lines extracted from the target short video being less than the specified number of lines can be understood as the number of short video lines identified from the target short video being less than the specified number of lines, wherein those skilled in the art can set a specific value for the specified number of lines based on actual experience, for example, it can be set to 5 sentences, and this is not limited to the embodiments of the present disclosure.

[0083] In practical applications, the main film video frames can be extracted from the entire main film video at intervals according to a certain frame extraction method (such as extracting m frames per second) and stored (for example, all the extracted main film video frames can be stored in a storage server) to obtain a main film frame extraction library. The embodiment of the present disclosure does not limit the construction method of the main film frame extraction library.

[0084] In step S124, a similarity calculation method known in the art may be used to perform similarity matching on each short video frame in the target short video with the main video frame in the main film frame library to obtain a frame matching result of each short video frame in the target short video. Alternatively, each short video frame in a plurality of short video frames extracted at intervals from the target short video may be matched with the main video frame in the main film frame library to obtain a frame matching result of each short video frame extracted from the target short video. Specifically, the similarity between the short video frame in the target short video and the main video frame in the main film frame library may be calculated, and the similarity exceeding 0 may be calculated. The main video frame with a specified similarity threshold (such as 80%) is determined as the main video frame that matches the short video frame, that is, if the similarity between the short video frame and a main video frame exceeds 80%, the short video frame is considered to match the main video frame, otherwise, the short video frame is considered to not match the main video frame; it should be understood that if the short video frame matches a main video frame, the main video frame that matches the short video frame and the frame time of the main video frame can be obtained, and the frame time is also the time when the main video frame appears in the main video, wherein, for a main video containing multiple episodes of video, the frame time may include a certain moment in a certain episode of the main video.

[0085] Among them, for each short video frame in the target short video or each extracted short video frame, the frame matching result of the short video frame in the target short video can be obtained according to the implementation method of the above-mentioned step S124, that is, whether it matches the main film video frame in the main film frame extraction library, and, when matching with the main film frame in the main film frame extraction library, the frame time of the matched main film video frame; and then in step S125, based on the frame time of the main film video frame that matches the short video frame in the target short video, the video segment indicated by the frame time of the main film video frame that matches the short video frame in the target short video can be determined.

[0086] In one example, the indicated video segment may be a video segment between the maximum and minimum values ​​of all frame moments of the matched feature video frame. For example, the frame moments of the feature video frame that matches the short video frame in the target short video include: 10:24:00, 10:24:10, 10:24:20, 10:24:30, then the video segment indicated by the frame moment of the feature video frame that matches the short video frame in the target short video is the video segment indicated from 10:24:00 to 10:24:30, and the starting moment of the video segment is 10:24:00 and the ending moment is 10:24:30.

[0087] In another example, the video segments obtained by stripping the main video in the previous article can be used. The indicated video segments can be video segments where frame times fall, such as video segments obtained by stripping the video at the four times 10:24:00, 10:24:10, 10:24:20, and 10:24:30.

[0088] In addition, it should be understood that the target short video may contain at least one video segment in the main video, and the frame moments of the main video frames in the same video segment are adjacent (the frame moments are adjacent, for example, the duration between two adjacent frame moments is less than or equal to the unit duration of the interval frame extraction). Therefore, the video segments corresponding to the multiple adjacent frame moments can be determined based on the frame moments of the main video frames, and the starting moment and the ending moment of the video segment can also be obtained, so as to obtain at least one video segment in the main video that matches the target short video and the starting moment and the ending moment of the video segment.

[0089] As described above, part of the target short video may not belong to the main video, that is, part of the target short video may not match the main video frames in the main film frame library. In this case, the short video frames that do not match the main video frames in the main film frame library can be directly filtered out; and, since the multiple target short videos in step S11 are obtained based on the user's attention level, the video content of the target short video may not match the main video at all, so that all short video frames in the target short video do not match the main video frames in the main film frame library. In this case, the target short video that does not match the main video at all can be directly filtered out, that is, the target short video that does not match the main video at all will not be processed subsequently, so that the target short video that matches the main video can be screened out from the multiple target short videos obtained in step S11.

[0090] In step S13, for any target short video among the multiple target short videos, when the target short video matches the main video, a first imitation short video of the target short video is determined according to the segment information of the video segment matching the target short video in the main video.

[0091] As described above, there may be one or more video segments in the main video that match the target short video. Therefore, in the case where there is only one video segment that matches the target short video in the main video, the corresponding video segment can be directly obtained from the storage server used to store each video segment in the main video according to the segment information of the video segment that matches the target short video in the main video, and the obtained video segment can be determined as the first imitation short video of the target short video; in the case where there are multiple video segments that match the target short video in the main video, the corresponding multiple video segments can be obtained from the storage server according to the segment information of the multiple video segments that match the target short video in the main video, and the obtained multiple video segments can be spliced ​​to obtain the first imitation short video of the target short video.

[0092] Considering that short videos in actual situations usually contain top titles and / or fancy words, therefore, the top title and / or fancy words in the target short video may also be referred to to generate a first imitation short video with the top title and / or fancy words added thereto, so as to obtain a first imitation short video with better publicity effect. Therefore, in a possible implementation, the above step S13, determining the first imitation short video of the target short video according to the segment information of the video segment matching the target short video in the feature video, may include:

[0093] Step S131, when there are multiple video segments in the feature video that match the target short video, the multiple video segments in the feature video that match the target short video are spliced ​​according to the segment information of the multiple video segments in the feature video that match the target short video to obtain an initial imitation short video corresponding to the target short video;

[0094] Step S132, according to the top title set and / or fancy word set of the target short video, add the top title and / or fancy word set to the initial imitation short video corresponding to the target short video to obtain the first imitation short video of the target short video.

[0095] In step S131, according to the segment information of the multiple video segments that match the target short video in the feature video, the multiple video segments that match the target short video in the feature video are spliced, and the multiple video segments that match the target short video can be spliced ​​in the order in which the multiple video segments that match the target short video appear in the target short video, so as to facilitate obtaining an initial imitation short video that conforms to the content logic in the target short video. In practical applications, those skilled in the art can use any video processing technology known in the art, for example, FFmpeg software can be used to realize the splicing of multiple video segments that match the target short video in the feature video, that is, all the matched video segments can be combined by FFmpeg software, and the embodiments of the present disclosure are not limited to this.

[0096] As described above, the top title set and / or fancy word set of the target short video can be obtained by grouping the character strings identified from the target short video. If the top title set contains top titles that are repeated multiple times, then in step S132, the top titles that are repeated multiple times in the top title set can be directly added to the top position of the initial imitation short video (the top position can be the position where the top title is located in the target short video) to obtain the first imitation short video with the top title added.

[0097] It can be known that the flower characters in the target short video are usually associated with the short video lines in the target short video. Therefore, the flower character set can also record the association relationship between the flower characters and the short video lines identified in the same short video frame. Then, in step S132, based on the association relationship between the flower characters and the short video lines, the flower characters associated with the short video lines can be added to the video frame where the main film lines matching the short video lines are located in the initial imitation short video to obtain the first imitation short video with the flower characters added. It should be understood that the first imitation short video can only have the top title added, can only have the flower characters added, or can have the top title and the flower characters added.

[0098] In actual applications, some target short videos may not originally have a top title and / or fancy words. If the target short video does not have a top title and / or fancy words, an artificial intelligence model can be used to generate a top title and / or fancy words based on the lines, plot introduction, and plot comments in the target short video, so that in step S132, the top title and / or fancy words generated by the model can be added to the initial imitation short video corresponding to the target short video to obtain the corresponding first imitation short video.

[0099] Considering that in actual situations, background music (BGM) is usually added to short videos to enhance the emotional expression of the short videos, therefore, in a possible implementation, in the above step S13, according to the segment information of the video segment matching the target short video in the feature video, determining the first imitation short video of the target short video may include:

[0100] Step S133, when there are multiple video segments in the main video that match the target short video, according to the segment information of the multiple video segments in the main video that match the target short video, the multiple video segments in the main video that match the target short video are spliced ​​to obtain an initial imitation short video corresponding to the target short video; the implementation method of step S133 can refer to the above step S131, which will not be described in detail here;

[0101] Step S134, using a generative artificial intelligence model to predict the emotion expressed in the initial imitation short video based on the lines corresponding to the initial imitation short video and the auxiliary information of the feature video; the auxiliary information includes a plot introduction and / or a plot comment;

[0102] Step S135, determining the first background music that matches the emotion expressed by the initial imitation short video from the preset background music library, and adding the first background music to the initial imitation short video to obtain the first imitation short video of the target short video.

[0103] In step S134, those skilled in the art may use any known, pre-trained generative artificial intelligence model in the art to predict the emotion expressed in the initial imitation short video based on the corresponding lines of the initial imitation short video and the auxiliary information of the feature video. The disclosed embodiment does not limit the type and structure of the generative artificial intelligence model. Among them, the generative artificial intelligence model may be instructed to predict the emotion expressed in the initial imitation short video by inputting a command prompt into the generative artificial intelligence model, and the disclosed embodiment does not limit this. The artificial intelligence model can be pre-trained based on the line samples, auxiliary information samples and emotion labels based on the training methods in the prior art.

[0104] In practical applications, we can obtain plot introductions and plot comments about the feature video from various drama review platforms, social platforms, video platforms and other online platforms. Among them, plot comments can be comments with a certain threshold of likes and reposts by users. These comments can be considered as high-quality comments that represent the hot topics that most users pay attention to, and can express users' views on the highlights of the feature video. In this way, we can use the auxiliary information of the feature video to assist the generative artificial intelligence model to more accurately predict the emotions expressed in the initial imitation short video based on the lines in the initial imitation short video.

[0105] Among them, each background music in the background music library can have a business identifier, and the business identifier can be used to describe the usage scenario of the background music, or to describe what kind of short video with emotion the background music is suitable for, that is, the business identifier indicates the emotion expressed by the background music. Therefore, in step S135, the emotion expressed by the initial imitation short video can be matched with the background music with the business identifier in the background music library to determine the first background music that matches the emotion expressed by the initial imitation short video from the background music library, and add the first background music to the initial imitation short video to obtain the first imitation short video of the target short video. For example, the background music library contains background music 1, background music 2, and background music 3. The business identifier of background music 1 is "suitable for scenes with sad emotions", the business identifier of background music 2 is "suitable for scenes with joy emotions", and the business identifier of background music 3 is "suitable for scenes with lonely emotions". If the emotion expressed by the initial imitation short video is joy, background music 2 can be matched to match the emotion expressed by the initial imitation short video.

[0106] As described above, there may be one video segment in the main video that matches the target short video. Therefore, when there is one video segment in the main video that matches the target short video, the one video segment that matches the target short video can be directly determined as the initial imitation short video, and then the above step S132 and / or the above steps S134 to S135 can be executed on the initial imitation short video to obtain the first imitation short video of the target short video.

[0107] It should be understood that for the initial imitation short video, step S132 can be executed simultaneously to add a top title and / or fancy words, and steps S134 to S135 can be executed to add a matching first background music. That is to say, the first imitation short video can include the first background music, as well as the top title and / or fancy words; of course, the first imitation short video may also only include the top title and / or fancy words, or only include the first background music, and the embodiments of the present disclosure are not limited to this.

[0108] In step S14, a plurality of second imitation short videos are generated according to a plurality of first imitation short videos of a plurality of target short videos matching the main video.

[0109] It should be understood that, through the above steps S12 to S13, the first imitation short video of each target short video in the multiple target short videos matching the main video can be determined from the multiple target short videos obtained in step S11, that is, multiple first imitation short videos can be obtained; wherein, the advantage of the first imitation short video generated based on the target short video is that the target short video has been verified by the market, and the quality of the first imitation short video obtained after imitation is also high, but the disadvantage is that one target short video can only generate one corresponding first imitation short video. In order to increase the output of short videos, after obtaining enough first imitation short videos of target short videos matching the main video, these first imitation short videos can be handed over to the artificial intelligence model for mixed editing (i.e., splicing clips from different first imitation short videos), so that more imitated second imitation short videos can be obtained, which is conducive to greatly increasing the production quantity of short videos.

[0110] Therefore, in a possible implementation method, the above-mentioned generation of multiple second imitation short videos based on multiple first imitation short videos of multiple target short videos matching the main video may include: using an artificial intelligence model to perform video remixing on the multiple first imitation short videos to obtain multiple second imitation short videos, wherein those skilled in the art can adopt artificial intelligence technology known in the art to develop and train artificial intelligence models for video remixing, and the embodiments of the present disclosure are not limited to this.

[0111] Considering that the production efficiency of directly mixing and cutting multiple first imitation short videos to obtain multiple second imitation short videos is low, the embodiment of the present disclosure also proposes that a new second imitation video can be generated by using a generative artificial intelligence model to generate a new second line combination according to the first line combination in the first imitation short video combined with the plot introduction and the plot commentary. Compared with using the model to process the video, the model has higher efficiency and lower cost in processing text such as lines, which is conducive to improving the generation efficiency of the second imitation short video and reducing the generation cost. Therefore, in a possible implementation method, the above step S14, generating multiple second imitation short videos according to multiple first imitation short videos of multiple target short videos matching the feature video, can include:

[0112] Step S141, when a plurality of first imitation short videos of a plurality of target short videos matching the feature video are obtained in excess of a specified number, a plurality of second dialogue combinations are generated using a generative artificial intelligence model according to first dialogue combinations in the plurality of first imitation short videos and auxiliary information related to the feature video; the auxiliary information includes plot introduction and / or plot commentary;

[0113] Step S142, determining the video segment where each line in each of the multiple second line combinations is located in the feature video, and determining multiple second imitation short videos corresponding to the multiple second line combinations based on the video segment where each line in each of the multiple second line combinations is located in the feature video.

[0114] In step S141, those skilled in the art may use any known generative artificial intelligence model in the art to realize generating multiple second lines combinations according to the first lines combinations in multiple first imitation short videos and auxiliary information related to the feature video, and the disclosed embodiment does not limit the type and structure of the generative artificial intelligence model. Among them, the generative artificial intelligence model may be instructed to automatically generate multiple second lines combinations according to the first lines combinations in multiple first imitation short videos and the drama review introduction and / or plot commentary related to the feature video by inputting a command prompt to the generative artificial intelligence model, and the disclosed embodiment does not limit this.

[0115] In practical applications, when the generative artificial intelligence model is instructed to generate multiple second line combinations by inputting an instruction prompt into the generative artificial intelligence model, in addition to indicating the auxiliary information of the first line combinations of multiple first imitation short videos and the feature video, some constraint information can also be added in the input prompt so that the generative artificial intelligence model can output the second line combination that meets the actual imitation requirements. For example, the constraint information may include the degree of change of the second line combination relative to the first line combination and the number of the generated second line combinations, etc. For example, the prompt input into the generative artificial intelligence model may be "please use the 100 input line combinations (i.e., the first line combination) combined with the input plot introduction and plot commentary to generate 1,000 new line combinations (i.e., the second line combination), wherein the degree of change of the new line combination relative to the input line combination does not exceed 10% (i.e., more than 10% of the input line combinations cannot be changed)". It should be understood that those skilled in the art can design the specific content of the prompt input into the generative artificial intelligence model according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0116] In step S142, determining the video segment in the feature video where each line of each second line combination in the plurality of second line combinations is located may include: for any second line combination in the plurality of second line combinations, performing string matching on each line of the second line combination with the feature lines in the feature line library to obtain a line matching result for each line in the second line combination, the line matching result including: whether it matches the feature lines in the feature line library, and, when matching the feature lines in the feature line library, the start time and the end time of the matched feature lines and the video segment where the matched feature lines are located; and determining the video segment in the feature video where each line of the second line combination is located according to the line matching result for each line in the second line combination.

[0117] Among them, the string matching method in the above step S122 can be referred to to achieve string matching of each line in the second line combination with the main film lines in the main film line library to obtain the line matching results of each line in the second line combination, which will not be described in detail here. Furthermore, after obtaining the line matching results of each line in the second line combination, the corresponding video segment can be obtained from the storage server for storing video segments for the start time and end time of the video segment where the successfully matched main film lines are located, that is, according to the line matching results of each line in the second line combination, the video segment where each line in the second line combination is located in the main film video is determined.

[0118] It should be understood that the second line combination is generated based on the first line combination, and the first line combination contains the lines of the main film, wherein the text in the second line combination can be understood as a rearranged combination of the original text in the first line combination, similar to a line remix, or the text in the second line combination is derived from the first line combination, that is, from the lines of the main film, so the lines in the second line combination can basically match the lines of the main film; each second line combination may include one or more lines, and in the above step S142, multiple second imitation short sentences corresponding to the multiple second line combinations are determined according to the video clips in which each line in each of the multiple second line combinations is located in the main film video. The video may include: when a second line combination includes a line, the video clip corresponding to the line can be determined as the second imitation video corresponding to the second line combination; when a second line combination includes multiple lines, the multiple video clips corresponding to the multiple lines in the second line combination can be spliced ​​according to the start time and the end time of the video clips corresponding to each line in the second line combination to obtain the second imitation video corresponding to the second line combination; wherein, for each second line combination in the multiple second line combinations, a second imitation short video corresponding to each line combination can be obtained, and multiple second line combinations can obtain multiple second imitation short videos.

[0119] In a possible implementation, the top title and / or fancy words in the target short video may be referred to to generate a second imitation short video with the top title and / or fancy words added thereto, so as to obtain a second imitation short video with better publicity effect. Specifically, in the above step S142, the multiple second imitation short videos corresponding to the multiple second line combinations are determined according to the video clips in the feature video where each line in each of the multiple second line combinations is located, including:

[0120] Step S1421, for any second line combination among the plurality of second line combinations, splicing the video segments where the lines in the second line combination are located in the feature video, to obtain an initial imitation short video corresponding to the second line combination;

[0121] Step S1422, according to the top title set and / or fancy word set of multiple target short videos matching the feature video, add the top title and / or fancy word set to the initial imitation short video corresponding to the second line combination to obtain the second imitation short video corresponding to the second line combination.

[0122] In step S1421, the video segments in the feature video where the lines in the second line combination that match the feature lines can be sequentially spliced ​​according to the start and end times of the video segments in the feature video where the lines in the second line combination are located, to obtain an initial imitation short video corresponding to the second line combination. If the second line combination includes only one line, the video segment in the feature video where the line is located can be determined as the initial imitation short video corresponding to the second line combination.

[0123] It should be understood that through the above steps S1214 and S1215, the top title set and / or flower character set of each target short video matching the main video can be obtained. Then, in the above step S1422, since there may be multiple top titles in the top title set of multiple target short videos, a top title can be randomly selected from the multiple top titles for the second line combination and added to the initial imitation short video corresponding to the second line combination. Alternatively, a top title that best matches the line content can be selected from the multiple top titles based on the line content of the second line combination and added to the initial imitation short video corresponding to the second line combination, so as to obtain a second imitation short video with a top title added. The embodiments of the present disclosure are not limited to this. As described above, the flower character set may also record the association between the flower characters identified in the same short video frame and the short video lines. Therefore, based on the association between the various flower characters in the flower character set and the short video lines, the flower characters associated with each line in the first line combination may be obtained. Since the characters in the second line combination as described above may be understood as a rearranged combination of the original characters in the first line combination, the flower characters associated with each line in the second line combination may be determined based on the flower characters associated with each line in the first line combination. Specifically, the flower characters associated with the lines in the second line combination that are the same as those in the first line combination may be determined as the flower characters associated with the lines in the second line combination, so that the flower characters associated with each line in the second line combination may be added to the initial imitation short video corresponding to the second line combination, to obtain a second imitation short video with the flower characters added.

[0124] Optionally, an artificial intelligence model can also be used to generate top titles and / or fancy words corresponding to each second line combination based on the line content in each second line combination and information such as the plot introduction and plot comments of the feature video. The above-mentioned top title set and / or fancy words set can respectively include the top titles and / or fancy words corresponding to each second line combination generated by the artificial intelligence model, and then in the above-mentioned step S1422, the top titles and / or fancy words corresponding to each second line combination generated by the artificial intelligence model can be added to the initial imitation short video corresponding to each second line combination to obtain a second imitation short video with the top title and / or fancy words added, which is not limited to this embodiment of the present disclosure.

[0125] As described above, background music is usually added to short videos to enhance the emotional expression of the short videos. Therefore, in a possible implementation, the above step S142, according to the video clips where each line in each second line combination in the plurality of second line combinations is located in the feature video, determines the plurality of second imitation short videos corresponding to the plurality of second line combinations, may include:

[0126] Step S1423, for any second line combination among the plurality of second line combinations, when the second line combination includes multiple lines, splice the video segments where the lines in the second line combination are located in the feature video to obtain an initial imitation short video corresponding to the second line combination; the implementation method of step S1423 can refer to the above step S1421, which will not be described in detail here;

[0127] Step S1424, using a generative artificial intelligence model to predict the emotion expressed by the initial imitation short video corresponding to the second line combination according to the line content of the second line combination and the auxiliary information;

[0128] Step S1425, determine the second background music that matches the emotion expressed by the initial imitation short video corresponding to the second line combination from the preset background music library, and add the second background music to the initial imitation short video corresponding to the second line combination to obtain the second imitation short video corresponding to the second line combination.

[0129] In step S1424, those skilled in the art may use any generative artificial intelligence model known in the art to predict the emotion expressed by the initial imitation short video corresponding to the second line combination based on the line content and auxiliary information of the second line combination. The embodiments of the present disclosure do not limit the type, structure, etc. of the generative artificial intelligence model.

[0130] As mentioned above, the plot introduction and plot comments about the feature video can be obtained from various drama review platforms, social platforms, video platforms and other online platforms, where the plot comments can be comments with a number of likes and reposts exceeding a certain threshold, and these plot comments can express the user's views on the highlight moments in the feature video. In this way, the auxiliary information of the feature video can be used to assist the generative artificial intelligence model to more accurately predict the emotions expressed by the initial imitation short video corresponding to the second line combination based on the lines in the second line combination.

[0131] As described above, each background music in the background music library may carry a business identification to describe the usage scenario of the background music, or to describe what kind of emotional short video the background music is suitable for. Therefore, in step S1425, the emotion expressed by the initial imitation short video corresponding to the second line combination can be matched with the background music with the business identification in the background music library to determine the second background music that matches the emotion expressed by the initial imitation short video corresponding to the second line combination from the background music library, and add the second background music to the initial imitation short video corresponding to the second line combination to obtain the second imitation short video of the second line combination.

[0132] As described above, the second line combination may include a line. In this case, a video clip matching the line can be directly determined as the initial imitation short video corresponding to the second line combination, and then the above step S1422 and / or the above steps S1424 to S1425 can be executed on the initial imitation short video to obtain the second imitation short video corresponding to the second line combination.

[0133] It should be understood that for the initial imitation short video corresponding to any second line combination, step S1422 can be simultaneously executed to add the top title and / or flower characters, and steps S1424 to S1425 can be executed to add the matching second background music, that is, the second imitation short video can include the second background music, as well as the top title and / or flower characters; of course, the second imitation short video can also only include the top title and / or flower characters, or only include the second background music, which is not limited in the embodiment of the present disclosure. For each second line combination in multiple second line combinations, the second imitation short video corresponding to each line combination can be obtained by executing the above steps S1421 to S1422 and / or executing the above steps S1423 to S1425, and multiple second line combinations can obtain multiple second imitation short videos.

[0134] In actual applications, after obtaining multiple first imitation short videos and multiple second imitation short videos, a video list of the generated multiple first imitation short videos and multiple second imitation short videos can be displayed on the front end. The video list can include video identifiers, feature videos corresponding to the videos, download addresses, and can also carry the release title of the feature video, related topics, and related content descriptions, so that users can view the generated multiple first imitation short videos and multiple second imitation short videos, and use the multiple first imitation short videos and multiple second imitation short videos as needed. For example, further use the multiple first imitation short videos and multiple second imitation short videos to promote the feature video. For example, the multiple first imitation short videos and multiple second imitation short videos can be published on various short video platforms, social platforms and other network platforms, so as to realize the promotion of the feature video to more platform users, which is conducive to increasing the attention and playback volume of the feature video and meeting the video platform's needs for the promotion of the feature video.

[0135] For example, Figure 4 A schematic diagram showing a process of a short video production method according to an embodiment of the present disclosure is shown as follows: Figure 4As shown, the target short video can be obtained and stored in a local database. After obtaining a video list of target short videos that have not been downloaded, the target short videos in the video list can be downloaded locally and marked (that is, a mark is added to the target short video). The short video lines can be identified by OCR technology every time a target short video is downloaded. The short video lines can be in the data format of a line string and stored in the local database; then the feature film lines that match the short video lines or the feature film video frames without lines can be matched from the feature film lines library to obtain the matching result of the target short video. The matching result can be in the data format of a JSON object, including the starting time and the ending time of the feature film lines; then, an imitation short video (including a first imitation short video and a second imitation short video) can be generated according to the matching result of the target short video, and information such as the release title, related topics, and content description of the imitation video can be added and stored in the local database, and the target short video can be marked to indicate that the target short video has been produced.

[0136] In practical applications, the short video production method of the above-mentioned embodiment of the present disclosure can utilize multiple devices to concurrently acquire multiple target short videos and batch produce a first imitation short video and a second imitation short video, which can further improve the efficiency of short video production; it should be understood that collaborative processes such as data synchronization and task scheduling can be performed between multiple devices, and the embodiment of the present disclosure does not limit this.

[0137] According to the method of the embodiment of the present disclosure, by using the target short video whose user attention degree is higher than the specified threshold and which matches the main video, a high-quality first imitation short video is generated which is similar in logic to the target short video, has a strong sense of rhythm, conforms to user preferences, etc. and matches the content of the main video, and then using multiple first imitation short videos that match the main video to generate more new second imitation short videos which are similar in logic to the target short video, have a strong sense of rhythm, conform to user preferences, etc. and match the content of the main video, low-cost and high-efficiency automatic batch production of high-quality imitation short videos is achieved. In an exemplary application scenario, it is beneficial to improve the publicity effect when using the first imitation short video and the second imitation short video to promote the main video, and meet the publicity needs of the video platform for the main video.

[0138] In the related technology, although it is possible to select lines that may be highlights by giving all the lines of a feature video in the form of prompts to a generative artificial intelligence model, and splicing the video clips corresponding to these lines to produce imitation short videos; however, the generation effect of this method is difficult to guarantee. Even if the most powerful model on the market is used and enough example data is added to the prompt, it is difficult to guarantee that the selected dialogue lines are in line with the user's preferences.

[0139] The short video production method of the embodiment of the present disclosure identifies the pictures and lines of the target short videos with high user attention on the Internet, as well as important information such as the top title and fancy words in the target short videos, and then matches them with all the lines and pictures in the main video to find the corresponding video clips, and finally splices these video clips in the order in which the lines and pictures in the target short video appear to generate a first imitation short video, so as to achieve the effect of 1:1 imitation of the target short video; after obtaining enough first imitation short videos of target short videos that match a main video, all the lines and the high-rated lines of the main video can be High-quality auxiliary information such as plot comments and plot introductions are given to the generative artificial intelligence model in the form of prompts, so that more new lines combinations can be produced through the generative artificial intelligence model, and then multiple new second imitation short videos can be generated. This can achieve a production efficiency of 1:N and achieve the effect of increasing the output of short videos. In an exemplary application scenario, a low-cost, batch-produced promotional short video method is realized to meet the promotional demands of various TV dramas, variety shows and other feature video content on the video platform, so that the promotion of imitation short videos can reach more target groups, which is conducive to increasing the playback volume of feature videos.

[0140] Figure 5 A block diagram of a short video production device according to an embodiment of the present disclosure is shown as follows: Figure 5 As shown, the device comprises:

[0141] An acquisition module 501 is used to acquire a plurality of target short videos, wherein the target short videos include short videos whose user attention degree is greater than a specified threshold;

[0142] A matching module 502 is used to match each of the multiple target short videos with the main video to obtain a matching result for each target short video; wherein the matching result includes: whether the target short video matches the main video, and, when the target short video matches the main video, segment information of the video segment in the main video that matches the target short video;

[0143] The first imitation module 503 is used to determine, for any target short video among the multiple target short videos, a first imitation short video of the target short video according to the segment information of the video segment matching the target short video in the main video, when the target short video matches the main video;

[0144] The second imitation writing module 504 is used to generate multiple second imitation writing short videos according to multiple first imitation writing short videos of multiple target short videos matching the main video; the multiple first imitation writing short videos and the multiple second imitation writing short videos are used to promote the main video.

[0145] In a possible implementation, the clip information includes the main film lines in the video clip, and the start time and the end time of the video clip; the matching of each target short video in the multiple target short videos with the main film video to obtain the matching result of each target short video includes: for any target short video in the multiple target short videos, extracting short video lines from the target short video to obtain a first line set corresponding to the target short video, wherein the first line set includes multiple short video lines appearing in the target short video; matching each short video line in the first line set with the main film lines in the main film line library The character string is matched to obtain the line matching result of each short video line; wherein the main film line library includes all the main film lines in the main film video and the starting time and ending time of the video segment where each main film line in all the main film lines is located; the line matching result includes: whether it matches the main film lines in the main film line library, and, when matching the main film lines in the main film line library, the starting time and ending time of the video segment where the matched main film lines are located; according to the line matching result of each short video line in the first line set, the matching result of the target short video is determined.

[0146] In a possible implementation, the segment information includes a main video frame in the video segment, and a start time and an end time of the video segment where the main video frame is located; the matching of each target short video in the multiple target short videos with the main video to obtain a matching result of each target short video includes: for any target short video in the multiple target short videos, when no short video lines are extracted from the target short video, or when the number of short video lines extracted from the target short video is less than a specified number of lines, similarity matching is performed between the short video frames in the target short video and the main video frames in the main frame library to obtain a frame matching result of the short video frames in the target short video; wherein the main frame library includes all main video frames extracted at intervals from the main video; the frame matching result includes: whether the main video frames in the main frame library are matched, and, when the main video frames in the main frame library are matched, the frame time of the matched main video frames; and determining the matching result of the target short video according to the frame matching result of the short video frames in the target short video.

[0147] In a possible implementation, the feature video includes multiple video episodes, and performing string matching on each short video line in the first line set with the feature film lines in the feature film line library to obtain a line matching result of each short video line includes: for any short video line in the first line set, performing string matching on the short video lines with the feature film lines in the feature film line library to obtain multiple feature film lines matching the short video lines, and selecting feature film lines appearing in the target set video from the multiple feature film lines matching the short video lines according to the target set video where the context lines of the short video lines are located; and determining the line matching result of the short video lines according to the start time and end time of the video clip where the selected feature film lines appearing in the target set video are located.

[0148] In a possible implementation, performing string matching on each short video line in the first line set with the main film lines in the main film line library to obtain a line matching result of each short video line includes: for any short video line in the first line set, performing string matching on the short video lines with the main film lines in the main film line library to obtain a plurality of main film lines matching the short video lines, performing similarity matching on the main film video frames where the plurality of main film lines matching the short video lines are located with the short video frames where the short video lines are located to obtain a main film video frame with the highest similarity to the short video frame where the short video lines are located; and determining the line matching result of the short video lines according to the main film lines corresponding to the main film video frame with the highest similarity to the short video frame where the short video lines are located and the start time and end time of the video clip where the short video lines are located.

[0149] In a possible implementation, the extracting short video lines from the target short video to obtain a first line set corresponding to the target short video includes: extracting short video frames from the target short video at n frames per second to obtain multiple extracted short video frames, and performing character recognition on each short video frame extracted from the target short video to obtain recognized character strings in the multiple short video frames and positions of the character strings, where n is a positive integer; clustering and grouping the recognized character strings in the multiple short video frames according to the positions of the recognized character strings in the multiple short video frames to obtain multiple character string groups; and determining a first character string group containing the largest number of character strings among the multiple character string groups as the first line set.

[0150] In a possible implementation, the first imitation short video of the target short video is determined based on the segment information of the video segments that match the target short video in the main video, including: when there are multiple video segments that match the target short video in the main video, according to the segment information of the multiple video segments that match the target short video in the main video, the multiple video segments that match the target short video in the main video are spliced ​​to obtain an initial imitation short video corresponding to the target short video; according to the top title set and / or fancy word set of the target short video, a top title and / or fancy word set is added to the initial imitation short video corresponding to the target short video to obtain the first imitation short video of the target short video.

[0151] In a possible implementation, the device further includes: a title determination module, used to determine a second string group among the multiple string groups, whose string positions are at the top of the short video frame and whose string repetition times exceed a specified threshold, as a top title set of the target short video; and / or, a fancy word determination module, used to determine a string group among the multiple string groups other than the first string group and the second string group as a fancy word set in the target short video.

[0152] In a possible implementation, the first imitation short video of the target short video is determined based on the segment information of the video segment that matches the target short video in the main video, including: when there are multiple video segments that match the target short video in the main video, according to the segment information of the multiple video segments that match the target short video in the main video, the multiple video segments that match the target short video in the main video are spliced ​​to obtain an initial imitation short video corresponding to the target short video; using a generative artificial intelligence model to predict the emotion expressed by the initial imitation short video according to the lines corresponding to the initial imitation short video and the auxiliary information of the main video; the auxiliary information includes a plot introduction and / or a plot commentary; determining a first background music that matches the emotion expressed by the initial imitation short video from a preset background music library, and adding the first background music to the initial imitation short video to obtain the first imitation short video of the target short video.

[0153] In a possible implementation, the generating of multiple second imitation short videos based on multiple first imitation short videos of multiple target short videos matching the main video includes: when more than a specified number of multiple first imitation short videos of multiple target short videos matching the main video are obtained, generating multiple second line combinations based on the first line combinations in the multiple first imitation short videos and auxiliary information related to the main video using a generative artificial intelligence model; the auxiliary information includes a plot introduction and / or a plot comment; determining the video segment in which each line in each of the multiple second line combinations is located in the main video, and determining multiple second imitation short videos corresponding to the multiple second line combinations based on the video segment in which each line in each of the multiple second line combinations is located in the main video.

[0154] In a possible implementation, determining the video segment in which each line of each second line combination in the plurality of second line combinations is located in the feature video includes: for any second line combination in the plurality of second line combinations, performing string matching on each line of the second line combination with feature lines in a feature line library to obtain a line matching result for each line in the second line combination, the line matching result including: whether it matches the feature lines in the feature line library, and, when matching the feature lines in the feature line library, the start time and the end time of the matched feature lines and the video segment in which the matched feature lines are located; and determining the video segment in which each line of the second line combination is located in the feature video according to the line matching result for each line in the second line combination.

[0155] In a possible implementation, the determining of a plurality of second imitation short videos corresponding to the plurality of second line combinations based on the video segments in which the respective lines in each second line combination in the plurality of second line combinations are located in the feature video includes: for any second line combination in the plurality of second line combinations, when the second line combination includes multiple lines, splicing the video segments in which the respective lines in the second line combination are located in the feature video to obtain an initial imitation short video corresponding to the second line combination; and adding a top title and / or an imitation short video to the initial imitation short video corresponding to the second line combination based on a top title set and / or an imitation character set of a plurality of target short videos matching the feature video to obtain a second imitation short video corresponding to the second line combination.

[0156] In a possible implementation, the determining of a plurality of second imitation short videos corresponding to the plurality of second line combinations according to the video segments in the feature video where the lines in each second line combination in the plurality of second line combinations are located includes: for any second line combination in the plurality of second line combinations, when the second line combination includes multiple lines, splicing the video segments in the feature video where the lines in the second line combination are located to obtain an initial imitation short video corresponding to the second line combination; predicting the emotion expressed by the initial imitation short video corresponding to the second line combination according to the line content of the second line combination and the auxiliary information using a generative artificial intelligence model; determining from a preset background music library second background music that matches the emotion expressed by the initial imitation short video corresponding to the second line combination, and adding the second background music to the initial imitation short video corresponding to the second line combination to obtain a second imitation short video corresponding to the second line combination.

[0157] According to the device of the embodiment of the present disclosure, by utilizing the target short video whose user attention degree is higher than the specified threshold and which matches the main video, a high-quality first imitation short video is generated which is similar in logic to the target short video, has a strong sense of rhythm, conforms to user preferences, etc. and matches the content of the main video. Then, multiple first imitation short videos that match the main video are utilized to generate more new second imitation short videos which are similar in logic to the target short video, have a strong sense of rhythm, conform to user preferences, etc. and match the content of the main video. This realizes low-cost and high-efficiency automatic batch production of high-quality imitation short videos. In an exemplary application scenario, this is conducive to improving the publicity effect when the first imitation short video and the second imitation short video are used to promote the main video, and meets the publicity needs of the video platform for the main video.

[0158] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0159] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0160] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0161] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0162] Figure 6 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 6 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0163] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0164] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0165] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0166] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0167] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0168] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, device instructions, device-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0169] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0170] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a device, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other device to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0171] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0172] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0173] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A short video production method, characterized in that: include: Acquire multiple target short videos, where the target short videos include short videos whose user attention level is greater than a specified threshold; Matching each target short video among the multiple target short videos with the main video to obtain a matching result for each target short video; wherein the matching result includes: whether the target short video matches the main video, and, when the target short video matches the main video, segment information of the video segment in the main video that matches the target short video; For any target short video among the multiple target short videos, when the target short video matches the main video, determining a first imitation short video of the target short video according to segment information of a video segment in the main video that matches the target short video; A plurality of second imitation short videos are generated according to a plurality of first imitation short videos of a plurality of target short videos matching the feature video.

2. The method according to claim 1, characterized in that The segment information includes the main lines in the video segment, as well as the start time and end time of the video segment. The step of matching each target short video among the multiple target short videos with the feature video to obtain a matching result for each target short video includes: For any target short video among the multiple target short videos, extract short video lines from the target short video to obtain a first line set corresponding to the target short video, where the first line set includes multiple short video lines appearing in the target short video; Performing string matching on each short video line in the first line set and the main film lines in the main film line library to obtain a line matching result of each short video line; wherein the main film line library includes all the main film lines in the main film video and the starting time and ending time of the video segment where each main film line in all the main film lines is located; the line matching result includes: whether it matches the main film lines in the main film line library, and, when matching the main film lines in the main film line library, the starting time and ending time of the video segment where the matched main film lines are located; The matching result of the target short video is determined according to the line matching result of each short video line in the first line set.

3. The method according to claim 2, characterized in that The segment information includes the main video frame in the video segment, and the start time and end time of the video segment where the main video frame is located. The step of matching each target short video among the multiple target short videos with the feature video to obtain a matching result for each target short video includes: For any target short video among the multiple target short videos, when short video lines are not extracted from the target short video, or the number of short video lines extracted from the target short video is less than the specified number of lines, the short video frames in the target short video are matched with the main video frames in the main film frame library for similarity, so as to obtain a frame matching result of the short video frames in the target short video; wherein the main film frame library includes all main film video frames extracted at intervals from the main film video; the frame matching result includes: whether it matches the main film video frames in the main film frame library, and, when matching the main film video frames in the main film frame library, the frame time of the matched main film video frames; According to the frame matching results of the short video frames in the target short video, the matching result of the target short video is determined.

4. The method according to claim 2, characterized in that: The feature video includes multiple episodes of video, and character string matching is performed on each short video line in the first line set with the feature lines in the feature line library to obtain a line matching result of each short video line, including: For any short video line in the first line set, when character string matching is performed between the short video line and the feature film lines in the feature film line library to obtain multiple feature film lines matching the short video line, based on the target set video where the context lines of the short video line are located, the feature film lines appearing in the target set video are selected from the multiple feature film lines matching the short video line; According to the start time and the end time of the selected video segment containing the main film lines appearing in the target set of videos, the lines matching result of the short video lines is determined.

5. The method according to claim 2, characterized in that: The step of performing string matching on each short video line in the first line set with the main film lines in the main film line library to obtain a line matching result of each short video line includes: For any short video line in the first line set, when character string matching is performed on the short video line and the main film lines in the main film line library to obtain multiple main film lines matching the short video line, similarity matching is performed on the main film video frames where the multiple main film lines matching the short video line are located and the short video frames where the short video lines are located to obtain the main film video frame with the highest similarity to the short video frame where the short video lines are located; The lines matching result of the short video lines is determined according to the main film lines corresponding to the main film video frame with the highest similarity to the short video frame where the short video lines are located and the starting time and the ending time of the video segment where the short video lines are located.

6. The method according to claim 2, characterized in that The step of extracting short video lines from the target short video to obtain a first line set corresponding to the target short video includes: Extracting short video frames from the target short video at n frames per second to obtain multiple extracted short video frames, and performing character recognition on each of the short video frames extracted from the target short video to obtain recognized character strings and positions of the character strings in the multiple short video frames, where n is a positive integer; Clustering and grouping the character strings identified in the multiple short video frames according to positions of the character strings identified in the multiple short video frames to obtain multiple character string groups; A first character string group including the largest number of character strings among the plurality of character string groups is determined as the first line set.

7. The method according to claim 6, characterized in that The step of determining a first imitation short video of the target short video according to the segment information of the video segment matching the target short video in the feature video comprises: In the case where there are multiple video segments in the main video that match the target short video, the multiple video segments in the main video that match the target short video are spliced ​​according to the segment information of the multiple video segments in the main video that match the target short video to obtain an initial imitation short video corresponding to the target short video; According to the top title set and / or fancy word set of the target short video, the top title and / or fancy word set are added to the initial imitation short video corresponding to the target short video to obtain a first imitation short video of the target short video.

8. The method according to claim 7, characterized in that The method further comprises: Determine a second string group in which the position of the string is at the top of the short video frame and the number of repetitions of the string exceeds a specified number threshold as the top title set of the target short video; and / or, The character string groups other than the first character string group and the second character string group in the multiple character string groups are determined as the flower character set in the target short video.

9. The method according to any one of claims 1 to 8, characterized in that: The step of determining a first imitation short video of the target short video according to the segment information of the video segment matching the target short video in the feature video comprises: In the case where there are multiple video segments in the main video that match the target short video, the multiple video segments in the main video that match the target short video are spliced ​​according to the segment information of the multiple video segments in the main video that match the target short video to obtain an initial imitation short video corresponding to the target short video; Using a generative artificial intelligence model, based on the lines corresponding to the initial imitation short video and the auxiliary information of the feature video, predict the emotion expressed by the initial imitation short video; the auxiliary information includes a plot introduction and / or a plot comment; A first background music that matches the emotion expressed by the initial imitation short video is determined from a preset background music library, and the first background music is added to the initial imitation short video to obtain a first imitation short video of the target short video.

10. The method according to claim 1, characterized in that The method of generating a plurality of second imitation short videos according to a plurality of first imitation short videos of a plurality of target short videos matching the feature video comprises: When a plurality of first imitation short videos of a plurality of target short videos matching the feature video are obtained in excess of a specified number, a plurality of second dialogue combinations are generated using a generative artificial intelligence model according to the first dialogue combinations in the plurality of first imitation short videos and auxiliary information related to the feature video; the auxiliary information includes a plot introduction and / or a plot comment; Determine the video segment in which each line of each second line combination in the multiple second line combinations is located in the feature video, and determine the multiple second imitation short videos corresponding to the multiple second line combinations based on the video segment in which each line of each second line combination in the multiple second line combinations is located in the feature video.

11. The method according to claim 10, characterized in that The step of determining the video segment where each line in each of the plurality of second line combinations is located in the feature video includes: For any second line combination among the plurality of second line combinations, character string matching is performed on each line in the second line combination with the main line in the main line library to obtain a line matching result of each line in the second line combination, wherein the line matching result includes: whether the line matches the main line in the main line library, and, when matching the main line in the main line library, the start time and the end time of the video clip where the matched main line and the matched main line are located; According to the dialogue matching result of each line in the second dialogue combination, the video segment where each line in the second dialogue combination is located in the feature video is determined.

12. The method according to claim 10, characterized in that The step of determining a plurality of second imitation short videos corresponding to the plurality of second line combinations according to the video segments in which the lines in each second line combination in the feature video are located includes: For any second line combination among the plurality of second line combinations, when the second line combination includes multiple lines, splicing the video segments where the lines in the second line combination are located in the feature video to obtain an initial imitation short video corresponding to the second line combination; According to the top title set and / or fancy word set of multiple target short videos matching the feature video, the top title and / or fancy word set are added to the initial imitation short video corresponding to the second line combination to obtain the second imitation short video corresponding to the second line combination.

13. The method according to any one of claims 10 to 12, characterized in that: The step of determining a plurality of second imitation short videos corresponding to the plurality of second line combinations according to the video segments in which the lines in each second line combination in the feature video are located includes: For any second line combination among the plurality of second line combinations, when the second line combination includes multiple lines, splicing the video segments where the lines in the second line combination are located in the feature video to obtain an initial imitation short video corresponding to the second line combination; Using a generative artificial intelligence model, based on the content of the second line combination and the auxiliary information, predict the emotion expressed by the initial imitation short video corresponding to the second line combination; Determine from a preset background music library the second background music that matches the emotion expressed by the initial imitation short video corresponding to the second line combination, and add the second background music to the initial imitation short video corresponding to the second line combination to obtain a second imitation short video corresponding to the second line combination.

14. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 13 when executing the instructions stored in the memory.

15. A computer program product, characterized in that The invention comprises a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device executes the method according to any one of claims 1 to 13.

Images