Video association method, device, electronic device and storage medium

Abstract

The embodiment of the present application provides a video association method, device, electronic device and storage medium, the method comprising: determining a target video album corresponding to a target short video according to at least one target person in a target short video; obtaining a line information set corresponding to the target short video, the line information set comprising N lines data, each line data corresponding to a first duration; retrieving at least one first episode corresponding to each line data and a second duration corresponding to each first episode in a line database of the target video album; selecting at least one first episode from multiple first episodes in the line information set according to the first duration and the second duration corresponding to the N lines data, determining the selected first episode as a target long video, and establishing an association between the target short video and the target long video. The present application realizes the association between a target long video and a target short video, and improves the efficiency of finding a target long video.

Description

Technical Field

[0001] The present application relates to the field of video technology, and in particular to a video association method, device, electronic device and storage medium. Background Art

[0002] When users watch short videos, if they see content that interests them, they will have a strong desire to watch the long video corresponding to the short video. However, considering the wide range of sources of short videos, the corresponding long video sources are usually not marked. If users want to find long videos associated with short videos, they not only need to analyze the short videos they watch and find out the names of the characters, but also need to search for other videos of the characters again. There are many steps in the operation and the search efficiency is low. Therefore, it is urgent to provide a method for determining the long videos associated with short videos and establishing the association relationship between long and short videos. Summary of the invention

[0003] The embodiments of the present application provide a video association method, device, electronic device and storage medium to solve the problem of how to establish an association relationship between a short video and a corresponding long video.

[0004] In a first aspect of an embodiment of the present application, a video association method is provided, including:

[0005] Determine a target video album corresponding to the target short video according to at least one target person in the target short video;

[0006] Acquire a dialogue information set corresponding to the target short video, wherein the dialogue information set includes N dialogue data corresponding to the target short video, each of which corresponds to a first duration, wherein N is a positive integer;

[0007] Retrieving, from the dialogue database corresponding to the target video album, at least one first episode corresponding to each of the dialogue data and a second duration corresponding to each of the first episodes;

[0008] According to the first duration time and the second duration time respectively corresponding to the N pieces of dialogue data, at least one first episode is selected from the multiple first episodes corresponding to the dialogue information set, the selected first episode is determined as the target long video, and an association is established between the target short video and the target long video.

[0009] In a second aspect of the embodiment of the present application, a video association device is also provided, including:

[0010] A determination module, used to determine a target video album corresponding to the target short video according to at least one target person in the target short video;

[0011] A first acquisition module is used to acquire a dialogue information set corresponding to the target short video, wherein the dialogue information set includes N dialogue data corresponding to the target short video, each of which corresponds to a first duration, wherein N is a positive integer;

[0012] A retrieval module, configured to retrieve, from a dialogue database corresponding to the target video album, at least one first episode corresponding to each of the dialogue data and a second duration corresponding to each of the first episodes;

[0013] The first association establishing module is used to select at least one of the first episodes from the multiple first episodes corresponding to the line information set according to the first duration time period and the second duration time period respectively corresponding to the N lines of line data, determine the selected first episode as the target long video, and establish an association between the target short video and the target long video.

[0014] In a third aspect of the embodiments of the present application, an electronic device is further provided, including:

[0015] at least one processor; and a memory communicatively coupled to the at least one processor;

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned video association method.

[0017] In a fourth aspect of the embodiments of the present application, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned video association method.

[0018] In a fifth aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, and the computer program implements the above-mentioned video association method when executed by a processor.

[0019] The embodiments of the present application include at least the following technical effects:

[0020] The technical solution of the present application determines the target video album corresponding to the target short video through the target person in the target short video, and then retrieves the first episode corresponding to each of the N lines of data in the target short video in the lines database corresponding to the target video album, and selects at least one first episode from the multiple first episodes corresponding to the target short video as the target long video corresponding to the target short video according to the first duration corresponding to the N lines of data in the target short video and the second duration corresponding to the first episode, and establishes an association between the target long video and the target short video, thereby improving the efficiency of querying the target long video. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0022] Figure 1 A schematic diagram of a flow chart of a video association method provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of the structure of a video association device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0026] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0027] The embodiment of the present application provides a video association method, which can be applied to a server, wherein the server can be a background server of a playback application.

[0028] like Figure 1 As shown, the method may include:

[0029] Step 101: determine a target video album corresponding to the target short video according to at least one target person in the target short video.

[0030] The video resources provided by the playback application include some video resources that are cut from TV series, movies or variety shows and whose video length is less than the preset time. This application refers to such videos as short videos. For example, a clip of a TV series, movie or variety show or a video spliced ​​from different clips.

[0031] Specifically, a short video generally contains one or more characters. For a target short video (a certain short video), at least one character appearing in the target short video can be determined as a target character, and a target video album corresponding to the target short video can be determined through the target character. The target video album can be a TV series, movie or variety show corresponding to the target short video, and the target video album can include at least one episode. For a TV series, the target video album is the name of the TV series, and the number of episodes included in the target video album is the number of episodes included in the TV series; for a movie, the target video album is the name of the movie, and the target video album includes one episode, which is the movie; for a variety show, the target video album is the name of the variety show, and the number of episodes included in the target video album is the number of episodes included in the variety show.

[0032] The target video album to which the target long video corresponding to the target short video belongs can be determined through the target person in the target short video.

[0033] Step 102, obtaining a dialogue information set corresponding to the target short video, wherein the dialogue information set includes N dialogue data corresponding to the target short video, each of which corresponds to a first duration, wherein N is a positive integer.

[0034] By processing the target short video, such as performing text recognition on the video frames in the target short video, the line data appearing in the target short video and the duration of each line data in the target short video, i.e., the first duration, can be obtained. Based on the N lines of line data obtained, a line information set corresponding to the target short video can be obtained.

[0035] Step 103 , searching, in the dialogue database corresponding to the target video album, at least one first episode corresponding to each of the dialogue data and a second duration corresponding to each of the first episodes.

[0036] The speech database corresponding to the target video album stores speech data corresponding to all episodes in the target video album. For each speech data in the speech data set corresponding to the target short video, a search is performed in the speech database to obtain the episode in which each speech data appears, i.e., the first episode, and the number of the first episodes is at least one. The duration of each speech data in the corresponding first episode, i.e., the second duration, can also be obtained.

[0037] Step 104, based on the first duration time and the second duration time respectively corresponding to the N lines of dialogue data, select at least one of the first episodes from the multiple first episodes corresponding to the line information set, determine the selected first episode as the target long video, and establish an association between the target short video and the target long video.

[0038] Each of the N lines of dialogue data in the target short video corresponds to the first episode in the target video album, wherein each line of dialogue data corresponds to at least one first episode, and the number of first episodes corresponding to the line information set is the collection of first episodes corresponding to each line of dialogue data in the N lines of dialogue data. Therefore, the number of first episodes corresponding to the line information set is multiple, and at least one first episode needs to be selected from multiple first episodes to determine the target long video and establish an association between the target short video and the target long video. Specifically, a video association database can be set on the server, and the video association database is used to store identification information corresponding to the target short video and identification information corresponding to the target long video that have an association relationship. After determining the target long video, the identification information corresponding to the target long video and the identification information corresponding to the target short video are respectively obtained, and the two identification information are stored in the video association database.

[0039] Specifically, at least one first episode can be selected as the target long video from the plurality of first episodes corresponding to the dialogue information set according to the first duration period and the second duration period respectively corresponding to the N pieces of dialogue data.

[0040] In the above implementation process of the present application, the target video album corresponding to the target short video is determined through the target person in the target short video, and then the first episode corresponding to each of the N lines of data in the target short video is retrieved in the lines database corresponding to the target video album. According to the first duration corresponding to the N lines of data in the target short video and the second duration corresponding to the first episode, at least one first episode is selected from the multiple first episodes corresponding to the target short video as the target long video corresponding to the target short video, and an association between the target long video and the target short video is established, thereby improving the efficiency of querying the target long video.

[0041] In an optional embodiment of the present application, step 101 determines a target video album corresponding to the target short video according to at least one target person in the target short video, including:

[0042] Extracting a first video frame from the target short video according to a first frame extraction frequency;

[0043] Extracting a person from the first video frame to obtain the target person;

[0044] In a video album database, a target video album associated with the target person is searched, wherein the video album database stores a plurality of video albums, and each of the video albums is associated with at least one person.

[0045] Specifically, the first video frame can be extracted from the target short video according to the first frame extraction frequency, wherein the value of the first frame extraction frequency can be determined according to the length of the target short video, the longer the target short video is, the smaller the first frame extraction frequency is, and the shorter the target short video is, the larger the first frame extraction frequency is, so as to ensure that the number of first video frames extracted according to the first frame extraction frequency can meet the preset conditions. Only when the number of first video frames meets the preset conditions can it be ensured that a sufficient number of characters can be extracted from the first video frames.

[0046] By performing character extraction on the first video frame, at least one character included in the first video frame can be obtained. Specifically, a face detection model can be used to obtain at least one face image in the first video frame, and then face recognition is performed on each face image through a face recognition model (the face recognition model can perform feature extraction on the face image to be identified, and search for a character matching the features corresponding to the face image from a character database, wherein a large number of characters and a feature set corresponding to each character are stored in the character database), and the character corresponding to each face image is determined. The target character can be a character whose appearance frequency is greater than a preset frequency threshold among at least one character extracted from the target short video.

[0047] A video album database is provided in the backend server, and a plurality of video albums are stored in the video album database, and each video album is associated with at least one person. Then, a target video album associated with a target person can be searched in the video album.

[0048] In the above implementation process of the present application, by determining the target person in the video frame of the target video, the target video album associated with the target person can be searched from the video album database, thereby improving the efficiency of searching for video albums.

[0049] In an optional embodiment of the present application, step 102 of obtaining a line information set corresponding to the target short video includes:

[0050] Extracting a second video frame from the target short video according to a second frame extraction frequency;

[0051] Performing text recognition on each of the second video frames to obtain corresponding lines in each of the second video frames;

[0052] Determining N pieces of dialogue data corresponding to the target short video according to the dialogue content corresponding to each of the second video frames;

[0053] Determining the speech information set according to the N pieces of speech data;

[0054] In the process of determining the N lines of dialogue data corresponding to the target short video, the first duration period corresponding to each line of dialogue data is determined based on the frame extraction moment corresponding to each second video frame.

[0055] Specifically, the second video frame can be extracted from the target short video according to the second frame extraction frequency, and all the lines in the target short video can be obtained by performing text recognition on each second video frame. For example, the second frame extraction frequency can be set to three frames per second to prevent missing the lines in the text recognition process. After determining the lines, each line data needs to be tracked to determine the duration of each line data in the target short video, that is, the first duration. Specifically, the first duration corresponding to each line data can be determined by the frame extraction time corresponding to the second video frame. For a certain line data, the frame extraction time corresponding to the second video frame where it first appears is the start time of the line data. Once there is no line or another line data appears in the second video frame after the start time, the frame extraction time corresponding to the previous second video frame where there is no line or another line data appears is determined as the end time of the line data.

[0056] Once another line data appears in the second video frame after the start time, the frame extraction time corresponding to the second video frame before the second video frame corresponding to the other line data appears can be determined as the end time of the line data.

[0057] In the above implementation process of the present application, the dialogue information set corresponding to the target short video and the first duration period corresponding to each dialogue data can be determined through the dialogue content in the video frame of the target short video.

[0058] In an optional embodiment of the present application, step 102 includes performing text recognition on each of the second video frames to obtain the line content in each of the second video frames, including:

[0059] Performing text recognition on each of the second video frames to obtain text in each of the second video frames and positions of the text in the second video frames;

[0060] Determine a target position in the second video frame according to the position of the text in each second video frame, wherein the target position is a position for displaying the content of the lines;

[0061] According to the target position, the lines are extracted from the text in each of the second video frames.

[0062] Specifically, in the process of obtaining the content of the lines in each second video frame, considering that all the characters appearing in the second video frame will be recognized during the text recognition process, it is necessary to determine the content of the lines. Since in a video, the lines generally have the following characteristics: the height of the lines is fixed (the lines are located at a fixed distance from the bottom edge of the image), the lines are located on the left, center or right of the image, and the length of the lines changes dynamically (the length of the lines is determined by the number of words and letters contained in the lines). Then, according to the frequency of updating the text at the position where the line content is displayed is greater than the frequency of updating the text at other positions, the target position for displaying the line content in the second video frame can be determined by analyzing the position of the text in each second video frame in the second video frame. After determining the target position, the text at other positions outside the target position in each second video frame can be determined as non-line text and filtered out, and the text at the target position in each second video frame is determined to be the line content corresponding to the second video frame.

[0063] In the above implementation process of the present application, by determining the target position for displaying the dialogue content in the target short video, the dialogue content can be determined from the text in the second video frame, and the dialogue content in each second video frame can be accurately identified to prevent other texts from being mistaken for dialogue content.

[0064] In an optional embodiment of the present application, after establishing the association between the target short video and the target long video in step 104, the method further includes:

[0065] Obtaining the second duration time period corresponding to each of the lines in the target short video in the target long video;

[0066] According to the second duration period, at least one target period in the target long video associated with the target short video is determined, and an association is established between the target short video and the target period, wherein a period obtained by splicing at least two consecutive second duration periods among multiple second duration periods is the target period.

[0067] Specifically, after establishing an association between the target short video and the target long video, the user can directly jump to the corresponding target long video by clicking the jump button on the playback interface of the target short video during the process of watching the target short video. However, if the duration of the target long video is longer than the preset duration, the user still cannot determine the specific position of the target short video in the target long video. Therefore, in order to improve the user experience, the target short video can also be associated with the target time period in the target long video.

[0068] Among them, the target time period specifically associated with the target short video in the target long video can be determined by the second duration of each line data in the target short video in the target long video. If the target short video is a complete segment in the target long video, the number of target time periods is one; if the target short video is spliced ​​from multiple segments in the target long video, the number of target time periods is multiple.

[0069] It should be noted that, by establishing an association between the target short video and the target long video, the user watching the target short video can send a jump instruction to the server by clicking the jump button set on the playback interface of the target short video. When the server detects the jump instruction triggered by the user, the target long video associated with the target short video is searched based on the association between the target short video and the target long video. For example, the target long video associated with the target short video is searched from the video association database (used to store the identification information corresponding to the target short video with an associated relationship and the identification information corresponding to the target long video). And the playback content of the current playback interface is switched from the target short video to the target long video, specifically, the link corresponding to the jump button can be set to the link corresponding to the target long video, so that after the user clicks the challenge button, it can jump to the corresponding target long video.

[0070] Furthermore, in order to enable users who are watching the target short video to jump to the target time period of the target long video corresponding to the target short video by clicking a jump button on the playback interface of the target short video, the time information of the target time period corresponding to the target long video can be added to the video association database (the video association database is used to store the identification information corresponding to the target short video and the identification information corresponding to the target long video that have an associated relationship) (the time information includes the start time and end time of the target time period in the target long video), thereby establishing an association between the target short video and the target time period in the target long video, so that users who are watching the target short video can jump to the jump button of the target time period corresponding to the target long video and the target time period corresponding to the target long video by clicking the indication on the playback interface of the target short video (when the server detects that the user clicks the jump button, it searches the video association database for the time information of the target time period corresponding to the target long video that has an associated relationship with the target short video, and sets the link corresponding to the jump button to the link of the target time period corresponding to the target long video according to the time information corresponding to the target time period).

[0071] The above-mentioned implementation process of the present application can associate the target short video with the target time period in the target long video, thereby realizing the precise positioning of the target short video in the target long video. The user can know which time period in the target long video the target short video currently being watched specifically comes from, thereby facilitating the user to find the target short video in the target long video and improving the user experience.

[0072] In an optional embodiment of the present application, in step 104, according to the first durations and the second durations respectively corresponding to the N pieces of speech data, at least one first episode is selected from the plurality of first episodes corresponding to the speech information set, and the selected first episode is determined as the target long video, including:

[0073] Determining whether the first speech data and the second speech data meet a merging condition according to the first duration and the second duration corresponding to the first speech data and the first duration and the second duration corresponding to the second speech data adjacent to the first speech data;

[0074] In the case where the first speech data and the second speech data meet the merging condition, merging the first speech data and the second speech data to obtain first merged speech data, and determining the second episode corresponding to the first merged speech data, the third duration corresponding to the second episode, and the fourth duration corresponding to the target short video;

[0075] In the case where the first merged speech data and the adjacent third speech data meet the merging condition, the first merged speech data and the third speech data are merged to obtain second merged speech data, and the merging step is continued based on the second merged speech data until the merging condition is no longer met or the merging is terminated when the last speech data in the target short video is merged, so as to determine the target long video;

[0076] When the first merged speech data and the third speech data do not meet the merging condition, the second episode is determined to be the target long video corresponding to the target short video, and the merging step is continued based on the third speech data until the merging condition is no longer met or the merging is terminated when the last speech data in the target short video is merged to determine the target long video.

[0077] Specifically, when determining the target long video corresponding to the target short video, the embodiment of the present application can merge the adjacent first and second speech data that meet the merging conditions in the target short video to obtain first merged speech data, determine the second episode corresponding to the first merged speech data, and then merge the first merged speech data with the adjacent third speech data.

[0078] It should be noted that the first speech data and the second speech data are two adjacent speech data, which may be two speech data located at the beginning of the target short video, or two speech data located at other positions in the target short video, and this application does not specifically limit this. In addition, the first speech data and the second speech data may be corresponding speech data in the target short video, or may be merged speech data obtained by merging corresponding speech data in the target short video. Among them, the third speech data adjacent to the first merged speech data may be speech data located before the first merged speech data, or may be speech data located after the first merged speech data.

[0079] If the first merged speech data and the third speech data meet the merge condition, then the first merged speech data and the third merged speech data are merged to obtain second merged speech data, and the merging step is continued based on the second merged speech data until the merge condition is no longer met or the merge is terminated when the last speech data in the target short video is merged, and it is determined that the third episode corresponding to the third merged speech data obtained before the merge condition is no longer met is the target long video corresponding to the target short video, and the fourth episode corresponding to the fourth merged speech data obtained when the merge is merged to the last speech data is the target long video corresponding to the target short video.

[0080] If the first merged speech data and the third speech data do not meet the merging condition, the second episode corresponding to the first merged speech data is determined to be the target long video corresponding to the target short video, and the merging step is continued based on the third speech data until the merging condition is no longer met or the merging is terminated when the last speech data in the target short video is merged, and the fifth episode corresponding to the fifth merged speech data obtained before the merging condition is no longer met is determined to be the target long video corresponding to the target short video, and the sixth episode corresponding to the sixth merged speech data obtained when the last speech data is merged is determined to be the target long video corresponding to the target short video.

[0081] In the above implementation process of the present application, by merging the dialogue data in the target short video, the number of the first episodes corresponding to the target short video can be gradually reduced, thereby reducing the search range of the target long video. When merging is impossible, the target long video can be determined based on the episodes corresponding to the merged dialogue data, thereby improving the efficiency of finding the target long video. Furthermore, the user can jump directly from the target short video to the corresponding target long video by clicking the jump button on the playback interface of the target short video indicating to jump to the target long video.

[0082] In an optional embodiment of the present application, the first speech data is located before the second speech data, and step 104 includes judging whether the first speech data and the second speech data meet a merging condition according to the first duration and the second duration corresponding to the first speech data and the first duration and the second duration corresponding to the second speech data adjacent to the first speech data, including:

[0083] According to the first duration periods respectively corresponding to the first speech data and the second speech data, obtaining a first duration between a first end time of the first speech data and a first start time of the second speech data;

[0084] For each first target episode, obtaining a second duration between a second end time corresponding to the first speech data and a second start time corresponding to the second speech data, wherein the first target episode includes the first speech data and the second speech data;

[0085] In the case that there is a target second duration in the at least one second duration whose difference with the first duration is less than a preset time threshold, it is determined that the first speech data and the second speech data meet the merging condition.

[0086] Specifically, it is possible to determine whether the first speech data and the second speech data meet the merging condition by the first duration of the first speech data in the target short video and the second duration of the first episode, and the first duration of the second speech data in the target short video and the second duration of the second episode. The duration between the first speech data and the second speech data in the target short video is the first duration, that is, the duration between the first end time of the first speech data and the first start time of the second speech data. The duration between the first speech data and the second speech data in the first target episode is the second duration, that is, the duration between the second end time of the first speech data and the second start time of the second speech data. The first target episode includes the first speech data and the second speech data, that is, the first target episode is the intersection of the first episode corresponding to the first speech data and the first episode corresponding to the second speech data. The number of the first target episodes is at least one, and each first target episode corresponds to a second duration, so the number of the second durations is also at least one.

[0087] In the case where there is a target second duration in at least one second duration whose difference with the first duration is less than the preset time threshold, it is determined that the first speech data and the second speech data meet the merging condition. In other words, when the duration between the first speech data and the second speech data in the first target episode is close to the duration between the first speech data and the second speech data in the target short video (the difference is less than the preset time threshold), it can be determined that the first speech data and the second speech data meet the merging condition.

[0088] In the above implementation process of the present application, by comparing the relationship between the duration of the interval between the first line data and the second line data in the first target episode and the duration of the interval between the first line data and the second line data in the target short video, it can be quickly determined whether the first line data and the second line data can be merged, thereby achieving a reduction in the number of first episodes corresponding to the target short video, and gradually narrowing the search range for the target long video.

[0089] Specifically, determining the second episode corresponding to the first merged dialogue data, the third duration corresponding to the second episode, and the fourth duration corresponding to the target short video includes:

[0090] Determine that the first episode corresponding to the target second duration is the second episode corresponding to the first combined lines;

[0091] Determine the time period between the start time of the first speech data in the second episode and the end time of the second speech data in the second episode as the third continuous time period;

[0092] Determine that the time period between the start time of the first speech data in the target short video and the end time of the second speech data in the target short video is the fourth duration period.

[0093] In an optional embodiment of the present application, in step 103, searching for at least one first episode corresponding to each piece of the speech data in the speech database corresponding to the target video album includes:

[0094] For each of the speech data, searching the speech database for target speech data whose matching degree with the speech data meets a preset condition;

[0095] The episode in which the target speech data is located is determined to be the first episode corresponding to the speech data.

[0096] During the process of text recognition, there may be certain misrecognitions. For example, the character "日" may be recognized as "目". In order to prevent the misrecognition from affecting the determination of the final first episode, when searching for line data in the line database, a preset matching degree threshold can be set. As long as the matching degree between the searched line data and the line data in the line database meets the preset condition, that is, is greater than the preset matching degree threshold, it is considered that the two line data match successfully. Then, the episode in which the target line data found in the line database is located is the first episode corresponding to the line data.

[0097] The above implementation process of this application can prevent the misrecognition in the process of line content recognition from affecting the determination of the first episode.

[0098] In an optional embodiment of this application, after establishing the association between the target short video and the target long video in step 101, the method further includes:

[0099] When detecting a jump instruction triggered by a user watching the target short video, based on the association between the target short video and the target long video, search for the target long video associated with the target short video, and control the playback content of the current playback interface to switch from the target short video to the target long video.

[0100] Specifically, after establishing the association relationship between the target short video and the target long video, the user watching the target short video can send a jump instruction to the server by clicking a jump button set on the playback interface of the target short video. When the server detects the jump instruction triggered by the user, based on the association between the target short video and the target long video, search for the target long video associated with the target short video. For example, search for the target long video associated with this target short video from a video association database (used to store the identification information corresponding to the target short video and the identification information corresponding to the target long video) with an association relationship. And switch the playback content of the current playback interface from the target short video to the target long video. Specifically, the link corresponding to the jump button can be set to the link corresponding to the target long video. Then, after the user clicks the challenge button, they can jump to the corresponding target long video, thus saving the time for the user to search for the target long video by themselves.

[0101] The above implementation process of this application enables the user watching the target short video to click the jump button to send a jump instruction to the server, directly jump from the target short video to the corresponding target long video, improving the user experience. At the same time, it can also realize the function of short video playback driving long video playback, effectively increasing the opportunities and scenarios for long video playback.

[0102] The above is the overall implementation process of the video association method provided in the embodiment of the present application. The target video album corresponding to the target short video is determined by the target person in the target short video, and then the first episode corresponding to each of the N lines of data in the target short video is retrieved in the lines database corresponding to the target video album. According to the first duration corresponding to the N lines of data in the target short video and the second duration corresponding to the first episode, at least one first episode is selected from the multiple first episodes corresponding to the target short video as the target long video corresponding to the target short video, and an association is established between the target long video and the target short video, thereby improving the efficiency of querying the target long video. Furthermore, it is convenient for the user to directly jump from the target short video to the corresponding target long video by clicking the jump button on the playback interface of the target short video indicating to jump to the target long video. At the same time, the function of short video playback driving long video playback can also be realized, effectively increasing the opportunities and scenarios for long video playback.

[0103] The present application also provides a video association device, such as Figure 2 As shown, the device comprises:

[0104] A determination module 201 is used to determine a target video album corresponding to the target short video according to at least one target person in the target short video;

[0105] A first acquisition module 202 is used to acquire a dialogue information set corresponding to the target short video, wherein the dialogue information set includes N dialogue data corresponding to the target short video, each of which corresponds to a first duration, wherein N is a positive integer;

[0106] A search module 203 is used to search, in the dialogue database corresponding to the target video album, at least one first episode corresponding to each of the dialogue data and a second duration corresponding to each of the first episodes;

[0107] The first association establishing module 204 is used to select at least one of the first episodes from the multiple first episodes corresponding to the line information set according to the first duration time period and the second duration time period respectively corresponding to the N lines of line data, determine the selected first episode as the target long video, and establish an association between the target short video and the target long video.

[0108] Optionally, the determining module 201 includes:

[0109] A first extraction submodule, configured to extract a first video frame from the target short video according to a first frame extraction frequency;

[0110] A first acquisition submodule, used to extract a person from the first video frame to obtain the target person;

[0111] The search submodule is used to search for a target video album associated with the target person in a video album database, wherein the video album database stores a plurality of video albums, each of which is associated with at least one person.

[0112] Optionally, the first acquisition module 202 includes:

[0113] A second extraction submodule, used to extract a second video frame from the target short video according to a second frame extraction frequency;

[0114] A second acquisition submodule is used to perform text recognition on each of the second video frames to obtain the corresponding lines in each of the second video frames;

[0115] A first determination submodule, configured to determine N lines of dialogue data corresponding to the target short video according to the dialogue content corresponding to each of the second video frames;

[0116] A second determination submodule, configured to determine the speech information set according to the N pieces of speech data;

[0117] In the process of determining the N lines of dialogue data corresponding to the target short video, the first duration period corresponding to each line of dialogue data is determined based on the frame extraction moment corresponding to each second video frame.

[0118] Optionally, the second acquisition submodule includes:

[0119] A first acquisition unit, configured to perform text recognition on each of the second video frames, and acquire text in each of the second video frames and a position of the text in the second video frame;

[0120] A first determining unit, configured to determine a target position in the second video frame according to a position of the text in each second video frame in the second video frame, wherein the target position is a position for displaying the content of the dialogue;

[0121] The extraction unit is used to extract the content of the lines from the text in each of the second video frames according to the target position.

[0122] Optionally, the device further comprises:

[0123] A second acquisition module is used to acquire the second duration period corresponding to each line data in the target short video in the target long video;

[0124] The second association establishing module is used to determine at least one target time period associated with the target short video in the target long video according to the second duration period, and establish an association between the target short video and the target time period, wherein a time period obtained by splicing at least two consecutive second duration periods among multiple second duration periods is the target time period.

[0125] Optionally, the first association establishing module 204 includes:

[0126] a judgment submodule, for judging whether the first speech data and the second speech data meet a merging condition according to a first duration period and a second duration period corresponding to the first speech data and a first duration period and a second duration period corresponding to the second speech data adjacent to the first speech data, wherein the first speech data is located before the second speech data;

[0127] a third determination submodule, configured to merge the first speech data and the second speech data to obtain first merged speech data when the first speech data and the second speech data meet a merging condition, and determine a second episode corresponding to the first merged speech data, a third duration corresponding to the second episode, and a fourth duration corresponding to the target short video;

[0128] A fourth determination submodule is used for merging the first merged speech data and the third speech data to obtain second merged speech data when the first merged speech data and the adjacent third speech data meet the merging condition, and continuing to perform the merging step based on the second merged speech data until the merging condition is no longer met or the merging is terminated when the last speech data in the target short video is merged, so as to determine the target long video;

[0129] The fifth determination submodule is used to determine that the second episode is the target long video corresponding to the target short video when the first merged dialogue data and the third dialogue data do not meet the merging condition, and continue to perform the merging step based on the third dialogue data until the merging condition is not met or the merging is terminated when the last dialogue data in the target short video is merged to determine the target long video.

[0130] Optionally, the first dialogue data is located before the second dialogue data, and the judgment submodule includes:

[0131] A second acquisition unit, configured to acquire a first duration between a first end time of the first speech data and a first start time of the second speech data according to first durations respectively corresponding to the first speech data and the second speech data;

[0132] A third acquisition unit is used to acquire, for each first target episode, a second duration between a second end time corresponding to the first speech data and a second start time corresponding to the second speech data, wherein the first target episode includes the first speech data and the second speech data;

[0133] A second determining unit is configured to determine that the first speech data and the second speech data satisfy a merging condition when there is a target second duration in the at least one second duration whose difference with the first duration is less than a preset time threshold.

[0134] Optionally, the third determining submodule includes:

[0135] A third determining unit, configured to determine that the first episode corresponding to the target second duration is the second episode corresponding to the first combined lines;

[0136] A fourth determining unit, configured to determine a period between a start time of the first speech data in the second episode and an end time of the second speech data in the second episode as the third continuous period;

[0137] The fifth determining unit is used to determine that the time period between the start time of the first dialogue data in the target short video and the end time of the second dialogue data in the target short video is the fourth duration period.

[0138] Optionally, the retrieval module 203 includes:

[0139] A search submodule, for searching, for each of the speech data, in the speech database for target speech data whose matching degree with the speech data meets a preset condition;

[0140] The sixth determination submodule is used to determine that the episode in which the target speech data is located is the first episode corresponding to the speech data.

[0141] Optionally, the video association device further includes:

[0142] The control module is used to, when detecting a jump instruction triggered by a user watching the target short video, search for a target long video associated with the target short video based on the association between the target short video and the target long video, and control the playback content of the current playback interface to switch from the target short video to the target long video.

[0143] The video association device provided in the embodiment of the present application determines the target video album corresponding to the target short video through the target person in the target short video, and then retrieves the first episode corresponding to each of the N lines of data in the target short video in the lines database corresponding to the target video album, and selects at least one first episode from the multiple first episodes corresponding to the target short video as the target long video corresponding to the target short video according to the first duration corresponding to the N lines of data in the target short video and the second duration corresponding to the first episode, and establishes an association between the target long video and the target short video, thereby improving the efficiency of querying the target long video.

[0144] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0145] The present application also provides an electronic device, including:

[0146] at least one processor; and a memory communicatively coupled to the at least one processor;

[0147] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned video association method.

[0148] The memory mentioned in the above electronic device may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the above processor.

[0149] The processor mentioned in the above electronic device can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0150] The embodiment of the present application also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the above-mentioned video association method.

[0151] An embodiment of the present application further provides a computer program product, including a computer program, which implements the above-mentioned video association method when executed by a processor.

[0152] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0153] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0154] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0155] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A video association method, characterized in that: The method comprises: Determine a target video album corresponding to the target short video according to at least one target person in the target short video; Acquire a dialogue information set corresponding to the target short video, wherein the dialogue information set includes N dialogue data corresponding to the target short video, each of which corresponds to a first duration, wherein N is a positive integer; Retrieving, from the dialogue database corresponding to the target video album, at least one first episode corresponding to each of the dialogue data and a second duration corresponding to each of the first episodes; According to the first duration and the second duration corresponding to the first speech data and the first duration and the second duration corresponding to the second speech data adjacent to the first speech data, it is determined whether the first speech data and the second speech data meet the merging condition; if the first speech data and the second speech data meet the merging condition, the first speech data and the second speech data are merged to obtain the first merged speech data, and the second episode corresponding to the first merged speech data, the third duration corresponding to the second episode, and the fourth duration corresponding to the target short video are determined; if the first merged speech data and the adjacent third speech data meet the merging condition, In the case of a condition where the first merged speech data and the third speech data are merged to obtain second merged speech data, and the merging step is continued based on the second merged speech data until the merging condition is not met or the merging is terminated when the last speech data in the target short video is merged, so as to determine the target long video; in the case where the first merged speech data and the third speech data do not meet the merging condition, the second episode is determined to be the target long video corresponding to the target short video, and the merging step is continued based on the third speech data until the merging condition is not met or the merging is terminated when the last speech data in the target short video is merged, so as to determine the target long video; An association is established between the target short video and the target long video.

2. The video association method according to claim 1, characterized in that: The step of determining a target video album corresponding to the target short video according to at least one target person in the target short video includes: Extracting a first video frame from the target short video according to a first frame extraction frequency; Performing person extraction on the first video frame to obtain the target person; In a video album database, a target video album associated with the target person is searched, wherein the video album database stores a plurality of video albums, and each of the video albums is associated with at least one person.

3. The video association method according to claim 1, characterized in that: The step of obtaining a dialogue information set corresponding to the target short video includes: Extracting a second video frame from the target short video according to a second frame extraction frequency; Performing text recognition on each of the second video frames to obtain corresponding lines in each of the second video frames; Determining N pieces of dialogue data corresponding to the target short video according to the dialogue content corresponding to each of the second video frames; Determining the speech information set according to the N pieces of speech data; In the process of determining the N lines of dialogue data corresponding to the target short video, the first duration period corresponding to each line of dialogue data is determined based on the frame extraction moment corresponding to each second video frame.

4. The video association method according to claim 3, characterized in that: The performing text recognition on each of the second video frames to obtain the dialogue content in each of the second video frames includes: Performing text recognition on each of the second video frames to obtain text in each of the second video frames and positions of the text in the second video frames; Determine a target position in the second video frame according to the position of the text in each second video frame, wherein the target position is a position for displaying the content of the lines; According to the target position, the lines are extracted from the text in each of the second video frames.

5. The video association method according to claim 1, characterized in that: After establishing the association between the target short video and the target long video, the method further includes: Obtaining the second duration time period corresponding to each of the lines in the target short video in the target long video; According to the second duration period, at least one target period in the target long video associated with the target short video is determined, and an association is established between the target short video and the target period, wherein a period obtained by splicing at least two consecutive second duration periods among multiple second duration periods is the target period.

6. The video association method according to claim 1, characterized in that: The first speech data is located before the second speech data, and judging whether the first speech data and the second speech data meet a merging condition according to the first duration and the second duration corresponding to the first speech data and the first duration and the second duration corresponding to the second speech data adjacent to the first speech data includes: According to the first duration periods respectively corresponding to the first speech data and the second speech data, obtaining a first duration between a first end time of the first speech data and a first start time of the second speech data; For each first target episode, obtaining a second duration between a second end time corresponding to the first speech data and a second start time corresponding to the second speech data, wherein the first target episode includes the first speech data and the second speech data; In the case that there is a target second duration in the at least one second duration whose difference with the first duration is less than a preset time threshold, it is determined that the first speech data and the second speech data meet the merging condition.

7. The video association method according to claim 6, characterized in that: The determining the second episode corresponding to the first merged dialogue data, the third duration corresponding to the second episode, and the fourth duration corresponding to the target short video includes: Determine that the first episode corresponding to the target second duration is the second episode corresponding to the first combined lines; Determine the time period between the start time of the first speech data in the second episode and the end time of the second speech data in the second episode as the third continuous time period; Determine that the time period between the start time of the first speech data in the target short video and the end time of the second speech data in the target short video is the fourth duration period.

8. The video association method according to claim 1, characterized in that: The step of retrieving at least one first episode corresponding to each piece of the line data from the line database corresponding to the target video album comprises: For each of the speech data, searching the speech database for target speech data whose matching degree with the speech data meets a preset condition; The episode in which the target speech data is located is determined to be the first episode corresponding to the speech data.

9. The video association method according to claim 1, characterized in that: After establishing the association between the target short video and the target long video, the method further includes: When a jump instruction triggered by a user watching the target short video is detected, based on the association between the target short video and the target long video, the target long video associated with the target short video is searched, and the playback content of the current playback interface is controlled to switch from the target short video to the target long video.

10. A video association device, characterized in that: The device comprises: A determination module, used to determine a target video album corresponding to the target short video according to at least one target person in the target short video; A first acquisition module is used to acquire a dialogue information set corresponding to the target short video, wherein the dialogue information set includes N dialogue data corresponding to the target short video, each of which corresponds to a first duration, wherein N is a positive integer; A retrieval module, configured to retrieve, from a dialogue database corresponding to the target video album, at least one first episode corresponding to each of the dialogue data and a second duration corresponding to each of the first episodes; The first association establishment module is used to determine whether the first speech data and the second speech data meet a merging condition according to a first duration and a second duration corresponding to the first speech data and a first duration and a second duration corresponding to the second speech data adjacent to the first speech data; if the first speech data and the second speech data meet the merging condition, merge the first speech data and the second speech data to obtain first merged speech data, and determine a second episode corresponding to the first merged speech data, a third duration corresponding to the second episode, and a fourth duration corresponding to the target short video; if the first merged speech data and the adjacent third speech data meet the merging condition, The first merged speech data and the third speech data are merged to obtain second merged speech data, and the merging step is continued based on the second merged speech data until the merging condition is not met or the merging is terminated when the last speech data in the target short video is merged to determine the target long video; when the first merged speech data and the third speech data do not meet the merging condition, the second episode is determined to be the target long video corresponding to the target short video, and the merging step is continued based on the third speech data until the merging condition is not met or the merging is terminated when the last speech data in the target short video is merged to determine the target long video, and an association between the target short video and the target long video is established.

11. An electronic device, characterized in that: include: at least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the video association method described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the video association method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the video association method according to any one of claims 1 to 9 when being executed by a processor.

Images