Video identification method and device, computer device and computer program

The method enhances video identification accuracy by combining local and platform reference videos to identify and skip similar segments, improving playback efficiency and comparison processing.

JP2024541832AActive Publication Date: 2024-11-13TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2024522040
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-06-20
Filing Date
2023-04-18
Publication Date
2024-11-13
Estimated Expiration
2043-04-18

AI Technical Summary

Technical Problem

Existing video identification methods struggle with low accuracy in identifying similar video segments such as openings and endings across different videos, which are crucial for video editing and comparison processes.

Method used

A method and device for video identification that involves obtaining a target video and a reference video, performing video frame matching to identify locally and globally similar segments, and determining an overall similar segment by combining the positions of these segments, using a combination of local and platform reference videos.

Benefits of technology

Improves the accuracy of identifying and skipping similar video segments like openings and endings, enhancing video playback efficiency and reducing data processing for comparison.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024541832000001_ABST
    Figure 2024541832000001_ABST
Patent Text Reader

Abstract

A video identification method executed by a computer device, comprising: a step (202) of acquiring a target video and a video set reference video in a video series video set, where the video series video set includes videos belonging to the same series; a step (204) of identifying a video set local similar segment in the target video to the video set reference video based on a first matching result obtained by video frame matching between the target video and the video set reference video; a step (206) of acquiring a platform reference video from a video platform to which the target video belongs; a step (208) of identifying a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video; and a step (210) of determining an overall similar segment in the target video to the video set reference video and the platform reference video based on the positions in the target video of the video set local similar segment and the platform global similar segment, respectively.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to a Chinese patent application filed with the China Patent Office on June 20, 2022, bearing application number 202210695301.5 and entitled "Video Identification Method and Apparatus, Computer Equipment, and Storage Medium," the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of computer technology, and in particular to a video identification method and apparatus, a computer device, a storage medium and a computer program product. [Background technology]

[0003] With the development of computer technology, various network video platforms have emerged, and in addition to the movie video resources on the network, people can also independently create various videos on the network video platform, including various kinds of videos such as lecture series, knowledge sharing, literary classes, commentary episodes, entertainment videos, etc., to meet the new viewing needs of viewers. Similar video segments such as openings and endings are usually created in videos on various network video platforms, and these video segments are not the content of the video itself, and when performing video comparison or video editing processing, these video segments need to be identified and removed. However, currently, the identification accuracy of similar video segments such as openings and endings in videos is low. Summary of the Invention

[0004] According to various embodiments provided herein, a video identification method and apparatus, a computer device, a computer readable storage medium, and a computer program product are provided.

[0005] In a first aspect, the present invention provides a method for video identification implemented by a computing device, said method comprising: Obtaining a target video and a video set reference video in a video series movie set, where the video series movie set includes videos that belong to the same series; identifying a video set locally similar segment in the target video to the video set reference video based on a first matching result obtained by video frame matching between the target video and the video set reference video; obtaining a platform reference video from a video platform to which the target video belongs; identifying a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video; determining overall similar segments in the target video to the video set reference video and the platform reference video based on the positions in the target video of the video set local similar segments and the platform global similar segments, respectively.

[0006] In a second aspect, the present invention further provides a video identification apparatus, said apparatus comprising: a video set video acquisition module configured to acquire a target video and a video set reference video in a video series video set, where the video series video set includes videos that belong to the same series; a local similar segment identification module configured to identify a video set locally similar segment in the target video to the video set reference video based on a first matching result obtained by video frame matching between the target video and the video set reference video; a platform video acquisition module configured to acquire a platform reference video from a video platform to which the target video belongs; a global similar segment identification module configured to identify a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video; and an overall similar segment determination module configured to determine overall similar segments in the target video to the video set reference video and the platform reference video based on positions in the target video of the video set local similar segments and the platform global similar segments, respectively.

[0007] In a third aspect, the present invention further provides a computing device comprising a memory in which computer readable instructions are stored, and a processor which, when executing the computer readable instructions, implements the above video identification method.

[0008] In a fourth aspect, the present invention further provides a computer-readable storage medium having stored thereon computer-readable instructions which, when executed by a processor, implement the above video identification method.

[0009] In a fifth aspect, the present invention further provides a computer program product comprising computer readable instructions which, when executed by a processor, implements the video identification method set forth above.

[0010] The details of one or more embodiments of the invention are set forth in the drawings and description which follow. Other features, objects, and advantages of the invention will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is an application environment diagram of a video identification method in one embodiment. [Diagram 2] 1 is a flow chart of a video identification method in one embodiment. [Diagram 3] 13 is a flowchart of a process for identifying platform global similar segments in one embodiment. [Figure 4] 1 is a flowchart for creating a user compilation video in one embodiment. [Diagram 5] 1 is a flow chart of video comparison in one embodiment. [Figure 6] FIG. 1 is an interface diagram introducing the opening of a platform screen in one embodiment. [Figure 7] FIG. 2 is a schematic diagram of an interface for playing video main content in one embodiment. [Figure 8] FIG. 13 is a schematic diagram of an interface introducing the ending of a platform screen in one embodiment. [Figure 9] FIG. 11 is a schematic interface diagram of the video platform introduction screen for the first period in one embodiment. [Figure 10] FIG. 11 is a schematic interface diagram of the video platform introduction screen for the second period in one embodiment. [Figure 11] 1 is an overall flowchart of a method for identifying openings and endings in one embodiment. [Figure 12] FIG. 1 is a schematic block diagram of an opening and ending mining method according to one embodiment. [Figure 13] FIG. 13 is a schematic diagram of an opening correction process in one embodiment. [Figure 14] FIG. 13 is a schematic diagram of an opening correction process in one embodiment. [Figure 15] FIG. 13 is a schematic diagram of matching segment information in one embodiment. [Figure 16] FIG. 2 is a schematic diagram illustrating time periods in one embodiment. [Figure 17] FIG. 13 is a schematic diagram illustrating an example of updating an end time when an overlapping portion exists in a time period. [Figure 18]FIG. 11 is a schematic diagram showing how to update the start point time when an overlapping portion exists in a time period in one embodiment. [Figure 19] FIG. 11 is a schematic diagram showing an example in which overlapping portions exist between time periods; [Figure 20] FIG. 13 is a schematic diagram of updating the count of recommended openings and endings in one embodiment. [Figure 21] FIG. 1 is a block diagram showing a configuration of a video identification device according to an embodiment. [Figure 22] FIG. 2 is a diagram showing the internal configuration of a computer device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present invention will be described in more detail below with reference to the accompanying drawings and examples in order to make the objectives, technical solutions and advantages of the present invention clearer. In addition, the specific examples described in this specification are merely for interpreting the present invention, and are not intended to limit the present invention.

[0013] The video identification method provided by the embodiment of the present invention can be applied to an application environment as shown in Fig. 1. Here, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated into the server 104, or can be located on a cloud, or can be located on another server. The server 104 performs video frame matching on a target video in a video series video set and a video set reference video, and identifies a video set local similar segment in the target video to the video set reference video according to the obtained first matching result, performs video frame matching on a platform reference video of a video platform to which the target video belongs to the target video, and identifies a platform global similar segment in the target video to the platform reference video according to the obtained second matching result, and determines an overall similar segment in the target video according to the positions in the target video of the video set local similar segment and the platform global similar segment. When the terminal 102 plays the target video, the server 104 sends segment information of the overall similar segments in the target video to the video set reference video and the platform reference video to the terminal 102, and the terminal 102 can skip the playback of the overall similar segments in the target video based on the received segment information, and if the overall similar segment is an opening or an ending, can skip the playback of the opening or the ending, so as to improve the video playback efficiency of the terminal 102. Furthermore, the video identification method provided in the present invention may be independently performed by the terminal 102 or the server 104, or may be jointly performed by the terminal 102 and the server 104 to realize the video identification process. Here, the terminal 102 includes, but is not limited to, various desktop computers, notebook computers, smartphones, tablet PCs, Internet of Things devices, and portable wearable devices.The Internet of Things devices include smart voice interaction devices, smart home appliances such as smart TVs and smart air conditioners, smart in-vehicle devices, airplanes, etc. The portable wearable devices include smart watches, smart bands, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster or cloud server consisting of multiple servers.

[0014] In one embodiment, as shown in Fig. 2, a video identification method is provided, which may be performed by an electronic device such as a terminal or a server alone, or may be performed by a terminal and a server together, and in the embodiment of the present invention, the method is described by taking an example of applying the method to the server in Fig. 1. The method includes the following steps:

[0015] In step 202, a target video and a video set reference video in a video series movie set are obtained, where the video series movie set includes videos belonging to the same series.

[0016] Here, a video series moving image set is a set of multiple videos belonging to the same series, and belonging to the same series can be divided according to different series dimensions based on actual needs. For example, the same serial drama can be considered to belong to the same series, and in this case, a set of each television video included in the drama can be considered to be a video series moving image set of the drama, and videos produced by the same creator can be considered to belong to the same series, and in this case, a set of each video produced by the creator can be a video series moving image set, and each video may have a different time. Furthermore, the same series may be videos on the same theme, videos produced in the same production location, etc. A video series moving image set includes multiple videos, and the multiple videos may have similar segments. For example, in the case of videos produced by the same creator, there may be an opening at the beginning of each video that introduces the creator, and there may also be an ending that summarizes the video, etc. The opening of a video is generally used to indicate the start of a video, and the ending of a video is used to indicate the end of a video. The opening and ending formats are various, specifically including but not limited to audio-video materials, text, logos, etc.

[0017] The target video is a video in the video series moving image set that is to be identified, that is, to identify a video segment similar to another video from the target video. For example, an opening and an ending are identified, and the opening and the ending are video segments similar to another video. The reference video is used as a reference for target video identification, that is, to identify a similar video segment in the target video based on the reference video. The moving image set reference video is a reference video sampled and extracted from the video series moving image set. The moving image set reference video and the target video belong to the same video series moving image set, and there may be a similar video segment between each video belonging to the same video series moving image set, so that the video can be skipped during playback or the main part of the video can be accurately edited. The number of videos in the moving image set reference video can be set according to actual needs. For example, the number of moving image set reference videos can be set to a fixed number, or based on the time of the target video and the number of videos included in the video series moving image set. For example, the longer the time of the target video, the more the number of moving image set reference videos can be set, and the more the number of videos included in the video series moving image set, the more the number of moving image set reference videos can be set. In addition, the number of movie set reference videos can be set to be a fixed ratio of the number of videos included in the video series movie set, for example, it can be set to 50%, so that if the number of videos included in the video series movie set is 20, the number of movie set reference videos can be 10, that is, 10 videos other than the target video are extracted from the video series movie set as movie set reference videos.

[0018] Specifically, when a video identification event is triggered, it indicates that video identification processing is required. The server obtains a target video and a video set reference video in a video series video set, specifically, the server determines a video series video set that is the object of the video identification event, the server queries the video series video set, determines a target video from the video series video set, and the server extracts a video set reference video from the video series video set, thereby obtaining a target video and a video set reference video that belong to the same video series video set. Furthermore, after determining the target video, the server determines a video series video set divided by the target video, thereby obtaining a target video and a video set reference video from the video series video set.

[0019] In step 204, a video set locally similar segment in the target video to the video set reference video is identified based on a first matching result obtained by video frame matching between the target video and the video set reference video.

[0020] Here, the video frame refers to each image frame in a video consisting of multiple video frames, that is, a video includes multiple video frames, and each video frame is one image. Video frame matching is performed on video frames belonging to different videos to determine matching video frames present in the different videos. For example, whether there are video frames with matching similarity, whether there are video frames with matching image content, etc. For example, a first video frame extracted from a first video is image matched with a second video frame extracted from a second video, thereby determining a video frame from the first video that matches a video frame of the second video, which may be a video frame that includes the same image content, for example, a video frame that includes both an opening content or an ending content. The first matching result is an image matching result obtained by performing video frame matching between a target video and a video set reference video, and the first matching result may specifically include matching video frames obtained by identifying the target video and the video set reference video. Regarding the video frame matching process between the target video and the video set reference video, similarity matching is performed on the video frames in the target video and the video frames in the video set reference video, and a first matching result including matching video frames between the target video and the video set reference video can be obtained based on the video frames corresponding to a similarity that satisfies a similarity threshold.

[0021] The similar segment refers to a video segment with a similar scene between different videos, and the video set locally similar segment refers to a video segment in a target video that is similar to a segment in a video set reference video. When the video set locally similar segment in a target video is similar to a segment in a reference video, the video set locally similar segment may be a video content overlapping the target video and the video set reference video, for example, the video content overlapping the target video and the video set reference video, specifically, the video content overlapping the opening, the ending, the advertisement, the platform introduction information, etc.

[0022] Specifically, the server identifies the target video and the video set reference video, and identifies a video segment in the target video that is similar to the video set reference video. The server performs video frame matching on the target video and the video set reference video, specifically, extracts video frames from the target video and the video set reference video, respectively, and performs image matching on the extracted video frames. For example, image similarity matching is performed to obtain a first matching result. The server identifies a video set local similar segment in the target video to the video set reference video based on the first matching result, specifically, the server determines a video set local similar segment based on time attributes of matching video frames of the target video and the video set reference video, such as timestamp positions of matching frames in the target video frames. Obtaining a video set local similar segment is to identify and obtain for the target video based on the video set reference video of the video series video set to which the target video belongs, and is a similar segment identified and obtained based on the local video, as opposed to being based on each video in the entire video platform.

[0023] For example, if in the obtained first matching result, the video frame at 1 second of the target video matches with the video frame at 3 second of the video set reference video, the video frame at 2 second of the target video matches with the video frame at 4 second of the video set reference video, the video frame at 3 second of the target video matches with the video frame at 5 second of the video set reference video, and the video frame at 4 second of the target video matches with the video frame at 6 second of the video set reference video, the server can determine that the video segments at 1 second to 4 seconds of the target video are video set locally similar segments to the video set reference video, thereby identifying and obtaining video set locally similar segments.

[0024] In step 206, a platform reference video from the video platform to which the target video belongs is obtained.

[0025] Here, the video platform refers to a platform that can provide video resources, and users can perform operations such as playing and watching videos, downloading, and favorites on the video platform. In a specific implementation, video creators distribute (distribute) the videos they have created to the video platform so that video viewers can watch them. The platform reference video is from the video platform to which the target video belongs, that is, it belongs to the same video platform as the target video. Specifically, a video extracted from the video platform to which the target video belongs can be used as a reference video to identify the target video.

[0026] Specifically, the server obtains the platform reference video, and in realization, the server determines the video platform to which the target video belongs, and obtains the platform reference video belonging to the video platform. In specific applications, the platform reference video is the original platform video directly obtained from the video platform, i.e., the platform video without further processing, and the platform reference video may be a video that has been edited on the original platform video, for example, a video segment cut out from the original platform video.

[0027] In step 208, a platform global similar segment in the target video to the platform reference video is identified based on a second matching result obtained by video frame matching between the target video and the platform reference video.

[0028] Here, the second matching result is an image matching result obtained by performing video frame matching between the target video and the platform reference video, and the second matching result may specifically include matching video frames obtained by identifying in the target video and the platform reference video, for example, whether there are video frames with matching similarity, whether there are video frames with matching image content, etc. The video frame matching process between the target video and the platform reference video may adopt the same processing method as the video frame matching between the target video and the video set reference video. The platform global similar segment refers to a video segment in the target video that is similar to a segment in the platform reference video.

[0029] Specifically, the server identifies the target video and the platform reference video, and identifies a video segment in the target video that is similar to the platform reference video. The server performs video frame matching on the target video and the platform reference video, specifically, extracts video frames from the target video and the platform reference video, respectively, and performs image matching on the extracted video frames to obtain a second matching result. The server identifies a platform global similar segment in the target video to the platform reference video based on the second matching result. Obtaining a platform global similar segment refers to identifying and obtaining the target video by the platform reference video in the video platform to which the target video belongs, and is a similar segment obtained by performing global video identification based on each video in the entire video platform.

[0030] In step 210, a global similar segment in the target video to the video set reference video and the platform reference video is determined based on the positions in the target video of the video set local similar segment and the platform global similar segment, respectively.

[0031] Here, the positions of the video set local similar segments and the platform global similar segments in the target video refer to the time stamp positions of the video set local similar segments and the platform global similar segments in the target video. For example, if the video set local similar segments are video segments from 2 to 6 seconds, the positions of the video set local similar segments in the target video may be the time stamp positions from 2 to 6 seconds, and if the platform global similar segments are video segments from 3 to 8 seconds, the positions of the platform global similar segments in the target video may be the time stamp positions from 3 to 8 seconds. The integrated similar segments are video identification results obtained by integration based on the video set local similar segments and the platform global similar segments.

[0032] Specifically, the server determines the positions of the video set local similar segment and the platform global similar segment in the target video, and determines the overall similar segment for the video set reference video and the platform reference video in the target video based on the positions. For example, if the position of the video set local similar segment is from 2 seconds to 6 seconds and the position of the platform global similar segment is from 3 seconds to 8 seconds, the server can merge the two positions and determine the video segment corresponding to the position from 2 seconds to 8 seconds as the overall similar segment in the target video. Furthermore, the user can adjust the overall similar segment to obtain a more accurate overall similar segment.

[0033] In specific application, after determining the comprehensive similarity segment in the target video to the video set reference video and the platform reference video, the comprehensive similarity segment can be a video segment that is used in the target video in a duplicated manner, such as video content such as opening, ending, advertisement or platform information, and when playing the target video, the comprehensive similarity segment can be skipped and played, thereby improving the playback efficiency.Furthermore, in the application scenario of video comparison, when there is a video content such as opening, ending or advertisement that is used in a duplicated manner in each video of the video series video set and comparison is not required, the comprehensive similarity segment can be cut out from the target video, so that video comparison can be performed with other video segments in the target video, reducing the amount of data of the video comparison process and improving the processing efficiency of the video comparison.

[0034] In the above video identification method, a video frame matching is performed between a target video and a video set reference video in a video series video set, and a video set local similar segment in the target video is identified for the video set reference video according to a first matching result obtained, and a video frame matching is performed between a platform reference video of a video platform to which the target video belongs and the target video, and a platform global similar segment in the target video is identified for the platform reference video according to a second matching result obtained, and a comprehensive similar segment in the target video is determined based on the positions of the video set local similar segment and the platform global similar segment in the target video, respectively. The video set local similar segment is obtained by identifying based on the video set reference video belonging to the same video series video set as the target video, and the platform global similar segment is obtained by identifying based on the platform reference video belonging to the same video platform as the target video, and the comprehensive similar segment obtained based on the positions of the video set local similar segment and the platform global similar segment in the target video takes into account the video similarity characteristics in the video series video set and the video similarity characteristics in the video platform, thereby improving the accuracy of identifying similar video segments in a video.

[0035] In one embodiment, the video identification method performs a refinement update on the movie set locally similar segments based on the refinement segments in the target video that include the refinement keywords, to obtain updated movie set locally similar segments.

[0036] Here, the correction keyword is a keyword for performing a correction process on the video identification of the target video to improve the accuracy of the video identification. The correction keyword can be various types of keywords, such as a keyword of platform introduction information, an advertisement keyword, a video introduction keyword, etc. For example, if the display content of a video segment from 2 seconds to 4 seconds of a certain video A is a video introduction keyword of "Episode N" or "Just fiction", the video segment can be considered to belong to a similar segment, not to the main video content in the target video. Furthermore, for example, if the display content of a video segment from 1 second to 2.5 seconds of a certain video B is platform introduction information of "XXX video platform", it can be determined that the video segment does not belong to the main video content in the target video, but to a similar segment that is overlapped and used in each video of the video platform. The correction segment is a video segment of the target video that needs a correction process on the video identification, and can be a video segment in the target video that includes the correction keyword. For example, if the video segment from 1 second to 2.5 seconds in the above video B contains the correction keyword "XXX video platform", the video segment from 1 second to 2.5 seconds in video B can be determined as the correction segment.

[0037] Specifically, the server determines a correction segment in the target video that includes the correction keyword, and when applied, the server can perform text identification on the video frame in the target video to identify a correction segment in the video frame of the target video that includes the correction keyword. The correction keyword can be preset according to actual needs, and can include various types of keywords, such as keywords of platform introduction information, advertising keywords, or video introduction keywords. The server performs correction update on the video set local similar segment based on the correction segment in the target video, specifically, uses the distribution of the correction segment in the target video, such as the position of the correction segment in the target video, performs correction update on the position of the video set local similar segment in the target video, updates the position of the video set local similar segment in the target video, and obtains the updated video set local similar segment. If the correction segment includes the correction keyword, the correction segment should also be considered as belonging to a similar segment that is overlapped and used in each video, and the correction segment should also be the result of video identification. For example, if the video set locally similar segment of a video C is the video segment from 2 seconds to 5 seconds, but the video C contains a correction segment of a correction keyword from 0 seconds to 2 seconds, the server can determine that the updated video set locally similar segment is the video segment from 0 seconds to 5 seconds, thereby performing a correction update on the video set locally similar segment based on the correction segment, and improving the accuracy of video identification.

[0038] Further, the step of determining overall similar segments in the target video to the video set reference video and the platform reference video based on the positions of the video set local similar segments and the platform global similar segments in the target video includes a step of determining overall similar segments in the target video to the video set reference video and the platform reference video based on the positions of the updated video set local similar segments and the platform global similar segments in the target video.

[0039] Specifically, the server determines a comprehensive similarity segment according to the updated video set local similarity segment and the platform global similarity segment. When applied, the server can determine the positions of the updated video set local similarity segment and the platform global similarity segment in the target video, and can determine a comprehensive similarity segment in the target video to the video set reference video and the platform reference video according to the positions.

[0040] In this embodiment, a correction update is performed on the video collection local similar segment based on the correction segment in the target video containing the correction keyword, and a comprehensive similar segment is determined based on the updated video collection local similar segment and the platform global similar segment. The correction keyword can be used to perform a correction update on the video collection local similar segment, video segments with overlapping correction keywords can be identified, and the accuracy of identifying similar video segments in videos can be improved.

[0041] In one embodiment, the step of performing a correction update on a video set locally similar segment based on a correction segment in the target video including the correction keyword and obtaining an updated video set locally similar segment includes the steps of determining a correction segment in the target video including the correction keyword, updating to a timestamp position of the video set locally similar segment in the target video based on a timestamp position of the correction segment in the target video and obtaining an updated timestamp position, and determining an updated video set locally similar segment in the target video based on the updated timestamp position.

[0042] Here, the timestamp position refers to the position of the timestamp in the video to which the video segment belongs, for example, if a video duration is 2 minutes, the timestamp is from 00:00 to 02:00, and if a video segment in the video is from 23 seconds to 59 seconds, the timestamp position of the video segment in the video is from 00:23 to 00:59. Different video segments in a video have different timestamp positions, and corresponding video segments can be determined from the video according to the timestamp positions.

[0043] Specifically, the server determines a correction segment in the target video that includes the correction keyword, for example, the server can perform text recognition on video frames in the target video, thereby determining a correction segment in the target video that includes the correction keyword. The server determines a timestamp position of the correction segment in the target video and a timestamp position of a video set locally similar segment in the target video. The server updates the timestamp position of the video set locally similar segment in the target video to obtain an updated timestamp position, and determines an updated video set locally similar segment in the target video based on the updated timestamp position.

[0044] For example, if the server determines that the correction segment in the target video containing the correction keyword is a video segment from seconds 30 to 31, the server can determine that the timestamp position of the correction segment is 00:30 to 00:31, and if the timestamp position of the video set locally similar segment in the target video is 00:26 to 00:30, the server obtains that the updated timestamp position is 00:26 to 00:31, i.e., the updated video set locally similar segment in the target video is the video segment from seconds 26 to 31.

[0045] In this embodiment, the timestamp position of the video set locally similar segment in the target video is updated according to the timestamp position of the correction segment in the target video, and the updated video set locally similar segment in the target video is determined based on the updated timestamp position, so that accurate correction updating of the video set locally similar segment can be performed based on the timestamp position, the accuracy of the video set locally similar segment can be ensured, and the accuracy of identifying similar video segments in the video can be improved.

[0046] In one embodiment, the step of determining a correction segment in the target video that includes the correction keyword includes the steps of performing text identification on video frames in the target video to obtain a text identification result, matching the text identification result with the correction keyword to obtain a matching result, and determining a correction segment from the target video that includes the correction keyword based on a video frame associated with the matching result that matches.

[0047] Here, the correction keywords can be preset according to actual needs, for example, a keyword library can be established, various types of correction keywords can be stored in the keyword library, the text identification result of the target video is matched with each type of correction keyword in the keyword library, and it is determined whether the target video contains a correction segment including the correction keyword.

[0048] Specifically, the server obtains video frames from the target video, for example, a plurality of video frames can be extracted at equal intervals, and the server performs text identification on each of the obtained video frames, respectively, to obtain a text identification result. The server obtains a pre-set correction keyword, matches the text identification result of the target video with the correction keyword, and obtains a matching result. The server screens the matching results that match, determines each video frame associated with the matching results that match, and determines a correction segment including the correction keyword from each target video. For example, for the first 10 seconds of the target video, one video frame is extracted every 0.5 seconds to obtain 20 video frames, and the server performs text identification on each video frame, respectively, and matches the text identification result of each video frame with the correction keyword. If the video frame associated with the matching result that match is the 18th to 20th video frames, the server can determine that the correction segment in the target video is the video segment between the 18th video frame and the 20th video frame, specifically, the video segment from the 9th to 10th seconds in the target video.

[0049] In this embodiment, text recognition is performed on video frames in the target video, and based on the matching result obtained by matching the text recognition result with the correction keyword, a correction segment containing the correction keyword is determined from the target video, and the correction segment in the target video can be accurately identified through a text search method. Furthermore, based on the correction segment, a correction update is performed on the video collection local similar segment to improve the accuracy of video recognition.

[0050] In one embodiment, the platform reference video includes platform public video segments obtained from the public video library of the video platform to which the target video belongs, and platform related videos obtained from the video platform. As shown in Figure 3, the process of identifying platform global similar segments, that is, identifying platform global similar segments in the target video to the platform reference video based on the second matching result obtained by video frame matching between the target video and the platform reference video, includes the following steps:

[0051] In step 302, video frame matching is performed on the target video and the platform public video segments to obtain a public video matching result.

[0052] Here, the public video library is related to a video platform and is used to store each platform public video segment in the video platform. The platform public video segment is a video segment publicly used for each video in the video platform. For example, in the case of video platform A, for a video uploaded to the video platform A, the video platform A adds a video segment introducing the video platform A to the uploaded video to indicate the origin of the video. For each video in the video platform, a video segment introducing the video platform A is publicly used, and the video segment is a platform public video segment. The platform public video segment may be one or more, and the time length and content of the platform public video segment may also be set by the video platform according to actual needs. Each video in the video platform includes a platform public video segment, and the platform public video segment does not belong to the main content of the video, but belongs to a similar video segment, and can be identified and deleted when editing the main content of the video or performing a video comparison process.

[0053] The platform-related video is a video obtained from the video platform to which the target video belongs, specifically, a video obtained by sampling from the video platform. The acquisition method of the platform-related video can be set according to actual needs, for example, a random sampling method can be adopted to extract from the video platform. In addition, the screening conditions such as the publication time, the theme content, the keyword, etc. can be set to screen each video of the video platform to obtain the platform-related video. The public video matching result is a matching result obtained by performing video frame matching between the target video and the platform public video segment.

[0054] Specifically, the platform reference video acquired by the server includes a platform public video segment acquired from the public video library of the video platform to which the target video belongs, for example, the server can determine the video platform to which the target video belongs, query the public video library of the video platform, and acquire the platform public video segment from the public video library. The server performs video frame matching between the target video and the platform public video segment to obtain a public video matching result.

[0055] In step 304, if no similar segments can be identified based on the public video matching result, video frame matching is performed on the target video and the platform related videos to obtain related video matching results.

[0056] Here, the related video matching result is a matching result obtained by performing video frame matching between the target video and the platform-related video, and may include matching video frames obtained by identifying the target video and the platform-related video.

[0057] Specifically, the server identifies a similar segment in the target video based on the public video matching result, and if the similar segment cannot be identified, it indicates that the target video has no video segment that is publicly available with the platform public video segment, in which case the server performs video frame matching on the target video and the platform related video to obtain a related video matching result.

[0058] In step 306, based on the related video matching results, identify platform-global similar segments in the target video to the platform-related videos.

[0059] Specifically, the server identifies platform-global similar segments in the target video to the platform-related videos based on the related video matching results. For example, the server determines matching video frames in the target video based on the related video matching results, and identifies platform-global similar segments in the target video to the platform-related videos based on the timestamp positions of each video frame.

[0060] In this embodiment, the platform reference video includes platform public video segments obtained from the public video library of the video platform to which the target video belongs, and platform related videos obtained from the video platform, and the server first performs a recognition process on the target video using the platform public video segments, and if no similar segments can be identified, performs a recognition process on the target video using the platform related videos to obtain platform global similar segments in the target video to the platform related videos. By first performing the recognition process using the platform public video segments, the suitability of similar segment recognition can be improved, the data volume of similar segment recognition can be reduced, and the processing efficiency of similar segment recognition can be improved, but if no similar segments can be identified using the platform public video segments, the recognition process can be performed using the platform related videos to ensure the accuracy of similar segment recognition.

[0061] In one embodiment, after a platform global similar segment to a platform related video in the target video is identified based on the related video matching result, the video identification method further includes a step of updating identification statistical parameters of the platform global similar segment to obtain updated identification statistical parameters, and if the updated identification statistical parameters satisfy the platform public judgment condition, updating the platform global similar segment as a platform public video segment in the public video library.

[0062] Here, the identification statistical parameters are parameters obtained by statistically calculating the identification process of the platform global similar segment. The parameter type of the identification statistical parameters can be set according to actual needs. For example, the identification statistical parameters may include the frequency of successful identification of the platform global similar segment, etc. For each platform global similar segment obtained by identification, the identification process of the platform global similar segment can be statistically calculated to obtain the identification statistical parameters. The platform public judgment condition is a judgment condition for judging whether the platform global similar segment is used as a platform public video segment. For example, the identification statistical parameters exceed a predetermined parameter threshold, specifically, the frequency exceeds a frequency threshold, etc.

[0063] Specifically, after the platform global similar segment for the platform related video in the target video is identified, the server can query the identification statistical parameters of the platform global similar segment, and the identification statistical parameters reflect the statistical results of successful identification of the platform global similar segment. The server updates the identification statistical parameters of the platform global similar segment. For example, if the identification statistical parameters of the platform global similar segment include the frequency of successful identification, specifically 5 times, the server can add 1 to the frequency and update the frequency of the identification statistical parameters to 6 times. The server queries a predetermined platform public judgment condition, and the server compares the updated identification statistical parameters with the platform public judgment condition. If the updated identification statistical parameters meet the platform public judgment condition, the server can use the platform global similar segment as a platform public video segment and update the platform global similar segment in the public video library, thereby realizing dynamic update to the public video library. In the subsequent video identification process, the server can first perform the video identification process with the platform global similar segment as the platform public video segment.

[0064] In this embodiment, after the platform global similar segment is successfully identified, the server updates the identification statistical parameters of the platform global similar segment. If the updated identification statistical parameters satisfy the platform public judgment conditions, the server updates the platform global similar segment as a platform public video segment in the public video library, thereby realizing dynamic updating to the public video library, ensuring the effectiveness of the platform public video segments in the public video library, and improving the accuracy and processing efficiency of the video similar segment identification process.

[0065] In one embodiment, obtaining a platform reference video from the video platform to which the target video belongs includes obtaining a platform public video segment from a public video library of the video platform to which the target video belongs.

[0066] Here, the public video library is related to a video platform and is used to store each platform public video segment in the video platform, and the platform public video segment is a video segment public to each video in the video platform. Specifically, the platform reference video acquired by the server includes the platform public video segment acquired from the public video library of the video platform to which the target video belongs. For example, the server determines the video platform to which the target video belongs, queries the public video library of the video platform, and acquires the platform public video segment from the public video library. In a specific application, the server can acquire all the platform public video segments in the public video library, and can also screen from the public video library. For example, screening is performed based on the release time, video theme, etc., to obtain the platform public video segment that meets the screening condition.

[0067] Further, the step of identifying a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video includes a step of identifying a platform global similar segment in the target video to the platform public video segment based on a second matching result obtained by video frame matching between the target video and the platform public video segment.

[0068] Specifically, the server performs video frame matching between the target video and the platform public video segment to obtain a second matching result, the second matching result may include matching video frames identified in the target video and the platform public video segment. The server identifies platform global similar segments in the target video for the platform public video segment based on the second matching result. For example, the server may determine platform global similar segments in the target video based on the positions in the target video of the matching video frames identified.

[0069] In this embodiment, the platform reference video includes a platform public video segment obtained from the public video library of the video platform to which the target video belongs, and the server performs an identification process based on the platform public video segment, thereby improving the suitability of similar segment identification, reducing the data volume of the similar segment identification process, and improving the processing efficiency of similar segment identification.

[0070] In one embodiment, the step of obtaining a platform reference video from the video platform to which the target video belongs includes the steps of determining the video platform to which the target video belongs and the correction keywords contained in the video frames of the target video, querying platform-related videos in the video platform that have an association relationship with the correction keywords, and screening to obtain the platform reference video from the platform-related videos according to the reference video screening conditions.

[0071] Here, the platform-related video is a video having a correction keyword related relationship obtained from the video platform to which the target video belongs. The related relationship between each video in the video platform and the correction keyword can be established in advance. For example, when a video is uploaded to the video platform, text identification is performed on the video frame of the video, and the correction keyword contained in the video is determined based on the text identification result, and the related relationship between the video and the correction keyword is established. The reference video screening condition is a screening condition set in advance for screening from the platform-related video to obtain the platform reference video. For example, it can be various screening conditions such as publication time, video theme, etc.

[0072] Specifically, the server determines the video platform to which the target video belongs, specifically, the server queries the video attribute information of the target video, and determines the video platform to which the target video belongs based on the video attribute information of the video attribute information. The server determines the correction keyword included in the video frame of the target video, specifically, the server performs text identification on the video frame of the target video, and can determine the correction keyword included in the video frame of the target video based on the text identification result. The server queries platform-related videos having a related relationship with the correction keyword from the video platform. For example, the server can query and obtain platform-related videos having a related relationship with the correction keyword based on the related relationship between each video and the keyword in the video platform. The server queries a predetermined reference video screening condition, such as a screening condition of the release time, and the server screens the platform-related videos based on the reference video screening condition to obtain a platform reference video that meets the reference video screening condition from the platform-related video. For example, if the release time of the target video is June 1, 2022, the reference video screening condition is that the release time is within one month of the target video release time, and the server screens platform reference videos from platform-related videos whose release time is between May 1, 2022 and June 1, 2022.

[0073] In this embodiment, the platform reference videos include platform-related videos having association relationships with the correction keywords obtained from the video platform, and are screened according to the reference video screening conditions, thereby performing global video identification processing using various videos in the video platform and controlling the number of platform reference videos, thereby reducing the amount of data used for similar segment identification processing overall using the platform reference videos, and improving the processing efficiency of similar segment identification while ensuring the accuracy of similar segment identification.

[0074] In one embodiment, the video identification method further includes the steps of performing video frame text identification in a platform video belonging to a video platform to obtain video keywords, performing matching in a keyword library based on the video keywords to determine target keywords that match with the video keywords, and establishing an association relationship between the platform video and the target keywords.

[0075] Here, the platform video refers to each video belonging to the video platform, and the video keyword is a keyword obtained by performing text identification from the platform video. Various keywords are stored in the keyword library, and the target keyword is a keyword that matches with the video keyword in the keyword library. Specifically, the server performs text identification on the platform video belonging to the video platform, for example, performs text identification on the video frame in the platform video, and obtains the video keyword contained in the video frame of the platform video. The server queries the keyword library, in which various correction keywords are stored, and the keyword library can be preset according to actual needs and dynamically updated and maintained. The server matches the video keyword in the keyword library to determine the target keyword that matches with the video keyword, and establishes an association relationship between the platform video and the target keyword, so that the corresponding platform video can be queried based on the keyword and the association relationship.

[0076] Furthermore, the step of querying, in the video platform, for platform-relevant videos having an association relationship with the remediation keyword includes the step of querying, in the video platform, for platform-relevant videos associated with the remediation keyword based on the association relationship.

[0077] Specifically, for each platform video in the video platform, the server determines its related relationship, and queries and obtains platform related videos related to the correction keyword based on the related relationship and the correction keyword.

[0078] In this embodiment, for each platform video in the video platform, an association relationship between the platform video and a keyword is established, and platform-related videos related to the correction keyword in the video platform are determined based on the association relationship, thereby improving the accuracy and processing efficiency of querying platform-related videos, and improving the accuracy and processing efficiency of identifying similar segments.

[0079] In one embodiment, the step of determining an overall similar segment in the target video to the video set reference video and the platform reference video based on the positions of the video set local similar segment and the platform global similar segment in the target video includes the steps of determining a first timestamp position of the video set local similar segment in the target video and a second timestamp position of the platform global similar segment in the target video, merging the first timestamp position and the second timestamp position to obtain an overall timestamp position, and determining an overall similar segment in the target video to the video set reference video and the platform reference video based on the overall timestamp position.

[0080] Here, the first timestamp position refers to the timestamp position of the video aggregation local similar segment in the target video, and the second timestamp position refers to the timestamp position of the platform global similar segment in the target video. The overall timestamp position is a timestamp position obtained by merging the first timestamp position and the second timestamp position. Based on the overall timestamp position, an overall similar segment can be determined from the target video.

[0081] Specifically, the server determines a first timestamp position of the video aggregation local similar segment in the target video and a second timestamp position of the platform global similar segment in the target video, specifically, the server determines the respective timestamp positions of the target video for the respective segment times of the video aggregation local similar segment and the platform global similar segment. The server merges the first timestamp position and the second timestamp position to obtain a comprehensive timestamp position. In a specific implementation, the server can directly merge the first timestamp position and the second timestamp position to obtain a comprehensive timestamp position, for example, when the first timestamp position is from 00:05 to 0:15 and the second timestamp position is from 00:02 to 00:06, the server can directly merge the first timestamp position and the second timestamp position to obtain a comprehensive timestamp position from 00:02 to 00:15. In addition, the server can further partially merge according to actual needs to obtain a comprehensive timestamp position. For example, if the first timestamp position is 00:05 to 00:15 and the second timestamp position is 00:04 to 00:14, the server obtains a range of 00:05 to 00:14 as the overall timestamp position based on the intersection position of the first timestamp position and the second timestamp position. The server determines an overall similar segment from the target video to the video set reference video and the platform reference video based on the obtained overall timestamp position. For example, if the overall timestamp position is 00:02 to 00:15, the server can determine a video segment from 2 seconds to 15 seconds from the target video as an overall similar segment to the video set reference video and the platform reference video.

[0082] In this embodiment, the first timestamp location of the video set local similar segment in the target video is merged with the second timestamp location of the platform global similar segment in the target video, and an overall similar segment to the video set reference video and the platform reference video in the target video is determined based on the overall timestamp location, thereby realizing the overall processing of the video set local similar segment and the platform global similar segment based on the timestamp location, so that the overall similar segment integrates the video similar characteristics in the video series video set and the video similar characteristics in the video platform, and improves the accuracy of identifying similar video segments in the video.

[0083] In one embodiment, the step of identifying a video set locally similar segment in the target video to the video set reference video based on a first matching result obtained by video frame matching between the target video and the video set reference video includes: performing image matching of video frames between the target video and the video set reference video to obtain a video frame pair, the video frame pair including a video frame to be identified belonging to the target video, and further including a video set reference video frame image-matched with the video frame to be identified in the video set reference video; determining a time offset of the video frame pair based on temporal attributes of the video frame to be identified and temporal attributes of the video set reference video frame in the video frame pair; and screening video frame pairs with matching time offsets, and determining a video set locally similar segment in the target video to the video set reference video based on temporal attributes of the video frame to be identified in the video frame pair obtained by screening.

[0084] Here, the video frame pair is an image pair composed of successfully matched video frames determined by image matching of video frames between the target video and the reference video. When the reference video is a video set reference video, the video frame pair includes a video frame to be identified belonging to the target video and a video set reference video frame image-matched with the video frame to be identified in the video set reference video, that is, the video frame to be identified and the video set reference video frame in the video frame pair are obtained by successful image matching, and the video frame to be identified in the video frame pair is from the target video, and the video set reference video frame is from the video set reference video.

[0085] The time attribute is used to indicate the time information of the corresponding video frame, and can represent the position of the video frame in the video. The time attribute may be the time stamp of the corresponding video frame in the video, or the frame number of the video frame, etc. For example, if the time attribute of a video frame is 2.0 seconds, it can indicate that the video frame is the 2.0 second video frame of the belonging video, and further, for example, if the time attribute of a video frame is 500, it can indicate that the video frame is the 500th frame of the belonging video. The time attribute can tag the position of the video frame in the belonging video, so that the appearance time of the video frame in the belonging video is determined. A video is obtained by combining multiple video frames according to time information, and each video frame of the video is set with a time attribute including time information. The time offset is used to represent the time interval between the appearance time of the identification target video frame in the target video and the appearance time of the reference video frame in the reference video among the video frame pairs. The time offset is obtained by the respective time attributes of the identification target video frame and the reference video frame. For example, in a video frame pair, the time attribute of the video frame to be identified may be 2 seconds, i.e., the video frame to be identified is the video frame at 2 seconds in the target video frame, while the time attribute of the video set reference video frame may be 3 seconds, i.e., the video frame at 2 seconds in the video set reference video, i.e., the video frame at 2 seconds in the target video and the video frame at 3 seconds in the video set reference video frame match, thereby obtaining that the time offset of the video frame pair is 1 second based on the difference between the time attribute of the video frame to be identified and the time attribute of the video set reference video frame.

[0086] Specifically, the server performs image matching of video frames between the target video and the video set reference video, specifically, image matching can be performed between the video frames in the target video and the video set reference video frames. For example, matching can be performed based on image similarity, and a video frame pair is determined based on the matching result. A video frame pair is an image pair consisting of video frames that have been successfully image matched, and in a video frame pair determined by performing image matching based on similarity, the image similarity between the identification target video frame and the video set reference video frame in the video frame pair is high, that is, the identification target video frame in the target video and the video set reference video frame in the video set reference video are similar, so they may be the same video content, for example, they may be a video frame belonging to an opening, or may be a video frame belonging to an ending. For the obtained video frame pair, the server determines the time attribute of the identification target video frame in the video frame pair and the time attribute of the video set reference video frame, specifically, the corresponding time attribute can be determined by querying the frame information of the identification target video frame and the video set reference video frame. The server determines the time offset of the video frame pair based on the obtained time attribute of the identification target video frame and the time attribute of the video set reference video frame. For example, when the time attribute is a quantified value, the server can obtain the time offset of the video frame pair based on the numerical difference between the time attribute of the identified video frame and the time attribute of the video set reference video frame. The server screens each video frame pair based on the time offset, and screens video frame pairs with matching time offsets. Specifically, the server can screen video frame pairs with the same numerical value of the time offset or the difference between the numerical values ​​is within a certain range.The server determines, based on the screened video frame pairs, a time attribute of the to-be-identified video frame in the screened video frame pairs, and obtains a video set local similar segment in the target video to the video set reference video based on the time attribute of the to-be-identified video frame. For example, after determining the time attribute of the to-be-identified video frame in the screened video frame pairs, the server can determine a start time and an end time based on the numerical magnitude of the time attribute of each to-be-identified video frame, so as to determine a video set local similar segment in the target video based on the start time and the end time.

[0087] In a specific application, the server can group the videos according to the magnitude of the numerical value of the time offset to obtain video frame pair sets corresponding to different time offsets, and the video frame pair set includes video frame pairs whose corresponding time offsets match each other. For example, when the time offsets of the obtained video frame pairs include three, 1s, 4s, and 5s, the server can use the video frame pair whose time offset is 1s as a first video frame pair set, and determine a video set local similar segment in the target video according to the time attributes of the identified video frames in the video frame pairs in the first video frame pair set, and the server can further use the video frame pairs whose time offset is 4s and 5s as a second video frame pair set, and determine a video set local similar segment in the target video according to the time attributes of the identified video frames in the video frame pairs in the second video frame pair set. The server can determine each video set local similar segment according to the time attributes of the identified video frames of the video frame pairs in each video frame pair set, and determine and merge the video set local similar segment according to each video frame pair set. For example, the server can delete overlapping video set locally similar segments or update partially overlapping video set locally similar segments, thereby obtaining video set locally similar segments in the target video for each video set reference video.

[0088] In this embodiment, image matching of video frames is performed for the target video and the video set reference video in the video series video set, a video frame pair is obtained including a video frame to be identified belonging to the target video and a video set reference video frame image-matched with the video frame to be identified, and a time offset of the video frame pair is determined according to the time attribute of the video frame to be identified and the time attribute of the video set reference video frame in the video frame pair, and the video frame pair with matching time offsets is screened, and a video set local similar segment is determined from the target video to the video set reference video according to the time attribute of the video frame to be identified in the video frame pair obtained by screening for the target video and the video set reference video in the video series video set. According to the time attribute of the video frame to be identified in the video frame pair with matching time offsets obtained by screening for the target video and the video set reference video in the video series video set, a time offset of the video frame pair is determined, and a video set local similar segment is determined in the target video to the video set reference video according to the time attribute of the video frame to be identified in the video frame pair with matching time offsets obtained by screening, and a similar video segment with different times can be flexibly determined according to the video frame pair image-matched, and the accuracy of identifying similar video segments in various videos can be improved.

[0089] In one embodiment, the step of screening video frame pairs having matching time offsets and determining a video set local similar segment in the target video to the video set reference video based on temporal attributes of the identified video frame in the screened video frame pairs includes the steps of performing numerical matching for the time offsets of each video frame pair and screening video frame pairs having numerically matching time offsets based on the numerical matching results, determining a start time and an end time based on temporal attributes of the identified video frame in the screened video frame pairs, and determining a video set local similar segment from the target video to the video set reference video based on the start time and the end time.

[0090] Here, the time offset represents the time interval between the appearance time of the video frame to be identified in the target video and the appearance time of the video set reference video frame in the video set reference video in the video frame pair. The specific form of the time offset is a quantified value, for example, a value in seconds, which indicates how many seconds the time difference is between the appearance time of the video frame to be identified and the video set reference video frame in the video frame pair in the video to which they belong. The numerical matching refers to matching the magnitude of the numerical value of the time offset of each video frame pair to obtain a numerical matching result. The numerical matching result may include a numerical difference between the time offset of each video frame pair, i.e., the numerical difference of the time offset. The start time refers to the video start time of the video segment, and the end time refers to the video end time of the video segment. Based on the start time and the end time, the start time is set as the video start time, and the end time is set as the video end time, so that the span time of the video is set from the start time to the end time, so that the corresponding video segment can be determined.

[0091] Specifically, the server performs numerical matching on the time offset of each video frame pair, specifically, performs numerical matching on the time offset of every two video frame pairs to obtain a numerical matching result. The server determines a video frame pair whose time offsets match numerically based on the obtained numerical matching result. For example, the numerical matching result includes a numerical difference between the time offsets of each video frame pair, and the server can determine a time offset whose difference between the time offsets of each video frame pair is smaller than a predetermined threshold as a numerically matching time offset, so that after obtaining a screened video frame pair whose time offsets match numerically based on the screened video frame pairs obtained by the numerically matching time offsets, the server determines a time attribute of the video frame to be identified in the screened video frame pair, specifically, can query frame information of each video frame to be identified, so as to obtain a time attribute of the video frame to be identified. The server determines a start time and an end time based on the time attribute of the video frame to be identified.

[0092] In a specific application, after obtaining the time attributes of the video frame to be identified in the video frame pair obtained by screening, the server can determine the time attribute with the smallest numerical value among them, and determine the start time based on the smallest time attribute, and the server can determine the time attribute with the largest numerical value among them, and determine the end time based on the largest time attribute. For example, in the application, if the sequence of the time attributes of the video frame to be identified in the video frame pair obtained by screening is {1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 15}, the server can determine 1s as the start time and 15s as the end time. The server can determine a video set local similar segment in the target video to the video set reference video based on the start time and the end time, for example, the server can determine the video segment from the start time to the end time in the target video as the video set local similar segment. For example, if the server determines 1s as the start time and 15s as the end time, the server can determine the video segment from the 1st second to the 15th second of the target video as the video set local similar segment to the video set reference video.

[0093] In this embodiment, numerical matching is performed on the time offsets of video frame pairs, and video frame pairs whose time offsets match numerically are screened based on the numerical matching result. A start time and an end time are determined based on the time attributes of the video frame to be identified in the video frame pair obtained by screening, and a video collection local similar segment is determined in the target video based on the start time and the end time. Thus, a video collection local similar segment is determined from the target video based on the video frame to be identified in the video frame pair obtained by screening, so that similar video segments can be flexibly determined based on the video frame to be identified at the frame level, and can be applied to videos containing similar video segments of different times, thereby improving the accuracy of identifying similar video segments in videos.

[0094] In one embodiment, the step of performing numerical matching on the time offsets of each video frame pair and screening video frame pairs whose time offsets match numerically based on the numerical matching result includes the steps of numerically comparing the time offsets of each video frame pair to obtain a numerical comparison result, screening video frame pairs whose numerical difference in time offset from each video frame pair is smaller than a numerical difference threshold based on the numerical comparison result, and performing offset updating for video frame pairs whose numerical difference in time offset is smaller than the numerical difference threshold to obtain video frame pairs whose time offsets match numerically.

[0095] Here, the numerical comparison refers to performing a comparison of the magnitude of a numerical value for the time offset of each video frame pair to obtain a numerical comparison result, and the numerical comparison result may include a numerical difference between the time offsets of each video frame pair. For example, if the time offset of video frame pair 1 is 1s and the time offset of video frame pair 2 is 2s, the numerical difference between the time offsets of video frame pair 1 and video frame pair 2 is 1s, that is, the numerical comparison result of the numerical comparison of the time offsets of video frame pair 1 and video frame pair 2 is 1s. The numerical difference threshold is flexibly set according to actual needs, and the numerical difference threshold is used to match the time offsets of each video frame pair. Specifically, the video frame pairs whose numerical difference in time offset is smaller than the numerical difference threshold can be used as the video frame pairs obtained by screening. The offset update is the update of the time offsets of the video frame pairs whose numerical difference in time offset is smaller than the numerical difference threshold in order to match the time offsets of the video frame pairs. For example, the time offsets of the video frame pairs can be unified and updated to the same time offset.

[0096] Specifically, the server performs a numerical comparison on the time offset of each video frame pair respectively to obtain a numerical comparison result, the numerical comparison result includes a numerical difference between the time offsets of each video frame pair, which can be obtained by the server by taking two differences for the time offsets of each video frame pair. The server determines a preset numerical difference threshold, and based on the numerical comparison result, screens out video frame pairs whose numerical difference in time offset is smaller than the numerical difference threshold from each video frame pair. Specifically, the server compares the numerical difference of the numerical comparison result with the numerical difference threshold to determine video frame pairs associated with time offsets whose numerical difference is smaller than the numerical difference threshold, and screens out the video frame pairs from each video frame pair. The server performs offset update on video frame pairs whose numerical difference in time offset is smaller than the numerical difference threshold, specifically, the server can unify and update the time offsets of the video frame pairs to the same numerical value. For example, the time offsets are updated to the minimum value in the video frame pairs whose numerical difference in time offset is smaller than the numerical difference threshold, thereby obtaining video frame pairs whose time offsets match numerically. For example, if the numerical difference threshold is 2s, and the video frame pairs whose numerical difference in time offset obtained by screening is less than the numerical difference threshold include two types of time offsets, 1s and 2s, the server updates the time offset of the video frame pair whose time offset is 2s to 1s, thereby obtaining each video frame pair whose time offset is 1s, i.e., obtaining video frame pairs whose time offsets match numerically.

[0097] In this embodiment, according to the time offset of each video frame pair, a numerical comparison is performed to obtain a numerical comparison result, a video frame pair whose numerical difference of time offset is less than a numerical difference threshold is screened from the video frame pair, and an offset update is performed on the video frame pair obtained by screening to obtain a video frame pair whose time offset is numerically matched, thereby obtaining a video frame pair for determining a video set local similar segment by screening, and a video set local similar segment for the video set reference video can be accurately identified from the target video by the video frame pair obtained by screening.

[0098] In one embodiment, the step of determining the start time and the end time based on the temporal attributes of the video frames to be identified in the video frame pairs obtained by screening includes the steps of obtaining a video frame pair list consisting of the video frame pairs obtained by screening; sorting each video frame pair in the video frame pair list in ascending order according to the numerical value of the time offset, and for video frame pairs having the same time offset, sorting in ascending order according to the numerical value of the time stamp of the video frame to be identified included therein, where the time stamp is determined by the temporal attribute of the video frame to be identified included therein; determining a temporal attribute distance between the temporal attributes of the video frames to be identified in adjacent video frame pairs in the video frame pair list; determining adjacent video frame pairs whose temporal attribute distance does not exceed a distance threshold as video frame pairs belonging to the same video segment; and determining the start time and the end time based on the time stamp of the video frames to be identified in the video frame pairs belonging to the same video segment.

[0099] Here, the video frame pair list is constructed by sorting according to the video frame pairs obtained by screening, and in the video frame pair list, each video frame pair obtained by screening is sorted in ascending order of the numerical value of the time offset, and the video frame pairs with the same time offset are sorted in ascending order of the numerical value of the time stamp of the included video frame to be identified, and the time stamp is determined by the time attribute of the included video frame to be identified, and the time stamp is the appearance time of the video frame to be identified in the target video. In the video frame pair list, the time offset numerical value is sorted in ascending order, and if the time offsets are the same, the time stamp numerical value of the video frame to be identified included in the video is sorted in ascending order. That is, in the video frame pair list, the smaller the time offset, the earlier the ranking, and in the case of video frame pairs with the same time offset, the smaller the time stamp of the included video frame to be identified, the earlier the ranking. The time attribute distance is determined based on the time attribute of the included video frame to be identified for adjacent video frame pairs in the video frame pair list, and represents the time interval between adjacent video frame pairs. The distance threshold is set in advance according to actual needs and is used to determine whether they belong to the same video segment; specifically, adjacent video frame pairs whose time attribute distance does not exceed the distance threshold can be determined as video frame pairs belonging to the same video segment, so that video segment aggregation processing can be adaptively performed for each video frame pair, and so that the start time and end time can be determined.

[0100] Specifically, the server obtains a video frame pair list obtained by sorting according to the video frame pairs obtained by screening. In a specific application, after the server obtains the video frame pairs by screening, the server sorts the video frame pairs obtained by screening in ascending order of the numerical value of the time offset, and for the video frame pairs having the same time offset, the server determines the time stamp according to the time attribute of the video frame to be identified contained in the video frame pair, and sorts in ascending order of the numerical value of the time stamp of the video frame to be identified, thereby obtaining a video frame pair list. In the video frame pair list, the server compares the time attributes of the video frames to be identified in adjacent video frame pairs, specifically, obtains the difference between the respective time attributes to obtain the time attribute distance. The server determines a predetermined distance threshold, compares the time attribute distance with the distance threshold, and determines from the video frame pair list adjacent video frame pairs whose time attribute distance does not exceed the distance threshold based on the comparison result, and determines adjacent video frame pairs whose time attribute distance does not exceed the distance threshold as video frame pairs belonging to the same video segment, that is, if the time attribute distance of the identified video frames in the adjacent video frame pairs is small, the adjacent video frame pairs are considered to belong to the same video segment, and are thus aggregated into a video segment based on the identified video frames in the video frame pairs. The server determines timestamps of the identified video frames in the video frame pairs belonging to the same video segment, and determines start time and end time based on the timestamps of each identified video frame. For example, the server determines the start time based on the timestamp with the smallest numerical value, and determines the timestamp with the largest numerical value as the end time, and the determined start time and end time are the start time and end time of the video segment to which the video frame pairs belonging to the same video segment both belong.

[0101] In this embodiment, based on a video frame pair list composed of video frame pairs obtained by screening, video frame pairs belonging to the same video segment are determined based on the temporal attribute distance between the temporal attributes of the video frames to be identified in adjacent video frame pairs, and the start time and end time are determined based on the timestamps of the video frames to be identified in the video frame pairs belonging to the same video segment, thereby realizing inference and mining of the video frames to be identified into video segments, and accurately identifying segments from the target video.

[0102] In one embodiment, the step of determining the start time and the end time based on the timestamps of the video frames to be identified in the video frame pair belonging to the same video segment includes the steps of determining a start video frame pair and an end video frame pair from the video frame pairs belonging to the same video segment based on the timestamps of the video frames to be identified in the video frame pair belonging to the same video segment, obtaining the start time based on the timestamp of the video frame to be identified in the start video frame pair, and obtaining the end time based on the timestamp of the video frame to be identified in the end video frame pair.

[0103] Here, the timestamp of the video frame to be identified is determined by the time attribute of the video frame to be identified, and the timestamp of the video frame to be identified represents the time when the video frame to be identified appears in the target video. The start video frame pair and the end video frame pair are determined by the magnitude of the timestamp of the video frame to be identified included in each video frame pair belonging to the same video segment. The timestamp of the video frame to be identified included in the start video frame pair may be the timestamp with the smallest numerical value among the timestamps of the video frames to be identified included in each video frame pair belonging to the same video segment, while the timestamp of the video frame to be identified included in the end video frame pair may be the timestamp with the largest numerical value, thereby determining the video frame to be identified included in the start video frame pair as the start video frame pair belonging to the same video segment, and determining the video frame to be identified included in the end video frame pair as the end video frame pair belonging to the same video segment.

[0104] Specifically, the server determines the timestamps of the video frames to be identified in the video frame pairs belonging to the same video segment, and based on the magnitude of each timestamp value, the server determines a start video frame pair and an end video frame pair from the video frame pairs belonging to the same video segment. Specifically, the server determines the video frame pair to which the video frame to be identified with the smallest timestamp belongs as the start video frame pair, and determines the video frame pair to which the video frame to be identified with the largest timestamp belongs as the end video frame pair. The server obtains a start time based on the timestamp of the video frame to be identified in the start video frame pair. For example, the server can determine the time corresponding to the timestamp as the start time. The server obtains an end time based on the timestamp of the video frame to be identified in the end video frame pair, and can determine the time corresponding to the timestamp as the end time.

[0105] In this embodiment, the server determines a start video frame pair and an end video frame pair based on the timestamps of the video frames to be identified in a video frame pair belonging to the same video segment, and determines the start time and end time based on the video frames to be identified contained in the start video frame pair and the end video frame pair, respectively, thereby realizing inference and mining into video segments based on the video frames to be identified belonging to the same video segment, and improving the accuracy of identifying similar video segments from the target video.

[0106] In one embodiment, the video identification method further includes a step of determining a segment overlap relationship between each video set locally similar segment based on a respective start time and end time of each video set locally similar segment, and performing segment updating for each video set locally similar segment based on the segment overlap relationship to obtain an updated video set locally similar segment for the video set reference video in the target video.

[0107] Here, when there are multiple segments in the video set locally similar segment identified in the target video for the video set reference video, each video set locally similar segment is updated based on the segment overlapping relationship between each video set locally similar segment, and an updated video set locally similar segment is obtained. The segment overlapping relationship refers to the overlapping relationship between the video set locally similar segments. For example, if the time range of the video set locally similar segment A is (2,5), that is, from the second second to the fifth second of the target video, and the time range of the video set locally similar segment B is (3,4), the video set locally similar segment A completely covers the video set locally similar segment B, in this case, the video set locally similar segment B can be deleted, and the video set locally similar segment A can be kept. When the time range of the video set locally similar segment C is (2,6) and the time range of the video set locally similar segment D is (5,8), the video set locally similar segment C and the video set locally similar segment D partially overlap, in this case, the video set locally similar segment C and the video set locally similar segment D can be extended and updated to obtain the updated video set locally similar segment CD(2,8); when the time range of the video set locally similar segment E is (4,8) and the time range of the video set locally similar segment F is (1,5), the video set locally similar segment E and the video set locally similar segment F partially overlap, in this case, the video set locally similar segment E and the video set locally similar segment F can be extended and updated to obtain the updated video set locally similar segment EF(1,8). Furthermore, when there is no overlap between multiple video set locally similar segments, for example, (2,5) and (7,10), in this case, it can be determined that all the video set locally similar segments that do not have overlap are the video identification result without performing a merge process on each video set locally similar segment. According to different segment overlapping relationships, different update methods can be set, which can ensure the accuracy of updating to the video set locally similar segments.

[0108] Specifically, when a video set locally similar segment is obtained, the server can determine a segment overlap relationship between each video set locally similar segment according to the start time and end time of each video set locally similar segment, for example, a segment overlap relationship of inclusion, partial overlap or no overlap. The server performs segment update for each video set locally similar segment according to the segment overlap relationship between each video set locally similar segment, specifically, performs processes such as merging, deleting, reserving, etc., for each video set locally similar segment to obtain an updated video set locally similar segment for the video set reference video in the target video.

[0109] In this embodiment, when a video set locally similar segment is identified and obtained from multiple segments, segment updating is performed based on the segment overlap relationship between each video set locally similar segment, thereby obtaining a more accurate video set locally similar segment and improving the accuracy of identifying a video set locally similar segment from the target video.

[0110] In one embodiment, the video set reference videos are at least two, and the step of screening video frame pairs having matching time offsets and determining a video set local similar segment in the target video to the video set reference video based on temporal attributes of the video frame to be identified in the video frame pairs obtained by screening includes the steps of screening video frame pairs having matching time offsets and determining an intermediate similar segment in the target video to the video set reference video based on temporal attributes of the video frame to be identified in the video frame pairs obtained by screening, and performing segment update for each intermediate similar segment in the target video to which an overlapping relationship exists among the intermediate similar segments to each video set reference video, to obtain a video set local similar segment in the target video to each video set reference video.

[0111] Here, there are at least two movie set reference videos, that is, at least two movie set reference videos are used to perform video frame matching processing with the target video. The intermediate similar segment refers to a similar segment obtained by identifying a single movie set reference video in the target video. The overlapping relationship refers to the overlapping relationship that exists between the intermediate similar segments obtained by identifying based on different movie set reference videos, and is specifically determined based on the time endpoints (including start time and end time) of each of the identified intermediate similar segments.

[0112] Specifically, the server obtains two or more movie set reference videos, and performs video identification processing on the target video and each of the two or more movie set reference videos to obtain intermediate similar segments in the target video for each movie set reference video. The server performs segment update on each intermediate similar segment that has an overlapping relationship among the intermediate similar segments in the target video for each movie set reference video, thereby obtaining movie set local similar segments in the target video for each movie set reference video.

[0113] In this embodiment, video identification is performed on the target video using multiple video set reference videos, and segment updating is performed on each intermediate similar segment based on the overlap relationship existing in each intermediate similar segment obtained by identification, and video set local similar segments for each of the video set reference videos in the target video are obtained, thereby further improving the accuracy of the video set local similar segments obtained by identification with reference to multiple video set reference videos, and improving the accuracy of similar segments from the target video.

[0114] In one embodiment, for intermediate similar segments in the target video for each video set reference video, the step of performing segment updates for each intermediate similar segment having an overlapping relationship, and obtaining video set local similar segments for each video set reference video in the target video includes the steps of performing segment position comparison for intermediate similar segments for each video set reference video in the target video to obtain segment comparison results, determining each intermediate similar segment having an overlapping relationship as a segment comparison result, and performing segment updates for each intermediate similar segment having an overlapping relationship based on the overlapping time length and statistics of each intermediate similar segment having an overlapping relationship, and obtaining video set local similar segments in the target video for each video set reference video.

[0115] Here, the segment position comparison refers to comparing the positions of the intermediate similar segments in the respective target videos, which are identified based on each video set reference video, to obtain a segment comparison result. The segment comparison result may include whether an overlapping relationship exists between each intermediate similar segment, and if an overlapping relationship exists, a segment update may be performed for each intermediate similar segment having an overlapping relationship to obtain a video set local similar segment for each video set reference video in the target video. The overlapping time length refers to the time of the overlapping segment where overlap exists between each intermediate similar segment having an overlapping relationship. For example, if the time range of the intermediate similar segment A determined by the first video set reference video is (2, 8), and the time range of the intermediate similar segment B determined by the second video set reference video is (5, 10), an overlapping relationship exists between the intermediate similar segment A and the intermediate similar segment B, the overlapping segment is (5, 8), and the overlapping time length is 4 seconds between the 5th second and the 8th second. The statistics may include the number of times the same intermediate similar segment is identified in the intermediate similar segment for each video set reference video identification in the target video. The larger the value of the statistic, the more times the corresponding intermediate similar segment has been identified, and thus the higher the probability that the intermediate similar segment belongs to the movie set locally similar segment.

[0116] Specifically, the server determines intermediate similar segments for each movie set reference video in the target video, and performs segment position comparison for each intermediate similar segment. The server can determine the start time and end time of each intermediate similar segment, and perform segment position comparison based on the start time and end time of each intermediate similar segment to obtain a segment comparison result. If the segment comparison result indicates that there is no overlapping relationship, there is no need to process the intermediate similar segments that do not have an overlapping relationship, and they can be reserved as movie set local similar segments for each movie set reference video in the target video. If the segment comparison result indicates that there is an overlapping relationship, that is, if there is segment overlap between each intermediate similar segment, the server determines each intermediate similar segment that has an overlapping relationship, and performs segment update for each intermediate similar segment that has an overlapping relationship. For example, various update processes such as deleting, merging, and reserving each intermediate similar segment are performed to obtain movie set local similar segments for each movie set reference video in the target video. The server determines each intermediate similar segment having an overlapping relationship as a result of the segment comparison, and determines the statistics of each intermediate similar segment having an overlapping relationship and the overlapping time length between each intermediate similar segment. The server performs segment update for each intermediate similar segment having an overlapping relationship based on the overlapping time length and statistics of each intermediate similar segment having an overlapping relationship, and obtains a video set local similar segment for each video set reference video in the target video. Specifically, the server can determine whether merging is necessary according to the length of the overlapping time length, and determine whether a hold or merge process is necessary according to the value of the statistics.

[0117] In this embodiment, segment position comparison is performed for intermediate similar segments in the target video with respect to each video set reference video, and segment update is performed for each intermediate similar segment that has an overlap relationship as a result of the segment comparison. Specifically, segment update is performed for each intermediate similar segment that has an overlap relationship based on the overlap time length and statistics of each intermediate similar segment that has an overlap relationship, thereby performing segment update for the overlap time length and statistics of each intermediate similar segment that has an overlap relationship, and by taking into account the characteristics between each intermediate similar segment, the effect of segment update can be improved, and the accuracy of identifying video set local similar segments from the target video can be improved.

[0118] In one embodiment, the step of performing segment position comparison for intermediate similar segments in the target video for each video set reference video to obtain a segment comparison result includes the steps of obtaining a similar segment list consisting of intermediate similar segments in the target video for each video set reference video, sorting each intermediate similar segment in the similar segment list in descending order of statistics and sorting the intermediate similar segments with the same statistics in ascending order of start time, and performing segment position comparison for each intermediate similar segment in the similar segment list to obtain a segment comparison result.

[0119] Here, the similar segment list is constructed by sorting the intermediate similar segments in the target video for each movie set reference video. In the similar segment list, each intermediate similar segment is sorted in descending order of statistics, and intermediate similar segments with the same statistics are sorted in ascending order of start time. That is, in the similar segment list, each intermediate similar segment is sorted in descending order of statistics, and corresponding intermediate similar segments with the same statistics are sorted in ascending order of start time.

[0120] Specifically, the server obtains a similar segment list composed of intermediate similar segments for each video set reference video in the target video, and the similar segment list can be obtained by the server in advance by sorting based on each intermediate similar segment. Specifically, first, each intermediate similar segment is sorted in descending order of statistics, and for intermediate similar segments with the same statistics, the server sorts them in ascending order of start time, thereby obtaining a similar segment list. The server performs segment position comparison for each intermediate similar segment in the similar segment list to obtain a segment comparison result. In a specific application, the server can perform segment position comparison in order from front to back according to the order of each intermediate similar segment in the similar segment list to obtain a segment comparison result.

[0121] Further, the step of performing a segment update for each intermediate similar segment for which an overlapping relationship exists, and obtaining a video set local similar segment for each of the video set reference videos in the target video includes a step of performing a segment update for a preceding intermediate similar segment by a succeeding intermediate similar segment among each intermediate similar segment for which an overlapping relationship exists, and obtaining a video set local similar segment for each of the video set reference videos in the target video, where the preceding intermediate similar segment is located before the succeeding intermediate similar segment in the similar segment list.

[0122] Here, in the similar segment list, the preceding intermediate similar segment is positioned before the following intermediate similar segment, i.e., compared to the preceding intermediate similar segment, the following intermediate similar segment is the intermediate similar segment arranged at the back of the similar segment list among the intermediate similar segments with which there is an overlapping relationship, and compared to the following intermediate similar segment, the preceding intermediate similar segment is the intermediate similar segment arranged at the front of the similar segment list. For example, if the similar segment list includes intermediate similar segment A and intermediate similar segment B, and the statistics of intermediate similar segment A are higher than the statistics of intermediate similar segment B, the similar segment list will have the intermediate similar segment A arranged before the intermediate similar segment B, so that the following intermediate similar segment is intermediate similar segment B and the preceding intermediate similar segment is intermediate similar segment A.

[0123] Specifically, the server can determine the following intermediate similar segment and the preceding intermediate similar segment for each intermediate similar segment with which a superposition relationship exists, and perform segment updates for the preceding intermediate similar segment based on the determined following intermediate similar segment, such as performing various update processes such as deletion, merging, and retention to obtain a video set local similar segment for each video set reference video in the target video.

[0124] In this embodiment, based on a similar segment list composed of intermediate similar segments for each video set reference video in the target video, for each intermediate similar segment that has an overlapping relationship, segment updating is performed on the intermediate similar segment in front of it by the intermediate similar segment in back of it, thereby ensuring accurate retention of intermediate similar segments with high statistics, improving the effect of segment updating, and improving the accuracy in identifying local similar segments of the video set from the target video.

[0125] In one embodiment, for intermediate similar segments in the target video for each video set reference video, a segment update is performed for each intermediate similar segment having an overlapping relationship, and a step of obtaining a video set local similar segment for each video set reference video in the target video includes a step of performing a segment update for each intermediate similar segment in the target video for each video set reference video having an overlapping relationship, to obtain an updated intermediate similar segment, a step of determining statistics of the updated intermediate similar segment, and a step of obtaining a video set local similar segment for each video set reference video in the target video based on the updated intermediate similar segment when the statistics of the updated intermediate similar segment exceed a statistics threshold.

[0126] Here, the statistics may include the cumulative number of times that the same intermediate similar segment is identified in the intermediate similar segment for each movie set reference video identification in the target video. The statistics threshold is used to determine whether the updated intermediate similar segment is a valid movie set local similar segment, and the statistics threshold can be set according to actual needs.

[0127] Specifically, the server performs segment update on each intermediate similar segment having an overlapping relationship in the intermediate similar segment for each movie set reference video in the target video, and obtains an updated intermediate similar segment. The server determines statistics of the updated intermediate similar segment. Specifically, the server performs statistical processing on the updated intermediate similar segment to obtain statistics of the updated intermediate similar segment. The server determines a predetermined statistics threshold, and if the statistics of the updated intermediate similar segment exceed the statistics threshold, the updated intermediate similar segment can be regarded as a valid movie set local similar segment, and the server obtains a movie set local similar segment for each movie set reference video in the target video based on the updated intermediate similar segment. For example, the server can use the updated intermediate similar segment as a movie set local similar segment for each movie set reference video in the target video.

[0128] In this embodiment, the validity of the updated intermediate similar segment is judged by a statistical threshold, and after the validity is judged, a video set local similar segment for each video set reference video in the target video is obtained based on the updated intermediate similar segment, and the validity of the identified video set local similar segment can be ensured.

[0129] In one embodiment, the video identification method further includes obtaining a public video matching the public video type in the target video based on the aggregate similar segments when the aggregate similar segments satisfy the criterion for determining the public video type.

[0130] Here, the public video type refers to the type of video publicly used for each video, including but not limited to opening, ending, and advertisement. The public video type can be set according to actual needs. The public video type determination condition is used to determine whether the type of the overall similar segment matches the public video type, specifically, the public video distribution area related to the public video type is compared with the overall similar segment, thereby determining whether the overall similar segment matches the public video type, and thereby determining the type of the overall similar segment. The matching of the public video with the public video type is, that is, matching the type of the public video with the public video type. The public video is a video segment of overlapping use whose type has been determined. For example, the public video can be a video content that can be overlappingly used in each video, such as an opening, ending, or advertisement.

[0131] Specifically, the server determines a judgment condition of the public video type, and if the synthetic similar segment meets the judgment condition, the server obtains a public video that matches the public video type in the target video based on the synthetic similar segment. For example, the judgment condition of the public video type is that it is in a public video distribution section related to the public video type, and the server determines a time period of the synthetic similar segment and determines whether the time period of the synthetic similar segment is already located in the public video distribution section, and if the time period of the synthetic similar segment is in the public video distribution section, the server obtains a public video that matches the public video type based on the synthetic similar segment. In this case, if the public video type is an opening type, the server can obtain an opening in the target video based on the synthetic similar segment, and specifically, the synthetic similar segment can be the opening of the target video.

[0132] In this embodiment, if the identified comprehensive similar segment satisfies the judgment condition for the public video type, a public video matching the public video type in the target video is obtained based on the comprehensive similar segment, thereby identifying a public video matching the public video type from the target video, and improving the identification accuracy when identifying a public video from the target video.

[0133] In one embodiment, when the overall similar segment satisfies the judgment condition of the public video type, the step of obtaining a public video matching the public video type in the target video based on the overall similar segment includes the steps of determining a public video distribution section to which the public video type of the target video is related, and when the time period of the overall similar segment is within the public video distribution section, obtaining a public video matching the public video type in the target video based on the overall similar segment.

[0134] Here, the public video distribution section is a time distribution section in the target video of the public video belonging to the public video type. For example, if the public video type is an opening type, the relevant time distribution section can be the first N seconds of the target video, for example, the first 20 seconds of the target video, that is, the time distribution section is 0s-20s. The time period of the comprehensive similar segment means the time span of the comprehensive similar segment obtained by identification in the target video, and can be determined based on the start time and end time of the comprehensive similar segment, and can be the time span directly from the start time to the end time.

[0135] Specifically, the server determines a public video distribution section associated with the public video type of the target video, and different public video types have different public video distribution sections. For example, if the public video type is an opening type, the associated public video distribution section may be the first N seconds of the video, while if the public video type is an ending type, the associated public video distribution section may be the last M seconds of the video. The server determines a time period of the overall similar segment, and specifically, the time period may be determined based on the start time and end time of the overall similar segment, and if the time period of the overall similar segment is in the public video distribution section associated with the public video type, it indicates that the overall similar segment is within a time span range corresponding to the public video type. The server obtains a public video matching the public video type in the target video based on the overall similar segment. For example, the server may obtain the overall similar segment as a public video matching the public video type in the target video. If the public video type is an ending type, the server determines the overall similar segment as an ending in the target video.

[0136] In this embodiment, based on the comparison result between the public video distribution section related to the public video type and the time period of the comprehensive similar segment, a public video matching the public video type in the target video is determined by the comprehensive similar segment, thereby ensuring the accuracy of the public video matching the public video type from the target video based on the predetermined public video distribution section, and improving the identification accuracy when identifying the public video from the target video.

[0137] In one embodiment, the video identification method further includes determining a start time and an end time of the public video, extracting a non-public video from the target video based on the start time and the end time in response to a video comparison trigger event, and performing a video comparison of the non-public video with the comparison target video.

[0138] Here, the public video is a video segment whose type is specified and used overlappingly. For example, the public video is a video content that can be reused in each video, such as an opening, an ending, or an advertisement. The start time of the public video refers to the time when the public video starts, and the end time of the public video refers to the time when the public video ends. The video comparison trigger event is a trigger event for comparing videos, and the similarity between the videos can be determined by comparing the videos. The non-public video is a video of another segment other than the public video in the target video, and the non-public video is considered to be the main video content of the target video, not a video segment used overlappingly. The comparison target video is a video required for video comparison, and the video similarity between the non-public video and the comparison target video can be determined by performing a video comparison of the non-public video with the comparison target video.

[0139] Specifically, the server determines the start time and end time of the public video, and responds to a video comparison trigger event, such as a video comparison event triggered by a user at a terminal, the server extracts and obtains a non-public video from the target video based on the start time and end time of the public video. Specifically, the server excludes the public video from the target video based on the start time and end time of the public video, thereby extracting and obtaining a non-public video of the target video. The server obtains a comparison target video, and performs video comparison with the non-public video obtained by extracting the comparison target video, thereby obtaining a video comparison result. The video comparison result can reflect the content similarity between the comparison target video and the extracted non-public video.

[0140] In this embodiment, based on the start time and end time of the public video, a non-public video is extracted from the target video for video comparison with the comparison target video, so that the non-public video in the target video can be accurately and quickly located, thereby improving the accuracy and processing efficiency of video comparison.

[0141] In one embodiment, the video identification method further includes the steps of determining a skip time point of the public video, playing the target video in response to a video playback event of the target video, and skipping and playing the public video when the playback of the target video reaches the skip time point.

[0142] Here, the skip time means the time when the public video is reached during the playback of the target video, which means the time when the public video needs to be skipped and not played. The video playback event is a trigger event for playing the target video. Specifically, the server determines a skip time in the public video, and the skip time can be at least one of the start time or the end time in the public video. The server responds to the video playback event for the target video, specifically, triggers a video playback event for the target video on the terminal by the user, plays the target video on the terminal, and skips and plays the public video when the playback of the target video reaches the skip time. That is, directly skips the public video and plays the non-public video in the target video. In a specific application, if the public video is an opening, the skip time can be the start time of the public video, that is, when playing the target video, the opening is skipped and the non-public video after the opening is directly played. Furthermore, if the public video is an ending, the skip time may be the end time of the public video, that is, when playing the target video, the ending is skipped and the playback is ended directly, or another video is switched and played.

[0143] In this embodiment, when the target video is being played back and the playback reaches a skip point of the public video, the public video is skipped and played back, so that the overlapping public video can be skipped and played back during the video playback, and the efficiency of the video playback can be improved.

[0144] In one embodiment, the step of performing image matching of video frames between the target video and the video set reference video to obtain video frame pairs further includes the steps of extracting a video frame to be identified from the target video and extracting a video set reference video frame from the video set reference video, respectively extracting video frame features of the video frame to be identified and the video frame features of the video set reference video frame, and performing feature matching of the video frame features of the video set reference video frame with the video frame features of the video set reference video frame, and obtaining video frame pairs based on the video frame to be identified and the video set reference video frame that have been successfully feature matched.

[0145] Specifically, after obtaining the target video and the video set reference video, the server performs video frame extraction on the target video and the video set reference video, specifically extracting a video frame to be identified from the target video and extracting a video set reference video frame from the video set reference video. The server extracts the video frame features of the video frame to be identified and the video frame features of the video set reference video frame, respectively, and the server performs feature extraction on the video frame to be identified and the video set reference video frame respectively through an image processing model, so as to obtain the video frame features of the video frame to be identified and the video set reference video frame. The server performs feature matching on the video frame features of the video set reference video frame, for example, performing feature distance matching, and can determine that the video frame to be identified and the video set reference video frame corresponding to the feature distance smaller than the feature distance threshold are successful in feature matching. The server obtains a video frame pair according to the video frame to be identified and the video set reference video frame that are successful in feature matching.

[0146] In this embodiment, video frames are extracted from the target video and the video set reference video to perform feature matching, and a video frame pair is obtained based on the identification target video frame and the video set reference video frame that have successfully been feature matched, thereby identifying similar video segments based on the video frame pair obtained by image matching, thereby improving the accuracy of identifying similar video segments.

[0147] In one embodiment, the step of extracting video frame features of the video frame to be identified and the video frame features of the video set reference video frame respectively includes a step of extracting video frame features of the video frame to be identified and the video frame features of the video set reference video frame respectively using an image processing model.

[0148] Here, the image processing model can be a pre-trained artificial neural network model, such as various forms of network models, such as convolutional neural network, residual network, etc. Specifically, the server uses the pre-trained image processing model to respectively extract the video frame features of the identification target video frame and the video frame features of the video set reference video frame. In specific application, the image processing model can be a pre-trained triple neural network model or a multi-task model.

[0149] Furthermore, the training step of the image processing model includes the steps of obtaining training sample images including classification labels, performing feature extraction and image classification on the training sample images using the image processing model to be trained to obtain sample image features and sample image categories of the training sample images, determining a model loss based on the sample image features, sample image category and classification labels, and continuing training after updating the image processing model to be trained based on the model loss, and obtaining the trained image processing model when training is completed.

[0150] Here, the training sample images include classification labels, and the training sample images can set a training dataset according to actual needs. The sample image features are image features obtained by performing feature extraction on the training sample images using the image processing model to be trained, and the sample image category is a classification result obtained by performing classification processing on the training sample images based on the image processing model to be trained. The model loss updates model parameters in the image processing model to be trained to ensure the convergence of the image processing model to be trained and to perform model training. Specifically, the server obtains training sample images including classification labels, and performs feature extraction and image classification on the training sample images using the image processing model to be trained, to obtain sample image features and sample image categories output by the image processing model to be trained. The server determines a model loss based on the sample image features, the sample image category, and the classification label, specifically, it can determine a triple loss based on the sample image features, and determine a classification loss based on the sample image category and the classification label. Specifically, it can be a cross-entropy loss, and the model loss can be obtained based on the triple loss and the classification loss. The server performs updated continued training on the image processing model to be trained based on the model loss, and obtains a fully trained image processing model upon completion of training. The fully trained image processing model can also perform image feature extraction on input image frames and image classification processing on the input image frames.

[0151] In this embodiment, the image processing model to be trained is updated and trained based on the model loss determined by the sample image features, the sample image category, and the classification label, and the trained image processing model extracts the video frame features of the video frame to be identified and the video frame features of the reference video frame of the video set, so that the image processing model can fully extract the video frame features of the input video frame, and improve the accuracy of video frame matching.

[0152] In one embodiment, the step of identifying a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video includes the steps of performing image matching of video frames between the target video and the platform reference video to obtain a video frame pair, where the video frame pair includes a video frame to be identified belonging to the target video and further includes a platform reference video frame image-matched with the video frame to be identified in the platform reference video; determining a time offset of the video frame pair based on the temporal attributes of the video frame to be identified in the video frame pair and the temporal attributes of the video set reference video frame; and screening video frame pairs with matching time offsets, and determining a platform global similar segment in the target video to the platform reference video based on the temporal attributes of the video frame to be identified in the video frame pair obtained by screening.

[0153] Specifically, the same identification method as that of the video set local similar segment can be adopted to identify the platform global similar segment in the target video to the platform reference video. The server performs image matching of video frames in the target video and the platform reference video, and for the obtained video frame pairs, the server determines the time attribute of the video frame to be identified in the video frame pair and the time attribute of the platform reference video frame. The server determines the time offset of the video frame pair according to the time attribute of the obtained video frame to be identified and the time attribute of the platform reference video frame. The server screens each video frame pair according to the time offset, and screens the video frame pairs with matching time offsets. The server determines the time attribute of the video frame to be identified in the video frame pair obtained by screening according to the video frame pairs obtained by screening, and obtains the platform global similar segment in the target video to the platform reference video according to the time attribute of the video frame to be identified.

[0154] In this embodiment, for the target video and the platform reference video, a time offset of a video frame pair is determined based on the time attributes of the image-matched video frame to be identified and the time attributes of the platform reference video frame, and a platform global similar segment in the target video for the platform reference video is determined based on the time attributes of the video frame to be identified in the video frame pair whose time offsets match after screening. Similar video segments with different time lengths are flexibly determined based on the image-matched video frame pairs, thereby improving the accuracy of identifying similar video segments in videos.

[0155] The present invention further provides an application scenario for applying the above video identification method, which is specifically applied to the application scenario as follows:

[0156] When reproducing a video, it is necessary to use relatively pure videos as a material library, and in particular to remove promotional content that has a beneficial effect on production from the video. For example, when it is necessary to generate a compilation video for a user, the pure video parts that do not have meaningless content such as user or platform advertisements from the videos previously uploaded by the user are screened as materials, and then, through the method of video smart synthesis, for example, one segment of video with the highest aesthetic evaluation score from each video is automatically extracted and collected to generate the user's compilation, at which time, it is very important to clean the opening, ending or non-main content of the short video or mini video uploaded by the user in advance.

[0157] In the case of such mini-videos for video users that are recorded by individual users through self-shooting, production, etc. and last less than 2 minutes to share their life, knowledge, knowledge, skills, and viewpoints, the opening and closing may include not only the video segments of the user's promotional information such as the user's logo and QR code information, but also the logo segment of the platform, and the length is 1 to 5 seconds, which is very short compared with movies and dramas. At the same time, some video creators may randomly replace and modify the opening and closing screens, and the information that the platform focuses on promoting varies depending on the time, and the opening and closing screens of the platform also change accordingly, thereby causing the opening and closing of each video to differ in the videos uploaded by users. In addition, over time, the opening and closing of the platform may not be correctly identified, because new promotional information has been added. How to effectively identify the opening and closing of the ultra-short opening and closing created by users themselves, and at the same time, adapt to the cleaning of the non-main video segments of the mini-videos whose opening and closing of the platform are stable only within a certain period of time, is an urgent problem to be solved in order to carry out secondary production of mini-videos. In addition, when mining the opening and ending of a mini video, it is necessary to consider whether there are openings and endings of the platform logo type. The most direct query method is to compare the target video with the global video in the video platform, that is, to query whether there are overlapping openings and endings between the target mini video and the global video. This requires a lot of time and resources, and is therefore not practical to apply.

[0158] Openings and endings can be information such as different screens, different subtitles, different logos, and text of different video themes, and it is difficult to uniformly identify specific patterns using machines, so traditional methods generally tag opening and ending information manually. However, manual tagging requires a large amount of tagging resources each time, which reduces processing efficiency. Traditional opening and ending mining processing methods are mostly multiple video inputs related to drama types, and cannot solve the identification of special openings and endings of We Media's self-produced materials, and can only support mining of openings and endings of videos whose opening and ending times are all fixed in multiple videos. In reality, most opening and ending times are not always strictly aligned, and when different video collection information, different opening scenes, etc. are inserted into the opening, it cannot be strictly guaranteed that the opening times are always aligned. In addition, the traditional opening and ending mining processing method can only support the identification of opening segments with equal time length or endings with equal length, so the location of openings and endings in videos with unequal time length is inaccurate. When using frame-level video features to identify openings and endings, the frame-level video features cannot guarantee that the main content of the text frame, such as the title, and other text type frame images can be successfully matched. That is, in reality, regardless of whether the text content is the same or not, the frame-level video features of all text types are similar, and any change in the time length of the text frame will result in the time of the opening being inaccurate. For example, after the distribution of a certain drama, the content is warned as inappropriate, and from a certain episode of the drama, a text frame of the summary content of this video is added to the opening, which causes the time length of the text frame to be different between the video of the episode and the previous video frame.In addition, many mini-videos do not have corresponding video sets, so there are no valid videos for opening and ending mining, and some mini-videos need to be compared with global videos, and global video comparison means that a large number of videos need to be mined, so each mining takes a long time and is difficult to realize. As for the processing method of building an opening and ending library to mine opening and endings, only those in the opening and ending library can be queried, and the opening and ending library is updated manually, making it difficult to perform opening and ending cutout processing for a large number of videos. Relying too much on manual labor makes it impossible to achieve automation, and automatic repetitive processing and maintenance are impossible.

[0159] In view of this, by analyzing the performance of openings and endings in the scope of global videos and local videos of the same user account, a search and identification method for video openings and endings is presented, which is combined with the search of frame-level time sequence similarity in the local and global scopes of videos based on the construction and query of a global common opening and ending library. Specifically, the construction and maintenance of a common opening and ending library improves the detection effect of the current openings and endings, and the efficient global scope video comparison list reduces the number of comparison videos that need to be mined for openings and endings in the global scope, thereby achieving the mining effect for newly increased openings and endings within a limited time. In addition, the mining of the local videos of the user account quickly identifies the opening and ending segments of the user that have no regularity, and finally merges the user's local mining results with the global results, thereby realizing the mining of video openings and endings. Here, dynamic global mining refers to a method of adopting a global common opening and ending mining method for global videos that are updated in real time and mining in real time based on the current video query method. In contrast, local identification is a method of mining openings and endings from videos of the same user or series as the reference video. The combination of global and local can obtain more comprehensive openings and endings, improving the accuracy of opening and ending identification.

[0160] The video identification method provided in this embodiment supports identification processing of opening and ending segments of any user of a video and opening and ending segments of a platform, and performs mining of a common opening and ending library based on a text OCR (Optical Character Recognition) identification recommendation global matching list, thereby reducing the overall video processing amount and ensuring the mining effect of common opening and ending. In addition, a similarity search of an image sequence is used to realize a cross search of two videos, thereby finding openings and endings that appear overlapping. By constructing a library in which common openings and endings are dynamically updated, it is supported to query openings and endings by searching the library when an input is queried, so that the response efficiency is improved and identification processing of openings and endings of various types of videos is supported. Compared with the conventional opening and ending identification processing method, the video identification method provided in this embodiment supports identification of openings and endings of variable length, and realizes identification of openings and endings that are not aligned in time or have variable time length by using a video frame similarity sequence search. Furthermore, by searching the common opening and ending library and efficiently extracting global videos, it can search and mine openings and endings, improve the mining ability of common opening and endings, and at the same time support the mining of new platform opening and endings, meet the requirement of dynamically maintaining and identifying common opening and endings due to the dynamic update of platform promotion during application, and at the same time control the global video scope of the search, thereby avoiding the resource time consumption being too high for global search of large amounts of data.In addition, by maintaining a common opening / ending and keyword library that supports global library search, it supports the removal of existing openings and endings, as well as the mining of newly added openings, endings or keywords in real time.It also provides the automatic repair ability with simple manual intervention for openings and endings that are missed in the search, further improving the accuracy of video opening and ending identification.

[0161] The video identification method provided in this embodiment can be applied to identifying the opening and ending of a mini video, so that the opening and ending are removed to obtain the main part of the mini video, and the mini video can be used for secondary production of the video, or the mini video can be used for video comparison. As shown in FIG. 4, in secondary production of a user's compilation video, the opening and ending are removed from all videos uploaded by a user, the main part of the video is reserved, and one video segment is cut out every 3 seconds from each video, and all the scenes of each video segment are aesthetically evaluated and scored, and the average score is taken as the aesthetic score of the video of the segment, and the highest aesthetic score of each video is obtained for all the videos of the user. A plurality of video segments are stitched, filter beautified, and output as the compilation video of the user. As shown in FIG. 5, in a user's video comparison application scenario, after identifying the opening and ending of a certain video uploaded by a user, the main part is reserved, and the main part is referenced to perform a similar time period matching with the past video library. If a matching video exists in the past video library, it is indicated whether the video already exists in the past video library or whether a similar video exists, thereby realizing a quick video comparison process. As shown in FIG. 6, when a video A on a certain video platform is played, there is an opening of the platform introduction screen of the video platform, specifically, the screen at 2 seconds. As shown in FIG. 7, the video content of the video A is played, specifically, the screen at 20 seconds (including a person) of the video A is played. As shown in FIG. 8, when the playback of the video A ends, the ending of the platform introduction screen of the bio platform is played next, specifically, the screen at 1 minute and 12 seconds. When editing the video A on the video platform, it is necessary to remove the opening and ending segments of the platform introduction screen in order to reserve the main video content.After multiple users upload videos, the platform adds platform logo segments to the same time period of the videos, so that the videos with the same logo segment can be found more quickly by global video queries in the same time period, and the matched segments are determined to be common endings. As shown in FIG. 9, during a first period, a video platform A includes text and an icon 901 in the opening and closing of the platform introduction screen. As shown in FIG. 10, after a certain period of time, during a second period, the opening and closing of the platform introduction screen of the video platform A includes, in addition to text and an icon 1001, promotion information 1002 for downloading, and specifically, the download link of the application platform.

[0162] Specifically, the video identification method provided in this embodiment is as shown in FIG. 11, where a query video is a target video to be identified, and a user video list of the query video is obtained, and each video in the user video list and the query video belong to the same user account. If the user video list is successfully obtained, opening and ending mining is performed by each video in the user video list to obtain openings and endings, and if the user video list is not successfully obtained, opening and ending mining is not performed on the user video list. In addition, the query video is identified as a common opening and ending, and if the opening and ending cannot be identified, a global video list in the video platform is obtained, and the global video list includes videos extracted from the video platform to which the query video belongs. The query video is mined for opening and ending based on the global video list to obtain openings and endings. The identification result of the common opening and ending is merged with the mining result of the user video list to obtain openings and endings, and then output, or the mining result of the global video list is merged with the mining result of the user video list to obtain openings and endings, and then output. Furthermore, for the mining result of the global video list, common openings and endings are extracted from the mining result, recommended openings and endings corresponding to the extracted common openings and endings are counted and updated, and if the common opening and ending determination condition is satisfied, the extracted common opening and ending is updated to the common opening and ending library after the Tth day.

[0163] Furthermore, for a query video, first, another video of the upload username is mined, where the mining includes searching for similar time periods between video pairs and correcting frame-level OCR keyword queries. If there are no search results after searching in the common opening and ending library, it indicates that the current query video may contain new openings and endings of platform logo type, and in this case, it is necessary to trigger global video mining. Specifically, the OCR platform keyword obtained by identification is used to find recent videos containing the same platform keyword from the global video to construct a global video list, and a search for similar time periods is performed using the query video and the global list video, and when there are results, it indicates that a new platform logo type has appeared. In this case, the search results are output by merging the video search results under the username, and at the same time, the new platform logo type is recommended to the common opening and ending library, and when there are no results, it indicates that this video does not have a matching opening and ending in the global. Furthermore, in order to ensure the automatic addition of common openings and endings, the new global common openings and endings obtained by mining each time are statistically processed by the opening and ending library to determine whether to recommend updating to the common opening and ending library.

[0164] As shown in FIG. 12, the video identification method provided in this embodiment includes processes such as global library query, local list mining, global list generation, global list mining, recording new openings and endings in the common opening and ending library, and maintaining keyword library. Specifically, in the global library query, the embedding features of the frame level image of the query video and the embedding features of the frame level image of the common opening and ending library can be directly adopted, specifically, the frame level images are respectively extracted from the query video and the video of the common opening and ending library, the frame level features of the extracted frame level images are obtained, and a similar time period search is performed based on the frame level features, and the matching time period is regarded as the opening and ending obtained by the search, and the identification result 1 can be obtained. Specifically, the matching time period of the query video and multiple openings and endings in the global library is obtained by querying, where the longest time period is regarded as the final search result. If no matching time period of the opening and ending is found, it is determined that the opening and ending in the query video cannot be identified based on the common opening and ending in the global library.

[0165] In the case of global list mining, by adopting the same processing method as that of local list mining, only the video list for searching is different, and frame level images are obtained from each of the query video and the video in the global list, and the frame level features of each frame level image are extracted to perform fixed segment sequence similarity search processing, and identification result 2 can be obtained. In the case of local list mining, the query video and each video in the user video list are formed into a video pair in a two-to-two manner, and frame level images are obtained for each video pair, and the frame level features of the frame level images are extracted to perform fixed segment sequence similarity search processing, thereby adopting the video frame images to generate similar segments through similar time period search, completing all video pair search, obtaining multiple similar segments, merging them to obtain local openings and endings, and obtaining identification result 4. Meanwhile, for the frame level images obtained in the video pairs, frame level OCR is adopted to find platform keywords from the keyword library, obtaining identification result 3, correcting identification result 4 according to identification result 3, and merging identification result 3 and identification result 4 to obtain a merged result.

[0166] Specifically, for the identification results 3 and 4, the identification result 4 is highly reliable opening and ending information obtained by searching two videos, and the identification result 3 is information on whether or not a frame is invalid, which is obtained by whether or not the screen contains some special vocabulary, and therefore the identification result 4 is corrected by the information of the identification result 3. Here, the role of the identification result 3 is to provide keywords for the opening and ending in the video. For example, the ending is a promotional screen for a certain video platform, which is an invalid screen for derivative video creations, and therefore it is necessary to remove invalid screens near the opening and ending using this special vocabulary. Specifically, a character search method can be adopted to remove screens with such characters from the main story. First, the characters to be removed are stored in a keyword library, and for the OCR obtained by identifying the input frame image, the OCR is queried to see whether the library keyword exists, and if the library keyword is hit, the frame is considered to be an invalid screen. The opening and ending times are corrected using the text search results by determining whether all the screens are invalid based on whether or not they are hit for all the frames.

[0167] In a specific application, for example, when the end time of the opening [2,18] is 18s, the classification information starting from the opening end time is searched for, and if more than 50% of the main screens from the end of the opening to the start of the ending are invalid, the invalid screens are not cleaned. If two or more invalid frames are included within 5s after the end of the opening, that is, in the 19th to 23rd frames, the opening end time is corrected to the time of the last invalid screen, and if there are consecutive invalid screens for a certain period of time after the end of the opening, the opening end time is directly corrected to the longest consecutive invalid time. Similarly, for the ending start time, a certain period of time going back from the start time is searched for, and if an invalid screen appears, the ending start time is corrected to the next 1 second of this invalid screen. As shown in FIG. 13, for opening 1, the time of opening 1 is extended to the end time of the invalid screen containing the platform keyword obtained by identification. As shown in FIG. 14, for ending 1, the time of ending 1 is extended to the start time of the invalid screen containing the platform keyword obtained by identification.

[0168] For the processing of querying by global library, mining by global list, or mining by local list, the fixed segment sequence similarity search processing can be performed based on the frame level features of the frame level images. Specifically, the common opening and ending in the global library, the global video in the global list, or the user video in the local list is used as the reference video of the query video to form a video pair with the query video. For the processing of frame level feature extraction, frames are extracted from the video to obtain frame level images, and the frame level features of each frame level image are extracted. For example, for a 6-second video with 25 FPS (Frames Per Second), one frame is extracted every second, extracting a total of six images, and then for the frame extraction images, the video frame features of each frame are obtained by the feature extractor, so that the video has six video frame features. When a frame extraction method of 3 frames per second is adopted, the final opening and ending identification time accuracy is 0.33 seconds, and for a mini video with a short time, if higher time accuracy is required, a denser frame extraction method with 10 frames per second and an accuracy of 0.1 seconds can be adopted to perform frame extraction processing. Here, the video frames can be extracted by an image feature extractor. The image feature extractor can adopt the ResNet-101 neural network pooling layer output trained based on the open source classification dataset Imagenet to convert each image into a 1x2048 image embedding vector. Here, Imagenet is an open source dataset for large-scale common object identification. The image feature extractor can also be realized based on different network structures and different pre-trained model weights.

[0169] Here, the image embedding is used to describe the features of image information, including image lower layer representation, image semantic features, etc., and the embedding is not limited to floating point features, but may be an image representation consisting of a binary feature vector, i.e., deep hash features. The embedding features in this embodiment may be binarized deep hash features. The image lower layer representation is an image embedding from the lower layer features of deep learning, which describes some representation information such as the texture of the whole image, feature placement, etc., and the image semantic feature is an image embedding from semantic learning, which describes the representation of a specific designated semantic content part in the image. For example, when used to describe the embedding of a dog, the feature of the position where the dog is located in the image is taken as the image representation.

[0170] The configuration of the ResNet-101 convolutional neural network (CNN) deep representation module is shown in Table 1 below.

[0171] [Table 1] Furthermore, by performing OCR classification on each frame-extracted image, text information on each image can be identified.

[0172] In the embedding-based sequence similarity search process, when performing video time period matching, for each video pair (i, r) consisting of a query video and a list video, the list video is a video in a global library, a global list, or a local list, i means the opening and ending query video waiting to be determined, and r means a list video serving as a reference video. Assuming there are three list videos, it is necessary to perform three algorithm calculations of sequence similarity search based on embedding1 and three algorithm calculations of sequence similarity search based on embedding2 for the query video i.

[0173] Specifically, for sequence similarity search, also called time period matching algorithm, it processes one pair of videos each time, and the input of each video is its embedding sequence. The threshold in the time period matching algorithm can be dynamically adjusted according to the needs of the service or the video being processed. The time period matching algorithm step specifically presets the distance threshold of the video frame feature embedding as t0=0.3, that is, the Euclidean distance of the two embeddings is less than 0.3, which indicates that the two embeddings are from similar frames, and the distance threshold can be flexibly set according to actual needs. Frame extraction is performed for the two videos in the video pair to obtain the embedding of each frame. For each frame j in video i, calculate the Euclidean distance between it and each frame embedding in video r, and the frames smaller than t0 are the similar frames of j, and obtain the similar frame or matching frame list sim-id-list of j, and at the same time, record the corresponding similar frame time deviation diff-time-list. For example, for a frame j=1, if the similar frame list sim-id-list is [1,2,3], it indicates that it is similar to the 1st, 2nd, and 3rd seconds of the r video, and if the time deviation diff-time-list is [0,1,2], it indicates the distance between the similar frames in sim-id-list and the time represented by the frame j=1. The default frame extraction is to extract one frame per second, so the frame number is the number of seconds. Therefore, we can obtain the similar frame list SL and the time deviation list TL of all frames in i.

[0174] All frames are traversed to compile statistics on the number of matching frames between i and r videos, i.e., the number of j matching with r video. If the number of matching frames is less than 1, then i and r videos do not have the same video segment, and the opening and ending cannot be mined. Otherwise, the time deviation dt is sorted to obtain the SL list. Specifically, for all matching frames in the SL, sort them in ascending order according to diff-time (i.e., dt), and if dt is the same, sort them in ascending order according to the number of video i. At the same time, reconstruct the corresponding diff-time-list according to this order, i.e., the one with a time difference of 0 is at the top, and the one with a time difference of 1 is next. For example, the new SL list is [10,11], [11,12], [2,4], [3,5], [4,6], [6,9], [7,10].

[0175] To reconstruct data by dt to obtain match-dt-list, specifically, for the list in the similar frame list SL of all frames of i, the time deviation is used as the primary key to reconstruct the list in ascending order of dt, and similar frames match-dt-list:{0:{count,start-id,match-id-list},....} with time deviations of 0, 1, 2... are obtained. For example, {2:{3,2,[[2,4],[3,5],[4,6]]},3:{2,6,[[6,9],[7,10]]}}, where 2 means a time difference of 2. For example, if the second frame of i and the fourth frame of the video vid2 are similar, the time difference between these two frames is 1. count is the number of similar frames in this time deviation. If the second frame of i above and the fourth frame of vid2 are similar, then count is incremented by 1. start-id is the smallest frame id of i in this time difference. For example, if the first frame of i and vid2 are dissimilar, but the second frame of i and the fourth frame of video vid2 are similar, then start-id will be 2.

[0176] Two dt lists in match-dt-list whose previous and next dt are less than 3 (i.e., matching pairs whose matching deviation is within 3s are merged) are merged, and the list with a larger dt is merged into the list with a smaller dt. At the same time, matching updates are performed for similar frames with a larger dt, and the matching frame list SL is updated. For example, in the above example, the list with a dt of 2 is merged into the list with a dt of 3, and finally, {2:{5,2,[[2,4],[3,5],[4,6],[6,8],[7,9]]}} can be obtained. Here, count is the sum of the counts of dt=2 and dt=3, and start-id is the frame where the smallest i video frame is found from the similar frame lists of dt=2 and dt=3. For the list of dt=3, the matched frame numbers are rewritten and merged. For example, [6,9] is rewritten to [6,8] and merged into the similar frame list of dt=2, and at the same time, the similar frame pairs whose frame numbers have been rewritten are updated in synchronization with the SL matching frame list of step 5), for example, to [10,11], [11,12], [2,4], [3,5], [4,6], [6,8], and [7,9]. As described above, if a merged frame list exists, the order of dt or frame id may be disturbed, so it is necessary to rearrange them. Specifically, dt is rearranged. That is, a process of rearranging dt again for the new SL list to obtain an SL list is performed, and a matching frame list rearranged in ascending order of dt (ascending order of frame id of video i) is obtained. Data is reconstructed with dt to obtain a match-dt-list, that is, the process of reconstructing data with dt to obtain a match-dt-list is performed again.

[0177] Calculate the time duration matching list match-duration-list. Specifically, set the time interval between two matching segments to be greater than T2 (e.g., 8s, if 1 second is 1 frame, the difference in frame number is 8), and for each dt (e.g., dt=2) in match-dt-list, for each frame srcT of video i at dt (e.g., 2 out of 2, 3, 4, 6, 7 exemplified above), if the difference between srcT and the previous srcT is greater than T2 (e.g., if the difference between 2 and the previous srcT11 is 9, it is greater than the interval threshold), merge the previous similar frame pair into one matching segment, start a new similar frame pair statistic from the current srcT, and save the similar frames in a temporary list tmplist. At dt=2, srcT=2, the similar frames in the previous temporary frame list are saved as matching segments, and the similar frames of the previous tmplist=[[10,11],[11,12]] are added to match-duration-list as matching segments, and the matching segment information is added as [10,11,11,12,1,2,2], where each value is [src-startTime,src-endTime,ref-startTime,ref-endTime,dt,duration,count], that is, the start frame and end frame of video i of the two-segment video, the start frame and end frame of the matching video, the dt of the matching segment, the duration of the matching segment, and the number of matched similar frames are saved in the matching segment. As shown in FIG. 15, the matching segment information includes information such as the start frame time of the target video, the end frame time of the target video, the start frame time of the matching video, and the end frame time of the matching video. The similar frames of this time are saved in the temporary list as tmplist=[[2,4]]. If the difference between srcT and the previous srcT is less than T2, save the current similar frame in a temporary list tmplist.For example, for dt2, save srcT=3, 4, 6, 7 in a temporary list, so that we have tmplist=[[2,4],[3,5],[4,6],[6,8],[7,9]]. If we are currently at the last similar frame of this dt (say srcT=7), construct a matching segment from the accumulated similar frames in tmplist and add it to match-duration-list. For example, add [2,7,4,9,2,6,5], where duration is 7-2+1 and count=5 is the count of similar frames, so match-duration-list=[[10,11,11,12,1,2,2],[2,7,4,9,2,6,5]]. The above match-duration-list is sorted in reverse order of the number of similar frames count, for example, match-duration-list=[[2,7,4,9,2,6,5],[10,11,11,12,1,2,2]].

[0178] Processing is performed when there is a time period that overlaps in match-duration-list. In the similar frame calculation, if distance calculation is performed by traversing all frames of two videos and operating on those that are similar within a certain threshold range, a certain frame is likely to be similar to multiple frames, which may result in a situation where two matched time periods overlap in time in match-duration-list, and such a situation needs to be handled. Specifically, the time length of the minimum matching segment is set to T3 (for example, set to 5, indicating that the shortest matching time length is 5s), and for time period i (referring to a time period consisting of src-startTime and src-endTime) in match-duration-list, for time period j=i+1 in match-duration-list, if time period j is included in time period i, j is deleted. As shown in FIG. 16, the start time of time period i is before the start time of time period j, and the end time of time period i is after the end time of time period j, that is, time period j is included in time period i, and j needs to be deleted. If there is an overlap between i and j, and the start time of i is the earliest start time, the start time of j is moved backward to the end time position of i, and j is updated. In this case, if the duration of time period j is less than T3, j is deleted, and if not, the old j is replaced with a new j. As shown in FIG. 17, the start time of time period i is before the start time of time period j, the end time of time period i is before the end time of time period j, i and j overlap, and the end time of time period i needs to be updated to the end time of time period j. If i and j overlap, and the start time of j is the earliest start time, the end time of j is moved forward to the start time position of i, and j is updated. In this case, if the duration of time period j is less than T3, j is deleted, and if not, the old j is replaced with a new j. As shown in FIG. 18, the start time of time period i is after the start time of time period j, and the end time of time period i is after the end time of time period j, i and j overlap, and the start time of time period i needs to be updated to the start time of time period j.Finally, return the matching duration information, e.g. match-duration-list=[[2,7,4,9,2,6,5],[10,11,11,12,1,2,2]], or just the matching segments [[2,7,4,9],[10,11,11,12]].

[0179] In the process of obtaining the same matching segment, after performing similarity sequence matching with the video list for the query video, three matching time periods are obtained, and the three time periods are aligned to obtain the same matching segment of the video list based on this embedding. Specifically, assuming that video i needs to be mined from video vid2, video vid3, and video vid4, the aforementioned video segment matching process is performed on each of the total N=3 video pairs, [I,vid2], [I,vid3], and [I,vid4], to obtain three matching information. For example, the video matching segments of the first pair are returned as [[2,7,4,9], [10,11,11,12]], the matching segments of the second pair are returned as [[2,7,4,9]], and the matching segments of the third pair are returned as [[2,7,4,10]]. Count the matching segments, e.g., count [2,7,4,9] twice, [2,7,4,10] once, and [10,11,11,12] once. Sort the matching segments in reverse order of count, and if the counts are the same, sort by src-startTime ascending, to get match-list=[[2,7,4,9],[2,7,4,10],[10,11,11,12]], count-list=[2,1,1].

[0180] Merge the overlapping matching segments in the match-list. Specifically, the effective overlap rate T4 is set to, for example, 0.5, which means that if the overlapping time length of two time periods occupies the time length of the target segment is equal to or greater than T4, the two counts need to be merged for calculation; the effective matching count T5 is set to, for example, 3, which means that if the count of the matching segment of a certain segment is greater than T5, this segment cannot be ignored. For a time period i (referring to a time period composed of src-startTime and src-endTime) in the match-list, for a time period j=i+1 in the match-list, if time period i includes time period j and the time length of the j segment is >0.5*time length of the i segment, delete j, and at the same time, set the count of the i segment = the count of the original i segment + the count of the j segment. If i and j overlap, the overlapping time length is >0.5*time length of the i segment, and the count of the j segment is greater than T5, merge the i and j segment times as the longest start and end time, and set the count of the i segment = the count of the original i segment + the count of the j segment. If the count of j segment is less than T5, delete j segment and set i segment count = original i segment count + j segment count. That is, in this case, i and j segments are not merged, only the i segment with the most occurrences is kept, and the count of j segment is reflected in the count of the new i segment. If i and j overlap and the overlap time length < 0.5 * i segment time length, j segment is discarded. As shown in FIG. 19, if the start time of time period i is before the start time of time period j and the end time of time period i is before the end time of time period j, i and j overlap and the end time of time period i needs to be updated to the end time of time period j. On the other hand, if the start time of time period i is after the start time of time period j and the end time of time period i is after the end time of time period j, i and j overlap and the start time of time period i needs to be updated to the start time of time period j.

[0181] A new video matching segment match-list (e.g., [[2,7,4,9], [10,11,11,12]]) and count count-list (e.g., [3,1]) are obtained. A valid overlapping appearance ratio threshold T6 is set, and in N pairs of video pair mining, a matching video segment has an overlapping appearance count x>N*T6, which means that it is a valid overlapping segment (e.g., T6=0.5). For the match-list, the valid time period is reserved, and match-list=[[2,7,4,9]] and count=[3] are obtained. Here, match-list is the identification result obtained by performing fixed segment sequence similarity search on frame-level features and different list videos.

[0182] In the case of generating a global list, for the obtained OCR keyword, find videos with the same OCR keyword from global videos in the past week or two weeks, and sample 10,000 videos from them to form a global list. Compared with the case of directly using all global videos to generate a global list, fewer videos are required for comparison because videos from the same platform, the same time period, or recent videos are adopted, and the target of updates is narrowed down, making it easier to mine new platform openings and endings. If there is no match with the OCR vocabulary in the keyword library, 10,000 videos are randomly sampled from global videos in the past week to form a global list. In order to efficiently generate a global list, OCR text is pre-extracted for global mini videos and the keyword library is queried, so that each word in the keyword library can be associated with a specific global mini video. The keyword library contains various keywords, and the videos in the video platform are associated with the keywords in the keyword library. In addition, the global list and the queried video have the same keyword, and at the same time, 10,000 videos with the same keyword are adopted and combined with 10,000 global random samples, which can improve generalization performance and improve keyword identification accuracy. As shown in FIG. 12, for a global new video, for example, a video newly uploaded by a user in a video platform, a frame-level image can be extracted from the global new video, character recognition can be performed on the frame-level image, and a keyword query can be performed using the character recognition result and each keyword in the keyword library to realize the collection of video information of the global new video. For example, a related relationship between the global new video and the corresponding keyword can be established. A video information collection process can also be performed on each video in the video platform to obtain a global list.

[0183] Regarding the maintenance of the keyword library, since video platforms are constantly emerging, new video platforms may appear, and the keyword library needs to be dynamically updated and maintained. The keywords of the platform logo segments of the opening and endings that appear on the new video platforms can be directly added to the library, thereby realizing the dynamic update and maintenance of the keyword library. Specifically, during the local list mining, the platform keywords of the query video can be obtained, and the obtained platform keywords can be updated into the keyword library.

[0184] Regarding the recording of new openings and endings in the common opening / ending library, recommended openings and endings can be generated from the anchor point identification result 1 or identification result 2 in the list mining, and stored in the recommendation library, and the number of appearances N1 and the number of new additions N2 of the openings and endings can be recorded. As shown in FIG. 20, a frame-level image obtained from the query video is used to perform a single video common sequence similarity search to obtain openings and endings, and then the number of appearances N1 and the number of new additions N2 of the openings and endings can be updated. In each of the above video list and single video mining, it is inquired whether the opening and ending are included in the recommendation library, and if it is determined that they are included each time, that is, if the opening and ending are included in the mining results of the openings and endings obtained in each of the above video list and single video mining, the number of appearances and the number of new additions of the openings and endings in the recommendation library, that is, N1 and N2, are updated. After T days, openings and endings with a large number of new additions are selected based on the number of new additions and stored in the common opening / ending library.

[0185] Specifically, after the openings and endings with high reliability are mined in the global list mining, the openings and endings can be used in the subsequent video global library query process. In order to ensure the validity of the common opening and ending library, one buffer library, namely, the recommended opening and ending library, can be adopted, which is used to store all the openings and endings generated by the global list mining, and the valid information N1, N2, where N1 is the number of times the openings and endings have appeared, and N2 is the number of times the openings and endings have newly appeared. For a certain opening and ending, N1 is recorded as 1 when it is added to the library, and N2 is recorded as 0. Every time a query video is input, a query is made in the recommended opening and ending library, and if a match with a certain opening and ending is found, 1 is added to the record of N2 of this opening and ending. After a certain period of time, assuming the time threshold is 7 days, sort in descending order based on the number of N2 records, obtain the top 10% of openings and endings with N2>100, obtain the openings and endings that are finally recommended within this period, and record the openings and endings that are finally recommended in the common opening and ending library. If this opening and ending have been recorded in the common opening and ending library, update the records of all recommended opening and ending libraries, i.e., update N1=original N1+N2, N2=0. This starts the statistics for the next period. In addition to N1 and N2, when adding to the library, T time can also be recorded to represent the number of days this video will be added to the library. Openings and endings that have been added to the library for a number of days that is a multiple of 7 days are statistically recorded every day, and if the N2 record is greater than the specified threshold, they are recorded in the common library, and at the same time, update the records of recommended openings and ending libraries that are a multiple of 7 days, i.e., update N1=original N1+N2, N2=0.This will start the statistics for the next period. Other threshold judgment strategies based on N1, N2, and T can also be adopted to update the common opening and ending library. In addition, the time period for updating the recommended opening and ending library to the global opening and ending library can be adjusted in real time, and updated when the traffic reaches a certain threshold based on the daily video traffic statistics.

[0186] A merged result is generated from the identification result 3 and the identification result 4, and the merged result is merged with the identification result 1 or the identification result 2. Since both of the two identification results are obtained based on searching multiple video pairs, the obtained matching time period is strong opening and ending information, that is, the reliability that the time period belongs to the opening and ending is very high. In this case, it is necessary to merge the two identification results to obtain the opening and ending that appear multiple times in the videos. Specifically, when the merged result is merged with the identification result 1 or the identification result 2, the merged result is merged with the opening time segment of the identification result 1 or the identification result 2, and the multiple segments are merged, and the maximum time is set as the opening end time, for example, [2,7], [9,15], [9,13], and after the merged time, [2,15] is output as the opening time period, and 15 is set as the end time. Similarly, for the merged result of the ending of the identification result 1 or the identification result 2, the minimum time is set as the ending start time, and a comprehensive identification result including the opening and ending obtained by comprehensive identification can be obtained.

[0187] The video identification method provided in this embodiment supports the identification of openings and endings with unequal lengths, and realizes the identification of openings and endings with time unaligned or unequal lengths by utilizing video frame embedding similarity sequence search. In addition, by combining the mining of local and global list embeddings, and the identification of openings and endings in the user dimension and platform dimension, the overall judgment effect is improved, and the openings and endings that appear in the platform dimension are avoided from being ignored in normal mining, so that the mini video content can be cleaned more neatly. In addition, for the mined global openings and endings, the mining of openings and endings and closed-loop management of common openings and endings are realized by methods such as recommended opening and ending library, full network recall statistics, and storage in the formal opening and ending library. In addition to opening and ending identification for mini videos, after limited modifications, the video identification method provided in this embodiment can also be applied to the opening and ending identification processing of other types of videos, such as long videos such as movies and dramas, for example, it is necessary to limit the video list of global mining for long videos, thereby avoiding comparison with excessive videos causing increased time consumption.

[0188] It should be understood that although the steps of the flowcharts according to the above-mentioned embodiments are shown in sequence according to the arrows, the steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified in this specification, the execution of the steps is not limited to a strict order, and the steps may be performed in other orders. Furthermore, at least some of the steps of the flowcharts according to the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed simultaneously, but may be performed at different times, and the order of execution of these steps or stages is not necessarily performed sequentially, but may be performed alternately with other steps or at least some of the steps or stages in other steps.

[0189] Based on the same inventive idea, the embodiments of the present application further provide a video identification device for realizing the above-mentioned video identification method. Since the problem solving embodiment provided by this device is similar to the embodiment described in the above-mentioned method, the specific limitations in the embodiment of one or more video identification devices provided below may refer to the limitations related to the above-mentioned video identification method, and will not be described again here.

[0190] In one embodiment, as shown in FIG. 21, a video identification device 2100 is provided, the device comprising: a video set video acquisition module 2102, a local similar segment identification module 2104, a platform video acquisition module 2106, a global similar segment identification module 2108 and an overall similar segment determination module 2110, wherein: The movie set video acquisition module 2102 is configured to acquire a target video and a movie set reference video in a video series movie set, where the video series movie set includes videos belonging to the same series; The locally similar segment identification module 2104 is configured to identify a video set locally similar segment in the target video to the video set reference video based on a first matching result obtained by video frame matching between the target video and the video set reference video; The platform video acquisition module 2106 is configured to acquire a platform reference video from a video platform to which the target video belongs; The global similar segment identification module 2108 is configured to identify a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video; The overall similar segment determination module 2110 is configured to determine overall similar segments in the target video to the video set reference video and the platform reference video based on the positions of the video set local similar segments and the platform global similar segments in the target video, respectively.

[0191] In one embodiment, the apparatus further includes a correction update module configured to perform a correction update on the video set local similar segment based on the correction segment including the correction keyword in the target video to obtain an updated video set local similar segment, and the overall similar segment determination module 2110 is further configured to determine an overall similar segment in the target video to the video set reference video and the platform reference video based on the positions of the updated video set local similar segment and platform global similar segment, respectively, in the target video.

[0192] In one embodiment, the correction updating module includes a correction segment determination module, a timestamp updating module, and a similar segment updating module, where the correction segment determination module is configured to determine a correction segment in the target video including the correction keyword, the timestamp updating module is configured to update a timestamp position of a video set locally similar segment in the target video based on a timestamp position of the correction segment in the target video to obtain an updated timestamp position, and the similar segment updating module is configured to determine an updated video set locally similar segment in the target video based on the updated timestamp position.

[0193] In one embodiment, the correction segment determination module is further configured to perform character identification on video frames in the target video to obtain character identification results, match the character identification results with the correction keyword to obtain a matching result, and determine a correction segment including the correction keyword from the target video based on a video frame associated with the matching result that matches.

[0194] In one embodiment, the platform reference video includes platform public video segments obtained from the public video library of the video platform to which the target video belongs and platform related videos obtained from the video platform, and the global similar segment identification module 2108 includes a public video matching module, a related video matching module, and a matching result processing module, where the public video matching module is configured to perform video frame matching between the target video and the platform public video segments to obtain public video matching results, and if no similar segments can be identified based on the public video matching results, the related video matching module is configured to perform video frame matching of the target video with the platform related video to obtain related video matching results, and the matching result processing module is configured to identify platform global similar segments in the target video to the platform related videos based on the related video matching results.

[0195] In one embodiment, the apparatus further includes a public video updating module configured to update discrimination statistics parameters of the platform global similar segment, obtain updated discrimination statistics parameters, and if the updated discrimination statistics parameters satisfy a platform public judgment condition, update the platform global similar segment as a platform public video segment in the public video library.

[0196] In one embodiment, the platform video acquisition module 2106 is further configured to acquire platform public video segments from a public video library of the video platform to which the target video belongs, and the global similar segment identification module 2108 is further configured to identify platform global similar segments in the target video for the platform public video segments based on a second matching result obtained by video frame matching between the target video and the platform public video segments.

[0197] In one embodiment, the platform video acquisition module 2106 includes a platform determination module, a related video query module, and a video screening module, where the platform determination module is configured to determine a video platform to which the target video belongs and the correction keywords contained in video frames in the target video, the related video query module is configured to query platform-related videos having a related relationship with the correction keywords in the video platform, and the video screening module is configured to screen from the platform-related videos to obtain a platform reference video according to the reference video screening conditions.

[0198] In one embodiment, the device further includes a relation building module configured to perform character identification on video frames in platform videos belonging to a video platform, obtain video keywords, perform matching in a keyword library based on the video keywords, determine target keywords matching the video keywords, and establish relational relationships between the platform videos and the target keywords, and the related video query module is further configured to query platform related videos related to the corrective keywords in the video platform based on the relational relationships.

[0199] In one embodiment, the overall similar segment determination module 2110 includes a timestamp determination module, a timestamp merging module, and an overall timestamp processing module, where the timestamp determination module is configured to determine a first timestamp location of a video set local similar segment in the target video and a second timestamp location of a platform global similar segment in the target video, the timestamp merging module is configured to merge the first timestamp location and the second timestamp location to obtain an overall timestamp location, and the overall timestamp processing module is configured to determine an overall similar segment in the target video to the video set reference video and the platform reference video based on the overall timestamp location.

[0200] In one embodiment, the local similar segment identification module 2104 comprises a video set video frame matching module, a video set offset determination module, and a video set video frame pair processing module, where the video set video frame matching module is configured to perform image matching of video frames between a target video and a video set reference video to obtain a video frame pair, the video frame pair including a to-be-identified video frame belonging to the target video and further including a video set reference video frame that image-matches with the to-be-identified video frame in the video set reference video, the video set offset determination module is configured to determine a time offset of the video frame pair based on temporal attributes of the to-be-identified video frame in the video frame pair and temporal attributes of the video set reference video frame, and the video set video frame pair processing module is configured to screen video frame pairs with matching time offsets, and determine a video set locally similar segment in the target video to the video set reference video based on temporal attributes of the to-be-identified video frame in the screened video frame pair.

[0201] In one embodiment, the video set video frame pair processing module is further configured to perform numerical matching on the time offsets of each video frame pair, screen video frame pairs whose time offsets match numerically based on the numerical matching result, determine a start time and an end time based on temporal attributes of the identified video frame in the screened video frame pairs, and determine a video set local similar segment from the target video to the video set reference video based on the start time and the end time.

[0202] In one embodiment, the moving image set video frame pair processing module is further configured to obtain a video frame pair list consisting of the video frame pairs obtained by screening, in which each video frame pair is sorted in order from smallest to largest numerical value of a time offset, and video frame pairs with the same time offset are sorted in order from smallest to largest numerical value of a time stamp of the included video frame to be identified, and the time stamp is determined based on the temporal attribute of the included video frame to be identified, in the video frame pair list, determine a temporal attribute distance between the temporal attributes of the video frames to be identified in adjacent video frame pairs, determine adjacent video frame pairs whose temporal attribute distance does not exceed a distance threshold as video frame pairs belonging to the same video segment, and determine a start time and an end time based on the time stamp of the video frames to be identified in the video frame pairs belonging to the same video segment.

[0203] In one embodiment, the moving image set video frame pair processing module is further configured to determine a start video frame pair and an end video frame pair from the video frame pairs belonging to the same video segment based on a timestamp of the identified video frame in the video frame pair belonging to the same video segment, obtain a start time based on the timestamp of the identified video frame in the start video frame pair, and obtain an end time based on the timestamp of the identified video frame in the end video frame pair.

[0204] In one embodiment, the moving image set video frame pair processing module is further configured to numerically compare the time offsets of each video frame pair respectively, obtain a numerical comparison result, and based on the numerical comparison result, screen video frame pairs from each video frame pair whose numerical difference in time offset is less than a numerical difference threshold, perform offset updating for the video frame pairs whose numerical difference in time offset is less than the numerical difference threshold, and obtain video frame pairs whose time offsets are numerically matched.

[0205] In one embodiment, the video set reference videos are at least two, and the video set video frame pair processing module is further configured to screen video frame pairs with matching time offsets, determine intermediate similar segments in the target video to the video set reference video based on temporal attributes of the identified video frame in the video frame pairs obtained by screening, perform segment update for each intermediate similar segment in the target video to each video set reference video that has an overlapping relationship among the intermediate similar segments, and obtain video set local similar segments in the target video to each video set reference video.

[0206] In one embodiment, the video set video frame pair processing module is further configured to perform segment updating for each intermediate similar segment in the target video that has an overlapping relationship among the intermediate similar segments for each video set reference video, obtain an updated intermediate similar segment, determine statistics of the updated intermediate similar segment, and if the statistics of the updated intermediate similar segment exceed a statistics threshold, obtain a video set local similar segment in the target video for each video set reference video based on the updated intermediate similar segment.

[0207] In one embodiment, the video set video frame pair processing module is further configured to perform segment position comparison for intermediate similar segments in the target video for each video set reference video, obtain segment comparison results, determine each intermediate similar segment for which an overlap relationship exists as a result of the segment comparison, perform segment update for each intermediate similar segment for which an overlap relationship exists based on the overlap time length and statistics of each intermediate similar segment for which an overlap relationship exists, and obtain video set local similar segments in the target video for each video set reference video.

[0208] In one embodiment, the video set video frame pair processing module is further configured to obtain a similar segment list consisting of intermediate similar segments in the target video for each video set reference video, in which each intermediate similar segment is sorted in order from largest to smallest statistics, and intermediate similar segments with the same statistics are sorted in order from earlier to later start times, and to perform segment position comparison for each intermediate similar segment in the similar segment list to obtain a segment comparison result.

[0209] In one embodiment, the video set video frame matching module is further configured to extract a video frame to be identified from the target video, extract a video set reference video frame from the video set reference video, extract video frame features of the video frame to be identified and the video frame features of the video set reference video frame respectively, perform feature matching of the video frame features of the video frame to be identified with the video frame features of the video set reference video frame, and obtain a video frame pair based on the video frame to be identified and the video set reference video frame that have been successfully feature matched.

[0210] In one embodiment, the video set video frame matching module is further configured to use an image processing model to extract video frame features of the video frame to be identified and the video frame features of the video set reference video frame, respectively, where the training step of the image processing model includes the steps of obtaining training sample images including classification labels, performing feature extraction and image classification on the training sample images by the image processing model to be trained, and obtaining sample image features and sample image categories of the training sample images, determining a model loss based on the sample image features, sample image categories and classification labels, and continuing training after updating the image processing model to be trained based on the model loss, and obtaining a trained image processing model when training is completed.

[0211] In one embodiment, the global similar segment identification module 2108 includes a global video frame matching module, a global offset determination module, and a global video frame pair processing module, where the global video frame matching module is configured to perform image matching of video frames between the target video and the platform reference video to obtain a video frame pair, where the video frame pair includes a to-be-identified video frame belonging to the target video and further includes a platform reference video frame that image-matches with the to-be-identified video frame in the platform reference video, the global offset determination module is configured to determine a time offset of the video frame pair based on temporal attributes of the to-be-identified video frame in the video frame pair and temporal attributes of the video set reference video frame, and the global video frame pair processing module is configured to screen video frame pairs with matching time offsets, and determine a platform global similar segment in the target video to the platform reference video based on temporal attributes of the to-be-identified video frame in the video frame pair obtained by screening.

[0212] In one embodiment, the apparatus further includes a video set identification and update module configured to determine a segment overlap relationship between each video set locally similar segment based on a start time and an end time of each video set locally similar segment, perform segment updating for each video set locally similar segment based on the segment overlap relationship, and obtain updated video set locally similar segments for the video set reference video in the target video.

[0213] In one embodiment, the apparatus further includes a public video judgment module configured to obtain a public video matching a public video type in the target video based on the overall similar segment when the overall similar segment meets a judgment condition of a public video type.

[0214] In one embodiment, the public video judgment module is further configured to determine a distribution interval of a public video related to the public video type of the target video, and obtain a public video matching the public video type in the target video based on the overall similar segment when the time period of the overall similar segment is within the distribution interval of the public video.

[0215] In one embodiment, the apparatus further includes a video comparison module configured to determine a start time and an end time of the public video, and in response to a video comparison trigger event, extract a non-public video from the target video based on the start time and the end time, and perform a video comparison of the non-public video with a comparison target video.

[0216] In one embodiment, the device further includes a video skip module configured to determine a skip time point in the public video, play the target video in response to a video playback event for the target video, and skip and play the public video when playback of the target video reaches the skip time point.

[0217] All or part of each module in the above video identification device can be realized by software, hardware, or a combination thereof. Each module may be integrated into a processor in a computer device in the form of hardware or may be independent, or may be stored in a memory in a computer device in the form of software, so that the processor can call the modules to execute the operations corresponding to the modules.

[0218] In one embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as shown in FIG. 22. The computer device includes a processor, a memory, an input / output interface (I / O: Input / Output) and a communication interface. Wherein, the processor, the memory and the input / output interface are connected by a system bus, and the communication interface is connected to the system bus by the input / output interface. Wherein, the processor of the computer device is configured to provide a calculation function and a control function. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer-readable instructions and a database. The internal memory provides an environment for the execution of the operating system and the computer-readable instructions in the non-volatile storage medium. The database of the computer device is configured to store video identification data. The input / output interface of the computer device is configured to allow the processor to exchange information with an external device. The communication interface of the computer device is configured to connect and communicate with an external terminal through a network. The computer-readable instructions, when executed by a processor, cause the processor to realize a video identification method. Those skilled in the art will understand that the structure shown in Figure 22 is only a partial structural block diagram of the technical solution of the present invention, and does not limit the computer device to which the technical solution of the present invention is applied, and a specific computer device may have more or less parts than those shown in the drawings, or may combine some parts, or have a different parts arrangement.

[0219] In one embodiment, there is further provided a computing device comprising a memory having computer readable instructions stored thereon, and a processor for executing the computer readable instructions to implement the steps in each of the method embodiments described above.

[0220] In one embodiment, a computer readable storage medium is provided having stored thereon computer readable instructions which, when executed by a processor, perform the steps in each of the method embodiments described above.

[0221] In one embodiment, a computer program product is provided that includes computer readable instructions which, when executed by a processor, implement the steps in each of the method embodiments described above.

[0222] In addition, the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) of the present invention are information and data approved by the user or fully approved by all parties, and the collection, use and processing of related data should comply with the relevant laws, regulations and standards of the relevant countries and regions. Furthermore, with respect to the platform promotion information of the present invention, the user can reject or conveniently reject the advertising push information.

[0223] It is obvious to those skilled in the art that the realization of all or part of the process of the method of the above embodiments can be completed by implementing related hardware with computer-readable instructions, and the above-mentioned computer-readable instructions can be stored in a non-volatile computer-readable storage medium, and when the computer-readable instructions are executed, the process of each of the above method embodiments can be included. Here, any reference to memory, database, or other medium used in each embodiment of the present invention can include at least one of non-volatile and volatile memory. The non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, the RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The database according to the embodiments of the present invention may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a distributed database based on a blockchain, and the like. The processor according to the embodiments of the present invention may be, but is not limited to, a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like.

[0224] Each technical feature in the above examples can be arbitrarily combined, and for the sake of brevity, all possible combinations of each technical feature in the above examples are not described, but as long as there is no contradiction in the combination of these technical features, they should all be considered as being within the scope described in this specification. The above-described examples only represent some embodiments of the present invention, and the description is specific and detailed, but should not be understood as a limitation on the patent scope of the present invention. It should be noted that those skilled in the art can also make some modifications and improvements without departing from the concept of the present invention, and all of these should be considered to be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the attached claims.

Claims

1. 1. A computer device implemented method for video identification, comprising: Obtaining a target video and a video set reference video in a video series video set, the video series video set including videos belonging to the same series; identifying a video set locally similar segment in the target video to the video set reference video based on a first matching result obtained by video frame matching between the target video and the video set reference video; obtaining a platform reference video from a video platform to which the target video belongs; identifying a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video; determining a global similarity segment in the target video to the video set reference video and the platform reference video based on the positions in the target video of the video set local similarity segment and the platform global similarity segment, respectively; Video Identification Methods.

2. The video identification method includes: The method further includes performing a correction update on the video set locally similar segment according to a correction segment in the target video including a correction keyword, to obtain an updated video set locally similar segment; determining a global similarity segment in the target video to the video set reference video and the platform reference video based on the positions of the video set local similarity segment and the platform global similarity segment in the target video, determining a global similarity segment in the target video to the video set reference video and the platform reference video based on the updated positions of the video set local similarity segments and the platform global similarity segments in the target video; 2. The method of claim 1, further comprising:

3. The step of performing a correction update on the video set locally similar segment based on a correction segment including a correction keyword in the target video to obtain an updated video set locally similar segment includes: determining a redemption segment in the target video that includes a redemption keyword; updating a timestamp position of the movie set locally similar segment in the target video according to a timestamp position of the correction segment in the target video to obtain an updated timestamp position; and determining an updated movie set local similar segment in the target video based on the updated timestamp position.

3. The method of claim 2, further comprising:

4. The step of determining a correction segment in the target video that includes a correction keyword includes: performing text identification on video frames in the target video to obtain a text identification result; matching the text identification result with the corrected keywords to obtain a matching result; determining a correction segment including the correction keyword from the target video based on a video frame associated with a matching result that the match is found to be a match; 4. The method of claim 3, further comprising:

5. The platform reference video includes a platform public video segment obtained from a public video library of a video platform to which the target video belongs, and a platform related video obtained from the video platform; identifying a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video, performing video frame matching between the target video and the platform public video segment to obtain a public video matching result; if no similar segment can be identified based on the public video matching result, performing video frame matching between the target video and the platform related video to obtain a related video matching result; and identifying platform-global similar segments in the target video to the platform-related videos based on the related video matching results.

2. The method of claim 1, further comprising:

6. After identifying a platform global similar segment in the target video to the platform related video based on the related video matching result, the video identification method includes: updating a discrimination statistic parameter of the platform global similar segment to obtain an updated discrimination statistic parameter; If the updated discrimination statistical parameters satisfy a platform public judgment condition, updating the platform global similar segment as a platform public video segment in the public video library.

6. The method of claim 5, further comprising:

7. The step of obtaining a platform reference video from a video platform to which the target video belongs includes: obtaining a platform public video segment from a public video library of a video platform to which the target video belongs; identifying a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video, and identifying a platform global similar segment in the target video to the platform public video segment based on a second matching result obtained by video frame matching between the target video and the platform public video segment.

2. The method of claim 1, further comprising:

8. The step of obtaining a platform reference video from a video platform to which the target video belongs includes: determining a video platform to which the target video belongs and a correction keyword contained in a video frame in the target video; Querying platform-related videos in the video platform that have a related relationship with the correction keyword; and screening the platform-related videos to obtain a platform reference video according to a reference video screening condition.

2. The method of claim 1, further comprising:

9. The video identification method includes: performing text recognition on video frames in platform videos belonging to the video platform to obtain video keywords; performing matching within a keyword library based on the video keywords to determine target keywords that match the video keywords; establishing an association between the platform video and the target keyword; The step of querying the platform-related videos having a related relationship with the correction keyword in the video platform includes: querying the video platform for platform-related videos related to the corrective keyword based on the related relationship; 9. The video identification method according to claim 8, characterized in that

10. determining a global similarity segment in the target video to the video set reference video and the platform reference video based on the positions of the video set local similarity segment and the platform global similarity segment in the target video, determining a first timestamp location of the video collection local similar segment in the target video and a second timestamp location of the platform global similar segment in the target video; merging the first timestamp location with the second timestamp location to obtain an overall timestamp location; determining a collective similar segment in the target video to the video collection reference video and the platform reference video based on the collective timestamp position.

2. The method of claim 1, further comprising:

11. Identifying a video set locally similar segment in the target video to the video set reference video based on a first matching result obtained by video frame matching between the target video and the video set reference video includes: performing image matching of video frames between the target video and the video set reference video to obtain a video frame pair, the video frame pair including a video frame to be identified belonging to the target video, and a video set reference video frame that image-matches the video frame to be identified in the video set reference video; determining a time offset of the video frame pair based on a temporal attribute of the identified video frame and a temporal attribute of a video set reference video frame in the video frame pair; and screening video frame pairs having matching time offsets, and determining a video set local similar segment in the target video to the video set reference video based on a temporal attribute of a video frame to be identified in the screened video frame pairs. A video identification method according to any one of claims 1 to 10, characterized in that

12. The step of screening the video frame pairs having matching time offsets and determining a video set local similar segment in the target video to the video set reference video based on a temporal attribute of a video frame to be identified in the video frame pairs obtained by screening includes: performing numerical matching on the time offsets of each of the video frame pairs, and screening video frame pairs whose time offsets match numerically based on the numerical matching result; determining a start time and an end time based on a time attribute of the identified video frame in the screened video frame pair; determining a video set local similar segment from the target video to the video set reference video based on the start time and the end time.

12. The method of claim 11, further comprising:

13. performing numerical matching on the time offsets of each of the video frame pairs; and screening video frame pairs having numerically matching time offsets based on the numerical matching results; numerically comparing the time offsets of each of the video frame pairs respectively to obtain a numerical comparison result; based on the numerical comparison result, screening each of the video frame pairs for video frame pairs whose numerical difference of time offset is less than a numerical difference threshold; performing offset updates on video frame pairs whose numerical difference in time offset is less than a numerical difference threshold to obtain video frame pairs whose time offsets match numerically.

13. The method of claim 12, further comprising:

14. The video set reference video is at least two, and the step of screening video frame pairs having matching time offsets and determining a video set local similar segment in the target video to the video set reference video based on a time attribute of a video frame to be identified in the video frame pairs obtained by screening includes: screening video frame pairs with matching time offsets, and determining intermediate similar segments in the target video to the movie set reference video based on temporal attributes of a target video frame in the screened video frame pairs; performing segment updating for each intermediate similar segment in the target video that has an overlapping relationship among the intermediate similar segments to each of the video set reference videos, to obtain a video set local similar segment in the target video to each of the video set reference videos.

12. The method of claim 11, further comprising:

15. The step of performing segment updating for each intermediate similar segment having an overlapping relationship among intermediate similar segments in the target video with respect to each of the video set reference videos to obtain a video set local similar segment in the target video with respect to each of the video set reference videos includes: performing segment position comparison for intermediate similar segments in the target video with respect to each of the movie set reference videos to obtain a segment comparison result; determining each intermediate similar segment having an overlapping relationship as a result of the segment comparison; and performing segment updating for each intermediate similar segment having an overlapping relationship based on the overlapping time length and statistics of each intermediate similar segment having an overlapping relationship, to obtain a video set local similar segment in the target video for each of the video set reference videos.

15. The method of claim 14, further comprising:

16. The step of performing image matching of video frames between the target video and the video set reference video to obtain video frame pairs includes: extracting a video frame to be identified from the target video and extracting a video set reference video frame from the video set reference video; extracting video frame features of the identification target video frame and video frame features of the movie set reference video frames respectively; performing feature matching between the video frame features of the identification target video frame and the video frame features of the video set reference video frame, and obtaining a video frame pair based on the identification target video frame and the video set reference video frame that have been successfully feature matched; 12. The method of claim 11, further comprising:

17. identifying a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video, performing image matching of video frames between the target video and the platform-reference video to obtain a video frame pair, the video frame pair including a target video frame belonging to the target video and a platform-reference video frame that image matches the target video frame in the platform-reference video; determining a time offset of the video frame pair based on a temporal attribute of the identified video frame and a temporal attribute of a video set reference video frame in the video frame pair; screening video frame pairs having matching time offsets, and determining a platform global similar segment in the target video to the platform reference video based on a time attribute of a target video frame in the screened video frame pairs.

2. The method of claim 1, further comprising:

18. 1. A video identification device, comprising: a video set video acquisition module configured to acquire a target video and a video set reference video in a video series video set, the video series video set including videos belonging to the same series; a local similar segment identification module configured to identify a video set locally similar segment in the target video to the video set reference video based on a first matching result obtained by video frame matching between the target video and the video set reference video; a platform video acquisition module configured to acquire a platform reference video from a video platform to which the target video belongs; a global similar segment identification module configured to identify a platform global similar segment in the target video to the platform reference video based on a second matching result obtained by video frame matching between the target video and the platform reference video; an overall similar segment determination module configured to determine an overall similar segment in the target video to the video set reference video and the platform reference video based on the positions in the target video of the video set local similar segment and the platform global similar segment, respectively; Equipped Video identification device.

19. A computing device comprising a memory in which computer readable instructions are stored and a processor which, when said computer readable instructions are executed, performs the steps of the method according to any one of claims 1 to 10 and 17.

20. A computer program product causing a processor to carry out the method according to any one of claims 1 to 10 and 17.

Citation Information

Patent Citations

  • Video clip identification method and device, equipment and storage medium

    CN114550070A

  • Moving picture processor, moving picture processing method and moving picture processing program

    JP2005130416A

  • Method, system, and program for topic guidance in video content using sequence pattern mining

    JP2019024192A

  • Video scene classification device and video scene classification method

    US20090257649A1