Video frame type identification method and device, electronic equipment and storage medium

Abstract

The present disclosure discloses a video frame type identification method and device, electronic equipment and storage medium, wherein the method comprises: obtaining a video frame sequence, calculating the image similarity of each video frame in a plurality of previous video frames and a target video frame respectively; determining whether the target video frame is a key frame according to the calculated image similarity. Since the video frame sequence includes the target video frame and a plurality of previous video frames arranged before the target video frame in the video frame sequence, the present disclosure not only determines whether the target video frame is a key frame according to one previous video frame adjacent to the target video frame, but also determines whether the target video frame is a key frame according to a plurality of previous video frames before the target video frame. Since the image similarity of a plurality of previous video frames before the target video frame and the target video frame is considered, the present disclosure can effectively improve the accuracy of key frame identification.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly relates to a video frame type identification method and device, an electronic device and a storage medium. BACKGROUND

[0002] Identifying a video frame type is very important for video coding technology. An I frame (I Frame) is an intra-frame coded frame, also known as a key frame. The key frame does not need to refer to other video frames, and thus has a higher coding quality, but also consumes a lot of code rate and system resources. Other non-key frames can be coded by referring to the key frame, thereby reducing the consumption of code rate.

[0003] Accurate identification of a key frame is very important for video coding technology. The key frame identification technology in the related art compares two adjacent video frames as a whole, determines an image similarity, and determines whether it is a key frame according to the image similarity, which cannot accurately identify the key frame. SUMMARY

[0004] In view of the above problems, the present disclosure provides a video frame type identification method, device, electronic device and storage medium which overcome the above problems or at least partially solve the above problems, and the technical solutions are as follows.

[0005] According to a first aspect of an embodiment of the present disclosure, a video frame type identification method is provided, comprising:

[0006] obtaining a video frame sequence, wherein the video frame sequence includes a target video frame and a plurality of previous video frames arranged before the target video frame in the video frame sequence;

[0007] calculating an image similarity of each of the plurality of previous video frames and the target video frame, respectively;

[0008] determining whether the target video frame is a key frame according to the calculated image similarity.

[0009] In a possible implementation manner of the first aspect, the calculating an image similarity of each of the plurality of previous video frames and the target video frame, respectively, comprises:

[0010] determining the image similarity of each of the plurality of previous video frames and the target video frame, respectively, according to a global image similarity and / or a local image similarity, wherein the global image similarity is a similarity of a global image of each of the plurality of previous video frames and a global image of the target video frame; and the local image similarity is a similarity of a local image in each of the plurality of previous video frames and a local image in the target video frame at a same position.

[0011] In a possible implementation manner of the first aspect, the obtaining of the local image similarity includes:

[0012] For each video frame in the presequence video frames, two first granularity image blocks respectively located in the video frame and the target video frame and having the same position are determined as an image block group, and similarity of the two first granularity image blocks in each image block group is calculated respectively and taken as the local image similarity, wherein the first granularity image blocks are image blocks obtained by respectively segmenting each video frame in the video frame sequence according to a first preset image segmentation manner.

[0013] In a possible implementation manner of the first aspect, the obtaining of the local image similarity includes:

[0014] The first granularity image blocks located in the same video frame are divided into a plurality of image block groups, two image block groups respectively located in a presequence video frame and the target video frame and matched are determined as an image block group, and similarity of the two image block groups in each image block group is calculated respectively and taken as the local image similarity, wherein the first granularity image blocks are image blocks obtained by respectively segmenting each video frame in the video frame sequence according to a first preset image segmentation manner, and the matched image block groups have the same position in the video frames.

[0015] In a possible implementation manner of the first aspect, the obtaining of the local image similarity includes:

[0016] For each video frame in the presequence video frames, two second granularity image blocks respectively located in the video frame and the target video frame and having the same position are determined as an image block group, and similarity of the two second granularity image blocks in each image block group is calculated respectively and taken as the local image similarity, wherein the second granularity image blocks are image blocks obtained by respectively segmenting each video frame in the video frame sequence according to a second preset image segmentation manner, and the first granularity image blocks and the second granularity image blocks have different sizes.

[0017] In a possible implementation manner of the first aspect, the determining of the image similarity between each presequence video frame and the target video frame according to the overall image similarity and / or the local image similarity includes:

[0018] For each presequence video frame, the overall image similarity and the local image similarities calculated according to the presequence video frame are weighted and averaged, and a result of the weighted and averaged calculation is determined as the image similarity between the presequence video frame and the target video frame.

[0019] In a possible implementation manner of the first aspect, the determining whether the target video frame is a key frame according to the calculated image similarity includes:

[0020] determining a maximum value among the calculated image similarities;

[0021] determining whether the target video frame is a key frame according to the maximum value.

[0022] In a possible implementation manner of the first aspect, the determining whether the target video frame is a key frame according to the calculated image similarity includes:

[0023] multiplying the calculated image similarity by a sequence weight of a previous video frame corresponding to the calculated image similarity, and determining whether the target video frame is a key frame according to a calculation result.

[0024] In a possible implementation manner of the first aspect, an upper limit of a number of video frames included in the video frame sequence is a first number, and the method further includes:

[0025] when the target video frame is a key frame, deleting each previous video frame of the target video frame from the video frame sequence, adding at least one subsequent video frame after the target video frame to the video frame sequence, determining one video frame in the at least one subsequent video frame as a new target video frame, and returning to execute the step of calculating the image similarity between each of the plurality of previous video frames and the target video frame;

[0026] and / or, when the target video frame is not a key frame, determining whether the number of video frames included in the video frame sequence is the first number, if yes, deleting a video frame with a highest sequence order in the video frame sequence from the video frame sequence, determining a video frame after the target video frame as a new target video frame and adding the new target video frame to the video frame sequence, and returning to execute the step of calculating the image similarity between each of the plurality of previous video frames and the target video frame.

[0027] According to a second aspect of the embodiments of the present disclosure, a video frame type identification device is provided, including:

[0028] a sequence obtaining unit configured to execute obtaining a video frame sequence, wherein the video frame sequence includes a target video frame and a plurality of previous video frames with a sequence order before the target video frame in the video frame sequence;

[0029] a similarity calculating unit configured to execute calculating an image similarity between each of the plurality of previous video frames and the target video frame;

[0030] The key frame determination unit is configured to determine whether the target video frame is a key frame according to the calculated image similarity.

[0031] In a possible implementation of the second aspect, the similarity calculation unit is configured to determine the image similarity of each of the plurality of previous video frames with the target video frame according to an overall image similarity and / or a local image similarity, wherein the overall image similarity is the similarity of the overall image of each of the plurality of previous video frames with the overall image of the target video frame, and the local image similarity is the similarity of a local image in each of the plurality of previous video frames with a local image in the target video frame at a same position.

[0032] In a possible implementation of the second aspect, the similarity calculation unit is further configured to obtain the local image similarity by:

[0033] For each of the previous video frames, two first-granularity image blocks at the same position in the video frame and in the target video frame are determined as an image block group, and the similarity of the two first-granularity image blocks in each image block group is calculated as a local image similarity, wherein the first-granularity image blocks are image blocks obtained by dividing each of the video frames in the video frame sequence according to a first preset image division manner.

[0034] In a possible implementation of the second aspect, the similarity calculation unit is further configured to obtain the local image similarity by:

[0035] Each first-granularity image block in the same video frame is divided into a plurality of image block groups, two matching image block groups in one previous video frame and in the target video frame are determined as an image block group, and the similarity of the two image block groups in each image block group is calculated as a local image similarity, wherein the first-granularity image blocks are image blocks obtained by dividing each of the video frames in the video frame sequence according to a first preset image division manner, and the matching image block groups are at the same position in the video frames.

[0036] In a possible implementation of the second aspect, the similarity calculation unit is further configured to obtain the local image similarity by:

[0037] For each video frame in the pre-sequence video frames, two second granularity image blocks respectively located in the video frame and the target video frame are determined as an image block group, and similarity of the two second granularity image blocks in each image block group is calculated respectively as a local image similarity, wherein the second granularity image blocks are image blocks obtained by dividing each video frame in the sequence of video frames according to a second preset image division manner, and the size of the first granularity image block is different from the size of the second granularity image block.

[0038] In a possible implementation of the second aspect, the similarity calculation unit is configured to perform: for each pre-sequence video frame, performing weighted average calculation on the overall image similarity and the local image similarity calculated according to the pre-sequence video frame, and determining the calculation result as the image similarity of the pre-sequence video frame and the target video frame.

[0039] In a possible implementation of the second aspect, the key frame determination unit is configured to perform: determining a maximum value in the calculated image similarities, and determining whether the target video frame is a key frame according to the maximum value.

[0040] In a possible implementation of the second aspect, the key frame determination unit is configured to perform: multiplying the calculated image similarity and the ordering weight of the corresponding pre-sequence video frame, and determining whether the target video frame is a key frame according to the calculation result.

[0041] In a possible implementation of the second aspect, an upper limit of the number of video frames included in the sequence of video frames is a first number, and the apparatus further comprises a first processing unit and / or a second processing unit,

[0042] The first processing unit is configured to perform: when the target video frame is a key frame, deleting each pre-sequence video frame of the target video frame from the sequence of video frames, adding at least one subsequent video frame after the target video frame to the sequence of video frames, and determining one of the at least one subsequent video frame as a new target video frame, and triggering the similarity calculation unit.

[0043] The second processing unit is configured to perform: when the target video frame is not a key frame, determining whether the number of video frames included in the sequence of video frames is the first number, if yes, deleting a video frame with the earliest order in the sequence of video frames from the sequence of video frames, determining a video frame after the target video frame as a new target video frame and adding the new target video frame to the sequence of video frames, and triggering the similarity calculation unit.

[0044] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises at least one processor, and at least one memory connected with the processor through a bus; wherein the processor, the memory and the bus complete mutual communication; the processor is configured to invoke program instructions in the memory to execute any one of the video frame type identification methods described above.

[0045] According to a fourth aspect of the embodiments of the present disclosure, a storage medium is provided, which stores a program, and the program is executed by a processor to implement any one of the video frame type identification methods described above.

[0046] According to the technical solutions described above, the video frame type identification method, device, electronic device and storage medium provided by the present disclosure can obtain a video frame sequence, calculate the image similarity of each video frame in a plurality of previous video frames and a target video frame respectively, and determine whether the target video frame is a key frame according to the calculated image similarity. Since the video frame sequence comprises the target video frame and a plurality of previous video frames arranged before the target video frame in the video frame sequence, the present disclosure not only determines whether the target video frame is a key frame according to one previous video frame adjacent to the target video frame, but also determines whether the target video frame is a key frame according to a plurality of previous video frames before the target video frame. Since the image similarity of the plurality of previous video frames before the target video frame and the target video frame is considered, the present disclosure can effectively improve the accuracy of key frame identification.

[0047] The above description is only a summary of the technical solutions of the present disclosure. In order to enable a clearer understanding of the technical means of the present disclosure, the contents of the specification can be implemented, and in order to enable the above and other purposes, features and advantages of the present disclosure to be more obvious and easy to understand, the specific implementation manner of the present disclosure is described below. BRIEF DESCRIPTION OF DRAWINGS

[0048] Various other advantages and benefits will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments, and are not meant to limit the present disclosure. Moreover, the same reference numerals are used throughout the accompanying drawings to represent same or similar components. In the drawings:

[0049] Figure 1 is a flowchart of a video frame type identification method according to an exemplary embodiment;

[0050] Figure 2 is a flowchart of a video frame type identification method according to another exemplary embodiment;

[0051] Figure 3 is a schematic diagram of a focus image and a background image according to an exemplary embodiment;

[0052] Figure 4 is a schematic diagram of first granularity image blocks obtained by segmenting a target video frame according to a first preset image segmentation manner, according to an example embodiment;

[0053] Figure 5 is a schematic diagram of first granularity image blocks obtained by segmenting a certain previous video frame according to a first preset image segmentation manner, according to an example embodiment;

[0054] Figure 6 is a schematic diagram of second granularity image blocks obtained by segmenting a target video frame according to a second preset image segmentation manner, according to an example embodiment;

[0055] Figure 7 is a schematic diagram of second granularity image blocks obtained by segmenting a certain previous video frame according to a second preset image segmentation manner, according to an example embodiment;

[0056] Figure 8 is a schematic diagram of dividing image blocks contained in a video frame into image block groups, according to an example embodiment;

[0057] Figure 9 is a schematic diagram of dividing image blocks in a previous video frame and a target video frame into image block groups respectively and obtaining image block group sets, according to an example embodiment;

[0058] Figure 10 is another schematic diagram of dividing image blocks in a previous video frame and a target video frame into image block groups respectively and obtaining image block group sets, according to an example embodiment;

[0059] Figure 11 is a structural schematic diagram of a video frame type identification device, according to an example embodiment;

[0060] Figure 12 is a structural schematic diagram of an electronic device, according to an example embodiment. DETAILED DESCRIPTION

[0061] Example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. While example embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0062] Figure 1 is a flowchart of a video frame type identification method, according to an example embodiment. As shown in FIG. 8, the video frame type identification method includes the following steps.Figure 1 As shown in the method can comprise:

[0063] S100, obtaining a video frame sequence, wherein the video frame sequence includes a target video frame and a plurality of presequence video frames arranged before the target video frame in the video frame sequence.

[0064] It can be understood that the video includes a plurality of video frames, which can be arranged in a certain order, which can be the order of video frame display when playing the video, of course, it can also be other order, for example: the order of transmission, etc., which is not limited by the present disclosure.

[0065] Among them, the plurality of presequence video frames can include key frames, or can not include key frames.

[0066] The video frame sequence obtained in step S100 can be a sequence composed of part of the video frames in the video, and each video frame in the video frame sequence can be adjacent in turn or be spaced apart by other video frames outside the video frame sequence.

[0067] The number of video frames in the video frame sequence obtained in step S100 can be not more than a preset number, by which the present disclosure can control the number of video frames in the video frame sequence, thereby limiting the number of video frames to be processed when identifying whether a target video frame is a key frame each time, and further reducing the amount of calculation and code rate consumption.

[0068] Among them, the video frame sequence in the present disclosure can be transmitted by other devices into the device executing the video frame type identification method provided by the present disclosure. For example, the video acquisition device such as camera obtains the video including a plurality of video frames after video acquisition, and then the video acquisition device sends the video frame sequence composed of a plurality of video frames to the device executing the video frame type identification method provided by the present disclosure. In other embodiments, the video frame sequence in the present disclosure can be generated by the device executing the video frame type identification method provided by the present disclosure. For example, the device executing the video frame type identification method provided by the present disclosure has video acquisition function.

[0069] Among them, the target video frame can be the last video frame in the video frame sequence, and the present disclosure can sequentially take each of the plurality of video frames as the target video frame and perform the processing of the method to identify whether each of the plurality of video frames is a key frame. Figure 1 As shown in the method can comprise:

[0070] The presequence video frame in the present disclosure is the video frame arranged before the target video frame in the video frame sequence. The "previous video frame" and "subsequent video frame" mentioned in the present disclosure refer to the front and rear arrangement in the video frame sequence.

[0071] Optionally, the video frame sequence can be stored in a preset storage space, and when it is needed to adjust the video frames in the video frame sequence, the video frame sequence stored in the preset storage space can be processed (such as deletion, addition, or modification of the video frames) to achieve the purpose.

[0072] S200, calculate the image similarity of each video frame in the plurality of previous video frames and the target video frame.

[0073] Optionally, the disclosure can calculate the overall image similarity of each video frame in the plurality of previous video frames and the overall image of the target video frame, and determine the overall similarity as the image similarity.

[0074] Optionally, the disclosure can calculate the local image similarity of each video frame in the plurality of previous video frames and the local image of the target video frame, and determine the image similarity of the previous video frame and the target video frame according to the local image similarity.

[0075] It can be understood that the disclosure can divide each video frame into the same number of image blocks, and then the disclosure can determine the local image similarity of at least part of the image blocks of the target video frame and the image blocks of the same position of a previous video frame. For example, the disclosure divides the target video frame into two parts by the middle line, and divides each previous video frame into two parts by the middle line. Then the disclosure processes each previous video frame as follows:

[0076] determine the first local image similarity of the upper part of the previous video frame and the upper part of the target video frame, and determine the second local image similarity of the lower part of the previous video frame and the lower part of the target video frame.

[0077] The local image similarity in the disclosure can be the similarity of an image block in the target video frame and an image block of the same position in a previous video frame. It can be understood that for the target video frame and a previous video frame, the disclosure can obtain at least one local image similarity according to the two video frames.

[0078] Optionally, after obtaining the plurality of local image similarities of the target video frame and a previous video frame, the disclosure can integrate the plurality of local image similarities to obtain an integrated similarity. Optionally, the specific process of integrating the plurality of local image similarities to obtain the integrated similarity can include:

[0079] The average of the plurality of local image similarities is calculated, and the average of the plurality of local image similarities is determined as the integrated similarity. Optionally, the average can be an arithmetic average or a geometric average or a weighted average. Optionally, the present disclosure can set different or same weights for local images in different positions and use the weights in the calculation process of the weighted average. Optionally, the weight of a local image located in a middle region of a video frame can be greater than the weight of a local image located in an edge region of the video frame. The present disclosure finds that the key region in an image is generally located in the middle of the image, and therefore the above-mentioned difference in the weights can make the key region in the image have a greater influence on the integrated similarity, and the image similarity can be better measured from the perspective of the key region.

[0080] The present disclosure can directly determine the integrated similarity as the image similarity between the previous video frame and the target video frame, or can determine the image similarity between the previous video frame and the target video frame according to the overall image similarity and the integrated similarity.

[0081] Of course, in other embodiments of the present disclosure, the image similarity between the previous video frame and the target video frame can be determined according to the overall image similarity and / or the local image similarity without obtaining the integrated similarity.

[0082] Of course, in other embodiments of the present disclosure, the overall image similarity between the overall image of each of a plurality of previous video frames and the overall image of the target video frame can be calculated, and the local image similarity between the local image in each of the plurality of previous video frames and the local image in the target video frame with the same position can be calculated, and then the image similarity between each of the plurality of previous video frames and the target video frame can be determined according to the overall image similarity and the local image similarity.

[0083] Since the overall image similarity and the local image similarity of the image are integrated, the reliability of the image similarity can be effectively improved, and the accuracy of the key frame recognition can be improved.

[0084] S300, determining whether the target video frame is a key frame according to the calculated image similarity.

[0085] Specifically, the disclosure can compare the image similarity with a preset similarity threshold, and determine whether the target video frame is a key frame according to the comparison result. Since the image similarity calculated in step S200 is multiple (each previous video frame has an image similarity with the target video frame), the disclosure can determine the maximum value in the image similarity, and determine whether the target video frame is a key frame according to the maximum value. The disclosure can also average the calculated image similarity, and then determine whether the target video frame is a key frame according to the calculation result. Of course, there are other methods for determining whether the target video frame is a key frame by calculating multiple image similarities, which are not limited by the disclosure.

[0086] Optionally, step S300 can specifically include:

[0087] Determine the maximum value in the calculated image similarity.

[0088] Determine whether the target video frame is a key frame according to the maximum value.

[0089] The disclosure can effectively reduce the number of similarities used by using the maximum value in the image similarity to determine whether the target video frame is a key frame, thereby reducing the calculation amount. At the same time, since the maximum value of the similarity reflects the maximum degree of similarity between the previous video frame and the target video frame, the key frame determined by the maximum value also has high accuracy.

[0090] Specifically, the disclosure can determine whether the target video frame is a key frame according to the size relationship between the maximum value in the image similarity and the preset similarity threshold. When the maximum value is not greater than the preset similarity threshold, it is determined that the target video frame is a key frame, otherwise it is determined that the target video frame is not a key frame. In other embodiments, when the maximum value is greater than the preset similarity threshold, the disclosure can also determine that the target video frame is a forward prediction frame, i.e. P frame (P-Frame).

[0091] Of course, the disclosure can also determine whether the target video frame is a key frame according to more image similarities. Optionally, step S300 can specifically include:

[0092] Multiply the calculated image similarity by the ordering weight of the corresponding previous video frame, and determine whether the target video frame is a key frame according to the calculation result.

[0093] Optionally, the ordering weight of the previous video frame with a high order in the video frame sequence is smaller than the ordering weight of the previous video frame with a low order in the video frame sequence.

[0094] Optionally, a sorting weight of a first preceding video frame in the sequence of video frames is less than a sorting weight of a second preceding video frame in the sequence of video frames. The first preceding video frame is sorted before the second preceding video frame in the sequence of video frames. The first preceding video frame is one of the preceding video frames in the sequence of video frames, and the second preceding video frame is another one of the preceding video frames in the sequence of video frames.

[0095] The disclosure can adjust the influence degree of the image similarity between each preceding video frame and the target video frame on determining whether the target video frame is a key frame through the sorting weight. By making the sorting weight of the preceding video frame with a higher sorting position less than the sorting weight of the preceding video frame with a lower sorting position, the disclosure makes the preceding video frame closer to the target video frame have a greater influence degree on determining whether the target video frame is a key frame, and can effectively improve the accuracy of the determined key frame.

[0096] The video frame type identification method provided by the disclosure can obtain a sequence of video frames, calculate the image similarity between each video frame in a plurality of preceding video frames and a target video frame, and determine whether the target video frame is a key frame according to the calculated image similarity. Since the sequence of video frames includes the target video frame and a plurality of preceding video frames sorted before the target video frame in the sequence of video frames, the disclosure not only determines whether the target video frame is a key frame according to one preceding video frame adjacent to the target video frame, but also determines whether the target video frame is a key frame according to a plurality of preceding video frames before the target video frame. Since the image similarity between a plurality of preceding video frames before the target video frame and the target video frame is considered, the disclosure can effectively improve the accuracy of key frame identification.

[0097] Figure 2 is a flowchart of a video frame type identification method according to another exemplary embodiment. As shown in Figure 2 , the method can include:

[0098] S100, obtaining a sequence of video frames, wherein the sequence of video frames includes a target video frame and a plurality of preceding video frames sorted before the target video frame in the sequence of video frames.

[0099] S210, determining the image similarity between each video frame in the plurality of preceding video frames and the target video frame according to the overall image similarity and / or the local image similarity, wherein the overall image similarity is the similarity between the overall image of each video frame in the plurality of preceding video frames and the overall image of the target video frame; and the local image similarity is the similarity between the local image in each video frame in the plurality of preceding video frames and the local image in the target video frame with the same position.

[0100] wherein step S210 is Figure 1An optional specific implementation of step S200 is shown.

[0101] S300, determining whether the target video frame is a key frame according to the calculated image similarity.

[0102] Since steps S100 and S300 have been described in the foregoing Figure 1 The embodiments shown will not be described again.

[0103] Optionally, the disclosure can calculate the overall image similarity of the overall image of each of the plurality of previous video frames and the overall image of the target video frame.

[0104] The disclosure can calculate the overall image similarity by normalizing the correlation coefficient. In actual application, when calculating the overall image similarity of one previous video frame and the target video frame, the disclosure can respectively obtain the image features of part or all of the pixel points in the previous video frame and the image features of part or all of the pixel points in the target video frame, and then calculate the overall image similarity by normalizing the correlation coefficient. Of course, the disclosure can also use other similarity measurement methods to calculate the overall image similarity, which is not limited herein.

[0105] The disclosure can calculate the local image similarity of the local image in each of the plurality of previous video frames and the local image in the target video frame with the same position.

[0106] Optionally, the disclosure can select part of the image from the video frame as the local image for calculating the local image similarity, for example: selecting the background image from the video frame as the local image for calculating the local image similarity. For another example: selecting the focal point image from the video frame as the local image for calculating the local image similarity. Wherein, one image can be divided into focal point image and background image according to the primary and secondary display content. Figure 3 The cartoon animals, baskets, sausages and music symbols in the image shown are focal point images, and the clouds, lawns and plants are background images. The disclosure can distinguish the focal point image and the background image according to various features of the image, for example: first recognizing the contours of each object in the image according to the difference between adjacent pixels, thereby recognizing a plurality of objects from the image. Then, according to the display position of each object in the image, the image complexity, the image clarity and other features to identify whether each object is a focal point image or a background image. Wherein, the display position of the focal point image is close to the center of the image, and the display position of the background image is close to the edge of the image; the image complexity of the focal point image is higher than that of the background image; the image clarity of the focal point image is higher than that of the background image.

[0107] Of course, the disclosure can also use other methods to identify the focal point image and the background image.

[0108] Of course, the disclosure can also use other methods to calculate the local image similarity of the local image in each of the plurality of pre-sequence video frames respectively consistent with the local image in the target video frame. For example: the local image similarity provided by the embodiment of the disclosure can specifically include:

[0109] For each of the pre-sequence video frames: determine two first granularity image blocks respectively located in the video frame and the target video frame as a group of image blocks, and calculate the similarity of the two first granularity image blocks in each group of image blocks as the local image similarity, wherein the first granularity image block is obtained by dividing each video frame in the video frame sequence according to a first preset image segmentation manner.

[0110] Optionally, the disclosure can divide each video frame in the video frame sequence into a plurality of first granularity image blocks according to the first preset image segmentation manner. Wherein the size of the first granularity image block is the first size.

[0111] Wherein, each first granularity image block in the same video frame does not overlap. In actual application, the disclosure can divide the video frame into image blocks of different sizes according to a plurality of different preset image segmentation manners.

[0112] Wherein, the first preset image segmentation manner can be a segmentation manner of dividing the video frame into NXM first granularity image blocks uniformly. Wherein, N and M are both natural numbers and both greater than 1, N represents the number of first granularity image blocks included in each row after segmentation, and M represents the number of first granularity image blocks included in each column after segmentation. Optionally, N=M. Optionally, the shape of the first granularity image block can be rectangular.

[0113] For the convenience of understanding, the following examples are given:

[0114] Suppose the video frame is as shown in Figure 3 , the disclosure can divide it into 4x4 first granularity image blocks as shown in Figure 4 . Wherein, the first granularity can be represented by the reciprocal of the number of first granularity image blocks, such as 1 / 16, or represented by other ways. Figure 4 The size of each first granularity image block in the Figure 3 is consistent, and the length and width of each first granularity image block are respectively one quarter of the length and one quarter of the width of the image.

[0115] It should be noted that step S221 not only divides the target video frame according to the first preset image segmentation manner, but also divides each pre-sequence video frame according to the first preset image segmentation manner. Since the same segmentation manner is used, the sizes of each first granularity image block in the target video frame and the pre-sequence video frame are consistent, and the image blocks having the same position in the target video frame and the pre-sequence video frame can be conveniently used for similarity calculation. It is assumed that Figure 4 is a schematic diagram of a first granularity image block obtained by dividing the target video frame according to the first preset image segmentation manner. Figure 5 is a schematic diagram of a first granularity image block obtained by dividing a certain pre-sequence video frame according to the first preset image segmentation manner. Then, by comparing Figure 4 and Figure 5 it can be known that the two images each include a first granularity image block having the same position, for example: Figure 4 the first granularity image block in the upper left corner of Figure 5 has the same position as the first granularity image block in the upper left corner of

[0116] The present disclosure can determine, for each video frame in the pre-sequence video frames, two first granularity image blocks respectively located in the video frame and the target video frame and having the same position as each other as an image block group, and calculate the similarity of the two first granularity image blocks in each image block group.

[0117] The local image similarity can include the similarity of the two first granularity image blocks in each image block group.

[0118] Since the pre-sequence video frame and the target video frame each include a first granularity image block having the same position, the present disclosure can determine, for each video frame in the pre-sequence video frames, two first granularity image blocks respectively located in the video frame and the target video frame and having the same position as each other as an image block group, and calculate the similarity of the two first granularity image blocks in each image block group. The comparison of the two first granularity image blocks having the same position is more important, and the similarity obtained by the comparison has higher confidence.

[0119] The present disclosure can calculate the similarity of the two first granularity image blocks in various ways, such as a normalized correlation coefficient. The way of calculating the similarity of the image blocks and the way of calculating the overall similarity can be the same or different, which is not limited in the present disclosure.

[0120] Since there is not only one pair of first granularity image blocks having the same position, the present disclosure can obtain multiple image block groups from each pre-sequence video frame and the target video frame. Since there are multiple pre-sequence video frames, the present disclosure can obtain more image block groups according to each pre-sequence video frame.

[0121] Of course, in addition to the first preset image segmentation manner, the disclosure can also use other preset image segmentation manners to segment the video frames, so as to obtain image blocks of different sizes and perform similarity calculation.

[0122] In other embodiments of the disclosure, the obtaining process of the local image similarity can include:

[0123] For each video frame in the preceding video frame sequence: determine two second granularity image blocks respectively located in the video frame and the target video frame and having the same position as each other as an image block group, and calculate the similarity of the two second granularity image blocks in each image block group respectively and take it as the local image similarity, wherein the second granularity image blocks are obtained by segmenting each video frame in the video frame sequence according to a second preset image segmentation manner, and the size of the first granularity image block is different from the size of the second granularity image block.

[0124] The disclosure can segment the video frames into image blocks of different granularities according to at least two different image segmentation manners, and determine the local image similarity of the image blocks of each granularity respectively. In this way, the disclosure can obtain the local image similarity under multiple scales, and can effectively improve the accuracy and robustness of the image similarity.

[0125] The disclosure can segment each video frame in the video frame sequence into a plurality of second granularity image blocks according to a second preset image segmentation manner.

[0126] The second size can be greater than the first size, or can be less than the first size.

[0127] The local image similarity can include the similarity of the two second granularity image blocks in each image block group.

[0128] The second preset image segmentation manner will be illustrated below:

[0129] Suppose the video frame is as shown in Figure 3 , the disclosure can segment it into 2x2 second granularity image blocks as shown in Figure 6 . The second granularity can be represented by the reciprocal of the number of second granularity image blocks, such as 1 / 4, or can be represented by other manners. Figure 6 The size of each second granularity image block is consistent in the image, and the length and width of each second granularity image block are respectively half of the length and half of the width of the image. Figure 3 The size of each second granularity image block is consistent in the image, and the length and width of each second granularity image block are respectively half of the length and half of the width of the image.

[0130] Suppose Figure 6 is a second granularity image block diagram obtained by segmenting the target video frame according to the second preset image segmentation manner, and suppose Figure 7This is a schematic diagram of the second-granularity image block obtained after segmenting a certain preceding video frame according to a second preset image segmentation method. Then, by comparison... Figure 6 and Figure 7 It can be seen that both images contain second-granularity image patches in the same location, for example: Figure 6 The second granularity image patch in the upper left corner and Figure 7 The position of the second granularity image block in the upper left corner is consistent.

[0131] It is understandable that calculating the similarity of image patches of different granularities can enable this disclosure to obtain local image similarity at more scales, which can effectively improve the accuracy and robustness of image similarity.

[0132] Optionally, in other embodiments, the process of obtaining local image similarity may include the following step one:

[0133] Step 1: Divide each first-granularity image block within the same video frame into multiple image block groups. Determine two matching image block groups located in a preceding video frame and a target video frame, respectively, as an image block group. Calculate the similarity between the two image block groups in each image block group and use it as the local image similarity. The first-granularity image block is obtained by segmenting each video frame in the video frame sequence according to the first preset image segmentation method. Matching image block groups are in the same position in the video frame.

[0134] This disclosure can divide each video frame in a video frame sequence into multiple first-granularity image blocks according to a first preset image segmentation method, wherein the size of the first-granularity image block is a first size.

[0135] In practical applications, this disclosure can execute step one multiple times for each video frame, with the number of image blocks contained in each resulting image block group varying each time step one is executed. For example: when this disclosure executes step one for a video frame for the first time, the number of image blocks in each resulting image block group is 2. When this disclosure executes step one for the same video frame for the second time, the number of image blocks in each resulting image block group is 4. When this disclosure executes step one for the same video frame for the third time, the number of image blocks in each resulting image block group is 6.

[0136] Optionally, each image patch group within an image patch group contains the same number of image patches.

[0137] By the above step one, the present disclosure can segment each video frame only once without multiple segmentations of the same video frame. In order to determine the similarity of the image blocks of different scales, the present disclosure can divide the first granularity image blocks into image block groups by using different image block division manners in step one. In this way, an image block group containing a certain number of first granularity image blocks is an image block of a scale matching the number. As shown in Figure 8 , the present disclosure divides a video frame into Figure 8 4×4 first granularity image blocks. Then, the four first granularity image blocks covered by the diagonal line are divided into an image block group. When the granularity is represented by the reciprocal of the number of image blocks, it can be seen from Figure 8 that the image block group covered by the diagonal line is actually an image block of 1 / 4 granularity. In this way, the present disclosure can obtain the image blocks of different scales and the similarity of the image blocks of different scales by segmenting each video frame only once.

[0138] The local image similarity can include the similarity of two image block groups in each image block group.

[0139] Optionally, after the present disclosure divides each video frame in the video frame sequence into a plurality of image blocks of a certain granularity according to a preset image segmentation manner, the present disclosure can divide N adjacent image blocks into an image block group according to the positional relationship of the image blocks, where N is not less than 2 and N is a positive integer.

[0140] The image blocks in an image block group can be adjacent to each other and form a region without gaps. When each video frame in the video frame sequence is segmented according to the above preset image segmentation method and the image block groups are divided in the same way, each video frame will have image block groups with consistent positions. For convenience, the following will be illustrated by Figure 9 .

[0141] Figure 9 From top to bottom, each dashed box contains two video frames, the video frame on the left in each dashed box is the same video frame, and the video frame on the right in each dashed box is the same video frame. As shown in Figure 9 , the present disclosure can uniformly segment the two video frames in Figure 9 into 4×4 image blocks of 1 / 16 granularity according to the same image segmentation method. Among them, Figure 9 , the diagonal line regions in the two video frames in the topmost dashed box contain the same four image blocks, i.e., Figure 9 , the four image blocks contained by the diagonal line regions in the two video frames in the topmost dashed box are image block groups respectively. The present disclosure can divide Figure 9The diagonal areas in the two video frames in the uppermost dashed box in FIG. 1 are determined as one image block group. The disclosure can determine Figure 9 The diagonal area (one image block group) of the left video frame in the uppermost dashed box in FIG. 2 is similar to Figure 9 The diagonal area (another image block group) of the right video frame in the uppermost dashed box in FIG. 2 is similar to

[0142] In response, the disclosure can also divide other image blocks in FIG. 2 into image block groups, thereby obtaining other image block groups. For example, Figure 9 The diagonal areas in the two video frames in the second layer dashed box in FIG. 3 each contain the same four image blocks. The disclosure determines the diagonal areas in the left and right video frames in the second layer dashed box as one image block group respectively, and determines the similarity of the two image block groups. By analogy, the disclosure can also divide other image blocks in FIG. 3 into image block groups, thereby obtaining other image block groups and calculating the similarity of the image block groups in each image block group. For example, Figure 9 The diagonal areas in the two video frames in the second layer dashed box in FIG. 3 each contain the same four image blocks. The disclosure determines the diagonal areas in the left and right video frames in the second layer dashed box as one image block group respectively, and determines the similarity of the two image block groups. By analogy, the disclosure can also divide other image blocks in FIG. 3 into image block groups, thereby obtaining other image block groups and calculating the similarity of the image block groups in each image block group. For example, Figure 9 The diagonal areas in the two video frames in the second layer dashed box in FIG. 3 each contain the same four image blocks. The disclosure determines the diagonal areas in the left and right video frames in the second layer dashed box as one image block group respectively, and determines the similarity of the two image block groups. By analogy, the disclosure can also divide other image blocks in FIG. 3 into image block groups, thereby obtaining other image block groups and calculating the similarity of the image block groups in each image block group. For example, Figure 9 The diagonal areas in the two video frames in the second layer dashed box in FIG. 3 each contain the same four image blocks. The disclosure determines the diagonal areas in the left and right video frames in the second layer dashed box as one image block group respectively, and determines the similarity of the two image block groups. By analogy, the disclosure can also divide other image blocks in FIG. 3 into image block groups, thereby obtaining other image block groups and calculating the similarity of the image block groups in each image block group. For example,

[0143] Figure 10 The image blocks contained in different image block groups in the same video frame in FIG. 4 are all different. Through this processing, the calculation amount can be effectively reduced.

[0144] In actual application, the disclosure can also divide one image block into multiple image block groups. For example, Figure 10 As shown in FIG. 5, the diagonal areas in each dashed box switch positions in turn according to the arrow. It can be seen that the four image blocks contained in the diagonal area in the uppermost dashed box have two same image blocks with the four image blocks contained in the diagonal area in the second layer dashed box on the left. Although Figure 1 The image block group division mode of FIG. 6 will increase the calculation amount, but since there are more image block groups and the positions are different, the sampling amount of the local image can be effectively improved, thereby improving the calculation accuracy of the image similarity.

[0145] Specifically, the disclosure can integrate the overall image similarity and the local image similarity into the image similarity according to a certain calculation mode, which can be summation, average, weighted summation, weighted average, etc. The average in the embodiment of the disclosure can be arithmetic average or geometric average.

[0146] The disclosure can determine the image similarity between each preceding video frame and the target video frame according to the overall image similarity and the local image similarity between the preceding video frame and the target video frame.

[0147] Optionally, step S210 can specifically include:

[0148] For each preceding video frame, the overall image similarity and the local image similarities calculated according to the preceding video frame are weighted and averaged to determine the image similarity between the preceding video frame and the target video frame.

[0149] Specifically, when the local image similarities include the local image similarities of the first granularity image blocks, the present disclosure can perform weighted and average calculation on the overall image similarity and the local image similarities of the first granularity image blocks calculated according to the target video frame and one preceding video frame according to the formula

[0150]

[0151] to determine the image similarity between the preceding video frame and the target video frame. Wherein, Sim is the image similarity between the preceding video frame and the target video frame, W is the weight of the overall image similarity, W1 is the weight of the local image similarity of the first granularity image blocks, Sim is the overall image similarity between the target video frame and the preceding video frame, Sim is the local image similarity of the two first granularity image blocks respectively located in the target video frame and the preceding video frame and numbered as j, and n is the number of the first granularity image blocks. 整体 整体 1，j

[0152] Specifically, when the local image similarities include the local image similarities of the first granularity image blocks and the local image similarities of the second granularity image blocks, the present disclosure can perform weighted and average calculation on the overall image similarity, the local image similarities of the first granularity image blocks and the local image similarities of the second granularity image blocks calculated according to the target video frame and one preceding video frame according to the formula

[0153]

[0154] to determine the image similarity between the preceding video frame and the target video frame. Wherein, Sim is the image similarity between the preceding video frame and the target video frame, W is the weight of the overall image similarity, W1 is the weight of the local image similarity of the first granularity image blocks, W2 is the weight of the local image similarity of the second granularity image blocks, Sim is the overall image similarity between the target video frame and the preceding video frame, Sim is the local image similarity of the two first granularity image blocks respectively located in the target video frame and the preceding video frame and numbered as j, Sim is the local image similarity of the two second granularity image blocks respectively located in the target video frame and the preceding video frame and numbered as j, and n is the number of the first granularity image blocks. 整体 整体 1，j 2，j ​​​​​​wherein, the first granularity image blocks with the same number are located at the same position in each video frame, and the second granularity image blocks with the same number are located at the same position in each video frame.

[0155] Optionally, the video frame type identification method provided by other embodiments of the present disclosure can further include:

[0156] When the target video frame is a key frame, each of the preceding video frames of the target video frame is deleted from the video frame sequence, at least one of the subsequent video frames after the target video frame is added to the video frame sequence, and one of the at least one of the subsequent video frames is determined as a new target video frame, and the step S200 is executed again. Figure 1 The step S200 is executed again.

[0157] It can be understood that when the target video frame is a key frame, the subsequent video frames of the target video frame can refer to the target video frame without referring to the preceding video frames of the target video frame. Therefore, the present disclosure can delete each of the preceding video frames of the target video frame from the video frame sequence when the target video frame is a key frame. Of course, the target video frame which is a key frame is still kept in the video frame sequence. Meanwhile, the present disclosure adds at least one of the subsequent video frames after the target video frame to the video frame sequence, and determines one of the at least one of the subsequent video frames as a new target video frame, so that the video frames after the newly determined key frame can be identified whether they are key video frames. Optionally, when the target video frame is a key frame, the present disclosure can add one of the subsequent video frames after the target video frame to the video frame sequence, or add multiple of the subsequent video frames after the target video frame to the video frame sequence. When multiple of the subsequent video frames after the target video frame are added to the video frame sequence, the present disclosure can determine the last one of the multiple of the subsequent video frames as a new target video frame, so that the calculation amount can be effectively reduced.

[0158] Optionally, the upper limit of the number of the video frames included in the video frame sequence is a first number, and the video frame type identification method provided by other embodiments of the present disclosure can further include:

[0159] When the target video frame is not a key frame, it is determined whether the number of the video frames included in the video frame sequence is the first number, if yes, the first one of the video frames in the video frame sequence is deleted from the video frame sequence, one of the video frames after the target video frame is determined as a new target video frame and added to the video frame sequence, and the step S200 is executed again. Figure 11 The step S200 is executed again.

[0160] When the target video frame is not a key frame, a video frame after the target video frame needs to be determined as a new target video frame and added to the video frame sequence. However, since the upper limit of the number of video frames included in the video frame sequence is the first number, another video frame needs to be deleted from the video frame sequence. Since the video frames with higher rankings have poorer timeliness, the disclosure can delete the video frame with the highest ranking in the video frame sequence from the video frame sequence. After the above processing is performed, the disclosure can continue to identify whether the new target video frame is a key frame.

[0161] Figure 11 is a structural schematic diagram of a video frame type identification device according to an exemplary embodiment, as Figure 11 indicated, the video frame type identification device can include:

[0162] The sequence obtaining unit 100 is configured to perform obtaining a video frame sequence, wherein the video frame sequence includes a target video frame and a plurality of previous video frames arranged before the target video frame in the video frame sequence.

[0163] The similarity calculating unit 200 is configured to perform calculating an image similarity between each of the plurality of previous video frames and the target video frame.

[0164] The key frame determining unit 300 is configured to perform determining whether the target video frame is a key frame according to the calculated image similarity.

[0165] The video frame type identification device provided by the embodiment of the disclosure can obtain a video frame sequence, calculate an image similarity between each of a plurality of previous video frames and a target video frame, and determine whether the target video frame is a key frame according to the calculated image similarity. Since the video frame sequence includes a target video frame and a plurality of previous video frames arranged before the target video frame in the video frame sequence, the disclosure not only determines whether the target video frame is a key frame according to one previous video frame adjacent to the target video frame, but also determines whether the target video frame is a key frame according to a plurality of previous video frames before the target video frame. Since the image similarity between the plurality of previous video frames before the target video frame and the target video frame is considered, the disclosure can effectively improve the accuracy of key frame identification.

[0166] Optionally, the similarity calculation unit 200 is configured to determine, according to the overall image similarity and / or the local image similarity, the image similarity of each of the plurality of previous video frames with the target video frame, respectively, wherein the overall image similarity is the similarity of the overall image of each of the plurality of previous video frames with the overall image of the target video frame, respectively; and the local image similarity is the similarity of the local image in each of the plurality of previous video frames with the local image in the target video frame at a same position, respectively.

[0167] Optionally, the similarity calculation unit 200 is further configured to obtain the local image similarity by:

[0168] For each of the plurality of previous video frames, two first granularity image blocks respectively located in the video frame and the target video frame at a same position are determined as an image block group, and the similarity of the two first granularity image blocks in each image block group is calculated as the local image similarity, respectively, wherein the first granularity image block is an image block obtained by dividing each of the video frames in the video frame sequence according to a first preset image division manner, respectively.

[0169] Optionally, the similarity calculation unit 200 is further configured to obtain the local image similarity by:

[0170] The first granularity image blocks located in the same video frame are divided into a plurality of image block groups, two matching image block groups respectively located in one of the previous video frames and the target video frame are determined as an image block group, and the similarity of the two image block groups in each image block group is calculated as the local image similarity, respectively, wherein the first granularity image block is an image block obtained by dividing each of the video frames in the video frame sequence according to a first preset image division manner, respectively, and the matching image block groups are at a same position in the video frames.

[0171] Optionally, the similarity calculation unit 200 is further configured to obtain the local image similarity by:

[0172] For each of the plurality of previous video frames, two second granularity image blocks respectively located in the video frame and the target video frame at a same position are determined as an image block group, and the similarity of the two second granularity image blocks in each image block group is calculated as the local image similarity, respectively, wherein the second granularity image block is an image block obtained by dividing each of the video frames in the video frame sequence according to a second preset image division manner, and the size of the first granularity image block is different from the size of the second granularity image block.

[0173] Optionally, the similarity calculation unit 200 is configured to perform: for each of the previous video frames, performing weighted average calculation on the overall image similarity and each of the local image similarities calculated according to the previous video frame, and determining the calculation result as the image similarity between the previous video frame and the target video frame.

[0174] Optionally, the key frame determination unit 300 is configured to perform: determining the maximum value among the calculated image similarities; and determining whether the target video frame is a key frame according to the maximum value.

[0175] Optionally, the key frame determination unit 300 is configured to perform: multiplying the calculated image similarity by the ordering weight of the corresponding previous video frame, and determining whether the target video frame is a key frame according to the calculation result.

[0176] Optionally, the upper limit of the number of video frames included in the video frame sequence is a first number, Figure 12 The apparatus can further include a first processing unit and / or a second processing unit,

[0177] The first processing unit is configured to perform: when the target video frame is a key frame, deleting each of the previous video frames of the target video frame from the video frame sequence, adding at least one subsequent video frame after the target video frame to the video frame sequence, and determining one of the at least one subsequent video frame as a new target video frame, and triggering the similarity calculation unit 200.

[0178] The second processing unit is configured to perform: when the target video frame is not a key frame, determining whether the number of video frames included in the video frame sequence is the first number, if yes, deleting the video frame with the first order in the video frame sequence from the video frame sequence, determining one video frame after the target video frame as a new target video frame and adding the new target video frame to the video frame sequence, and triggering the similarity calculation unit 200.

[0179] The video frame type identification apparatus includes a processor and a memory, and the sequence obtaining unit, the similarity calculation unit, and the key frame determination unit are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0180] The processor includes a core, and the core retrieves the corresponding program units from the memory. The core can be one or more, and the video frame type is identified by adjusting the core parameters.

[0181] The embodiments of the present disclosure provide a storage medium having a program stored thereon, and the program is executed by a processor to realize the video frame type identification method.

[0182] This disclosure provides a processor for running a program, wherein the program executes a video frame type recognition method during runtime.

[0183] like Figure 1 As shown, an exemplary embodiment of this disclosure provides an electronic device 70, which includes at least one processor 701, at least one memory 702 connected to the processor 701, and a bus 703; wherein the processor 701 and the memory 702 communicate with each other through the bus 703; the processor 701 is used to call program instructions in the memory to execute the aforementioned video frame type recognition method. The device described herein may be a server, PC, PAD, mobile phone, etc.

[0184] This disclosure also provides a computer program product, which, when executed on a data processing device, is adapted to perform a program that initializes at least one step included in the video frame type identification method described above.

[0185] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes ​ A device that provides the functions specified in one or more boxes.

[0186] In a typical configuration, the device includes one or more processors (CPUs), memory, and a bus. The device may also include input / output interfaces, network interfaces, etc.

[0187] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM, and memory includes at least one memory chip. Memory is an example of computer-readable media.

[0188] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0189] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0190] Those skilled in the art will appreciate that embodiments of the present disclosure can be provided as a method, system or computer program product. Accordingly, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0191] The above is only an embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art can make various modifications and changes to the present disclosure. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present disclosure shall be included in the scope of claims of the present disclosure.

Claims

1. A method for identifying video frame types, characterized in that, include: Obtain a video frame sequence, wherein the video frame sequence includes a target video frame and a plurality of preceding video frames ordered in the video frame sequence before the target video frame; Calculate the image similarity between each of the plurality of preceding video frames and the target video frame; The target video frame is determined as a keyframe based on the calculated image similarity. The step of calculating the image similarity between each of the plurality of preceding video frames and the target video frame includes: Based on overall image similarity and local image similarity, the image similarity between each video frame in the plurality of preceding video frames and the target video frame is determined, wherein the overall image similarity is the similarity between the overall image of each video frame in the plurality of preceding video frames and the overall image of the target video frame; the local image similarity is the similarity between a local image in each video frame in the plurality of preceding video frames and a local image in the target video frame that is in the same position. The process of obtaining the local image similarity includes: For each video frame in the preceding video frame: two first-granularity image blocks that are located in the same position in the video frame and the target video frame are determined as an image block group. The similarity between the two first-granularity image blocks in each image block group is calculated and used as the local image similarity. The first-granularity image block is an image block obtained by segmenting each video frame in the video frame sequence according to a first preset image segmentation method. or, The process of obtaining the local image similarity includes: Each first-granularity image block located within the same video frame is divided into multiple image block groups. Two matching image block groups located in a preceding video frame and a target video frame are determined as an image block group. The similarity between the two image block groups in each image block group is calculated and used as the local image similarity. The first-granularity image block is obtained by segmenting each video frame in the video frame sequence according to a first preset image segmentation method. The matching image block groups are in the same position in the video frame.

2. The method according to claim 1, characterized in that, The step of determining the image similarity between each of the plurality of preceding video frames and the target video frame based on overall image similarity and local image similarity includes: For each preceding video frame: a weighted average is calculated on the overall image similarity and each local image similarity obtained from the preceding video frame, and the calculation result is determined as the image similarity between the preceding video frame and the target video frame.

3. The method according to claim 1, characterized in that, Determining whether the target video frame is a keyframe based on the calculated image similarity includes: Determine the maximum value among the calculated image similarities; The target video frame is determined as a keyframe based on the maximum value.

4. The method according to claim 1, characterized in that, Determining whether the target video frame is a keyframe based on the calculated image similarity includes: The calculated image similarity is multiplied by the ranking weight of its corresponding preceding video frame, and the result is used to determine whether the target video frame is a keyframe.

5. The method according to claim 1, characterized in that, The maximum number of video frames contained in the video frame sequence is a first number, and the method further includes: When the target video frame is a keyframe, each preceding video frame of the target video frame is deleted from the video frame sequence, at least one subsequent video frame after the target video frame is added to the video frame sequence, and one of the video frames in the at least one subsequent video frame is determined as a new target video frame. Then, the process of calculating the image similarity between each of the plurality of preceding video frames and the target video frame is returned. And / or, when the target video frame is not a keyframe, determine whether the number of video frames in the video frame sequence is the first number. If so, delete the first video frame in the video frame sequence from the video frame sequence, determine the next video frame after the target video frame as the new target video frame and add it to the video frame sequence, and return to the step of calculating the image similarity between each of the plurality of preceding video frames and the target video frame.

6. A video frame type recognition device, characterized in that, include: A sequence acquisition unit is configured to acquire a video frame sequence, wherein the video frame sequence includes a target video frame and a plurality of preceding video frames ordered in the video frame sequence before the target video frame; The similarity calculation unit is configured to perform the calculation of the image similarity between each of the plurality of preceding video frames and the target video frame; The keyframe determination unit is configured to determine whether the target video frame is a keyframe based on the calculated image similarity. The similarity calculation unit calculates the image similarity between each of the plurality of preceding video frames and the target video frame, including: Based on overall image similarity and local image similarity, the image similarity between each video frame in the plurality of preceding video frames and the target video frame is determined, wherein the overall image similarity is the similarity between the overall image of each video frame in the plurality of preceding video frames and the overall image of the target video frame; the local image similarity is the similarity between a local image in each video frame in the plurality of preceding video frames and a local image in the target video frame that is in the same position. The process of obtaining the local image similarity includes: For each video frame in the preceding video frame: two first-granularity image blocks that are located in the same position in the video frame and the target video frame are determined as an image block group. The similarity between the two first-granularity image blocks in each image block group is calculated and used as the local image similarity. The first-granularity image block is an image block obtained by segmenting each video frame in the video frame sequence according to a first preset image segmentation method. or, The process of obtaining the local image similarity includes: Each first-granularity image block located within the same video frame is divided into multiple image block groups. Two matching image block groups located in a preceding video frame and a target video frame are determined as an image block group. The similarity between the two image block groups in each image block group is calculated and used as the local image similarity. The first-granularity image block is obtained by segmenting each video frame in the video frame sequence according to a first preset image segmentation method. The matching image block groups are in the same position in the video frame.

7. An electronic device, characterized in that, The electronic device includes at least one processor, and at least one memory and bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the video frame type recognition method according to any one of claims 1 to 5.

8. A storage medium having a program stored thereon, which, when executed by a processor, implements the video frame type identification method according to any one of claims 1 to 5.

Images