Video key frame extraction method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a method, device, electronic device and storage medium for extracting video key frames, the method comprising: obtaining each initial video frame; determining the target reference parameters of each initial video frame; the target reference parameters include spatial reference parameters and / or temporal reference parameters; filtering each initial video frame based on the target reference parameters to determine at least one video frame to be processed; clustering the video frames to be processed to determine at least one target key frame. Through the technical solution of this embodiment, the technical effects of filtering low-quality frames, removing redundant frames, and improving the rate and quality of key frame extraction are achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to image processing technology, and in particular to a video key frame extraction method, device, electronic device and storage medium. Background Art

[0002] Key frame extraction of videos is of great significance in information extraction, semantic conversion, retrieval, classification, etc. For example, when conducting content review of video files, key frame extraction technology can be used to improve review efficiency. The original content review was to review the entire video, that is, all video frames. Now only the extracted key frames need to be reviewed, which can improve the review efficiency while reducing the interference of invalid video frames on the review results.

[0003] In addition to video content review, the application scenarios of key frame extraction technology can also include video retrieval, video structuring, video summarization, etc. Traditional key frame extraction methods are based on global features and local features. However, this method of selecting key frames has large errors, high consumption of computing resources, redundancy of extracted key frames, and low quality of key frame images. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, electronic device and storage medium for extracting key frames of a video, so as to achieve the technical effects of filtering low-quality frames, removing redundant frames, and improving the rate and quality of key frame extraction.

[0005] In a first aspect, an embodiment of the present invention provides a method for extracting a video key frame, the method comprising:

[0006] Obtain each initial video frame;

[0007] Determining target reference parameters of each of the initial video frames; the target reference parameters include spatial reference parameters and / or temporal reference parameters;

[0008] Filtering the initial video frames based on the target reference parameter to determine at least one video frame to be processed;

[0009] Clustering is performed on the video frames to be processed to determine at least one target key frame.

[0010] In a second aspect, an embodiment of the present invention further provides a video key frame extraction device, the device comprising:

[0011] An initial video frame acquisition module, used to acquire each initial video frame;

[0012] A target reference parameter determination module, used to determine the target reference parameters of each initial video frame; the target reference parameters include spatial reference parameters and / or temporal reference parameters;

[0013] a module for determining video frames to be processed, configured to filter the initial video frames based on the target reference parameter to determine at least one video frame to be processed;

[0014] The target key frame determination module is used to perform clustering processing on the video frames to be processed to determine at least one target key frame.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] a storage device for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the video key frame extraction method as described in any one of the embodiments of the present invention.

[0019] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video key frame extraction method as described in any one of the embodiments of the present invention.

[0020] The technical solution of the embodiment of the present invention obtains each initial video frame, determines the target reference parameters of each initial video frame, and then filters each initial video frame based on the target reference parameters to determine at least one video frame to be processed, so as to filter out low-quality video frames and converted video frames from the initial video frames, cluster the video frames to be processed, and determine at least one target key frame, thereby solving the problems of large errors in extracting key frames, large consumption of computing resources, and low image quality of key frames, and achieving the technical effects of filtering low-quality frames, removing redundant frames, and improving the rate and quality of key frame extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of a flow chart of a video key frame extraction method provided in the first embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a flow chart of a video key frame extraction method provided in Embodiment 2 of the present invention;

[0024] Figure 3 A schematic diagram of a flow chart of a video key frame extraction method provided in Embodiment 3 of the present invention;

[0025] Figure 4 A schematic diagram of a flow chart of a video key frame extraction method provided in Embodiment 4 of the present invention;

[0026] Figure 5 A flowchart of a video key frame extraction method provided in Embodiment 5 of the present invention;

[0027] Figure 6 A schematic diagram of the structure of a video key frame extraction device provided by Embodiment 6 of the present invention;

[0028] Figure 7 This is a structural schematic diagram of an electronic device provided in Embodiment 7 of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0030] Embodiment 1

[0031] Figure 1 A flow chart of a video key frame extraction method provided in Embodiment 1 of the present invention is provided. This embodiment is applicable to the case of extracting high-quality key frames from a large number of video frames of a video. The method can be performed by a video key frame extraction device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device. Optionally, the electronic device can be a mobile terminal, etc.

[0032] like Figure 1 The method of this embodiment specifically includes the following steps:

[0033] S110: Acquire each initial video frame.

[0034] The initial video frames may be all video frames in the video from which key frames are to be extracted.

[0035] Specifically, when it is determined to extract key frames for a certain video, the video can be decomposed to obtain all video frames. For example, OpenCV (a cross-platform computer vision and machine learning software library) can be used to decompose the video to obtain each frame of the video.

[0036] S120: Determine target reference parameters of each initial video frame.

[0037] The target reference parameter includes a spatial reference parameter and / or a temporal reference parameter. The spatial reference parameter may be a reference parameter extracted from the video frame content. The temporal reference parameter may be a reference parameter extracted by comparing the previous and next video frames.

[0038] Specifically, the target reference parameters of each initial video frame can be determined from the time dimension and / or space dimension according to the requirements of video key frame extraction. For example, target reference parameters such as brightness parameter, contrast parameter, and balance parameter of each initial video frame can be extracted from the space dimension, and target reference parameters such as lens edge change rate of each initial video frame can also be extracted from the time dimension.

[0039] S130: Filter each initial video frame based on the target reference parameter to determine at least one video frame to be processed.

[0040] The video frames to be processed may be the initial video frames remaining after filtering, which are used for subsequent clustering processing.

[0041] Specifically, it is possible to determine whether each initial video frame is a low-quality video frame according to the spatial reference parameters of each initial video frame, for example, whether it is too dark, blurred, uneven, etc. It is also possible to determine whether each initial video frame is a converted video frame according to the temporal reference parameters of each initial video frame. Furthermore, the low-quality video frames and the converted video frames can be filtered according to the target reference parameters to determine at least one video frame to be processed.

[0042] Optionally, the video frame to be processed may be determined according to a target reference parameter threshold: each initial video frame is filtered according to the target reference parameter threshold and the target reference parameter to determine at least one video frame to be processed.

[0043] The target reference parameter threshold may be a threshold set according to requirements related to video key frames, and is used to determine whether each initial video frame is suitable for clustering processing to extract key frames.

[0044] Specifically, if the target reference parameter of the initial video frame meets the target reference parameter threshold, the initial video frame can be used as the video frame to be processed. If the target reference parameter of the initial video frame does not meet the target reference parameter threshold, the initial video frame can be filtered out.

[0045] S140: Perform clustering processing on the video frames to be processed to determine at least one target key frame.

[0046] The target key frame may be a to-be-processed video frame used to represent the video.

[0047] Specifically, the features of the video frame to be processed may be extracted, for example, global features such as color histogram features of the video frame to be processed, features extracted using VGG (Visual Geometry Group, computer vision group), etc. Then, clustering processing is performed using a clustering algorithm based on the extracted features to obtain at least one target key frame.

[0048] Exemplarily, the features of each video frame to be processed can be clustered by K-means (K-means clustering algorithm), and clustering can generate K classes. The frame closest to the cluster center in each class is used as the target key frame corresponding to the class.

[0049] The technical solution of the embodiment of the present invention obtains each initial video frame, determines the target reference parameters of each initial video frame, and then filters each initial video frame based on the target reference parameters to determine at least one video frame to be processed, so as to filter out low-quality video frames and converted video frames from the initial video frames, cluster the video frames to be processed, and determine at least one target key frame, thereby solving the problems of large errors in extracting key frames, large consumption of computing resources, and low image quality of key frames, and achieving the technical effects of filtering low-quality frames, removing redundant frames, and improving the rate and quality of key frame extraction.

[0050] Embodiment 2

[0051] Figure 2 This is a flow chart of a video key frame extraction method provided by Embodiment 2 of the present invention. This embodiment is based on the above embodiments. For the determination method of the spatial reference parameter, please refer to the technical solution of this embodiment. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0052] like Figure 2 As shown, the method of this embodiment specifically includes the following steps:

[0053] S210: Acquire each initial video frame.

[0054] S220: Determine spatial reference parameters of each initial video frame.

[0055] Specifically, a spatial reference parameter can be calculated based on each initial video frame to determine whether each initial video frame is of low quality based on the spatial reference parameters of each initial video frame. For example, whether each initial video frame is too dark can be determined based on a brightness parameter in the spatial reference parameters of each initial video frame; whether each initial video frame is blurred can be determined based on a contrast parameter in the spatial reference parameters of each initial video frame; whether each initial video frame is uneven can be determined based on a balance parameter in the spatial reference parameters of each initial video frame, etc.

[0056] Optionally, the brightness parameter of the initial video frame may be determined by the following steps:

[0057] Step 1: Determine each channel component corresponding to each pixel point in the current initial video frame.

[0058] The initial video frame is composed of three channels of RGB (red, green, blue), and the channel components can be the red channel component, the green channel component and the blue channel component of the pixel point in the initial video frame.

[0059] Specifically, the current initial video frame may be processed by channel division, for example, by decomposing the current initial video frame using OpenCV, etc. For each channel frame obtained by decomposition, each channel component corresponding to each pixel point may be determined.

[0060] Step 2: Determine the brightness parameters of the current initial video frame according to the width, height and each channel component of the current initial video frame.

[0061] The brightness parameter may be a spatial reference parameter used to measure the brightness of a video frame.

[0062] Specifically, a corresponding weight may be assigned to each channel, and the brightness map of the current initial video frame may be determined according to each channel component and each weight. Furthermore, the pixels in the brightness map may be normalized and averaged to obtain the brightness parameter of the current initial video frame.

[0063] Optionally, the brightness parameter of the current initial video frame may be determined according to the following formula:

[0064]

[0065] Where y1 represents the brightness parameter of the current initial video frame, i represents the i-th row of the current initial video frame, j represents the j-th column of the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, and I r Represents the red channel image of the current initial video frame, ω r Represents the weight corresponding to the red channel image, I g Represents the green channel image of the current initial video frame, ω gIndicates the weight corresponding to the green channel image, I b Represents the blue channel image of the current initial video frame, ω b Indicates the weight corresponding to the blue channel image.

[0066] Specifically, weighted sums are performed on each channel component of each pixel of the current initial video frame, and the sum values ​​of all pixels are accumulated to obtain the brightness parameter and value of the current initial video frame. Then, the brightness parameter and value are divided by 255 to obtain a normalized value, and then divided by m×n to obtain an average value after averaging, so that the obtained value is used as the brightness parameter of the current initial video frame.

[0067] It should be noted that the normalization process by dividing by 255 and the averaging process by dividing by m×n are both for the purpose of facilitating the comparison of brightness parameters of different initial video frames. In practical applications, the normalization process and the averaging process can be used together, one or the other, or neither. The specific method can be adjusted according to the actual application requirements.

[0068] It should also be noted that the range of the brightness parameter is [0,1]. The larger the parameter value, the brighter the current video frame, and the smaller the parameter value, the darker the current video frame. The weight corresponding to each channel can be set according to actual needs, for example: r =0.2,ω g =0.7,ω b =0.1, etc.

[0069] Optionally, the contrast parameter of the initial video frame may be determined by the following steps:

[0070] Step 1: Convert the current initial video frame into a current grayscale video frame, and determine the grayscale value corresponding to each pixel in the current grayscale video frame.

[0071] The grayscale video frame may be a grayscale image obtained by representing the initial video frame in grayscale. The grayscale value may be a value corresponding to each pixel in the grayscale video frame.

[0072] Specifically, the current initial video frame may be subjected to grayscale processing to obtain a current grayscale video frame, and then the grayscale value of each pixel in the current grayscale video frame may be obtained.

[0073] Step 2: Determine the gradient value corresponding to each pixel point according to each grayscale value.

[0074] The gradient value includes the transverse gradient and the longitudinal gradient.

[0075] Specifically, according to the calculation method of the gradient value, the horizontal gradient of each pixel point and the vertical gradient of each pixel point can be calculated.

[0076] Step 3: Determine the contrast parameter according to the width, height, grayscale value and gradient value of the current initial video frame.

[0077] The contrast parameter may be a spatial reference parameter used to measure whether a video frame is blurred or not.

[0078] Specifically, the contrast map of the current initial video frame can be determined according to the gradient value and grayscale value of each pixel point. Furthermore, the contrast parameters of the current initial video frame can be obtained by taking an average value of the pixels in the contrast map.

[0079] Optionally, the contrast parameter of the current initial video frame may be determined according to the following formula:

[0080]

[0081] or,

[0082]

[0083] Where y2 represents the contrast parameter of the current initial video frame, i represents the i-th row of the current initial video frame, j represents the j-th column of the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, and I gray Represents the grayscale image of the current initial video frame, Δx ij Represents the horizontal gradient of the pixel point in the i-th row and j-th column of the current initial video frame, Δy ij Represents the vertical gradient of the pixel at the i-th row and j-th column of the current initial video frame.

[0084] Specifically, the grayscale value of each pixel of the current initial video frame can be multiplied by the horizontal gradient and the vertical gradient of the pixel to obtain the horizontal gradient grayscale value Δx ij I gray (i, j) and the vertical gradient gray value Δy ij I gray (i, j). Furthermore, the horizontal gradient grayscale value and the vertical gradient grayscale value can be squared, summed, and then squared to determine the contrast parameter of the current pixel. Alternatively, the horizontal gradient grayscale value and the vertical gradient grayscale value can be absolute values, summed, and then determined to determine the contrast parameter of the current pixel. Both of the above methods are to make the contrast parameter non-negative. After determining the contrast parameter corresponding to each pixel, all contrast parameters can be summed and divided by the size of the current initial video frame (height × width) to obtain the average value to obtain the contrast parameter of the current initial video frame.

[0085] It should be noted that the averaging process is performed by dividing by m×n in order to facilitate comparison of contrast parameters of different initial video frames. In practical applications, whether the averaging process is used or not can be adjusted according to actual application requirements.

[0086] It should also be noted that the larger the parameter value of the contrast parameter is, the clearer the current video frame is, and the smaller the parameter value is, the blurrier the current video frame is.

[0087] Optionally, the equalization parameter of the initial video frame may be determined by the following steps:

[0088] Step 1: Perform histogram equalization processing on the current initial video frame to determine the grayscale equalization value corresponding to each grayscale level.

[0089] The grayscale level may be different levels determined after histogram equalization processing, and the grayscale balance value may be a frequency value corresponding to each grayscale level.

[0090] Specifically, performing histogram equalization processing on the current initial video frame can enhance the contrast of the current initial video frame to make the current initial video frame approximately uniform. Furthermore, the frequency corresponding to each gray level can be determined as the gray balance value.

[0091] Step 2: Determine the balance parameter according to the gray balance value and the preset balance ratio.

[0092] The preset equalization ratio may be a preset percentage used to sum the grayscale equalization values. The equalization parameter may be a spatial reference parameter used to measure whether the video frame is uniform.

[0093] Specifically, the grayscale balance values ​​of each gray level are arranged from large to small, and the sum of the grayscale balance values ​​of the preset balance ratios arranged in front is calculated as the balance parameter.

[0094] Optionally, the equalization parameter of the current initial video frame is determined according to the following formula:

[0095] y3=top per (norm_hist(I gray ))

[0096] Among them, y3 represents the balance parameter of the current initial video frame, I gray Represents the grayscale image of the current initial video frame, norm_hist(I gray ) represents the grayscale balance value, per represents the preset balance ratio, top per Indicates the value of the preset equalization ratio.

[0097] Specifically, the grayscale balance value of the preset balance ratio before statistics is used as the balance parameter. The smaller the balance parameter, the more uniform the current initial video frame is, and the larger the balance parameter, the more non-uniform the current initial video frame is.

[0098] It should be noted that the preset balance ratio can be a value set according to demand, such as 5%. The larger the sum of grayscale balance values ​​within the preset balance ratio, the more concentrated the grayscale of the current initial video frame, which means the grayscale value of the current initial video frame is more uneven.

[0099] S230: Filter each initial video frame based on the spatial reference parameter to determine at least one video frame to be processed.

[0100] S240: Perform clustering processing on the video frames to be processed to determine at least one target key frame.

[0101] The technical solution of the embodiment of the present invention obtains each initial video frame, determines the spatial reference parameters of each initial video frame, and then filters each initial video frame based on the spatial reference parameters to determine at least one video frame to be processed, so as to filter out low-quality video frames from the initial video frames and improve the quality of the video frames. The video frames to be processed are clustered to determine at least one target key frame, which solves the problems of large errors in extracting key frames, large consumption of computing resources and low image quality of key frames, and achieves the technical effects of filtering low-quality frames, removing redundant frames, and improving the rate and quality of key frame extraction.

[0102] Embodiment 3

[0103] Figure 3 This is a flow chart of a method for extracting a video key frame provided by Embodiment 3 of the present invention. This embodiment is based on the above embodiments. For the determination method of the time reference parameter, please refer to the technical solution of this embodiment. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0104] like Figure 3 As shown, the method of this embodiment specifically includes the following steps:

[0105] S310: Acquire each initial video frame.

[0106] S320: Determine a time reference parameter of each initial video frame.

[0107] Specifically, a temporal reference parameter may be calculated according to each initial video frame, so as to determine whether each initial video frame is a conversion video frame according to a spatial reference parameter of each initial video frame.

[0108] It should be noted that each video may be synthesized by combining different lenses, and there may be some special effects during lens conversion. These video frames belong to conversion video frames and can be removed by comparing the previous and next video frames.

[0109] Optionally, the time reference parameters of the initial video frame can be determined in the following two ways:

[0110] Method 1: determine a first norm based on a current initial video frame and a previous initial video frame adjacent to the current initial video frame; determine a second norm based on the current initial video frame and a subsequent initial video frame adjacent to the current initial video frame; determine a lens edge change rate of the current initial video frame based on the width, height, first norm and second norm of the current initial video frame.

[0111] Specifically, the initial video frame during the shot conversion period can be determined by the frame difference. Optionally, since the frame difference during the shot conversion period is relatively large, a preset change rate ratio can be set to determine the range of the initial video frame to be filtered, thereby narrowing the range for subsequent filtering.

[0112] Optionally, the lens edge change rate of the current initial video frame is determined according to the following formula:

[0113]

[0114] Wherein, y4 represents the lens edge change rate of the current initial video frame, i represents the sequence number of the current initial video frame, I(i) represents the i-th initial video frame, i.e., the current initial video frame, I(i-1) represents the i-1-th initial video frame, i.e., the previous initial video frame adjacent to the current initial video frame, I(i+1) represents the i+1-th initial video frame, i.e., the next initial video frame adjacent to the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, norm(I(i)-I(i-1)) represents the first norm, norm(I(i)-I(i+1)) represents the second norm, per represents the preset change rate ratio, top per Indicates the value of the current preset rate of change ratio.

[0115] Specifically, the difference between the current initial video frame and the previous initial video frame adjacent to the current initial video frame can be calculated, and the difference between the current initial video frame and the next initial video frame adjacent to the current initial video frame can be calculated to obtain the difference between the two frames. Then, the norm of the two frame differences is calculated to determine the frame difference. The average frame difference between the current initial video frame and the previous and next initial video frames can be determined by dividing by 2, and the average frame difference of each pixel of the current initial video frame can be determined by dividing by m×n, and the average frame difference is used as the lens edge change rate of the current initial video frame.

[0116] Method 2: for the current initial video frame and the previous initial video frame adjacent to the current initial video frame, determine the current edge value and the neighbor edge value; determine the fade-in edge change rate based on the neighbor edge value and the current dilated edge value corresponding to the current edge value; determine the fade-out edge change rate based on the neighbor edge value, the current edge value and the neighbor dilated edge value corresponding to the neighbor edge value; determine the lens edge change rate of the current initial video frame based on the fade-in edge change rate and the fade-out edge change rate.

[0117] Specifically, special effects such as fade-in, fade-out, dissolve, etc. may appear during the shot conversion. In this case, edge extraction can be performed on the current initial video frame and the previous initial video frame adjacent to the current initial video frame to determine whether special effects such as fade-in, fade-out, etc. exist.

[0118] For example, for fade-in special effects, during the lens transition, the video frame usually changes from blurry to clear. At this time, the edge image of the current initial video frame can be compared to perform morphological expansion processing, that is, blurring processing. The sum of the current expanded edge value after blurring processing and the neighbor edge value of the previous initial video frame is "AND" operated, and then the sum of the neighbor edge values ​​of the previous initial video frame is divided by point. Finally, the fade-in edge change rate can be obtained by subtracting from 1. Similarly, the calculation principle of the fade-out edge change rate can be known.

[0119] Optionally, the fade-in edge change rate of the current initial video frame is determined according to the following formula:

[0120]

[0121] Among them, y in Indicates the fade-in edge change rate of the current initial video frame, i indicates the sequence number of the current initial video frame, I edge (i-1) represents the neighbor edge value, I edge_dilate (i) represents the current dilation edge value.

[0122] The fade-out edge change rate of the current initial video frame is determined according to the following formula:

[0123]

[0124] Among them, y out Indicates the fade-out edge change rate of the current initial video frame, i indicates the sequence number of the previous initial video frame, I edge (i) represents the current edge value, I edge (i-1) represents the neighbor edge value, I edge_dilate (i-1) represents the neighbor dilation edge value.

[0125] The lens edge change rate of the current initial video frame is determined according to the following formula:

[0126] y5=max(y in ,y out )

[0127] Among them, y5 represents the lens edge change rate of the current initial video frame, y in Indicates the fade-in edge change rate of the current initial video frame, y out Indicates the fade-out edge change rate of the current initial video frame.

[0128] Specifically, taking the fade-in effect as an example, the current initial video frame and the previous video frame adjacent to the current initial video frame have basically the same content, but the fade-in effect will produce edge blur. Therefore, the edge image of the current initial video frame after expansion has a large overlap with the edge image of the previous initial video frame. The value y after subtracting 1.0 in It will be relatively small, which is convenient for setting the threshold value to judge the conversion of video frames. The fade-out effect is similar to the above content, so I will not repeat it.

[0129] It should be noted that the overlap rate is subtracted from 1.0 only for the convenience of calculation and comparison. If the subtraction is not performed, it is also in line with the principle of lens edge change rate calculation. in ,y out ) can comprehensively determine whether the current initial video frame has a fade-in or fade-out effect. If they are used as time reference parameters for subsequent filtering, they also meet the requirements of this solution.

[0130] S330: Filter each initial video frame based on the time reference parameter to determine at least one video frame to be processed.

[0131] S340: Perform clustering processing on the video frames to be processed to determine at least one target key frame.

[0132] The technical solution of the embodiment of the present invention obtains each initial video frame, determines the time reference parameter of each initial video frame, and then filters each initial video frame based on the time reference parameter to determine at least one video frame to be processed, so as to filter out the video frames at the time of lens conversion from the initial video frames, improve the quality of the video frames, cluster the video frames to be processed, and determine at least one target key frame, thereby solving the problems of large errors in extracting key frames, large consumption of computing resources, and low image quality of key frames, and achieving the technical effect of filtering video frames at the time of lens conversion, removing redundant frames, and improving the rate and quality of key frame extraction.

[0133] Embodiment 4

[0134] Figure 4This is a flow chart of a method for extracting video key frames provided by Embodiment 4 of the present invention. Based on the above embodiments, this embodiment can refer to the technical solution of this embodiment for the way of clustering the video frames to be processed. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0135] like Figure 4 As shown, the method of this embodiment specifically includes the following steps:

[0136] S410: Acquire each initial video frame.

[0137] S420: Determine target reference parameters for each initial video frame.

[0138] S430: Filter each initial video frame based on the target reference parameter to determine at least one video frame to be processed.

[0139] S440: Determine the target shot quantity according to the sequence number of each to-be-processed video frame and a preset step length.

[0140] The sequence number may be a sequence number assigned to each initial video frame in sequence. The preset step size may be a preset sequence number interval for determining the number of shots. The target number of shots may be the number of shots included in the current video.

[0141] Specifically, since low-quality video frames and converted video frames are filtered based on the target reference parameters, the serial numbers of adjacent video frames to be processed may be discontinuous. If the serial number interval of adjacent video frames to be processed is too large, it can be considered that the two video frames to be processed belong to two shots. That is, the target shot number can be determined by comparing the difference between the serial numbers of adjacent video frames to be processed with the preset step size.

[0142] Optionally, the number of shots can be determined in the following manner: if the difference between the serial numbers of two adjacent video frames to be processed is greater than or equal to a preset step length, the target number of shots is increased by one; if the difference between the serial numbers of two adjacent video frames to be processed is less than a preset step length, the target number of shots remains unchanged.

[0143] For example, the sequence numbers of the initial video frames are 1-200, and the sequence numbers of the video frames to be processed obtained by filtering are 1-25, 30-36, 38-45, 55-70, 72-75, 88-97, 105-120, 130-150, 155-169, 175-199. If the preset step size is 3, 1-25, 30-45, 55-75, 88-97, 105-120, 130-150, 155-169, 175-199 can be used as video frames in different target shots, that is, the number of target shots is 8.

[0144] S450: Perform clustering processing based on the number of target shots and each to-be-processed video frame, and use the to-be-processed video frame corresponding to each cluster center as a target key frame.

[0145] Specifically, the number of target shots is taken as the number of cluster centers, and each video frame to be processed is clustered based on the number. After each cluster center is determined, the video frame closest to the cluster center in each class is determined as the video frame to be processed corresponding to the cluster center, and then, these video frames to be processed are used as target key frames.

[0146] It should be noted that, usually, a 1-second video includes 24 initial video frames. A 2-minute video can obtain 7 to 8 seconds of target key frames through filtering and clustering.

[0147] The technical solution of the embodiment of the present invention obtains each initial video frame, determines the target reference parameter of each initial video frame, and then filters each initial video frame based on the target reference parameter to determine at least one video frame to be processed, so as to filter out low-quality video frames and converted video frames from the initial video frames, and determines the target number of shots according to the sequence number of each video frame to be processed and a preset step size to determine the number of clustering centers, and clustering is performed based on the target number of shots and each video frame to be processed, and the video frame to be processed corresponding to each clustering center is used as the target key frame to solve the problems of large errors in extracting key frames, large consumption of computing resources and low image quality of key frames, and achieves the technical effect of filtering low-quality frames, removing redundant frames, and improving the rate and quality of key frame extraction.

[0148] Embodiment 5

[0149] As an optional implementation scheme of the above embodiments, Figure 5 The flowchart of a method for extracting key frames of a video provided by Embodiment 5 of the present invention is shown in FIG.

[0150] like Figure 5 As shown, the method of this embodiment specifically includes the following steps:

[0151] 1. Input video frame (initial video frame).

[0152] Specifically, the video data of the key frame to be selected is obtained, and each video frame (usually about 24 frames per second) is obtained using an open source tool (eg, OpenCV).

[0153] 2. Filter low-quality video frames in the video frames.

[0154] Among them, the low-quality video frame can be a blurred video frame, a video frame with poor definition, a video frame with uneven grayscale, etc.

[0155] Specifically, darker video frames can be filtered, that is, filtering is performed according to the brightness of the video frames. After determining the brightness value (brightness parameter) of the video frame, the video frames with brightness values ​​less than or equal to T1 are filtered out through the brightness threshold T1 (such as: 0.08, etc.), and the video frames with brightness values ​​greater than T1 are retained for subsequent judgment. Blurred video frames can also be filtered, that is, filtered out according to the blurriness of the video frame. For blurred video frame judgment, the color information can be weakened, and grayscale images can be used for judgment. From the perspective of the image content of the video frame, if the sharpness of the image content is higher, it means that the contrast of the image content details is higher and the video frame is clearer. Conversely, the contrast of the image content details of the blurred video frame is low, that is, the sharpness is low. After determining the blur value (contrast parameter) of the video frame, the video frames with blur values ​​less than or equal to T2 are filtered out through the blur threshold T2 (such as: 0.1, etc.), and the video frames with blur values ​​greater than T2 are retained for subsequent judgment. Video frames with uneven grayscale levels can also be filtered. After determining the equalization value (equalization parameter) of the video frame, the video frames with equalization values ​​greater than or equal to T3 are filtered out through the equalization threshold T3 (such as 0.8), and the video frames with equalization values ​​less than T3 are retained for subsequent judgment.

[0156] 3. Filter the converted video frames in the video frames.

[0157] The converted video frame may be a video frame with special effects during the lens conversion.

[0158] Specifically, the video conversion frames can be filtered according to the comparison of the previous and next video frames. If the lens edge change rate is determined by the inter-frame difference value, some conversion video frames can be filtered out by the hyperparameter threshold T4. This is because the inter-frame difference value of the conversion video frames of the general lens is large. If the lens edge change rate is determined by the fade-in edge change rate and the fade-out edge change rate, the conversion video frames related to the lens conversion special effects can be filtered out by the hyperparameter threshold T5.

[0159] 4. Filter the redundant video frames in the video frames and determine the target key frames.

[0160] The redundant video frame may be a video frame having a high similarity with the target key frame.

[0161] Specifically, after filtering low-quality video frames and converted video frames from the spatial dimension and the temporal dimension, high-quality video frames can be obtained. Furthermore, redundant video frames can be filtered from high-quality video frames by feature clustering. The difference in the serial numbers of two adjacent video frames can be calculated, that is, the serial number Index of the latter video frame minus the serial number Index of the previous video frame. If the difference is greater than the preset step size Step, the two video frames before and after can be judged as the two shots before and after. The reason is that: low-quality video frames and converted video frames can be filtered out by steps 2 and 3, and the remaining video frames are spaced at this time. Two video frames with too large an interval can be regarded as video frames of different shots. After determining the number of target shots, the number of target shots is used as the number of cluster centers, and the Kmeans clustering method is used to cluster according to the underlying features of the image content of the video frame, such as: color histogram, LBP (Local Binary Pattern, local binary pattern) feature, HOG (Histogram of Oriented Gradient, directional gradient histogram) feature, etc. Finally, after determining each class center, the video frame closest to the class center in each shot can be determined as the target key frame.

[0162] The technical solution of the embodiment of the present invention, by inputting video frames, filtering low-quality video frames and converting video frames in the video frames, and then filtering redundant video frames in the video frames, determines the target key frames, thereby solving the problems of large errors in extracting key frames, large consumption of computing resources, and low image quality of key frames, and achieves the technical effect of filtering low-quality frames, removing redundant frames, and improving the rate and quality of key frame extraction.

[0163] Embodiment 6

[0164] Figure 6 This is a structural schematic diagram of a video key frame extraction device provided in Embodiment 6 of the present invention, the device includes: an initial video frame acquisition module 610, a target reference parameter determination module 620, a to-be-processed video frame determination module 630 and a target key frame determination module 640.

[0165] Among them, the initial video frame acquisition module 610 is used to acquire each initial video frame; the target reference parameter determination module 620 is used to determine the target reference parameters of each initial video frame; the target reference parameters include spatial reference parameters and / or time reference parameters; the to-be-processed video frame determination module 630 is used to filter the to-be-processed video frames based on the target reference parameters to determine at least one to-be-processed video frame; the target key frame determination module 640 is used to cluster the to-be-processed video frames to determine at least one target key frame.

[0166] Optionally, the spatial reference parameters include brightness parameters, and the target reference parameter determination module 620 is also used to determine the channel components corresponding to each pixel point in the current initial video frame; determine the brightness parameters of the current initial video frame based on the width, height and the channel components of the current initial video frame.

[0167] Optionally, the target reference parameter determination module 620 is further configured to determine the brightness parameter of the current initial video frame according to the following formula:

[0168]

[0169] Wherein, y1 represents the brightness parameter of the current initial video frame, i represents the i-th row of the current initial video frame, j represents the j-th column of the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, and I r represents the red channel image of the current initial video frame, ω r Represents the weight corresponding to the red channel image, I g Represents the green channel image of the current initial video frame, ω g Represents the weight corresponding to the green channel image, I b Represents the blue channel image of the current initial video frame, ω b Indicates the weight corresponding to the blue channel image.

[0170] Optionally, the spatial reference parameters include contrast parameters, and the target reference parameter determination module 620 is also used to convert the current initial video frame into a current grayscale video frame, and determine the grayscale value corresponding to each pixel in the current grayscale video frame; determine the gradient value corresponding to each pixel according to the grayscale values; the gradient value includes a horizontal gradient and a vertical gradient; determine the contrast parameter according to the width, height, grayscale value and gradient value of the current initial video frame.

[0171] Optionally, the target reference parameter determination module 620 is further configured to determine a contrast parameter of the current initial video frame according to the following formula:

[0172]

[0173] or,

[0174]

[0175] Wherein, y2 represents the contrast parameter of the current initial video frame, i represents the i-th row of the current initial video frame, j represents the j-th column of the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, and Igray Represents the grayscale image of the current initial video frame, Δx ij represents the horizontal gradient of the pixel point at the i-th row and j-th column of the current initial video frame, Δy ij Represents the vertical gradient of the pixel point in the i-th row and j-th column of the current initial video frame.

[0176] Optionally, the spatial reference parameters include a balance parameter, and the target reference parameter determination module 620 is also used to perform histogram equalization on the current initial video frame to determine the grayscale balance value corresponding to each grayscale level; and determine the balance parameter based on the grayscale balance value and a preset balance ratio.

[0177] Optionally, the target reference parameter determination module 620 is further configured to determine the equalization parameter of the current initial video frame according to the following formula:

[0178] y3=top per (norm_hist(I gray ))

[0179] Wherein, y3 represents the balance parameter of the current initial video frame, I gray Represents the grayscale image of the current initial video frame, norm_hist(I gray ) represents the grayscale balance value, per represents the preset balance ratio, top per Indicates the value of the preset balancing ratio mentioned above.

[0180] Optionally, the time reference parameters include a lens edge change rate, and the target reference parameter determination module 620 is further used to determine a first norm based on a current initial video frame and a previous initial video frame adjacent to the current initial video frame; determine a second norm based on the current initial video frame and a subsequent initial video frame adjacent to the current initial video frame; and determine the lens edge change rate of the current initial video frame based on the width and height of the current initial video frame, the first norm and the second norm.

[0181] Optionally, the target reference parameter determination module 620 is further configured to determine the shot edge change rate of the current initial video frame according to the following formula:

[0182]

[0183] Wherein, y4 represents the lens edge change rate of the current initial video frame, i represents the sequence number of the current initial video frame, I(i) represents the i-th initial video frame, that is, the current initial video frame, I(i-1) represents the i-1-th initial video frame, that is, the previous initial video frame adjacent to the current initial video frame, I(i+1) represents the i+1-th initial video frame, that is, the next initial video frame adjacent to the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, norm(I(i)-I(i-1)) represents the first norm, norm(I(i)-I(i+1)) represents the second norm, per represents the preset change rate ratio, top per Indicates the value of the preset change rate ratio mentioned above.

[0184] Optionally, the time reference parameter includes a lens edge change rate, and the target reference parameter determination module 620 is further used to determine a current edge value and a neighbor edge value for a current initial video frame and a previous initial video frame adjacent to the current initial video frame; determine a fade-in edge change rate based on the neighbor edge value and a current dilation edge value corresponding to the current edge value; determine a fade-out edge change rate based on the neighbor edge value, the current edge value and a neighbor dilation edge value corresponding to the neighbor edge value; and determine a lens edge change rate of the current initial video frame based on the fade-in edge change rate and the fade-out edge change rate.

[0185] Optionally, the target reference parameter determination module 620 is further configured to determine the fade-in edge change rate of the current initial video frame according to the following formula:

[0186]

[0187] Among them, y in represents the fade-in edge change rate of the current initial video frame, i represents the sequence number of the current initial video frame, and I edge (i-1) represents the neighbor edge value, I edge_dilate (i) represents the current dilation edge value;

[0188] The fade-out edge change rate of the current initial video frame is determined according to the following formula:

[0189]

[0190] Among them, y out represents the fade-out edge change rate of the current initial video frame, i represents the sequence number of the current initial video frame, and I edge (i) represents the current edge value, I edge (i-1) represents the neighbor edge value, I edge_dilate(i-1) represents the neighbor dilation edge value;

[0191] The lens edge change rate of the current initial video frame is determined according to the following formula:

[0192] y5=max(y in ,y out )

[0193] Wherein, y5 represents the lens edge change rate of the current initial video frame, y in Indicates the fade-in edge change rate of the current initial video frame, y out Indicates the fade-out edge change rate of the current initial video frame.

[0194] Optionally, the target key frame determination module 640 is further used to determine the target shot number according to the sequence number of each video frame to be processed and a preset step size; perform clustering processing based on the target shot number and the video frames to be processed, and use the video frames to be processed corresponding to each cluster center as target key frames.

[0195] Optionally, the target key frame determination module 640 is further configured to increase the target shot number by one if the sequence number difference between two adjacent video frames to be processed is greater than or equal to a preset step length; and to keep the target shot number unchanged if the sequence number difference between two adjacent video frames to be processed is less than a preset step length.

[0196] The technical solution of the embodiment of the present invention obtains each initial video frame, determines the target reference parameters of each initial video frame, and then filters each initial video frame based on the target reference parameters to determine at least one video frame to be processed, so as to filter out low-quality video frames and converted video frames from the initial video frames, cluster the video frames to be processed, and determine at least one target key frame, thereby solving the problems of large errors in extracting key frames, large consumption of computing resources, and low image quality of key frames, and achieving the technical effects of filtering low-quality frames, removing redundant frames, and improving the rate and quality of key frame extraction.

[0197] The video key frame extraction device provided in the embodiment of the present invention can execute the video key frame extraction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0198] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0199] Embodiment 7

[0200] Figure 7 This is a structural schematic diagram of an electronic device provided in Embodiment 7 of the present invention. Figure 7 A block diagram of an exemplary electronic device 70 suitable for use in implementing exemplary embodiments of the present invention is shown. Figure 7 The electronic device 70 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0201] like Figure 7 As shown, the electronic device 70 is in the form of a general-purpose computing device. The components of the electronic device 70 may include, but are not limited to: one or more processors or processing units 701, a system memory 702, and a bus 703 connecting different system components (including the system memory 702 and the processing unit 701).

[0202] Bus 703 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0203] The electronic device 70 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 70, including volatile and non-volatile media, removable and non-removable media.

[0204] The system memory 702 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 704 and / or cache memory 705. The electronic device 70 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 706 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 703 via one or more data medium interfaces. The system memory 702 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0205] A program / utility 708 having a set (at least one) of program modules 707 may be stored, for example, in system memory 702, such program modules 707 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 707 generally perform the functions and / or methods of the embodiments described herein.

[0206] The electronic device 70 may also communicate with one or more external devices 709 (e.g., keyboards, pointing devices, displays 710, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 70, and / or communicate with any device that enables the electronic device 70 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 711. Furthermore, the electronic device 70 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 712. As shown, the network adapter 712 communicates with other modules of the electronic device 70 via the bus 703. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 70, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0207] The processing unit 701 executes various functional applications and data processing by running the programs stored in the system memory 702, such as implementing the video key frame extraction method provided in the embodiment of the present invention.

[0208] Embodiment 8

[0209] Embodiment 8 of the present invention further provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute a video key frame extraction method when executed by a computer processor, the method comprising:

[0210] Obtain each initial video frame;

[0211] Determining target reference parameters of each of the initial video frames; the target reference parameters include spatial reference parameters and / or temporal reference parameters;

[0212] Filtering the initial video frames based on the target reference parameter to determine at least one video frame to be processed;

[0213] Clustering is performed on the video frames to be processed to determine at least one target key frame.

[0214] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0215] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0216] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0217] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0218] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A video key frame extraction method, characterized in that: include: Obtain each initial video frame; Determining target reference parameters of each of the initial video frames; The target reference parameters include spatial reference parameters and / or temporal reference parameters; Filtering the initial video frames based on the target reference parameter to determine at least one video frame to be processed; Performing clustering processing on the video frames to be processed to determine at least one target key frame; The spatial reference parameter includes a balance parameter, and the step of determining the target reference parameter of each initial video frame includes: Performing histogram equalization processing on the current initial video frame to determine the grayscale equalization value corresponding to each grayscale level; Determining a balance parameter according to the gray balance value and a preset balance ratio; The preset balancing ratio is a preset percentage, which is used to sum the gray balance values. Specifically, the gray balance values ​​of each gray level are arranged from large to small, and the sum of the gray balance values ​​of the preset balancing ratios arranged in front is calculated as the balancing degree parameter. The clustering process of the video frames to be processed includes: extracting features of the video frames to be processed by using a computer vision group, and performing clustering process by using a clustering algorithm according to the extracted features; The step of clustering the video frames to be processed to determine at least one target key frame includes: Determine the target number of shots according to the sequence number of each video frame to be processed and the preset step length; wherein the sequence number is a sequence number assigned to each initial video frame in sequence, the preset step length is a preset sequence number interval for determining the number of shots, and the target number of shots is the number of shots contained in the current video; Performing clustering processing based on the target shot quantity and the video frames to be processed, and taking the video frames to be processed corresponding to the cluster centers as target key frames; The step of determining the target number of shots according to the sequence number of each video frame to be processed and a preset step size includes: If the difference between the serial numbers of two adjacent video frames to be processed is greater than or equal to the preset step length, the number of target shots is increased by one; If the difference between the serial numbers of two adjacent video frames to be processed is less than the preset step size, the number of target shots remains unchanged.

2. The method according to claim 1, characterized in that The spatial reference parameter includes a brightness parameter, and the determining of the target reference parameter of each initial video frame includes: Determine each channel component corresponding to each pixel point in the current initial video frame; The brightness parameter of the current initial video frame is determined according to the width, height and each channel component of the current initial video frame.

3. The method according to claim 2, characterized in that Determining the brightness parameter of the current initial video frame according to the width, height and each channel component of the current initial video frame includes: The brightness parameter of the current initial video frame is determined according to the following formula: Wherein, y1 represents the brightness parameter of the current initial video frame, i represents the i-th row of the current initial video frame, j represents the j-th column of the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, and I r represents the red channel image of the current initial video frame, ω r Represents the weight corresponding to the red channel image, I g Represents the green channel image of the current initial video frame, ω g Represents the weight corresponding to the green channel image, I b Represents the blue channel image of the current initial video frame, ω b Indicates the weight corresponding to the blue channel image.

4. The method according to claim 1, characterized in that: The spatial reference parameter includes a contrast parameter, and the determining of the target reference parameter of each initial video frame includes: Converting the current initial video frame into a current grayscale video frame, and determining the grayscale value corresponding to each pixel in the current grayscale video frame; Determine the gradient value corresponding to each pixel point according to each grayscale value; the gradient value includes a horizontal gradient and a vertical gradient; The contrast parameter is determined according to the width, height, the grayscale value and the gradient value of the current initial video frame.

5. The method according to claim 4, characterized in that Determining the contrast parameter according to the width, height, grayscale value and gradient value of the current initial video frame includes: The contrast parameter of the current initial video frame is determined according to the following formula: or, Wherein, y2 represents the contrast parameter of the current initial video frame, i represents the i-th row of the current initial video frame, j represents the j-th column of the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, and I gray Represents the grayscale image of the current initial video frame, Δx ij represents the horizontal gradient of the pixel point at the i-th row and j-th column of the current initial video frame, Δy ij Represents the vertical gradient of the pixel point in the i-th row and j-th column of the current initial video frame.

6. The method according to claim 1, characterized in that The step of determining a balance parameter according to the gray balance value and a preset balance ratio includes: The equalization parameter of the current initial video frame is determined according to the following formula: y3=top per (norm_hist(I gray )) Wherein, y3 represents the balance parameter of the current initial video frame, I gray Represents the grayscale image of the current initial video frame, norm_hist(I gray ) represents the gray balance value, per represents the preset balance ratio, top per Indicates the value of the preset balancing ratio mentioned above.

7. The method according to claim 1, characterized in that The time reference parameter includes a lens edge change rate, and the determining of the target reference parameter of each initial video frame includes: Determine a first norm according to a current initial video frame and a previous initial video frame adjacent to the current initial video frame; Determine a second norm according to the current initial video frame and a subsequent initial video frame adjacent to the current initial video frame; The shot edge change rate of the current initial video frame is determined according to the width, the height, the first norm and the second norm of the current initial video frame.

8. The method according to claim 7, characterized in that Determining the lens edge change rate of the current initial video frame according to the width, height, the first norm and the second norm of the current initial video frame includes: The lens edge change rate of the current initial video frame is determined according to the following formula: Wherein, y4 represents the lens edge change rate of the current initial video frame, i represents the sequence number of the current initial video frame, I(i) represents the i-th initial video frame, that is, the current initial video frame, I(i-1) represents the i-1-th initial video frame, that is, the previous initial video frame adjacent to the current initial video frame, I(i+1) represents the i+1-th initial video frame, that is, the next initial video frame adjacent to the current initial video frame, m represents the height of the current initial video frame, n represents the width of the current initial video frame, norm(I(i)-I(i-1)) represents the first norm, norm(I(i)-I(i+1)) represents the second norm, per represents the preset change rate ratio, top per Indicates the value of the preset change rate ratio mentioned above.

9. The method according to claim 1, characterized in that: The time reference parameter includes a lens edge change rate, and the determining of the target reference parameter of each initial video frame includes: Determine a current edge value and a neighbor edge value for a current initial video frame and a previous initial video frame adjacent to the current initial video frame; Determining a fade-in edge change rate based on the neighbor edge value and a current dilation edge value corresponding to the current edge value; Determine a fade-out edge change rate based on the neighbor edge value, the current edge value, and a neighbor dilation edge value corresponding to the neighbor edge value; Based on the fade-in edge change rate and the fade-out edge change rate, a shot edge change rate of the current initial video frame is determined.

10. The method according to claim 9, characterized in that The step of determining the fade-in edge change rate based on the neighbor edge value and the current dilation edge value corresponding to the current edge value includes: The fade-in edge change rate of the current initial video frame is determined according to the following formula: Among them, y in represents the fade-in edge change rate of the current initial video frame, i represents the sequence number of the current initial video frame, and I edge (i-1) represents the neighbor edge value, I edge_dilate (i) represents the current dilation edge value; Accordingly, determining the fade-out edge change rate based on the neighbor edge value, the current edge value and the neighbor dilation edge value corresponding to the neighbor edge value includes: The fade-out edge change rate of the current initial video frame is determined according to the following formula: Among them, y out represents the fade-out edge change rate of the current initial video frame, i represents the sequence number of the current initial video frame, and I edge (i) represents the current edge value, I edge (i-1) represents the neighbor edge value, I edge_dilate (i-1) represents the neighbor dilation edge value; Correspondingly, determining the lens edge change rate of the current initial video frame based on the fade-in edge change rate and the fade-out edge change rate includes: The lens edge change rate of the current initial video frame is determined according to the following formula: y5=max(y in ,and out ) Wherein, y5 represents the lens edge change rate of the current initial video frame, y in Indicates the fade-in edge change rate of the current initial video frame, y out Indicates the fade-out edge change rate of the current initial video frame.

11. A video key frame extraction device, characterized in that: include: An initial video frame acquisition module, used to acquire each initial video frame; A target reference parameter determination module, used to determine the target reference parameters of each initial video frame; The target reference parameters include spatial reference parameters and / or temporal reference parameters; a module for determining video frames to be processed, configured to filter the initial video frames based on the target reference parameter to determine at least one video frame to be processed; A target key frame determination module, used to perform clustering processing on the video frames to be processed to determine at least one target key frame; The spatial reference parameters include a balance parameter, and the target reference parameter determination module is further used to perform histogram equalization processing on the current initial video frame to determine the grayscale balance value corresponding to each grayscale level; and determine the balance parameter according to the grayscale balance value and a preset balance ratio; The preset balancing ratio is a preset percentage, which is used to sum the gray balance values. Specifically, the gray balance values ​​of each gray level are arranged from large to small, and the sum of the gray balance values ​​of the preset balancing ratios arranged in front is calculated as the balancing degree parameter. The clustering process of the video frames to be processed includes: extracting features of the video frames to be processed by using a computer vision group, and performing clustering process by using a clustering algorithm according to the extracted features; The target key frame determination module is further used to determine the target number of shots according to the sequence number of each to-be-processed video frame and the preset step length; wherein the sequence number is a sequence number assigned to each initial video frame in sequence, the preset step length is a preset sequence number interval for determining the number of shots, and the target number of shots is the number of shots contained in the current video; clustering is performed based on the target number of shots and the to-be-processed video frames, and the to-be-processed video frames corresponding to each cluster center are used as target key frames; Wherein, the target key frame determination module is also used to increase the target shot number by one if the sequence number difference between two adjacent video frames to be processed is greater than or equal to a preset step length; if the sequence number difference between two adjacent video frames to be processed is less than the preset step length, the target shot number remains unchanged.

Images