Key frame extraction method, system and device, medium and program product
By performing low-pass filtering and clearness index calculation on video frames and automatically filtering keyframes, the problems of insufficient clarity and low efficiency of keyframes in the existing technology are solved, and efficient keyframe extraction and modeling effects are improved.
Patent Information
- Application Number
- CN202411088358.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-15
AI Technical Summary
When using technologies such as neural radiation fields for three-dimensional reconstruction or rendering, the acquired keyframe clarity is insufficient, resulting in the subsequent modeling effect being affected, and manual screening is required, which is inefficient.
By acquiring multiple image frames of the video, performing low-pass filtering, calculating pixel intensity changes and pixel intensity changes after filtering, determining clarity indicators, automatically filtering out keyframes, and reducing manual intervention.
It improves the efficiency of keyframe extraction, reduces the dependence of manual screening, obtains more accurate image clarity evaluation, and improves the effect of subsequent modeling.
Smart Images

Figure CN118967635A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a key frame extraction method, system, device, medium and program product. Background Art
[0002] When the existing technology uses technologies such as neural radiation fields for three-dimensional reconstruction or rendering, it is necessary to obtain materials of the target object. The obtained material is a video that includes the target object. After the key frames of the video are extracted, the key frames are used as input data for reconstruction or rendering. Currently, mobile phones are usually used to take photos and videos. Due to the influence of real-scene shooting and camera performance, the quality of some photos is poor, which will affect the subsequent modeling effect. Key frames are usually obtained by extracting images from videos. Key frames with insufficient clarity will need to be manually screened again, resulting in reduced efficiency. Summary of the invention
[0003] The embodiments of the present invention provide a key frame extraction method, system, device, medium and program product to solve the technical problem that the existing key frame extraction is inefficient and relies on manual labor.
[0004] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:
[0005] In a first aspect, an embodiment of the present invention provides a key frame extraction method, comprising:
[0006] Acquire multiple image frames of a video;
[0007] Performing low-pass filtering on each of the image frames to obtain a filtered image frame;
[0008] determining a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame;
[0009] Determining a clarity index of the image frame according to a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame;
[0010] The multiple image frames are filtered according to the clarity index to obtain key frames in the multiple image frames.
[0011] Optionally, performing low-pass filtering on each of the image frames to obtain a filtered image frame includes:
[0012] Performing horizontal filtering on the image frame using a horizontal filter to obtain a first filtered image frame;
[0013] The image frame is filtered in a vertical direction using a vertical filter to obtain a second filtered image frame.
[0014] Optionally, determining the pixel intensity change of the image frame and the pixel intensity change of the filtered image frame includes:
[0015] Determine a pixel intensity difference of the image frame in the horizontal direction according to pixel values of two adjacent columns of pixels in the horizontal direction of the image frame;
[0016] Determining a pixel intensity difference of the image frame in the vertical direction according to pixel values of two adjacent rows of pixels in the vertical direction of the image frame;
[0017] Determining a pixel intensity difference in the horizontal direction of the filtered image frame according to pixel values of two adjacent columns of pixels in the horizontal direction of the filtered image frame;
[0018] The pixel intensity difference in the vertical direction of the filtered image frame is determined according to the pixel values of two adjacent rows of pixels in the vertical direction of the filtered image frame.
[0019] Optionally, determining the clarity index of the image frame according to the pixel intensity change of the image frame and the pixel intensity change of the filtered image frame includes:
[0020] Determine the pixel intensity difference before and after filtering in the horizontal direction according to the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference of the filtered image frame in the horizontal direction;
[0021] Determine the pixel intensity difference before and after filtering in the vertical direction according to the pixel intensity difference of the image frame in the vertical direction and the pixel intensity difference of the filtered image frame in the vertical direction;
[0022] Determine a pixel intensity difference in the horizontal direction according to the pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction;
[0023] Determine a vertical direction pixel intensity difference according to the vertical direction pixel intensity difference of the image frame and the vertical direction pixel intensity difference before and after filtering;
[0024] The larger value of the horizontal pixel intensity difference and the vertical pixel intensity difference is used as the clarity index of the image frame.
[0025] Optionally, the pixel intensity difference in the horizontal direction is determined according to the pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction;
[0026] Determining a vertical pixel intensity difference according to the pixel intensity difference of the image frame in the vertical direction and the pixel intensity difference before and after filtering in the vertical direction includes:
[0027] Normalizing the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference before and after filtering in the horizontal direction to obtain the pixel intensity difference in the horizontal direction;
[0028] The pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction are normalized to obtain the pixel intensity difference in the vertical direction.
[0029] Optionally, filtering the multiple image frames according to the clarity index to obtain key frames in the multiple image frames includes:
[0030] Divide the video into a plurality of segments, each of which includes a plurality of image frames;
[0031] The image frame with the highest definition index in each segment is selected as the key frame of the segment.
[0032] In a second aspect, an embodiment of the present invention provides a key frame extraction system, including:
[0033] An acquisition module, used for acquiring multiple image frames of a video;
[0034] A filtering module, used for performing low-pass filtering on each of the image frames to obtain a filtered image frame;
[0035] a change module, configured to determine a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame;
[0036] An index module, used for determining a clarity index of the image frame according to a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame;
[0037] A filtering module is used to filter the multiple image frames according to the clarity index to obtain key frames in the multiple image frames.
[0038] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the key frame extraction method described in the first aspect are implemented.
[0039] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the key frame extraction method as described in the first aspect above are implemented.
[0040] In a fifth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the key frame extraction method as described in the first aspect above.
[0041] In the present application, firstly, an image frame of a video is acquired; a filtered image frame is obtained by low-pass filtering the image frame; a pixel intensity change of the image frame and a pixel intensity change of the filtered image frame are determined; then, according to the pixel intensity change of the image frame and the pixel intensity change of the filtered image frame, a clarity index of the image frame is determined; finally, the image frame is filtered according to the clarity index to obtain a key frame. The present application can reduce manual intervention in image clarity evaluation and obtain an accurate image clarity evaluation index by low-pass filtering the image frame and calculating the pixel intensity change of the filtered image frame and the pixel intensity change of the unprocessed image frame, comparing the two intensity changes and determining the clarity index of the image frame, so as to automatically screen key frames, improve the extraction efficiency of key frames, and reduce manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0043] Figure 1 is a flow chart of a key frame extraction method provided by an embodiment of the present invention;
[0044] Figure 2 is a detailed flow chart of a key frame extraction method provided by an embodiment of the present invention;
[0045] Figure 3 is a schematic diagram of converting a grayscale image provided by an embodiment of the present invention;
[0046] Figure 4 is a schematic diagram of a filtering result provided by an embodiment of the present invention;
[0047] Figure 5 is a schematic diagram of clarity comparison provided by an embodiment of the present invention;
[0048] Figure 6 It is a structural schematic diagram of a key frame extraction system provided by an embodiment of the present invention;
[0049] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] Using a low-pass filter to filter a clear image will make it blurry, but using the same low-pass filter to filter a blurred image will make it difficult for the naked eye to perceive changes in the image; the image is blurred because high-frequency details are lost, so the embodiment of the present application uses a low-pass filter to filter high-frequency details to reproduce this phenomenon. Based on the above phenomenon, the clarity of the image can be measured by blurring the image and comparing the changes between adjacent elements before and after the low-pass filtering step. Therefore, in order to efficiently extract key frames, the embodiment of the present application uses a low-pass filter to filter the same image frame, and compares the changes between pixels as the basis for the clarity of the image; a large change between the original image and the blurred image means that the image is clear, and a small change between the original image and the blurred image means that the original image is blurred.
[0052] Please refer to Figure 1 and Figure 2 , an embodiment of the present invention provides a key frame extraction method, comprising:
[0053] Step 11: Get the image frame of the video;
[0054] Step 12: performing low-pass filtering on the image frame to obtain a filtered image frame;
[0055] Step 13: Determine a pixel intensity change of the image frame and a pixel intensity change of the filtered image frame;
[0056] Optionally, the intensity can be represented by pixel value. If it is a grayscale image, it can be represented by grayscale value. If the grayscale image is converted into floating-point data, it can be represented by floating-point data corresponding to the pixel. The change in pixel intensity is represented by the change in pixel value, grayscale value or floating-point data corresponding to the pixel.
[0057] Step 14: determining a clarity index of the image frame according to a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame;
[0058] Step 15: Filter the image frame according to the clarity index to obtain a key frame.
[0059] In the present application, firstly, an image frame of a video is acquired; a filtered image frame is obtained by low-pass filtering the image frame; a pixel intensity change of the image frame and a pixel intensity change of the filtered image frame are determined; then, according to the pixel intensity change of the image frame and the pixel intensity change of the filtered image frame, a clarity index of the image frame is determined; finally, the image frame is filtered according to the clarity index to obtain a key frame. In an embodiment of the present application, the image frame can be low-pass filtered and the pixel intensity change of the filtered image frame and the pixel intensity change of the unprocessed image frame can be calculated, and the two intensity changes can be compared and the clarity index of the image frame can be determined, thereby reducing manual intervention in image clarity evaluation and obtaining an accurate image clarity evaluation index, so as to automatically screen key frames, improve the extraction efficiency of key frames, and reduce manual intervention.
[0060] Optionally, the image frame is a grayscale image;
[0061] The step of performing low-pass filtering on the image frame to obtain a filtered image frame includes:
[0062] The grayscale image is low-pass filtered using a plurality of filters to obtain a plurality of filtered image frames.
[0063] Optionally, in an embodiment of the present application, if the image frame of the video is a color image, such as an RGB image, then when the RGB image is low-pass filtered, each channel (R, G, B) needs to be processed independently, and then the processed channels are merged to form the final output image.
[0064] Optionally, in the embodiments of this application, please refer to Figure 3 , the color image frame can be converted into a grayscale image. The grayscale image is a single-channel image and only a single channel needs to be processed. The computing resources and time required to process the grayscale image are significantly reduced.
[0065] Optionally, in an embodiment of the present application, the grayscale image is further converted into a floating-point format and recorded as an image. Each pixel in the grayscale image corresponds to a floating-point data, and the floating-point data can be used to represent the intensity of this pixel. The integer value of each pixel (for example, 0 to 255) is converted into a floating-point value (for example, 0.0 to 1.0). Floating-point numbers provide higher precision and facilitate standardization operations.
[0066] Optionally, in the embodiment of the present application, multiple filters can be selected to perform low-pass filtering on the same image to generate multiple filtered image frames. After processing and comparing the multiple filtered image frames, the clarity index of the image frame is determined. Using filters with different parameters can detect and enhance features in different directions in the image, making the final result more comprehensive.
[0067] Optionally, different filters may be in different directions, for example, a horizontal filter, a vertical filter, and a diagonal filter. In the embodiment of the present application, a horizontal filter F is selected. hor and a vertical filter to F ver Perform filtering.
[0068] Optionally, the number of filters is 2;
[0069] The step of using a plurality of filters to low-pass filter the grayscale image to obtain a plurality of filtered image frames comprises:
[0070] Performing horizontal filtering on the image frame using a horizontal filter to obtain a first filtered image frame;
[0071] The image frame is filtered in a vertical direction using a vertical filter to obtain a second filtered image frame.
[0072] Optionally, most of the prior art uses IIR bandpass filters. The IIR bandpass filter allows frequency signals within a range to pass through, and weakens the signals outside the range. The filtering range is determined by designing the parameters of the IIR digital filter. However, the IIR bandpass filter needs to design IIR filter parameters according to different types, and different parameters need to be designed according to different blur types of the image (such as fog and out-of-focus are different blur types). It is difficult to use, the parameters are not easy to adjust, and the processing still relies on manual intervention, and it is also easy to produce inaccurate evaluation results.
[0073] Therefore, the horizontal filter F is used in the embodiment of the present application. hor and a vertical filter to F ver Filtering is performed. Horizontal filtering is usually sensitive to changes in the horizontal direction, while vertical filtering is sensitive to changes in the vertical direction. There is no need to design low-pass filter parameters. Based on the intensity comparison before and after filtering, a reliable clarity reference value can be directly given.
[0074] F hor =|1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9|;
[0075] F ver =F ver T ;
[0076] The above filter is used to filter the image horizontally and vertically, and the image clarity is obtained by analyzing and counting the differences between adjacent pixels. The filter of the embodiment of the present application is simple in design, fixed in parameters, clear in function, only filters high-frequency signals, is easy to use, and has strong universality.
[0077] Further, optionally, the determining the pixel intensity change of the image frame and the pixel intensity change of the filtered image frame includes:
[0078] Determine a pixel intensity difference of the image frame in the horizontal direction according to pixel values of two adjacent columns of pixels in the horizontal direction of the image frame;
[0079] Determining a pixel intensity difference of the image frame in the vertical direction according to pixel values of two adjacent columns of pixels in the vertical direction of the image frame;
[0080] Determining a pixel intensity difference in the horizontal direction of the filtered image frame according to pixel values of two adjacent columns of pixels in the horizontal direction of the filtered image frame;
[0081] The pixel intensity difference in the vertical direction of the filtered image frame is determined according to the pixel values of two adjacent columns of pixels in the vertical direction of the filtered image frame.
[0082] Optionally, the image frames are subjected to a horizontal filter F hor and a vertical filter to F ver Filtering is performed, and the result is b hor , b ver , if the image frame is converted to grayscale before filtering, the image after filtering is still grayscale, but the high-frequency details are filtered out and become blurred. The pixels in the image are represented by (i, j), m and n represent the width and height of the image, and the pixel intensity difference is calculated using the following formulas:
[0083] The vertical intensity difference after filtering:
[0084] D_b ver (i,j)=|b ver (i,j)-b ver (i-1,j)|,i={x∈N1≤x≤m},j={y∈N0≤y≤m};
[0085] Horizontal intensity difference after filtering:
[0086] D_b hor (i,j)=|b hor (i,j)-b hor (i,j-1)|,i={x∈N0≤x≤m},j={y∈N1≤y≤m};
[0087] Vertical intensity difference before filtering:
[0088] D_g ver (i,j)=|g ver (i,j)-g ver (i-1,j)|,i={x∈N1≤x≤m},j={y∈N0≤y≤m};
[0089] Horizontal intensity difference before filtering:
[0090] D_g hor (i,j)=|g hor (i,j)-g hor (i, j-1)|, i={x∈N0≤x≤m}, j={y∈N1≤y≤m}.
[0091] Optional, please refer to Figure 4 , Figure 4 The left part of represents the part of the image frame after it is converted into a grayscale image and horizontally filtered. Figure 4 The right part of represents the horizontal intensity difference after filtering, D_b hor (ij) indicates the difference is calculated in the horizontal direction, b hor (ij) represents the pixel intensity of the next column; b hor (ij-1) represents the pixel intensity of the previous column; where, Figure 4 The absolute value of column B minus column A is equal to the value of column N, and so on. The difference of each column is calculated to get Figure 4 The horizontal intensity difference table on the right side of the image frame can be further used to obtain four intensity difference tables, which are the horizontal and vertical differences before filtering, i.e., the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference of the image frame in the vertical direction; and the horizontal and vertical differences after filtering, i.e., the pixel intensity difference of the filtered image frame in the horizontal direction and the pixel intensity difference of the filtered image frame in the vertical direction.
[0092] Optionally, in an embodiment of the present application, the intensity difference calculation in the horizontal and vertical directions before and after filtering can more comprehensively capture the changing characteristics of the image frame before and after processing.
[0093] Optionally, determining the clarity index of the image frame according to the pixel intensity change of the image frame and the pixel intensity change of the filtered image frame includes:
[0094] Determine the pixel intensity difference before and after filtering in the horizontal direction according to the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference of the filtered image frame in the horizontal direction;
[0095] Determine the pixel intensity difference before and after filtering in the vertical direction according to the pixel intensity difference of the image frame in the vertical direction and the pixel intensity difference of the filtered image frame in the vertical direction;
[0096] Determine a pixel intensity difference in the horizontal direction according to the pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction;
[0097] Determine a vertical direction pixel intensity difference according to the pixel intensity difference of the image frame in the vertical direction and the pixel intensity difference before and after filtering in the vertical direction;
[0098] The larger value of the horizontal pixel intensity difference and the vertical pixel intensity difference is used as the clarity index of the image frame.
[0099] Further, optionally, in the embodiment of the present application, the difference before and after filtering in the horizontal and vertical directions is calculated by the following formula:
[0100] The vertical difference table before filtering minus the vertical difference table after filtering is used to obtain the pixel intensity difference before and after vertical filtering. If the data is less than 0, this point is set to 0:
[0101] D ver =max(0,D_g ver (ij)-D_b ver (ij));
[0102] The horizontal difference table before filtering minus the vertical difference table after filtering can be used to obtain the pixel intensity difference before and after filtering in the horizontal direction. If the data is less than 0, this point is set to 0:
[0103] D hor =max(0,D_g hor (ij)-D_b hor (ij));
[0104] Determining the horizontal pixel intensity difference and the vertical pixel intensity difference includes:
[0105] First, calculate the sum of the intensity differences (the intensity of a pixel can also be its corresponding grayscale value):
[0106] The vertical difference table of the image frame, the sum of all data in the table:
[0107] s_D_g ver =∑D_g ver (i,j);
[0108] Horizontal difference table of image frames, sum of all data in the table:
[0109] s_D_g hor =∑D_g hor (i,j)
[0110] Sum all the pixel intensity differences before and after vertical filtering:
[0111] s_D ver =∑D ver (i,j);
[0112] Sum all the horizontal pixel intensity differences:
[0113] s_D hor =∑D hor (i,j).
[0114] Optionally, through the above steps, four difference values s_D_g are obtained ver is the vertical difference sum before filtering (the vertical difference sum of the image frame); s_D_g hor is the horizontal difference sum before filtering (the horizontal difference sum of the image frame); s_D ver is the sum of the vertical differences before and after filtering (the sum of all pixel intensity differences before and after vertical filtering); s_D hor It is the sum of the differences between the horizontal differences before and after filtering (the sum of all the pixel intensity differences before and after filtering in the horizontal direction). Since these data are not convenient for measuring clarity, the above four data are converted into two differences in the horizontal and vertical directions, and normalized to between 0 and 1.
[0115] Optionally, the pixel intensity difference in the horizontal direction is determined according to the pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction;
[0116] Determining a vertical pixel intensity difference according to the pixel intensity difference of the image frame in the vertical direction and the pixel intensity difference before and after filtering in the vertical direction includes:
[0117] Normalizing the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference before and after filtering in the horizontal direction to obtain the pixel intensity difference in the horizontal direction;
[0118] The pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction are normalized to obtain the pixel intensity difference in the vertical direction.
[0119] Specifically:
[0120]
[0121] Optionally, using the larger value of the horizontal pixel intensity difference and the vertical pixel intensity difference as the clarity index of the image frame includes:
[0122] Select the maximum blur between vertical and horizontal differences = max(R ver -R hor ), the larger the blur, the clearer the image.
[0123] Please refer to Figure 5 The clarity of the left image is 0.462720, and the clarity of the right image is 0.651024. It can be seen that through the clarity selection method in the embodiment of the present application, a clearer image frame can be selected.
[0124] Optionally, filtering the image frame according to the clarity index to obtain a key frame includes:
[0125] dividing the video into a plurality of segments;
[0126] The image frame with the highest definition index in each segment is selected as the key frame of the segment.
[0127] Optionally, in an embodiment of the present application, a clarity index can be evaluated for all data frames in the video in the image clarity detection stage, and the number of frames N to be extracted can be set according to requirements, and then all video frames can be divided into N segments, and the video frames with the highest clarity are selected in each segment. In this way, the clearest N frames of data in the time series are achieved, thereby improving the efficiency of extracting video key frames and reducing reliance on manual screening.
[0128] In summary, in the process of extracting video key frames, it is only necessary to extract N frames of high-definition pictures in the target video according to the detected picture clarity to obtain key frames for reconstruction, and the video is divided into N segments, and clarity detection is performed on all video frames in each segment. In each segment, the video frame with the highest clarity is selected, so as to achieve the extraction of the clearest N frames of data in the time series. N can be preset according to actual conditions. The larger N is, the more video frames are extracted. In addition, video frames can also be extracted according to preset thresholds. For example, when the clarity of the picture exceeds a certain threshold, the video frames exceeding the preset threshold are extracted as input for model reconstruction, so as to improve the final generation effect of the model and reduce the reliance on manual work in the key frame extraction process.
[0129] Please refer to Figure 6 The embodiment of the present application further provides a key frame extraction system 20, comprising:
[0130] An acquisition module 21 is used to acquire image frames of a video;
[0131] A filtering module 22, configured to perform low-pass filtering on the image frame to obtain a filtered image frame;
[0132] a variation module 23, configured to determine a variation in pixel intensity of the image frame and a variation in pixel intensity of the filtered image frame;
[0133] An index module 24, configured to determine a clarity index of the image frame according to a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame;
[0134] The filtering module 25 is used to filter the image frame according to the clarity index to obtain a key frame.
[0135] Optional,
[0136] The filtering module 22 comprises:
[0137] A second filtering module, configured to use a horizontal filter to filter the image frame in a horizontal direction to obtain a first filtered image frame;
[0138] The image frame is filtered in a vertical direction using a vertical filter to obtain a second filtered image frame.
[0139] Optionally, the changing module 23 includes:
[0140] A variation submodule, used to determine a pixel intensity difference of the image frame in the horizontal direction according to pixel values of two adjacent columns of pixels in the horizontal direction of the image frame;
[0141] Determining a pixel intensity difference of the image frame in the vertical direction according to pixel values of two adjacent columns of pixels in the vertical direction of the image frame;
[0142] Determining a pixel intensity difference in the horizontal direction of the filtered image frame according to pixel values of two adjacent columns of pixels in the horizontal direction of the filtered image frame;
[0143] The pixel intensity difference in the vertical direction of the filtered image frame is determined according to the pixel values of two adjacent columns of pixels in the vertical direction of the filtered image frame.
[0144] Optionally, the indicator module 24 includes:
[0145] An indicator submodule, used to determine the pixel intensity difference before and after filtering in the horizontal direction according to the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference of the filtered image frame in the horizontal direction;
[0146] Determine the pixel intensity difference before and after filtering in the vertical direction according to the pixel intensity difference of the image frame in the vertical direction and the pixel intensity difference of the filtered image frame in the vertical direction;
[0147] Determine a pixel intensity difference in the horizontal direction according to the pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction;
[0148] Determine a vertical direction pixel intensity difference according to the vertical direction pixel intensity difference of the image frame and the vertical direction pixel intensity difference before and after filtering;
[0149] The larger value of the horizontal pixel intensity difference and the vertical pixel intensity difference is used as the clarity index of the image frame.
[0150] Optionally, the indicator submodule includes:
[0151] A normalization module, used for normalizing the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference before and after the horizontal filtering to obtain the pixel intensity difference in the horizontal direction;
[0152] The pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction are normalized to obtain the pixel intensity difference in the vertical direction.
[0153] Optionally, the filtering module 25 includes:
[0154] A filtering submodule, used for dividing the video into a plurality of segments;
[0155] The image frame with the highest definition index in each segment is selected as the key frame of the segment.
[0156] The key frame extraction system 20 provided in the embodiment of the present invention can realize Figure 1 The various processes implemented in the method embodiment achieve the same technical effect and will not be described again here to avoid repetition.
[0157] The embodiment of the present invention provides an electronic device 30, see Figure 7 As shown, Figure 7 This is a principle block diagram of an electronic device 30 according to an embodiment of the present invention, which includes a processor 31, a memory 32, and a program or instruction stored in the memory 32 and executable on the processor 31. When the program or instruction is executed by the processor, the steps in any key frame extraction method of the present invention are implemented.
[0158] An embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of an embodiment of a key frame extraction method such as any one of the above-mentioned items is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0159] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0160] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes of any of the key frame extraction method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.
[0161] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0162] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.
[0163] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0165] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A key frame extraction method, It is characterized in that include: Acquire multiple image frames of a video; Performing low-pass filtering on each of the image frames to obtain a filtered image frame; determining a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame; Determining a clarity index of the image frame according to a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame; The multiple image frames are filtered according to the clarity index to obtain key frames in the multiple image frames.
2. The method according to claim 1, It is characterized in that The low-pass filtering of each of the image frames to obtain a filtered image frame comprises: Performing horizontal filtering on the image frame using a horizontal filter to obtain a first filtered image frame; The image frame is filtered in a vertical direction using a vertical filter to obtain a second filtered image frame.
3. The method according to claim 1 or 2, It is characterized in that The determining of the pixel intensity change of the image frame and the pixel intensity change of the filtered image frame comprises: Determine a pixel intensity difference of the image frame in the horizontal direction according to pixel values of two adjacent columns of pixels in the horizontal direction of the image frame; Determining a pixel intensity difference of the image frame in the vertical direction according to pixel values of two adjacent rows of pixels in the vertical direction of the image frame; Determining a pixel intensity difference in the horizontal direction of the filtered image frame according to pixel values of two adjacent columns of pixels in the horizontal direction of the filtered image frame; The pixel intensity difference in the vertical direction of the filtered image frame is determined according to the pixel values of two adjacent rows of pixels in the vertical direction of the filtered image frame.
4. The method according to claim 3, It is characterized in that Determining the clarity index of the image frame according to the pixel intensity change of the image frame and the pixel intensity change of the filtered image frame includes: Determine the pixel intensity difference before and after filtering in the horizontal direction according to the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference of the filtered image frame in the horizontal direction; Determine the pixel intensity difference before and after filtering in the vertical direction according to the pixel intensity difference of the image frame in the vertical direction and the pixel intensity difference of the filtered image frame in the vertical direction; Determine a pixel intensity difference in the horizontal direction according to the pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction; Determine a vertical direction pixel intensity difference according to the vertical direction pixel intensity difference of the image frame and the vertical direction pixel intensity difference before and after filtering; The larger value of the horizontal pixel intensity difference and the vertical pixel intensity difference is used as the clarity index of the image frame.
5. The method according to claim 4, It is characterized in that Determining a pixel intensity difference in the horizontal direction according to the pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction; Determining a vertical pixel intensity difference according to the pixel intensity difference of the image frame in the vertical direction and the pixel intensity difference before and after filtering in the vertical direction includes: Normalizing the pixel intensity difference of the image frame in the horizontal direction and the pixel intensity difference before and after filtering in the horizontal direction to obtain the pixel intensity difference in the horizontal direction; The pixel intensity difference in the horizontal direction of the image frame and the pixel intensity difference before and after filtering in the horizontal direction are normalized to obtain the pixel intensity difference in the vertical direction.
6. The method according to claim 1, It is characterized in that The filtering the multiple image frames according to the clarity index to obtain key frames in the multiple image frames includes: Divide the video into a plurality of segments, each of which includes a plurality of image frames; The image frame with the highest definition index in each segment is selected as the key frame of the segment.
7. A key frame extraction system, It is characterized in that include: An acquisition module, used for acquiring multiple image frames of a video; A filtering module, used for performing low-pass filtering on each of the image frames to obtain a filtered image frame; a change module, configured to determine a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame; An index module, used for determining a clarity index of the image frame according to a change in pixel intensity of the image frame and a change in pixel intensity of the filtered image frame; A filtering module is used to filter the multiple image frames according to the clarity index to obtain key frames in the multiple image frames.
8. An electronic device, It is characterized in that include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the key frame extraction method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium, It is characterized in that A computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the key frame extraction method according to any one of claims 1 to 6 are implemented.
10. A computer program product, It is characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the key frame extraction method according to any one of claims 1 to 6.
