Method and device for determining key frame

Abstract

The invention discloses a method and device for determining a key frame, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: extracting multiple frames of images from a file, wherein the file is a video or a dynamic graph; for any extracted non-first-frame image, acquiring a pixel number ratio of the non-first-frame image in each predetermined feature space, and determining a feature vector of the non-first-frame image according to the pixel number ratio; any feature space is determined by a preset value interval of each channel in the multi-frame image; judging whether the non-first-frame image is similar to the extracted previous-frame image of the non-first-frame image or not according to the feature vector of the non-first-frame image; and if not, determining the non-first-frame image as the key frame in the file. According to the embodiment, the occupation of redundant frames on computing resources can be avoided, and the overall processing efficiency of the video file or the dynamic graph file is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for determining a key frame. Background Art

[0002] In the process of processing video files or dynamic image files, it is generally necessary to capture some images from the video files or dynamic image files to perform subsequent processing such as target detection and image comparison. In the prior art, in order to avoid missing key information in the video files or dynamic image files, images are generally captured at smaller time intervals (for example, 1 second). However, this will generate too many redundant frames (frames with a high degree of similarity to existing frames). These redundant frames are difficult to provide useful information, and a large amount of computing resources are required for processing in the later stage, which affects the overall processing efficiency and processing effect of the video files or dynamic image files. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a method and device for determining key frames, which can determine non-redundant key frames from a video file or a dynamic image file, thereby avoiding the occupation of computing resources by redundant frames and improving the overall processing efficiency and processing effect of the video file or the dynamic image file.

[0004] To achieve the above objective, according to one aspect of the present invention, a method for determining a key frame is provided.

[0005] The method for determining a key frame in an embodiment of the present invention includes: extracting multiple frames of images from a file, where the file is a video or a dynamic image; for any non-first frame image extracted, obtaining the ratio of the number of pixels of the non-first frame image in each predetermined feature space, and determining the feature vector of the non-first frame image according to the ratio of the number of pixels; wherein any feature space is determined by a preset value interval of each channel in the multiple frame images; judging whether the non-first frame image is similar to the previous frame image extracted from the non-first frame image based on the feature vector of the non-first frame image; if not similar, determining the non-first frame image as a key frame in the file.

[0006] Optionally, extracting multiple frames of images from the file includes: when the file is a video, periodically extracting multiple frames of images from the file at preset time intervals; when the file is a dynamic image, extracting each frame of image contained in the file.

[0007] Optionally, the method further comprises: after extracting a plurality of frames of images from the file in video form, converting any extracted frame of images into a target format having a plurality of channels.

[0008] Optionally, the method further comprises: after extracting multiple frames of images from the file, performing proportional pixel sampling in the width direction and the height direction on any extracted frame of image to obtain a predetermined number of pixels in the frame of image.

[0009] Optionally, the method further includes: before extracting multiple frames of images from the file, dividing each channel into at least one value interval; combining the value intervals of each channel into a feature space; wherein a feature space is determined by a value interval of each channel.

[0010] Optionally, the proportion of the number of pixels of the non-first frame image in each predetermined feature space is obtained, and the feature vector of the non-first frame image is determined according to the proportion of the number of pixels, including: judging the feature space where each pixel is located according to the pixel value of each pixel in the channel in the non-first frame image; counting the number of pixels in any feature space; dividing the number of pixels by the total number of pixels in the non-first frame image to obtain the proportion of the number of pixels in the feature space; arranging the proportion of the number of pixels of each feature space in a preset order to obtain the feature vector of the non-first frame image.

[0011] Optionally, judging whether the non-first frame image is similar to the extracted previous frame image based on the feature vector of the non-first frame image includes: determining the similarity between the non-first frame image and the previous frame image based on the feature vector of the non-first frame image and the feature vector of the previous frame image; in response to the similarity being less than a preset similarity threshold, determining that the non-first frame image is not similar to the previous frame image; in response to the similarity being not less than a preset similarity threshold, determining that the non-first frame image is similar to the previous frame image.

[0012] To achieve the above objective, according to another aspect of the present invention, a device for determining a key frame is provided.

[0013] The apparatus for determining a key frame according to the embodiment of the present invention may include: an image extraction unit, a feature vector determination unit, and a similarity detection unit.

[0014] Among them, the image extraction unit is used to extract multiple frames of images from a file, and the file is a video or a dynamic image; the feature vector determination unit is used to obtain the pixel number ratio of the non-first frame image in each predetermined feature space for any non-first frame image extracted, and determine the feature vector of the non-first frame image according to the pixel number ratio; wherein any feature space is determined by the preset value interval of each channel in the multiple frame images; the similarity detection unit is used to determine whether the non-first frame image is similar to the previous frame image extracted from the non-first frame image based on the feature vector of the non-first frame image; if not similar, the non-first frame image is determined as the key frame in the file.

[0015] Optionally, the image extraction unit is further used to: when the file is a video, periodically extract multiple frames of images from the file at preset time intervals; when the file is a dynamic image, extract each frame of image contained in the file.

[0016] Optionally, the image extraction unit is further used to: after extracting multiple frames of images from the file in video form, convert any extracted frame of image into a target format with multiple channels.

[0017] Optionally, the image extraction unit is further used to: after extracting multiple frames of images from the file, perform proportional pixel sampling in the width direction and the height direction on any extracted frame of image to obtain a predetermined number of pixels in the frame of image.

[0018] Optionally, the device further includes: a feature space determination unit, used to divide each channel into at least one value interval before extracting multiple frames of images from the file; combine the value intervals of each channel into a feature space; wherein a feature space is determined by a value interval of each channel.

[0019] Optionally, the feature vector determination unit is further used to: determine the feature space where each pixel is located according to the pixel value of each pixel in the channel in the non-first frame image; count the number of pixels in any feature space; divide the number of pixels by the total number of pixels in the non-first frame image to obtain the pixel number ratio of the feature space; arrange the pixel number ratios of each feature space in a preset order to obtain the feature vector of the non-first frame image.

[0020] Optionally, the similarity detection unit is further used to: determine the similarity between the non-first frame image and the previous frame image based on the feature vector of the non-first frame image and the feature vector of the previous frame image; in response to the similarity being less than a preset similarity threshold, determine that the non-first frame image is not similar to the previous frame image; in response to the similarity being not less than a preset similarity threshold, determine that the non-first frame image is similar to the previous frame image.

[0021] To achieve the above objective, according to another aspect of the present invention, an electronic device is provided.

[0022] An electronic device of the present invention comprises: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining key frames provided by the present invention.

[0023] To achieve the above objective, according to yet another aspect of the present invention, a computer-readable storage medium is provided.

[0024] A computer-readable storage medium of the present invention stores a computer program, and when the program is executed by a processor, the method for determining a key frame provided by the present invention is implemented.

[0025] To achieve the above objective, according to yet another aspect of the present invention, a computer program product is provided.

[0026] A computer program product of the present invention includes a computer program. When the computer program is executed by a processor, the method for determining a key frame provided by the present invention is implemented.

[0027] According to the technical solution of the present invention, the embodiments of the above invention have the following advantages or beneficial effects: First, multiple frames of images are extracted from a file in the form of a video or dynamic image. For any non-first frame image extracted, the pixel count ratio of the non-first frame image in each predetermined feature space is obtained, and the feature vector of the non-first frame image is determined based on the pixel count ratio; then, the non-first frame image is judged based on the feature vector of the non-first frame image to determine whether it is similar to the previous frame image extracted from the non-first frame image; if not similar, the non-first frame image is determined as the key frame in the file. Through the above steps, the key frames of non-redundant frames can be determined from the file, thereby avoiding the occupation of subsequent computing resources by redundant frames and improving the overall processing efficiency and processing effect of video files or dynamic image files.

[0028] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention. Figure 1 is a schematic diagram of the main steps of the method for determining a key frame in an embodiment of the present invention; Figure 2 is a schematic diagram of the execution steps of the method for determining a key frame in an embodiment of the present invention; Figure 3 is a schematic diagram of characteristic vector calculation in an embodiment of the present invention; Figure 4 is a schematic diagram of components of an apparatus for determining a key frame in an embodiment of the present invention; Figure 5 is an exemplary system architecture diagram in which embodiments of the present invention may be applied; Figure 6 It is a schematic diagram of the structure of an electronic device used to implement the method for determining a key frame in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0031] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0032] Figure 1 Schematic diagram of main steps of a method for determining a key frame in an embodiment of the present invention.

[0033] like Figure 1 As shown, the method for determining a key frame in the embodiment of the present invention can be executed by a server, and the specific execution steps are as follows: Step S101: extract multiple frames of images from a file. The file may be a video or a dynamic image. The dynamic image may be a dynamic image in a GIF (Graphics Interchange Format) format. Each frame of the extracted image may be referred to as an extracted frame.

[0034] Step S102: For any non-first frame image extracted, obtain the pixel number ratio of the non-first frame image in each predetermined feature space, and determine the feature vector of the non-first frame image according to the pixel number ratio; wherein, the non-first frame image refers to each frame image except the first frame image in the extracted multi-frame image, and the non-first frame image currently performing calculation can be called the current frame. Feature space refers to a multi-dimensional space formed by limiting the multi-dimensional feature values ​​of pixels (i.e., the pixel values ​​of multiple channels). Any feature space is determined by the preset value interval of each channel in the multi-frame image. The above channels are also determined by the image format, for example, the channels in RGB (red, green, and blue) format are channel R, channel G, and channel B, and the channels in HSV (hue, saturation, brightness) format are channel H, channel S, and channel V. This step is used to determine the feature vector of the non-first frame image.

[0035] Step S103: Determine whether the non-first frame image is similar to the extracted previous frame image according to the feature vector of the non-first frame image; if not, determine the non-first frame image as a key frame in the file. After determining the feature vector of the current frame, the server can perform this step to calculate the similarity between the current frame and its previous frame image (the image is earlier in the file than the current frame and is the extracted frame closest to the current frame), and then determine whether the two are similar. If not, it means that there is useful information in the current frame, and the current frame is determined as a key frame.

[0036] Through the above steps, the key frames of non-redundant frames can be determined from the file, thereby avoiding the occupation of subsequent computing resources by redundant frames and improving the overall processing efficiency of video files or dynamic image files.

[0037] Figure 2 is a schematic diagram of the execution steps of the method for determining a key frame in an embodiment of the present invention, see Figure 2 .

[0038] Step S201: The server extracts multiple frames of images from the file. In an embodiment of the present invention, after obtaining a file in the form of a video or a dynamic image, the server reads the byte stream information therein and extracts multiple frames of images therefrom. In one embodiment, if the file is a video, the server periodically extracts multiple frames of images from the file at a preset time interval. In actual applications, the server can perform extraction based on the instantiated FFmpegFrameGrabber class; if the file is a dynamic image, the server extracts each frame of image contained in the file, for example, extracts all of them. In actual applications, the server can perform extraction based on the instantiated GifDecoder class.

[0039] Step S202: The server converts the extracted images into a target format. This step is optional. In this step, the server converts the extracted images into a target format for subsequent processing. For example, the target format can be any applicable format such as RGB, HSV, YUV (Y represents brightness, U and V represent chrominance), etc. Each target format contains multiple channels. For example, the RGB format contains three channels of R, G, and B. The following description takes the target format of RGB as an example.

[0040] In the embodiment of the present invention, if the target format is RGB, the server may convert the image extracted from the video file from the YUV format to the RGB format, and the conversion formula is as follows:

[0041] R=1.164(Y-16)+1.596(V-128)

[0042] G=1.164(Y-16)-0.813(V-128)-0.391(U-128)

[0043] B=1.164(Y - 16+2.018(U–128)

[0044] Since images extracted from dynamic image files are generally in RGB format, no conversion is required.

[0045] Step S203: The server performs global proportional sampling on the current frame. This step is optional. In order to reduce the amount of subsequent calculations, in this step, the server performs global proportional sampling on the current frame, that is, performs sampling at the same ratio in both width and height directions, so as to obtain a predetermined number of pixels in one frame of image.

[0046] In one embodiment, the server may calculate the sampling step length step in a single direction (width direction or height direction) according to the following formula:

[0047]

[0048] Where N represents the above predetermined number, width represents the image width, height represents the image height, and ⌊ ⌋ represents rounding down. The above global proportional sampling method helps to maintain the global spatial feature distribution of the image and prevent the loss of key information.

[0049] Preferably, in the embodiment of the present invention, for the first frame image extracted, the server can directly determine the first frame image as a key frame and record it to save the information therein, or determine the first frame image as a key frame after sampling, and the first frame image can also execute step S204 to calculate the feature vector. As a preferred solution, after extracting the image from the file, the server can execute a known algorithm to transform the value range of the pixel value in each image to a standard range, for example, the pixel values ​​of R, G, and B are all limited to [0,255].

[0050] Step S204: The server determines the feature vector of the current frame. Figure 3 , the server may pre-divide each channel of the target format into a value interval, and then combine the value intervals of each channel into a feature space. Specifically, the server may pre-divide each channel into at least one value interval. When a channel is divided into multiple value intervals, these value intervals may have the same value span or different value spans. For example, if the value of the R channel ranges from zero to 255, the server may divide the R channel into the following four value intervals with the same value span: [0,63], [64,127], [128,191], [192,255]. Thereafter, the server combines the value intervals of each channel to form multiple feature spaces. Specifically, the server selects a value interval from each channel, and these value intervals are combined into a feature space, which is determined by a value interval of each channel. By traversing the value intervals of each channel in the above manner, all feature spaces are formed. If each of the three channels is divided into four value intervals, a total of 64 feature spaces are formed. Figure 3 The n in represents the total number of feature spaces.

[0051] After determining the pixels of the current frame as the non-first frame image, the server can first determine the feature space where each pixel is located according to the pixel value of each pixel in each channel in the current frame, and then count the number of pixels in any feature space (i.e. Figure 3 n1, n2...n64 in the feature space), divide the number of pixels by the total number of pixels in the current frame to obtain the pixel ratio of the feature space (i.e. Figure 3v1, v2, ..., v64 in ). Finally, the server may determine the feature vector of the current frame according to the pixel quantity ratio, for example, the pixel quantity ratios of each feature space are arranged in a preset order to obtain the feature vector of the current frame; for another example, the pixel quantity ratios of each feature space are first calculated based on a preset function, and then the calculation results are arranged in a preset order to obtain the feature vector of the current frame. The above functions may be selected as needed, for example, an integer function, a modulus function, etc.

[0052] It should be noted that the above step of forming a feature space can be performed before the step of extracting multiple frames of images from the file (this is the preferred method), can be performed after extracting the images, or can be performed after extracting and sampling the images.

[0053] Step S205: The server calculates the similarity between the current frame and the previous frame. In this step, the server can calculate the similarity based on the feature vectors of the current frame and the previous frame, and the algorithm used can be the inner product method, cosine similarity, Pearson correlation coefficient, Euclidean distance and the like.

[0054] Step S206: The server determines whether the calculated similarity is less than a preset similarity threshold (for example, 0.95 can be taken). If it is determined to be less than the similarity threshold, it means that the current frame is not similar to the previous frame, and there is useful information in the current frame that is not in the previous frame, then step S207 is executed to determine that the current frame is a key frame and record the current frame; if it is determined to be not less than the similarity threshold, it means that the current frame is similar to the previous frame, there is no useful information in the current frame that is not in the previous frame, or the current frame contains less useful information, then step S208 is executed to determine that the current frame is a redundant frame and delete the current frame. After performing the above processing on each frame image, the recorded key frames can form an image data set to perform subsequent target detection and other links.

[0055] In this way, the embodiment of the present invention can determine the key frames of non-redundant frames from the file, thereby avoiding the occupation of subsequent computing resources by redundant frames and improving the overall processing efficiency of video files or dynamic image files. In addition, the detection of the above key frames and redundant frames is based on the similarity comparison between the current frame and the previous frame. This method is convenient for quickly locating the two frames that are compared with each other, thereby improving the comparison efficiency. Compared with the comparison of the current frame with all historical captured frames, the comparison of the current frame with all historical key frames, the comparison of the current frame with the latest key frame, etc., it has faster computing efficiency and helps to improve the overall processing efficiency of video files or dynamic image files.

[0056] The method for determining key frames in the embodiment of the present invention can be applied to any scenario involving video frame cutting and dynamic image frame cutting, such as anti-fraud scenarios and video comparison scenarios. In the anti-fraud scenario, the method for determining key frames in the embodiment of the present invention can be executed to detect key frames in the file, and then perform detection of specific targets such as payment QR codes on the key frames; in the video comparison scenario, the method for determining key frames in the embodiment of the present invention can be executed to detect key frames in two videos to be compared, and then compare the key frames in the two videos to be compared to obtain the video comparison result.

[0057] In the technical solution of the embodiment of the present invention, a high-efficiency key frame detection technology for videos or dynamic images is proposed, which is based on pixel sampling and implemented by feature space vector encoding. The global spatial feature distribution of the image is maintained through global proportional sampling. The key frames are detected and redundant frames are eliminated by extracting the spatial vector encoding of the frames and calculating the similarity, thereby saving computing resources in the downstream links of video detection. Solutions suitable for various application scenarios can also be formed by reasonably adjusting the capture time interval and the similarity threshold.

[0058] It should be noted that the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information that may be involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0059] For the above-mentioned method embodiments, for the convenience of description, they are expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, and some steps can actually be performed in other orders or simultaneously. In addition, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary to implement the present invention.

[0060] In order to better implement the above-mentioned solution of the embodiment of the present invention, relevant devices for implementing the above-mentioned solution are also provided below.

[0061] See also Figure 4 As shown, the apparatus 400 for determining key frames provided by the embodiment of the present invention may include: an image extraction unit 401 , a feature vector determination unit 402 , and a similarity detection unit 403 .

[0062] Among them, the image extraction unit 401 is used to extract multiple frames of images from a file, and the file is a video or a dynamic image; the feature vector determination unit 402 is used to obtain the pixel ratio of the non-first frame image in each predetermined feature space for any non-first frame image extracted, and determine the feature vector of the non-first frame image according to the pixel ratio; wherein any feature space is determined by the preset value interval of each channel in the multiple frame images; the similarity detection unit 403 is used to determine whether the non-first frame image is similar to the previous frame image extracted from the non-first frame image based on the feature vector of the non-first frame image; if not similar, the non-first frame image is determined as the key frame in the file.

[0063] In the embodiment of the present invention, the image extraction unit 401 is further used to: when the file is a video, periodically extract multiple frames of images from the file at preset time intervals; when the file is a dynamic image, extract each frame of image contained in the file.

[0064] As a preferred solution, the image extraction unit 401 is further used for: after extracting multiple frames of images from a file in the form of a video, converting any extracted frame of image into a target format with multiple channels.

[0065] Preferably, the image extraction unit 401 is further used for: after extracting multiple frames of images from the file, performing proportional pixel sampling in the width direction and the height direction on any extracted frame of image to obtain a predetermined number of pixels in the frame of image.

[0066] In one embodiment, the device 400 further includes: a feature space determination unit, which is used to divide each channel into at least one value interval before extracting multiple frames of images from the file; and combine the value intervals of each channel into a feature space; wherein a feature space is determined by a value interval of each channel.

[0067] Optionally, the feature vector determination unit 402 is further used to: determine the feature space where each pixel is located according to the pixel value of each pixel in the channel in the non-first frame image; count the number of pixels in any feature space; divide the number of pixels by the total number of pixels in the non-first frame image to obtain the pixel number ratio of the feature space; arrange the pixel number ratios of each feature space in a preset order to obtain the feature vector of the non-first frame image.

[0068] In addition, in an embodiment of the present invention, the similarity detection unit 403 is further used to: determine the similarity between the non-first frame image and the previous frame image based on the feature vector of the non-first frame image and the feature vector of the previous frame image; in response to the similarity being less than a preset similarity threshold, determine that the non-first frame image is not similar to the previous frame image; in response to the similarity being not less than a preset similarity threshold, determine that the non-first frame image is similar to the previous frame image.

[0069] According to the technical solution of the embodiment of the present invention, the key frames of non-redundant frames can be determined from the file, thereby avoiding the occupation of subsequent computing resources by redundant frames and improving the overall processing efficiency and processing effect of video files or dynamic image files.

[0070] Figure 5 An exemplary system architecture 500 is shown to which the method for determining a key frame or the apparatus for determining a key frame according to the embodiments of the present invention can be applied.

[0071] like Figure 5 As shown, the system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505 (this architecture is only an example, and the components included in the specific architecture may be adjusted according to the specific circumstances of the invention). The network 504 is used to provide a medium for a communication link between the terminal devices 501, 502, 503 and the server 505. The network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0072] The user can use the terminal devices 501, 502, 503 to interact with the server 505 through the network 504 to receive or send messages, etc. Various client applications can be installed on the terminal devices 501, 502, 503, such as a video frame cutting application (only an example).

[0073] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0074] The server 505 may be a server that provides various services, such as a background server (only an example) that provides support for a video frame cutting application operated by a user using the terminal devices 501, 502, 503. The background server may process the received request for determining a key frame, and feed back the processing result (such as the determined key frame - only an example) to the terminal devices 501, 502, 503.

[0075] It should be noted that the method for determining key frames provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the device for determining key frames is generally disposed in the server 505 .

[0076] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0077] The present invention also provides an electronic device. The electronic device of the embodiment of the present invention comprises: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining key frames provided by the present invention.

[0078] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of an electronic device suitable for implementing an embodiment of the present invention. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0079] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0080] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

[0081] In particular, according to the embodiments disclosed in the present invention, the process described in the main step diagram above can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the main step diagram. In the above embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit 601, the above functions defined in the system of the present invention are executed.

[0082] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0083] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes: an image extraction unit, a feature vector determination unit and a similarity detection unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the image extraction unit may also be described as "a unit that provides the extracted image to the feature vector determination unit".

[0085] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the device, the steps executed by the device include: extracting multiple frames of images from a file, the file is a video or a dynamic image; for any non-first frame image extracted, obtain the pixel number ratio of the non-first frame image in each predetermined feature space, and determine the feature vector of the non-first frame image according to the pixel number ratio; wherein any feature space is determined by the preset value interval of each channel in the multiple frame images; based on the feature vector of the non-first frame image, determine whether the non-first frame image is similar to the previous frame image extracted from the non-first frame image; if not similar, determine the non-first frame image as the key frame in the file.

[0086] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method for determining a key frame provided by the present invention is implemented.

[0087] In the technical solution of the embodiment of the present invention, multiple frames of images are first extracted from a file in the form of a video or dynamic image. For any non-first frame image extracted, the percentage of the number of pixels of the non-first frame image in each predetermined feature space is obtained, and the feature vector of the non-first frame image is determined according to the percentage of the number of pixels; thereafter, it is determined whether the non-first frame image is similar to the previous frame image extracted from the non-first frame image according to the feature vector of the non-first frame image; if not similar, the non-first frame image is determined as a key frame in the file. Through the above steps, the key frames of non-redundant frames can be determined from the file, thereby avoiding the occupation of subsequent computing resources by redundant frames and improving the overall processing efficiency and processing effect of video files or dynamic image files.

[0088] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining a key frame, characterized in that: include: Extract multiple frames of images from a file, wherein the file is a video or a dynamic image; For any non-first frame image extracted, obtain the pixel number ratio of the non-first frame image in each predetermined feature space, and determine the feature vector of the non-first frame image according to the pixel number ratio; wherein any feature space is determined by the preset value interval of each channel in the multiple frame images; Determine whether the non-first frame image is similar to the extracted previous frame image according to the feature vector of the non-first frame image; if not similar, determine the non-first frame image as a key frame in the file.

2. The method according to claim 1, characterized in that The step of extracting multiple frames of images from a file comprises: When the file is a video, periodically extracting multiple frames of images from the file at preset time intervals; When the file is a dynamic image, each frame image contained in the file is extracted.

3. The method according to claim 1, characterized in that The method further comprises: After extracting a plurality of frames of images from a file in the form of a video, any extracted frame of image is converted into a target format having a plurality of channels.

4. The method according to claim 1, characterized in that: The method further comprises: After extracting multiple frames of images from the file, equal-proportional pixel sampling in the width direction and the height direction is performed on any extracted frame of image to obtain a predetermined number of pixels in the frame of image.

5. The method according to claim 1, characterized in that The method further comprises: Before extracting the multiple frames of images from the file, each channel is divided into at least one value interval; The value intervals of each channel are combined into the feature space; wherein one feature space is determined by one value interval of each channel.

6. The method according to claim 5, characterized in that The obtaining of the pixel quantity ratio of the non-first frame image in each predetermined feature space, and determining the feature vector of the non-first frame image according to the pixel quantity ratio, includes: Determine the feature space where each pixel is located according to the pixel value of each pixel in the channel in the non-first frame image; Counting the number of pixels in any feature space; dividing the number of pixels by the total number of pixels in the non-first frame image to obtain the percentage of the number of pixels in the feature space; The pixel quantity ratios of each feature space are arranged in a preset order to obtain the feature vector of the non-first frame image.

7. The method according to claim 1, characterized in that The determining, based on the feature vector of the non-first frame image, whether the non-first frame image is similar to an extracted previous frame image of the non-first frame image comprises: Determining the similarity between the non-first frame image and the previous frame image according to the feature vector of the non-first frame image and the feature vector of the previous frame image; In response to the similarity being less than a preset similarity threshold, determining that the non-first frame image is not similar to the previous frame image; In response to the similarity being not less than a preset similarity threshold, it is determined that the non-first frame image is similar to the previous frame image.

8. A device for determining a key frame, characterized in that: include: An image extraction unit, used to extract multiple frames of images from a file, wherein the file is a video or a dynamic image; A feature vector determination unit, for obtaining, for any non-first frame image extracted, a pixel number ratio of the non-first frame image in each predetermined feature space, and determining a feature vector of the non-first frame image according to the pixel number ratio; wherein any feature space is determined by a preset value interval of each channel in the multiple frames of images; A similarity detection unit is used to determine whether the non-first frame image is similar to the extracted previous frame image according to the feature vector of the non-first frame image; if not similar, the non-first frame image is determined as a key frame in the file.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images