Video frame extraction method and apparatus, and computer device and medium

Abstract

Provided in the embodiments of the present application are a video frame extraction method and apparatus, and a computer device and a medium. The method comprises: acquiring an initial video frame set to be processed, wherein the initial video frame set includes a plurality of initial video frames, and the plurality of initial video frames are consecutive video frames in a video; performing brightness filtering processing on the initial video frames in the initial video frame set, so as to obtain a candidate video frame set; acquiring coding information which corresponds to the candidate video frame set, wherein the coding information includes frame types and timestamps which correspond to candidate video frames in the candidate video frame set; and performing frame extraction processing on the candidate video frame set on the basis of the coding information, so as to obtain a target video frame set, such that the accuracy of video frame extraction can be effectively improved.

Description

Video frame extraction method, device, computer equipment and medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on April 17, 2023, with application number 2023104101320 and application name “A video frame extraction method, device, computer equipment and medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of video processing technology, and in particular to a video frame extraction method, device, computer equipment and medium. Background Art

[0003] Currently, video processing often involves frame extraction. Common methods include interval extraction and uniform extraction with a set total frame count. However, these methods often struggle to guarantee a certain level of frame extraction accuracy. For example, interval extraction can result in many duplicate frames for static, unchanging video time slices. Furthermore, for video time slices with numerous transitions or frequent scene changes, many critical frames may be missed. Therefore, improving frame extraction accuracy has become a hot topic of research.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a video frame extraction method, apparatus, computer equipment, and medium, which can effectively improve the accuracy of video frame extraction.

[0006] A first aspect of an embodiment of the present application discloses a video frame extraction method, the method comprising:

[0007] Acquire an initial video frame set to be processed, wherein the initial video frame set includes a plurality of initial video frames, and the plurality of initial video frames are continuous video frames in a video;

[0008] Performing brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set;

[0009] Obtaining encoding information corresponding to the candidate video frame set, the encoding information including a frame type and a timestamp corresponding to each candidate video frame in the candidate video frame set;

[0010] The candidate video frame set is subjected to frame extraction processing based on the encoding information to obtain a target video frame set.

[0011] A second aspect of an embodiment of the present application discloses a video frame extraction device, comprising:

[0012] A first acquisition unit is configured to acquire an initial video frame set to be processed, wherein the initial video frame set includes a plurality of initial video frames, and the plurality of initial video frames are consecutive video frames in a video;

[0013] a filtering unit, configured to perform brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set;

[0014] A second acquiring unit is configured to acquire encoding information corresponding to the candidate video frame set, where the encoding information includes a frame type and a timestamp corresponding to each candidate video frame in the candidate video frame set;

[0015] The frame extraction unit is used to perform frame extraction processing on the candidate video frame set based on the encoding information to obtain a target video frame set.

[0016] A third aspect of an embodiment of the present application discloses a computer device, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect above.

[0017] A fourth aspect of an embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the first aspect.

[0018] The fifth aspect of the embodiments of the present application discloses a computer program product or a computer program, wherein the computer program product or the computer program includes program instructions, and when the program instructions are executed by a processor, the method of the first aspect is implemented.

[0019] In an embodiment of the present application, an initial video frame set to be processed can be obtained, and the initial video frame set can include multiple initial video frames, and the multiple initial video frames are continuous video frames in a video; then, brightness filtering processing can be performed on each initial video frame in the initial video frame set to obtain a candidate video frame set; further, encoding information corresponding to the candidate video frame set can be obtained, and frame extraction processing can be performed on the candidate video frame set based on the encoding information to obtain a target video frame set. By implementing the above method, the quality of the extracted video frames can be improved by brightness filtering processing of the video frames; the number of key frames can also be increased in combination with the decoding information of the video to improve the accuracy of frame extraction and the quality of the video frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG1 is a schematic diagram of the architecture of a video frame extraction system provided in an embodiment of the present application;

[0021] FIG2 is a flow chart of a video frame extraction method provided in an embodiment of the present application;

[0022] FIG3 is a schematic diagram of a method for determining a frame extraction requirement according to an embodiment of the present application;

[0023] FIG4 is a flow chart of another video frame extraction method provided in an embodiment of the present application;

[0024] FIG5 is a flow chart of another video frame extraction method provided in an embodiment of the present application;

[0025] FIG6 is a schematic structural diagram of a video frame extraction device provided in an embodiment of the present application;

[0026] FIG7 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0028] The present application proposes a video frame extraction scheme. The scheme works as follows: an initial video frame set to be processed can be obtained, and frame extraction processing can be performed on the initial video frame set to obtain a target video frame set. The initial video frame set includes multiple initial video frames, and the multiple initial video frames are consecutive video frames in a video. In a specific implementation, encoding information corresponding to a candidate video frame set can be obtained, such as the encoding information including the frame type and timestamp corresponding to each candidate video frame in the candidate video frame set. Based on the encoding information, frame extraction processing can be performed on the initial video frame set to obtain the target video frame set. In one implementation, before using the encoding information to extract the initial video frame set, brightness filtering processing can be performed on each initial video frame in the initial video frame set to obtain a candidate video frame set. The encoding information can then be used to extract the candidate video frame set to obtain the target video frame set. In the above embodiment, the brightness filtering processing of the video frames can be used to improve the quality of the extracted video frames. Furthermore, the number of key frames can be increased in combination with the video decoding information to improve the accuracy of frame extraction and the quality of the video frames.

[0029] In a specific implementation, the execution subject of the above-mentioned video frame extraction scheme can be a computer device, which can be a terminal or a server. The terminal mentioned here can be a smart phone, tablet computer, laptop computer, desktop computer and other devices; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc.

[0030] For example, when the computer device is a server, an embodiment of the present application provides a video frame extraction system, as shown in Figure 1, the video frame extraction system may include at least one terminal and at least one server; the terminal can obtain the initial video frame set to be processed, and upload the obtained initial video frame set to the server (i.e., the computer device), so that the server can obtain the initial video frame set to be processed, and extract the required target video frame set from the initial video frame set to be processed, that is, the server can use a video frame extraction scheme to implement frame extraction of the initial video frame set.

[0031] Please refer to FIG. 2 , which is a flow chart of a video frame extraction method provided in an embodiment of the present application. The video frame extraction method described in this embodiment can be applied to the above-mentioned computer device. As shown in FIG. 2 , the method may include:

[0032] S201: Obtain an initial video frame set to be processed.

[0033] The initial video frame set may include multiple initial video frames, which may be consecutive video frames in a video. The video may be of any type, such as film, education, or sports, and may be of any length, such as 20 seconds or 8 minutes.

[0034] In one implementation, after obtaining the initial video frame set, the initial video frame set can be scaled to control the resolution of each initial video frame in the initial video frame set. For example, the resolution of each initial video frame can be reduced to reduce the storage space occupied by the initial video frame set. In a specific implementation, the target resolution of the required initial video frame can be pre-set. The target resolution can be smaller than the original resolution of each initial video frame in the initial video frame set. For example, the target resolution can be set to 720×1280, so as to scale the initial video set based on the target resolution, thereby obtaining the scaled initial video set. The scaling process can be understood as an interpolation process, which requires the use of a corresponding scaling algorithm (interpolation algorithm). For example, the scaling algorithm can be any one of nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, etc.

[0035] In one implementation, step S201 may be triggered when a frame extraction requirement for a video is obtained.

[0036] Optionally, a frame extraction request for a video can be obtained on a computer device to determine that a frame extraction requirement for the video has been obtained. The frame extraction request can be triggered by an object (which can be any user) performing relevant operations on a user operation interface. For example, when the object needs to use the key frames of the video for subsequent reasoning tasks, the relevant operations can be performed on the user operation interface output by the terminal used to send a frame extraction request for the video to the computer device. For example, as shown in Figure 3, a user operation interface can be displayed on the terminal screen of the terminal used by the object, and the user operation interface can include at least a video input area 301 and a confirmation control 302. If the target object wants to obtain the matching product corresponding to the video, the relevant information of the video can be entered in the video input area 301 first (such as the video directly, or the link address corresponding to the video); then, a trigger operation can be performed on the confirmation control 302 (such as clicking, pressing, etc.); after the terminal detects that the confirmation control 302 is triggered, the video can be obtained based on the information in the input area 301. After the terminal obtains the video, a frame extraction request carrying the video can be sent to the computer device. The frame extraction request can carry the video so that the video frames contained in the video can be subsequently extracted.

[0037] Optionally, when a frame extraction timing task is triggered, it can be determined that the frame extraction requirement for the video has been obtained. For example, a frame extraction timing task can be set, and when the trigger condition in the frame extraction timing task is triggered, it can be determined that the frame extraction requirement has been obtained. In one embodiment, a large number of videos can be stored in a certain designated area, and the trigger condition can be that the current time reaches the preset processing time, or the remaining storage space of the designated storage area exceeds the preset remaining storage space, etc. The video that needs to be frame extracted can be a video in the designated area.

[0038] S202 : Perform brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set.

[0039] Luminance filtering may refer to filtering using the luminance values ​​of pixels in the initial video frame; that is, filtering may be performed on each initial video frame in the initial video frame set based on the luminance value to obtain a candidate video frame set. Considering that the same luminance filtering may be performed on each initial video frame, the luminance filtering process will be specifically described below using any initial video frame in the initial video frame set as an example.

[0040] It's understandable that the brightness of an image refers to the size of its pixels. A larger pixel value indicates a brighter image, and a smaller pixel value indicates a darker image. Furthermore, pixel values ​​in an image range from 0 to 255, meaning the brightness of an image can be represented by values ​​between 0 and 255. Pixel values ​​closer to 0 indicate lower brightness, while values ​​closer to 255 indicate higher brightness.

[0041] Based on this, it can be seen that in the embodiment of the present application, the brightness value of a pixel in any initial video frame ranges from [0, 255]. For example, the brightness value of a pixel in any initial video frame is 200, and the brightness value of another pixel is 185.

[0042] In one implementation, for any initial video frame in the initial video frame set, the brightness value of each pixel in the initial video frame can be obtained; if the brightness value of the pixel in the initial video frame meets a preset condition, the initial video frame can be used as a candidate video frame and added to the candidate video frame set. It should be understood that video frames are generally in YUV format, where Y represents brightness and U and V represent chrominance. Therefore, the Y value of the initial video frame can be directly obtained, and this Y value is the brightness value.

[0043] Among them, the preset conditions may include any one of the first preset condition, the second preset condition and the third preset condition. The first preset condition can be used to filter out initial video frames that are too dark, the second preset condition can be used to filter out initial video frames that are too bright, and the third preset condition can be used to filter out initial video frames in which the proportion of black is too large.

[0044] Based on this, it can be seen that the first preset condition can be that the minimum brightness value among the brightness values ​​of each pixel point is greater than a preset minimum brightness value. In other words, if the minimum brightness value among the brightness values ​​of all pixels of any initial video frame is greater than the preset minimum brightness value, it can be determined that the initial video frame is not too dark, and the initial video frame can be retained. If the minimum brightness value among the brightness values ​​of all pixels of any initial video frame is not greater than the preset minimum brightness value, it can be determined that the initial video frame is too dark, and the initial video frame can be filtered out.

[0045] The second preset condition may be that the maximum brightness value among the brightness values ​​of each pixel point is less than a preset maximum brightness value. In other words, if the maximum brightness value among the brightness values ​​of all pixels in any initial video frame is less than the preset maximum brightness value, it can be determined that the initial video frame is not too bright, and the initial video frame can be retained. If the maximum brightness value among the brightness values ​​of all pixels in any initial video frame is not less than the preset maximum brightness value, it can be determined that the initial video frame is too bright, and the initial video frame can be filtered out.

[0046] The third preset condition may be that a pixel ratio corresponding to any initial video frame is greater than a preset ratio. The pixel ratio may be the ratio of pixels having a brightness value of each pixel less than a preset threshold. In other words, if the pixel ratio corresponding to any initial video frame is greater than the preset ratio, i.e., it is determined that the black content of any initial video frame is not large, then the initial video frame may be retained. If the pixel ratio corresponding to any initial video frame is not greater than the preset ratio, i.e., it is determined that the black content of any initial video frame is large, then the initial video frame may be filtered out.

[0047] The specific values ​​of the preset minimum brightness value, preset maximum brightness value, preset threshold value, and preset ratio mentioned above can be pre-set, and their specific values ​​are not limited. For example, the minimum brightness value ∈ [0, 255], the preset maximum brightness value ∈ [0, 255], the preset threshold value ∈ [0, 255], and the preset ratio ∈ [0, 1].

[0048] For example, it is assumed that the preset minimum brightness value is 16, the preset maximum brightness value is 235, the preset threshold is 30, and the preset ratio is 32%.

[0049] Then, if the minimum brightness value of any initial video frame is greater than 16, the initial video frame may be added to the candidate video frame set, and if the minimum brightness value of the initial video frame is less than or equal to 16, the initial video frame may be filtered out.

[0050] If the maximum brightness value of any initial video frame is less than 235, the initial video frame may be added to the candidate video frame set, and if the minimum brightness value of the initial video frame is greater than or equal to 235, the initial video frame may be filtered out.

[0051] If the pixel ratio of any initial video frame determined under the preset threshold of 30 is greater than 32%, then the any initial video frame can be added to the candidate video frame set, and if the minimum brightness value of any initial video frame is less than or equal to 32%, then the any initial video frame can be filtered out.

[0052] Through the above filtering operation, initial video frames that are too dark, too bright, or have a large proportion of black can be filtered out from the initial video frame set to ensure the image quality of the final video frame set.

[0053] S203: Obtain encoding information corresponding to the candidate video frame set.

[0054] The encoding information corresponding to the candidate video frame set may include the frame type and timestamp corresponding to each candidate video frame in the candidate video frame set; in other words, the encoding information may include the encoding information of each candidate video frame in the candidate video frame set, and the encoding information of any candidate video frame may include the corresponding frame type and timestamp of the candidate video frame. The frame type of a candidate video frame may include any one of an I-frame, a P-frame, and a B-frame, and the timestamp may refer to the playback time of the candidate video frame in the video.

[0055] In one implementation, after encoding and decoding a video, continuous video frames included in the video and encoding information of the video can be obtained. The encoding information of the video can include the frame type and timestamp corresponding to each video frame in the video. The continuous video frames mentioned here are the initial video frame set in step S201. Then, after obtaining the encoding information of the video, the encoding information of each candidate video frame can be extracted from the encoding information of the video, so that the encoding information corresponding to the candidate video frame set is formed by the encoding information of each candidate video frame.

[0056] S204 , performing frame extraction processing on the candidate video frame set based on the coding information to obtain a target video frame set.

[0057] In one implementation, one or more of the first frame extraction strategy and the second frame extraction strategy may be adopted and frame extraction processing may be performed on the candidate video frame set based on the encoding information to obtain the target video frame set.

[0058] The first frame extraction strategy may include: extracting frames from the candidate video frame set using either one or more of the following methods: retaining the target frame type and retaining candidate video frames with a target interval; and then extracting the extracted candidate video frame set again using the pixel difference or timestamp difference between two adjacent candidate video frames to obtain the target video frame set. The target frame type may be an I frame, and the target interval may be sampling the candidate video frame set at an interval of N frames, where N may be any value, such as 3, 5, etc.

[0059] The second frame extraction strategy may refer to: on the premise of extracting the candidate video frame set based on the pixel difference between two adjacent candidate video frames combined with the frame type, the pixel difference between two adjacent candidate video frames combined with the timestamp difference between the two adjacent candidate video frames is used to extract the frame set again after the frame extraction, so as to obtain the target video frame set.

[0060] In an embodiment of the present application, an initial video frame set to be processed is obtained, and each initial video frame in the initial video frame set is subjected to brightness filtering to obtain a candidate video frame set, thereby ensuring the video frame quality of each candidate video frame in the candidate video frame set. Furthermore, encoding information corresponding to the candidate video frame set can be obtained, and frame extraction processing can be performed on the candidate video frame set based on the encoding information to obtain a target video frame set. In combination with the video decoding information, the number of key frames is increased, thereby improving the accuracy of frame extraction and the quality of video frames, thereby achieving efficient video frame extraction.

[0061] Please refer to FIG. 4 , which is a flow chart of another video frame extraction method provided in an embodiment of the present application. The video frame extraction method described in this embodiment can be applied to the above-mentioned computer device. As shown in FIG. 4 , the method may include:

[0062] S401: Obtain an initial video frame set to be processed.

[0063] S402 : Perform brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set.

[0064] S403: Obtain encoding information corresponding to the candidate video frame set.

[0065] The encoding information includes the frame type and timestamp corresponding to each candidate video frame in the candidate video frame set.

[0066] The specific implementation of steps S401-S403 may refer to the specific implementation of steps S201-S203 above, which will not be repeated here.

[0067] S404 , performing frame extraction processing on the candidate video frame set based on the coding information using a first frame extraction strategy to obtain a first target video frame set.

[0068] In one implementation, first, the candidate video frame set may be extracted according to a target extraction method to obtain a first video frame set. The target extraction method may include one or more of a first extraction method and a second extraction method. The first extraction method may be extracting candidate video frames of a target frame type from the candidate video frame set, and the second extraction method may be sampling the candidate video frame set at a target interval.

[0069] Based on this, if the target extraction method is the first extraction method, the specific implementation of determining the first video frame set can be: extracting candidate video frames of the target frame type from the candidate video frame set, and using the extracted candidate video frames as the first video frames, thereby obtaining the first video frame set. Among them, the target frame type can refer to an I frame, that is, the first video frame set can be constructed from candidate video frames of the frame type I frame. It can be understood that an I frame represents a key frame in a video, and a key frame can be understood as a picture in the candidate video frame that can be completely retained. Then, through the above method, the more critical video frames in the candidate video frame set can be retained.

[0070] If the target extraction method is the second extraction method, the specific implementation of determining the first video frame set may be: sampling the candidate video frame set at a target interval to obtain the first video frame set. The target interval may refer to sampling the candidate video frame set at an interval of N frames. The target interval may be preset, and its specific value is not limited. For example, the target interval (i.e., N value) may be 3, 5, etc. For example, if the target interval is 3, it means that a candidate video frame can be extracted every 3 candidate video frames, and the extracted candidate video frame can be used as the first video frame. In this way, the number of video frames in the final video frame set can be reduced to reduce the amount of computation required for subsequent video frame processing.

[0071] If the target extraction method is the first target extraction method and the second extraction method, the specific implementation of determining the first video frame set can be: extracting candidate video frames of the target frame type from the candidate video frame set, and using the extracted candidate video frames as the first reference video frame set; sampling the candidate video frame set at the target interval to obtain the second reference video frame set; after obtaining the first reference video frame set and the second reference video frame set, the first reference video frame set and the second reference video frame set can be merged to obtain the first video frame set.

[0072] It is understandable that when processing the candidate video frame set in the first target extraction method and the second extraction method, the same candidate video frame may be extracted. Subsequently, the first reference video frame set and the second reference video frame set may be merged. The merging process may be a union process, that is, the union of the first reference video frame set and the second reference video frame set may be used as the first video frame set. It can be seen that through this video frame extraction method, the more critical frames in the video frames can be retained, and the number of video frames can also be appropriately reduced.

[0073] Optionally, after obtaining the first video frame set, the first video frame set may be used as the first target video frame set.

[0074] Optionally, after obtaining the first video frame set, the first video frame set can be filtered using the pixel values ​​or timestamps of each first video frame in the first video frame set to obtain a first target video frame set. That is, after obtaining the first video frame set, the first video frame set can be subjected to further frame extraction processing to obtain a frame-extracted video frame set, which can be referred to as the first target video frame set. In a specific implementation, each first video frame in the first video frame set can be traversed, and the currently traversed first video frame can be compared with the previous first video frame. Based on the comparison result, a determination can be made as to whether to add the currently traversed first video frame to the first target video frame set. For ease of description, the currently traversed first video frame can be referred to as the current first video frame. If the comparison result shows that the pixel difference between the currently traversed first video frame and the previous first video frame is greater than a preset pixel difference, or that the timestamp difference between the current first video frame and the previous first video frame is greater than a first preset timestamp difference, then the current first video frame can be added to the first target video frame set. Then, after traversing each first video frame in the first video frame set, the final first target video frame set can be obtained.

[0075] Among them, the pixel gap can refer to the average value of the pixel difference between the two video frames. Based on this, it can be seen that the pixel gap between the current first video frame and the previous first video frame is determined as follows: first, the pixel value of each pixel in the current first video frame can be obtained, and the pixel value of each pixel in the previous video frame can be obtained; then, based on the pixel value of each pixel in the current first video frame and the pixel value of each pixel in the previous video frame, the pixel gap between the current first video frame and the previous first video frame can be determined. Optionally, for any pixel in the current first video frame, the pixel difference for any pixel can be determined based on the pixel value of the any pixel and the pixel value of the pixel corresponding to the any pixel in the previous video frame. For example, the difference between the pixel value of the any pixel and the pixel value of the pixel corresponding to the any pixel can be used as the pixel difference. Considering that the difference can be positive or negative, in order to better characterize the pixel gap between the two video frames, the absolute value of the difference can be taken as the pixel difference.

[0076] Through the above method, the pixel difference for each pixel in the current first video frame can be determined; then, the average of the pixel differences for each pixel can be used as the pixel difference between the current first video frame and the previous first video frame. It can be understood that the pixels of each video frame in the video are the same, that is, each pixel in the current first video frame has a corresponding pixel in the previous first video frame.

[0077] For example, if the pixel values ​​of all pixels in the current first video frame are a1, a2, ..., an, and the pixel values ​​of all pixels in the previous first video frame are b1, b2, ..., bn, then the computer device can calculate the pixel difference between the current first video frame and the previous first video frame using the following formula (1): [|a1-b1|+|a2-b2|+...+|an-bn] / n (1)

[0078] Among them, the method for determining the timestamp difference between the current first video frame and the previous first video frame is as follows: the timestamp of the current first video frame and the timestamp of the previous first video frame can be obtained first; then, the timestamp difference can be determined based on the timestamp of the current first video frame and the timestamp of the previous first video frame, such as the difference between the timestamp of the previous first video frame and the timestamp of the current first video frame can be used as the timestamp difference.

[0079] Among them, the preset pixel gap and the first preset timestamp gap can be pre-set, the preset pixel gap is greater than 0, the first preset timestamp gap is greater than 0, and the specific values ​​are not limited. Through the above method, the first video frame with a larger pixel gap or the first video frame with a larger time gap can be retained, ensuring the frame gap between video frames, which can effectively avoid the extraction of duplicate frames. For example, if only frame extraction methods such as "interval timestamp frame" and "uniform frame extraction with a set total number of frames" are used to extract frames for the video, for static and unchanging video time slices, more duplicate video frames may be extracted; and by using the pixel gap or timestamp gap between two frames in the embodiment of the present application, the number of duplicate frames extracted can be effectively reduced, thereby improving the accuracy of frame extraction.

[0080] S405 , performing frame extraction processing on the candidate video frame set based on the coding information using a second frame extraction strategy to obtain a second target video frame set.

[0081] In one implementation, each candidate video frame in the candidate video frame set can be traversed first, and the first pixel difference between the current candidate video frame and the previous candidate video frame can be obtained. The first pixel difference refers to the pixel difference between the current candidate video frame and the previous candidate video frame. The specific implementation of this first pixel difference can be found in the method for determining the pixel difference between the current first video frame and the previous first video frame in step S405, and will not be further described here. Then, based on the first pixel difference and the frame type of the current candidate video frame, the candidate video frame set can be filtered to obtain a second video frame set.

[0082] The specific implementation of the above-mentioned determination of the second video frame set may be described as follows.

[0083] If the frame type of the current candidate video frame is an I frame type and the first pixel difference is greater than the first preset difference, the current candidate video frame can be added to the second video frame set; if the frame type of the current candidate video frame is a P frame type and the first pixel difference is greater than the second preset difference, the current candidate video frame can be added to the second video frame set; if the frame type of the current candidate video frame is a B frame type and the first pixel difference is greater than the third preset difference, the current candidate video frame can be added to the second video frame set.

[0084] Among them, the first preset gap is smaller than the second preset gap, and the second preset gap is smaller than the third preset gap. Considering that the criticality of video frames of large I-frame type, P-frame type and B-frame type in the video gradually decreases, when using the preset gaps (i.e., the first preset gap, the second preset gap, and the third preset gap here) to screen candidate video frames, the preset gaps corresponding to the I-frame type, P-frame type and B-frame type, respectively, can be gradually reduced, that is, the first preset gap mentioned above is smaller than the second preset gap, and the second preset gap is smaller than the third preset gap. Through this setting method, as many key video frames as possible can be retained to improve the accuracy of frame extraction.

[0085] Through the above-mentioned frame extraction method, the pixel gap between video frames can be used to avoid the extraction of duplicate frames, and the frame type can be combined to extract key frames to improve the accuracy of frame extraction. For example, if only frame extraction methods such as "interval timestamp frame" and "uniform frame extraction with a set total number of frames" are used to extract frames for a video, for static and unchanging video time slices, more duplicate video frames may be extracted; and for video time slices with a lot of transition content or frequent scene changes, more key frames may be missed. By combining the frame type and the pixel gap between two frames in the embodiment of the present application, the number of duplicate frames extracted can be effectively reduced, and the number of key frames extracted can be increased, thereby effectively improving the accuracy of frame extraction.

[0086] Optionally, after obtaining the second video frame set, the second video frame set may be used as the second target video frame set.

[0087] Optionally, after obtaining the second video frame set, the second video frame set can also be filtered based on the timestamps of each second video frame in the second video frame set to obtain a second target video frame set. In a specific implementation, each second video frame in the second video frame set can be traversed, and the second pixel gap between the current second video frame currently traversed and the previous second video frame can be obtained. Among them, the second pixel gap refers to the pixel gap between the current second video frame and the previous second video frame. The specific implementation of the second pixel gap can refer to the method for determining the pixel gap between the current first video frame and the previous first video frame in step S404, which will not be repeated here. Then, the second video frame set can be filtered based on the second pixel gap, the timestamp of the current second video frame, and the timestamp of the previous second video frame to obtain a second target video frame set.

[0088] The specific implementation of the above-mentioned determination of the second target video frame set can be found in the following description.

[0089] First, the timestamp difference between the timestamp of the current second video frame and the timestamp of the previous second video frame can be determined, wherein the specific implementation of the timestamp difference here can refer to the method for determining the timestamp difference in step S404, which will not be repeated here. Then, based on the timestamp difference and the second pixel difference, it is determined whether to add the current second video frame to the second target video frame set. In one embodiment, if the timestamp difference is greater than the second preset timestamp difference and the second pixel difference is greater than the fourth preset difference, the current second video frame can be added to the second target video frame set; if the timestamp difference is less than or equal to the second preset timestamp difference and the second pixel difference is greater than the fifth preset difference, the current second video frame can be added to the second target video frame set.

[0090] The fifth preset gap is greater than the fourth preset gap. It is understandable that, when ensuring the gap between two video frames, if the time stamp difference between the two video frames is large, the pixel gap between the two video frames can be set to be smaller. Correspondingly, if the time stamp difference between the two video frames is small, the pixel gap between the two video frames can be set to be larger. In this case, the fifth preset gap can be set to be larger than the fourth preset gap.

[0091] Through the above-mentioned frame extraction method, the pixel gap and timestamp gap between video frames can be used to effectively avoid the extraction of duplicate frames, and the extraction of key frames can be improved, thereby improving the accuracy of frame extraction. For example, if only frame extraction methods such as "interval timestamp frames" and "uniform frame extraction with a set total number of frames" are used to extract frames from a video, for static and unchanging video time slices, more duplicate video frames may be extracted; and for video time slices with a lot of transition content or frequent scene changes, more key frames may be missed. By combining the pixel gap and timestamp gap in the embodiments of the present application, the number of duplicate frames extracted can be effectively reduced, and the number of key frames extracted can be increased, thereby effectively improving the accuracy of frame extraction.

[0092] S406: Merge the first target video frame set and the second target video frame set to obtain a target video frame set.

[0093] In one implementation, considering that the first target video frame set and the second target video frame set may have the same video frames, they may be merged to obtain the target video frame set. The merging process may be a union process, that is, the union of the first target video frame set and the second target video frame set may be used as the target video frame set.

[0094] For example, if the first target video frame set includes: video frame 1, video frame 2, video frame 3, video frame 4; the second target video frame set includes: video frame 2, video frame 4, video frame 5, video frame 6; then the target video frame set includes: video frame 1, video frame 2, video frame 3, video frame 4, video frame 5, video frame 6.

[0095] In one implementation, after acquiring the target video frame set, quality control can be performed on the target video frame set to control the image compression quality of each target video frame in the target video frame set. For example, each target video frame can be compressed to reduce the storage space occupied by the target video frame set. In a specific implementation, a desired image compression quality parameter can be pre-set, and image compression processing can be performed on the target video set based on the quality parameter to obtain the compressed target video set. The image compression processing can be any of JPEG compression, WebP compression, etc., without limitation.

[0096] For ease of understanding, the video frame extraction method proposed in the embodiment of the present application is further described below in conjunction with Figure 5. In the description, the frame extraction processing at each stage is understood as the processing of a module, that is, the video frame extraction method can be divided into the processing of each module as shown in Figure 5. It can be understood that some modules will output every video frame flowing into it, and some modules will also filter out some of their input video frames, thereby realizing the frame extraction processing of the video. The functions of each module are described below.

[0097] Module A may be a scaling module, which may be used to control the resolution of each initial video frame in the initial video frame set. For example, the configuration parameters of module A may include: the resolution of the output video frame (i.e., the target resolution). The processing of module A may be to process each initial video frame in the initial video frame set based on the target resolution and the scaling algorithm, thereby obtaining the initial video frame set after scaling processing, such as the scaling algorithm may be bicubic interpolation. That is, the input of module A is the initial video frame set, and the output is the initial video frame set after scaling processing.

[0098] It should be noted that the configuration parameters mentioned in the embodiments of the present application can be understood as parameters that need to be pre-set (configured). Different parameter configurations can be used for different business scenarios (such as auditing, content understanding, etc.).

[0099] Module B can be used to perform brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set. That is, it can be used to filter out video frames that are too bright, too dark, or have a large proportion of black. It can be understood that the input of module B is each initial video frame in the initial video frame set, and the output is the candidate video frame set.

[0100] The processing process of Module B can be described as follows: For each input initial video frame, calculate the luminance component Y (i.e., luminance value) of each pixel point in the initial video frame, and count the minimum luminance value and the maximum luminance value among all luminance values. Denote the minimum luminance value and the maximum luminance value as Ymin and Ymax respectively.

[0101] It is also possible to calculate the proportion of pixel points (i.e., pixel ratio) among all luminance values that are less than a preset threshold (th_black threshold), and denote this pixel ratio as black_percentage.

[0102] The condition for the initial video frame input to be filtered by this Module B can be the following expression, that is, the initial video frame that meets this condition can be added to the candidate video frame set.

[0103] (Ymin > th_Y1) ‖ (Ymax < th_Y2) ‖ (black_percentage > th_bp)

[0104] The configuration parameters include: th_Y1 (value range 0 - 255), th_Y2 (value range 0 - 255), th_black (value range 0 - 255), th_bp (value range 0 - 1). Among them, th_Y1 represents the preset minimum luminance value, th_Y2 represents the preset maximum luminance value, th_black represents the preset threshold, and th_bp represents the preset ratio; "Ymin > th_Y1" represents the first preset condition above, "Ymax < th_Y2" represents the second preset condition above, and "black_percentage > th_bp" represents the third preset condition above.

[0105] After Module B, it can be divided into two parallel branches, and the final output frame set can include the union of the output results of the two branches. Among them, the first branch can include Module C and Module D, and the second branch can include Module E and Module F. The first branch can be understood as the implementation process of frame extraction processing on the candidate video frame set based on the coding information using the first frame extraction strategy to obtain the first target video frame set, and the second branch can be understood as the implementation process of frame extraction processing on the candidate video frame set based on the coding information using the second frame extraction strategy to obtain the second target video frame set. Based on this, it can be known that the output results of the two branches are the first target video frame set and the second target video frame set respectively, that is, the union between the first target video frame set and the second target video frame set can be used as the final output frame set (i.e., the target video frame set).

[0106] The following specifically introduces each module in these two branches.

[0107] Regarding Module C:

[0108] Assume that the video frame currently input by module C is F n , where F n It can be understood as a candidate video frame in the above-mentioned candidate video frame set, such as the nth frame in the candidate video frame set, the video frame F of the C module n The following conditions must be met:

[0109] Frame type is I frame ‖(n mod th_sample == 0)

[0110] mod is the modulo operation.

[0111] Configuration parameters include: th_sample (value >= 1). th_sample represents the target interval (ie, N).

[0112] Wherein, "frame type is I frame" indicates the first extraction method, and "n mod th_sample == 0" indicates the second extraction method.

[0113] As mentioned above, the C module can be understood as a selector that retains I frames and video frames with an interval of N.

[0114] For D module:

[0115] Assume that the video frame currently input to the D module is F n , the corresponding timestamp is T n , the most recent video frame that passes through the D module is F n-1 , the corresponding timestamp is T n-1 Here F n It can be understood as the first video frame currently traversed in the above-mentioned first video frame set (i.e., the current first video frame), F n-1 It can be understood as the first video frame before the current first video frame. n The following conditions must be met: (Diff(F n -F n-1 )>th_D_pixel)‖(T n –T n-1 >th_D_time)

[0116] Diff represents the average value of pixel differences between two video frames (the current first video frame and the previous first video frame), that is, the pixel gap.

[0117] Configuration parameters include: th_D_pixel (value>0), th_D_time (value>0). th_D_pixel represents the preset pixel difference, th_D_time represents the first preset timestamp difference; "Diff(F n -F n-1)>th_D_pixel” indicates that the pixel difference between the current first video frame and the previous first video frame is greater than the preset pixel difference, “T n –T n-1 >th_D_time” indicates that the time stamp difference between the current first video frame and the previous first video frame is greater than the first preset time stamp difference.

[0118] As mentioned above, the D module can be understood as a selector for determining a frame difference or a duration between adjacent frames. Here, the frame difference can refer to the pixel difference between the current first video frame and the previous first video frame, and the duration between adjacent frames can refer to the timestamp difference between the current first video frame and the previous first video frame.

[0119] For E module:

[0120] Assume that the video frame currently input by module E is F n , the most recent video frame that passes through the E module is F n-1 , where F n It can be understood as the candidate video frame currently traversed in the above candidate video frame set (i.e., the current candidate video frame), F n-1 It can be understood as the previous candidate video frame of the current candidate video frame. The video frame F through the E module n The following conditions must be met: Diff(F n -F n-1 )>th_E_pixel_I if F n The frame type is I‖ Diff(F n -F n-1 )>th_E_pixel_P if F n The frame type is P‖ Diff(F n -F n-1 )>th_E_pixel_B if F n Frame type is B‖

[0121] Diff represents the average value of pixel differences between two video frames (the current candidate video frame and the previous candidate video frame), that is, the first pixel difference.

[0122] Configuration parameters include: th_E_pixel_I (value>0), th_E_pixel_P (value>0), th_E_pixel_B (value>0). th_E_pixel_I represents the first preset difference, th_E_pixel_P represents the second preset difference, and th_E_pixel_B represents the third preset difference; "Diff(F n -F n-1 )>th_E_pixel_I if F n"Frame type is I" means: if the frame type of the current candidate video frame is I frame type, and the first pixel difference is greater than the first preset difference; "Diff(F n -F n-1 )>th_E_pixel_P if F n "Frame type is P" means: if the frame type of the current candidate video frame is P frame type, and the first pixel difference is greater than the second preset difference; "Diff(F n -F n-1 )>th_E_pixel_B if F n The frame type is B” indicates that: if the frame type of the current candidate video frame is a B frame type, and the first pixel difference is greater than a third preset difference.

[0123] As mentioned before, the E module can be understood as a selector that takes the frame type into consideration.

[0124] For F modules:

[0125] Assume that the video frame currently input by the F module is F n , the corresponding timestamp is T n , the most recent video frame that passes through the F module is F n-1 , the corresponding timestamp is T n-1 Here F n It can be understood that the second video frame currently traversed in the above-mentioned second video frame set (ie, the current second video frame) is F n-1 It can be understood as the second video frame before the current second video frame. n The following conditions must be met: Diff(F n -F n-1 )>th_F_pixel_1 if T n –T n-1 >th_F_time‖ Diff(F n -F n-1 )>th_F_pixel_2 if T n –T n-1 <=th_F_time

[0126] Diff represents an average value of pixel differences between two video frames (the current second video frame and the previous second video frame), that is, a second pixel difference.

[0127] Configuration parameters include: th_F_pixel_1 (value>0), th_F_pixel_2 (value>0), th_F_tim (value>0). th_F_pixel_1 represents the fourth preset gap, th_F_pixel_2 represents the fifth preset gap, th_F_time represents the second preset timestamp gap; "Diff(F n -F n-1 )>th_F_pixel_1 if T n –T n-1 >th_F_time” means: if the timestamp difference is greater than the second preset timestamp difference, and the second pixel difference is greater than the fourth preset difference; “Diff(F n -F n-1 )>th_F_pixel_2 if T n –T n-1 <=th_F_time” indicates that if the timestamp difference is less than or equal to the second preset timestamp difference, and the second pixel difference is greater than the fifth preset difference.

[0128] As mentioned above, the F module can be understood as a selector for frame difference determination combined with adjacent frame durations. Here, adjacent frame durations can refer to the timestamp difference between the current second video frame and the previous second video frame, and the frame difference can refer to the pixel difference between the current second video frame and the previous second video frame.

[0129] The G module may be a quality control module, which may be used to control the image compression quality of each target video frame in the target video frame set. For example, the configuration parameters of the G module may include: outputting quality parameters for image compression of the target video frames. The G module may process each target video frame in the target video frame set based on the quality parameters and a compression method, thereby obtaining a target video frame set after image compression processing, such as JPEG compression, WebP compression, etc.

[0130] In one implementation, the target video frame set extracted by the embodiment of the present application can be applied to various inference services, which can be cloud-based AI inference services, such as auditing, content understanding and other services. Compared with the frame extraction methods such as "interval time frame" and "uniform frame extraction with a set total number of frames", it can effectively avoid the extraction of duplicate frames, improve the extraction of key frames, and thus improve the extraction accuracy. In addition, when using the frame extraction methods such as "interval time frame" and "uniform frame extraction with a set total number of frames" to extract video frames, if the number of extracted frames is small, some of the key frames required for these inference services may usually be lost; and if the number of extracted frames is increased in order to retain more key frames, firstly, the computational cost of the inference service will be significantly increased, and secondly, some invalid frames or low-quality frames may cause the computational accuracy of the inference service to decrease. The embodiment of the present application can use as few video frames as possible to extract key frames and reduce the extraction of duplicate frames. From the above specific description of the video frame extraction method, it can be seen that the embodiment of the present application can provide a more general, fast, and parameter-flexible video frame extraction method that can adapt to various inference services. This method can combine the video encoding information and multi-level serial and parallel image processing algorithms to maintain lightweight computing while effectively improving the frame extraction accuracy and the quality of the extracted video frames. The method has a certain degree of versatility and can customize parameter sets for each inference service to maximize the satisfaction of the frame extraction requirements of different inference services. In summary, the video frame extraction method in the embodiment of the present application is an efficient and general frame extraction method.

[0131] In an embodiment of the application, an initial video frame set to be processed can be obtained, and each initial video frame in the initial video frame set can be subjected to brightness filtering to obtain a candidate video frame set, thereby ensuring the video frame quality of each candidate video frame in the candidate video frame set. Furthermore, the encoding information corresponding to the candidate video frame set can be obtained; a first frame extraction strategy can be used to extract frames from the candidate video frame set based on the encoding information to obtain a first target video frame set; and a second frame extraction strategy can be used to extract frames from the candidate video frame set based on the encoding information to obtain a second target video frame set; thereby, the first target video frame set and the second target video frame set can be merged to obtain a target video frame set. Through the above method, the encoding information of the video and the multi-stage serial-parallel image processing algorithm can be combined to maintain lightweight calculation while effectively improving the frame extraction accuracy and the quality of the extracted video frames.

[0132] Please refer to FIG6 , which is a schematic diagram of the structure of a video frame extraction device provided in an embodiment of the present application. The video frame extraction device described in this embodiment includes:

[0133] A first acquiring unit 601 is configured to acquire an initial video frame set to be processed, wherein the initial video frame set includes a plurality of initial video frames, and the plurality of initial video frames are consecutive video frames in a video;

[0134] A filtering unit 602 is configured to perform brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set;

[0135] A second acquiring unit 603 is configured to acquire encoding information corresponding to the candidate video frame set, where the encoding information includes a frame type and a timestamp corresponding to each candidate video frame in the candidate video frame set;

[0136] The frame extraction unit 604 is configured to perform frame extraction processing on the candidate video frame set based on the coding information to obtain a target video frame set.

[0137] In one implementation, the frame extraction unit 604 is specifically configured to:

[0138] Using a first frame extraction strategy to perform frame extraction processing on the candidate video frame set based on the encoding information to obtain a first target video frame set;

[0139] Using a second frame extraction strategy to perform frame extraction processing on the candidate video frame set based on the encoding information to obtain a second target video frame set;

[0140] The first target video frame set and the second target video frame set are merged to obtain a target video frame set.

[0141] In one implementation, the frame extraction unit 604 is specifically configured to:

[0142] Extracting the candidate video frame set according to a target extraction method to obtain a first video frame set; the target extraction method includes one or more of a first extraction method and a second extraction method, the first extraction method extracting candidate video frames of a target frame type from the candidate video frame set, and the second extraction method sampling the candidate video frame set at a target interval;

[0143] Traversing each first video frame in the first video frame set, if a pixel difference between a current first video frame and a previous first video frame is greater than a preset pixel difference, or a timestamp difference between the current first video frame and the previous first video frame is greater than a first preset timestamp difference, then adding the current first video frame to the first target video frame set.

[0144] In one implementation, the frame extraction unit 604 is specifically configured to:

[0145] Traversing each candidate video frame in the candidate video frame set, and obtaining a first pixel difference between a current candidate video frame and a previous candidate video frame;

[0146] filtering the candidate video frame set based on the first pixel difference and the frame type of the current candidate video frame to obtain a second video frame set;

[0147] Traversing each second video frame in the second video frame set, and obtaining a second pixel difference between a current second video frame traversed and a previous second video frame;

[0148] The second video frame set is screened based on the second pixel difference, the timestamp of the current second video frame, and the timestamp of the previous second video frame to obtain a second target video frame set.

[0149] In one implementation, the frame extraction unit 604 is specifically configured to:

[0150] If the frame type of the current candidate video frame is an I frame type, and the first pixel difference is greater than a first preset difference, adding the current candidate video frame to the second video frame set;

[0151] If the frame type of the current candidate video frame is a P frame type, and the first pixel difference is greater than a second preset difference, adding the current candidate video frame to the second video frame set;

[0152] If the frame type of the current candidate video frame is a B frame type, and the first pixel difference is greater than a third preset difference, adding the current candidate video frame to the second video frame set;

[0153] The first preset gap is smaller than the second preset gap, and the second preset gap is smaller than the third preset gap.

[0154] In one implementation, the frame extraction unit 604 is specifically configured to:

[0155] Determining a timestamp difference between the timestamp of the current second video frame and the timestamp of the previous second video frame;

[0156] If the timestamp difference is greater than a second preset timestamp difference, and the second pixel difference is greater than a fourth preset difference, adding the current second video frame to a second target video frame set;

[0157] If the timestamp difference is less than or equal to the second preset timestamp difference, and the second pixel difference is greater than a fifth preset difference, adding the current second video frame to the second target video frame set;

[0158] The fifth preset gap is greater than the fourth preset gap.

[0159] In one implementation, the frame extraction unit 604 is specifically configured to:

[0160] For any initial video frame in the initial video frame set, obtaining a brightness value of each pixel in the any initial video frame;

[0161] If the brightness value of the pixel in any of the initial video frames meets a preset condition, then adding the any of the initial video frames to the candidate video frame set;

[0162] Among them, the preset condition includes any one of a first preset condition, a second preset condition and a third preset condition, the first preset condition is that the minimum brightness value among the brightness values ​​of each pixel point is greater than the preset minimum brightness value, the second preset condition is that the maximum brightness value among the brightness values ​​of each pixel point is less than the preset maximum brightness value, and the third preset condition is that the pixel ratio corresponding to any initial video frame is greater than the preset ratio, and the pixel ratio is the ratio of pixels in the brightness value of each pixel point that is less than the preset threshold.

[0163] It is understood that the division of units in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. The functional units in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0164] Please refer to Figure 7, which is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device includes: a processor 701 and a memory 702. Optionally, the computer device may also include a network interface 703. The processor 701, memory 702, and network interface 703 can exchange data.

[0165] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0166] The memory 702 may include a read-only memory and a random access memory, and provides program instructions and data to the processor 701. A portion of the memory 702 may also include a non-volatile random access memory. When the processor 701 calls the program instructions, it is used to execute:

[0167] Acquire an initial video frame set to be processed, wherein the initial video frame set includes a plurality of initial video frames, and the plurality of initial video frames are continuous video frames in a video;

[0168] Performing brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set;

[0169] Obtaining encoding information corresponding to the candidate video frame set, the encoding information including a frame type and a timestamp corresponding to each candidate video frame in the candidate video frame set;

[0170] The candidate video frame set is subjected to frame extraction processing based on the encoding information to obtain a target video frame set.

[0171] In one implementation, the processor 701 is specifically configured to:

[0172] Using a first frame extraction strategy to perform frame extraction processing on the candidate video frame set based on the encoding information to obtain a first target video frame set;

[0173] Using a second frame extraction strategy to perform frame extraction processing on the candidate video frame set based on the encoding information to obtain a second target video frame set;

[0174] The first target video frame set and the second target video frame set are merged to obtain a target video frame set.

[0175] In one implementation, the processor 701 is specifically configured to:

[0176] Extracting the candidate video frame set according to a target extraction method to obtain a first video frame set; the target extraction method includes one or more of a first extraction method and a second extraction method, the first extraction method extracting candidate video frames of a target frame type from the candidate video frame set, and the second extraction method sampling the candidate video frame set at a target interval;

[0177] Traversing each first video frame in the first video frame set, if a pixel difference between a current first video frame and a previous first video frame is greater than a preset pixel difference, or a timestamp difference between the current first video frame and the previous first video frame is greater than a first preset timestamp difference, then adding the current first video frame to the first target video frame set.

[0178] In one implementation, the processor 701 is specifically configured to:

[0179] Traversing each candidate video frame in the candidate video frame set, and obtaining a first pixel difference between a current candidate video frame and a previous candidate video frame;

[0180] filtering the candidate video frame set based on the first pixel difference and the frame type of the current candidate video frame to obtain a second video frame set;

[0181] Traversing each second video frame in the second video frame set, and obtaining a second pixel difference between a current second video frame traversed and a previous second video frame;

[0182] The second video frame set is screened based on the second pixel difference, the timestamp of the current second video frame, and the timestamp of the previous second video frame to obtain a second target video frame set.

[0183] In one implementation, the processor 701 is specifically configured to:

[0184] If the frame type of the current candidate video frame is an I frame type, and the first pixel difference is greater than a first preset difference, adding the current candidate video frame to the second video frame set;

[0185] If the frame type of the current candidate video frame is a P frame type, and the first pixel difference is greater than a second preset difference, adding the current candidate video frame to the second video frame set;

[0186] If the frame type of the current candidate video frame is a B frame type, and the first pixel difference is greater than a third preset difference, adding the current candidate video frame to the second video frame set;

[0187] The first preset gap is smaller than the second preset gap, and the second preset gap is smaller than the third preset gap.

[0188] In one implementation, the processor 701 is specifically configured to:

[0189] Determining a timestamp difference between the timestamp of the current second video frame and the timestamp of the previous second video frame;

[0190] If the timestamp difference is greater than a second preset timestamp difference, and the second pixel difference is greater than a fourth preset difference, adding the current second video frame to a second target video frame set;

[0191] If the timestamp difference is less than or equal to the second preset timestamp difference, and the second pixel difference is greater than a fifth preset difference, adding the current second video frame to the second target video frame set;

[0192] The fifth preset gap is greater than the fourth preset gap.

[0193] In one implementation, the processor 701 is specifically configured to:

[0194] For any initial video frame in the initial video frame set, obtaining a brightness value of each pixel in the any initial video frame;

[0195] If the brightness value of the pixel in any of the initial video frames meets a preset condition, then adding the any of the initial video frames to the candidate video frame set;

[0196] Among them, the preset condition includes any one of a first preset condition, a second preset condition and a third preset condition, the first preset condition is that the minimum brightness value among the brightness values ​​of each pixel point is greater than the preset minimum brightness value, the second preset condition is that the maximum brightness value among the brightness values ​​of each pixel point is less than the preset maximum brightness value, and the third preset condition is that the pixel ratio corresponding to any initial video frame is greater than the preset ratio, and the pixel ratio is the ratio of pixels in the brightness value of each pixel point that is less than the preset threshold.

[0197] An embodiment of the present application further provides a computer storage medium having program instructions stored therein. When the program is executed, the program may include some or all of the steps of the video frame extraction method in the embodiment corresponding to FIG. 2 or FIG. 4 .

[0198] It should be noted that for the aforementioned various method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0199] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0200] The present application also provides a computer program product or computer program, which includes program instructions that, when executed by a processor, implement some or all of the steps in the above-described method. For example, the program instructions are stored in a computer-readable storage medium. A processor of a computer device reads the program instructions from the computer-readable storage medium and executes the program instructions, causing the computer device to perform the steps performed in the above-described method embodiments.

[0201] The above is a detailed introduction to a video frame extraction method, device, computer equipment and medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A video frame extraction method, characterized in that: The method comprises: Acquire an initial video frame set to be processed, wherein the initial video frame set includes a plurality of initial video frames, and the plurality of initial video frames are continuous video frames in a video; Performing brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set; Obtaining encoding information corresponding to the candidate video frame set, the encoding information including a frame type and a timestamp corresponding to each candidate video frame in the candidate video frame set; The candidate video frame set is subjected to frame extraction processing based on the encoding information to obtain a target video frame set.

2. The method according to claim 1, characterized in that: The step of performing frame extraction processing on the candidate video frame set based on the encoding information to obtain a target video frame set includes: Using a first frame extraction strategy to extract frames from the candidate video frame set based on the encoding information to obtain a first target video frame set; Using a second frame extraction strategy to extract frames from the candidate video frame set based on the encoding information to obtain a second target video frame set; The first target video frame set and the second target video frame set are merged to obtain a target video frame set.

3. The method according to claim 2, characterized in that The step of adopting a first frame extraction strategy to extract frames from the candidate video frame set based on the encoding information to obtain a first target video frame set includes: Extracting the candidate video frame set according to a target extraction method to obtain a first video frame set; the target extraction method includes one or more of a first extraction method and a second extraction method, the first extraction method is to extract candidate video frames of a target frame type from the candidate video frame set, and the second extraction method is to sample the candidate video frame set at a target interval; Traverse each first video frame in the first video frame set, and if a pixel difference between a current first video frame and a previous first video frame is greater than a preset pixel difference, or a timestamp difference between the current first video frame and the previous first video frame is greater than a first preset timestamp difference, add the current first video frame to the first target video frame set.

4. The method according to claim 2, characterized in that: The adopting the second frame extraction strategy to extract frames from the candidate video frame set based on the encoding information to obtain a second target video frame set includes: Traversing each candidate video frame in the candidate video frame set, and obtaining a first pixel difference between a current candidate video frame traversed and a previous candidate video frame; filtering the candidate video frame set based on the first pixel difference and the frame type of the current candidate video frame to obtain a second video frame set; Traversing each second video frame in the second video frame set, and obtaining a second pixel difference between a current second video frame traversed and a previous second video frame; The second video frame set is screened based on the second pixel difference, the timestamp of the current second video frame, and the timestamp of the previous second video frame to obtain a second target video frame set.

5. The method according to claim 4, characterized in that The filtering the candidate video frame set based on the first pixel difference and the frame type of the current candidate video frame to obtain a second video frame set includes: If the frame type of the current candidate video frame is an I frame type, and the first pixel difference is greater than a first preset difference, adding the current candidate video frame to the second video frame set; If the frame type of the current candidate video frame is a P frame type, and the first pixel difference is greater than a second preset difference, adding the current candidate video frame to a second video frame set; If the frame type of the current candidate video frame is a B frame type, and the first pixel difference is greater than a third preset difference, adding the current candidate video frame to the second video frame set; The first preset gap is smaller than the second preset gap, and the second preset gap is smaller than the third preset gap.

6. The method according to claim 4, characterized in that The filtering the second video frame set based on the second pixel difference, the timestamp of the current second video frame, and the timestamp of the previous second video frame to obtain a second target video frame set includes: Determine a timestamp difference between the timestamp of the current second video frame and the timestamp of the previous second video frame; If the timestamp difference is greater than a second preset timestamp difference, and the second pixel difference is greater than a fourth preset difference, adding the current second video frame to a second target video frame set; If the timestamp gap is less than or equal to the second preset timestamp gap, and the second pixel gap is greater than a fifth preset gap, adding the current second video frame to the second target video frame set; Among them, the fifth preset gap is greater than the fourth preset gap.

7. The method according to claim 1, characterized in that The performing brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set includes: For any initial video frame in the initial video frame set, obtain the brightness value of each pixel in the any initial video frame; If the brightness value of the pixel point in any of the initial video frames meets the preset condition, then adding the any of the initial video frames to the candidate video frame set; Among them, the preset condition includes any one of a first preset condition, a second preset condition and a third preset condition, the first preset condition is that the minimum brightness value among the brightness values ​​of each pixel point is greater than a preset minimum brightness value, the second preset condition is that the maximum brightness value among the brightness values ​​of each pixel point is less than a preset maximum brightness value, and the third preset condition is that the pixel ratio corresponding to any initial video frame is greater than a preset ratio, and the pixel ratio is the ratio of pixels whose brightness values ​​of each pixel point are less than a preset threshold.

8. A video frame extraction device, characterized in that: include: A first acquisition unit is used to acquire an initial video frame set to be processed, wherein the initial video frame set includes a plurality of initial video frames, and the plurality of initial video frames are continuous video frames in a video; A filtering unit, configured to perform brightness filtering on each initial video frame in the initial video frame set to obtain a candidate video frame set; A second acquisition unit, configured to acquire encoding information corresponding to the candidate video frame set, wherein the encoding information includes a frame type and a timestamp corresponding to each candidate video frame in the candidate video frame set; A frame extraction unit is used to extract frames from the candidate video frame set based on the encoding information to obtain a target video frame set.

9. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images