Static frame filtering method, system and storage medium
By using the static frame filtering method of HSV color model and dual-channel mechanism in video processing, the problem of difficult balance between speed and injury rate when eliminating static frames is solved, and fast and robust static frame filtering is achieved, reducing errors Processing ratio, suitable for video surveillance and other fields.
Patent Information
- Application Number
- CN202210526733.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-16
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-05-16
AI Technical Summary
In video processing, traditional methods are difficult to balance the processing speed and the injury rate when removing static frames. They not only need to quickly process a large number of repeated frames, but also avoid accidentally deleting valuable information, especially in unexpected situations such as light changes.
Using a dual-channel mechanism based on the HSV color model, the hue, saturation or brightness value of the video frame is determined, and the dual-channel is established to filter out static frames. The overall proportion and preset classification range are used to determine whether the frame is a static frame. And delete the static frames in the video integration step and retain the valid frames.
It realizes fast and robust static frame filtering, reduces the proportion of error processing, reduces the interference of unexpected situations, and improves the processing efficiency in the video surveillance field.
Smart Images

Figure CN114821442B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of video stream processing technology, in particular to the field of mobile device technology, and specifically to a static frame filtering method, system and storage medium. Background Art
[0002] When processing video images, there are often many repeated static frames. Most of these static frames have repeated content and do not contain important information, but they take up storage space and computing space. Before performing algorithm operations, they often need to be removed in advance, otherwise it will greatly waste relevant computing resources. That is, static frame filtering is: in the field of video processing, in order to reduce the waste of computing resources caused by repeated video content, it is often necessary to quickly filter repeated, unchanged video images, and then only retain video images with significant changes. For example, when roadside cameras are used for security monitoring, they often automatically delete unchanged video content and only retain images with pedestrians or cars passing by.
[0003] However, there is often a dilemma in removing static frames. On the one hand, it should be as simple and fast as possible, otherwise the processing time and resource consumption will be very huge when there are many repeated static frames, and even exceed the original processing cost; on the other hand, it is necessary to avoid removing valuable information and causing "accidental injury".
[0004] Traditional methods only consider the processing of individual frames, which can be easily interfered with by mistake (for example, birds that flash quickly may also be captured); but if the entire video information is processed, the calculation amount is usually large and the processing is relatively slow. Summary of the invention
[0005] The embodiment of the present application provides a static frame filtering method, system and storage medium, which utilizes the judgment of the overall frame state in the channel to realize the judgment of whether a single frame is a static frame. This judgment method is fast and more robust, and is not easily affected by unexpected changes (such as light blocking) to affect the overall judgment result, and more effectively screens out static frames and deletes them. While maintaining fast processing, the present application maintains an extremely low misprocessing ratio, reduces interference caused by unexpected situations, and solves a difficulty in the video field.
[0006] The present invention provides a static frame filtering method, comprising the following steps:
[0007] The video frame data processing step processes the video frames of the monitored area collected by the monitoring device frame by frame, wherein the judgment parameters corresponding to the video frames include one or more combinations of hue value, saturation value or brightness value, and the values of the corresponding video frames based on the judgment parameters are obtained and recorded using the HSV color model, the RNG color model or the RGB color model; the values of the judgment parameters are the color values of the corresponding video frames;
[0008] A dual-channel establishment step is to establish a dual channel for video frames and corresponding determination parameter values, wherein the dual channel has a color value range including a target video frame, the dual channel has a fixed number of cached video frames, receives input video frames from its input end, and outputs video frames from its output end; the dual channel has a preset classification ratio range;
[0009] In the step of determining the overall proportion, the video frames enter the dual channel from the input end of the dual channel frame by frame in sequence, and when the number of video frames entering the dual channel is the largest, the proportion of the values of the corresponding determination parameters in all frames in the current dual channel within the color value range of the target video frame is determined and output;
[0010] In the single-frame classification step, if the current dual-channel output ratio is within the preset classification ratio range of the dual-channel, the last video frame input in the current dual-channel is recorded as a valid frame; otherwise, it is regarded as a static frame;
[0011] The step of determining and classifying is repeated, the first video frame input into the current dual channels is transferred out from the output end, and the step of determining the overall proportion and the step of classifying the single frame are repeated for the next video frame input into the dual channels, until all the video frames are classified; and
[0012] In the video integration step, static frames in the video frames output by the dual channels are deleted, and valid frames are stitched together to form a new video.
[0013] In some embodiments, before the video frame data processing step, the method further includes:
[0014] A monitoring device setting step, wherein a monitoring device is set at a location corresponding to a monitored area; and
[0015] The step of setting the numerical range of the determination parameter of the monitored area is to set the numerical range of the determination parameter of the monitored area according to the monitoring video screen content of the monitored area.
[0016] In some embodiments, in the step of establishing a dual channel, the preset classification ratio range of the dual channel has an upper limit value and a lower limit value; the upper limit value and the lower limit value can be set manually.
[0017] In some embodiments, in the step of determining the overall proportion, when the number of video frames entering the dual channels is less than the maximum number of video frames in the dual channels, filling is continued until the maximum number is reached.
[0018] In some embodiments, in the step of determining the overall proportion, all video frames entering the dual channels need to obtain the value of the determination parameter corresponding to each video frame by the color model.
[0019] In some embodiments, in the single frame classification step, the single frame classification step specifically includes:
[0020] The interference frame classification step, if the proportion of the current dual-channel output is less than the upper limit value, then the last video frame input in the current dual-channel is recorded as an interference frame, and the static frame includes the interference frame;
[0021] Invalid frame classification step: if the proportion of the current dual-channel output is greater than the lower limit, the last video frame input in the current dual-channel is recorded as an invalid frame, and the static frame includes the invalid frame.
[0022] In some embodiments, the video integration step is performed synchronously when the output ends of the dual channels output video frames.
[0023] The present application also provides a system, comprising: a memory and a processor; the memory stores a computer program, and the processor is used to execute the computer program to implement each step in the static frame filtering method described in any one of the above items.
[0024] The present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to implement each step of the static frame filtering method described in any one of the above items.
[0025] The advantages of this application are: on the basis of maintaining fast processing, it also maintains a very low proportion of misprocessing, reducing interference caused by unexpected situations. This application can be widely used in a variety of fields that require video surveillance, such as security detection, traffic control, smart parking, etc. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The technical solution and other beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.
[0027] Figure 1 A flowchart of a static frame filtering method provided in an embodiment of the present application.
[0028] Figure 2 A schematic diagram of the dual-channel processing mechanism provided in an embodiment of the present application.
[0029] Figure 3 A flowchart of the single frame classification steps provided in an embodiment of the present application.
[0030] Figure 4 The embodiment of the present application provides a schematic diagram of a newly-entered frame that meets the HSV judgment and enters the dual channel.
[0031] Figure 5The embodiment of the present application provides a schematic diagram of a newly entered frame that does not comply with the HSV judgment and enters the dual channel.
[0032] Figure 6 A schematic diagram of a frame marked as a valid frame provided in an embodiment of the present application. DETAILED DESCRIPTION
[0033] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0034] For details, please refer to Figure 1 , an embodiment of the present application provides a static frame filtering method, comprising the following steps S1-S8.
[0035] S1, a monitoring device setting step, wherein a monitoring device is set at a location corresponding to a monitored area.
[0036] For example, if you choose to monitor a red bicycle lane, you would want to only keep the video recording of bicycles or pedestrians passing by, and delete the static frames where no one passes by.
[0037] When monitoring, choose a camera position that can capture the monitored content and has a strong color feature. For example, the red area has a more obvious color feature and can cover passing bicycles and pedestrians. Therefore, the camera lens of the monitoring device can be set to face the red bicycle lane, and the focal length can be adjusted to focus only on the red bicycle lane area to form the monitored area.
[0038] In some cases, such as when the color features of a specific monitoring area are not obvious enough, color markings can be sprayed artificially.
[0039] Selecting an area with more obvious color features as the monitored area can facilitate the identification of the background image, and it is also easier to set the judgment parameters of the monitored area later. It can also avoid the subtle influence caused by light changes in the morning and evening.
[0040] S2, a step of setting a numerical range of the determination parameter of the monitored area, setting a numerical range of the determination parameter of the monitored area according to the monitoring video screen content of the monitored area.
[0041] The determination parameter includes one or more combinations of hue value (H), saturation value (S) or brightness value (V).
[0042] In this embodiment, the hue value (H) is selected as the judgment parameter of the monitored area, which is easy to identify and obtain. Moreover, as mentioned above, the red color of the red bicycle lane is easy to identify. By processing each picture frame using the HSV method and judging the HSV color interval of each pixel in the picture, the hue value (H) of the red bicycle lane can be obtained. In the monitoring video picture, the main shooting picture is red, so the HSV method is used to mainly take the hue (value), and the red hue value range is 0-10 or 156-180. That is, if the hue value of the picture after HSV processing is 0-10 or 165-180, it means that the picture frame is mainly red.
[0043] S3, a video frame data processing step, processing the video frames of the monitored area collected by the monitoring device frame by frame, wherein the judgment parameters corresponding to the video frames include one or more combinations of hue value, saturation value or brightness value, and the values of the corresponding video frames based on the judgment parameters are obtained and recorded using a color model such as HSV, RNG or RGB. The values of the judgment parameters are color values of the corresponding video frames.
[0044] HSV (Hue, Saturation, Value) is a common image processing method in the field of computer vision. It is a color space created by AR Smith in 1978 based on the intuitive characteristics of color, also known as the Hexcone Model. Through the HSV method, the corresponding hue (H), saturation (S), and brightness (V) values in the image can be quickly extracted, where: hue values are 0-180; saturation values are 0-255; brightness values are 0-255.
[0045] The present application preferably adopts the HSV color model to obtain the value of the corresponding judgment parameter, which is relatively simple and feasible. That is, HSV processing is performed on each video frame of the video to be processed, and at least one of its hue (H), saturation (S), and brightness (V) values is obtained and recorded, and the HSV processing result of the video frame is associated with the frame number of the frame.
[0046] S4, a dual-channel establishment step, establishing a dual channel for video frames and their corresponding judgment parameter values, wherein the dual channel has a color value range that includes the target video frame, the dual channel caches a fixed number of video frames, receives input video frames from its input end, and outputs video frames from its output end; the dual channel presets a classification proportion range.
[0047] like Figure 2As shown in the figure, the dual-channel processing mechanism is described, and the HSV results after frame-by-frame processing are sequentially input into the cache dual channels. The dual channels only retain the results of a fixed number of frames, such as 100 frames. When the number of frames in the previous channel is less than 100 frames, it will continue to fill until it reaches 100 frames; after reaching it, every time a new frame enters from the input end, a frame will be transferred from the output end on the other side. This cycle is repeated until all video frames to be processed have been judged.
[0048] The dual-channel preset classification ratio range is selected to be 20%-80%, which can effectively avoid, for example, interference frames with a ratio below 20% and still picture frames with a ratio above 80%. That is, the dual-channel preset classification has an upper limit and a lower limit; the upper limit is 20% and the lower limit is 80%. The upper limit and the lower limit can be manually set according to actual practical conditions, and the two limit values can be adjusted according to actual conditions, for example, 10%-90%.
[0049] S5, a step of determining the overall proportion, wherein the video frames enter the dual channel from the input end of the dual channel frame by frame in sequence, and when the number of video frames entering the dual channel is the largest, the proportion of the values of the corresponding determination parameters in all frames in the current dual channel within the color value range of the target video frame is determined, and the proportion is output.
[0050] When the number of video frames entering the dual channels is less than the maximum number of video frames in the dual channels, filling is continued until the maximum number is reached.
[0051] Taking the previous case of determining the red area as an example, determine how many proportions of the 100 frames in the dual channel meet the HSV determination range, such as the hue value of 0-10 or 165-180, and record the proportion of frames that meet this range to the total number of frames. For example, among the 100 frames in the current dual channel, 70 frames meet the red hue determination, then the output result of this determination is 70%.
[0052] It can be understood that the role of this ratio is to determine the overall color ratio of the 100 frames in the current dual channel. Based on the background color of red, if people or objects of other colors enter the detection area, part of the red area will be blocked, and the corresponding picture frame will be input into the dual channel, which will affect the output result of the overall ratio. Therefore, according to the ratio of the red area, it can be distinguished whether a person or object has entered the detection area.
[0053] When, for example, the vehicle exists for a very short time, the number of corresponding video frames (which means the same as picture frames) is small, and the impact on the output result of the overall proportion is small, it can be regarded as an interference frame.
[0054] In order to exclude the content of these interfering frames, only when a large number of video frames have a significant impact on the output result of the overall ratio, they are identified as valid frames. Therefore, this application designs a method to define the state of the last video frame input in the current dual channel based on the result of the overall tone ratio, which can effectively determine the content of each picture frame.
[0055] Although this judgment method has a certain lag, the frequency of picture acquisition is high. Even if some dynamic pictures are identified as static frames or invalid frames, this does not affect the overall demand for the output picture. On the contrary, this judgment method only needs to judge the tone ratio, and the processing speed is very fast, which reduces the demand for software and hardware, reduces costs and improves processing speed.
[0056] Furthermore, in order to ensure that the video frames entering the dual channels are all data processed by the video frame data processing step S3, in the overall proportion determination step S5, all video frames entering the dual channels need to obtain the value based on the determination parameter corresponding to each video frame by the color model, so as to determine whether all video frames have been determined before the threshold.
[0057] S6, single frame classification step, if the current dual-channel output ratio is within the preset classification ratio range of the dual channels, then the last video frame input in the current dual channels is recorded as a valid frame; otherwise, it is recorded as a static frame.
[0058] In the single frame classification step S6, a hue determination step for each video frame may also be included, so as to assist in determining whether each video frame is a valid frame or a static frame.
[0059] like Figure 3 As shown, the single frame classification step S6 specifically includes:
[0060] S61, interference frame classification step, if the proportion of the current dual-channel output is less than the upper limit, then the last video frame input in the current dual-channel is recorded as an interference frame, and the static frame includes the interference frame;
[0061] S62, invalid frame classification step, if the proportion of the current dual-channel output is greater than the lower limit, then the last video frame input in the current dual-channel is recorded as an invalid frame, and the static frame includes the invalid frame.
[0062] The invalid frames are actually the images of the monitored area, which are generally red in color. Since the video frames of the monitored area are extracted by extracting the values that meet the judgment parameters (that is, the value range of the judgment parameters of the monitored area, the red hue value range is 0-10 or 156-180), at this time, the number of video frames that meet the judgment parameters in the dual channels accounts for more than 80%. Therefore, the most recently shot video frame is judged as an invalid frame and classified as a static frame, which can be deleted later.
[0063] For example, when small area colors such as birds and plastic bags enter the detection area, even if the corresponding picture frames are large in number, the color proportion in each frame is very small, which does not meet the range of the video frame judgment parameters and has little impact on the output result of the overall proportion in the dual channels, and can be regarded as an invalid frame.
[0064] Although the first 99 video frames entering the dual channel for the first time are not classified and determined, these first 99 video frames do not affect the demand for viewing dynamic images. These first 99 video frames can be chosen to be retained or deleted.
[0065] S7, looping the determination and classification steps, transferring the first video frame input into the current dual channels from its output end, repeating the overall proportion determination step and the single frame classification step for the next video frame newly input into the dual channels, until all video frames obtain classification results.
[0066] Repeat this process and introduce a new frame. At this time, the last judged frame becomes the 2nd frame, and the original 100th frame is transferred out of the dual channel. At this time, the number of frames in the new dual channel that meet the HSV judgment is counted, and this cycle repeats.
[0067] as follows Figure 4 , if the new frame is a frame that meets the HSV judgment, the proportion of frames that meet the judgment in the entire channel increases; otherwise, Figure 5 , if the new incoming frame does not meet the HSV judgment, the proportion of frames that meet the judgment in the entire channel will decrease; Figure 6 Until the number of frames that do not meet the HSV judgment exceeds a certain number, the last frame that enters the dual channel is marked as a valid frame, even if the last frame that enters the dual channel is actually a static picture, it will be marked as a valid frame.
[0068] S8, video integration step, deleting static frames in the video frames output by the dual channels, and combining valid frames to form a new video.
[0069] The picture frames previously marked as static frames are deleted, only valid frames are retained, and the valid frames are stitched into new valid videos.
[0070] The retained valid frames contain the corresponding time information, which is convenient for checking the time nodes of the corresponding dynamic drawings. Although the end of each section marked as a valid frame is actually a still picture, and the end of each section marked as a static frame is actually a dynamic picture, due to the large number of video frames, this difference can be ignored.
[0071] Moreover, the subsequent input of dual-channel video frames causes the value ratio of the corresponding judgment parameters to change. The overall change is used to determine the type of the latest dual-channel video frame input, which can effectively exclude the influence of a single frame as a dynamic picture (such as light blocking) on the overall judgment result. It also avoids the judgment method of each video frame in the video stream, improving the processing efficiency.
[0072] In some embodiments, the video integration step is performed synchronously when the output end of the dual channel outputs the video frame. The dual channel receives new input while outputting the determined video frame, which greatly improves the processing efficiency.
[0073] Therefore, this method uses the judgment of the overall frame status in the channel to determine whether a single frame is a static frame. This judgment method is fast and more robust, and is not easily affected by unexpected changes (such as light blocking) to affect the overall judgment result, and it is more effective in screening out static frames and deleting them.
[0074] The present application also provides a system, comprising: a memory and a processor; the memory stores a computer program, and the processor is used to execute the computer program to implement each step in the static frame filtering method described in any one of the above items.
[0075] The present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to implement each step of the static frame filtering method described in any one of the above items.
[0076] The advantages of this application are: for specific video scenes, a static frame filtering method combining color processing and dual-channel mechanism is provided. This method maintains a very low misprocessing ratio while maintaining fast processing, reducing interference caused by unexpected situations. This application can be widely used in a variety of fields that require video surveillance, such as security detection, traffic control, smart parking, etc.
[0077] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0078] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the technical solution and core idea of the present application. Ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A static frame filtering method, characterized in that: Includes steps: The video frame data processing step processes the video frames of the monitored area collected by the monitoring device frame by frame, wherein the judgment parameters corresponding to the video frames include one or more combinations of hue value, saturation value or brightness value, and the values of the corresponding video frames based on the judgment parameters are obtained and recorded using the HSV color model, the RNG color model or the RGB color model; the values of the judgment parameters are the color values of the corresponding video frames; A dual-channel establishment step is to establish a dual channel for video frames and corresponding determination parameter values, wherein the dual channel has a color value range including a target video frame, the dual channel has a fixed number of cached video frames, receives input video frames from its input end, and outputs video frames from its output end; the dual channel has a preset classification ratio range; In the step of determining the overall proportion, the video frames enter the dual channel from the input end of the dual channel frame by frame in sequence, and when the number of video frames entering the dual channel is the largest, the proportion of the values of the corresponding determination parameters in all frames in the current dual channel within the color value range of the target video frame is determined and output; In the single-frame classification step, if the current dual-channel output ratio is within the preset classification ratio range of the dual-channel, the last video frame input in the current dual-channel is recorded as a valid frame; otherwise, it is regarded as a static frame; The determination and classification steps are looped, a video frame first input into the current dual channels is transferred out from its output end, and the overall proportion determination step and the single frame classification step are repeated for the next video frame newly input into the dual channels until all video frames obtain classification results; as well as In the video integration step, static frames in the video frames output by the dual channels are deleted, and valid frames are stitched together to form a new video.
2. The static frame filtering method according to claim 1, characterized in that: Before the video frame data processing step, the method further includes: A monitoring device setting step, wherein a monitoring device is set at a location corresponding to a monitored area; and The step of setting the numerical range of the determination parameter of the monitored area is to set the numerical range of the determination parameter of the monitored area according to the monitoring video screen content of the monitored area.
3. The static frame filtering method according to claim 1, characterized in that: In the step of establishing the dual channels, the dual channel preset classification has an upper limit value and a lower limit value; the upper limit value and the lower limit value can be set manually.
4. The static frame filtering method according to claim 1, characterized in that: In the step of determining the overall proportion, when the number of video frames entering the dual channels is less than the maximum number of video frames in the dual channels, filling is continued until the maximum number is reached.
5. The static frame filtering method according to claim 1, characterized in that: In the step of determining the overall proportion, all video frames entering the dual channels need to obtain a value based on the determination parameter corresponding to each video frame by the color model.
6. The static frame filtering method according to claim 3, characterized in that: The single frame classification step specifically includes: The interference frame classification step, if the proportion of the current dual-channel output is less than the upper limit value, then the last video frame input in the current dual-channel is recorded as an interference frame, and the static frame includes the interference frame; Invalid frame classification step: if the proportion of the current dual-channel output is greater than the lower limit, the last video frame input in the current dual-channel is recorded as an invalid frame, and the static frame includes the invalid frame.
7. The static frame filtering method according to claim 1, characterized in that: The video integration step is performed synchronously when the output ends of the dual channels output video frames.
8. A system, characterized in that: include: Memory and processor; The memory stores a computer program, and the processor is used to execute the computer program to implement each step in the static frame filtering method according to any one of claims 1 to 7.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement each step in the static frame filtering method according to any one of claims 1 to 7.
