Rainy day identification method and device and camera

Abstract

The invention discloses a rainy day identification method and device and a camera. The method comprises the following steps: acquiring a target video stream of a scene to be monitored; polling each video frame in the target video stream, and determining a raindrop pixel block in the current video frame; judging whether the corresponding video frame is a rainy day video frame according to the raindrop pixel block in each video frame in the target video stream, and counting the number of the rainy day video frames in the target video stream; and when the total number of the raindrop pixel blocks in the target video stream and / or the frame number of the rainy day video frames meet a first preset condition, determining that the scene to be monitored is a rainy day scene. Through the technical scheme provided by the embodiment of the invention, whether any to-be-monitored scene is a rainy day scene or not can be accurately identified based on the target video stream.

Description

Technical Field

[0001] The present invention relates to the technical field of cameras, and in particular to a rainy day recognition method, device and camera. Background Art

[0002] In the field of intelligent video surveillance, for some fine-grained tasks and special image tasks, rainy days will destroy the original structure of the image, causing a large number of false detections or affecting the accuracy of subsequent image operations. At present, the industry mainly adopts two solutions: the first solution is to remove rain from the captured image when the monitoring scene is determined to be a rainy scene, and the second solution is to adjust the operation of subsequent image processing tasks in a timely manner when the monitoring scene is determined to be a rainy scene. Therefore, accurate rainy day recognition is a prerequisite for the subsequent fine-grained task development and image restoration.

[0003] In the related art, three rainy day recognition methods are provided: the first is rainy day recognition based on regional marking, which mainly calculates the gradient distribution variance and regional brightness of the two marked areas, and then calculates the covariance norm and compares it with a preset threshold to determine whether it is a rainy day. This scheme only uses some manually marked areas for detection and judgment, and the error is relatively large. It does not take into account the global image features. It is very sensitive to the manually selected marked areas, which cannot be too small or too large, and the area with a large image gradient cannot be selected. In addition, the preset threshold involved is highly related to the selected area range. For different backgrounds such as buildings, roads, and sky, the preset threshold setting is also different. The second is to use road monitoring for rainy day recognition. This scheme can only be used in road scenes and when vehicles need to pass, and the scene restrictions are large. The third is to identify rainy days based on single-frame images. First, a suitable area is selected from the video image for background detection, and then each pixel is accumulated and judged as a foreground to generate a foreground survival map. Finally, the number and distribution of foreground survival maps are used to determine whether it is a rainy day. This solution relies too much on the selection of the detection area and performs rainy day detection based on a single-frame image, resulting in low accuracy of the detection results. Summary of the invention

[0004] The present invention provides a rainy day recognition method, device and camera for accurately recognizing rainy day scenes.

[0005] According to one aspect of the present invention, a rainy day recognition method is provided, comprising:

[0006] Obtain the target video stream of the scene to be monitored;

[0007] Polling each video frame in the target video stream to determine a raindrop pixel block in a current video frame;

[0008] Determining whether a corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream, and counting the number of rainy day video frames in the target video stream;

[0009] When the total number of raindrop pixel blocks in the target video stream and / or the number of the rainy day video frames meets a first preset condition, it is determined that the scene to be monitored is a rainy day scene.

[0010] According to another aspect of the present invention, there is provided a rainy day recognition device, comprising:

[0011] A target video stream acquisition module is used to acquire a target video stream of a scene to be monitored;

[0012] A raindrop pixel block determination module, used to poll each video frame in the target video stream to determine a raindrop pixel block in a current video frame;

[0013] A rainy day video frame judgment module, used to judge whether the corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream, and count the number of rainy day video frames in the target video stream;

[0014] The rainy scene recognition module is used to determine that the scene to be monitored is a rainy scene when the total number of raindrop pixel blocks in the target video stream and / or the number of frames of the rainy video frames meet a first preset condition.

[0015] According to another aspect of the present invention, there is provided a camera, the camera comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the rainy day identification method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the rainy day recognition method described in any embodiment of the present invention when executed.

[0020] The rainy day recognition scheme of the embodiment of the present invention includes: obtaining a target video stream of a scene to be monitored; polling each video frame in the target video stream to determine the raindrop pixel block in the current video frame; judging whether the corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream, and counting the number of rainy day video frames in the target video stream; when the total number of raindrop pixel blocks in the target video stream and / or the number of rainy day video frames meet a first preset condition, determining that the scene to be monitored is a rainy day scene. Through the technical scheme provided by the embodiment of the present invention, it is possible to accurately identify whether any scene to be monitored is a rainy day scene based on the target video stream.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a rainy day identification method provided according to Embodiment 1 of the present invention;

[0024] Figure 2 is a structural schematic diagram of a rainy day recognition device provided according to Embodiment 2 of the present invention;

[0025] Figure 3 It is a structural schematic diagram of a camera for implementing the rainy day recognition method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Embodiment 1

[0029] Figure 1 A flowchart of a rainy day identification method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of identifying whether the monitoring scene of a camera is a rainy day scene. The method can be executed by a rainy day identification device. The rainy day identification device can be implemented in the form of hardware and / or software. The rainy day identification device can be configured in the camera. Figure 1 As shown, the method includes:

[0030] S110: Obtain a target video stream of the scene to be monitored.

[0031] The camera may be a ball camera, a gun camera, or a hemispherical camera, and the embodiment of the present invention does not limit the type of the camera. In the embodiment of the present invention, the camera obtains a target video stream of the scene to be monitored, wherein the target video stream contains at least two video frames.

[0032] S120: poll each video frame in the target video stream to determine a raindrop pixel block in a current video frame.

[0033] In an embodiment of the present invention, for each video frame in the target video stream, a raindrop pixel block in the current video frame is determined, that is, the raindrop pixel blocks in each video frame in the target video stream are determined respectively. The raindrop pixel block can be understood as a raindrop pixel area composed of adjacent raindrop pixel points in the video frame. Exemplarily, raindrops have a great influence on the brightness of the image. Therefore, each video frame in the target video stream is polled, and for each pixel point in the current video frame, a corresponding Y component Gaussian distribution model is established according to the Y component corresponding to the current pixel point of each video frame in the target video stream; the Y component mean and Y component standard deviation of the current pixel point in the video frame before the current video frame in the target video stream are determined according to the Y component Gaussian distribution model; when the Y component, Y component mean and Y component standard deviation of the current pixel point meet certain preset conditions, the current pixel point can be determined to be a raindrop pixel point. The adjacent raindrop pixel points in the current video frame are connected to generate a raindrop pixel block. Optionally, when the width of the connected generated raindrop pixel block is greater than a preset width threshold (such as 16 pixels), the raindrop pixel block can be excluded. There may be one or more raindrop pixel blocks in each video frame. Of course, there may be no raindrop pixel blocks in the video frame. It should be noted that the size, number and position of the raindrop pixel blocks contained in each video frame may be the same or different, and this is not limited in the embodiment of the present invention.

[0034] Each video frame in the target video stream may be divided into at least two grid areas according to the resolution of the video frame, wherein the grid areas divided into the video frames have the same size. For example, the size of each grid area of ​​the video frame is s. std *s std The raindrop pixel block included in the video frame may be in one grid area or may span one or more grid areas. Therefore, a target raindrop pixel block spanning at least two grid areas is determined from all the raindrop pixel blocks in the video frame.

[0035] S130 , judging whether a corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream, and counting the number of rainy day video frames in the target video stream.

[0036] In an embodiment of the present invention, for each video frame in the target video stream, it is determined whether the current video frame is a rainy day video frame based on the raindrop pixel blocks in the current video frame. Exemplarily, when the number of raindrop pixel blocks contained in the video frame is greater than a preset number threshold, the video frame is determined to be a rainy day video frame. Also exemplarily, when the ratio of the sum of the areas of the raindrop pixel blocks contained in the video frame to the area of ​​the video frame is greater than a preset ratio threshold, the video frame is determined to be a rainy day video frame. It should be noted that the embodiment of the present invention does not limit the method of determining whether a video frame is a rainy day video frame based on the raindrop pixel blocks. The number of rainy day video frames in the target video stream is counted.

[0037] Optionally, judging whether the corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream respectively includes: determining the number of raindrop pixel blocks in the current video frame for each video frame in the target video stream; when the number of raindrop pixel blocks is greater than a preset number threshold, determining the center point position information of each raindrop pixel block in the current video frame respectively; for each raindrop pixel block in the current video frame, determining the target distance between the current raindrop pixel block and the target raindrop pixel block according to the center point position information; wherein the target raindrop pixel block is the raindrop pixel block that is closest to the current raindrop pixel block among the raindrop pixel blocks; when the target distance corresponding to each raindrop pixel block satisfies a fourth preset condition, determining the current video frame is a rainy day video frame. The advantage of such a setting is that it can accurately judge whether each video frame in the target video stream is a rainy day video frame.

[0038] Exemplarily, the number of raindrop pixel blocks contained in each video frame in the target video stream is determined respectively. When the number of raindrop pixel blocks is greater than a preset number threshold, the center point position information of each raindrop pixel block in the video frame is obtained respectively. For example, the center point coordinates of the circumscribed rectangle of the raindrop pixel block can be used as the center point position information of the raindrop pixel block. For each raindrop pixel block in the video frame, the distance between the current raindrop pixel block and each other raindrop pixel block is calculated according to the center point position information, and the raindrop pixel block with the smallest distance to the current raindrop pixel block is used as the target raindrop pixel block. It can be understood that the target raindrop pixel block is the raindrop pixel block that is closest to the current raindrop pixel block among the raindrop pixel blocks in the video frame. The distance between the current raindrop pixel block and the target raindrop pixel block is used as the target distance, that is, the minimum distance is used as the target distance. Among them, the Euclidean distance between the center points of the circumscribed rectangles of two raindrop pixel blocks can be used as the distance between the two raindrop pixel blocks. In this way, the target distance between each raindrop pixel block in the video frame and its closest target raindrop pixel block can be determined. When the target distances corresponding to the raindrop pixel blocks meet the preset conditions, the current video frame is determined to be a rainy day video frame. For example, the average of the target distances corresponding to the raindrop pixel blocks is calculated, and when the average is greater than a preset distance threshold, the current video frame is determined to be a rainy day video frame.

[0039] Optionally, when the target distances corresponding to the raindrop pixel blocks meet a fourth preset condition, determining that the current video frame is a rainy day video frame includes: calculating a distance mean of the target distances corresponding to the raindrop pixel blocks; when the distance mean meets the following condition, calculating a distance standard deviation of the target distances corresponding to the raindrop pixel blocks; When the distance standard deviation satisfies the following conditions, it is determined that the current video frame is a rainy day video frame: Among them, σ min represents the distance standard deviation, d min represents the mean of the distances, d i represents the target distance corresponding to the i-th raindrop pixel block, n represents the total number of raindrop pixel blocks contained in the current video frame, w represents the width of the current video frame, h represents the length of the current video frame, and s std Indicates the length of the pre-divided grid area in the current video frame.

[0040] Exemplarily, the mean distance d of the target distance corresponding to each raindrop pixel block in the video frame is calculated min ,in, When the distance to the mean d min Greater than When , the distance standard deviation of the target distance is calculated according to the target distance corresponding to each raindrop pixel block, where the distance standard deviation when , it is determined that the video frame is a rainy day video frame.

[0041] Optionally, the method further includes: when the number of the raindrop pixel blocks is less than the preset number threshold, determining that the current video frame is a non-rainy day video frame; or when the distance mean satisfies the following conditions, determining that the current video frame is a non-rainy day video frame: Alternatively, when the distance standard deviation satisfies the following conditions, it is determined that the current video frame is a non-rainy day video frame:

[0042] S140: When the total number of raindrop pixel blocks in the target video stream and / or the number of rainy video frames meets a first preset condition, determine that the scene to be monitored is a rainy scene.

[0043] In an embodiment of the present invention, the number of raindrop pixel blocks contained in each video frame in the target video stream is counted and summed to obtain the total number of all raindrop pixel blocks in the target video stream. When the total number of all raindrop pixel blocks in the target video stream and / or the number of frames of all rainy day video frames in the target video stream meet a preset condition, the scene to be monitored is determined to be a rainy day scene. For example, when the total number of raindrop pixel blocks is greater than a preset total number threshold, the scene to be monitored is determined to be a rainy day scene. For another example, when the number of frames of rainy day video frames is greater than a preset frame number threshold, the scene to be monitored is determined to be a rainy day scene. Optionally, when the total number of raindrop pixel blocks is greater than a preset total number threshold and the number of frames of rainy day video frames is greater than a preset frame number threshold, the scene to be monitored is determined to be a rainy day scene.

[0044] Optionally, when the total number of raindrop pixel blocks in the target video stream and the number of frames of the rainy video frames meet a first preset condition, determining that the scene to be monitored is a rainy scene includes: when the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas corresponding to all video frames in the target video stream is greater than a first preset threshold, and the ratio of the number of frames of the rainy video frames to the number of frames of the video frames contained in the target video stream is greater than a second preset threshold, determining that the scene to be monitored is a rainy scene; or, when the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas corresponding to all video frames in the target video stream is less than or equal to the first preset threshold, and the ratio of the number of frames of the rainy video frames to the number of frames of the video frames contained in the target video stream is greater than a third preset threshold, determining that the scene to be monitored is a rainy scene; wherein the third preset threshold is greater than the second preset threshold.

[0045] Exemplarily, the number of grid areas of each video frame in the target video stream is summed to obtain the total number of grid areas contained in all video frames in the target video stream. When the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas is greater than the first preset threshold, and the ratio of the number of frames of rainy day video frames to the number of frames of video frames contained in the target video stream is greater than the second preset threshold, the scene to be monitored is determined to be a rainy day scene. Alternatively, when the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas is less than or equal to the first preset threshold, and the ratio of the number of frames of rainy day video frames to the number of frames of video frames contained in the target video stream is greater than the third preset threshold, the scene to be monitored is determined to be a rainy day scene. For example, the first preset threshold may be 50%, the second preset threshold may be 0.3, and the third preset threshold may be 0.6. If the target video stream contains N video frames, it can be understood that when the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas is greater than 50%, and the number of frames of rainy day video frames is greater than 0.3N, the scene to be monitored is determined to be a rainy day scene. Alternatively, when the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas is less than 50%, and the number of rainy day video frames is greater than 0.6N, it is determined that the scene to be monitored is a rainy day scene.

[0046] The rainy day recognition method of the embodiment of the present invention includes: obtaining a target video stream of a scene to be monitored; polling each video frame in the target video stream to determine the raindrop pixel block in the current video frame; judging whether the corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream, and counting the number of rainy day video frames in the target video stream; when the total number of raindrop pixel blocks in the target video stream and / or the number of rainy day video frames meet a first preset condition, determining that the scene to be monitored is a rainy day scene. Through the technical solution provided by the embodiment of the present invention, it is possible to accurately identify whether any scene to be monitored is a rainy day scene based on the target video stream.

[0047] In some embodiments, the target video stream is a video stream in a luminance and chrominance YUV format; determining the raindrop pixel block in the current video frame includes: for each pixel in the current video frame, when the current pixel is determined to be the background according to the Y component of the current pixel and the current pixel is determined to be the foreground according to the UV component of the current pixel, determining the current pixel as a raindrop pixel; connecting adjacent raindrop pixel points in the current video frame to generate a raindrop pixel block. The advantage of this setting is that the raindrop pixel blocks of each video frame in the target video stream can be accurately determined.

[0048] Exemplarily, the camera collects images in YUV420 encoding mode, so that the target video stream of the scene to be monitored is a video stream in YUV format. Among them, the Y component refers to brightness information, and the UV component refers to color information. For raindrops, the difference in brightness feature expression is greater than the difference in color feature expression, that is, the impact of raindrops on the Y component of the image is much greater than the impact of raindrops on the UV component of the image. The pixel changes caused by other interferences are different in the YUV component. In an embodiment of the present invention, for each pixel in the video frame, it is determined whether the current pixel is the background according to the Y component and UV component corresponding to the current pixel. When the current pixel is determined to be the background according to the Y component of the current pixel and the current pixel is determined to be the foreground according to the UV component of the current pixel, the current pixel is determined to be a raindrop pixel. According to the above method, all raindrop pixels in each video frame in the target video stream are determined. Then, adjacent raindrop pixel blocks in the video frame can be regionally connected based on the two-step scanning method to generate raindrop pixel blocks.

[0049] Optionally, determining the current pixel as the background according to the Y component of the current pixel and determining the current pixel as the foreground according to the UV component of the current pixel includes: establishing a corresponding Y component Gaussian distribution model and a UV component Gaussian distribution model according to the Y component and UV component corresponding to the current pixel of each video frame in the target video stream, respectively; determining the Y component mean and Y component standard deviation of the current pixel in the video frame before the current video frame in the target video stream according to the Y component Gaussian distribution model, and determining the UV component mean and UV component standard deviation of the current pixel in the video frame before the current video frame in the target video stream according to the UV component Gaussian distribution model; when the Y component of the current pixel, the Y component mean and the Y component standard deviation meet a second preset condition, it indicates that the current pixel is determined to be the background according to the Y component of the current pixel; when the UV component of the current pixel, the UV component mean and the UV component standard deviation meet a third preset condition, it indicates that the current pixel is determined to be the foreground according to the UV component of the current pixel.

[0050] Exemplarily, according to the Y component corresponding to the same pixel position of each video frame in the target video stream, a corresponding Y component Gaussian distribution model is established, and according to the UV component corresponding to the same pixel position of each video frame in the target video stream, a corresponding UV component Gaussian distribution model is established. It can be understood that each pixel point of the target video stream in YUV format is divided into Y channel and UV channel for single Gaussian background modeling. For example, the YUV format is YUV420 format. Since YUV420 is only one-fourth of the number of video frame pixels in the UV component, each UV component value will correspond to 4 pixels. Therefore, when modeling, each UV component pixel corresponds to 4 pixels on the Y component, and these 4 pixels are equivalent on the UV component. Taking the Y component as an example, the time series change process of the Y component of each video frame in the target video stream is described as follows: {X1, X2,…, X t =I Y (x p ,y p ,i):1≤i≤t,0≤p≤w,0≤q≤h, where X represents the total number of video frames in the target video stream, t represents the total number of video frames contained in the target video stream, and I Y (x p ,y p ,i) indicates that the i-th frame of the target video stream is in (x p ,y p ) pixel position, w represents the width of the video frame, and h represents the length of the video frame. The changes in the Y component and UV component of different pixels in the video frame are independent. It can be considered that the changes in the Y component and UV component of the video frame at the same pixel position in the target video stream obey the Gaussian distribution.

[0051] The Y component mean and the Y component standard deviation of the current pixel in the previous video frame of the current video frame are determined according to the Y component Gaussian distribution model, and when the Y component, the Y component mean and the Y component standard deviation of the current pixel meet the second preset condition, it indicates that the current pixel is determined as the background according to the Y component of the current pixel. Optionally, the second preset condition includes: Among them, I Y (x, y, t) represents the Y component of the current pixel, represents the mean value of the Y component, that is, the mean value of the Y component Gaussian distribution obeyed by the current pixel point of the video frame up to the t-1 frame, represents the standard deviation of the Y component, that is, the standard deviation of the Y component Gaussian distribution obeyed by the current pixel point of the video frame up to the t-1 frame, α represents the dynamic coefficient, and α is dynamically updated based on the video frame before the current video frame in the target video stream.

[0052] The UV component mean and UV component standard deviation of the current pixel in the previous video frame are determined according to the UV component Gaussian distribution model; when the UV component, UV component mean and UV component standard deviation of the current pixel meet the third preset condition, it indicates that the current pixel is determined as the foreground according to the UV component of the current pixel. Optionally, the third preset condition includes: Among them, I UV (x, y, t) represents the UV component of the current pixel. represents the mean value of the UV component, that is, the mean value of the Gaussian distribution of the UV component obeyed by the current pixel point of the video frame up to the t-1 frame, represents the standard deviation of the UV component, that is, the standard deviation of the UV component Gaussian distribution obeyed by the current pixel point of the video frame up to the t-1 frame, and θ represents the coefficient constant.

[0053] The Y component represents the change in brightness, and any moving object will cause brightness changes. It is known from reality that rainy days exist in continuous frames for a long period of time. After raindrops are detected in a certain area, due to the similarity of the video frame structure, subsequent raindrops are also likely to appear in this area. Therefore, the judgment threshold of α in this area can be dynamically reduced to improve the detection effect of raindrops. The UV component represents the change in chromaticity and is used to filter out non-raindrop detection targets, such as high-altitude parabolic or wind-blown animal objects fluttering and swinging, etc., which have larger chromaticity changes than raindrops. On the one hand, the UV component has a large change amplitude, and θ can distinguish raindrops from other objects by using empirical values. On the other hand, selecting a constant can reduce the amount of calculation and memory usage. In summary, for the same pixel point, if the Y component is judged as the foreground and the UV component is judged as the background, the pixel point is considered to be a raindrop pixel point.

[0054] Optionally, when a certain pixel of the current video frame is determined as the background through the Y component, the mean and variance of the Y component Gaussian distribution model obeyed by the pixel are updated, and the updating process is as follows: in, Indicates the mean of the Y component Gaussian distribution obeyed by the current pixel point of the video frame up to frame t, Indicates the mean of the Y component Gaussian distribution obeyed by the current pixel point of the video frame up to frame t-1, Indicates the standard deviation of the Y component Gaussian distribution of the current pixel in the video frame up to frame t. It represents the standard deviation of the Gaussian distribution of the Y component of the current pixel in the video frame up to the t-1 frame, I Y (x, y, t) represents the Y component of the current pixel in the tth video frame (i.e., the current video frame). Similarly, when the pixel is determined to be the background through the UV component, the mean and variance of the UV component Gaussian distribution model that the pixel obeys are updated. The update process is as follows: in, Indicates the mean value of the Gaussian distribution of the UV component obeyed by the current pixel point of the video frame up to frame t, Indicates the mean of the Gaussian distribution of the UV component obeyed by the current pixel point of the video frame up to frame t-1, Indicates the standard deviation of the Gaussian distribution of the UV component obeyed by the current pixel point of the video frame up to frame t, Indicates the standard deviation of the Gaussian distribution of the UV component obeyed by the current pixel point of the video frame up to the t-1 frame, I UV (x, y, t) represents the UV component of the current pixel in the tth video frame (i.e., the current video frame). Δ represents the update rate.

[0055] Optionally, the updating process of α includes: determining a target grid area in which the current pixel is located in at least two grid areas pre-divided in the current video frame; wherein the at least two grid areas are of the same size; determining the number of consecutive frames in which raindrops are detected in the target grid area from the video frames before the current video frame in the target video stream; wherein, when the maximum total number of pixels of adjacent raindrop pixels in the target grid area is less than a preset pixel threshold, it is determined that raindrops are detected in the target grid area; and updating the α corresponding to each pixel in the target grid area according to the consecutive number of frames.

[0056] In the embodiment of the present invention, for the Y component, it can be seen that α is the key to determine whether each pixel is foreground or background, but for raindrops, they exist continuously in the video frame for a long period of time and will show a regular distribution according to the structure of the video frame. Therefore, the embodiment of the present invention provides a method for dynamically adjusting α.

[0057] Specifically, each video frame in the target video stream is divided into at least two grid areas, wherein the grid areas into which the video frames are divided have the same size. For example, the size of each grid area of ​​the video frame is s. std *s stdThe so-called standard cell for a video frame of 720P size is 16 pixels. Of course, if it is a video frame of other resolution sizes, the size of the grid area can be scaled in proportion. It can be seen from the neighborhood relationship of the image that the structure in a small grid area can be regarded as approximately equal, so all pixels in the same grid area can share the same α. Determine the target grid area where the current pixel is located from at least two grid areas pre-divided in the current video frame, and then determine the number of consecutive frames in which raindrops are detected in the target grid area from the video frame before the current video frame in the target video stream. Exemplarily, the current video frame is the nth video frame, then the number of consecutive frames in which raindrops are detected in the target grid area where the current pixel is located is determined from the first n-1 video frames in the target video stream. For example, the target video stream contains 10 video frames, and the video frames in which raindrops are detected in the target grid area where a certain pixel is located in the 10 video frames are the 1st frame, the 2nd frame, the 3rd frame, the 4th frame and the 9th frame respectively. Then the number of consecutive frames in which raindrops are detected in the target grid area in the 10 video frames are: 1, 2, 3, 4, 0, 0, 0, 0, 1, 0. If the current video frame is the 5th video frame, the number of consecutive frames in which raindrops are detected in the target grid area in the first 4 video frames of the target video stream is 4. Among them, the process of determining whether raindrops are detected in the target grid area in the first n-1 video frames in the target video stream includes: respectively counting the maximum total number of adjacent raindrop pixels in the target grid area in each of the first n-1 video frames, and judging whether the maximum total number of pixels is less than a preset pixel threshold (such as 16). If so, it is determined that raindrops are detected in the target grid area in the video frame, otherwise, it is determined that no raindrops are detected in the target grid area in the video frame. The α corresponding to each pixel in the target grid area of ​​the current video frame is updated according to the number of consecutive frames.

[0058] Optionally, updating the α corresponding to each pixel point in the target grid area according to the number of consecutive frames includes: updating the α corresponding to each pixel point in the target grid area according to the following formula:

[0059]

[0060] Wherein, f represents the number of consecutive frames, and λ represents a constant. For example, λ is an empirical value, and its value range may be [10, 100].

[0061] In some embodiments, obtaining a target video stream of a scene to be monitored includes: obtaining a target video stream of a window queue of the scene to be monitored; wherein the size of the window queue is t frames of images; after determining that the scene to be monitored is a rainy day scene, it also includes: when the video frames of consecutive k*t frames in the window queue are non-rainy day video frames, determining that the scene to be monitored is a rainy day scene ends, wherein k is a constant less than 1. Exemplarily, obtaining a target video stream of a scene to be monitored with a window queue size of t frames of images, that is, taking t frames of video frames as detection objects, to determine whether the scene to be monitored is a rainy day scene. Polling each video frame in the target video stream of the window queue, sequentially determining the target raindrop pixel block in each video frame and determining whether each video frame is a rainy day video frame. Starting from the first video frame in the window queue, sequentially determining the total number of raindrop pixel blocks and / or the number of rainy day video frames in the first i video frames in the target video stream, and determining whether they meet the first preset condition, when the total number of raindrop pixel blocks and / or the number of rainy day video frames begin to meet the first preset condition, it indicates that the scene to be monitored begins to rain. When k*t consecutive video frames in the window queue are non-rainy day video frames, it is determined that the scene to be monitored is a rainy day scene, that is, the rain in the scene to be monitored ends. For example, k can be taken as 0.1. Optionally, after k*t consecutive video frames in the window queue are non-rainy day video frames, when rainy day video frames are detected again, the operations of S110-S140 are re-executed. Optionally, after the target video stream in the window queue is detected, when t frames of new target video stream are updated in the window queue, the operations of S110-S140 are executed again.

[0062] Embodiment 2

[0063] Figure 2 This is a schematic diagram of the structure of a rainy day recognition device provided by Embodiment 2 of the present invention. Figure 2 As shown, the device comprises:

[0064] A target video stream acquisition module 210 is used to acquire a target video stream of a scene to be monitored;

[0065] A raindrop pixel block determination module 220, configured to poll each video frame in the target video stream to determine a raindrop pixel block in a current video frame;

[0066] A rainy day video frame determination module 230 is used to determine whether a corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream, and count the number of rainy day video frames in the target video stream;

[0067] The rainy scene recognition module 240 is used to determine that the scene to be monitored is a rainy scene when the total number of raindrop pixel blocks in the target video stream and / or the number of frames of the rainy video frames meets a first preset condition.

[0068] Optionally, the target video stream is a video stream in luminance and chrominance YUV format;

[0069] The raindrop pixel block determination module is used to:

[0070] For each pixel point in the current video frame, a corresponding Y component Gaussian distribution model and a UV component Gaussian distribution model are established according to the Y component and UV component corresponding to the current pixel point of each video frame in the target video stream;

[0071] Determine a Y component mean and a Y component standard deviation of the current pixel in the video frame before the current video frame in the target video stream according to the Y component Gaussian distribution model, and determine a UV component mean and a UV component standard deviation of the current pixel in the video frame before the current video frame in the target video stream according to the UV component Gaussian distribution model;

[0072] When the Y component of the current pixel, the mean value of the Y component and the standard deviation of the Y component meet a second preset condition, and when the UV component of the current pixel, the mean value of the UV component and the standard deviation of the UV component meet a third preset condition, determine that the current pixel is a raindrop pixel;

[0073] Adjacent raindrop pixel points in the current video frame are connected to generate a raindrop pixel block.

[0074] Optionally, the second preset condition includes:

[0075]

[0076] Among them, I Y (x, y, t) represents the Y component of the current pixel, represents the mean value of the Y component, represents the standard deviation of the Y component, α represents a dynamic coefficient, and α is dynamically updated based on the video frame before the current video frame in the target video stream.

[0077] Optionally, the third preset condition includes:

[0078]

[0079] Among them, I UV (x, y, t) represents the UV component of the current pixel. represents the mean value of the UV component, represents the standard deviation of the UV component, and θ represents the coefficient constant.

[0080] Optionally, the update process of α includes:

[0081] Determine a target grid area where the current pixel point is located among at least two grid areas pre-divided in the current video frame; wherein the at least two grid areas are of the same size;

[0082] Determine the number of consecutive frames in which raindrops are detected in the target grid area from the video frames before the current video frame in the target video stream; wherein, when the maximum total number of pixels of adjacent raindrop pixels in the target grid area is less than a preset pixel threshold, it is determined that raindrops are detected in the target grid area;

[0083] The α corresponding to each pixel point in the target grid area is updated according to the number of consecutive frames.

[0084] Optionally, updating the α corresponding to each pixel point in the target grid area according to the number of consecutive frames includes:

[0085] Update the α corresponding to each pixel in the target grid area according to the following formula:

[0086]

[0087] Wherein, f represents the number of consecutive frames, and λ represents a constant.

[0088] Optionally, the rainy day video frame judgment module includes:

[0089] a raindrop pixel block number determination unit, configured to determine the number of raindrop pixel blocks in a current video frame for each video frame in the target video stream;

[0090] a center point position information determining unit, configured to respectively determine the center point position information of each raindrop pixel block in the current video frame when the number of the raindrop pixel blocks is greater than a preset number threshold;

[0091] a target distance determination unit, configured to determine, for each raindrop pixel block in the current video frame, a target distance between a current raindrop pixel block and a target raindrop pixel block according to the center point position information; wherein the target raindrop pixel block is a raindrop pixel block that is closest to the current raindrop pixel block among the raindrop pixel blocks;

[0092] The rainy day video frame determining unit is used to determine that the current video frame is a rainy day video frame when the target distance corresponding to each raindrop pixel block meets a fourth preset condition.

[0093] Optionally, the rainy day video frame determination unit is used to:

[0094] Calculating the distance mean of the target distance corresponding to each raindrop pixel block;

[0095] When the distance mean satisfies the following conditions, the distance standard deviation of the target distance corresponding to each raindrop pixel block is calculated:

[0096]

[0097] When the distance standard deviation satisfies the following conditions, it is determined that the current video frame is a rainy day video frame:

[0098]

[0099] Among them, σ min represents the distance standard deviation, d min represents the mean of the distances, d i represents the target distance corresponding to the i-th raindrop pixel block, n represents the total number of raindrop pixel blocks contained in the current video frame, w represents the width of the current video frame, h represents the length of the current video frame, and s std Indicates the length of the pre-divided grid area in the current video frame.

[0100] Optionally, the device further includes a non-rainy day video frame determination module, which is used to:

[0101] When the number of the raindrop pixel blocks is less than the preset number threshold, determining that the current video frame is a non-rainy video frame;

[0102] Alternatively, when the distance mean satisfies the following conditions, it is determined that the current video frame is a non-rainy day video frame:

[0103]

[0104] Alternatively, when the distance standard deviation satisfies the following conditions, it is determined that the current video frame is a non-rainy day video frame:

[0105]

[0106] Optionally, the target video stream acquisition module is used to:

[0107] Obtain the target video stream of the window queue of the scene to be monitored; wherein the size of the window queue is t frame images;

[0108] Also includes:

[0109] The rainy day end determination module is used to determine that after determining that the monitored scene is a rainy day scene, when the video frames of consecutive k*t frames in the window queue are non-rainy day video frames, determine that the monitored scene is over as a rainy day scene, wherein k is a constant less than 1.

[0110] Optionally, the rainy scene recognition module is used to:

[0111] When the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas corresponding to all video frames in the target video stream is greater than a first preset threshold, and the ratio of the number of frames of the rainy day video frames to the number of frames of the video frames included in the target video stream is greater than a second preset threshold, it is determined that the scene to be monitored is a rainy day scene; or,

[0112] When the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas corresponding to all video frames in the target video stream is less than or equal to the first preset threshold, and the ratio of the number of frames of the rainy video frames to the number of frames of the video frames contained in the target video stream is greater than a third preset threshold, it is determined that the scene to be monitored is a rainy scene; wherein the third preset threshold is greater than the second preset threshold.

[0113] The rainy day recognition device provided in the embodiment of the present invention can execute the rainy day recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0114] Embodiment 3

[0115] Figure 3 A schematic diagram of a camera 10 that can be used to implement an embodiment of the present invention is shown. The camera is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The camera can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0116] like Figure 3As shown, the camera 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the camera 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0117] A number of components in the camera 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the camera 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0118] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the rainy day recognition method.

[0119] In some embodiments, the rainy day recognition method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the camera 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the rainy day recognition method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the rainy day recognition method in any other suitable manner (e.g., by means of firmware).

[0120] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0122] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a camera having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the camera. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0125] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0126] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0127] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A rainy day recognition method, characterized in that: include: Obtain the target video stream of the scene to be monitored; Polling each video frame in the target video stream to determine a raindrop pixel block in a current video frame; Determining whether a corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream, and counting the number of rainy day video frames in the target video stream; When the total number of raindrop pixel blocks in the target video stream and / or the number of the rainy day video frames meets a first preset condition, it is determined that the scene to be monitored is a rainy day scene.

2. The method according to claim 1, characterized in that The target video stream is a video stream in luminance and chrominance YUV format; Determine the raindrop pixel block in the current video frame, including: For each pixel point in the current video frame, a corresponding Y component Gaussian distribution model and a UV component Gaussian distribution model are established according to the Y component and UV component corresponding to the current pixel point of each video frame in the target video stream; Determine a Y component mean and a Y component standard deviation of the current pixel in the video frame before the current video frame in the target video stream according to the Y component Gaussian distribution model, and determine a UV component mean and a UV component standard deviation of the current pixel in the video frame before the current video frame in the target video stream according to the UV component Gaussian distribution model; When the Y component of the current pixel, the mean value of the Y component and the standard deviation of the Y component meet a second preset condition, and when the UV component of the current pixel, the mean value of the UV component and the standard deviation of the UV component meet a third preset condition, determine that the current pixel is a raindrop pixel; Adjacent raindrop pixel points in the current video frame are connected to generate a raindrop pixel block.

3. The method according to claim 2, characterized in that The second preset condition includes: Among them, I Y (x, y, t) represents the Y component of the current pixel, represents the mean value of the Y component, represents the standard deviation of the Y component, α represents a dynamic coefficient, and α is dynamically updated based on the video frame before the current video frame in the target video stream; The third preset condition includes: Among them, I UV (x, y, t) represents the UV component of the current pixel. represents the mean value of the UV component, represents the standard deviation of the UV component, and θ represents the coefficient constant.

4. The method according to claim 3, characterized in that The update process of α includes: Determine a target grid area where the current pixel point is located among at least two grid areas pre-divided in the current video frame; wherein the at least two grid areas are of the same size; Determine the number of consecutive frames in which raindrops are detected in the target grid area from the video frames before the current video frame in the target video stream; wherein, when the maximum total number of pixels of adjacent raindrop pixels in the target grid area is less than a preset pixel threshold, it is determined that raindrops are detected in the target grid area; The α corresponding to each pixel point in the target grid area is updated according to the number of consecutive frames.

5. The method according to claim 4, characterized in that Updating α corresponding to each pixel point in the target grid area according to the number of consecutive frames includes: Update the α corresponding to each pixel in the target grid area according to the following formula: Wherein, f represents the number of consecutive frames, and λ represents a constant.

6. The method according to claim 1, characterized in that Determining whether a corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream respectively includes: For each video frame in the target video stream, determining the number of raindrop pixel blocks in the current video frame; When the number of the raindrop pixel blocks is greater than a preset number threshold, respectively determining the center point position information of each raindrop pixel block in the current video frame; For each raindrop pixel block in the current video frame, determine a target distance between the current raindrop pixel block and a target raindrop pixel block according to the center point position information; wherein the target raindrop pixel block is a raindrop pixel block that is closest to the current raindrop pixel block among the raindrop pixel blocks; When the target distances corresponding to the raindrop pixel blocks meet a fourth preset condition, it is determined that the current video frame is a rainy day video frame.

7. The method according to claim 6, characterized in that When the target distances corresponding to the raindrop pixel blocks meet a fourth preset condition, determining that the current video frame is a rainy day video frame includes: Calculating the distance mean of the target distance corresponding to each raindrop pixel block; When the distance mean satisfies the following conditions, the distance standard deviation of the target distance corresponding to each raindrop pixel block is calculated: When the distance standard deviation satisfies the following conditions, it is determined that the current video frame is a rainy day video frame: Among them, σ min represents the distance standard deviation, d min represents the mean of the distances, d i represents the target distance corresponding to the i-th raindrop pixel block, n represents the total number of raindrop pixel blocks contained in the current video frame, w represents the width of the current video frame, h represents the length of the current video frame, and s std Indicates the length of the pre-divided grid area in the current video frame.

8. The method according to claim 1, characterized in that When the total number of raindrop pixel blocks in the target video stream and the number of frames of the rainy day video frames meet a first preset condition, determining that the scene to be monitored is a rainy day scene includes: When the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas corresponding to all video frames in the target video stream is greater than a first preset threshold, and the ratio of the number of frames of the rainy day video frames to the number of frames of the video frames included in the target video stream is greater than a second preset threshold, it is determined that the scene to be monitored is a rainy day scene; or, When the ratio of the total number of raindrop pixel blocks in the target video stream to the total number of grid areas corresponding to all video frames in the target video stream is less than or equal to the first preset threshold, and the ratio of the number of frames of the rainy video frames to the number of frames of the video frames contained in the target video stream is greater than a third preset threshold, it is determined that the scene to be monitored is a rainy scene; wherein the third preset threshold is greater than the second preset threshold.

9. A rainy day recognition device, characterized in that: include: A target video stream acquisition module is used to acquire a target video stream of a scene to be monitored; A raindrop pixel block determination module, used to poll each video frame in the target video stream to determine a raindrop pixel block in a current video frame; A rainy day video frame judgment module, used to judge whether the corresponding video frame is a rainy day video frame according to the raindrop pixel blocks in each video frame in the target video stream, and count the number of rainy day video frames in the target video stream; The rainy scene recognition module is used to determine that the scene to be monitored is a rainy scene when the total number of raindrop pixel blocks in the target video stream and / or the number of frames of the rainy video frames meet a first preset condition.

10. A camera, characterized in that: The camera comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the rainy day identification method according to any one of claims 1 to 8.

Images