A Camera Damage Detection Method and System Based on Average Optical Flow Gradient

Abstract

The present invention relates to a camera damage detection method and system based on the average optical flow gradient, which obtains the current frame image and the previous frame image; obtains the grayscale image of the current frame image and the grayscale image of the previous frame image; inputs the grayscale image of the current frame image and the grayscale image of the previous frame image, calculates the dense optical flow field between the two frames of images, and each point in the optical flow field is the displacement amount of the corresponding pixel point in the x and y directions; calculates the mean value of the optical flow amplitude based on the dense optical flow field; calculates the gradient of the mean value of the optical flow amplitude, denoted as the average optical flow gradient, and if the average optical flow gradient meets the preset judgment condition, it is considered that the camera is damaged. Compared with the prior art, in the automatic detection of camera damage, the present invention mainly realizes it by detecting the drastic change of the camera screen, describes the change of the camera screen with the average optical flow gradient as a feature, and determines whether the camera is damaged by judging whether the current optical flow feature change exceeds the set threshold.

Description

Technical Field

[0001] The present invention relates to the field of camera damage detection, and more particularly to a camera damage detection method and system based on average optical flow gradient. Background Art

[0002] With the development and progress of the country, the application of social security monitoring measures is becoming more and more extensive. Surveillance cameras are important security means and are generally equipped in places such as banks, schools, and the boundaries of military camps. When the camera collects video information, if it is blocked by sundries or maliciously damaged by humans, it cannot capture the actual situation on the scene, which may pose a security risk. The traditional method of manually monitoring the surveillance video to check the working status of the camera is feasible for a small number of cameras. However, if the number of cameras is huge, it is difficult for manual monitoring to detect in real time whether the camera is damaged.

[0003] The existing methods for detecting camera damage mostly involve installing a housing on the camera and combining sensors such as pressure, touch, and human proximity. When someone damages the camera, the sensor receives the signal and alarms. However, these devices require redesigning the camera housing, which will increase the material cost. For example, in Chinese Patent CN112712660A, a detection film is added to the surface of the existing camera for a device that generates a touch detection signal. The touch signal generated by the detection film is transmitted to an analog-to-digital conversion ADC amplifier to convert the touch analog value into a digital value, and it is determined whether the camera is touched according to the digital value. A transmitter for signal detection emits an ultrasonic touch detection signal. The detection film is used as a touch signal detection device, and it is determined whether the camera is touched according to the numerical change of the ultrasonic signal emitted by the signal transmitter on the detection film. This method can detect when someone touches the detection film to disassemble the monitor, but it cannot detect the occlusion or torsion of the camera body. Chinese Patent CN111246071B discloses a remotely monitored video camera with automatic warning of damage. This camera has an additional bracket, and functions such as automatic warning are realized by fixing the camera on this device. A microprocessor, an alarm, and a vibration sensor are installed in the control box of the bracket, and both the alarm and the vibration sensor are electrically connected to the microprocessor. When the signal intensity received by the vibration sensor is greater than 50mm, the vibration sensor transmits the vibration signal to the microprocessor, and the microprocessor transmits an instruction to the alarm. Similarly, the detection method of this design protection device is difficult to detect damage forms such as camera occlusion and smearing.

[0004] In addition, with the development of microprocessors, video images can be directly processed at the surveillance camera end. Subsequently, methods for detecting damage by processing the image data of video using some algorithms have emerged. Generally, the foreground and background of the video image are separated through algorithms and processed into grayscale images. By comparing the area size of the foreground image with a preset threshold, it is determined whether the camera is blocked. Then, based on the existing background dataset, the difference between the current background of the image and the existing background data is compared. If it is greater than the threshold, it is determined that the camera has been moved. For example, Chinese Patent CN105761261A discloses a method for determining whether an accident occurs to a camera by comparing the grayscale value of the current camera image with that of a normal background image. The global grayscale mean of the image is obtained through the grayscale mean algorithm, and then the foreground of the current image is estimated using the Kalman filtering theory. Then, a normal background set is determined and collected. Subsequently, the foreground and background of the image are segmented using the background difference method. The method for obtaining the foreground image using the background difference method is to perform a difference between the input image and the background model to determine the foreground and extract the foreground image. When using the background difference method, the background is constructed through a Kalman filter. For the specific content of the background difference method, reference can be made to "Qu Jingjing, Xin Yunhong. A Moving Target Detection Method Combining Inter-Frame Difference and Background Difference [J]. Acta Photonica Sinica, 2014, 43(07): 219-226". This method determines whether the camera is blocked by calculating the size of the foreground area of the image and comparing the area with a preset threshold. In addition, by comparing the current background of the image with the normal background set, if the change is greater than the threshold, it is determined that the camera has been moved. This method can identify two situations of camera movement or blockage, but it is necessary to determine the normal background dataset of the camera image in order to compare it to obtain the change difference. This increases the cost of marking the normal background set, and the method has low autonomy. At the same time, this method only utilizes the grayscale information of the camera image. In the face of complex image information and large amounts of image noise such as complex damage forms, the robustness of the method is poor.

[0005] In summary, existing camera protection devices will incur additional material costs, and sensors for detecting touch and vibration cannot detect situations such as blockage and movement, and vice versa. There are various forms of camera damage. At present, existing methods for detecting blockage, movement, etc. handle different forms of damage using different methods. The abnormal determination of camera damage is relatively complex and cannot cover all aspects. The method of using the background difference method to extract the foreground and background of the camera image for comparison only compares the grayscale image of the current video frame with the grayscale image of the normal background set. This may be because there are many objects (people or vehicles) in the image, occupying the main scene of the image, which will also lead to a large difference in the grayscale image of the image, and the robustness of the algorithm is poor. Since in reality, there are various forms of camera damage, including being moved, blocked, and struck by external forces. Therefore, it is crucial to seek a method that can detect camera damage in multiple situations. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a camera damage detection method and system based on average optical flow gradient, which can detect various damage forms, can be widely applied to automatic alarm for damage of surveillance cameras in personal residences, banks, schools, military camp boundaries, etc., and has broad practical application prospects.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A camera damage detection method based on average optical flow gradient includes the following steps:

[0009] S1. The camera collects video images to obtain the current frame image and the previous frame image;

[0010] S2. Perform grayscale processing to obtain the grayscale image of the current frame image and the grayscale image of the previous frame image;

[0011] S3. Input the grayscale image of the current frame image and the grayscale image of the previous frame image, calculate the dense optical flow field between the two frames of images, and each point in the optical flow field is the displacement amount of the corresponding pixel point in the x direction and the y direction;

[0012] S4. Based on the dense optical flow field, calculate the average value of the optical flow amplitude;

[0013] S5. Based on the average value of the optical flow amplitude, calculate the gradient of the average value of the optical flow amplitude, denoted as the average optical flow gradient. If the average optical flow gradient meets the preset judgment condition, it is considered that the camera is damaged.

[0014] Further, in step S2, the following formula is used for grayscale processing:

[0015] Gray = (Red + Green + Blue) / 3

[0016] Where Gray represents the grayscale value of the pixel after grayscale processing, and Red, Green, and Blue respectively represent the R, G, and B channel pixel values of the pixel before grayscale processing.

[0017] Further, in step S3, the Farneback method is used to calculate the dense optical flow of the image to obtain the displacement amount of each pixel point in the x direction and the displacement amount in the y direction;

[0018] According to the linear motion hypothesis, there is a motion equation in a two-dimensional plane:

[0019] d = Sp

[0020] Where: p = [p1 p2 p3 p4 p5 p6] T

[0021] d is the optical flow field to be solved, X = (x, y) T is the two-dimensional pixel coordinate of the image, p represents the motion parameter, substituting it back into the neighborhood information constraint equation and omitting X, using the subscript i as the index of the pixel points within the neighborhood:

[0022]

[0023] ω i is the weight function of the pixel point, A and Δb are intermediate parameters in the calculation process of the Farneback method;

[0024] Use the ridge regression algorithm to solve the constraint equation:

[0025] p R = ∑(ωS T AS + kI) -1 ∑ωs T A T Δb

[0026] where k represents the ridge parameter, p R represents the motion parameter obtained by ridge regression calculation, I is the identity matrix, after obtaining the motion parameter p R substitute it back into the motion equation to obtain the optical flow field d.

[0027] Furthermore, in step S4, the calculation formula for the mean value of the optical flow amplitude is:

[0028]

[0029] where f t represents the mean value of the optical flow amplitude at time t, w represents the pixel width of the camera screen, h represents the pixel height of the camera screen, Flow x (x, y) represents the displacement of the pixel point (x, y) in the x direction, Flow y (x, y) represents the displacement of the pixel point (x, y) in the y direction.

[0030] Furthermore, in step S5, the calculation formula for the gradient of the mean value of the optical flow amplitude is:

[0031] Δf t = f t - f t-1

[0032] where Δf t represents the average optical flow gradient, f tRepresents the average optical flow amplitude at time t, that is, the average optical flow amplitude calculated from the current frame image and the previous frame image, f t-1 Represents the average optical flow amplitude at time t - 1, that is, the average optical flow amplitude calculated from the previous frame image and the frame before the previous frame image.

[0033] Furthermore, in step S5, the preset judgment conditions are as follows:

[0034] It is considered that the average optical flow gradient of the video frame obeys the Gaussian distribution. Under normal circumstances, the average optical flow gradient Δf t The probability within (μ - 3σ, μ + 3σ) is 99.74. When the camera is damaged, the data distribution of the average optical flow gradient Δf t Will exceed the interval (μ - 3σ, μ + 3σ):

[0035] P(|Δf i - u| > 3σ) ≤ 0.003

[0036]

[0037] Where N represents the time series of the frame before the current frame of the video. When the camera is damaged, then Δf t > μ + 3σ or Δf t < μ - 3σ. When the camera is working properly, μ - 3σ ≤ Δf t ≤ μ + 3σ.

[0038] A camera damage detection system based on the average optical flow gradient, including:

[0039] A data acquisition module, connected to the camera. The camera acquires video images, and the data acquisition module obtains the current frame image and the previous frame image;

[0040] A preprocessing module that grayscales the images to obtain the grayscale images of the current frame image and the previous frame image;

[0041] An optical flow field calculation module that takes the grayscale image of the current frame image and the grayscale image of the previous frame image as inputs and calculates the dense optical flow field between the two frames of images. Each point in the optical flow field is the displacement of the corresponding pixel point in the x - direction and y - direction;

[0042] An optical flow amplitude average calculation module that calculates the average optical flow amplitude based on the dense optical flow field;

[0043] A detection module that calculates the gradient of the average optical flow amplitude based on the average optical flow amplitudes at different times, denoted as the average optical flow gradient. If the average optical flow gradient meets the preset judgment conditions, it is considered that the camera is damaged.

[0044] Further, in the optical flow amplitude mean calculation module, the calculation formula for the optical flow amplitude mean is as follows:

[0045]

[0046] where f t represents the optical flow amplitude mean at time t, w represents the pixel width of the camera screen, h represents the pixel height of the camera screen, Flow x (x, y) represents the displacement of the pixel point (x, y) in the x direction, and Flow y (x, y) represents the displacement of the pixel point (x, y) in the y direction.

[0047] Further, in the detection module, the gradient Δf of the optical flow amplitude mean t is:

[0048] Δf t = f t - f t-1

[0049] where f t represents the optical flow amplitude mean at time t, that is, the optical flow amplitude mean calculated from the current frame image and the previous frame image, and f t-1 represents the optical flow amplitude mean at time t-1, that is, the optical flow amplitude mean calculated from the previous frame image and the frame before the previous frame image.

[0050] Further, in the detection module, the preset judgment conditions are as follows:

[0051] It is considered that the average optical flow gradient of the video screen follows a Gaussian distribution. Under normal circumstances, the probability that the average optical flow gradient Δf t is within (μ - 3σ, μ + 3σ) is 99.74. When the camera is damaged, the data distribution of the average optical flow gradient Δf t will exceed the interval (μ - 3σ, μ + 3σ):

[0052] P(|Δf i - u| > 3σ) ≤ 0.003

[0053]

[0054] where N represents the time series of the previous frame of the current frame of the video. When the camera is damaged, then Δf t > μ + 3σ or Δf t < μ - 3σ. When the camera is working normally, μ - 3σ ≤ Δf t ≤ μ + 3σ.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) Under normal circumstances, the camera's picture changes relatively slowly. At the same time, the camera is fixedly installed, and there will be no jitter in the camera under normal circumstances. When criminals damage the camera by means of occlusion, movement, and hitting, etc., it will cause a large amplitude of jitter in the camera's picture. The damage to the camera presents various forms, and in most cases, it will cause a drastic change in the camera's picture. In the automatic detection of camera damage in the present invention, it is mainly achieved by detecting the drastic change in the camera's picture. The change in the camera's picture is described by the average optical flow gradient, and by determining whether the current optical flow feature change exceeds the set threshold, it is determined whether the camera is damaged.

[0057] (2) This application only needs to calculate the average optical flow and its gradient of the camera picture to complete most damage detections, eliminating the need to design detection algorithms for different forms of camera damage, reducing labor costs. At the same time, no additional protection device is required, and there is no need to redesign the camera housing, reducing the material costs of sensors such as vibration, pressure, and touch.

[0058] (3) Compared with the method of collecting background image datasets for detection, this application uses the average optical flow gradient of the camera picture for camera damage detection, eliminating the process of labeling the background dataset of the picture and the need to collect and determine the background dataset, solving the cumbersome and complex problem of the classic anomaly detection based on image grayscale that requires subjective collection of background set data by humans. At the same time, since the average optical flow gradient method reflects the changes between the current and the previous moment of the video, it has a certain robustness for the recognition of picture situations such as the gathering of people and vehicles, and can exclude them.

[0059] (4) The calculation of the mean value of the optical flow amplitude and the calculation of the average optical flow gradient are proposed. The overall optical flow information and the change amplitude of the camera picture are characterized by the mean value of the optical flow amplitude and the average optical flow gradient. Whether the camera has been damaged by behaviors such as being moved or attacked by throwing can be judged through the average optical flow gradient of the whole picture, improving the robustness of detecting camera damage and being able to give feedback on various forms of damage.

[0060] (5) The ridge regression algorithm is used to optimize the calculation process of the Farneback method, greatly improving the calculation stability and the accuracy of the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is the flow chart of the present invention;

[0062] Figure 2 is the schematic diagram of calculating the optical flow vector of the image by the Farneback method;

[0063] Figure 3 are the mean value of the optical flow amplitude and the average optical flow gradient of the video on the time axis. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and the detailed implementation manners and specific operation processes are given. However, the protection scope of the present invention is not limited to the following embodiments.

[0065] In the accompanying drawings, components with the same structure are denoted by the same numerical labels, and components with similar structures or functions everywhere are denoted by similar numerical labels. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, some parts in the drawings are appropriately exaggerated.

[0066] Embodiment 1:

[0067] A camera damage detection method based on the average optical flow gradient, comprising the following steps:

[0068] S1. The camera captures video images to obtain the current frame image and the previous frame image;

[0069] Generally speaking, when the camera is working, it will capture a continuous frame sequence, that is, a video. The sampling interval can be set as needed, such as 0.1 second, or directly use the continuous frame image sequence captured by the camera to obtain two adjacent frame images. Among them, the current frame image is the frame image captured at time t, and the previous frame image is the frame image captured at time -1.

[0070] S2. Perform grayscale processing to obtain the grayscale image of the current frame image and the grayscale image of the previous frame image;

[0071] In this embodiment, the gray value is calculated by equally proportionally superimposing the three primary colors (red, green, blue) of each pixel of the image, and the following formula is used for grayscale processing:

[0072] Gray = (Red + Green + Blue) / 3

[0073] Among them, Gray represents the gray value of the pixel after grayscale processing, and Red, Green, and Blue respectively represent the R, G, and B channel pixel values of the pixel before grayscale processing.

[0074] In other implementation manners, the weighted average method can also be used to set different weights for each channel to calculate the gray value.

[0075] S3. Input the grayscale image of the current frame image and the grayscale image of the previous frame image, and calculate the dense optical flow field between the two frame images. Each point in the optical flow field is the displacement amount of the corresponding pixel point in the x direction and the y direction;

[0076] The optical flow of the screen is defined as the motion pattern of objects in the image between consecutive frames. It may be caused by the movement of objects or the camera and can be represented as displacement vectors in a two-dimensional vector field, indicating the movement of pixels from one frame to another, also known as the optical flow vector. Currently, there are many algorithms for optical flow calculation. According to the design requirements of this application, the Farneback method is selected to calculate the dense optical flow of the image, obtaining the displacement in the x direction and the displacement in the y direction for each pixel point.

[0077] The Farneback method can calculate the dense optical flow of the image, that is, the optical flow Flow of each pixel in the image x , Flow y , this application uses the Farneback method optimized based on ridge regression to calculate the optical flow field of each pixel in the entire screen, and its principle is as follows:

[0078] The core idea of the Farneback optical flow method is to assume that the image gradient is constant and the local optical flow is constant, and regard the input image as a two-dimensional function, that is, approximate the screen of the camera (or called the camera) using a quadratic polynomial:

[0079] I(X) = X T AX + b T X + c

[0080] Among them, the variable is the two-dimensional coordinate X, X = (x, y) T is the two-dimensional coordinate of the pixel of the image, A is a 2×2 symmetric matrix, and the parameters in the matrix will obtain actual values after coefficientization. b is a two-dimensional vector, and c is a constant. After coefficientizing this formula, we have:

[0081] I(x, y) = r1 + r2x + r3y + r4x 2 + r5y 2 + r6xy

[0082] In the formula, c = r1;

[0083] Perform an ideal transformation through the global displacement d, where Construct a new signal:

[0084]

[0085] That is, if there are changes in adjacent frames of the video screen, the change in the image can be expressed as:

[0086]

[0087] Among them, d is the displacement d = (Flow x (x, y), Flowy (x, y)) T

[0088] According to the assumption of the optical flow method, the corresponding coefficients should be equal after the above formula is expanded, so we have:

[0089] A1 = A2, b2 = b1 - 2A1d,

[0090] Furthermore, the calculation formula for the displacement change d is obtained, which is also the optical flow field:

[0091]

[0092] In the actual situation, the above ideal situation cannot be used. It is very unrealistic that all signals are a polynomial and the global transformation involves two signals. It is difficult for the corresponding coefficients to satisfy the equal assumption. Therefore, the mean value of adjacent frames needs to be used for approximation, and the approximation formula is defined as follows:

[0093]

[0094] where X represents the two-dimensional coordinates of all pixels in the current frame

[0095] Then the optical flow field d is transformed to obtain:

[0096] A(X)d = Δb(X)

[0097] Then, the neighborhood information of pixels is used to weight A(X) and it is constrained by the neighborhood information to obtain:

[0098]

[0099] where ω is the weight function of each pixel in the neighborhood, which is determined by the Gaussian distribution, ΔX represents the coordinates of all pixels in the neighborhood, and I represents the neighborhood set of the current frame.

[0100] In order to introduce the coordinate information of pixels in the neighborhood, according to the linear motion assumption, there is a motion equation in the two-dimensional plane:

[0101] d = Sp

[0102] where: p = [p1 p2 p3 p4 p5 p6] T

[0103] Substitute it back into the neighborhood information constraint equation and omit X, using the subscript i as the index of the pixel points in the neighborhood:

[0104]

[0105] where i is the index of the pixel points in the neighborhood, ω iis the weight function of the pixel points in the neighborhood;

[0106] The Farneback method solves the above constraint equation by using least squares regression to calculate the motion parameter p. For ease of understanding, i is omitted:

[0107] p = ∑(ωS T AS) -1 ∑ωS T A T Δb

[0108] The constraint equation belongs to an overdetermined equation. When solving this equation by least squares, the solution is prone to instability, which will lead to inaccurate optical flow field parameters. When solving the overdetermined equation by least squares, ωS T AS is close to singularity. Adding a positive constant matrix kI to ωS T AS will greatly reduce the possibility of it being close to singularity, that is, (ωS T AS + kI) -1 has much higher stability of the solution than (ωS T AS) -1 is much higher. Therefore, this patent proposes to use the ridge regression algorithm to solve the constraint equation:

[0109] p R = ∑(ωS T AS + kI) -1 ∑ωS T A T Δb

[0110] where k represents the ridge parameter, and p R represents the motion parameter obtained by ridge regression calculation. Generally, k = 0.1, and I is the identity matrix. After obtaining the motion parameter p R , substitute it into the motion equation to obtain the optical flow field d.

[0111] S4. Calculate the mean value of the optical flow amplitude based on the dense optical flow field;

[0112] When the camera is damaged, in most cases, it will cause a drastic change in the camera screen, which will lead to a drastic change in the optical flow of the screen. In order to reflect the overall change of the screen, this application uses the mean value of the optical flow amplitude of the entire screen as the optical flow feature to describe the change in the video. The calculation formula for the mean value of the optical flow amplitude is:

[0113]

[0114] where f t represents the mean value of the optical flow amplitude at time t, w represents the pixel width of the camera screen, h represents the pixel height of the camera screen, Flow x (x, y) represents the displacement of the pixel point (x, y) in the x direction, Flow y(x, y) represents the displacement of the pixel point (x, y) in the y direction.

[0115] S5. Based on the mean optical flow amplitude, the gradient of the mean optical flow amplitude is calculated and recorded as the average optical flow gradient. If the average optical flow gradient meets the preset judgment condition, it is considered that the camera is damaged.

[0116] From the calculation formula of the mean optical flow amplitude in step S4, it can be seen that when the local part of the picture changes, the mean optical flow amplitude of the whole picture changes slightly; when the whole picture changes, the optical flow value of each pixel will change greatly, so that f t To further explain how fast the camera image changes, this application uses the gradient of the mean optical flow amplitude (hereinafter referred to as the average optical flow gradient) to explain:

[0117] The calculation formula for the gradient of the mean optical flow amplitude is:

[0118] Δf t =f t -f t-1

[0119] Where Δf t represents the average optical flow gradient, f t represents the mean optical flow amplitude at time t, that is, the mean optical flow amplitude calculated from the current frame image and the previous frame image, f t-1 It represents the mean optical flow amplitude at time t-1, that is, the mean optical flow amplitude calculated from the previous frame image and the previous frame image.

[0120] The larger the absolute value of the average optical flow gradient is, the more dramatic the changes in the picture are. The changes in optical flow caused by pedestrians, vehicles, etc. in the normal camera picture will hardly cause a large change in the average optical flow gradient; while forms of damage such as shaking, occlusion, movement, etc. will cause a large change in the average optical flow amplitude of the picture. At the same time, these forms of damage show that the picture changes faster, and the absolute value of the average optical flow gradient will be larger. The change in the average optical flow gradient value reflects the speed of change of the average optical flow amplitude per unit time, which is reflected in the camera as the intensity of the picture change. The camera can be judged by setting a threshold. In order to judge whether the camera is damaged based on the average optical flow gradient value, the judgment conditions of this application propose the use of a statistical confidence test method, also known as the 3-sigma criterion.

[0121] Assuming that the average optical flow gradient of the video screen follows a Gaussian distribution, under normal circumstances, the average optical flow gradient Δf t The probability of being within (μ-3σ, μ+3σ) is 99.74. When the camera is damaged, the average optical flow gradient Δf t The data distribution will be outside the interval (μ-3σ, μ+3σ):

[0122] P(|Δf i -u| > 3σ) ≤ 0.003

[0123]

[0124] Where N represents the time series of the previous frame (at time t - 1) of the current frame of the video, which can be understood as the total number of frames from a certain moment in the time series of the video (which can be the start moment of the video, or a selected moment or a moment within a certain time interval before time t) to time t - 1. When the camera is damaged, then there is Δf t > μ + 3σ or Δf t < μ - 3σ. When the camera is working normally, μ - 3σ ≤ Δf t ≤ μ + 3σ.

[0125] Thus, a damage signal signal can be fed back to the terminal according to the average optical flow gradient:

[0126]

[0127] When the terminal receives "1", it means the camera is damaged; when it receives "0", it means it is normal.

[0128] Embodiment 2:

[0129] This application also protects a camera damage detection system based on the average optical flow gradient, including:

[0130] A data acquisition module, connected to the camera. The camera acquires video images, and the data acquisition module obtains the current frame image and the previous frame image;

[0131] A preprocessing module, which performs grayscale processing on the images to obtain the grayscale images of the current frame image and the previous frame image;

[0132] An optical flow field calculation module, taking the grayscale image of the current frame image and the grayscale image of the previous frame image as inputs, calculates the dense optical flow field between the two frames of images. Each point in the optical flow field is the displacement amount of the corresponding pixel point in the x direction and the y direction;

[0133] An optical flow amplitude mean calculation module, based on the dense optical flow field, calculates the mean value of the optical flow amplitude;

[0134] A detection module, based on the mean value of the optical flow amplitude, calculates the gradient of the mean value of the optical flow amplitude, denoted as the average optical flow gradient. If the average optical flow gradient meets the preset judgment condition, it is considered that the camera is damaged.

[0135] The relevant content about the optical flow field, the mean value of the optical flow amplitude, the average optical flow gradient, and the judgment condition, etc., has been described in Embodiment 1 and will not be elaborated here.

[0136] After the camera is fixedly installed, the video frame is first processed into a grayscale image, and then the Farneback method is used to calculate the optical flow vector Flow between two consecutive frames of the video x , Flow y , such as Figure 2 shown. Then calculate the mean value of the optical flow amplitude of the frame, and calculate the average optical flow gradient according to the mean value of the optical flow amplitude at adjacent times. As Figure 3 shown, the mean value of the optical flow amplitude and the average optical flow gradient calculated at different times can be seen. It can be seen that in the Frame1 box, there is no drastic change in the optical flow information of the frame. Observing the camera frame at this time, it is the normal walking of the crowd. In the Frame2 box, the optical flow information of the frame changes drastically, and there are large fluctuations in the mean value of the optical flow amplitude and the average optical flow gradient. Observing the camera frame at this time, someone throws a stone at the camera. This shows that the present application can better detect whether the camera is damaged, and has good detection effect and strong robustness.

[0137] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of the present invention through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. A camera damage detection method based on average optical flow gradient, characterized in that Including the following steps: S1. The camera captures video images to obtain the current frame image and the previous frame image; S2. Perform grayscale processing to obtain the grayscale image of the current frame image and the grayscale image of the previous frame image; S3. Input the grayscale image of the current frame image and the grayscale image of the previous frame image, and calculate the dense optical flow field between the two frames of images. Each point in the optical flow field is the displacement of the corresponding pixel point in the x direction and the y direction; S4. Based on the dense optical flow field, calculate the mean value of the optical flow magnitude; S5. Based on the mean value of the optical flow magnitude, calculate the gradient of the mean value of the optical flow magnitude, denoted as the average optical flow gradient. If the average optical flow gradient meets the preset judgment condition, it is considered that the camera is damaged; In step S3, the Farneback method optimized based on ridge regression is used to calculate the dense optical flow of the image, and the displacement of each pixel point in the x direction and the displacement in the y direction are obtained; According to the linear motion hypothesis, there is a motion equation in the two-dimensional plane: d = Sp Wherein: p = [p1 p2 p3 p4 p5 p6] T d is the optical flow field to be solved, X = (x, y) T is the two-dimensional pixel coordinate of the image, p represents the motion parameter, substitute it back into the neighborhood information constraint equation and omit X, using the subscript i as the index of the pixel points within the neighborhood: ω i is the weight function of the pixel point, and A and Δb are intermediate parameters in the calculation process of the Farneback method; Use the ridge regression algorithm to solve the constraint equation: where k represents the ridge parameter, and p R represents the motion parameter obtained by ridge regression calculation, I is the identity matrix, and after obtaining the motion parameter p R it is substituted back into the motion equation to obtain the optical flow field d.

2. The camera damage detection method based on average optical flow gradient according to claim 1, characterized in that, In step S2, the following formula is used for grayscale processing: Gray = (Red + Green + Blue) / 3 where Gray represents the grayscale value of the pixel after grayscale processing, and Red, Green, and Blue respectively represent the R, G, and B channel pixel values of the pixel before grayscale processing.

3. The camera damage detection method based on the average optical flow gradient according to claim 1, wherein In step S4, the calculation formula for the mean value of the optical flow magnitude is: Among them, f t represents the average value of the optical flow amplitude at time t, w represents the pixel width of the camera screen, h represents the pixel height of the camera screen, Flow x (x, y) represents the displacement of the pixel point (x, y) in the x direction, Flow y (x, y) represents the displacement of the pixel point (x, y) in the y direction.

4. A method for detecting camera damage based on average optical flow gradient according to claim 1, characterized in that, In step S5, the calculation formula for the gradient of the mean value of the optical flow magnitude is: Δf t = f t - f t-1 Among them, Δf t represents the average optical flow gradient, and f t represents the average value of the optical flow amplitude at time t, that is, the average value of the optical flow amplitude calculated from the current frame image and the previous frame image, and f t-1 represents the average value of the optical flow amplitude at time t-1, that is, the average value of the optical flow amplitude calculated from the previous frame image and the frame image before that.

5. The camera damage detection method based on the average optical flow gradient according to claim 1, wherein, In step S5, the preset judgment condition is as follows: It is considered that the average optical flow gradient of the video image follows a Gaussian distribution. Under normal circumstances, the average optical flow gradient Δf t has a probability of 99.74 within (μ - 3σ, μ + 3σ). When the camera is damaged, the average optical flow gradient Δf t data distribution will exceed the interval (μ - 3σ, μ + 3σ): P(|Δf i - u| > 3σ) ≤ 0.003 Among them, N represents the time series of the previous frame of the current frame of the video. When the camera is damaged, there is Δf t > μ + 3σ or Δf t < μ - 3σ. When the camera is working properly, μ - 3σ ≤ Δf t ≤ μ + 3σ.

6. A camera damage detection system based on the average optical flow gradient, characterized in that, A camera damage detection method based on the average optical flow gradient according to any one of claims 1-5 includes: A data acquisition module, connected to the camera. The camera captures video images, and the data acquisition module obtains the current frame image and the previous frame image; A preprocessing module that performs grayscale processing on the image to obtain the grayscale image of the current frame image and the grayscale image of the previous frame image; An optical flow field calculation module that takes the grayscale image of the current frame image and the grayscale image of the previous frame image as inputs, and calculates the dense optical flow field between the two frames of images. Each point in the optical flow field is the displacement of the corresponding pixel point in the x direction and the y direction; An optical flow magnitude mean value calculation module that calculates the mean value of the optical flow magnitude based on the dense optical flow field; A detection module that calculates the gradient of the mean value of the optical flow magnitude based on the mean values of the optical flow magnitude at different times, denoted as the average optical flow gradient. If the average optical flow gradient meets the preset judgment condition, it is considered that the camera is damaged.

7. The camera damage detection system based on average optical flow gradient according to claim 6, wherein In the optical flow magnitude mean value calculation module, the calculation formula for the mean value of the optical flow magnitude is: Among them, f t represents the average value of the optical flow amplitude at time t, w represents the pixel width of the camera screen, h represents the pixel height of the camera screen, Flow x (x, y) represents the displacement of the pixel point (x, y) in the x direction, Flow y (x, y) represents the displacement of the pixel point (x, y) in the y direction.

8. The camera damage detection system based on the average optical flow gradient according to claim 6, characterized in that, In the detection module, the gradient Δf of the mean value of the optical flow amplitude t is as follows: Δf t = f t - f t-1 Among them, f t represents the average optical flow amplitude at time t, that is, the average optical flow amplitude calculated from the current frame image and the previous frame image. f t-1 represents the average optical flow amplitude at time t - 1, that is, the average optical flow amplitude calculated from the previous frame image and the frame before the previous frame image.

9. The camera damage detection system based on average optical flow gradient according to claim 6, wherein, In the detection module, the preset judgment condition is as follows: It is considered that the average optical flow gradient of the video image follows a Gaussian distribution. Under normal circumstances, the average optical flow gradient Δf t has a probability of 99.74 within (μ - 3σ, μ + 3σ). When the camera is damaged, the average optical flow gradient Δf t data distribution will exceed the interval (μ - 3σ, μ + 3σ): P(|Δf i - u| > 3σ) ≤ 0.003 Where N represents the time series of the previous frame of the current video frame. When the camera is damaged, there is Δf t > μ + 3σ or Δf t < μ - 3σ. When the camera is working properly, μ - 3σ ≤ Δf t ≤ μ + 3σ.

Images