Method and device for acquiring effective alarm image of moving object

Abstract

The invention relates to an image processing technology, and discloses a method and device for obtaining an effective alarm image of a moving object, and the method comprises the steps: determining the information of the moving object according to an initial picture frame and a subsequent picture frame; calculating the first exposure of the moving object and the second exposure of the moving object, calculating the second exposure of the moving object according to the information of the moving object, and caching the object moving frame image corresponding to the second exposure of the object; the first exposure of the moving object is used for displaying a real-time moving object image, and the cached moving frame image of the second exposure of the moving object is used for displaying the moving object image during alarming. According to the invention, the image frames with bright overall scene and good background image identification are displayed in a real-time preview mode, and the image frames with clear and distinguishable moving objects and poor background brightness are individually cached and uploaded as alarm video images.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to a method and device for obtaining effective alarm images of moving objects. Background Art

[0002] Cameras are increasingly demanding night vision distance and low-light image quality. CMOS sensors are becoming increasingly sensitive, and fill lights are becoming increasingly brighter. At the same time, digital image processing technology typically uses a center-weighted approach to exposure control, resulting in overexposure of objects 1 to 3 meters from the camera. As objects approach, occupying a certain percentage of the frame, the exposure is rapidly reduced to ensure that close-up objects are not overexposed. However, this can lead to a dark background and prone to color cast. This makes it difficult to simultaneously maintain image brightness and legibility for distant backgrounds. Multi-frame fusion wide dynamic range (WDR) technology can ensure background legibility while suppressing overexposed areas. However, this can lead to unusually large image smearing and brightness steps for moving objects and areas where there is a transition between long and short exposures. Furthermore, short exposure response speed and accuracy are limited.

[0003] Conventional automatic exposure algorithms overexpose alarm images and videos triggered by objects entering the frame, making it impossible to identify the object, resulting in a significant amount of invalid nighttime alarm data. Using multi-frame fusion technology, wide dynamic range processing is performed when an object enters the frame. This results in numerous ghosting and trailing areas behind the object in the alarm and preview images, as well as abnormal brightness and color in the fusion transition zone. This also compromises the clarity of moving objects and the overall image, severely impacting the image quality of the preview and alarm data. Furthermore, short exposures lack sufficient response speed and accuracy, often failing to effectively identify moving objects.

[0004] For example, the prior art, CN201711318583.2, discloses: 1. Constantly calculating the average brightness of the target object detected in the picture and the average brightness of the entire picture; 2. Comparing the difference between the average brightness of the target object and the desired brightness; 3. When the brightness difference of the target object exceeds the set tolerance, changing the target brightness of the entire picture, and adjusting the brightness of the entire picture to achieve the desired brightness of the target object. Summary of the Invention

[0005] The present invention addresses the problems in the prior art of abnormally large-scale tailing and brightness step sensations in images of moving objects and areas where long and short exposures are fused together, as well as insufficient short exposure response speed and accuracy, and provides a method and device for obtaining effective alarm images of moving objects.

[0006] In order to solve the above technical problems, the present invention is solved by the following technical solutions: A method for obtaining an effective alarm image of a moving object, the method comprising: Determining the moving object information based on the initial image frame and subsequent image frames; Acquiring a first exposure value and a second exposure value of a moving object, acquiring the first exposure value of the moving object through a camera, calculating the second exposure value of the moving object based on information about the moving object, and caching an object motion frame image corresponding to the second exposure value of the object; Display of moving object images: the first exposure value of the moving object is used for displaying real-time moving object images, and the moving frame images cached at the second exposure value of the moving object are used for displaying moving object images during alarm.

[0007] Preferably, the information of the moving object includes the direction of the moving object, the speed of the moving object and the pixel region R where the moving object is located.

[0008] Preferably, determining the moving object information includes: The pixel frame difference D(x, y, t) of a moving object is calculated based on the initial frame image I(x, y, t-Δt) and the subsequent frame image I(x, y, t). D(x, y, t) = I(x, y, t) - I(x, y, t-Δt), where Δt is the frame interval. Determine the parameters of the moving object, including its direction and speed dp(t) / dt, and the pixel region R where the moving object is located. Calculate the direction and speed of the moving object and the region of the moving object using the pixel frame difference of the moving object. Among them, dp(t) / dt = [dx / dt, dy / dt]^T; R=P(x,y), where T is the transpose operator that converts a row vector into a column vector.

[0009] Preferably, the second exposure value H1 of the moving object is calculated based on the brightness corresponding to the image of the moving object at different positions and the first exposure value H0. H1=B*H0 / C; Wherein, B is the expected brightness of the moving object, C is the predicted brightness of the pixel region R2 of the next frame when the moving object is located in the characteristic pixel region R1, and H0 is the first exposure value of the moving object image.

[0010] Preferably, the predicted brightness C of the pixel region R2 of the next frame when the moving object is located in the characteristic pixel region R1 is calculated based on the fill light energy and the first exposure value corresponding to different positions of the moving object image; C= k*H0*f2(φ); Where f2(φ) is the fill light energy function fitted according to the fill light distribution curve, k is the lens aperture conversion coefficient related to the lens and sensor; φ is the fitting angle.

[0011] Preferably, the field of view angle φ corresponding to the pixel position (x, y) is determined by the field of view angle function according to the corresponding relationship between the lens image height and the field of view angle and the pixel size of the imaging chip; φ=f1(x0+v x Δt,y0+v y Δt); Among them, f1(…) is the distribution function, x0 is the x coordinate of the starting frame image, and y0 is the starting frame image y coordinate, v x is the speed in the x direction, v x =dx / dt; v y is the velocity in the y direction, v y =dy / dt; Δt is the frame interval.

[0012] In order to solve the above technical problems, the present application also provides a device for obtaining an effective alarm image of a moving object, which is used to implement any of the above methods for obtaining an effective alarm image of a moving object, and includes: A moving object information determination module determines the moving object information based on the initial image frame and subsequent image frames; A module for acquiring a first exposure value and a second exposure value of a moving object, which acquires a first exposure value of the moving object through a camera, calculates a second exposure value of the moving object based on information about the moving object, and caches a motion frame image of the object corresponding to the second exposure value of the object; The display module of the moving object image uses the first exposure value of the moving object to display the real-time moving object image, and the moving frame image cached by the second exposure value of the moving object is used to display the moving object image when an alarm is triggered.

[0013] The present invention has significant technical effects due to the adoption of the above technical solutions.

[0014] The present invention uses image data frames as differences to determine the motion state and pixel area of ​​an object, and combines the image exposure and the night vision fill light distribution curve to infer the expected brightness of the object's subsequent image frames. By setting the inferred appropriate exposure for the subsequent frames, the overexposure abnormality of the object can be quickly and effectively suppressed.

[0015] This application solves the "either black or white" contradiction of the traditional exposure method, as well as the problems of image quality degradation of multi-frame fusion wide dynamic and insufficient short exposure response efficiency and accuracy.

[0016] The present invention displays image frames with bright overall scenes and good background image recognition in the form of real-time preview, and caches image frames with clear moving objects but poor background brightness separately and uploads them as alarm video images. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of digital image video stream.

[0018] Figure 2 It is the fill light distribution curve.

[0019] Figure 3 It is a schematic diagram of image frame distribution on the device side of the present invention.

[0020] Figure 4 It is a schematic diagram of the real-time preview frame distribution of the present invention.

[0021] Figure 5 It is a schematic diagram of the alarm screen frame distribution of the present invention.

[0022] Figure 6 It is a schematic diagram of judging motion state by pixel difference between previous and next frames according to the present invention. DETAILED DESCRIPTION

[0023] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0024] Example 1

[0025] A method for obtaining an effective alarm image of a moving object, the method comprising: Determining the moving object information based on the initial image frame and subsequent image frames; Acquiring a first exposure value and a second exposure value of a moving object, acquiring the first exposure value of the moving object through a camera, calculating the second exposure value of the moving object based on information about the moving object, and caching an object motion frame image corresponding to the second exposure value of the object; Display of moving object images: the first exposure value of the moving object is used for displaying real-time moving object images, and the moving frame images cached at the second exposure value of the moving object are used for displaying moving object images during alarm.

[0026] The information of the moving object includes the direction of the moving object, the speed of the moving object and the pixel region R where the moving object is located.

[0027] The determination of moving object information includes: The pixel frame difference D(x, y, t) of a moving object is calculated based on the initial frame image I(x, y, t-Δt) and the subsequent frame image I(x, y, t). D(x, y, t) = I(x, y, t) - I(x, y, t-Δt), where Δt is the frame interval. Determine the parameters of the moving object, including its direction and speed dp(t) / dt, and the pixel region R where the moving object is located. Calculate the direction and speed of the moving object and the region of the moving object using the pixel frame difference of the moving object. Among them, dp(t) / dt = [dx / dt, dy / dt]^T; R=P(x,y), where T is the transpose operator that converts a row vector into a column vector.

[0028] The second exposure value H1 of the moving object is calculated based on the brightness of the image of the moving object at different positions and the first exposure value H0. H1=B*H0 / C; Wherein, B is the expected brightness of the moving object, C is the predicted brightness of the pixel region R2 of the next frame when the moving object is located in the characteristic pixel region R1, and H0 is the first exposure value of the moving object image.

[0029] The predicted brightness C of the pixel region R2 of the next frame when the moving object is located in the characteristic pixel region R1 is calculated based on the fill light energy and the first exposure value corresponding to different positions of the moving object image; C= k*H0*f2(φ); Where f2(φ) is the fill light energy function fitted according to the fill light distribution curve, k is the lens aperture conversion coefficient related to the lens and sensor, and φ is the fitting angle.

[0030] The field of view angle φ corresponding to the pixel position (x, y) is determined by the field of view angle function based on the corresponding relationship between the lens image height and the field of view angle and the pixel size of the imaging chip; φ=f1(x0+v x Δt,y0+v y Δt); Among them, f1(…) is the distribution function, x0 is the x coordinate of the starting frame image, and y0 is the starting frame image y coordinate, v x is the speed in the x direction, v x =dx / dt; v y is the velocity in the y direction; v y =dy / dt; Δt is the frame interval.

[0031] Example 2

[0032] This embodiment is a device for obtaining an effective alarm image of a moving object, which is used to implement a method for obtaining an effective alarm image of a moving object, including: A moving object information determination module determines the moving object information based on the initial image frame and subsequent image frames; A module for acquiring a first exposure value and a second exposure value of a moving object, which acquires a first exposure value of the moving object through a camera, calculates a second exposure value of the moving object based on information about the moving object, and caches a motion frame image of the object corresponding to the second exposure value of the object; The display module of the moving object image uses the first exposure value of the moving object to display the real-time moving object image, and the moving frame image cached by the second exposure value of the moving object is used to display the moving object image when an alarm is triggered.

[0033] Example 3

[0034] Based on Example 1, this embodiment is as above Figure 1 , a digital image consists of multiple static image frames, usually 15 frames, 20 frames, 25 frames, 30 frames, etc., which refer to video data composed of 15, 20, 25, and 30 separate image frames. Figure 3 、 Figure 4 、 Figure 5 signal; When an object enters the picture, the initial picture frame and subsequent frames are subtracted; That is, D(x,y,t) = I(x,y,t) - I(x,y,t-Δt), Where I(x,y,t) represents the pixel intensity at position (x,y) at time t, and Δt is the frame interval. It can be used to determine whether there is a moving object and its trajectory, and thus the velocity vector of the moving object. dp(t) / dt = [dx / dt, dy / dt]T; Among them, dx / dt and dy / dt are the velocity components in the x and y directions respectively, T is the transpose operator to convert the row vector into a column vector. Column vectors are usually used to describe physical quantities such as speed and force to facilitate matrix multiplication), and the pixel area where the object is located R=P(x,y), P represents the pixel set at position (x,y), such as Figure 6 As shown; The camera lens and CMOS can uniquely determine the field of view of the image at different image heights, which can be achieved by using the comparison table shown in Table 1, and can ultimately be fitted into an angle distribution function φ = f1 (x, y). Figure 2The fill light distribution curve reflects the fill light intensity distribution function Y=f2(φ)= f2(f1(x,y)). The fill light energy corresponding to different positions in the image can be calculated using f2(f1(x,y)).

[0035] Table 1 Comparison table of field of view angles at different image height positions.

[0036]

[0037] Count the average brightness A of the pixel area where the object is located. Assuming that the object's suitable brightness is B, the current object is located in the area R=P(x0,y0). After one frame of time, calculate the position of the moving object; R = P(x0+v x Δt,y0+v y Δt); Among them, v x = dx / dt, v y = dy / dt, v x 、v y Represent the speed of movement in the x and y directions respectively, and Δt is the frame interval time. Then the fill light intensity corresponding to the position of the moving object in the next frame is: Y=f2(f1(x0+v x Δt,y0+v y Δt)) The first exposure H0 is the initial exposure (shutter time + gain), A=k*H0f2(f1(x0,y0)); Where k is the lens aperture conversion coefficient, k=A / H0f2(f1(x0,y0)); The predicted brightness C of the next frame pixel area R2 of the feature pixel area R1; C= k*H0f2(f1(x0+v x Δt,y0+v y Δt)).

[0038] Then force the next frame exposure, i.e. the second exposure H1; H1=B*H0 / C=B*H0 / kf2(f1(x0+v x Δt,y0+v y Δt))= =B*H0f2(f1(x0,y0)) / Af2(f1(x0+v x Δt,y0+v y Δt)); At this time, the brightness of the object in the next frame will be more appropriate, and the exposure frame with the appropriate brightness of the object is extracted and stored in the buffer area.

[0039] The initial exposure H0 is maintained for the subsequent frames, such as frames 2 to 5, and the above steps are continued until the object leaves the frame or stops moving.

[0040] Upload the exposure frame with the appropriate brightness of the object as the alarm data, and use the high-brightness image captured with the original exposure H0 as the real-time preview image. This ensures the validity of the alarm data and allows it to be presented in the form of a low-frame rate video stream, while ensuring the brightness stability of the entire scene and background recognition.

Claims

1. A method for obtaining an effective alarm image of a moving object, the method comprising: Determining the moving object information based on the initial image frame and subsequent image frames; Acquiring a first exposure value and a second exposure value of a moving object, acquiring the first exposure value of the moving object through a camera, calculating the second exposure value of the moving object based on information about the moving object, and caching an object motion frame image corresponding to the second exposure value of the object; Display of moving object images: the first exposure value of the moving object is used for displaying real-time moving object images, and the moving frame images cached at the second exposure value of the moving object are used for displaying moving object images during alarm.

2. The method for obtaining an effective alarm image of a moving object according to claim 1, characterized in that: The information of the moving object includes the direction of the moving object, the speed of the moving object and the pixel region R where the moving object is located.

3. The method for obtaining an effective alarm image of a moving object according to claim 1, characterized in that: The determination of moving object information includes: The moving object pixel frame difference D(x, y, t) is calculated based on the starting frame image I(x, y, t-Δt) and the subsequent frame image I(x, y, t). D(x, y, t) = I(x, y, t) - I(x, y, t-Δt); where Δt is the frame interval. Determine the parameters of the moving object, including its direction and speed dp(t) / dt, and the pixel region R where the moving object is located. Calculate the direction and speed of the moving object and the region of the moving object using the pixel frame difference of the moving object. Among them, dp(t) / dt = [dx / dt, dy / dt]^T; R=P(x,y), where T is the transpose operator that converts a row vector into a column vector.

4. The method for obtaining an effective alarm image of a moving object according to claim 1, characterized in that: The second exposure value H1 of the moving object is calculated based on the brightness of the image of the moving object at different positions and the first exposure value H0. H1=B*H0 / C; Wherein, B is the expected brightness of the moving object, C is the predicted brightness of the pixel region R2 of the next frame when the moving object is located in the characteristic pixel region R1, and H0 is the first exposure value of the moving object image.

5. The method for obtaining an effective alarm image of a moving object according to claim 4, characterized in that: The predicted brightness C of the pixel region R2 of the next frame when the moving object is located in the characteristic pixel region R1 is calculated based on the fill light energy and the first exposure value corresponding to different positions of the moving object image; C= k*H0*f2(φ); Where f2(φ) is the fill light energy function fitted according to the fill light distribution curve, k is the lens aperture conversion coefficient related to the lens and sensor, and φ is the fitting angle.

6. The method for obtaining an effective alarm image of a moving object according to claim 5, characterized in that: The field of view angle φ corresponding to the pixel position (x, y) is determined by the field of view angle function based on the corresponding relationship between the lens image height and the field of view angle and the pixel size of the imaging chip; φ=f1(x0+v x Δt,y0+v y Δt); Among them, f1(…) is the distribution function, x0 is the x coordinate of the starting frame image, y0 is the y coordinate of the starting frame image, v x is the speed in the x direction, v x =dx / dt; v y is the velocity in the y direction; v y =dy / dt; Δt is the frame interval.

7. A device for obtaining effective alarm images of moving objects, characterized in that: A method for obtaining an effective alarm image of a moving object according to any one of claims 1 to 6, comprising: A moving object information determination module determines the moving object information based on the initial image frame and subsequent image frames; A module for acquiring a first exposure value and a second exposure value of a moving object, which acquires a first exposure value of the moving object through a camera, calculates a second exposure value of the moving object based on information about the moving object, and caches a motion frame image of the object corresponding to the second exposure value of the object; The display module of the moving object image uses the first exposure value of the moving object to display the real-time moving object image, and the moving frame image cached by the second exposure value of the moving object is used to display the moving object image when an alarm is triggered.

Images