A method and apparatus for acquiring an effective alarm image of a moving object

Abstract

The present application relates to image processing technology, disclose a kind of method and device for obtaining effective alarm image of moving object, its method includes: the determination of moving object information, according to initial picture frame and subsequent picture frame, the determination of moving object information;The calculation of moving object first exposure and moving object second exposure, the second exposure of moving object is calculated according to the information of moving object, and the object motion frame image corresponding to object second exposure is cached;Moving object first exposure is used for the display of real-time moving object image, and the motion frame image cached by moving object second exposure is used for the display of moving object image when alarm.The whole scene bright, the image frame with better background image recognition is displayed in the form of real-time preview, and the image frame with clear and discernible moving object but poor background brightness is cached separately and uploaded as alarm video image.

Description

Technical Field

[0001] This invention relates to image processing technology, and more particularly to a method and apparatus for acquiring effective alarm images of moving objects. Background Technology

[0002] Currently, cameras have increasingly higher requirements for night vision distance and low-light image performance. The light sensitivity of CMOS sensors is improving, and the brightness of supplementary lights is increasing. However, digital image processing technology typically uses a central weighting method for exposure control, leading to overexposure of objects 1-3 meters away from the camera. As the object moves closer and occupies a certain proportion of the frame, the exposure is rapidly reduced to prevent overexposure, but this results in a dark background and color cast anomalies. It's difficult to simultaneously maintain both brightness and clarity in distant backgrounds. While multi-frame fusion wide dynamic range technology can ensure background visibility while suppressing overexposed areas, it exhibits significant trailing and brightness abruptness in areas with moving objects and areas where long and short exposures are blended. Furthermore, its short exposure response speed and accuracy are insufficient.

[0003] Conventional automatic exposure algorithms often result in overexposed alarm images and videos triggered when an object enters the frame, making it impossible to identify the object and leading to a large amount of invalid alarm data at night. While multi-frame fusion technology performs wide dynamic range processing on objects entering the frame, alarm images and previews often exhibit significant ghosting, trailing areas, and abnormal brightness and color in the fusion transition zone behind the object. Furthermore, the clarity of the moving object and the overall image is somewhat sacrificed, severely impacting the quality of the preview and alarm data images. Additionally, short exposure response speed and accuracy are insufficient, typically failing to effectively identify moving objects.

[0004] As disclosed in the prior art, CN201711318583.2, the following methods are used: 1. Calculate the average brightness of the target object detected in the image and the average brightness of the entire image in real time; 2. Compare the difference between the average brightness of the target object and the desired brightness; 3. When the brightness difference of the target object exceeds the set tolerance, change the target brightness of the entire image, and adjust the brightness of the target object to the desired state based on the adjustment of the brightness of the entire image. Summary of the Invention

[0005] This invention addresses the problems in existing technologies regarding the abnormal large-scale trailing and brightness abruptness in images of moving objects and areas of excessive fusion between long and short exposures, as well as the insufficient response speed and accuracy of short exposures. It provides a method and apparatus for acquiring effective alarm images of moving objects.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A method for acquiring a valid alarm image of a moving object, the method comprising:

[0008] The determination of moving object information is based on the initial frame and subsequent frames.

[0009] The first exposure and the second exposure of the moving object are obtained by acquiring the first exposure of the moving object through the camera, calculating the second exposure of the moving object based on the information of the moving object, and caching the motion frame image of the object corresponding to the second exposure.

[0010] The display of moving object images: the first exposure of the moving object is used for real-time display of the moving object image, and the second exposure of the moving object is used to buffer the motion frame image for display of the moving object image when an alarm is triggered.

[0011] Preferably, the information of the moving object includes the direction of the moving object, the speed of the moving object, and the pixel region R in which the moving object is located.

[0012] Preferably, the determination of information about the moving object includes:

[0013] The pixel frame difference D(x, y, t) of a moving object is calculated based on the difference between the initial frame image I(x, y, t-Δt) and the subsequent frame image I(x, y, t) of the moving object. D(x, y, t) = I(x, y, t) - I(x, y, t-Δt); where Δt is the frame interval.

[0014] The parameters of the moving object are determined, including the direction and velocity dp(t) / dt of the moving object, and the pixel region R in which the moving object is located. The direction, velocity and region of the moving object are calculated by the pixel frame difference of the moving object.

[0015] Among them, dp(t) / dt = [dx / dt, dy / dt]^T;

[0016] R = P(x, y), where T is the transpose operator for converting a row vector into a column vector.

[0017] Preferably, the second exposure H1 of the moving object is calculated based on the brightness of the image of the moving object at different positions and the first exposure H0.

[0018] H1 = B * H0 / C;

[0019] Where B is the expected brightness of the moving object, C is the predicted brightness of the moving object located in the next frame pixel region R2 of the feature pixel region R1, and H0 is the first exposure of the moving object image.

[0020] Preferably, the predicted brightness C of the next frame pixel region R2 of the moving object located in the feature pixel region R1 is calculated based on the fill light energy and the first exposure of the moving object image at different positions.

[0021] C = k * H0 * f2(φ);

[0022] Where f2(φ) is the energy function of the fill light fitted based on the fill light distribution curve, k is the lens aperture conversion coefficient related to the lens and sensor, and φ is the fitted angle.

[0023] Preferably, the field of view angle φ corresponding to the pixel position (x,y) is determined based on the relationship between the lens image height and the field of view angle and the pixel size of the imaging chip;

[0024] φ=f1(x0+v x Δt,y0+v y Δt);

[0025] Where f1(...) is the distribution function, x0 is the x-coordinate of the starting frame image, and y0 is the starting frame image.

[0026] y-coordinate, v x Let v be the velocity in the x-direction. x =dx / dt; v y v is the velocity in the y-direction. y =dy / dt; Δt is the frame interval.

[0027] To address the aforementioned technical problems, this application also provides an apparatus for acquiring a valid alarm image of a moving object, which implements any of the methods for acquiring a valid alarm image of a moving object, comprising:

[0028] The moving object information determination module determines the moving object information based on the initial frame and subsequent frames.

[0029] The module for acquiring the first exposure and the second exposure of a moving object acquires the first exposure of the moving object through the camera, calculates the second exposure of the moving object based on the information of the moving object, and caches the motion frame image of the object corresponding to the second exposure.

[0030] The moving object image display module uses a first exposure value for real-time display of the moving object image and a second exposure value for buffering motion frame images for displaying the moving object image when an alarm occurs.

[0031] The present invention has significant technical effects due to the adoption of the above technical solutions.

[0032] This invention uses the difference between image data frames to determine the motion state and pixel area of ​​an object, and combines the image exposure and the light distribution curve of the night vision fill light to calculate the expected brightness of the object in subsequent image frames. By setting the calculated appropriate exposure for subsequent frames, the overexposure of the object can be quickly and effectively suppressed.

[0033] This application resolves the contradiction of the traditional exposure method of "black and white", as well as the problems of image quality degradation due to multi-frame fusion with wide dynamic range and insufficient efficiency and accuracy of short exposure response.

[0034] This invention displays image frames with a bright overall scene and good background image recognition in real-time preview form, while image frames with clearly identifiable moving objects but poor background brightness are cached separately and uploaded as alarm video images. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of a digital image and video stream.

[0036] Figure 2 This is the light distribution curve of the fill light.

[0037] Figure 3 This is a schematic diagram of the image frame distribution on the device side of the present invention.

[0038] Figure 4 This is a schematic diagram of the real-time preview frame distribution of the present invention.

[0039] Figure 5 This is a schematic diagram of the alarm screen frame distribution of the present invention.

[0040] Figure 6 This is a schematic diagram illustrating the motion state determination based on the pixel difference between consecutive frames according to the present invention. Detailed Implementation

[0041] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0042] Example 1

[0043] A method for acquiring a valid alarm image of a moving object, the method comprising:

[0044] The determination of moving object information is based on the initial frame and subsequent frames.

[0045] The first exposure and the second exposure of the moving object are obtained by acquiring the first exposure of the moving object through the camera, calculating the second exposure of the moving object based on the information of the moving object, and caching the motion frame image of the object corresponding to the second exposure.

[0046] The display of moving object images: the first exposure of the moving object is used for real-time display of the moving object image, and the second exposure of the moving object is used to buffer the motion frame image for display of the moving object image when an alarm is triggered.

[0047] Information about a moving object includes its direction, speed, and the pixel region R in which it is located.

[0048] Determining information about moving objects includes:

[0049] The pixel frame difference D(x, y, t) of a moving object is calculated based on the difference between the initial frame image I(x, y, t-Δt) and the subsequent frame image I(x, y, t) of the moving object. D(x, y, t) = I(x, y, t) - I(x, y, t-Δt); where Δt is the frame interval.

[0050] The parameters of the moving object are determined, including the direction and velocity dp(t) / dt of the moving object, and the pixel region R in which the moving object is located. The direction, velocity and region of the moving object are calculated by the pixel frame difference of the moving object.

[0051] Among them, dp(t) / dt = [dx / dt, dy / dt]^T;

[0052] R = P(x, y), where T is the transpose operator for converting a row vector into a column vector.

[0053] The second exposure value H1 for a moving object is calculated based on the brightness of the moving object's image at different positions and the first exposure value H0.

[0054] H1 = B * H0 / C;

[0055] Where B is the expected brightness of the moving object, C is the predicted brightness of the moving object located in the next frame pixel region R2 of the feature pixel region R1, and H0 is the first exposure of the moving object image.

[0056] The predicted brightness C of the moving object located in the next frame pixel region R2 of the feature pixel region R1 is calculated based on the fill light energy and the first exposure of the moving object image at different positions.

[0057] C = k * H0 * f2(φ);

[0058] Where f2(φ) is the energy function of the fill light fitted based on the fill light distribution curve, k is the lens aperture conversion coefficient related to the lens and sensor, and φ is the fitted angle.

[0059] The field of view angle φ corresponding to the pixel position (x,y) is determined based on the relationship between the lens image height and the field of view angle and the pixel size of the imaging chip.

[0060] φ=f1(x0+v x Δt,y0+v y Δt);

[0061] Where f1(...) is the distribution function, x0 is the x-coordinate of the starting frame image, and y0 is the starting frame image.

[0062] y-coordinate, v x Let v be the velocity in the x-direction. x =dx / dt; v y v is the velocity in the y-direction. y =dy / dt; Δt is the frame interval.

[0063] Example 2

[0064] This embodiment is a device for acquiring a valid alarm image of a moving object, which is used to implement a method for acquiring a valid alarm image of a moving object, comprising:

[0065] The moving object information determination module determines the moving object information based on the initial frame and subsequent frames.

[0066] The module for acquiring the first exposure and the second exposure of a moving object acquires the first exposure of the moving object through the camera, calculates the second exposure of the moving object based on the information of the moving object, and caches the motion frame image of the object corresponding to the second exposure.

[0067] The moving object image display module uses a first exposure value for real-time display of the moving object image and a second exposure value for buffering motion frame images for displaying the moving object image when an alarm occurs.

[0068] Example 3

[0069] Based on Example 1, this example is as follows. Figure 1 Digital images are composed of multiple still image frames, typically 15 frames, 20 frames, 25 frames, 30 frames, etc., referring to video data composed of 15, 20, 25, and 30 individual image frames, respectively. For example... Figure 3 , Figure 4 , Figure 5 Indication;

[0070] When an object enters the frame, the frame difference is calculated between the initial frame and subsequent frames.

[0071] That is, D(x,y,t) = I(x,y,t) - I(x,y,t-Δt),

[0072] Where I(x,y,t) represents the pixel intensity at position (x,y) at time t, and Δt is the frame interval. It can be obtained whether there is a moving object and the trajectory of the moving object, and thus the velocity vector of the moving object can be known.

[0073] dp(t) / dt = [dx / dt, dy / dt]T;

[0074] Where dx / dt and dy / dt are the velocity components in the x and y directions, respectively, T is a transpose operator that converts a row vector into a column vector (column vectors are commonly used to describe physical quantities such as velocity and force for easier matrix multiplication), and R = P(x,y) is the pixel region where the object is located, where P represents the set of pixels at position (x,y). Figure 6 As shown;

[0075] The camera lens and CMOS sensor uniquely determine the field of view at different image heights, as shown in Table 1, and can ultimately be fitted to an angular distribution function φ=f1(x,y). Then... Figure 2 The fill light distribution curve reflects the fill light intensity distribution function Y = f2(φ) = f2(f1(x,y)). Therefore, the fill light energy corresponding to different positions in the image can be calculated from f2(f1(x,y)).

[0076] Table 1. Comparison of field of view angles at different image heights.

[0077]

[0078] The average brightness A of the pixel area where the object is located is calculated. Assuming the suitable brightness of the object is B, the current area where the object is located is R=P(x0,y0). After one frame, the position of the moving object is calculated.

[0079] R = P(x0 + v) x Δt,y0+v y Δt);

[0080] Among them, v x = dx / dt, v y = dy / dt, v x v y Let x and y represent the motion velocities in the x and y directions, respectively, and Δt be the frame interval. Then, the fill light intensity corresponding to the position of the moving object in the next frame is:

[0081] Y=f2(f1(x0+v x Δt,y0+v y Δt))

[0082] The first exposure H0 is the initial exposure (shutter speed + gain).

[0083] A = k * H0f2(f1(x0,y0));

[0084] Where k is the lens aperture conversion factor, k=A / H0f2(f1(x0,y0));

[0085] The predicted brightness C of the next frame pixel region R2 of the feature pixel region R1;

[0086] C = k*H0f2(f1(x0+v) x Δt,y0+v y Δt).

[0087] Then force the exposure of the next frame, namely the second exposure H1;

[0088] H1=B*H0 / C=B*H0 / kf2(f1(x0+v x Δt,y0+v y Δt))= =B*H0f2(f1(x0,y0)) / Af2(f1(x0+v x Δt,y0+v y Δt));

[0089] At this point, the brightness of the object in the next frame will be more suitable, and the exposure frame with suitable brightness will be extracted and stored in the buffer.

[0090] For the next few frames, such as frames 2 to 5, maintain the initial exposure level H0 and continue the above steps until the object leaves the frame or stops moving.

[0091] By uploading exposure frames with appropriate object brightness as alarm data and using the high-brightness images captured at the original exposure level H0 as real-time preview images, the validity of the alarm data can be guaranteed, and low frame rate video streams can be allowed to be presented, while ensuring the overall scene brightness stability and background visibility.

Claims

1. A method for obtaining an effective alarm image of a moving object, the method comprising: determining moving object information based on an initial frame and a subsequent frame of the moving object; obtaining a first exposure of the moving object and a second exposure of the moving object, wherein the first exposure of the moving object is obtained by a camera, the second exposure of the moving object is calculated based on the information of the moving object, and a moving frame image corresponding to the second exposure of the moving object is cached; displaying the moving object image, wherein the first exposure of the moving object is used for displaying a real-time moving object image, and the moving frame image cached by the second exposure of the moving object is used for displaying a moving object image when an alarm occurs; calculating the second exposure H1 of the moving object based on the brightness corresponding to different positions of the moving object image and the first exposure H0 of the moving object H1=B*H0 / C; wherein B is an expected brightness of the moving object, C is a predicted brightness of a next frame pixel region R2 of the moving object located in a characteristic pixel region R1, and H0 is the first exposure of the moving object image; the predicted brightness C of the next frame pixel region R2 of the moving object located in the characteristic pixel region R1 is calculated based on the light energy of a light compensation lamp corresponding to different positions of the moving object image and the first exposure; C=k*H0*f2(φ); wherein f2(φ) is a light compensation lamp energy function fitted based on a light compensation lamp light distribution curve, k is a lens aperture conversion coefficient related to a lens and a sensor, and φ is a fitted angle; a field angle φ corresponding to a pixel position (x, y) is determined based on a relationship between an image height and a field angle and a pixel size of an imaging chip. φ = f1(x0+ v x Δt, y0+ v y Δt); where f1(...) is the distribution function, x0is the starting frame image x coordinate, y0is the starting frame image y coordinate, v x is the x direction motion velocity, v x = dx / dt; v y is the y direction motion velocity; v y = dy / dt; Δt is the frame interval.

2. The method of claim 1, wherein, The information of the moving object includes a direction of the moving object, a speed of the moving object, and a pixel region R in which the moving object is located.

3. The method of claim 1, wherein, The determination of the moving object information comprises: calculating a moving object pixel frame difference D(x, y, t) based on a starting frame image I(x, y, t-Δt) of the moving object and a subsequent frame image I(x, y, t) of the moving object, wherein D(x, y, t)=I(x, y, t)-I(x, y, t-Δt); and Δt is a frame interval; determining parameters of the moving object, wherein the parameters of the moving object include a direction and a speed dp(t) / dt of the moving object, and a pixel region R in which the moving object is located; the direction and the speed of the moving object and the pixel region R in which the moving object is located are calculated based on the moving object pixel frame difference. wherein dp(t) / dt=[dx / dt, dy / dt]^T. R=P(x, y), wherein T is a transpose symbol for converting a row vector into a column vector.

4. A device for acquiring a valid alarm image of a moving object, characterized in that, A method for obtaining an effective alarm image of a moving object according to any one of claims 1-3, the method comprising: a moving object information determination module configured to determine moving object information based on an initial frame and a subsequent frame of the moving object; a moving object first exposure and a moving object second exposure obtaining module configured to obtain a first exposure of the moving object by a camera, calculate a second exposure of the moving object based on the information of the moving object, and cache a moving frame image corresponding to the second exposure of the moving object; and displaying the moving object image, wherein the first exposure of the moving object is used for displaying a real-time moving object image, and the moving frame image cached by the second exposure of the moving object is used for displaying a moving object image when an alarm occurs. The display module of the moving object image, the first exposure quantity of the moving object is used for the display of the real-time moving object image, and the second exposure quantity of the moving object is used for the display of the moving frame image cached in the alarm.

Images