A real-time bullet hole detection algorithm based on dynamic background estimation

Abstract

A kind of bullet hole real-time detection algorithm based on dynamic background estimation, comprising the following steps: S1, the video image of real-time target surface is obtained by image acquisition device, and initial image data stream is obtained by frame extraction operation;S2, the initial image data stream is preprocessed to obtain preprocessed image;S3, the preprocessed image is processed based on time and space image de-bobbing positioning algorithm, eliminate target surface shaking and locate the detection area of image after eliminating shaking;S4, the detection area obtained by the above S3 is used to construct complete initial background image, and the detection area of each subsequent frame image is continuously background estimated;S5, the continuous background estimation result obtained in S4 is forward difference with initial background image, whether the current frame image has new bullet hole is judged according to difference result;S6, background image is updated according to bullet hole detection result and image time flow.

Description

Technical Field

[0001] This invention relates to the field of bullet hole detection technology, specifically a real-time bullet hole detection algorithm based on dynamic background estimation. Background Technology

[0002] Soldiers improve their shooting accuracy through shooting training. During live-fire training, target reporting is mainly done manually. After shooting, a designated person goes to the corresponding target position, removes the target paper, checks the number of bullet holes and their corresponding ring numbers, and then manually counts the total number of rings. This target reporting method is not only labor-intensive, but also relatively slow, has low accuracy, and cannot report ring count information in real time.

[0003] In existing technologies, there are also a few automatic target reporting methods that improve the speed of target reporting and enable real-time target reporting.

[0004] Currently, automatic target reporting includes sensor system target reporting and image processing bullet hole recognition technology. Among them, sensor system target reporting is relatively expensive. Image processing bullet hole recognition technology analyzes the acquired target surface image in real time during shooting, analyzes the changes in the target surface image to determine whether there are new bullet holes and the location of the new bullet holes. However, during shooting, the bullet hitting the target surface causes the target surface to shake, and the shaking can last for a period of time, resulting in blurry images, which can easily lead to false alarms or missed alarms. The accuracy of the bullet hole detection ring count is not high. To accurately detect whether there are new bullet holes on the target surface and the number of bullet holes, it is necessary to wait for the target surface to stabilize before detection, which increases the shooting interval. Summary of the Invention

[0005] To address the issue of low accuracy in bullet hole detection in existing technologies, this invention provides a real-time bullet hole detection algorithm based on dynamic background estimation. This algorithm can detect newly added bullet holes on the target surface in real time, eliminating the impact of target surface jitter on the detection results and improving the accuracy of bullet hole detection.

[0006] To achieve the above objectives, the specific solution adopted by this invention is as follows: a real-time bullet hole detection algorithm based on dynamic background estimation, comprising the following steps:

[0007] S1, acquire real-time video images of the target surface through an image acquisition device, and perform frame extraction to obtain the initial image data stream;

[0008] S2, preprocess the initial image data stream to obtain a preprocessed image;

[0009] S3, a time- and space-based image de-shaking and localization algorithm, processes the preprocessed image to eliminate target surface shaking and locate the detection area of ​​the image after shaking removal;

[0010] S4. Construct a complete initial background image using the detection region obtained in S3 above, and perform continuous background estimation on the detection region of each subsequent frame image.

[0011] S5, perform a forward difference between the continuous background estimation result obtained in S4 and the initial background image, and determine whether there are any new bullet holes in the current frame image based on the difference result;

[0012] S6 updates the background image based on the bullet hole detection results and the time flow of the image.

[0013] As an optimization scheme for the above-mentioned real-time bullet hole detection algorithm based on dynamic background estimation: In S1, the target position in the initial image data stream is located in the middle of the image, and the area of ​​the target surface region occupies one-quarter to one-third of the total area of ​​the initial image data stream.

[0014] As an alternative optimization of the aforementioned real-time bullet hole detection algorithm based on dynamic background estimation, S2 includes:

[0015] S21, calibrate the image acquisition device to obtain its intrinsic and extrinsic parameters, and perform distortion correction on the initial image data stream based on the intrinsic and extrinsic parameters;

[0016] S22, the image processed by S21 above is converted to grayscale using a three-channel weighting method to obtain a grayscale image;

[0017] S23, perform convolution filtering on the above grayscale image to reduce noise while enhancing the detail information of the grayscale image, and obtain a filtered image;

[0018] S24: Preset the original point pairs and target point pairs, and correct the full-size filtered image to a custom target size to obtain a preprocessed image.

[0019] As another optimization scheme for the above-mentioned real-time bullet hole detection algorithm based on dynamic background estimation: In S24, the target size is calculated by the actual target surface area and corresponds to the number of rings on the target surface.

[0020] As an alternative optimization of the aforementioned real-time bullet hole detection algorithm based on dynamic background estimation, S3 includes:

[0021] S31, in the time dimension, adjust the weights of each preprocessed image frame to eliminate target surface jitter:

[0022] NFrame(x,y)=bFrame(x,y)×W0+nFrame(x,y)×W1

[0023] Where NFrame is the image after jitter removal in the current frame, nFrame is the preprocessed image in the current frame, bFrame is the preprocessed image in the previous frame, (x,y) is the pixel position, W0 and W1 are weights, W0+W1=1, and W0>W1.

[0024] S32, the target surface has pre-set spiral marks, and the detection area is located spatially using the spiral mark contour tree:

[0025]

[0026] Where Fc is the set of contours in the image after jitter removal in the current frame, Fp is the center point of the corresponding contour, Fd is the vertex of the detection region in the image after jitter removal in the current frame, and f(F c ,F p ) represents the mapping from the center point of the contour, φ(F) p ,F d This represents the mapping from the center point coordinates to the image coordinates.

[0027] As another optimization scheme for the above-mentioned real-time bullet hole detection algorithm based on dynamic background estimation, S32 includes:

[0028] S321, Perform custom Wiener filtering on the image after jitter removal, and set a local adaptive threshold to perform partition binarization on the above image to obtain a binarized image;

[0029] S322 performs edge detection on the binarized image using a local two-dimensional pattern operator and records all detected edge contours;

[0030] S323, the contour features of the herringbone mark are used to filter the edge contours and locate the position of the herringbone mark. The center of the herringbone mark is extracted to obtain the four vertices of the target area.

[0031] S323, the coordinates of the four vertices of the target area are mapped onto the image after jitter removal and this part of the area is cropped as the detection area.

[0032] As an alternative optimization of the aforementioned real-time bullet hole detection algorithm based on dynamic background estimation, S4 includes:

[0033] S41, preset pixel threshold;

[0034] S42, Perform pixel value difference and take the absolute value of the detection area of ​​three consecutive adjacent frames of images after jitter removal;

[0035] S43, compare the absolute value of the pixel value difference with the pixel threshold. If the absolute value of the pixel value difference is greater than the preset threshold, it is determined that the pixel has changed in the image after the jitter is eliminated in three consecutive adjacent frames.

[0036] S44, remove all changed pixels in the image after jitter removal to obtain the background image of the current frame;

[0037] S45. Repeat steps S41-S44. After 20-30 updates and iterations, a complete initial background image is obtained.

[0038] As an alternative optimization of the aforementioned real-time bullet hole detection algorithm based on dynamic background estimation, S5 includes:

[0039] S51, preset conditional threshold for image pixels;

[0040] S52 determines the location and degree of change in the detection area by the pixel life value increase / decrease mechanism and energy concentration. If the degree of change in 3 to 5 consecutive frames is above the condition threshold, it indicates that there is a new bullet hole in the current frame image; otherwise, it is determined that there is no new bullet hole in the current frame image.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. This invention provides a real-time bullet hole detection algorithm based on dynamic background estimation. The preprocessed image is processed by a temporal and spatial image de-jitter localization algorithm to eliminate target surface jitter and locate the effective area of ​​the image after jitter elimination. This can eliminate the influence of target surface jitter on the detection results and improve the accuracy of bullet hole detection.

[0043] 2. In this invention, the initial image data stream is preprocessed to reduce the computational load of the algorithm and improve its running speed while preserving the effective area.

[0044] 3. In this invention, the background image is updated based on the bullet hole detection results and image changes, enabling real-time and accurate detection of newly added bullet holes. Detailed Implementation

[0045] The technical solution of the present invention will be further described in detail below with reference to specific embodiments. Parts not described or disclosed in detail in the following embodiments of the present invention should be understood as prior art known or should be known by those skilled in the art.

[0046] A real-time bullet hole detection algorithm based on dynamic background estimation includes the following steps:

[0047] S1. Acquire real-time video images of the target surface through an image acquisition device, and perform frame extraction to obtain an initial image data stream. When acquiring real-time video images of the target surface, the clarity of the video images should be no less than 720P. At the same time, a spiral mark is set on the target surface.

[0048] In S1, the target position in the initial image data stream is located in the middle of the image, and the area of ​​the target surface occupies one-quarter to one-third of the total area of ​​the initial image data stream.

[0049] S2 preprocesses the initial image data stream to obtain a preprocessed image, eliminating image distortion caused by the image acquisition device itself and image warping caused by the device's installation position, adjusting the image size, and removing redundancy. While retaining the effective information of the initial image data stream, it significantly reduces the computational load of the algorithm and improves the algorithm's running speed.

[0050] Specifically, S2 includes:

[0051] S21, the image acquisition device is calibrated to obtain its intrinsic and extrinsic parameters, and distortion correction is performed on the initial image data stream based on these parameters. The intrinsic parameters include focal length, radial distortion level, scaling factor, principal point position, and image size; the extrinsic parameters include translation vector and rotation matrix. Distortion correction of the initial image data stream using these intrinsic and extrinsic parameters eliminates image distortion caused by the image acquisition device itself and its installation location.

[0052] S22: The image processed in S21 is converted to grayscale using a three-channel weighted method to obtain a grayscale image. The image after distortion correction is a color image. The color image is mapped to the RGB color gamut, and then mapped to the grayscale gamut using a three-channel weighted method. This compresses the data volume, reduces the computational load of the algorithm, and thus improves the running speed of the algorithm.

[0053] S23, perform convolution filtering on the grayscale image to reduce noise while enhancing detail information, resulting in a filtered image; this facilitates subsequent detection of the effective region.

[0054] S24: Preset original point pairs and target point pairs, correct the full-size filtered image to a custom target size to obtain a preprocessed image. The target size is calculated based on the actual target surface size, and its ring number corresponds to the ring number on the actual target surface. This removes redundant parts of the filtered image.

[0055] S3, based on a temporal and spatial image de-jitter localization algorithm, processes the preprocessed image to eliminate target surface jitter and locate the detection area of ​​the image after jitter elimination. Temporally, it adjusts the weight of the pixels in the next frame of the preprocessed image in the current frame of the preprocessed image to eliminate the influence of jitter on the preprocessed image. Spatially, it determines the target surface detection area through an image localization algorithm, thereby improving the accuracy of the invention in detecting bullet holes and the number of bullet hole rings.

[0056] S3 includes:

[0057] S31, in the time dimension, adjust the weights of each preprocessed image frame to eliminate target surface jitter:

[0058] NFrame(x,y)=bFrame(x,y)×W0+nFrame(x,y)×W1

[0059] Wherein, NFrame is the image after jitter removal in the current frame, nFrame is the preprocessed image in the current frame, bFrame is the preprocessed image in the next frame, (x,y) is the pixel position, W0 and W1 are weights, W0+W1=1, and W0>W1, and W0 and W1 can be adjusted according to the actual scene.

[0060] S32, the target surface has pre-set spiral marks, and the detection area is located spatially using the spiral mark contour tree:

[0061]

[0062] Where Fc is the set of contours in the image after jitter removal in the current frame, Fp is the center point of the corresponding contour, Fd is the vertex of the effective region in the image after jitter removal in the current frame, and f(F c ,F p ) represents the mapping from the center point of the contour, φ(F) p ,F d This represents the mapping from the center point coordinates to the image coordinates.

[0063] Specifically, S32 includes:

[0064] S321, Perform custom Wiener filtering on the image after jitter elimination, and set a local adaptive threshold to perform partition binarization on the above image to obtain a binarized image; partition binarization of the Wiener-filtered image is used to adapt to uneven lighting conditions in nature.

[0065] S322 performs edge detection on the binarized image using a local two-dimensional pattern operator and records all detected edge contours.

[0066] S323, the three-level chain contour feature of the spiral mark filters the edge contour, locates the position of the spiral mark, and extracts the center of the spiral mark to obtain the four vertices of the target area;

[0067] S323, the coordinates of the four vertices of the target area are mapped onto the jitter-free image, and this portion is cropped as the detection area. The cropped detection area serves as the data source for background estimation in subsequent steps.

[0068] S4. Construct a complete initial background image using the detection area obtained in S3 above, and perform continuous background estimation on the effective area of ​​each subsequent frame image.

[0069] This embodiment uses three consecutive frames of images I n I n+1 I n+2 For example, the steps include:

[0070] S41, preset pixel threshold.

[0071] S42, perform pixel value difference analysis on the detection area of ​​three consecutive adjacent frames after jitter removal and take the absolute value; subtract the pixel value of the current frame from the pixel value of the next frame image to obtain comparison difference images D1 and D2, i.e., D1 = I n+1 -I n D2 = I n+2 -I n+1 And take the absolute values ​​of D1 and D2.

[0072] S43, compare the absolute value of the pixel value difference with the pixel threshold. If the absolute value of the pixel value difference is greater than the preset threshold, then it is determined that the pixel has changed in the image after jitter removal in three consecutive adjacent frames. Specifically, when the pixel value of the same pixel on D1 and D2 is greater than the preset threshold, then the pixel is in I... n I n+1 I n+2 All of the above are subject to change.

[0073] S44, remove all changed pixels on the image after jitter removal to obtain the background image of the current frame. Specifically, remove all changed pixels in the detection area of ​​the current frame to obtain the background image of the current frame.

[0074] S45. Repeat steps S41-S44. After 20-30 updates and iterations, a complete initial background image is obtained.

[0075] S5. Perform a forward difference between the continuous background estimation result obtained in S4 and the initial background image, and determine whether there are any new bullet holes in the current frame image based on the difference result.

[0076] S51, a preset conditional threshold for image pixels, used to compare with the degree of change in the detection area.

[0077] S52, the position and degree of change in the detection area are determined by the pixel life value increase / decrease mechanism and energy concentration. The degree of change in the detection area of ​​the current frame is compared with the condition threshold. If the degree of change in the detection area of ​​the current frame is less than or equal to the condition threshold, it indicates that there is no new bullet hole in the current detection area. If the degree of change in the detection area of ​​the current frame is greater than the condition threshold, the duration of this degree of change is determined. If the degree of change in the detection area of ​​3 to 5 consecutive frames is above the condition threshold, it indicates that there is a new bullet hole in the current frame image. Otherwise, it is determined that there is no new bullet hole in the current frame image. That is, the degree of change in the detection area of ​​the current frame does not continue to the monitoring area of ​​the next frame, so there is no new bullet hole in the current frame detection area.

[0078] The life value increment / decrement mechanism is used to statistically analyze pixel changes. First, a pixel life map of the same size as the monitoring area is constructed, and the life values ​​of the corresponding pixels are recorded. The life values ​​of the pixels are updated based on the results of the background image difference. The life values ​​of pixels that have changed are incremented, and the life values ​​of pixels that have not changed are decremented, thereby determining the specific location of the change within the detection area. Finally, the energy concentration of the local window within the changed area is calculated to determine whether it is a newly added bullet hole.

[0079] S6. Update the background image based on the bullet hole detection results and the temporal flow of the image. The update of the background image is related to whether there are any new bullet holes and the temporal flow of the image. If it is determined that a new bullet hole has appeared in the detection area of ​​the current frame, count all pixels of the bullet hole and its area, and update the background image by replacing the gray value of the pixel at the same position in the background image with the gray value of the pixel in the current frame image. If no new bullet holes are added for a certain period of time, the background image is updated automatically.

[0080] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A real-time bullet hole detection algorithm based on dynamic background estimation, characterized in that, Includes the following steps: S1, acquire real-time video images of the target surface through an image acquisition device, and perform frame extraction to obtain the initial image data stream; In S1, the target position in the initial image data stream is located in the middle of the image, and the area of ​​the target surface region occupies one-quarter to one-third of the total area of ​​the initial image data stream. S2, preprocessing the initial image data stream to obtain a preprocessed image; S2 includes: S21, calibrate the image acquisition device to obtain its intrinsic and extrinsic parameters, and perform distortion correction on the initial image data stream based on the intrinsic and extrinsic parameters; S22, the image processed by S21 above is converted to grayscale using a three-channel weighting method to obtain a grayscale image; S23, perform convolution filtering on the above grayscale image to reduce noise while enhancing the detail information of the grayscale image, and obtain a filtered image; S24, preset the original point pairs and target point pairs, and correct the full-size filtered image to a custom target size to obtain a preprocessed image; S3, a time- and space-based image de-jitter localization algorithm, processes the preprocessed image to eliminate target surface jitter and locate the detection area of ​​the image after jitter removal; S3 includes: S31, in the time dimension, adjust the weights of each preprocessed image frame to eliminate target surface jitter: in, The image after removing jitter from the current frame. Preprocess the image for the current frame. The image is a preprocessed image of the previous frame, where (x, y) is the pixel position, W0 and W1 are the weights, W0 + W1 = 1, and W0 > W1; S32, the target surface has pre-set spiral marks, and the detection area is located spatially using the spiral mark contour tree: Where Fc is the set of contours in the image after jitter removal in the current frame, Fp is the center point of the corresponding contour, and Fd is the vertex of the detection region in the image after jitter removal in the current frame. Represents the mapping from the center point of the contour This represents the mapping from the center point coordinates to the image coordinates; S32 includes: S321, Perform custom Wiener filtering on the image after jitter removal, and set a local adaptive threshold to perform partition binarization on the above image to obtain a binarized image; S322 performs edge detection on the binarized image using a local two-dimensional pattern operator and records all detected edge contours; S323, the contour features of the herringbone mark are used to filter the edge contours and locate the position of the herringbone mark. The center of the herringbone mark is extracted to obtain the four vertices of the target area. S323, the coordinates of the four vertices of the target area are mapped onto the image after jitter removal and this part of the area is cropped as the detection area; S4, constructing a complete initial background image using the detection regions obtained in S3, and performing continuous background estimation on the detection regions of each subsequent frame; S4 includes: S41, preset pixel threshold; S42, Perform pixel value difference and take the absolute value of the detection area of ​​three consecutive adjacent frames of images after jitter removal; S43, compare the absolute value of the pixel value difference with the pixel threshold. If the absolute value of the pixel value difference is greater than the preset threshold, it is determined that the pixel has changed in the image after the jitter is eliminated in three consecutive adjacent frames. S44, remove all changed pixels in the image after jitter removal to obtain the background image of the current frame; S45, repeat steps S41-S44, and after 20-30 updates and iterations, obtain a complete initial background image; S5, perform a forward difference between the continuous background estimation result obtained in S4 and the initial background image, and determine whether there are any new bullet holes in the current frame image based on the difference result; S6 updates the background image based on the bullet hole detection results and the time flow of the image.

2. The real-time bullet hole detection algorithm based on dynamic background estimation as described in claim 1, characterized in that: In S24, the target size is calculated from the actual target surface area and corresponds to the number of rings on the target surface.

3. The real-time bullet hole detection algorithm based on dynamic background estimation as described in claim 1, characterized in that: S5 includes: S51, preset conditional threshold for image pixels; S52 determines the location and degree of change in the detection area by the pixel life value increase / decrease mechanism and energy concentration. If the degree of change in 3 to 5 consecutive frames is above the condition threshold, it indicates that there is a new bullet hole in the current frame image; otherwise, it is determined that there is no new bullet hole in the current frame image.

Images