Gun calibration method based on automatic bullet hole detection, medium, equipment and product

Abstract

The invention provides a gun calibration method based on automatic bullet hole detection, a medium, equipment and a product, and relates to the technical field of infrared image processing, and the method comprises the steps: capturing an infrared image of a bullet hole in real time through an infrared thermal imager, carrying out the preprocessing, and constructing a bullet hole detection data set; constructing a bullet hole detection model based on the YOLOv5 network, adding a multi-scale feature fusion module in the YOLOv5 network, and fusing the shallow-layer features, the middle-layer features and the deep-layer features extracted by the backbone network into new deep-layer features; spatial pyramid pooling is replaced by hollow spatial pyramid pooling; inputting the bullet hole detection data set into the model to obtain bullet hole position data; according to the position data of the bullet hole, converting the coordinates of the bullet hole, and calculating the offset of the bullet hole; and according to the bullet hole offset, ballistic deviation is analyzed, and ballistic deviation is corrected. The method can break through environmental limitation, realizes real-time bullet hole detection, and improves gun calibration precision.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared image processing, and in particular to a gun calibration method, medium, equipment and product based on automatic bullet hole detection. Background Art

[0002] In traditional gun calibration, shooters typically rely on manual observation of the distribution of bullet holes on target paper and rely on experience to determine the direction and amplitude of crosshair adjustment. This method is not only inefficient but also subject to significant subjective influences, making it difficult to achieve accurate and repeatable calibration results. With the development of computer vision technology, early research has attempted to combine image processing algorithms (such as edge detection, threshold segmentation, and morphological operations) with geometric transformations (such as affine transformation and perspective transformation) to achieve coarse target image registration and bullet hole detection. However, these methods perform poorly in complex scenarios. For example, when the target surface is wrinkled, stained, has uneven lighting, or is partially occluded, the robustness and accuracy of traditional algorithms are significantly reduced. Furthermore, because bullet holes are typically small (especially at long range), manual feature-based registration methods struggle to achieve sub-pixel positioning accuracy, limiting the practical application value of gun calibration systems. Therefore, algorithm innovation and sensor technology upgrades are urgently needed to improve the system's gun calibration accuracy and environmental adaptability. Summary of the Invention

[0003] The purpose of the present invention is to solve the problem that traditional gun calibration methods are difficult to achieve sub-pixel accuracy, and to propose a gun calibration method based on automatic bullet hole detection, comprising the following steps:

[0004] S1. Use an infrared thermal imager to capture infrared images of bullet holes in real time, perform preprocessing, and construct a bullet hole detection dataset.

[0005] S2. Build a bullet hole detection model based on the improved YOLOv5 network. Replace the spatial pyramid pooling (SPP) in the YOLOv5 backbone network with atrous spatial pyramid pooling (ASPP). Add a multi-scale feature fusion module to the YOLOv5 network. The multi-scale feature fusion module fuses the shallow, middle, and deep features extracted by the backbone network into new deep features, which are then input into the neck network.

[0006] S3, inputting the bullet hole detection data set into the model to obtain bullet hole location data;

[0007] S4. Convert the bullet hole coordinates according to the bullet hole position data and calculate the bullet hole offset;

[0008] S5. Analyze the trajectory deviation based on the bullet hole offset and correct the trajectory offset.

[0009] Furthermore, preprocessing includes: non-uniformity compensation, image denoising, and contrast enhancement.

[0010] Furthermore, the dilated spatial pyramid pooling has five branches. The first branch includes a 1×1 standard convolution, the second to fourth branches all include a 3×3 dilated convolution, and the fifth branch is expressed as:

[0011]

[0012] Among them, B5 represents the output of the fifth branch, F represents the input image of the void space pyramid pooling, GAP() represents the global average pooling operation, Conv 1×1 Represents a 1×1 standard convolution operation, Upsample represents upsampling, H and W represent the height and width of the image F respectively, and F ：,i,j represents the (i, j) pixel of image F;

[0013] The outputs of the five branches of the dilated spatial pyramid pooling are fused and expressed as:

[0014] F ASPP =Conv 1×1 (Concat(B1,B2,B3,B4,B5)),

[0015] Among them, F ASPP Represents the output of dilated spatial pyramid pooling, B1~B5 are the outputs of the five branches of dilated spatial pyramid pooling respectively, and Concat() represents the concatenation operation.

[0016] Furthermore, the output size of the dilated convolution is expressed as:

[0017]

[0018] Among them, O represents the size of the output feature map, I represents the size of the input feature map, p represents the padding size, k represents the equivalent convolution kernel size of the dilated convolution, d represents the dilation rate, and s represents the step size.

[0019] Furthermore, the multi-scale feature fusion module fuses the shallow features, mid-level features, and deep features extracted by the backbone network into new deep features. Specifically:

[0020] The middle-layer features and deep-layer features are adjusted to the same resolution as the shallow-layer features through upsampling, and are concatenated with the shallow-layer features along the channel dimension. Then, the number of channels is adjusted using 1×1 convolution to obtain new deep-layer features.

[0021] Furthermore, S4 is specifically:

[0022] The pixel coordinates of the bullet holes in the bullet hole position data are converted into the physical coordinates of the actual target surface, which can be expressed as follows:

[0023]

[0024] Among them, (X d ,Y d ) represents the physical coordinates of the actual target surface, (u d ,v d ) represents the pixel coordinates of the bullet hole in the position data of the bullet hole, (c x ,c y ) represents the image center coordinate, D represents the shooting distance, f x Indicates the focal length of the camera in the X-axis direction, f y Indicates the focal length of the camera in the Y-axis direction;

[0025] Calculate the trajectory offset based on the initial aiming point:

[0026]

[0027] Among them, Δx represents the x-axis trajectory offset, Δy represents the y-axis trajectory offset, and (x0, y0) represents the initial aiming point coordinates.

[0028] Furthermore, S5 is specifically:

[0029] Horizontal calibration: when Δx>0, adjust to the left by |Δx| / k MOA; when Δx<0, adjust to the right by |Δx|k MOA;

[0030] Vertical calibration: when Δy>0, adjust downward by |Δy| / k MOA; when Δy<0, adjust upward by |Δy| / k MOA;

[0031] Where k is the range correlation coefficient.

[0032] The present invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the above-mentioned gun calibration method based on automatic bullet hole detection.

[0033] The present invention also proposes an electronic device, including a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the above-mentioned gun calibration method based on automatic bullet hole detection.

[0034] The present invention also provides a computer program product, comprising a computer program / instruction, which implements the steps of the above-mentioned gun calibration method based on automatic bullet hole detection when executed by a processor.

[0035] The beneficial effects brought about by the technical solution provided by the present invention are:

[0036] This method utilizes an infrared thermal imager to capture real-time infrared thermal images of bullet holes, overcoming the environmental limitations of visible light and enabling all-weather bullet hole detection. Thermal signature enhancement enables the detection of small or obscured bullet holes. Infrared thermal imaging, combined with deep learning algorithms, rapidly locates the impact point and analyzes trajectory deviation. This method offers significant advantages over traditional manual calibration methods that rely on visual inspection, rangefinders, or paper targets: It operates stably in complex environments such as low light and smoke, accelerating detection speed and improving calibration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of a gun calibration method based on automatic bullet hole detection according to an embodiment of the present invention;

[0038] Figure 2 This is a structural diagram of a bullet hole detection model according to an embodiment of the present invention;

[0039] Figure 3 It is a block diagram of an electronic device in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0041] The flow chart of the gun calibration method based on automatic bullet hole detection according to the embodiment of the present invention is as follows: Figure 1 , specifically including the following steps:

[0042] S1. Use an infrared thermal imager to capture infrared images of bullet holes in real time, perform preprocessing, and construct a bullet hole detection dataset.

[0043] An infrared thermal imager was used to capture real-time infrared images, including those before and after the shooting, in different weather conditions, at different angles, at different distances, and in different scenes. In this embodiment of the present invention, a high-sensitivity, long-wave, uncooled infrared thermal imager (resolution ≥512×640, NETD ≤50mk, frame rate ≥30Hz) was used. During infrared image acquisition, 200 images were collected for each of the following weather conditions (sunny / overcast / rainy / snowy), time periods (morning / noon / evening), distances (25-500 meters), and angles (±30° deflection). To ensure sample diversity and avoid overfitting during training, the image samples must also include negative samples, such as bullet hole occlusion or blurred bullet hole in a complex background. To improve the robustness of the method, data enhancement was performed on the original images. Each image was randomly processed using a processing method, such as cropping, rotation, noise addition (e.g., Gaussian noise addition), or optical processing. The enhanced dataset totaled 2,400 images.

[0044] These infrared images are preprocessed, including: (1) Non-uniformity correction: The two-point correction method is used to correct the non-uniformity of infrared images. The two-point correction method is based on the assumption of linear response of the detector pixel. Only two reference temperature points are needed to calibrate the gain (slope) and offset (intercept) of each pixel. It has low computational complexity, strong real-time performance, and is suitable for dynamic scenes. (2) Image denoising: The non-local mean denoising (NLM) method is used. NLM uses redundant information in the image to remove noise by finding similar areas in the image and performing weighted averaging on these areas. The NLM algorithm considers the information of the entire image, so it can better preserve image details while denoising. (3) Contrast enhancement: The contrast-limited adaptive histogram equalization (CLAHE) method is used. The local adaptive enhancement of the CLAHE method preserves details and is suitable for complex lighting scenes. The flexible and adjustable parameters can adapt to diverse needs and can significantly improve the distinction between the target and the background of the infrared thermal image, while preserving the details of the high dynamic range (HDR) image and enhancing the visual effect.

[0045] The preprocessed infrared image dataset is divided into a training set and a validation set in a ratio of 8:2, with 1920 images in the training set and 480 images in the validation set. The training set is annotated using LabelImg.

[0046] S2. A bullet hole detection model is constructed based on the improved YOLOv5 network. A multi-scale feature fusion module (MSFF) is added to the YOLOv5 network, and the spatial pyramid pooling (SPP) in the YOLOv5 backbone network is replaced by atrous spatial pyramid pooling (ASPP).

[0047] The improved YOLOv5 includes a backbone network (Backbone) feature extraction part, a neck network (Neck) feature fusion part, a detection head (Head) part, and a multi-scale feature fusion module. Figure 2 . Backbone includes a convolution module (conv), a C3 module, and an ASPP module. The first convolution module of the backbone network performs initial feature extraction on the input image. The convolution module and the C3 module are stacked and used multiple times in the backbone network. The C3 module adopts the cross-stage local network (CSP) idea to divide the feature map into two parts, one part performs convolution operations, and the other part is directly connected to the output. It is composed of multiple small convolution layers and residual connections, and features are extracted by stacking these small modules. The input image first enters the convolution module to extract preliminary features and increase the number of channels. Then, the feature map passes through multiple conv+C3 modules in sequence. The embodiment of the present invention has 4 conv+C3 modules. The C3 module continuously extracts and fuses features, gradually deepening the feature expression capability of the network. Finally, the feature map enters the ASPP module, where it is further enriched with feature information through pooling operations at different scales, outputting a feature map containing multi-scale features. In this embodiment of the present invention, the second C3 module outputs shallow features P1, the third C3 module outputs mid-level features P2, and the ASPP module outputs deep features P3. The multi-scale feature fusion module fuses the shallow, mid-level, and deep features extracted by the backbone network into new deep features. Specifically, features P2 and P3 are adjusted to the same resolution as P1 through upsampling to obtain P4 and P5, and P1, P4, and P5 are fused to obtain fused features P6: P1, P4, and P5 are first concatenated along the channel dimension, and then a 1×1 convolution is used to adjust the number of channels to reduce the amount of computation to obtain P6. The multi-scale features extracted by the backbone network (shallow features P1, mid-level features P2, and deep features P6) are passed to the FPN of the neck network for subsequent processing. The subsequent processing process is the same as that of the traditional YOLOv5 network. The neck network adopts the Feature Pyramid Networks (FPN) and Path Aggregation Network (PAN) structures. FPN transfers semantic information from high dimension to low dimension, and PAN transfers semantic information again from low dimension to high dimension.

[0048] The dilated spatial pyramid pooling has five branches. The first branch includes a 1×1 standard convolution. The second to fourth branches all include 3×3 dilated convolutions, with the dilation rates d set to 6, 12, and 18, respectively. The fifth branch is expressed as:

[0049]

[0050] Among them, B5 represents the output of the fifth branch, F represents the input image of the void space pyramid pooling, GAP() represents the global average pooling operation, Conv 1×1 Represents a 1×1 standard convolution operation, Upsample represents upsampling, H and W represent the height and width of the image F respectively, and F ：,i,j represents the (i, j) pixel of image F;

[0051] The outputs of the five branches of the dilated spatial pyramid pooling are fused and expressed as:

[0052] F ASPP =Conv 1×1 (Concat(B1,B2,B3,B4,B5)),

[0053] Among them, F ASPP Represents the output of dilated spatial pyramid pooling, B1~B5 are the outputs of the five branches of dilated spatial pyramid pooling respectively, and Concat() represents the concatenation operation.

[0054] The output size of the dilated convolution is expressed as:

[0055]

[0056] Among them, O represents the size of the output feature map, I represents the size of the input feature map, p represents the padding size, k represents the equivalent convolution kernel size of the dilated convolution, d represents the dilation rate, s represents the step size, and when the step size s = 1 and the padding p = d, O = I.

[0057] Then, deconvolution is used to gradually upsample and restore the spatial resolution. The deconvolution operation is calculated as follows:

[0058]

[0059] Where s is the stride, p is the padding, k is the kernel size, W(i,j) is the learnable weight of pixel (i,j), O(x,y) represents the deconvolution output image with a resolution of (x,y), and I(x·s+ip,y·s+jp) represents the deconvolution input image with a resolution of (x·s+ip,y·s+jp).

[0060] S3. Input the bullet hole detection dataset into the model to obtain bullet hole location data. First, the model is trained using the training set, and the trained model is verified using the validation set. The trained model is then used to detect the infrared image to obtain bullet hole location data.

[0061] S4. Convert the bullet hole coordinates according to the bullet hole position data and calculate the bullet hole offset.

[0062] It can be expressed as:

[0063]

[0064] Among them, (X d ,Y d ) represents the physical coordinates of the actual target surface, (u d ,v d ) represents the pixel coordinates of the bullet hole in the position data of the bullet hole, (c x ,c y ) represents the image center coordinate, D represents the shooting distance, f x Indicates the focal length of the camera in the X-axis direction, f y Indicates the focal length of the camera in the Y-axis direction.

[0065] Calculate the trajectory offset based on the initial aiming point:

[0066]

[0067] Among them, Δx represents the x-axis trajectory offset, Δy represents the y-axis trajectory offset, and (x0, y0) represents the initial aiming point coordinates.

[0068] S5. Analyze and correct trajectory deviation based on the hole offset. When x-axis trajectory offset Δx > 0, the impact point is offset to the right; when x-axis trajectory offset Δx < 0, the impact point is offset to the left. When y-axis trajectory offset Δy > 0, the impact point is offset to the high side; when y-axis trajectory offset Δy < 0, the impact point is offset to the low side.

[0069] According to the above conclusions, trajectory deviation correction is performed: horizontal calibration: when Δx>0, adjust |Δx| / k MOA to the left, and when Δx<0, adjust |Δx| / k MOA to the right; vertical calibration: when Δy>0, adjust |Δy| / k MOA downward, and when Δy<0, adjust |Δy| / k MOA upward; where k is the range correlation coefficient (k=0.291@100m), and MOA (Minute of Angle) is the gun calibration unit.

[0070] The significant advantages of the present invention are:

[0071] 1. Break through environmental limitations and achieve reliable detection around the clock with infrared thermal imaging;

[0072] 2. Improved algorithm to improve positioning accuracy;

[0073] 3. Have intelligent adaptive capabilities;

[0074] 4. Shorten calibration cycle through efficient closed-loop control.

[0075] In an exemplary embodiment, a computer-readable storage medium is included. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the firearm calibration method based on automatic bullet hole detection is implemented.

[0076] See also Figure 3 In an exemplary embodiment, an electronic device is also included, including at least one processor, at least one memory, and at least one communication bus.

[0077] The memory stores a computer program including computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through a communication bus to execute the above-mentioned gun calibration method based on automatic bullet hole detection.

[0078] In an exemplary embodiment, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above-mentioned gun calibration method based on automatic bullet hole detection when executed by a processor.

[0079] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A gun calibration method based on automatic bullet hole detection, characterized in that: The following steps are involved: S1. Use an infrared thermal imager to capture infrared images of bullet holes in real time, perform preprocessing, and construct a bullet hole detection dataset. S2. Build a bullet hole detection model based on the improved YOLOv5 network. Replace the spatial pyramid pooling in the YOLOv5 backbone network with the dilated spatial pyramid pooling. Add a multi-scale feature fusion module to the YOLOv5 network. The multi-scale feature fusion module fuses the shallow features, middle features, and deep features extracted by the backbone network into new deep features. The shallow features, middle features, and new deep features are input into the neck network. S3, inputting the bullet hole detection data set into the model to obtain bullet hole location data; S4. Convert the bullet hole coordinates according to the bullet hole position data and calculate the bullet hole offset; S5. Analyze the trajectory deviation based on the bullet hole offset and correct the trajectory offset.

2. A gun calibration method based on automatic bullet hole detection according to claim 1, characterized in that: Preprocessing includes: Non-uniformity compensation, image denoising, and contrast enhancement.

3. The gun calibration method based on automatic bullet hole detection according to claim 1, characterized in that: The dilated spatial pyramid pooling has five branches. The first branch includes 1×1 standard convolution, the second to fourth branches include 3×3 dilated convolution, and the fifth branch is expressed as: Among them, B5 represents the output of the fifth branch, F represents the input image of the void space pyramid pooling, GAP( ) represents the global average pooling operation, Conv 1×1 Represents a 1×1 standard convolution operation, Upsample represents upsampling, H and W represent the height and width of the image F respectively, and F ：,i,j represents the (i, j) pixel of image F; The outputs of the five branches of the dilated spatial pyramid pooling are fused and expressed as: F ASPP =Conv 1×1 (Concat(B1,B2,B3,B4,B5)), Among them, F ASPP Represents the output of dilated spatial pyramid pooling, B1~B5 are the outputs of the five branches of dilated spatial pyramid pooling respectively, and Concat() represents the concatenation operation.

4. The gun calibration method based on automatic bullet hole detection according to claim 3, characterized in that: The output size of the dilated convolution is expressed as: Among them, O represents the size of the output feature map, I represents the size of the input feature map, p represents the padding size, k represents the equivalent convolution kernel size of the dilated convolution, d represents the dilation rate, and s represents the step size.

5. The gun calibration method based on automatic bullet hole detection according to claim 1, characterized in that: The multi-scale feature fusion module fuses the shallow features, mid-level features, and deep features extracted by the backbone network into new deep features. Specifically: The middle-layer features and deep-layer features are adjusted to the same resolution as the shallow-layer features through upsampling, and are concatenated with the shallow-layer features along the channel dimension. Then, the number of channels is adjusted using 1×1 convolution to obtain new deep-layer features.

6. The gun calibration method based on automatic bullet hole detection according to claim 1, characterized in that: S4 is specifically: The pixel coordinates of the bullet holes in the bullet hole position data are converted into the physical coordinates of the actual target surface, which can be expressed as follows: Among them, (X d ,Y d ) represents the physical coordinates of the actual target surface, (u d ,v d ) represents the pixel coordinates of the bullet hole in the position data of the bullet hole, (c x ,c y ) represents the image center coordinate, D represents the shooting distance, f x Indicates the focal length of the camera in the X-axis direction, f y Indicates the focal length of the camera in the Y-axis direction; Calculate the trajectory offset based on the initial aiming point: Among them, Δx represents the x-axis trajectory offset, Δy represents the y-axis trajectory offset, and (x0, y0) represents the initial aiming point coordinates.

7. The gun calibration method based on automatic bullet hole detection according to claim 1, characterized in that: S5 is specifically: Horizontal calibration: when Δx>0, adjust to the left by |Δx| / k MOA; when Δx<0, adjust to the right by |Δx| / k MOA; Vertical calibration: when Δy>0, adjust downward by |Δy| / k MOA; when Δy<0, adjust upward by |Δy| / k MOA; Where k is the range correlation coefficient.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images