Edge end bullet hole detection method based on deep learning

Abstract

The invention provides an edge end bullet hole detection method based on deep learning. The method comprises the following steps: S1, obtaining an original image; s2, preprocessing the original image; s3, inputting the preprocessed original image into a deep neural network for bullet hole detection; s4, processing prediction information of bullet hole detection; s5, outputting a detection result; according to the edge end bullet hole detection method based on deep learning, end-to-end design is performed according to bullet hole detection requirements, and the method mainly comprises the following steps: recalculating and adjusting the size of a detected anchor according to the size of a bullet hole; adjusting the network input size for a small target; an attention mechanism is added, and bullet hole detection precision is improved; a backbone network is optimized, and reasoning time consumption is reduced; a detection head is modified to pay attention to small target detection; the model is quantized after being trained, the reasoning speed is increased, and the detection time consumption of the algorithm is greatly reduced while the high detection precision is kept.

Description

Technical Field

[0001] This invention belongs to the field of image processing and target tracking technology, specifically relating to a deep learning-based method for detecting bullet holes at the edge. Background Technology

[0002] Bullet hole detection is the process of detecting the point of impact after live-fire shooting, and reporting the target by determining the point of impact.

[0003] Existing bullet hole detection algorithms mainly include deep learning-based methods and traditional algorithms. Traditional computer vision methods detect bullet holes by utilizing features such as color, shape, and texture of images. They are suitable for situations where bullet hole features are obvious and the image background is simple. However, they are sensitive to factors such as lighting changes and image noise, which may lead to false positives or false negatives. Deep learning-based bullet hole detection methods involve labeling and training the collected data, using neural networks for feature extraction to determine the size and location of the bullet holes. This detection method generally has low accuracy and is time-consuming, failing to meet the needs of rapid edge detection. Summary of the Invention

[0004] To address the issues of low accuracy, long processing time, and inability to meet the needs of rapid edge detection in existing deep learning-based bullet hole recognition methods.

[0005] The purpose of this invention is to provide a deep learning-based method for detecting bullet holes at the edge of a bullet hole, comprising the following steps: S1. Obtain the original image; S2. Preprocess the original image; S3. Input the preprocessed original image into a deep neural network for bullet hole detection; S4. Process the predicted information for bullet hole detection; S5. Output the detection results.

[0006] Furthermore, S2, preprocessing the original image, specifically includes the following steps: S21. The original image is converted to RGB using a transformation method. S22. Perform scaling and filling operations on the converted image; S23. Normalize the scaled and filled image by dividing each pixel of the image by 255.

[0007] Furthermore, step S22, scaling and filling the converted image, specifically includes the following steps: S221. Calculate the scaling ratio: First, calculate the aspect ratio of the target size to the original image size, that is, the ratio of the target width to the original image width and the ratio of the target height to the original image height. Select the smaller ratio as the scaling ratio. S222. After scaling the image according to the scaling ratio, if the target size is inconsistent with the scaled image size, then fill the scaled image to ensure that the scaled image size is consistent with the target size. S223. Output a scaled image that matches the target size.

[0008] Furthermore, the specific process of inputting the preprocessed original image into the deep neural network for bullet hole detection in step S3 is as follows: S31. Input the scaled image obtained in step S2 into the depth convolution-based feature extraction backbone network for feature extraction; S32. Input the features extracted in step S31 into a feature aggregation network based on the fusion of high-level semantics and location information to perform multi-scale feature fusion; S33. Input the features fused in step S31 into the multi-scale head detection head to classify and regress the target. The prediction information includes the coordinates of the upper left corner of the prediction box, the width and height of the prediction box, the confidence score, and the category information.

[0009] Furthermore, the specific process of processing the predicted information for bullet hole detection in step S4 is as follows: S41. Decode the coordinate information of the prediction box, as shown below: in, It is the sigmoid function. , , , It is the offset predicted by the network. , These are the coordinates of the anchor frame's center point. , These are the width and height of the anchor frame; S42. Based on the set confidence threshold, filter the predicted boxes and retain those with a confidence level higher than the confidence threshold. S43. Perform non-maximum suppression on the filtered prediction boxes and delete prediction boxes with large overlap.

[0010] Furthermore, the specific process of performing non-maximum suppression on the filtered prediction boxes and deleting prediction boxes with large overlaps in step S43 is as follows: S431. Sort the predicted boxes by confidence level; S432. Add the prediction box with the highest confidence level to the final output list and remove it from the prediction box list; S433. Calculate the IoU (intersection-over-union) between the prediction box with the highest confidence and other prediction boxes, which is the intersection of the two bounding boxes divided by their union. S434. Delete prediction boxes with IoU greater than the threshold; S435. Repeat the above process until the prediction box list is empty.

[0011] Furthermore, the deep neural network includes a backbone network, a neck network, and a head detection head; The backbone network includes a first CBS module, a second CBS module, a first C3K2-DFF-1 module, a third CBS module, a second C3K2-DFF-1 module, a fourth CBS module, a third C3K2-DFF-1 module, a fifth CBS module, a fourth C3K2-DFF-1 module, an SPPF module, and an FFA module connected in sequence. The second C3K2-DFF-1 module outputs the first backbone, the third C3K2-DFF-1 module outputs the second backbone, and the FA module outputs the third backbone. The neck network includes a first Upsample module, a first Concat module, a fifth C3K2-DFF-1 module, a second Upsample module, a second Concat module, a sixth C3K2-DFF-1 module, a sixth CBS module, a third Concat module, a seventh C3K2-DFF-1 module, a seventh CBS module, a fourth Concat module, and an eighth C3K2-DFF-1 module connected in sequence; the fifth C3K2-DFF-1 module is connected to the third Concat module; the first backbone output is connected to the first Upsample module, and the first backbone output is also connected to the fourth Concat module; the sixth C3K2-DFF-1 module is the first neck output, the seventh C3K2-DFF-1 module is the second neck output, and the eighth C3K2-DFF-1 module is the third neck output. The head detection head includes a Head-1 module, a Head-2 module, a Head-3 module, and a fifth Concat module. The Head-1 module is connected to the fifth Concat module, the Head-2 module is connected to the fifth Concat module, and the Head-3 module is connected to the fifth Concat module. The fifth Concat module has a head output terminal. The first neck output terminal is connected to the Head-1 module, the second neck output terminal is connected to the Head-2 module, and the third neck output terminal is connected to the Head-1 module.

[0012] This invention provides a deep learning-based edge bullet hole detection method. Addressing the needs of bullet hole detection, it employs an end-to-end design, primarily involving: recalculating and adjusting the detection anchor size based on bullet hole dimensions; adjusting the network input size for small targets; adding an attention mechanism to improve bullet hole detection accuracy; optimizing the backbone network to reduce inference time; modifying the detection head to focus on small target detection; and quantizing the model after training to improve inference speed. These measures significantly reduce the algorithm's detection time while maintaining high detection accuracy.

[0013] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a deep learning-based edge bullet hole detection method.

[0015] Figure 2 This is a flowchart illustrating a deep learning-based edge bullet hole detection method.

[0016] Figure 3 This is a schematic diagram of the bullet hole detection results using a deep learning-based edge bullet hole detection method. Detailed Implementation

[0017] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the specific implementation methods, structural features and effects of the present invention are described in detail below with reference to the accompanying drawings and embodiments.

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "aligned", "overlapping", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0020] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature; in the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0021] Example 1

[0022] To address the issues of low accuracy, long processing time, and inability to meet the needs of rapid edge detection in existing deep learning-based bullet hole recognition methods.

[0023] This embodiment provides, for example Figure 1 The method for detecting bullet holes at the edge of a device based on deep learning, as shown, includes the following steps: S1. Obtain the original image; S2. Preprocess the original image; S3. Input the preprocessed original image into a deep neural network for bullet hole detection; S4. Process the predicted information for bullet hole detection; S5. Output the detection results.

[0024] Furthermore, S2, preprocessing the original image, specifically includes the following steps: S21. Use a transformation method to perform color conversion on the original image, converting the colors of the original image to RGB; for example, if the image has three channels (such as 0, 1, 2), change channels 0, 1, 2 to channels 2, 1, 0, and convert BGR to RGB. S22. Perform scaling and filling operations on the converted image; S23. Normalize the scaled and filled image by dividing each pixel of the image by 255.

[0025] Furthermore, step S22, scaling and filling the converted image, specifically includes the following steps: S221. Calculate the scaling ratio: First, calculate the aspect ratio of the target size to the original image size, that is, the ratio of the target width to the original image width, and the ratio of the target height to the original image height. Select the smaller ratio as the scaling ratio; for example, the aspect ratio of the target size: 640x640 to the original image size: 1920x1080 is 640 / 1920 for width and 640 / 1080 for height. S222. After scaling the image according to the scaling ratio, if the target size and the scaled image size are inconsistent, the scaled image is padded to ensure that the scaled image size is consistent with the target size. For example, if the target size is 640x640 and the scaled image size is not exactly the same, and the scaled image size is 640x360, then the shorter blank area of ​​the scaled image will be filled with color (usually black), forming a black border similar to that on a letterbox. For example, if the target size is 640x640 and the scaled image size is 640x360, then pixels need to be padded in the 360 ​​dimension to make the final image size 640x640. S223. Output a scaled image that matches the target size.

[0026] Furthermore, the specific process of inputting the preprocessed original image into the deep neural network for bullet hole detection in step S3 is as follows: S31. Input the scaled image obtained in step S2 into the depthwise convolution-based feature extraction backbone network for feature extraction, resulting in three feature maps, corresponding to... Figure 2 Feature map B1, feature map B2, and feature map B3; S32. Input the features extracted in step S31 into a feature aggregation network based on high-level semantic and location information fusion (neck network) for multi-scale feature fusion to obtain a three-way fused feature map. Figure 2 Feature maps N1, N2, and N3; S33. Input the features fused in step S31 into the multi-scale head detection head to classify and regress the target. The prediction information includes the coordinates of the upper left corner of the prediction box, the width and height of the prediction box, the confidence score, and the category information. Figure 2 H1 in the middle.

[0027] Furthermore, such as Figure 2 As shown, the deep neural network includes a backbone network, a neck network, and a head detection head; The backbone network includes a first CBS module, a second CBS module, a first C3K2-DFF-1 module, a third CBS module, a second C3K2-DFF-1 module, a fourth CBS module, a third C3K2-DFF-1 module, a fifth CBS module, a fourth C3K2-DFF-1 module, an SPPF module, and an FFA module connected in sequence. The second C3K2-DFF-1 module outputs the first backbone, the third C3K2-DFF-1 module outputs the second backbone, and the FA module outputs the third backbone. The neck network includes a first Upsample module, a first Concat module, a fifth C3K2-DFF-1 module, a second Upsample module, a second Concat module, a sixth C3K2-DFF-1 module, a sixth CBS module, a third Concat module, a seventh C3K2-DFF-1 module, a seventh CBS module, a fourth Concat module, and an eighth C3K2-DFF-1 module connected in sequence; the fifth C3K2-DFF-1 module is connected to the third Concat module; the first backbone output is connected to the first Upsample module, and the first backbone output is also connected to the fourth Concat module; the sixth C3K2-DFF-1 module is the first neck output, the seventh C3K2-DFF-1 module is the second neck output, and the eighth C3K2-DFF-1 module is the third neck output. The head detection head includes a Head-1 module, a Head-2 module, a Head-3 module, and a fifth Concat module. The Head-1 module is connected to the fifth Concat module, the Head-2 module is connected to the fifth Concat module, and the Head-3 module is connected to the fifth Concat module. The fifth Concat module has a head output terminal. The first neck output terminal is connected to the Head-1 module, the second neck output terminal is connected to the Head-2 module, and the third neck output terminal is connected to the Head-1 module.

[0028] The backbone network uses a CSP (Concurrent Spinning Process) structure to divide the network into blocks, reducing the number of parameters and enhancing expressive power. First, the input feature map is divided into two parts, as shown in the figure, into half the number of channels. Two 1x1 convolutional layers are used for this division. The feature map in part 1 is left unprocessed. The feature map in part 2 undergoes a series of block processing steps before passing through a 1x1 convolutional layer: the Transition block. Finally, the feature maps from the two branches are concatenated along the channel direction and fused through the final 1x1 convolutional layer: the Transition block. SPPF (Spatial Pyramid Pooling - Fast) is an improved spatial pyramid pooling module for object detection models, designed to efficiently fuse multi-scale feature information, improving detection accuracy and speed. As a fast alternative to the traditional SPP module, it significantly reduces computational overhead while maintaining high accuracy. SPPF captures contextual information from different receptive fields by using multiple max-pooling layers in parallel, but its design is more efficient. Unlike traditional SPP, which requires multiple pooling operations, SPPF uses a single pooling operation followed by a cascading approach to fuse multi-scale features, reducing redundant computation. PPA structure: The main advantage of PPA lies in its multi-branch feature extraction strategy. PPA employs a parallel multi-branch method, with each branch responsible for extracting features at different scales and levels. This multi-branch strategy helps capture the multi-scale features of targets, thereby improving the accuracy of small target detection. Specifically, this strategy involves three parallel branches: a local convolution branch, a global convolution branch, and a sequential convolution branch, used to efficiently extract features such as object edges, corners, and colors.

[0029] The neck section incorporates a PANet structure: Path Aggregation Network, a network structure used in object detection to enhance feature pyramids. It introduces bottom-up path enhancement on top of Feature Pyramid Network (FPN) to improve the performance of multi-scale object detection. Bottom-up path enhancement: This design adds a short bottom-up path to the original top-down path of FPN. This new path starts from the low-level high-resolution features and efficiently transfers precise localization information from low-level layers, such as edges and textures, to high-level features through iterative downsampling and feature fusion operations. This better integrates feature information at different scales, improving the detection performance for small and occluded targets.

[0030] The head section employs a multi-scale prediction head to improve the detection performance for targets at different scales. The head primarily consists of a classification head (outputting the probability that each candidate region belongs to a different category) and a regression head (predicting the target's location, i.e., the coordinates of the bounding box). The detection model extracts features, fuses features, and predicts outputs from the normalized image. The backbone network extracts features at different levels and scales from the input image, which are then passed to subsequent network parts for further processing. The neck section is responsible for further processing based on the basic features extracted by the backbone. Its main task is multi-scale feature fusion. Since objects in an image can vary greatly in size, target detection needs to handle both large and small objects simultaneously. The neck extracts features at multiple scales to ensure the model can handle targets of various sizes. The head is the final part of the network, responsible for classifying and regressing the target based on the feature map fused by the neck. The head has three output heads, responsible for predicting large, medium, and small targets respectively. The predicted bounding box information includes: the coordinates of the top-left corner, the width and height of the predicted bounding box, confidence score, and category information.

[0031] Furthermore, the specific process of processing the predicted information for bullet hole detection in step S4 is as follows: S41. Decode the coordinate information of the prediction box, as shown below: in, It is the sigmoid function. , , , It is the offset predicted by the network. , These are the coordinates of the anchor frame's center point. , These are the width and height of the anchor frame; S42. Based on the set confidence threshold (the confidence threshold is set to 0.5), the predicted boxes are filtered, and predicted boxes with a confidence level higher than the confidence threshold are retained. Confidence level is the degree of confidence that the predicted box in the detection model's inference contains the corresponding target. When the confidence level of the model prediction is greater than or equal to the set confidence threshold, the information of the predicted box is retained; when the confidence level of the model prediction is less than the set confidence threshold, the information of the predicted box is filtered out. S43. Perform non-maximum suppression on the filtered prediction boxes and delete prediction boxes with large overlap.

[0032] Furthermore, the specific process of performing non-maximum suppression on the filtered prediction boxes and deleting prediction boxes with large overlaps in step S43 is as follows: S431. Sort the predicted boxes by confidence level. The specific process is as follows: recursively divide the sequence to be sorted into two smaller parts, sort these two parts separately, and finally merge the sorted subsequences. Specifically, it selects a pivot element to divide the sequence into left and right subsequences, where all elements in the left subsequence are smaller than the pivot element, and all elements in the right subsequence are larger than the pivot element. Then, the same process is recursively applied to the left and right subsequences until each subsequence contains only one element, thus achieving the final ordered state. S432. Add the prediction box with the highest confidence level to the final output list and remove it from the prediction box list; S433. Calculate the IoU (intersection-over-union) between the prediction box with the highest confidence and other prediction boxes, which is the intersection of the two bounding boxes divided by their union. S434. Delete prediction boxes with IoU greater than the threshold; S435. Repeat the above process until the prediction box list is empty.

[0033] This invention provides a deep learning-based edge bullet hole detection method. Addressing the needs of bullet hole detection, it employs an end-to-end design, primarily involving: recalculating and adjusting the detection anchor size based on bullet hole dimensions; adjusting the network input size for small targets; adding an attention mechanism to improve bullet hole detection accuracy; optimizing the backbone network to reduce inference time; modifying the detection head to focus on small target detection; and quantizing the model after training to improve inference speed. This significantly reduces the algorithm's detection time while maintaining high detection accuracy. Figure 3 The diagram shown is a schematic of the bullet hole detection result of the edge bullet hole detection method based on deep learning shown in this embodiment.

[0034] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A deep learning-based method for detecting bullet holes at the edge of an object, characterized in that, Includes the following steps: S1. Obtain the original image; S2. Preprocess the original image; S3. Input the preprocessed original image into a deep neural network for bullet hole detection; S4. Process the predicted information for bullet hole detection; S5. Output the detection results.

2. The edge-end bullet hole detection method based on deep learning as described in claim 1, characterized in that: S2, the preprocessing of the original image, specifically includes the following steps: S21. The original image is converted to RGB using a transformation method. S22. Perform scaling and filling operations on the converted image; S23. Normalize the scaled and filled image by dividing each pixel of the image by 255.

3. The edge-end bullet hole detection method based on deep learning as described in claim 2, characterized in that: S22, the scaling and filling operation on the converted image, specifically includes the following steps: S221. Calculate the scaling ratio: First, calculate the aspect ratio of the target size to the original image size, that is, the ratio of the target width to the original image width and the ratio of the target height to the original image height. Select the smaller ratio as the scaling ratio. S222. After scaling the image according to the scaling ratio, if the target size is inconsistent with the scaled image size, then fill the scaled image to ensure that the scaled image size is consistent with the target size. S223. Output a scaled image that matches the target size.

4. The edge-end bullet hole detection method based on deep learning as described in claim 1, characterized in that: The specific process of inputting the preprocessed original image into a deep neural network for bullet hole detection in step S3 is as follows: S31. Input the scaled image obtained in step S2 into the backbone network for feature extraction based on depth convolution to perform feature extraction. S32. Input the features extracted in step S31 into the Neck network, which is a feature aggregation network based on the fusion of high-level semantics and location information, to perform multi-scale feature fusion. S33. Input the features fused in step S31 into the multi-scale head detection head to classify and regress the target. The prediction information includes the coordinates of the upper left corner of the prediction box, the width and height of the prediction box, the confidence score, and the category information.

5. The edge-end bullet hole detection method based on deep learning as described in claim 1, characterized in that: The specific process of processing the predicted information for bullet hole detection in S4 is as follows: S41. Decode the coordinate information of the prediction box, as shown below: in, It is the sigmoid function. , , , It is the offset predicted by the network. , These are the coordinates of the anchor frame's center point. , These are the width and height of the anchor frame; S42. Based on the set confidence threshold, filter the predicted boxes and retain those with a confidence level higher than the confidence threshold. S43. Perform non-maximum suppression on the filtered prediction boxes and delete prediction boxes with large overlap.

6. The edge-end bullet hole detection method based on deep learning as described in claim 5, characterized in that: The specific process of S43, which involves performing non-maximum suppression on the filtered prediction boxes and deleting prediction boxes with large overlap, is as follows: S431. Sort the predicted boxes by confidence level; S432. Add the prediction box with the highest confidence level to the final output list and remove it from the prediction box list; S433. Calculate the intersection-over-union (IoU) between the prediction box with the highest confidence and other prediction boxes, which is the intersection of two bounding boxes divided by their union. S434. Delete prediction boxes with IoU greater than the threshold; S435. Repeat the above process until the prediction box list is empty.

7. The edge-end bullet hole detection method based on deep learning as described in claim 1, characterized in that: The deep neural network includes a backbone network, a neck network, and a head detection head. The backbone network includes a first CBS module, a second CBS module, a first C3K2-DFF-1 module, a third CBS module, a second C3K2-DFF-1 module, a fourth CBS module, a third C3K2-DFF-1 module, a fifth CBS module, a fourth C3K2-DFF-1 module, an SPPF module, and an FFA module connected in sequence. The second C3K2-DFF-1 module leads to the first backbone output terminal, the third C3K2-DFF-1 module leads to the second backbone output terminal, and the FA module leads to the third backbone output terminal; The neck network includes a first Upsample module, a first Concat module, a fifth C3K2-DFF-1 module, a second Upsample module, a second Concat module, a sixth C3K2-DFF-1 module, a sixth CBS module, a third Concat module, a seventh C3K2-DFF-1 module, a seventh CBS module, a fourth Concat module, and an eighth C3K2-DFF-1 module connected in sequence. The fifth C3K2-DFF-1 module is connected to the third Concat module; the first backbone output is connected to the first Upsample module, and the first backbone output is also connected to the fourth Concat module; the sixth C3K2-DFF-1 module is the first neck output, the seventh C3K2-DFF-1 module is the second neck output, and the eighth C3K2-DFF-1 module is the third neck output; The head detection head includes a Head-1 module, a Head-2 module, a Head-3 module, and a fifth Concat module. The Head-1 module is connected to the fifth Concat module, the Head-2 module is connected to the fifth Concat module, and the Head-3 module is connected to the fifth Concat module. The fifth Concat module has a head output terminal. The first neck output terminal is connected to the Head-1 module, the second neck output terminal is connected to the Head-2 module, and the third neck output terminal is connected to the Head-1 module.

Images