Bullet hole detection method and device based on computer vision

Abstract

This specification provides a bullet hole detection method and apparatus based on computer vision, comprising: after the start of target shooting, acquiring a target image input from a front end; identifying and segmenting the effective area of ​​the target image through target surface positioning; removing useless background parts from the target image input from the front end; and outputting a processed target image; detecting changes in the target image acquired before and after shooting at the same target position using a ROI detection algorithm based on the processed target image, eliminating old bullet holes and identifying new bullet holes; and performing bullet hole detection by combining image processing technology with deep learning technology based on the identified new bullet holes.

Description

Technical Field

[0001] This document relates to the field of bullet hole detection technology, and in particular to a bullet hole detection method and device based on computer vision. Background Technology

[0002] Firing training for armored equipment is a crucial component of training for tanks and armored infantry units. From basic to advanced training, firing training is divided into multiple levels, training gunners to gradually master firing maneuvers, techniques, and skills to improve their shooting proficiency. One aspect of this training is "gun-in-cannon" coaching firing, which is based on live-fire exercises with coaxial machine guns and anti-aircraft machine guns, but with firing maneuvers and techniques aligned with the artillery firing requirements of combat firing training. Because gun ammunition is relatively cheaper than artillery shells, firing training is easier to organize and implement, and is well-suited to the repeated training needs of new gunners. Therefore, it holds a special and important position in armored equipment firing training.

[0003] During coaching shooting training, shooters need to search for, aim at, and fire at different types of targets, using near-real combat scenarios and training methods to improve their combat shooting skills and ultimately enhance combat effectiveness. Coaching shooting differs from other shooting training in two main ways: first, it requires setting up multiple different types of targets simultaneously; second, the score is determined by the number of hits within a specified time, eliminating the issue of scoring by ring numbers. Therefore, the final score requires meticulously checking the number of hits on multiple targets. Currently, in grassroots units and even military academies, coaching shooting training mostly uses the traditional manual method of target reporting. Target checking personnel are pre-positioned near the target area as needed. After a shooter completes shooting, they mark the target or use binoculars or other equipment to observe the hits as a basis for performance evaluation. This traditional method has several shortcomings: first, it is inefficient. Manual target checking requires pausing shooting training to ensure the safety of personnel involved. Since each shooter completes shooting training, target checking takes up a significant portion of the training time, severely impacting overall training efficiency. Second, the accuracy rate is relatively low. Because the targets in the training shooting range are far apart, it's impossible to mark or change the target paper as frequently as in ordinary individual combat shooting training. Therefore, when reporting targets, the target inspector needs to remember the location of old bullet holes to compare and find new ones. When there are many bullet impact points on the target surface, errors are easily made, leading to inaccurate reporting results. To avoid this problem, some units choose not to check the bullet impact points on the target surface during training, but instead have the target inspector judge whether a hit has occurred based on the trajectory of the tracer rounds and the smoke on the target. In this way, although it no longer relies on memory, this subjective judgment is highly dependent on personal experience, making it difficult to guarantee the accuracy and objectivity of the reporting results. Another solution is to assign multiple people to observe each target simultaneously. While this improves accuracy to some extent, it consumes a lot of manpower, affects training enthusiasm, and is often counterproductive; thirdly, the training effect is poor. During breaks in shooting practice, trainees only know how many shots hit the target, but they cannot know the exact location of the impact points. This prevents them from reviewing, experiencing, and understanding the shooting techniques and key points, identifying shortcomings, and finding ways to improve, in order to verify these findings in the next round of shooting. Therefore, it is necessary to improve the target detection and reporting method to replace manual reporting, in order to meet the needs of actual shooting training in the military and solve problems such as low efficiency and poor training effectiveness.

[0004] In existing technologies, different types of automatic target reporting systems often fail to simultaneously meet the needs of grassroots military shooting training in terms of cost, operating costs, and environmental adaptability. Among them, vision-based automatic target reporting systems are the most suitable for promotion in the military and to replace manual target reporting due to their low cost, ease of maintenance, and portability. Their performance mainly depends on the algorithm; however, because the accuracy and reliability of existing algorithms still need improvement, they have not yet been widely adopted in practical applications. Therefore, it is necessary to improve and optimize their algorithms to enhance the performance of vision-based automatic target reporting systems, enabling them to effectively perform target reporting tasks in training shooting.

[0005] Existing target localization algorithms have varying degrees of shortcomings and are primarily designed for chest-ring targets. This paper focuses on armored vehicle firing training, which requires the identification and localization of various target types, making the situation more complex and challenging. Theoretically, using deep learning-based algorithms is more feasible and effective. To address the problems of current algorithms, this paper primarily employs deep learning methods to build a semantic segmentation model and improves several aspects, including the network architecture and loss function, to complete the semantic segmentation of the effective regions of various target types, thus achieving target localization.

[0006] Existing bullet hole detection methods almost universally have their own limitations. Most current technologies tend to rely on traditional methods to address these issues, making it difficult to overcome the bottlenecks of traditional image processing techniques. Consequently, these algorithms often struggle to adapt to the training environment of coach shooting, characterized by varying lighting conditions, target jitter, and repeated use by multiple users. To address this, this paper takes into full account the practical needs of coach shooting training and proposes a novel approach: combining traditional image processing techniques with deep learning. This results in a bullet hole detection algorithm specifically designed for newly added bullet holes. Furthermore, improvements are made to address the limitations of existing algorithms, solving the problems of identifying adhered bullet holes and detecting incremental bullet holes. Summary of the Invention

[0007] The purpose of this invention is to provide a bullet hole detection method and apparatus based on computer vision, which aims to solve the above-mentioned problems in the prior art.

[0008] This invention provides a bullet hole detection method based on computer vision, comprising:

[0009] After the shooting begins, the target image input from the front end is acquired. The effective area of ​​the target image is identified and segmented by target surface positioning. The useless background part in the target image input from the front end is removed, and the processed target image is output.

[0010] Based on the processed target image, the ROI detection algorithm is used to detect the changes in the target image before and after shooting at the same target position, eliminating old bullet holes and identifying new bullet holes.

[0011] Based on the identified new bullet holes, image processing technology and deep learning technology are combined to perform bullet hole detection.

[0012] This invention provides a bullet hole detection device based on computer vision, comprising:

[0013] The elimination module is used to acquire the target image input from the front end after the shooting starts, identify and segment the effective area of ​​the target image through target surface positioning, remove the useless background part in the target image input from the front end, and output the processed target image.

[0014] The ROI detection algorithm module is used to detect changes in the target images acquired before and after shooting at the same target position based on the processed target image using the ROI detection algorithm, thereby eliminating old bullet holes and identifying new bullet holes.

[0015] The bullet hole detection module is used to perform bullet hole detection by combining image processing technology with deep learning technology based on the determined newly added bullet holes.

[0016] This invention also provides a bullet hole detection device based on computer vision, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the above-described bullet hole detection method based on computer vision.

[0017] This invention also provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, implements the steps of the above-described computer vision-based bullet hole detection method.

[0018] This invention employs a target surface localization and target type recognition algorithm based on semantic segmentation, enabling it to adapt to high-resolution input. The target surface localization algorithm achieves pixel-level high-precision semantic segmentation of the effective areas of various target types, effectively solving the problem of identifying multiple target types in training shooting and the interference of complex outdoor shooting range environments on target surface localization, while also laying the foundation for subsequent processing. A registration-difference-based ROI detection algorithm is used to detect changing regions in continuous target surface images, extracting all newly added bullet hole regions in the image. The bullet hole detection algorithm of this invention achieves high-precision detection of bullet holes in images under complex scenarios, and can identify isolated bullet holes and adhered bullet holes. Combining the target surface localization algorithm and the ROI detection algorithm, incremental detection of bullet holes in continuous images is finally achieved. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a computer vision-based bullet hole detection method according to an embodiment of the present invention;

[0021] Figure 2 This is a simplified flowchart of the target reporting algorithm according to an embodiment of the present invention;

[0022] Figure 3 This is a flowchart illustrating the target localization algorithm according to an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram illustrating the processing results of the target image processed using Gaussian filtering, median filtering, and bilateral filtering according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of sample images collected according to an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of some samples before and after annotation in an embodiment of the present invention;

[0026] Figure 7 This is a schematic diagram of the RefineNet network structure according to an embodiment of the present invention;

[0027] Figure 8 This is a schematic diagram of the RefineNet module structure according to an embodiment of the present invention;

[0028] Figure 9 These are schematic diagrams of the network structures of three schemes according to embodiments of the present invention;

[0029] Figure 10 This is a schematic diagram of a partial segmentation result of Scheme B in an embodiment of the present invention;

[0030] Figure 11 This is a detailed flowchart of ROI detection according to an embodiment of the present invention;

[0031] Figure 12 This is a schematic diagram of the final registration result of an embodiment of the present invention;

[0032] Figure 13 This is the final three-channel histogram of this embodiment of the invention;

[0033] Figure 14 This is a schematic diagram of the improved feature matching results according to an embodiment of the present invention;

[0034] Figure 15 This is a schematic diagram of the final registration result of the improved feature matching according to an embodiment of the present invention;

[0035] Figure 16 This is a schematic diagram of the ROI detection results according to an embodiment of the present invention;

[0036] Figure 17 This is a detailed flowchart of the final bullet hole detection algorithm according to an embodiment of the present invention;

[0037] Figure 18 This is a path map for contour extraction from a binary image according to an embodiment of the present invention;

[0038] Figure 19 This is a schematic diagram of the coarse extraction results according to an embodiment of the present invention;

[0039] Figure 20 This is a schematic diagram of sample data for a deep neural network model according to an embodiment of the present invention;

[0040] Figure 21 This is a schematic diagram of the network structure of ResNet50 according to an embodiment of the present invention;

[0041] Figure 22 This is a schematic diagram of the structure of the ECA Block according to an embodiment of the present invention;

[0042] Figure 23 This is a schematic diagram of the improved ECA model structure according to an embodiment of the present invention;

[0043] Figure 24 This is a schematic diagram of the bullet hole detection results according to an embodiment of the present invention;

[0044] Figure 25 This is a schematic diagram of a bullet hole detection device based on computer vision according to Embodiment 1 of the present invention;

[0045] Figure 26 This is a schematic diagram of a bullet hole detection device based on computer vision, which is a second embodiment of the present invention. Detailed Implementation

[0046] In armored vehicle firing training, "gun-in-the-might" training holds a special and important position. Compared to ordinary firing training, training requires setting up multiple different types of targets simultaneously, and the score is determined by the number of hits within a specified time, eliminating the issue of scoring rings. Currently, most grassroots units and even military academies use traditional manual target reporting methods in training, which suffers from low training efficiency and accuracy. With the continuous advancement of military modernization, using automatic target reporting systems to replace outdated manual methods has become a development trend. Visual imaging-based automatic target reporting systems, with their low cost and ease of maintenance, have become the preferred choice for training.

[0047] This invention addresses the problem of automatic target reporting in armored equipment shooting training. Based on a review and analysis of the current status of target detection and reporting, a target reporting scheme based on visual imaging was chosen as the research direction. The paper comprehensively considers the impact of various factors such as illumination changes, target surface jitter, and imaging changes on imaging and detection. It combines traditional image processing techniques with deep learning techniques to design an automatic target reporting algorithm. The paper discusses and implements three main problems in "gun-in-cannon" training: target detection and recognition, identification of adhering bullet holes, and incremental bullet hole detection. Specifically: a target surface segmentation and type recognition algorithm improves the RefineNet model, solving the target surface localization problem in complex outdoor shooting range environments based on semantic segmentation; a new bullet hole region detection algorithm achieves ROI detection based on registration difference through feature matching of images before and after shooting; and a new bullet hole detection algorithm introduces an attention mechanism and modified activation function into the ResNet model, achieving the detection of small-sized isolated bullet holes and adhering bullet holes, improving detection accuracy and success rate. The algorithm designed in this paper was tested using a large number of target surface images collected from various complex scenarios. The experimental results show that the algorithm has high accuracy and anti-interference ability, which can meet the needs of automatic target reporting in coach shooting training and effectively promote the application process of automatic target reporting system.

[0048] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0049] Method Implementation Examples

[0050] According to embodiments of the present invention, a bullet hole detection method based on computer vision is provided. Figure 1 This is a flowchart of a computer vision-based bullet hole detection method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the bullet hole detection method based on computer vision according to an embodiment of the present invention specifically includes:

[0051] Step 101: After the shooting starts, the target image input from the front end is acquired. The effective area of ​​the target image is identified and segmented by target surface positioning. The useless background part in the target image input from the front end is removed, and the processed target image is output.

[0052] Step 102: Based on the processed target image, the ROI detection algorithm is used to detect the changes in the target image collected before and after shooting at the same target position, and old bullet holes are eliminated to determine new bullet holes.

[0053] Step 103: Based on the determined newly added bullet holes, combine image processing technology with deep learning technology to perform bullet hole detection.

[0054] Step 101 specifically includes:

[0055] Step 1011: The target image input from the front end is smoothed using a bilateral filter through a smoothing filter.

[0056] Step 1012: Using deep learning methods, a semantic segmentation model is built and trained to perform target segmentation; specifically, the following processing is included: establishing a dataset for the semantic segmentation model and performing pixel-level annotation on the data in the dataset, wherein the dataset specifically includes: real target images captured by a camera at different times and in different environments, and scaled-down self-made target images captured by a camera, and the annotation types include: rocket launcher, machine gun, frontal infantry fighting vehicle, frontal tank, tank firing point, markers and background;

[0057] Based on the RefineNet model, the network depth and width are increased to build a semantic segmentation model;

[0058] The labeled dataset is divided into training, validation and test sets. Target images are acquired at a resolution of 3120×3120 and enhanced. A weighted cross-entropy loss function is introduced to weight the target images, and the Adam optimizer is used to train the semantic segmentation model.

[0059] Target surface segmentation is performed using a trained semantic segmentation model.

[0060] Step 1013: Remove useless background from the target image after target surface segmentation to achieve target surface localization and output the processed target image.

[0061] Step 102 specifically includes:

[0062] Step 1021: Read two frames of target images before and after firing, and perform histogram matching on the two frames of target images: calculate the mapping function based on the existing grayscale distribution and the desired target distribution, where p r (r) represents the existing distribution, denoted by p. z (z) represents the target distribution, which is the distribution p of the image to be processed. r (r) Transform according to Formula 1, and calculate the result s. k The value is rounded to obtain the discrete form of the histogram equalization:

[0063]

[0064] Where M×N is the size of the image, n j The grayscale value is r j The number of pixels, where L represents the number of gray levels in the image;

[0065] Step 1022, based on the target histogram distribution p r The value of (r) is calculated for q = 0, 1, 2, ..., L-1 as 2, and the result G is rounded to obtain the transformation function G(Z). q ):

[0066]

[0067] Step 1023, when k = 0, 1, 2, ..., L-1, for each s k All are matched according to the value of the transformation function G, and the corresponding z is matched accordingly. q Thus, G(Z) q ) and s k The value is closest to , and all mappings from s to z are recorded, for any s k If its corresponding z q If there is more than one value, then the smallest value is selected.

[0068] Step 1024: After performing histogram equalization on the image to be processed, based on the mapping obtained in the previous step, calculate the equalized pixel values ​​s in the image. k Mapped to the corresponding z q This yields the result image after histogram matching.

[0069] Step 1025: Determine the feature detector and obtain feature points and feature descriptors through the feature detector. Then, establish a feature matcher, use the target image after shooting as a template, and match the feature points between it and the target image before shooting. Finally, correct the previous target image based on the matching results to complete the registration of the registered image. In the feature matching process, the target image is first masked based on the segmentation results, retaining only the marker pixels, and then the SIFT+BF algorithm is used for feature matching.

[0070] Step 1026: Perform a difference operation on the registered target image, directly subtract the pixel values ​​of the two images to obtain the difference result, and perform thresholding on the difference result after direct difference. Based on the Otsu method, the threshold T is determined according to formula 4-9. According to the threshold T and formula 3, the gray value of each pixel in the image is directly set to 0 or 1, that is, black or white, so that it presents a black and white effect.

[0071]

[0072] Where D(x,y) is the thresholding result, f t (x,y) and f t+1 (x,y) represents the two images before and after;

[0073] Step 1027, let the proportion of foreground pixels be ω0, and its average grayscale value be μ0; let the proportion of background pixels be ω1, and its average grayscale value be μ1, then we have:

[0074] ω0+ω1=1 Formula 4;

[0075] Step 1028, the overall average grayscale value of the image is:

[0076] μ=ω0μ0+ω1μ1 Formula 5;

[0077] The between-class variance is:

[0078] g=ω0(μ0-μ) 2 +ω1(μ1-μ) 2 Formula 6;

[0079] Equation 6 can be simplified to:

[0080] g = ω0ω1(μ0-μ1) 2 Formula 7;

[0081] Let p i Let i be the probability of gray level i appearing, then we have:

[0082]

[0083]

[0084] Step 1029: Traverse L gray levels, and find the gray level k that maximizes g, which is the threshold T.

[0085] Step 10210: Search for connected components in the difference results and remove all connected components with fewer than 5 pixels to eliminate some point and strip noise.

[0086] Step 10211: Add a minimum bounding box based on the search results of the connected components, and then map the bounding box onto the original image after shooting. Directly perform masking on the parts outside the ROI in the image to determine the newly added bullet holes.

[0087] Step 103 specifically includes:

[0088] Step 1031: Coarsely extract bullet holes in the region of interest using image binarization and contour extraction;

[0089] Step 1032: Build and train a deep neural network classification model: Use the Labelme tool to label the bullet hole regions in the localized target image into two categories: isolated bullet holes and adhered bullet holes. Generate a ground truth map of the bullet hole regions based on the JSON file. Extract suspected bullet hole regions from the image through image binarization and contour extraction, extract the detection results, save them as sample image data, and locate their corresponding regions in the ground truth map. Then calculate the intersection-union ratio (IU) between the two. If the IU of the bullet hole ground truth map and the coarse extraction result exceeds 0.5, the extracted sample is classified into the corresponding bullet hole class; otherwise, it is classified into the non-bullet hole class. This establishes the dataset for the deep neural network classification model. Build a deep neural network classification model based on the ResNet classification model, incorporating an attention mechanism and the Mish activation function. Divide the labeled dataset into training, validation, and test sets. Perform data augmentation on the dataset, introduce a focus loss function to weight the dataset, and use the Adamax optimizer to train the deep neural network classification model. Use the trained deep neural network classification model to identify the coarse extraction results.

[0090] Step 1033: Judge the results of the coarse extraction after identification, remove non-bullet hole parts, identify isolated bullet holes and adhered bullet holes, and finally complete the detection of bullet holes.

[0091] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0092] This invention focuses on the algorithm research of an automatic target reporting system based on visual imaging, taking into account the actual situation of shooting training in grassroots troops. It mainly designs three algorithms: target surface localization, ROI detection, and bullet hole detection. A simplified flowchart of the target reporting algorithm is shown below. Figure 2As shown in the diagram, after a round of target shooting is completed, the target image collected by the front end is transmitted to the back end, where target surface positioning, ROI detection, and bullet hole detection are performed sequentially, thus completing the entire process of an automatic target reporting algorithm.

[0093] 1. Target Localization Algorithm. Target localization involves identifying and segmenting the effective area of ​​the target, removing useless background parts from the input image, and laying the groundwork for subsequent processing. This invention, based on the characteristic of training shooting where multiple different types of targets are used simultaneously, mainly proposes a target semantic segmentation method based on deep learning technology. It improves the classic semantic segmentation model to address specific problems encountered and conducts experiments and analysis.

[0094] 2. ROI Detection Algorithm. ROI detection involves identifying changes in images acquired before and after shooting at the same target position to eliminate the influence of old bullet holes during shooting training, allowing subsequent algorithms to focus on detecting newly added bullet holes. This invention proposes an ROI detection algorithm based on image registration difference, which has been improved to suit the actual situation of coach shooting training and has been experimentally verified and analyzed.

[0095] 3. Bullet Hole Detection Algorithm. After preprocessing, the next step is to detect bullet holes. This invention combines traditional image processing techniques with deep learning techniques to propose a bullet hole detection method. A novel approach is used to build the dataset, and existing classification models are improved to suit the proposed algorithm. Finally, the algorithm is tested overall, and the results are analyzed.

[0096] The three algorithms mentioned above will be explained in detail below.

[0097] Since the images input from the front end often contain a large number of useless background areas and noise, it is necessary to eliminate noise interference as much as possible and accurately identify and extract the effective target area in order to successfully perform bullet hole detection. This chapter first processes the target image using a smoothing filter, and then uses deep learning methods to build and train a semantic segmentation model to achieve target localization. The specific algorithm flow is as follows: Figure 3 As shown.

[0098] For automatic target reporting systems based on computer vision technology, the first step is to eliminate noise generated during the acquisition and reading of target images. This is typically achieved using smoothing filters to denoise the target image. It's important to note that denoising should preserve the original image features as much as possible. Incomplete noise removal can negatively impact subsequent steps, while excessive smoothing can lead to the loss of bullet hole features, increasing detection difficulty and having the opposite effect. Smoothing filters can be divided into linear and nonlinear filters in the spatial domain. Unlike linear filtering, nonlinear filtering is closer to a logical operation, including median filtering and bilateral filtering. Median filtering sorts the pixel values ​​within the kernel so that the center pixel takes the median value; it effectively suppresses nonlinear noise such as impulse noise and salt-and-pepper noise. Bilateral filtering's kernel function consists of a spatial domain kernel and a value domain kernel, allowing it to suppress noise while preserving image edge information. Its expression is shown below:

[0099]

[0100] w(i,j,k,l)=d(i,j,k,l)×r(i,j,k,l)

[0101]

[0102]

[0103] Here, kernel d(i,j,k,l) ​​represents the distance between the current point and the points in the neighborhood, while kernel r(i,j,k,l) ​​represents the distance between their gray values.

[0104] The target image was processed using Gaussian filtering, median filtering, and bilateral filtering respectively. The processing results are as follows: Figure 4 As shown. By Figure 4 As can be seen from (a)-(d), Gaussian filtering and bilateral filtering have similar overall denoising effects, but median filtering is not very effective at denoising Gaussian noise. Furthermore, for the algorithm in this embodiment of the invention, the more obvious the bullet hole features in the image, the higher the accuracy of subsequent bullet hole detection. Figure 4 As can be seen from (e)-(h), compared to Gaussian filtering, the bullet hole details are clearer after bilateral filtering and median filtering. Therefore, Gaussian filtering has certain limitations in protecting bullet hole features. In summary, bilateral filtering can eliminate the influence of noise as much as possible without destroying bullet hole features. Therefore, this embodiment of the invention uses bilateral filtering to process target images.

[0105] Target segmentation: After eliminating noise interference, target segmentation is necessary to remove useless background from the image. Traditional segmentation methods mainly focus on low-level image features, which has limitations and is not suitable for complex multi-target scenarios such as training shooting. Therefore, deep neural networks are introduced here, and a fully supervised learning approach is used to achieve semantic segmentation of the target.

[0106] Creating a dataset is the first and crucial step in conducting deep learning experiments; a high-quality dataset can even improve the model training effect. Since there are currently no open-source datasets related to target images, the data used in these embodiments of the invention were all collected independently and mainly consist of the following two parts:

[0107] (1) A total of 4,802 real target images were collected by the camera at different times and under different environments.

[0108] (2) 588 scaled-down images of the self-made target (including markers) captured by a camera.

[0109] like Figure 5 As shown, since the collected sample images are not of the same size, all the collected samples were cropped into squares to unify the aspect ratio for ease of subsequent use, and a dataset containing 5390 target images of various types was established. Each sample image contains one target and up to four markers.

[0110] Next, the samples in the dataset need to be labeled. Since shooting training instructors use various types of targets, the segmentation task in this embodiment of the invention belongs to a multi-class semantic segmentation problem. Here, the image labeling software Labelme is used to perform pixel-level labeling on several commonly used target images, labeling them into 7 categories: rocket launcher (target A), machine gun (target B), infantry fighting vehicle in front (target C), tank in front (target D), tank firing point (target E), marker, and background. Samples before and after labeling are shown below. Figure 6 As shown, where, Figure 6 (a)-(c) show the comparison of the collected real target images before and after annotation. Figure 6 (d) Comparison of self-made target images with markers before and after annotation.

[0111] Image segmentation refers to separating pixel regions belonging to different objects, such as separating pedestrians and vehicles from the background in a street scene. Semantic segmentation, on the other hand, builds upon ordinary segmentation by identifying the category of each region, that is, giving them their semantic meaning.

[0112] An improved RefineNet semantic segmentation model: Training deep convolutional neural network models typically involves numerous convolution and pooling operations, inevitably leading to significant compression of the initial image resolution and loss of some data. To address this issue, the RefineNet network architecture was proposed. Structurally similar to U-Net, both can be simply understood as consisting of upsampling and downsampling pathways; RefineNet can also be considered a variant of U-Net. This model is primarily used to solve pixel-level high-precision semantic segmentation problems, achieving excellent results in semantic segmentation tests on public datasets such as PASCAL VOC, NYUDv2, and Person-Parts. The RefineNet network structure is as follows: Figure 7 As shown.

[0113] Figure 7 The left-hand downward propagation path is based on the ResNet network, extracting semantic features from the image through downsampling and feeding the four sets of output features into the corresponding RefineNet module. The right-hand upward propagation path is based on a multi-path refinement structure, using the RefineNet module to fuse features from each level, thereby maximizing the use of available information during the downsampling process. The RefineNet module structure is as follows: Figure 8 As shown.

[0114] The RefineNet module consists of three modules: Residual Convolution Unit (RCU), Multi-resolution Fusion, and Chained Residual Pooling. The RCU module is a model extracted from ResNet; each input first passes through two RCU modules to fine-tune the weights of the pre-trained ResNet. The Multi-resolution Fusion module fuses multiple inputs onto a high-resolution feature map; the convolution process before upsampling is primarily to adjust the numerical scale of different features. The Chained Residual Pooling module is used to obtain contextual background information for larger image regions and optimize the resulting segmentation; its multiple pooling modules help capture relatively large image information.

[0115] In summary, RefineNet is an excellent model, with its main advantage over other models lying in its multi-scale feature learning capabilities. It's worth noting that while RefineNet achieves high scores on multiple public sets, its input sample resolution is not particularly high; for example, the images in the PASCAL VOC dataset are only 500×486 pixels. In the algorithm of this invention, target segmentation requires maintaining high-resolution input to prevent the loss of bullet hole information. The original RefineNet network structure may not be suitable for such large images, therefore, it is necessary to improve its structure to make it suitable for the application scenario presented in this paper. To address this issue, this invention proposes three effective solutions, whose network structures are as follows: Figure 9 As shown. To effectively expand the network structure, all three schemes use two different scales of input. Scheme A has a simpler structure, allowing each RefineNet module to receive features from four scales simultaneously, mainly expanding the network width. Scheme B cascades five RefineNet modules, focusing more on expanding the model depth. Scheme C can be seen as a compromise between the first two improvements. Although the three improvements differ greatly, their network structures still retain the core structure of multi-path optimization. At the same time, moderately increasing the network depth and width does not necessarily lead to serious overfitting or gradient vanishing / exploding degradation problems.

[0116] Model Training: The main parameters for training the semantic segmentation model are shown in Table 1.

[0117] Table 1 Main parameters for model training

[0118]

[0119] During training, the sample images in the dataset were input in batches. To prevent the model from learning incorrect data distribution features and to reduce generalization error, the order of all images was shuffled. The 5390 samples were then placed into the training set, validation set, and test set in a 7:1:2 ratio, as detailed in Table 2. To ensure objectivity, the three sets were independent of each other, with no overlap in their samples.

[0120] Table 2 Details of Dataset Partitioning

[0121]

[0122] Since the segmented images are also used for bullet hole detection, they should be kept at high resolution to reduce the loss of image information during compression. However, excessively large sample sizes can also lead to problems such as difficulty in labeling and slow model convergence. After weighing the options, this embodiment of the invention acquires target images at a resolution of 3120×3120 and inputs them into the model.

[0123] To improve the model's generalization ability, enabling it to adapt to various scenarios and preventing overfitting due to insufficient training data, data augmentation of the sample images should be performed before training. This involves introducing the `transforms` module from the `torchvision` toolset to perform rotation, horizontal flipping, and brightness and saturation adjustments on the original input image with a certain probability, thus achieving data augmentation. It is crucial to ensure that the corresponding mask image of the original image undergoes the same processing; otherwise, the dataset will be corrupted.

[0124] Batch size represents the number of samples selected in a single training iteration. While introducing it may reduce the overall training speed of the model, a suitable batch size value can make the gradient descent direction more accurate. Based on past experience and considering the effects of GPU performance, training set size, and network model structure, a batch size of 4 is set.

[0125] Epoch represents the number of training rounds the model undergoes. In each training round, the deep neural network learns all the sample images in the training set completely. Setting the Epoch value relies heavily on personal experience. After several rounds of pre-training testing, to ensure model convergence, it was ultimately decided to set the Epoch value to 100.

[0126] The learning rate is a crucial hyperparameter in deep learning experiments, determining the magnitude of model parameter updates during training. A larger learning rate leads to faster gradient descent but may cause oscillations around the minimum, preventing the model from converging effectively. Introducing a smaller learning rate inevitably results in slower convergence but is more likely to minimize the loss. This experiment balances training speed and accuracy. After multiple pre-training trials, an initial learning rate of 0.0025 was chosen, and a learning rate controller was introduced to exponentially decay with each training epoch.

[0127] Multi-class image segmentation problems often employ the categorical cross-entropy loss function as the loss function. The purpose of this experiment is to identify and extract the target surface region, laying the groundwork for the subsequent bullet hole detection process. From this perspective, to minimize the impact of semantic segmentation errors on bullet hole detection, it is crucial to avoid misclassifying bullet hole pixels on the target as useless background pixels; that is, tolerating false positives to a certain extent, while tolerating false negatives is very low. Therefore, a weighted cross-entropy loss function (WCE) is introduced to weight the positive samples (target pixels), and its expression is shown below:

[0128]

[0129] Among them, y i p represents the true category of the pixel. i ω represents the probability of predicting a sample as class i, and ω is the weight. WCE essentially uses ω to describe the importance of a sample in the loss function; setting ω > 1 can reduce false negatives.

[0130] The choice of optimizer has a significant impact on model training; even with the same model and dataset, using different optimizers can lead to different training results. The Adam algorithm used in this chapter is an optimization algorithm that adaptively adjusts the learning rate based on momentum and the gradient of the objective function. Its expression is shown below:

[0131] m t =β1m t-1 +(1-β1)g t

[0132]

[0133] It is not difficult to see that m t and v t Gradient g t The first and second moment estimates are both initialized to 0. In the initial stage of parameter iteration, they both exhibit a shift towards the initial values; therefore, bias corrections are applied.

[0134]

[0135]

[0136] Then update the parameters:

[0137]

[0138] Here, η is the global learning rate, initially set to β1 = 0.9 and β2 = 0.999, with a small value ε = 1 × 10⁻⁶ added to the denominator. -8 To prevent

[0139] To evaluate the semantic segmentation performance of the model, this embodiment of the invention analyzes and compares the experimental results from both subjective and objective perspectives. In the objective evaluation, two commonly used image segmentation evaluation metrics, Mean Pixel Accuracy (MPA) and Mean Intersection over Union (MIoU), are selected. The segmentation results of the trained model are compared with manually labeled results, as shown in the following expressions:

[0140]

[0141]

[0142] Average pixel precision refers to calculating the proportion of correctly classified pixels in each category of a segmentation task and averaging the ratios across all categories. Average intersection-union ratio (IUU) is calculated in image semantic segmentation tasks by summing the IUU values ​​for each category (the ratio of the intersection to the union of the predicted and actual values), taking the average, and when it reaches 1, it indicates that the intersection and union of the actual and predicted values ​​for each category are exactly the same. In the formula, p... ii p represents the number of pixels correctly classified into class i. ij This represents the number of pixels that should belong to class i but were incorrectly classified as class j. k represents the number of classes (excluding background) in the image segmentation task. In principle, k should be taken as 6 here. However, considering that the focus of the research is on the segmentation effect of each model on the target, introducing marker classes with very low segmentation difficulty when calculating the mean would cause significant interference to subsequent analysis. Therefore, in this embodiment of the invention, marker classes are removed when calculating MPA and MIoU, and k is taken as 5.

[0143] Of the three improved methods proposed in this embodiment of the invention, it is clear that scheme A performs worse than the original model and is a reverse optimization. Scheme B, on the other hand, achieves excellent results, even surpassing DeepLabv3+ numerically, and is the highest among the models. Scheme C shows only a very small improvement over the original RefineNet-res152, which is almost negligible. Therefore, for the application scenario presented in this paper, if too much emphasis is placed on expanding the width, the high-channel-count feature maps generated during training may be difficult for the model to accept. Directly increasing the depth of the RefineNet network is the most effective approach. Thus, this embodiment of the invention selects scheme B as the final segmentation model, and its partial segmentation results are as follows: Figure 10 As shown.

[0144] In summary, the target localization algorithm according to embodiments of the present invention first uses bilateral filtering to preprocess the image for denoising. Then, based on deep learning technology, a semantic segmentation model based on RefineNet is built and trained to complete the target localization. During this process, a dataset was established by collecting and labeling a large number of sample images. Commonly used deep learning semantic segmentation models were summarized and analyzed. Addressing the shortcomings of the original RefineNet model, its model structure was improved to adapt to high-resolution input. To address the problem of bullet hole information loss during segmentation, the loss function was improved to reduce false negatives. Finally, the algorithm was tested and analyzed. Test results show that the target localization algorithm achieves pixel-level high-precision semantic segmentation of the effective area of ​​various types of targets, effectively solving the problem of identifying multiple types of targets in training shooting and the interference of complex outdoor shooting range environments on target localization. It also lays the foundation for subsequent processing.

[0145] ROI Detection: After completing target localization and removing useless background, the next step is to focus on bullet hole detection. The importance of new bullet hole detection has already been explained earlier. To detect new bullet holes on the target, this chapter uses image registration to detect the changes in the target image before and after firing; this part is the region of interest (ROI). The specific algorithm flow is as follows... Figure 11 As shown.

[0146] Histogram matching can alter the grayscale distribution of an image to match it with a histogram of a specified image or function. Since shooting training is conducted outdoors, the images taken before and after shooting may exhibit significant differences in brightness due to cloud cover, changes in lighting, and other factors. Therefore, histogram matching is used here to adjust the grayscale distribution of the image, eliminating these differences and thus making the results of feature matching and difference operations more accurate.

[0147] The core of this method lies in calculating a mapping function based on the existing grayscale distribution and the desired target distribution, where p is used. r (r) represents the existing distribution, denoted by p. z If (z) represents the target distribution, then the matching steps are as follows:

[0148] (1) The distribution p of the image to be processed r (r) is transformed according to the following formula, and the calculated result s is... k The value is rounded to obtain the discrete form of the histogram equalization.

[0149]

[0150] Where M×N is the size of the image, n j The grayscale value is r j The number of pixels, where L represents the number of gray levels in the image.

[0151] (2) Based on the target histogram distribution p r The value of (r) is calculated for q = 0, 1, 2, ..., L-1 according to the following formula, and the result G is rounded to obtain the transformation function G(Z). q ).

[0152]

[0153] (3) When k = 0, 1, 2, ..., L-1, for each s k All are matched according to the value of the transformation function G, and the corresponding z is matched accordingly. q Thus, G(Z) q ) and s k The value is closest to , and all mappings from s to z are recorded. It is important to note that for any s... k If its corresponding z q If there is more than one value, then the smallest value is selected.

[0154] (4) After performing histogram equalization on the image to be processed, based on the mapping obtained in the previous step, calculate the equalized pixel values ​​s in the image. k Mapped to the corresponding z q This yields the result image after histogram matching.

[0155] In the algorithm of this embodiment, histogram matching is performed on the two input images in the RGB color space. The acquired second image is used as the target image, and the first image is used as the source image for matching. The only difference from the algorithm described above is that before matching, the image needs to be split into three images according to the RGB channels, the corresponding channels are registered separately, and then a single color image is synthesized. The final registration result is as follows. Figure 12 As shown, its corresponding histogram is as follows: Figure 13 As shown.

[0156] Image registration can be viewed as the process of matching images acquired under different conditions, including grayscale-based registration, feature-based registration, and transform-domain-based registration methods. After histogram matching, the registration difficulty of the two images before and after firing is greatly reduced. Here, we choose an image feature-based method for registration. Registration first requires determining a feature detector to obtain feature points and feature descriptors. Then, a feature matcher is built, using the post-firing image as a template to match feature points with the pre-firing image. Finally, the pre-firing image is corrected based on the matching results, thus completing the registration.

[0157] Features are unique parts of an image. To register two images, their features must first be extracted. Feature detection includes point detection, line detection, and edge detection. When applied to feature matching, point detection is often chosen. Many existing feature point detection algorithms exist, such as Harris, SIFT, SURF, FAST, BRIEF, and ORB. This invention focuses on SIFT, SURF, and ORB, three widely recognized high-performance algorithms.

[0158] In feature detection, to address the problem of visual invariance, the concept of scale-invariant features was proposed. This means that consistent feature points can be detected in images of an object acquired at various scales, and each detection result has its corresponding scale factor. Theoretically, the ratio of the scale factor of the same feature point in two images at different scales is equal to the ratio of the image scales. SIFT, or Scale-Invariant Feature Transform, is a well-known scale-invariant feature detection algorithm, commonly used for blob detection, and is currently recognized as one of the best-performing algorithms. Its general steps are: first, constructing a scale space; then, searching for extreme points and selecting keypoints; and finally, calculating feature descriptors for the surrounding areas of the selected results. SURF, short for Speeded-Up Robust Feature, is also a scale-invariant feature detection algorithm, generally used for corner detection. The SURF algorithm can be seen as a variant of SIFT, mainly improving detection speed by taking the derivative of Haar's law, and theoretically, it can basically meet real-time processing requirements. ORB (Oriented Fast and Rotated Brief) uses the FAST (features from accelerated segmenttest) algorithm to detect feature points and the BRIEF (Binary Robust Independent ElementaryFeatures) algorithm to calculate the descriptors of the feature points. Its calculation speed is very fast and it can adapt to certain image transformations such as scaling and rotation.

[0159] Feature matching is a crucial step in the registration process. The general approach is to extract feature points one by one from the template image, then use an algorithm to find the feature point with the highest matching degree in the target image and record the matching pair. The OpenCV library provides two commonly used matching algorithms: Brute Force Matcher and FLANN-based Matcher. The basic idea of ​​Brute Force Matcher is to select each feature point in the template image, traverse all possible matches in the target image, and finally return the keypoint with the highest matching degree. FLANN (Fast Library for Approximate Nearest Neighbors) is a fast approximate nearest neighbor search algorithm, theoretically faster, but less accurate than Brute Force. Furthermore, this paper introduces the K-Nearest Neighbor optimization algorithm: for a keypoint in the template image, select the K closest keypoints in the other image by Euclidean distance. Among these K keypoints, calculate the ratio of the nearest distance to the second nearest distance. If the ratio is lower than a set threshold T, the matching pair is retained. In practical applications, K=2 is often set as the parameter, but selecting the threshold T is more complicated. Due to the high dimensionality of the feature space, there are usually many other erroneous matches within the similarity distance of the erroneous matches, so the calculated ratio value will be relatively large. Appropriately reducing the value of T can filter out more erroneous matches, but setting the value of T too low will also result in too few matching points. In this experiment, T=0.5 is preset.

[0160] For the application scenario of this invention embodiment, 40 sets of target images were selected, and the above algorithm was used for comparative testing. Since the format of the ORB operator feature descriptor information is difficult to match with the FLANN algorithm, the ORB+FLANN combination was eliminated, and five algorithms—SIFT+FLANN, SIFT+BF, SURF+FLANN, SURF+BF, and ORB+BF—were used for experiments. To calculate the matching accuracy, some feature point pairs in the test images were manually labeled before the experiment, and their standard transformation matrix was calculated based on the labeling results. Thus, for each keypoint (x0, y0) in the previous image, its matching point (x1, y1) in the subsequent image can be obtained through the standard transformation matrix. During formal testing, if the distance between the matching point (x2, y2) and (x1, y1) output by the algorithm is less than 2 pixels, the feature point is considered to be correctly matched. For the feature matcher, when using the same feature detection algorithm, brute-force matching has a relatively high accuracy, and the processing speed is not excessively reduced compared to FLANN, remaining in the millisecond range, which is within an acceptable range. With the same matcher, the accuracy difference between SIFT and SURF is not significant, with SIFT having relatively higher accuracy but sacrificing some processing speed. Although the ORB algorithm is very fast, its shortcomings in scale invariance and rotation invariance result in very low accuracy.

[0161] Feature matching test results: A solution to this problem is proposed here: attaching markers to the target surface to aid registration. In practical applications, the markers are very small compared to the target itself. During training shooting, shooters at a distance of hundreds of meters cannot see the markers on the target, so they will not affect training. In fact, the marker class in the target image was segmented in the previous chapter, and its label and location information were obtained. Here, the image is first masked based on the segmentation results, retaining only the marker pixels, and then the SIFT+BF algorithm is used for feature matching. This is equivalent to eliminating interference from other parts and performing feature detection and matching only on the marker parts with very obvious features, but the output transformation matrix is ​​still applicable to the entire image. The improved feature matching results and the final registration results are shown below. Figure 14 and Figure 15 As shown in the test, the average accuracy of registration using this method reached 95.93%, and the average processing time was 325.27ms, which meets the requirements of the algorithm in this paper.

[0162] To extract the target variation region, a difference operation is performed on the registered images. The difference operation typically involves directly subtracting the pixel values ​​of two images. Although histogram matching has been performed previously, the result of direct difference inevitably contains a lot of low-grayscale noise. To address this issue, this paper's algorithm improves upon this by thresholding the result after direct difference. Based on the threshold, the grayscale value of each pixel in the image is directly set to 0 or 1, i.e., black or white, to achieve a black and white effect. The expression is as follows:

[0163]

[0164] Clearly, D(x,y) in the above equation is the thresholding result, f t (x,y) and f t+1 (x, y) represents two images, and the key to the problem lies in determining the threshold T. Threshold selection methods include globally fixed thresholding, locally adaptive thresholding, and Otsu's method. Here, we calculate the threshold using Otsu's method. The part of the image we want to retain is called the foreground, and the part that needs to be eliminated is called the background. Let the proportion of foreground pixels be ω0, and its average gray value be μ0; let the proportion of background pixels be ω1, and its average gray value be μ1. Then we have: ω0 + ω1 = 1

[0165] The overall average gray level of the image is: μ = ω0μ0 + ω1μ1

[0166] The inter-class variance is: g = ω0(μ0 - μ) 2 +ω1(μ1-μ) 2

[0167] The above equation can be simplified to: g=ω0ω1(μ0-μ1) 2

[0168] Let p i Let i be the probability of gray level i appearing, then we have:

[0169]

[0170]

[0171]

[0172] By iterating through L gray levels, the gray level k that maximizes g is the threshold T.

[0173] After the above operations, although most of the low-grayscale noise has been eliminated, a lot of high-grayscale noise remains after thresholding. This noise is mainly caused by rounding errors introduced during perspective matrix transformation calculation in image registration. Next, connected components in the difference results are searched, and all connected components with fewer than 5 pixels are removed. This is done to eliminate some small point and band noise. Finally, a minimum bounding box is added based on the search results, and then the bounding box is mapped onto the original image after shooting, as shown below. Figure 16 As shown. To detect newly added bullet holes, simply mask the area outside the ROI in the image. This completes the entire ROI detection algorithm process. It's worth noting that although a significant amount of noise interference has been eliminated, the detection results for the changed areas in the image are still relatively coarse; some noise remains on the outer edge of the target. The accuracy is far from meeting the final requirements for bullet hole detection. Therefore, the next step is to perform precise detection of the bullet holes within the ROI.

[0174] In summary, the ROI detection algorithm based on registration difference according to embodiments of the present invention first eliminates interference caused by illumination changes through histogram matching, then achieves image registration based on feature matching results, and finally performs difference processing on the registration results to complete ROI detection. During this process, to address the difficulty of existing feature matching algorithms in adapting to situations with large target surface jitter, a solution is proposed: feature matching is performed only on markers on the target based on the segmentation results, significantly improving registration accuracy. Furthermore, to address the problem of generating a large amount of low grayscale noise during difference processing, the difference operation is improved, and thresholding processing eliminates most of the noise. Finally, the algorithm was tested and analyzed. Test results show that the ROI detection algorithm achieves the detection of changed regions in continuous target surface images and extracts all newly added bullet hole regions in the image.

[0175] After detecting the region of interest (ROI), all changed regions were extracted, including newly added bullet hole pixels in the current image and some useless target areas, significantly narrowing the scope of bullet hole detection and filtering out old bullet holes on the target. The next step is to proceed with bullet hole detection based on this. This chapter first uses image processing methods to coarsely extract bullet holes from the ROI. Then, a deep neural network classification model is built and trained to evaluate the coarse extraction results, discarding non-bullet hole portions, identifying isolated bullet holes and adhered bullet holes, and finally completing the bullet hole detection. The specific algorithm flow is as follows: Figure 17 As shown.

[0176] Coarse bullet hole extraction is mainly achieved through two steps: image binarization and contour extraction. Binarization prepares the image for successful contour extraction; its task is to eliminate noise interference as much as possible while preserving all bullet hole pixels. Contour extraction can be seen as locating the binarized result, a step in the subsequent bullet hole recognition process.

[0177] Since the grayscale value of the bullet hole pixels in the image is much lower than that of the surrounding areas, a local adaptive thresholding method is used to binarize the ROI. The threshold for each point is determined based on the distribution of pixel values ​​within its neighborhood. This ensures that all bullet hole pixels are extracted while eliminating some noise. The calculation first requires determining the size of the neighborhood window, N×N. For a point f(x,y) in the image, the mean value T of the pixel values ​​within its N×N range is calculated. T is then used as the threshold to binarize the point (x,y).

[0178] Next, contour extraction is performed on the binary image. Common contour extraction methods include edge detection, thresholding, and extraction of high-frequency information from the Fourier transform. This paper uses the `findContours` function from the OpenCV library to implement contour extraction. First, the binarized image is read, and a raster scan is performed from left to right and from top to bottom. Then, the parent boundary is determined based on the boundary type and tracked. The operation ends when the bottom right pixel of the image is reached. The process is as follows: Figure 18 As shown.

[0179] After contour extraction, a minimum bounding box is created based on the extraction results, completing the coarse extraction of bullet holes. The result of the coarse extraction is the suspected bullet hole region in the image, including all bullet hole regions and some non-bullet hole regions, as shown in the figure. Figure 19 As shown.

[0180] The next step is to analyze the coarsely extracted results, removing all non-bullet hole areas to achieve accurate bullet hole detection. Here, deep learning is used for bullet hole recognition. An image classification model is built and trained, and based on the model's output, bullet hole pixels are retained and their types are determined, thus completing the bullet hole detection task.

[0181] Dataset Construction: The training objective of the deep neural network model is to distinguish between isolated bullet holes, connected bullet holes, and non-bullet holes (noise) in the input image. This is equivalent to a multi-class supervised learning problem, requiring a large number of bullet hole and non-bullet hole sample images to build the dataset. The most direct data acquisition method is to manually extract sample data from the target image and classify it. However, this is not only time-consuming and labor-intensive, but also inefficient. Furthermore, the collection standards for noise samples are difficult to measure, making manual annotation impossible. This will lead to difficulties in ensuring the overall quality of the dataset, which can easily negatively impact the model's bullet hole recognition performance.

[0182] To avoid the aforementioned problems, a more convenient and efficient method is used to build the dataset. First, the Labelme tool is used to label the bullet hole regions in the located target image into two categories: isolated bullet holes and adhered bullet holes. A ground truth map of the bullet hole regions is generated based on a JSON file. Then, the bullet hole coarse extraction method proposed earlier is used to extract suspected bullet hole regions from the image. The detection results are cropped, saved as sample image data, and their corresponding regions in the ground truth map are located. Next, the Intersection over Union (IoU) ratio between the two is calculated; the specific calculation method has been described previously and will not be repeated here. If the IoU ratio between the bullet hole ground truth map and the coarse extraction result exceeds 0.5, the cropped sample is classified into the corresponding bullet hole class; otherwise, it is classified into the non-bullet hole class. In this way, sample data suitable for the algorithm of this embodiment is obtained, such as... Figure 20 As shown. Following the above approach, a program was written to batch process 715 target images, creating a dataset containing 5308 isolated bullet hole samples, 603 adhered bullet hole samples, and 5549 non-bullet hole samples.

[0183] Image classification refers to the task of assigning labels to a given image from a predefined set of categories using a certain algorithm. It is a hot research topic in computer vision and a key technology for many applications such as face recognition, text recognition, and industrial visual inspection. Existing technologies have seen the emergence of a series of classic models such as VGG, ResNet, and GoogleNet. ResNet50 is a classic model in the ResNet series, containing 50 two-dimensional convolutional layers. It first performs convolution operations on the input, then passes it through four sets of residual blocks, and finally through fully connected layers to complete the classification task. Its structure is as follows: Figure 21 As shown, a deep neural network model based on ResNet50 was built, and some improvements were made to the original network structure.

[0184] Introducing an attention mechanism: Since the bullet hole images are very small, when training the original ResNet50 model, the bullet hole information in the feature map is basically destroyed after several rounds of downsampling. It is difficult for the feature information to be passed into the depth of the network, and the classification effect will inevitably be affected. Therefore, we consider introducing an attention mechanism to capture high-level semantic information, enhance the model's learning of bullet hole features, and improve its classification accuracy.

[0185] In computer vision, attention mechanisms can be viewed as a resource allocation mechanism that redistributes resources that were originally evenly distributed to objects of varying importance. For deep neural networks, the resources that attention mechanisms focus on are the weights within each hidden layer. Attention mechanisms can be broadly categorized into hard attention and soft attention. Hard attention often focuses only on regions of interest, while other regions are ignored; image cropping is a common example of this. In deep learning, soft attention mechanisms are more commonly used, including spatial attention and channel attention. Spatial attention maps spatial information from feature maps to another space while preserving key information; it can be understood as directing the model's attention to a specific area. Channel attention, typically represented by SENet, adjusts the weights between channels in convolutional layers; it can be understood as directing the model's attention to a specific area. Because the attention given to a region by soft attention is continuously distributed and differentiable, its gradient can be calculated using a neural network, and the specific weights of the attention can be learned through forward propagation and backward feedback. At present, most scholars are committed to developing more complex attention modules to pursue higher performance, which will lead to an increase in the complexity of the network model, and sometimes will affect the training effect. To solve this problem, Qilong Wang et al. proposed the ECA (Efficient Channel Attention)

[65] structure in 2020. ECA is a lightweight channel attention module, which is an improved version of SENet. It can balance the performance and complexity of the model. Although it only involves a few parameters, it can bring significant performance improvement when using common sets for training. The structure of ECABlock is as follows. Figure 22 As shown.

[0186] GAP (Global Average Pooling) averages and compresses each two-dimensional feature channel of the input tensor into a single real number, resulting in a global receptive field of H×W. σ is the activation function. ECA can be implemented using a fast one-dimensional convolution with kernel k, where k represents the range of local cross-channel interactions, and k is typically 5. It uses an adaptive method to determine k, avoiding manual adjustment through cross-channel interactions, as shown in the following formula:

[0187]

[0188] Where |t| odd It represents the nearest odd number to t.

[0189] However, for the bullet hole classification model to be trained in this embodiment of the invention, after a large number of small-sized samples in the dataset are compressed by global average pooling with a large receptive field, the information represented by the image is already very blurred. In this case, adjusting the weights through one-dimensional convolution will theoretically not yield good results. Therefore, this embodiment of the invention further improves ECA based on the existing model. First, GAP is replaced with SP (Soft Pooling). Compared to commonly used max pooling and average pooling series, it can reduce feature redundancy while alleviating information loss. In SoftPool, the input feature map is downsampled by a 2×2 convolution kernel. Its output is the sum of the exponential weights of each pixel within the convolution kernel range. The weight is the ratio of the natural exponent in the activation region to the sum of the natural exponents of all activations in the neighborhood R, i.e.:

[0190] Therefore, its output can be expressed as:

[0191] At this point, since the output tensor size has changed, it's no longer possible to connect them using one-dimensional convolution. Therefore, we consider using depthwise separable convolution to replace the original one-dimensional convolutional layer. Depthwise separable convolution consists of a depthwise convolution (DW) and a pointwise convolution (PW) cascaded together. The DW primarily reduces the number of parameters, while the PW adjusts the model channels. It has significantly fewer parameters than a typical two-dimensional convolution, making it a lightweight convolutional layer. Here, we use a 3×3 depthwise separable convolution kernel without changing the number of input channels. In summary, the improved ECA model structure is as follows: Figure 23As shown, for ease of description, the embodiments of the present invention will be referred to as ECA modifications thereafter. Finally, the IdentityBlock and Conv Block of the ResNet50 model are modified, and the ECA modifications are inserted into its basic convolutional blocks.

[0192] Improved Activation Function: The activation function, also known as the activation function, primarily enables neural networks to express nonlinear information. ResNet50 uses the original ReLU activation function by default. This embodiment uses the smoother Mish activation function instead of the traditional ReLU to connect the layers. Its expression is as follows: mish(x) = xtanh(ln(1+e)) x )).

[0193] The main parameters during the training process of the image classification model are shown in Table 3:

[0194] Table 3 Main parameters for model training

[0195]

[0196] The samples in the bullet hole dataset were shuffled and then distributed to the training set, validation set, and test set in a ratio of 7:1:2. The specific details of the division are shown in Table 4. There is no overlap among the three sets.

[0197] Table 4. Details of Dataset Partitioning

[0198]

[0199] To prevent overfitting and enhance the model's generalization ability, data augmentation is introduced, rotating and horizontally flipping the sample images with a certain probability. Additionally, due to the large number of negative samples in the dataset and the somewhat unbalanced ratio of positive to negative samples, a focal loss function is introduced to reduce the weight of the large number of simple negative samples during training. Its expression is shown below:

[0200]

[0201] The Adamax optimizer is a variant of Adam, primarily simplifying the boundary conditions for the upper bound of the learning rate. The difference between their expressions lies in the calculation method of the second-order moment estimate and the parameter update rule. They use equation v... t =max(β2v) t-1 ,|g t |) and formula Perform calculations

[0202] Based on the bullet hole classification dataset established in this embodiment of the invention, the improved model and several other commonly used networks were trained. Similar to the target segmentation experiment, each model used the same hyperparameters as much as possible during training, but some values ​​were fine-tuned. After 100 rounds of training, all models converged. The improved model from this embodiment of the invention was used to identify bullet holes and remove noise. The bullet hole detection results are as follows: Figure 24 As shown.

[0203] The algorithm of this invention can achieve high-precision detection quickly, with both false negative and false positive rates within acceptable ranges. It can accurately detect bullet holes and exhibits good anti-interference capabilities in complex environments.

[0204] In summary, the bullet hole detection algorithm of this invention first uses binarization and contour extraction to coarsely extract bullet holes from the target image. Then, based on deep learning technology, a ResNet-based classification model is built and trained to recognize the coarsely extracted results, thus completing the bullet hole detection. In this process, a more convenient and efficient method is proposed to build the dataset, and commonly used deep learning image classification models are summarized and analyzed. Addressing the shortcomings of the original ResNet50 model, an improved attention mechanism and activation function are introduced to improve classification accuracy, and the problem of imbalanced samples in the dataset is solved by modifying the loss function. Finally, the algorithm is tested and analyzed as a whole. The test results show that the bullet hole detection algorithm achieves high-precision detection of bullet holes in images under complex scenarios and can identify isolated and connected bullet holes. Combined with target localization and ROI detection algorithms, incremental detection of bullet holes in continuous images is finally achieved.

[0205] The technical solutions of the embodiments of the present invention have the following beneficial effects:

[0206] (1) A target localization and target type identification algorithm based on semantic segmentation is proposed. Based on the characteristics of the dataset used in this paper, the structure of the RefineNet semantic segmentation model is improved to adapt to high-resolution input. Furthermore, to address the problem of bullet hole information loss during segmentation, the loss function is improved, reducing false negatives. The target localization algorithm achieves pixel-level high-precision semantic segmentation of the effective area of ​​various target types, effectively solving the problem of identifying multiple target types in training shooting and the interference of complex outdoor shooting range environments on target localization. It also lays the foundation for subsequent processing.

[0207] (2) A ROI detection algorithm based on registration difference is proposed. The algorithm first eliminates interference caused by illumination changes through histogram matching. Then, addressing the difficulty of existing feature matching algorithms in handling large target surface jitter, a solution is proposed: feature matching is performed only on markers on the target based on the segmentation results, significantly improving registration accuracy. Finally, to address the problem of a large amount of low-grayscale noise generated during difference, the difference operation is improved, and most of the noise is eliminated through thresholding. The ROI detection algorithm achieves the detection of changed regions in continuous target surface images, extracting all newly added bullet hole regions in the image.

[0208] (3) A bullet hole detection algorithm was proposed. To address the difficulty in extracting features from small-sized bullet hole samples, an improved attention mechanism and activation function were introduced to enhance classification accuracy based on the original ResNet model. Furthermore, the imbalance of samples in the dataset was addressed by modifying the loss function. The bullet hole detection algorithm achieves high-precision detection of bullet holes in images under complex conditions and can identify both isolated and connected bullet holes. By combining target localization and ROI detection algorithms, incremental detection of bullet holes in continuous images was finally achieved.

[0209] Device Example 1

[0210] According to an embodiment of the present invention, a bullet hole detection device based on computer vision is provided. Figure 25 This is a schematic diagram of a computer vision-based bullet hole detection device according to an embodiment of the present invention, as shown below. Figure 25 As shown, the computer vision-based bullet hole detection device according to an embodiment of the present invention specifically includes:

[0211] The target removal module 250 is used to acquire the target image input from the front end after the shooting begins, identify and segment the effective area of ​​the target image through target surface positioning, remove the useless background parts from the target image input from the front end, and output the processed target image; specifically used for:

[0212] The target image input from the front end is smoothed using a bilateral filter through a smoothing filter.

[0213] A semantic segmentation model was built and trained using deep learning methods to perform target segmentation. Specifically, a dataset for the semantic segmentation model was established, and the data in the dataset was annotated at the pixel level. The dataset specifically includes: real target images captured by a camera at different times and under different environments, and scaled-down self-made target images captured by a camera. The annotation types include: rocket launcher, machine gun, frontal infantry fighting vehicle, frontal tank, tank firing point, markers, and background. Based on the RefineNet model, the network depth and width were increased to build the semantic segmentation model. The annotated dataset was divided into training, validation, and test sets. Target images were captured at a resolution of 3120×3120, and the target images were enhanced by introducing a weighted cross-entropy loss function to weight the target images. The Adam optimizer was used to train the semantic segmentation model.

[0214] Target surface segmentation is performed using a trained semantic segmentation model.

[0215] Remove useless background from the target image after target surface segmentation to achieve target surface localization and output the processed target image;

[0216] ROI detection algorithm module 252 is used to detect changes in the target image acquired before and after firing at the same target position based on the processed target image using the ROI detection algorithm, eliminating old bullet holes and identifying new bullet holes; specifically used for:

[0217] Read two frames of target images before and after firing, and perform histogram matching on the two frames: calculate a mapping function based on the existing grayscale distribution and the desired target distribution, where p r (r) represents the existing distribution, denoted by p. z (z) represents the target distribution, which is the distribution p of the image to be processed. r (r) Transform according to Formula 1, and calculate the result s. k The value is rounded to obtain the discrete form of the histogram equalization:

[0218]

[0219] Where M×N is the size of the image, n j The grayscale value is r j The number of pixels, where L represents the number of gray levels in the image;

[0220] Based on the target histogram distribution p r The value of (r) is calculated for q = 0, 1, 2, ..., L-1 as 2, and the result G is rounded to obtain the transformation function G(Z). q ):

[0221]

[0222] When k = 0, 1, 2, ..., L-1, for each s k All are matched according to the value of the transformation function G, and the corresponding z is matched accordingly. q Thus, G(Z) q ) and s k The value is closest to , and all mappings from s to z are recorded, for any s k If its corresponding z q If there is more than one value, then the smallest value is selected.

[0223] After performing histogram equalization on the image to be processed, based on the mapping obtained in the previous step, the equalized pixel values ​​s in the image are calculated. k Mapped to the corresponding z q This yields the result image after histogram matching.

[0224] A feature detector is determined, and feature points and feature descriptors are obtained through the feature detector. Then, a feature matcher is built, and the target image after shooting is used as a template to match the feature points with the target image before shooting. Finally, the previous target image is corrected according to the matching results, thereby completing the registration of the registered image. In the feature matching process, the target image is first masked according to the segmentation results, retaining only the marker pixels, and then the SIFT+BF algorithm is used for feature matching.

[0225] The registered target image is subjected to a difference operation. The pixel values ​​of the two images are directly subtracted to obtain the difference result. The difference result after direct difference is thresholded. Based on the Otsu method, the threshold T is determined according to formula 4-9. According to the threshold T and formula 3, the gray value of each pixel in the image is directly set to 0 or 1, that is, black or white, so that it presents a black and white effect.

[0226]

[0227] Where D(x,y) is the thresholding result, f t (x,y) and f t+1 (x,y) represents the two images before and after;

[0228] Let the proportion of foreground pixels be ω0, and its average grayscale value be μ0; let the proportion of background pixels be ω1, and its average grayscale value be μ1, then we have:

[0229] ω0+ω1=1 Formula 4;

[0230] The overall average grayscale value of the image is:

[0231] μ=ω0μ0+ω1μ1 Formula 5;

[0232] The between-class variance is:

[0233] g=ω0(μ0-μ) 2 +ω1(μ1-μ) 2 Formula 6;

[0234] Equation 6 can be simplified to:

[0235] g = ω0ω1(μ0-μ1) 2 Formula 7;

[0236] Let p i Let i be the probability of gray level i appearing, then we have:

[0237]

[0238]

[0239] The threshold T is obtained by iterating through L gray levels and finding the gray level k that maximizes g.

[0240] The connected components in the difference results are searched, and all connected components with fewer than 5 pixels are removed to eliminate some point and strip noise.

[0241] Based on the search results of connected components, a minimum bounding box is added, and then the bounding box is mapped onto the original image after shooting. The part outside the ROI in the image is directly masked to determine the newly added bullet hole.

[0242] The bullet hole detection module 254 is used to perform bullet hole detection by combining image processing technology with deep learning technology based on the determined newly added bullet holes. Specifically, it is used for:

[0243] Coarse extraction of bullet holes in the region of interest is performed using image binarization and contour extraction.

[0244] A deep neural network classification model was built and trained: Using the Labelme tool, bullet hole regions in the localized target image were labeled into two categories: isolated bullet holes and adhered bullet holes. A ground truth map of the bullet hole regions was generated based on a JSON file. Suspected bullet hole regions were extracted from the image through image binarization and contour extraction. The detection results were cropped and saved as sample image data, and their corresponding regions in the ground truth map were located. The intersection-over-union (IoU) ratio between the two was then calculated. If the IoU ratio of the bullet hole ground truth map and the coarse extraction result exceeded 0.5, the cropped sample was classified into the corresponding bullet hole class; otherwise, it was classified as a non-bullet hole class. This established the dataset for the deep neural network classification model. A deep neural network classification model based on the ResNet classification model, incorporating an attention mechanism and the Mish activation function, was built. The labeled dataset was divided into training, validation, and test sets. Data augmentation was performed on the dataset, and a focus loss function was introduced to weight the dataset. The Adamax optimizer was used to train the deep neural network classification model. The trained deep neural network classification model was then used to identify the coarse extraction results.

[0245] The results of the initial coarse extraction are evaluated, non-bullet hole parts are removed, isolated bullet holes and adhered bullet holes are identified, and finally the bullet hole detection is completed.

[0246] The embodiments of the present invention are system embodiments corresponding to the above method embodiments. The specific operation of each module can be understood by referring to the description of the method embodiments, and will not be repeated here.

[0247] Device Example 2

[0248] This invention provides a bullet hole detection device based on computer vision, such as... Figure 26 As shown, it includes: a memory 260, a processor 262, and a computer program stored on the memory 260 and executable on the processor 262, wherein the computer program, when executed by the processor 262, performs the steps as described in the method embodiment.

[0249] Device Example 3

[0250] This invention provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor 262, implements the steps described in the method embodiment.

[0251] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0252] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A computer vision-based bullet hole detection method, characterized by, The application relates to a target image processing method and device. After starting shooting, a target image input from a front end is acquired, an effective area of the target image is recognized and segmented through target surface positioning, useless background parts in the target image input from the front end are removed, and a processed target image is output; According to the processed target image, a change part of the target image collected before and after shooting at the same target position is detected through an ROI detection algorithm, and new bullet holes are determined by eliminating old bullet holes; According to the determined new bullet holes, image processing technology and deep learning technology are combined to detect the bullet holes, specifically including: A bullet hole in a region of interest is coarsely extracted through image binarization and contour extraction; A deep neural network classification model is built and trained: the bullet hole regions in the positioned target image are labeled as isolated bullet hole types and adhered bullet hole types by using a Labelme tool, a bullet hole region true value graph is generated according to a json file, a suspected bullet hole region in the image is extracted through image binarization and contour extraction, a detection result is intercepted, and the sample image data is saved and positioned in the corresponding region of the true value graph, then the intersection-over-union ratio between the two is calculated, if the intersection-over-union ratio of the bullet hole true value graph and the coarsely extracted result exceeds 0.5, the intercepted sample is classified as the corresponding bullet hole type, otherwise, the sample is classified as a non-bullet hole type, thereby establishing a data set of the deep neural network classification model, building a deep neural network classification model based on a ResNet classification model and introducing an attention mechanism and a Mish excitation function, dividing the labeled data set into a training set, a verification set and a test set, performing data enhancement processing on the data set, introducing a focal loss function to weight the data set, and using an Adamax optimizer to train the deep neural network classification model; the trained deep neural network classification model is used to identify the coarsely extracted result. The coarsely extracted result after identification is judged, non-bullet hole parts are removed, isolated bullet holes and adhered bullet holes are identified, and finally the bullet hole detection is completed.

2. The method of claim 1, wherein, After starting shooting, a target image input from a front end is acquired, an effective area of the target image is recognized and segmented through target surface positioning, useless background parts in the target image input from the front end are removed, and a processed target image is output, specifically including: A target image input from a front end is smoothed through a smoothing filter and a bilateral filter; A semantic segmentation model is built and trained by using a deep learning method to perform target surface segmentation; Useless backgrounds are removed from the target image after target surface segmentation, target surface positioning is realized, and a processed target image is output.

3. The method of claim 2, wherein, The semantic segmentation model is built and trained by using a deep learning method to perform target surface segmentation, specifically including: A data set of the semantic segmentation model is established, and data in the data set is pixel-level labeled, wherein the data set specifically includes real target images collected by a camera at different times and in different environments and self-made target images collected by the camera and scaled down in proportion, and the labeling types include a rocket launcher, a machine gun, a front infantry tank, a front tank, a tank firing point, a marker and a background; Based on the RefineNet model, the depth and width of the network are increased, and a semantic segmentation model is built; The labeled data set is divided into a training set, a validation set and a test set, a target image of 3120x3120 resolution is collected, and the target image is enhanced, a weighted cross-entropy loss function is introduced for weighting, and an Adam optimizer is used for training of the semantic segmentation model; The target surface is segmented by the trained semantic segmentation model.

4. The method of claim 1, wherein, According to the processed target image, the change part of the target image collected before and after shooting at the same target position is detected by the ROI detection algorithm, and the old bullet hole is eliminated to determine the new bullet hole, which specifically includes: Two target images before and after shooting are read, and the two target images are subjected to histogram matching: a mapping function is calculated according to an existing gray distribution and an expected target distribution, wherein, represents the existing distribution, and represents the target distribution, and the distribution of the image to be processed is transformed according to formula 1, and the value of the calculation result is rounded to obtain a discrete form of histogram equalization: Formula 1 ; wherein, is the size of the image, is the number of pixels having a gray value of denotes the number of gray levels in the image; According to the target histogram distribution , the values of are calculated as 2, and the calculated results are rounded to obtain the transform function : Official 2; When , for each , the value of the transform function is matched to the corresponding , so that is closest to , and all mappings from to are recorded, for any , if its corresponding value is more than one, the smallest value is selected from it; After histogram equalization is performed on the image to be processed, each equalized pixel value in the image is mapped to a corresponding pixel value in the reference image according to the mapping obtained in the previous step to obtain a result image after histogram matching , thereby obtaining a result image after histogram matching A feature detector is determined, and feature points and feature descriptors are obtained through the feature detector, and then a feature matcher is established, the target image after shooting is taken as a template, and the feature points between the target image before shooting are matched, and finally the previous target image is corrected according to the matching result, so that the image registration is completed, wherein during feature matching, the target image is masked according to the segmentation result, only the marker pixels are retained, and then the SIFT+BF algorithm is used for feature matching; The target image after registration is subjected to difference operation, the pixel values of the two images are directly subtracted, and the difference result is obtained, the threshold value T is determined according to the OTSU method according to formula 4-9, and the gray value of each pixel in the image is directly set to 0 or 1, that is, black or white, according to the threshold value T and formula 3, so that it shows black and white effect; Official 3; wherein, is a thresholded result, and denotes the two images before and after. Let the foreground pixel ratio be , and the average gray value of the foreground pixel be ; the background pixel ratio be , and the average gray value of the background pixel be , then there is: Official 4; The overall average gray value of the image is: Equation 5; The interclass variance is: Official 6; Formula 6 is simplified as: Official 7; Let To The probability of the occurrence of a gray level, then we have: Equation 8; Equation 9; traversing one gray scale, such that the gray scale taking the maximum value value is the threshold value ; The connected domain in the difference result is searched, and all connected domains with a pixel number less than 5 are removed to eliminate part of the point noise and strip noise; According to the search result of the connected domain, a minimum bounding box is added, and then the bounding box is mapped to the original image after shooting, and the part outside the ROI in the image is directly masked to determine the new bullet hole.

5. A computer vision-based bullet hole detection apparatus, characterized by, It includes: The elimination module is used for acquiring the target image input by the front end after starting to shoot, identifying and segmenting the effective area of the target image through target surface positioning, eliminating the useless background part in the target image input by the front end, and outputting the processed target image; The ROI detection algorithm module is used for detecting the change part of the target image collected before and after shooting at the same target position according to the processed target image through the ROI detection algorithm, eliminating the old bullet hole to determine the new bullet hole; The bullet hole detection module is used for combining image processing technology and deep learning technology to detect the bullet hole according to the determined new bullet hole; specifically for: Using image binarization and contour extraction to coarsely extract the bullet hole in the region of interest; The deep neural network classification model is built and trained: the bullet hole regions in the target image after positioning are labeled as isolated bullet hole class and adhered bullet hole class by using the Labelme tool, the bullet hole region true value image is generated according to the json file, the suspected bullet hole regions in the image are extracted by image binarization and contour extraction, the detection result is intercepted, saved as sample image data, and the corresponding region in the true value image is located, then the intersection over union between the two is calculated, if the intersection over union of the bullet hole true value image and the rough extraction result is greater than 0.5, the intercepted sample is classified as the corresponding bullet hole class, otherwise it is classified as a non-bullet hole class, thereby establishing the data set of the deep neural network classification model, building the deep neural network classification model based on the ResNet classification model with the introduction of the attention mechanism and the Mish excitation function, dividing the labeled data set into a training set, a validation set and a test set, performing data enhancement processing on the data set, introducing a focal loss function for the data set for weighting, and using an Adamax optimizer to train the deep neural network classification model; the rough extraction result is recognized by the trained deep neural network classification model. The rough extraction result after recognition is judged, and the non-bullet hole part is removed, the isolated bullet hole and the adhered bullet hole are recognized, and finally the bullet hole detection is completed.

6. The apparatus of claim 5, wherein The removing module is specifically configured to: smoothly filter the target image input from the front end using bilateral filtering through a smoothing filter; build and train a semantic segmentation model using a deep learning method to segment the target surface; remove the useless background from the target image after target surface segmentation to realize target surface positioning and output the processed target image; The ROI detection algorithm module is specifically configured to: Two target images before and after shooting are read, and the two target images are subjected to histogram matching: a mapping function is calculated according to an existing gray distribution and an expected target distribution, wherein, the existing distribution is denoted by f(x), the target distribution is denoted by g(x), the distribution of a to-be-processed image is denoted by f(x), and the mapping function is denoted by T(x). The existing distribution is denoted by f(x), the target distribution is denoted by g(x), the distribution of a to-be-processed image is denoted by f(x), and the mapping function is denoted by T(x). The to-be-processed image is subjected to transformation according to formula 1, and the value of the calculation result is rounded to obtain a discrete form of histogram equalization. The to-be-processed image is subjected to transformation according to formula 1, and the value of the calculation result is rounded to obtain a discrete form of histogram equalization. Formula 1 ; wherein, is the size of the image, is the number of pixels having a gray value of denotes the number of gray levels in the image;​ According to the target histogram distribution , the values of are calculated as 2, and the calculated results are rounded to the values of the transform function : Official 2; When , for each , the value of the transform function is matched to the corresponding , so that is closest to the value of , and all mappings from to are recorded, for any , if there are more than one corresponding , the smallest value is selected from them; After histogram equalization is performed on the image to be processed, each equalized pixel value in the image is mapped to a corresponding pixel value in the reference image according to the mapping obtained in the previous step to obtain a result image after histogram matching , thereby obtaining a result image after histogram matching determine a feature detector, obtain feature points and feature descriptors through the feature detector, then establish a feature matcher, use the target image after shooting as a template, match the feature points between the target image before shooting, and finally correct the previous target image according to the matching result to complete image registration, wherein, during feature matching, first mask the target image according to the segmentation result to only keep the marker pixels, and then use the SIFT+BF algorithm for feature matching; perform difference operation on the registered target image, directly subtract the pixel values of the two images to obtain the difference result, perform thresholding processing on the directly difference result, determine the threshold T according to the OTSU method according to formula 4-9, and set the gray value of each pixel in the image to 0 or 1, i.e. black or white, according to the threshold T and formula 3, so that it shows black and white effect; Official 3; wherein is a thresholded result, and denotes the two images before and after. Let the foreground pixel ratio be , and the average gray value of the foreground pixel be ; the background pixel ratio be , and the average gray value of the background pixel be , then there is: Official 4; The overall average gray value of the image is: Formula 5; The inter-class variance is: Official 6; Simplify formula 6 to: Official 7; Let For The probability of the occurrence of a gray level, then we have: Official 8; Official 9; traversing one gray scale, such that the gray scale taking the maximum value value is the threshold value ; Search for the connected domain in the difference result, and remove all connected domains with less than 5 pixels, to eliminate part of the point noise and strip noise; According to the search result of the connected domain, add the minimum bounding box, then map the bounding box to the original image after shooting, directly mask the part outside the ROI in the image, and determine the new bullet hole.

7. The apparatus of claim 6, wherein, The culling module is specifically used for: A data set of the semantic segmentation model is established, and data in the data set is pixel-level labeled, wherein the data set specifically includes: real target target images collected by using a camera at different times and in different environments, and self-made target target images collected by using the camera and scaled down, and the labeling types include: a rocket launcher, a machine gun, a frontal infantry tank, a frontal tank, a tank firing point, a marker, and a background; Based on the RefineNet model, the network depth and width are increased to build the semantic segmentation model; The labeled data set is divided into a training set, a validation set, and a test set, target target images with a resolution of 3120x3120 are collected, the target target images are subjected to enhancement processing, a weighted cross-entropy loss function is introduced to weight the target target images, and an Adam optimizer is used to train the semantic segmentation model; The target surface is segmented by using the trained semantic segmentation model.

8. A computer vision-based bullet hole detection apparatus, characterized by, It comprises: A memory, a processor, and a computer program stored on the memory and executable on the processor, which, when executed by the processor, implements the steps of the computer vision-based bullet hole detection method according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an information transmission implementation program, and the program, when executed by the processor, implements the steps of the computer vision-based bullet hole detection method according to any one of claims 1 to 4.

Images