A method for detecting small and weak craters in high-resolution aerial images

Abstract

The present application relates to a crater target detection method, in particular to a weak and small crater target detection method in high-resolution aerial images, which solves the technical problems of insufficient applicability, weak robustness and low detection accuracy in weak and small crater detection methods in low signal-to-noise ratio images in real scenes. First, obtain the high-resolution original image of the explosion scene, and divide it into multiple image subblocks; second, randomly select a small number of image subblocks, manually label the training samples, extract multi-scale features, generate positive, negative, difficult and easy balanced anchor boxes through the balanced region candidate network, jointly optimize, and obtain the training model; third, based on the training parameter model, the remaining image subblocks are calculated through the multi-scale feature extraction network, the region candidate network, the region alignment network and the detection head network, and the position and confidence of the crater are inferred; finally, the detection results in the image subblocks are merged.

Description

TECHNICAL FIELD

[0001] The present application relates to a crater target detection method, in particular to a weak and small crater target detection method in high-resolution aerial images. BACKGROUND

[0002] The position, number, distribution, coverage area and other parameters of the craters formed by the explosion of the tiny shrapnel of fire equipment after touching the ground are of great significance to the performance research or explosion damage assessment of fire equipment. The basis for obtaining these parameters is crater detection. The research on crater detection technology mainly includes three categories: first, the actual measurement of production / identification departments, which obtains the crater parameters by manual field measurement in a professionally laid test site. This method is accurate but complex and expensive to implement; second, simulation calculation through simulation technology, which establishes a simulation system for the explosion of the equipment based on the analysis model of the equipment, calculates the explosion effect of the equipment, and then calculates the related attributes of the crater. This method can simulate a variety of explosion forms, but its accuracy depends heavily on the accuracy of the simulation model; third, information processing technology based on images, which obtains the crater target information by imaging the explosion scene and using image information processing methods. This method is simple and intuitive, and the cost performance is high. It is the most widely used method at present.

[0003] In the research on image analysis and processing-based methods, the general processing flow is as follows: first, use a high-definition, high-resolution camera on a UAV to take photos from above the explosion scene or use a satellite to remotely sense the area to obtain images of the explosion scene; second, use statistical learning methods in traditional machine learning to manually design some targeted statistical features, such as Haar-like features, Hough transform, strong light and shadow area features, and interest point features, to extract low- and medium-level appearance information of the crater target, such as boundary, shape, and texture; finally, use unsupervised classification methods, such as decision trees, support vector machines, and template matching, to classify and identify the region features based on similarity measurement of the features. The features designed by unsupervised statistical machine learning methods are abstract descriptions of the target appearance in a specific scene. When the scene changes to a real scene with low signal-to-noise ratio, the detection accuracy and robustness are not high when facing complex and variable weak and small crater target detection tasks.

[0004] Therefore, the medium- and low-level features designed only by the statistical attributes of the target appearance do not truly describe the high-level semantic information of the target, and the strong essential discriminant attributes of the target category are not extracted, resulting in low accuracy and poor robustness in real scene crater detection. SUMMARY

[0005] The purpose of this invention is to address the technical problems of insufficient applicability, low detection accuracy, and weak robustness in methods for detecting small craters in low signal-to-noise ratio images in real-world scenarios, and to provide a method for detecting small crater targets in high-resolution aerial images.

[0006] The design concept of this invention is:

[0007] To address the issues of low accuracy and weak robustness in detecting small craters in low signal-to-noise ratio scenarios, this invention proposes a deep learning-based method for small crater target detection. First, a drone equipped with a camera acquires scene images, generating high-precision digital orthophotos based on oblique photogrammetry, and then segmenting these images into small image blocks. Second, image blocks are randomly selected, and training samples are manually labeled. A multi-scale feature extraction method combining high-level semantics with accurate boundary responses and high-resolution mid-to-low-level appearances is extracted using a backbone framework fused with a residual network, a feature pyramid network, and a path enhancement network. Third, a balanced region candidate network generates positive, negative, and difficult / easy balanced anchor boxes, which are then jointly optimized to obtain a training model. Next, based on the training model, test images are processed by a multi-scale feature extraction network, a region candidate network, a region alignment network, and a detection head network to infer the location and confidence of the craters. Finally, the detection results from the image blocks are merged.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A method for detecting weak crater targets in high-resolution aerial images, characterized by the following steps:

[0010] Step 1: Obtain a high-resolution original image of the explosion scene. The high resolution refers to a pixel spatial resolution better than 0.5 cm.

[0011] Step 2: Divide the original image obtained in Step 1 into segments with a fixed step size s to obtain image sub-blocks with a resolution of H×W;

[0012] Step 3: Randomly select image sub-blocks with a proportion of r from the image sub-blocks obtained in Step 2 and record them as the training set, where 0.1≤r≤0.3. The remaining image sub-blocks are recorded as the test set. Manually label the crater targets in each image sub-block in the training set as positive sample instances to obtain manually labeled ground truth anchor boxes.

[0013] Step 4: Input each image sub-block in the training set and its corresponding annotation file into the target detection network. Through end-to-end offline training optimization, such as extracting multi-scale features, generating IOU-equalized candidate anchor boxes, obtaining region alignment features, classifying and regressing region alignment features, etc., obtain the training model for crater target detection.

[0014] Step 5: Based on the trained model in Step 4, use the object detection network to perform online inference on the test set to obtain the probability and location of crater detection in the test set;

[0015] Step 6: Combine the probability and location of crater detections obtained in Step 5 to obtain the crater detection results.

[0016] Furthermore, step four specifically involves:

[0017] 4.1 Multi-scale feature extraction

[0018] 4.1.1. Use the ResNet50 residual network to extract five stages of features S for each image sub-block in the training set. j Where j = 2, 3, 4, 5, 6; starting from stage S3, the length and width scales of the features acquired in each stage are reduced to half of the previous stage, while the number of channels remains unchanged;

[0019] 4.1.2 Extracting features from the feature pyramid network from top to bottom, wherein each layer of the feature pyramid network has features F j The residual network characteristics S of the corresponding stage j After resampling by the convolutional network and its upper-layer features F j+1 It is constructed by upsampling and fusing with nearest neighbor interpolation by 2 times. By combining lateral connections and top-down approach, the high-level semantic information of the feature pyramid network is gradually propagated to the low-level layers, and multi-scale feature pyramid network feature maps are extracted.

[0020] 4.1.3 Extracting path enhancement network features from bottom to top, wherein each layer of the path enhancement network has features P j From its next layer feature P j-1 Feature pyramid layer F, which is downsampled by 2x and laterally connected by a spatial convolutional network j Integration and composition;

[0021] The scales of the feature maps acquired in stages S2(F2, P2) to S6(F6, P6) are 256*256*256, 128*128*256, 64*64*256, 32*32*256, and 16*16*256, respectively.

[0022] 4.2 Generation of Candidate Anchor Box Samples for IOU Equalization

[0023] 4.2.1 At each location of each feature layer of the path enhancement network obtained in step 4.1.3, construct multiple arrays with different scales and aspect ratios, consisting of a 4-dimensional array α = (x... α ,y α ,w α ,h α A horizontal anchor frame is formed by x, where xα Let x be the coordinate of the center point of the horizontal anchor frame along the x-axis, and y be the coordinate of the center point of the horizontal anchor frame along the x-axis. α Let w be the coordinate of the center point of the horizontal anchor frame along the y-axis. α h is the width of the horizontal anchor frame. α The height of the horizontal anchor frame;

[0024] 4.2.2 Construct a region candidate network consisting of a spatial convolutional sliding window and two convolutional layers: a classification convolutional layer and a regression convolutional layer. The classification and regression convolutional layers are used to classify and regress the region containing the horizontal anchor box. The regression layer generates the offset Δ of the predicted candidate anchor box relative to the horizontal anchor box.

[0025] Δ=(Δ x ,Δ y ,Δ w ,Δ h )

[0026] Where, Δ x Δ represents the offset of the center points of the two points along the x-axis. y Δ represents the offset of the center point of both points along the y-axis. w Δ is the offset between the widths of the two. h This represents the offset in height between the two.

[0027] 4.2.3 The predicted candidate anchor box β = (x, y, w, h) is calculated using the following formula:

[0028]

[0029] Where e is a constant, x is the coordinate of the center point of the predicted candidate anchor box in the x-axis direction, y is the coordinate of the center point of the predicted candidate anchor box in the y-axis direction, w is the width of the predicted candidate anchor box, and h is the height of the predicted candidate anchor box.

[0030] 4.2.4 Calculate the confidence score p for predicting candidate anchor boxes as foreground. i Select p i The top m predicted candidate anchor boxes are used as candidate anchor box samples, where m is an integer and 1000≤m≤5000;

[0031] 4.2.5 Calculate the IOU value between each candidate anchor box sample selected in the regional candidate network and the manually labeled ground truth anchor box, and select positive anchor box samples, simple negative anchor box samples and difficult negative anchor box samples according to the IOU value.

[0032] 4.2.6. The region candidate network is trained and optimized based on the selected positive anchor box samples, simple negative anchor box samples and difficult negative anchor box samples. When the convergence condition is met, the final candidate anchor boxes are generated.

[0033] The loss function L1 for training and optimizing the region candidate network is:

[0034]

[0035] Among them, t i t represents the offset of the candidate anchor frame relative to the horizontal anchor frame. i =(t x ,t y ,t w ,t h ), t x t represents the offset of the center point of the candidate anchor frame relative to the center point of the horizontal anchor frame along the x-axis. y t represents the offset of the candidate anchor frame center point relative to the horizontal anchor frame center point along the y-axis. w t represents the offset of the candidate anchor frame width relative to the horizontal anchor frame width. h The offset of the candidate anchor frame height relative to the horizontal anchor frame height;

[0036] This represents the offset of the truth anchor frame relative to the horizontal anchor frame. This represents the offset of the center point of the truth anchor frame relative to the center point of the horizontal anchor frame along the x-axis. This represents the offset of the center point of the truth anchor frame relative to the center point of the horizontal anchor frame along the y-axis. This represents the offset of the width of the true anchor frame relative to the width of the horizontal anchor frame. This represents the offset of the true anchor frame height from the horizontal anchor frame height.

[0037] N reg This represents the number of positive anchor box samples;

[0038] N cls This represents the total number of positive anchor box samples, simple negative anchor box samples, and difficult negative anchor box samples.

[0039] The loss function for the classification layer that detects crater locations in image sub-blocks;

[0040] The loss function for the crater detection location regression layer in image sub-blocks;

[0041] p i The confidence level of the candidate anchor box predicted as a prospect;

[0042] The confidence level of the candidate anchor box corresponding to the ground truth anchor box;

[0043] λ is the proportionality coefficient, λ = 5;

[0044] in,

[0045]

[0046] x * y * These represent the coordinates of the center point of the manually calibrated true value anchor frame along the x-axis and y-axis, respectively. * h * These are the width and height of the anchor frame for manually calibrated true values, respectively.

[0047] 4.3 Training the detection model

[0048] 4.3.1 Project the candidate anchor boxes obtained in step 4.2.6 onto each layer of the feature map output by the path enhancement network. Divide each candidate anchor box region into k equal parts in the vertical and horizontal directions to obtain k×k grids with the same aspect ratio as the candidate anchor box. Cut out the three-dimensional features at the corresponding positions of the feature map projection and stretch them into a one-dimensional vector to obtain the region alignment features.

[0049] 4.3.2 Input the region alignment features into the detection head network for training and optimization, perform classification and position regression on the region alignment features, and obtain the trained model;

[0050] The loss function L for training and optimizing the region alignment features by inputting them into the detection head network is:

[0051]

[0052] Where, N cls The total number of candidate anchor boxes;

[0053] L cls Let be the classification loss function for the candidate anchor boxes;

[0054] L reg The regression loss function for the candidate anchor boxes;

[0055] N reg This represents the number of positive anchor box samples.

[0056] Further, in step 4.2.5, the selection of positive anchor box samples, simple negative anchor box samples, and difficult negative anchor box samples based on IOU values ​​specifically involves:

[0057] (1) Select candidate anchor boxes whose IOU value with any manually calibrated true anchor box is higher than 0.7 as positive anchor box samples, and denot the number of positive anchor box samples as N. reg ;

[0058] (2) Among the candidate anchor frames whose IOU value with any manually calibrated truth anchor frame is less than 0.3, randomly select N. regN is used as a simple negative anchor box sample; if the number of candidate anchor boxes whose IOU value with any manually calibrated ground truth anchor box is less than 0.3 is less than N. reg If so, then select all candidate anchor boxes that meet this condition;

[0059] (3) If the IOU value of the candidate anchor box and any manually calibrated true anchor box is between 0.3 and 0.7, then divide 0.3 to 0.7 into 4 equal sub-intervals, and randomly select N in each sub-interval. reg N candidate anchor boxes are selected as hard negative anchor box samples; if the number of samples in a certain sub-interval is less than N... reg If so, all candidate anchor boxes within that sub-interval are selected as difficult negative anchor box samples.

[0060] Furthermore, step five specifically includes:

[0061] 5.1 Input the test set and the training model obtained in step 4 into the object detection network, and use the training model to extract multi-scale feature maps according to the method in step 4.1;

[0062] 5.2. Based on the multi-scale feature map extracted in step 5.1, use the training model to calculate the predicted candidate anchor boxes in the region candidate network according to the methods in steps 4.2.1-4.2.3, and obtain n candidate anchor boxes according to the method in step 4.2.4, where n is an integer and 1000≤n≤5000;

[0063] 5.3 Obtain region alignment features according to the method in step 4.3.1;

[0064] 5.4 Input the alignment features of each region into the detection head network, and use the trained model to output the confidence score of the candidate anchor box as a crater and the position coordinates of the candidate anchor box as a crater, so as to obtain the probability and position of crater detection in the image sub-block.

[0065] Furthermore, in step six, the crater detection results in the image sub-blocks obtained in step five are specifically as follows:

[0066] For detection results where there is no overlap in the image sub-blocks, the results are directly converted to the original image based on the offset of the starting point of the segmentation in the original image.

[0067] For overlapping detection results in image sub-blocks, the probability of crater target detection is the product of the crater detection probabilities of each image sub-block, and the target region of the crater is the minimum outer moment of the overlapping regions of crater detection positions in each image sub-block.

[0068] Furthermore, in step one, the original image is a series of scene images collected by a drone carrying a camera flying at high altitude along a planned route, and a high-resolution digital orthophoto image generated by oblique photogrammetry, or a high-resolution image directly taken by the drone at high altitude, or an image collected by a remote sensing satellite.

[0069] Furthermore, step three specifically includes:

[0070] From the image sub-blocks in step two, randomly select image sub-blocks with a proportion of r and denote them as the training set, where 0.1≤r≤0.3; manually label each image sub-block in the training set as a positive sample instance and save it as an XML or TXT format annotation file to obtain the manually labeled ground truth anchor boxes; each annotation file is named according to the coordinate position of the segmentation starting point in the original image;

[0071] In this invention, the open-source labelImg tool can be used to manually annotate image sub-blocks, and the coordinate values ​​of the top left and bottom right vertices of the image sub-blocks can be used to represent the annotation file.

[0072] Furthermore, in steps two and five, the resolution of the image sub-block is 1024×1024, and the step size s is 824.

[0073] Furthermore, in step 4.2.1, the horizontal anchor frame has three different sizes {0.5, 1, 2} and three aspect ratios {1:2, 1:1, 2:1}.

[0074] Further, in step 4.3.1, each candidate anchor frame region is divided into k equal parts in the longitudinal and transverse directions, where k = 7.

[0075] Compared with the prior art, the present invention has the following beneficial technical effects:

[0076] 1. The method for detecting weak crater targets in high-resolution aerial images provided by this invention is based on deep learning. It eliminates the need for manually designing features based on low- and mid-level appearance attributes such as corners, boundaries, shapes, and textures of craters, thus overcoming the inadequacy of manually designed features in complex and ever-changing scenarios. It utilizes a data-driven deep convolutional neural network to deeply mine the high-level essential features of targets from sample data with diverse shapes, making it more applicable and better able to handle the complex and ever-changing crater target shapes in real low signal-to-noise ratio scenarios.

[0077] 2. The method for detecting weak crater targets in high-resolution aerial images provided by this invention uses a top-down feature extraction framework that integrates residual networks and feature pyramid networks. This framework can generate multi-scale features with strong semantic information at each layer, overcoming the shortcomings of insufficient semantic information in the middle and low layers and low resolution in the high layers. At the same time, it integrates a bottom-up path enhancement network, which shortens the information flow path between high and low layers and obtains high-level semantic features that strongly respond to the target boundary. This overcomes the shortcomings of inaccurate localization of high-level semantic features and is conducive to the robust and accurate extraction of features of weak crater targets.

[0078] 3. In the method for detecting weak crater targets in high-resolution aerial images provided by the present invention, the region candidate network with IOU equalization generates anchor box samples that are equalized for positive, negative and difficult and easy samples, which overcomes the shortcomings of model optimization that tends to favor a large number of negative samples and easy samples; the equalization of difficult and easy samples makes the loss function optimized towards the direction of difficult samples with stronger discrimination performance, thereby improving the robustness of the training parameter model.

[0079] 4. The method for detecting small crater targets in high-resolution aerial images provided by this invention uses oblique photography to generate orthophoto images with high spatial resolution, which is beneficial for detecting small craters with a scale of only 3 to 4 centimeters. At the same time, it is not affected by the distortion of traditional images, which is beneficial for applications such as accurate crater density distribution and equipment damage effect assessment based on crater detection results. Attached Figure Description

[0080] Figure 1 A flowchart of the method for detecting weak crater targets in high-resolution aerial images provided by the present invention;

[0081] Figure 2 This is an example diagram illustrating the use of labelImg software to mark crater targets in step three of this embodiment of the invention.

[0082] Figure 3 This is an example diagram of the XML file saved by the labelImg software in step three of this embodiment of the invention;

[0083] Figure 4 This is a diagram illustrating the overall framework of the algorithms used in steps four and five of this invention.

[0084] Figure 5 This is a framework diagram of multi-scale feature extraction in step 4.1 of this embodiment of the invention;

[0085] Figure 6 This is a schematic diagram illustrating the calculation of the Intersection over Union (IOU) value in step 4.2.5 of this embodiment of the invention;

[0086] Figure 7This is a diagram showing the detection results of weak crater targets in an image sub-block obtained in an embodiment of the present invention;

[0087] Figure 8 This is a map showing the location of weak crater targets in different image sub-blocks obtained in an embodiment of the present invention. Detailed Implementation

[0088] The present invention provides a method for detecting small crater targets in high-resolution aerial imagery, with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the invention and are not intended to limit the scope of protection of the invention.

[0089] A method for detecting weak crater targets in high-resolution aerial imagery, such as Figure 1 As shown, it includes the following steps:

[0090] Step 1: Acquire high-resolution raw images of the explosion scene. High resolution refers to a spatial resolution of pixels better than 0.5 cm. The raw images are high-resolution digital orthophotos generated by a series of scene images collected by a drone carrying a camera flying at high altitude along a planned flight path and then processed by the oblique photogrammetry software Context Capture, or high-resolution images directly taken by the drone at high altitude, or images collected by remote sensing satellites.

[0091] Step 2: Divide the original image obtained in Step 1 into segments with a fixed step size of 824 to obtain image sub-blocks with a resolution of 1024×1024. Name the image sub-blocks according to the coordinates of the segmentation starting point in the original image. The step size only needs to ensure that there is a certain overlap between adjacent image sub-blocks.

[0092] Step 3: Randomly select a sub-block of image size r from one of the original images obtained in Step 2, and denote it as the training set (r = 0.3). The remaining sub-blocks are designated as the test set. That is, select sub-blocks of image size 0.3 times the total number of sub-blocks from all image sub-blocks. The proportion of the training set is determined based on the network structure, crater target attributes, image quality, etc. Manually label the crater targets in each sub-block of the training set as positive sample instances to obtain manually labeled ground truth anchor boxes.

[0093] like Figure 2As shown, the open-source labelImg tool is used to manually label each image sub-patch in the training set as a positive sample instance, resulting in manually labeled ground truth anchor boxes. These anchor boxes record the category attribute and precise location of the image sub-patch. Each image sub-patch in the training set corresponds to a labeling file. Each labeling file is represented using the coordinates of the top-left and bottom-right vertices of a horizontal bounding box. The labeling files are saved in XML or TXT format, such as... Figure 3 The image shows an example of a tag file saved in XML format.

[0094] Step 4, as follows Figure 4 As shown, each image sub-block in the training set and its corresponding annotation file are input into the object detection network. End-to-end offline training optimization is performed by extracting multi-scale features, generating IOU-equalized candidate anchor boxes, obtaining region alignment features, and classifying and regressing region alignment features to obtain the training model for crater object detection. The specific steps are as follows:

[0095] 4.1, such as Figure 5 As shown, multi-scale feature extraction is performed.

[0096] 4.1.1. Extracting multi-stage features using residual networks: ResNet50 residual network is used to extract 5 stages S for each image sub-block in the training set. j The features are defined by j = 2, 3, 4, 5, 6. Starting from stage S3, the length and width scales of the features acquired in each stage are reduced to half of those in the previous stage, while the number of channels remains unchanged at 256. Three-dimensional features are typically represented by length, width, and number of channels.

[0097] ResNet50 consists of 50 layers of convolutional neural network, divided into 5 stages. Each stage consists of a series of bottleneck structures, which are composed of 1×1, 3×3, and 1×1 convolutional layers and jumper connections.

[0098] 4.1.2 Extracting features from the feature pyramid network from top to bottom, F is the feature of each layer of the feature pyramid network. j The residual network characteristics S of the corresponding stage j Resampled by a 1×1 convolutional network and its upper-layer features F j+1 It is constructed by upsampling and fusing with nearest neighbor interpolation at twice the speed. By combining lateral connections and top-down approach, the high-level semantic information of the feature pyramid network is gradually propagated to the low-level layers, and multi-scale feature pyramid network feature maps are extracted.

[0099] 4.1.3 Extracting path enhancement network features from bottom to top, each layer of the path enhancement network has features P j From its next layer feature P j-1 Feature pyramid layer F, which is downsampled by 2x and laterally connected by a 3×3 spatial convolutional networkj By integrating the components, the path length of information flow from the lower-level feature P2 (which transmits the precise spatial location response) to the higher-level feature P6 is shortened, and semantic features with precise location responses are extracted.

[0100] In the above steps, the scales of the feature maps obtained in stages S2 (F2, P2) to S6 (F6, P6) are 256*256*256, 128*128*256, 64*64*256, 32*32*256, and 16*16*256, respectively.

[0101] 4.2 Generation of Candidate Anchor Box Samples for IOU Equalization

[0102] 4.2.1 At each location of each feature layer of the path enhancement network obtained in step 4.1.3, construct nine nodes with three different scales {0.5, 1, 2} and three aspect ratios {1:2, 1:1, 2:1}, composed of a 4-dimensional array α = (x α ,y α ,w α ,h α The horizontal anchor frame is formed by x, where x α Let x be the coordinate of the center point of the horizontal anchor frame along the x-axis, and y be the coordinate of the center point of the horizontal anchor frame along the x-axis. α Let w be the coordinate of the center point of the horizontal anchor frame along the y-axis. α h is the width of the horizontal anchor frame. α The height of the horizontal anchor frame.

[0103] 4.2.2 Construct a region candidate network consisting of a 3×3 spatial convolutional sliding window and two 1×1 convolutional layers, one for classification and one for regression. The classification and regression convolutional layers are used to classify and regress the location of the horizontal anchor box. The regression layer generates the offset Δ of the predicted candidate anchor box relative to the horizontal anchor box.

[0104] Δ=(Δ x ,Δ y ,Δ w ,Δ h )

[0105] Where, Δ x Δ represents the offset of the center points of the two points along the x-axis. y Δ represents the offset of the center point of both points along the y-axis. w Δ is the offset between the widths of the two. h This represents the offset between their heights.

[0106] 4.2.3 The predicted candidate anchor box β = (x, y, w, h) is calculated using the following formula:

[0107]

[0108] Where e is a constant, x is the coordinate of the center point of the predicted candidate anchor box in the x-axis direction, y is the coordinate of the center point of the predicted candidate anchor box in the y-axis direction, w is the width of the predicted candidate anchor box, and h is the height of the predicted candidate anchor box.

[0109] 4.2.4 Calculate the confidence score p for predicting candidate anchor boxes as foreground. i The first m predicted candidate anchor boxes are used as candidate anchor box samples, where m is an integer and 1000≤m≤5000. In this embodiment, m is 2000.

[0110] 4.2.5 Calculate the Interchange of Value (IOU) between each candidate anchor box selected in the regional candidate network and the manually labeled ground truth anchor box, and select positive anchor box samples, simple negative anchor box samples, and difficult negative anchor box samples based on the IOU value. Specifically:

[0111] (1) Select candidate anchor box samples whose IOU value with any true anchor box is higher than 0.7 as positive anchor box samples, and denot the number of positive anchor box samples as N. reg ;

[0112] (2) From the candidate anchor box samples whose IOU value with any true anchor box is less than 0.3, randomly select N reg N is considered as a simple negative anchor box sample; if the number of candidate anchor box samples whose IOU value with any manually labeled ground truth anchor box is less than 0.3 is less than N. reg If so, then select all candidate anchor boxes that meet this condition;

[0113] (3) If the IOU value between the candidate anchor box sample and any true anchor box is between 0.3 and 0.7, then divide 0.3 to 0.7 into 4 equal sub-intervals, and randomly select N in each sub-interval. reg N candidate anchor box samples are used as difficult negative anchor box samples; if the number of samples in a certain sub-interval is less than N, reg If so, all candidate anchor box samples within that interval are selected as difficult negative anchor box samples.

[0114] Based on this, the selected negative anchor box samples include simple samples that are easily classified and distributed in the background region far from the target, and false positive samples that are difficult to classify and concentrated in the region around the target. The ratio of positive anchor box samples to negative anchor box samples is equal to or approximately 1:5, ensuring balanced sampling of positive and negative anchor box samples. At the same time, the ratio of simple negative anchor box samples to difficult negative anchor box samples is equal to or approximately 1:4, ensuring the decisive role of difficult negative anchor box samples in loss function optimization.

[0115] like Figure 6As shown, the Intersection over Union (IOU) ratio of two horizontal anchor frames (A, B) can be calculated using the following formula:

[0116]

[0117] 4.2.6. Train and optimize the region candidate network based on the selected positive and negative anchor box samples. When the convergence condition is met, candidate anchor boxes are generated.

[0118] The loss function L1 for training and optimizing the region candidate network is:

[0119]

[0120] Among them, t i t represents the offset of the candidate anchor frame relative to the horizontal anchor frame. i =(t x ,t y ,t w ,t h ), t x t represents the offset of the center point of the candidate anchor frame relative to the center point of the horizontal anchor frame along the x-axis. y t represents the offset of the candidate anchor frame center point relative to the horizontal anchor frame center point along the y-axis. w t represents the offset of the candidate anchor frame width relative to the horizontal anchor frame width. h The offset of the candidate anchor frame height relative to the horizontal anchor frame height;

[0121] This represents the offset of the truth anchor frame relative to the horizontal anchor frame. This represents the offset of the center point of the truth anchor frame relative to the center point of the horizontal anchor frame along the x-axis. This represents the offset of the center point of the truth anchor frame relative to the center point of the horizontal anchor frame along the y-axis. This represents the offset of the width of the true anchor frame relative to the width of the horizontal anchor frame. This represents the offset of the true anchor frame height from the horizontal anchor frame height.

[0122] N reg This represents the number of positive anchor box samples;

[0123] N cls This represents the total number of positive anchor box samples, simple negative anchor box samples, and difficult negative anchor box samples.

[0124] The loss function for the anchor box classification layer is defined as foreground and background.

[0125] The loss function for the anchor box regression layer;

[0126] p i The confidence level of the candidate anchor box predicted as a prospect;

[0127] The confidence level of the candidate anchor box corresponding to the ground truth anchor box;

[0128] λ is the proportionality coefficient, λ = 5;

[0129] in:

[0130]

[0131] x * y * These are the coordinates of the center point of the manually calibrated true value anchor frame in the x-axis and y-axis directions, respectively. * h * These are the width and height of the anchor frame for manually calibrated true values, respectively.

[0132] 4.3 Training the detection model

[0133] 4.3.1 Project the candidate anchor boxes obtained in step 4.2.6 onto the feature maps of each layer of the path enhancement network. Each candidate anchor box region is divided into 7 equal parts in length and width to obtain k×k grids with the same aspect ratio as the candidate anchor box. Cut out 7*7*256 three-dimensional features at the corresponding positions of the feature map projection and stretch them into one-dimensional vectors to obtain region alignment features.

[0134] 4.3.2 Input the region alignment features into the detection head network consisting of two 1024×1024 fully connected layers for training and optimization, perform classification and position regression on the region alignment features, and obtain the training parameter model;

[0135] The loss function L for training and optimizing the detection head network by inputting region alignment features is:

[0136]

[0137] Where, N cls The total number of candidate anchor boxes;

[0138] L cls Let be the classification loss function for the candidate anchor boxes;

[0139] L reg The regression loss function for the candidate anchor boxes;

[0140] N reg This represents the number of positive anchor box samples.

[0141] Step 5: Based on the trained model from Step 4, use the object detection network to perform online inference on the test set to obtain the probability and location of crater detection in image sub-blocks. The specific steps are as follows:

[0142] 5.1 Input the test set and the training parameter model obtained in step four into the object detection network. Using the weight parameters of the training parameter model obtained in step four, extract feature maps of five scales: 256*256*256, 128*128*256, 64*64*256, 32*32*256 and 16*16*256, respectively, according to the method in step 4.1.

[0143] 5.2. Based on the multi-scale feature map extracted in step 5.1, using the training model obtained in step 4, calculate the predicted candidate anchor boxes in the region candidate network according to the methods in steps 4.2.1-4.2.3, and obtain n candidate anchor box samples according to the method in step 4.2.4, where n is an integer and 1000≤n≤5000; in this embodiment, n is 2000.

[0144] 5.3 Obtain region alignment features according to the method in step 4.3.1;

[0145] 5.4 Input the alignment features of each region into the detection head network, use the training model obtained in step four to output the confidence of candidate anchor boxes as craters and the position coordinates of candidate anchor boxes as craters, and obtain the probability and location of crater detection in the test set.

[0146] like Figure 7 The image shown is a result of detecting small crater targets in one of the image sub-blocks. Figure 7 Displays the detection location and category score of the crater. For example... Figure 8 The image shown is a map illustrating the location of weak crater targets in another different image sub-block. Figure 8 It only displays the detection location of the craters.

[0147] Step 6: Merge the crater detection results from the test set obtained in Step 5 to obtain the detection results of all craters in the original image.

[0148] Since the original image is segmented with a fixed step size s, and the image resolution is H×W, where s is less than H and W, there may be overlapping detection results in the image sub-blocks. For non-overlapping detection results in the image sub-blocks, they are directly converted to the original image based on the offset of the segmentation starting point's coordinate position in the original image. For overlapping detection results in the image sub-blocks, the probability of crater target detection is the product of the crater detection probabilities of each image sub-block, and the target region of the crater is the minimum bounding rectangle of the overlapping regions of the crater detection positions in each image sub-block.

[0149] It should be noted that in this invention, the training set and the test set can come from the same original image, with a small portion of the image sub-blocks selected as the training set to train the model, and the remaining image sub-blocks used as the test set to detect craters based on the model; or they can come from different explosion scenarios, using multiple original images, such as using image sub-blocks from the original images of previous scenarios as the training set, and image sub-blocks from the original images of the current scenario as the test set.

Claims

1. A method for detecting weak crater targets in high-resolution aerial images, characterized in that, Includes the following steps: Step 1: Obtain a high-resolution original image of the explosion scene. High resolution means that the spatial resolution of pixels is better than 0.5 cm. Step 2: Divide the original image obtained in Step 1 into segments with a fixed step size s to obtain image sub-blocks with a resolution of H×W; Step 3: Randomly select image sub-blocks with a proportion of r from the image sub-blocks obtained in Step 2 and record them as the training set, where 0.1≤r≤0.

3. The remaining image sub-blocks are recorded as the test set. Manually label the crater targets in each image sub-block in the training set as positive sample instances to obtain manually labeled ground truth anchor boxes. Step 4: Input each image sub-block and its corresponding annotation file from the training set into the object detection network. The training model for crater object detection is obtained by extracting multi-scale features, generating IOU-equalized candidate anchor boxes, acquiring region alignment features, and classifying and regressing region alignment features. Specifically: 4.1 Multi-scale feature extraction 4.1.

1. Use the ResNet50 residual network to extract five stages of features S for each image sub-block in the training set. j Where j = 2, 3, 4, 5, 6; starting from stage S3, the length and width scales of the features acquired in each stage are reduced to half of the previous stage, while the number of channels remains unchanged; 4.1.2 Extracting features from the feature pyramid network from top to bottom, wherein each layer of the feature pyramid network has features F j The residual network characteristics S of the corresponding stage j After resampling by the convolutional network and its upper-layer features F j+1 It is formed by upsampling and fusion at twice the nearest neighbor interpolation; 4.1.3 Extracting path enhancement network features from bottom to top, wherein each layer of the path enhancement network has features P j From its next layer feature P j-1 Feature pyramid layer F, which is downsampled by 2x and laterally connected by a spatial convolutional network j Fusion composition; 4.2 Generation of Candidate Anchor Box Samples for IOU Equalization 4.2.1 At each location of each feature layer of the path enhancement network obtained in step 4.1.3, construct multiple arrays with different scales and aspect ratios, consisting of a 4-dimensional array α = (x... α ,y α ,w α ,h α A horizontal anchor frame is formed by x, where x α Let x be the coordinate of the center point of the horizontal anchor frame along the x-axis, and y be the coordinate of the center point of the horizontal anchor frame along the x-axis. α Let w be the coordinate of the center point of the horizontal anchor frame along the y-axis. α h is the width of the horizontal anchor frame. α The height of the horizontal anchor frame; wherein the horizontal anchor frame has 3 different sizes {0.5, 1, 2} and 3 aspect ratios {1:2, 1:1, 2:1}; 4.2.2 Construct a region candidate network consisting of a spatial convolutional sliding window and two convolutional layers: a classification convolutional layer and a regression convolutional layer. The classification and regression convolutional layers are used to classify and regress the location of the horizontal anchor box. The regression layer generates the offset Δ of the predicted candidate anchor box relative to the horizontal anchor box. Δ=(Δ x ,D y ,D w ,D h ) Where, Δ x Δ represents the offset of the center points of the two points along the x-axis. y Δ represents the offset of the center point of both points along the y-axis. w The offset between the two widths, Δ h This represents the height offset between the two. 4.2.3 The predicted candidate anchor box β = (x, y, w, h) is calculated using the following formula: Where e is a constant, x is the coordinate of the center point of the predicted candidate anchor box in the x-axis direction, y is the coordinate of the center point of the predicted candidate anchor box in the y-axis direction, w is the width of the predicted candidate anchor box, and h is the height of the predicted candidate anchor box. 4.2.4 Calculate the confidence score p for predicting candidate anchor boxes as foreground. i Select p i The top m predicted candidate anchor boxes are used as candidate anchor box samples, where m is an integer and 1000≤m≤5000; 4.2.5 Calculate the Interchange of Value (IOU) between each candidate anchor box sample selected in the region candidate network and the ground truth anchor box, and select positive anchor box samples, simple negative anchor box samples, and difficult negative anchor box samples based on the IOU value; specifically: (1) Select candidate anchor boxes whose IOU value with any true anchor box is higher than 0.7 as positive anchor box samples, and denot the number of positive anchor box samples as N. reg ; (2) Among the candidate anchor boxes whose IOU value with any truth anchor box is less than 0.3, randomly select N. reg N is used as a simple negative anchor box sample; if the number of candidate anchor boxes whose IOU value is less than 0.3 with any ground truth anchor box is less than N. reg If so, then select all candidate anchor boxes; (3) If the IOU value of the candidate anchor box and any true anchor box is between 0.3 and 0.7, then divide 0.3 to 0.7 into 4 equal sub-intervals, and randomly select N in each sub-interval. reg N candidate anchor boxes are used as hard negative anchor box samples; if the number of samples in a certain sub-interval is less than N reg If so, all candidate anchor boxes within that sub-interval are selected as difficult negative anchor box samples. 4.2.

6. The region candidate network is trained and optimized based on the selected positive anchor box samples, simple negative anchor box samples and difficult negative anchor box samples. When the convergence condition is met, the final candidate anchor boxes are generated. 4.3 Obtaining the training model 4.3.1 Project the candidate anchor boxes obtained in step 4.2.6 onto each layer of the feature map output by the path enhancement network. Each candidate anchor box region is divided into k equal parts in the vertical and horizontal directions. Cut out the three-dimensional features at the corresponding positions of the feature map projection and stretch them into a one-dimensional vector to obtain the region alignment features. 4.3.2 Input the region alignment features into the detection head network for training and optimization, perform classification and position regression on the region alignment features, and obtain the trained model; Step 5: Based on the trained model in Step 4, use the object detection network to perform online inference on the test set to obtain the probability and location of crater detection in the test set; Step 6: Combine the probability and location of crater detections obtained in Step 5 to obtain the crater detection results.

2. The method for detecting weak crater targets in high-resolution aerial images according to claim 1, characterized in that, In step 4.2.6, the loss function L1 used to train and optimize the region candidate network is: Among them, t i t represents the offset of the candidate anchor frame relative to the horizontal anchor frame. i =(t x ,t y ,t w ,t h ), t x t represents the offset of the center point of the candidate anchor frame relative to the center point of the horizontal anchor frame along the x-axis. y t represents the offset of the candidate anchor frame center point relative to the horizontal anchor frame center point along the y-axis. w t represents the offset of the candidate anchor frame width relative to the horizontal anchor frame width. h This represents the offset of the candidate anchor frame's height relative to the horizontal anchor frame's height. This represents the offset of the truth anchor frame relative to the horizontal anchor frame. This represents the offset of the center point of the truth anchor frame relative to the center point of the horizontal anchor frame along the x-axis. This represents the offset of the center point of the truth anchor frame relative to the center point of the horizontal anchor frame along the y-axis. This represents the offset of the width of the true anchor frame relative to the width of the horizontal anchor frame. This represents the offset of the true anchor frame height from the horizontal anchor frame height. N reg This represents the number of positive anchor box samples; N cls This represents the total number of positive anchor box samples, simple negative anchor box samples, and difficult negative anchor box samples. The loss function for the anchor box classification layer; The loss function for the anchor box regression layer; p i The confidence level of the candidate anchor box predicted as a prospect; The confidence level of the candidate anchor box corresponding to the ground truth anchor box; λ is the proportionality coefficient, λ = 5; in, x * y * These are the coordinates of the center point of the manually calibrated true value anchor frame in the x-axis and y-axis directions, respectively. * h * These are the width and height of the anchor frame for manually calibrated true values, respectively. In step 4.3.2, the loss function L for inputting the region alignment features into the detection head network for training and optimization is: Among them, L cls Let be the classification loss function for the candidate anchor boxes; L reg is the regression loss function for candidate anchor boxes.

3. The method for detecting weak crater targets in high-resolution aerial images according to claim 2, characterized in that, Step five specifically involves: 5.1 Input the test set and the training model obtained in step 4 into the object detection network, and use the training model to extract multi-scale feature maps according to the method in step 4.1; 5.

2. Based on the multi-scale feature map extracted in step 5.1, use the training model to calculate the predicted candidate anchor boxes in the region candidate network according to the methods in steps 4.2.1-4.2.3, and obtain n candidate anchor boxes according to the method in step 4.2.4, where n is an integer and 1000≤n≤5000; 5.3 Obtain region alignment features according to the method in step 4.3.1; 5.4 Input the alignment features of each region into the detection head network, and use the trained model to output the confidence score of the candidate anchor box as a crater and the position coordinates of the candidate anchor box as a crater, so as to obtain the probability and position of crater detection in the image sub-block.

4. The method for detecting weak crater targets in high-resolution aerial images according to claim 3, characterized in that, In step six, the crater detection results in the image sub-blocks obtained in step five are specifically as follows: For detection results where there is no overlap in the image sub-blocks, the results are directly converted to the original image based on the offset of the starting point of the segmentation in the original image. For overlapping detection results in image sub-blocks, the probability of crater target detection is the product of the crater detection probabilities of each image sub-block, and the target region of the crater is the minimum outer moment of the overlapping regions of crater detection positions in each image sub-block.

5. The method for detecting weak crater targets in high-resolution aerial images according to claim 4, characterized in that, In step one, the original images are a series of scene images collected by a drone carrying a camera flying at high altitude along a planned route, and are generated as high-resolution digital orthophotos through oblique photogrammetry, or high-resolution images directly taken by the drone at high altitude, or images collected by remote sensing satellites.

6. The method for detecting weak crater targets in high-resolution aerial images according to claim 5, characterized in that, Step three specifically involves: From the image sub-blocks obtained in step two, randomly select image sub-blocks with a proportion of r as the training set, where r = 0.3, and the remaining image sub-blocks as the test set; manually label the crater targets in each image sub-block in the training set as positive sample instances, and save them as annotation files in XML or TXT format to obtain manually labeled ground truth anchor boxes; each annotation file is named according to the coordinate position of the segmentation starting point in the original image.

7. The method for detecting weak crater targets in high-resolution aerial images according to claim 6, characterized in that, In steps two and five, the resolution of the image sub-block is 1024×1024, and the step size s is 824.

8. The method for detecting weak crater targets in high-resolution aerial images according to claim 7, characterized in that, In step 4.3.1, each candidate anchor frame region is divided into k equal parts in the longitudinal and transverse directions, where k = 7.

Images