Intelligent falling pit detection method based on YOLOv5

Abstract

The invention discloses a YOLOv5-based intelligent falling pit detection method. The method comprises the steps of constructing a training set and a verification set based on a public data set; the method comprises the following steps: designing a YOLOv5 falling pit detection model which comprises an input end preprocessing unit, a backbone network unit, a neck unit and a target detection head unit which are connected in sequence; designing a loss function according to the model structure; training a YOLOv5 falling pit detection model by utilizing the training set and combining a loss function, optimizing model parameters by continuously minimizing the loss function on the training data set to obtain a series of trained YOLOv5 falling pit detection models, and selecting a model with the best effect on the verification set as a final YOLOv5 detection model; the final YOLOv5 detection model is deployed on a satellite, and a YOLOv5 troillage detection reasoning module is formed; an original image needing to be processed is generated on a satellite and input into a YOLOv5 trocar detection reasoning module, and the position coordinates and confidence of a detection frame where a detected trocar is located are generated and used for assisting in subsequent landing point recognition. The method can assist in improving the landing quality of deep space exploration.

Description

Technical Field

[0001] The present invention relates to an intelligent crater detection method based on YOLOv5, which is applicable to on-orbit crater detection of lunar surface images, especially large-scale remote sensing image processing on orbit, and belongs to the field of intelligent data processing of payloads. Background Art

[0002] During the process of the detector landing and selecting a suitable landing site, due to the topographic features of the crater, it becomes an ideal object for landing site selection. Therefore, the system needs to detect the crater. A flat terrain can reduce the risk of the lander tipping over or sliding when it touches the lunar surface, ensuring a safer and more stable landing process. In addition, the open view of the crater also helps the lander navigate and communicate on the lunar surface, enhancing the success rate of the mission. Therefore, in lunar exploration missions, it is crucial to detect and analyze craters and find safe landing sites.

[0003] In the complex environment of the moon, the image data has low resolution, complex noise, and high distortion. Existing intelligent crater detection methods have the following problems: The detection rate of deep learning-based detection methods has been improved compared with traditional methods. However, due to the weak and complex noise environment of lunar surface images and the small number of publicly available crater detection datasets, existing methods still have detection errors. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: Overcoming the deficiencies of the prior art, providing an intelligent crater detection method based on YOLOv5, which has the advantages of high recognition rate and high flexibility compared with traditional crater detection methods, is suitable for on-orbit deployment to assist the lunar surface lander to find a suitable landing site, and improve the landing quality.

[0005] The technical solution of the present invention is: An intelligent crater detection method based on YOLOv5, including:

[0006] Constructing a training set and a validation set based on a public dataset;

[0007] Designing a YOLOv5 crater detection model;

[0008] Designing a loss function according to the model structure;

[0009] Training the YOLOv5 crater detection model using the training set and in combination with the loss function, optimizing the model parameters by continuously minimizing the loss function on the training dataset, obtaining a series of trained YOLOv5 crater detection models, and selecting the model with the best performance on the validation set as the final YOLOv5 detection model;

[0010] Deploying the final YOLOv5 detection model to the satellite to form a YOLOv5 crater detection inference module;

[0011] Generate the original image to be processed on the satellite, input it into the YOLOv5 crater detection inference module, and generate the position coordinates and confidence of the detected crater in the detection box, which are used to assist in subsequent landing point recognition.

[0012] Preferably, when constructing the training set and validation set based on the public dataset, use the publicly available YOLOLens lunar crater dataset, select the images with the most craters to construct the crater detection dataset, and adjust the size of each image in the dataset to be the same;

[0013] Divide the constructed crater detection dataset into a training set and a validation set according to the ratio of 8:2. The training dataset is used to optimize the model parameters, and the validation set is used to select the optimal model;

[0014] When constructing the crater detection dataset, label the craters in the images in the YOLO format.

[0015] Preferably, the YOLOv5 crater detection model includes an input preprocessing unit, a backbone network unit, a neck unit, and a target detection head unit, and the units are connected in sequence. Specifically:

[0016] The preprocessing unit outputs the preprocessed image to the backbone network unit. The backbone network unit processes it and outputs three-scale backbone feature images to the neck unit. The neck unit processes it and outputs three different-scale neck feature maps to the target detection head unit. The target detection head unit processes it and outputs three different-scale detection head feature maps.

[0017] Preferably, the input preprocessing unit in the YOLOv5 crater detection model performs image preprocessing on the given image. Specifically:

[0018] Perform edge detection on the given lunar surface image to obtain the edge detection result, and then perform weighted accumulation on the given lunar surface image and the edge detection result to obtain the initial input image;

[0019] Perform further processing on the initial input image, including resizing the image to 640*640, normalizing, format conversion, and Mosaic image enhancement to obtain the preprocessed image.

[0020] Preferably, the backbone network in the YOLOv5 crater detection model uses the CSPDarknet53 network to extract features. Specifically:

[0021] For the preprocessed image, slice it through the Focus module, output an image with reduced height and width, and integrate the height and width of the sliced image through Concat;

[0022] Use the Conv convolution module to extract features from the integrated image and output the first feature image; then, after performing three sets of BottleneckCSP1 and Conv convolution operations on the extracted first feature image, obtain the second feature image of the image; use the SSP module to perform max pooling operations on the second feature image respectively to further extract features from multiple aspects, and aggregate the pooled feature maps through Concat to obtain the third feature image;

[0023] Record the first feature image, the second feature image, and the third feature image as backbone feature maps of three scales.

[0024] Preferably, the neck in the YOLOv5 crater detection model uses the BottleneckCSP2 module to reduce the number of model parameters. Specifically:

[0025] Perform upsampling and downsampling operations on the backbone feature maps of the three scales output by the backbone network in sequence to obtain neck feature maps of three different scales;

[0026] Among them, the upsampling process is completed by 2 groups of BottleneckCSP2, a Conv convolution with a size of 1 and a stride of 1, Upsample interpolation, and Concat connection.

[0027] Preferably, the detection head in the YOLOv5 crater detection model uses multi-scale feature maps for detection, using large images to detect small targets and small images to detect large targets. Specifically:

[0028] Perform Conv2d convolution operations on the three neck feature maps of different scales obtained by the neck to finally obtain detection head feature maps of three different scales.

[0029] Preferably, based on the detection head feature maps of the three different scales output by the detection head, after performing convolution network operations on them, obtain outputs of three scales, and each scale output contains the bounding box position, the bounding box category, and the bounding box confidence.

[0030] Preferably, the loss function includes three parts:

[0031] Classification loss, calculated according to the output bounding box category, used to measure the accuracy of target category prediction, and use the binary cross-entropy loss function;

[0032] Localization loss, calculate the localization loss according to the bounding box position, used to measure the accuracy of the predicted target bounding box, and use the intersection over union loss function;

[0033] Confidence loss, calculated according to the bounding box confidence, used to measure the accuracy of bounding box confidence prediction, and use the binary cross-entropy loss function.

[0034] Preferably, the final loss function is the sum of the classification loss, the localization loss, and the confidence loss.

[0035] The present invention has the following advantages compared with the prior art:

[0036] (1) The present invention adopts the method of ground training combined with on-orbit inference to realize on-orbit crater detection of remote sensing images. The detection results are transmitted to the lunar surface lander to assist the lunar surface lander in finding a suitable landing point. It has the advantages of high recognition rate and high flexibility, and is suitable for on-orbit deployment to assist the lunar surface lander in finding a suitable landing point and improving the landing quality.

[0037] (2) The present invention proposes to first detect craters using the edge detection method and superimpose the crater results and the original image, which improves the detection effect. This patent proposes to introduce other knowledge outside the original image, that is, the edge detection result matrix, to improve the detection results. The edge detection result matrix can be obtained through any existing edge detection model. The introduction method is to weighted sum the original image and the edge detection result matrix as the input image, and the weight is a hyperparameter, which is manually adjusted during training. Description of the Drawings

[0038] Figure 1 It is a schematic structural diagram of the YOLOv5 crater detection model of the present invention. Detailed Embodiments

[0039] The present invention discloses an intelligent crater detection method based on YOLOv5, which is characterized by including the following steps:

[0040] Generally divided into two stages. The first stage completes the training of the target detection model on the ground, and the second stage completes the on-orbit deployment and inference of the target detection model on the satellite.

[0041] Step 1: Train the YOLOv5 crater detection model on the ground, as Figure 1 shown.

[0042] Step 1.1: Prepare training and validation data sets for training the model. A crater detection data set was built based on the public data. Based on the publicly available YOLOLens lunar impact crater data set, the images with the most qualified craters were selected to construct the crater detection data set, which contains 1100 images. Each image was cropped or expanded to a size of 2048×2048, and the crater targets were labeled in the YOLO format. All images were divided into a training set and a validation set in a ratio of 8:2. The training data set was used to optimize the model parameters, and the validation set was used to select the optimal model.

[0043] Step 1.2: Design the structure of the crater detection model. The single-stage object detection YOLOv5 network structure is adopted, as Figure 1As shown, it includes an input - end pre - processing unit, a backbone network, a neck, and a target detection head.

[0044] Input the given lunar - surface image X into the pre - processing unit. Specifically:

[0045] Perform edge detection on the given lunar - surface image X to obtain the edge - detection result. After weighted accumulation of the lunar - surface image X and the edge - detection result, an initial input image is obtained. Further process the initial input image, including resizing the image to 640*640, normalization, format conversion, and Mosaic image enhancement, to obtain the pre - processed image.

[0046] The pre - processed image is input into the backbone network (using CSPDarknet53) to extract features. For an image with an input size of 640*640*3, slice it through the Focus module to reduce the height and width of the image, and the output image size is 320*320. Then, through Concat, integrate the height and width of the sliced image to increase the number of channels of the input image. At this time, the number of image channels is 64. Secondly, perform feature extraction on the integrated image through a Conv convolution module with a size of 3 and a stride of 2, and the output image size is 160*160*128 (the first feature image). Then, after 3 groups of BottleneckCSP1 and Conv convolution operations on the extracted feature map, a feature map with an image size of 20*20*1024 (the second feature image) is obtained. Use the SSP module for the 20*20 feature map to improve the model accuracy. The SSP module performs 4 maximum - pooling operations on the image with sizes of 1*1, 5*5, 9*9, and 13*13 respectively to extract features from multiple aspects. Aggregate the four groups of pooled feature maps through Concat to obtain a feature map with a size of 80*80*512 (the third feature image). The above three feature maps are recorded as backbone feature maps of three scales.

[0047] The neck uses the BottleneckCSP2 module to reduce the number of model parameters. Perform up - sampling and down - sampling operations on the backbone feature maps of the three scales in sequence, and finally obtain three feature maps with different scales of 20*20*1024, 40*40*512, and 80*80*256. The up - sampling process is completed by 2 groups of BottleneckCSP2, a Conv convolution with a size of 1 and a stride of 1, Upsample interpolation, and Concat connection.

[0048] The detection head uses multi - scale feature maps for detection, using large images to detect small targets and small images to detect large targets. For the three different - scale feature maps of the neck, through Conv2d convolution operations, finally obtain three feature maps with sizes of 80*80*255, 40*40*255, and 20*20*255 respectively.

[0049] Step 1.3: Design the loss function according to the model structure. Based on the feature maps of three sizes output by the detection head, after convolutional network operations, outputs of three scales are obtained. Each scale of output contains the bounding box position, bounding box category, and bounding box confidence. Consider three parts of loss according to the three outputs. Calculate the classification loss according to the output bounding box category to measure the accuracy of target category prediction, and use the binary cross-entropy loss function. Calculate the localization loss according to the bounding box position to measure the accuracy of the predicted target bounding box, and use the IOU (Intersection over Union) loss function. Calculate the confidence loss according to the bounding box confidence to measure the accuracy of bounding box confidence prediction, and use the BCE (binary cross-entropy) loss function. The final loss function is the sum of the above three parts of loss for the three scales.

[0050] Step 1.4: Train and select the YOLOv5 crater detection model based on the training set and validation set. Continuously minimize the loss function on the training dataset to optimize the model parameters, and obtain a series of trained YOLOv5 crater detection models. And select the model with the best performance on the validation set as the final YOLOv5 detection model for use.

[0051] Step 2: Deploy the YOLOv5 crater detection model on the satellite and on the ground to complete on-orbit inference.

[0052] Step 2.1: Deploy the trained YOLOv5 crater detection model, image preprocessing code, and crater detection inference code on the satellite.

[0053] Step 2.2: Use the YOLOv5 crater detection model on orbit. The preprocessed lunar surface image is generated on the satellite and input into the image preprocessing and YOLOv5 crater detection inference module code. Call the YOLOv5 crater detection model file to generate the position coordinates and confidence of the detection boxes where the detected crater targets are located. It is used to assist subsequent landing point identification.

[0054] An intelligent crater detection method based on YOLOv5 of the present invention includes the following steps:

[0055] First, complete the training of the YOLOv5 crater detection model on the ground. After the training is completed, deploy the trained model, parameters, and inference code on the satellite to realize on-orbit crater detection of lunar surface images.

[0056] An intelligent crater detection method based on YOLOv5 proposed by the present invention is implemented by using the single-stage object detection idea. Based on the method of ground training and on-orbit reasoning, it realizes the on-orbit detection of craters in lunar surface images. The detected crater target positions and confidence results are transmitted to the lander to help the lander find a more suitable landing site. This method solves the problems of low recognition rate and poor flexibility of traditional crater detection methods. A crater dataset is built according to the public dataset to solve the problem of missing data samples for lunar surface crater detection.

[0057] The content not detailedly described in the specification of the present invention belongs to the prior art well-known to those skilled in the art.

Claims

1. An intelligent crater detection method based on YOLOv5, characterized in that include: Build training and validation sets based on public datasets; Design the YOLOv5 crater detection model; Design loss function according to model structure; The YOLOv5 crater detection model is trained using the training set and the loss function. The model parameters are optimized by continuously minimizing the loss function on the training data set to obtain a series of trained YOLOv5 crater detection models, and the model with the best effect on the validation set is selected as the final YOLOv5 detection model. Deploy the final YOLOv5 detection model on board to form the YOLOv5 crater detection inference module; The original image to be processed is generated on the satellite and input into the YOLOv5 crater detection inference module to generate the position coordinates and confidence of the detection box where the crater is located, which is used to assist in the subsequent landing point identification.

2. The intelligent crater detection method based on YOLOv5 according to claim 1, characterized in that: When building training and validation sets based on public datasets, we used the public YOLOLens lunar impact crater dataset to select images with the most craters, build a crater detection dataset, and adjust each image in the dataset to a consistent size. The constructed crater detection dataset is divided into a training set and a validation set in a ratio of 8:

2. The training dataset is used to optimize the model parameters, and the validation set is used to select the optimal model. While constructing the crater detection dataset, the YOLO format is used to annotate the craters in the image.

3. The intelligent crater detection method based on YOLOv5 according to claim 1, characterized in that: The YOLOv5 crater detection model includes an input preprocessing unit, a backbone network unit, a neck unit, and a target detection head unit. Each unit is connected in sequence. Specifically: The preprocessing unit outputs the preprocessed image to the backbone network unit. After processing, the backbone network unit outputs three scales of backbone feature images to the neck unit. After processing, the neck unit outputs three different scales of neck feature maps to the target detection head unit. After processing, the target detection head unit outputs three different scales of detection head feature maps.

4. The intelligent crater detection method based on YOLOv5 according to claim 3, characterized in that: The input preprocessing unit in the YOLOv5 crater detection model performs image preprocessing on the given image. Specifically: Perform edge detection on a given lunar surface image to obtain an edge detection result, and perform weighted accumulation of the given lunar surface image and the edge detection result to obtain an initial input image; The initial input image is further processed, including image resizing to 640*640, normalization, format conversion, and Mosaic image enhancement to obtain a preprocessed image.

5. The intelligent crater detection method based on YOLOv5 according to claim 3, characterized in that: The backbone network in the YOLOv5 crater detection model uses the CSPDarknet53 network to extract features. Specifically: For the preprocessed image, slice it through the Focus module, output an image with reduced height and width, and integrate the height and width of the sliced ​​image through Concat; The Conv convolution module is used to extract features from the integrated image and output the first feature image. Then, the extracted first feature image is subjected to three sets of BottleneckCSP1 and Conv convolution operations to obtain the second feature image. The SSP module is used to perform maximum pooling operations on the second feature image, further extract features from multiple aspects, and the pooled feature maps are aggregated through Concat to obtain the third feature image. The first feature image, the second feature image, and the third feature image are recorded as backbone feature maps of three scales.

6. The intelligent crater detection method based on YOLOv5 according to claim 3, characterized in that: The neck of the YOLOv5 crater detection model uses the BottleneckCSP2 module to reduce the number of model parameters. Specifically: The three-scale backbone feature maps output by the backbone network are upsampled and downsampled in turn to obtain three different-scale neck feature maps; The upsampling process is completed by 2 groups of BottleneckCSP2, Conv convolution with size 1 and step length 1, Upsample difference and Concat connection.

7. The intelligent crater detection method based on YOLOv5 according to claim 3, characterized in that: The detection head in the YOLOv5 crater detection model uses multi-scale feature maps for detection, using large images to detect small targets and small images to detect large targets. Specifically: The three neck feature maps of different scales obtained from the neck are subjected to Conv2d convolution operation to finally obtain three detection head feature maps of different scales.

8. The intelligent crater detection method based on YOLOv5 according to claim 3, characterized in that: Based on the three different scales of detection head feature maps output by the detection head, three scales of output are obtained after the convolution network operation. The output of each scale includes the bounding box position, bounding box category, and bounding box confidence.

9. The intelligent crater detection method based on YOLOv5 according to claim 8, characterized in that: The loss function consists of three parts: Classification loss, calculated based on the output bounding box category, is used to measure the accuracy of target category prediction and uses a binary cross entropy loss function; Positioning loss, which is calculated based on the bounding box position and is used to measure the accuracy of the predicted target bounding box. It uses the intersection-over-union loss function; Confidence loss, calculated based on the bounding box confidence, is used to measure the bounding box confidence prediction accuracy using the binary cross entropy loss function.

10. The intelligent crater detection method based on YOLOv5 according to claim 9, characterized in that: The final loss function is the sum of classification loss, positioning loss and confidence loss.

Images