A Deep Learning-Based Detection Method for Robot Armor Plates

Abstract

The present invention relates to the technical field of image processing, and particularly to a method for detecting a robot armor plate based on deep learning, including collecting pictures of the robot armor plate; adding a path connected to the shallow feature layer of the network, adding a prediction scale for small targets, and adding an SPP module to fuse XNOR-Net; setting network parameters; building a training platform; conducting armor plate detection tests; S6: using the PID control method to control the pan-tilt to aim at the target armor plate. The present invention adds an output prediction scale with strong small target detection ability, enhances the detection effect of small targets, introduces the spatial pyramid pooling SPP module, obtains global features and local features through kernels of different sizes, fuses different scale receptive fields in the network model, and enriches the feature information; fuses XNOR-Net, and achieves the purpose of both reducing the storage space and accelerating by performing binary operations on the weights and inputs.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method for detecting robot armor plates based on deep learning. Background Art

[0002] In recent years, robot technology has developed rapidly. Especially with the development of artificial intelligence technology, robot vision perception technology has also become a research hotspot and has broad application prospects in many fields. The RoboMaster robot competition is a global shooting confrontation robot competition. As a global robot competition platform, the importance of vision algorithms in the competition is self-evident. In order to win the competition, it is necessary to detect the armor plates of the robots. After successfully detecting the armor plates, their coordinates will be transmitted to the control terminal of one's own side, and the control terminal will control the pan-tilt head and automatically launch small balls to hit the armor plates. Whether the armor plates can be correctly recognized is the key to winning the competition.

[0003] In the RoboMaster competition, contestants often use color separation, contour extraction, and contour matching methods to identify and strike the armor plates, but they often spend a lot of time adjusting parameters on site and are extremely vulnerable to the influence of light and environmental conditions, resulting in an increased false recognition rate of the armor plates.

[0004] For embedded devices such as robots, the space and computing power for deep learning are far from enough. Therefore, how to improve the detection accuracy of multi-scale targets in the complex scenarios of actual competitions for target detection algorithms and how to combine target detection technology with embedded devices such as robots have become a major problem. Therefore, for target detection based on convolutional neural networks, it is of great significance to study a method that can improve the detection speed while ensuring the detection accuracy. Summary of the Invention

[0005] Aiming at the deficiencies of existing algorithms, the present invention can accurately and quickly detect robot armor plates in the complex scenarios of actual competitions, and can be carried on an embedded platform for real-time detection according to the actual needs of the competition.

[0006] The technical solution adopted by the present invention is as follows: A method for detecting robot armor plates based on deep learning includes the following steps:

[0007] S1. Obtain a data set;

[0008] Collect pictures of robot armor plates in different competition scenarios, construct a data set of robot armor plates, and preprocess the data set;

[0009] Further, it includes the following steps:

[0010] S11. Collect pictures of the robot armor plates in different competition scenarios. The pictures include the red robot armor plates, the blue robot armor plates, and the red and blue armor plates interfered by the competition environment.

[0011] S12. Classify the collected dataset samples and divide them into a training set and a test set according to a certain proportion.

[0012] S13. The resolution of the collected dataset is too large, which increases the computational burden. Therefore, perform a crop operation on the dataset.

[0013] The robot armor plate recognition scenario requires pictures from the first perspective. After the crop operation, an image similar to the first perspective image is obtained, which is convenient for subsequent image detection.

[0014] S14. Use the YOLO-MARK tool to annotate the pictures of the red and blue robot armor plates, generating label data corresponding to the pictures. The labels include the coordinate positions and types of the robot armor plates. The types include red armor plates and blue armor plates.

[0015] S2. Construct a robot armor plate detection network.

[0016] In the yolov4-tiny network, add a path connected to the shallow feature layer of the network, add an output prediction scale with strong small target detection ability, and add an SPP module to the neck after the backbone network of the network model. Then fuse the XNOR-Net in the yolov4-tiny network to construct an improved yolov4-tiny network suitable for real-time detection of robot armor plates.

[0017] Furthermore, the construction of the robot armor plate detection network includes:

[0018] S21. For the yolov4-tiny network, add a path connected to the shallow feature layer of the network, add an output prediction scale with strong small target detection ability, and fuse the shallow and deep features of the network through upsampling on the original network structure to enhance the detection effect of small targets.

[0019] S22. Introduce the SPP module to obtain global features and local features through kernels of different sizes, fuse different scale receptive fields in the network model, and enrich the feature information. At the cost of very little computational amount, higher detection accuracy is obtained.

[0020] S23. Fuse the XNOR-Net in the yolov4-tiny network, use XNOR and bitcount to replace the multiplication operation in the traditional convolution, and at the same time perform binarization operations on the weights and inputs to achieve the purpose of reducing the network storage space and accelerating the network.

[0021] The convolution operation multiplies the convolution kernel by a certain area of the input to obtain the final value. Assume the input is X, the convolution kernel is W, the scaling factors are α and β, the binary activation is H, and the binary weight is B:

[0022] Quantize the activated input: X≈βH

[0023] Quantize the weight: W≈αB

[0024] Then we get:

[0025] X T W≈βH T αB

[0026] In addition:

[0027] H,B∈{+1,-1} n andβ,α∈R +

[0028] Therefore, the following optimization formula is obtained:

[0029] α * ,B * ,β * ,H * =argmin α,B,β,H ‖X⊙W-βαH⊙B‖

[0030] The input and weight calculated by the optimization formula are the solutions of the binarized optimal values;

[0031] S3. Set the network parameters for the improved yolov4-tiny network;

[0032] Furthermore, it includes the following steps:

[0033] S31. Use the K-means++ algorithm to re-cluster the robot armor plate target dataset to obtain more accurate and representative anchor boxes;

[0034] Furthermore, it includes the following steps:

[0035] S311. Randomly select a sample point from the dataset as the initial clustering center C1;

[0036] S312. First, calculate the shortest distance between each sample and the currently existing clustering centers (i.e., the distance to the nearest clustering center), denoted as D(x); then calculate the probability of each sample point being selected as the next clustering center Finally, select the next clustering center according to the roulette method;

[0037] S313. Repeat S311 and S322 until K clustering centers are selected;

[0038] S32. Use CIOU to replace IOU as the regression optimization loss function;

[0039] Further, it includes the following steps:

[0040] Compared with the traditional loss function Intersection over Union (IOU), CIOU can avoid the problem that the value of the loss function IOU is 0 when the predicted box does not intersect with the ground truth box and the problem that IOU cannot accurately reflect the degree of overlap between the predicted box and the ground truth box; Use CIOU to measure the distance and overlap degree between the target box and the predicted box, coordinate the distance, overlap rate, scale, and penalty term between the target and the anchor box, making the regression of the target box more stable, and not having problems such as divergence during the training process like IOU and Generalized Intersection over Union (GIOU), and use the ratio of the length and width of the predicted box as the penalty term to make the effect of the predicted box more stable;

[0041] S33. Set the pixel of the network input image, batch size, mini-batch, decay rate of the weight, initial learning rate, and number of iterations;

[0042] S4. Build a training platform and put the dataset into the improved yolov4-tiny network for iterative training;

[0043] S5. Transfer the trained network to the robot embedded platform Jetson TX2 for armor plate detection and testing;

[0044] Further, it includes:

[0045] S51. NVIDIA Jetson TX2 environment configuration;

[0046] S52. Image preprocessing, implemented by opencv. The image read by the camera is in BGR format, converted to the RGB format expected by Darknet, scaled to the network input size, and the aspect ratio of the original image is maintained during the scaling process, and the pixel values of the image are normalized;

[0047] S53. Input the preprocessed image into the improved network for forward calculation to detect the armor plate;

[0048] S54. Non-maximum suppression. First, rank the confidence levels of all predicted boxes on the image, select the predicted box with the highest confidence level as the benchmark, then calculate the Intersection over Union (IOU) between the remaining predicted boxes and the benchmark predicted box. If it is greater than the set threshold, delete the bounding box, and then select the one with the highest confidence level from the remaining bounding boxes as the benchmark, and repeat the above process until the best predicted boxes that meet the criteria are finally obtained;

[0049] S55. The average detection precision (mAP) is used as the evaluation index for the detection precision of the yolov4-tiny network, and the filtered detection results are output;

[0050] S6. The robot receives the information of the target armor plate through the improved yolov4-tiny network, and uses the PID control method to control the pan-tilt to aim at the target armor plate and fire bullets at the target;

[0051] Furthermore, it includes:

[0052] Two parts: the upper computer and the lower computer. The upper computer part runs on TX2, and the lower computer runs on the control chip STM32. The two communicate through the serial port, and the communication content is the pan-tilt attitude, that is, the rotation angle of the pan-tilt. In the upper computer part, real-time pictures in the competition are obtained through the camera, the robot armor plate is detected, and the desired pan-tilt attitude is obtained by using the conversion between the image pixel coordinate system and the real-world coordinate system. This attitude is sent to the lower computer through the serial port for cascade PID control of the pan-tilt attitude. The attitude control runs independently in the lower computer and is faster.

[0053] The beneficial effects of the present invention are:

[0054] 1. The neural network yolov4-tiny with strong real-time performance is used to detect the robot armor plate, which has strong robustness and can still be detected and recognized in the case of complex competition background, light interference, shadow occlusion, blurred armor plate, etc.

[0055] 2. Based on the yolov4-tiny network, by improving the yolov4-tiny network, a path connected to the shallow feature layer of the network is added, and an output prediction scale with strong small target detection ability is added. The shallow and deep features of the network are fused by the upsampling method on the original network structure, enhancing the detection effect of small targets. At the same time, the spatial pyramid pooling SPP module is introduced, and global features and local features are obtained through kernels of different sizes. Different scale receptive fields are fused in the network model, enriching the feature information, and obtaining higher detection accuracy at the cost of very little computation.

[0056] 3. The RoboMaster competition has high requirements for real-time performance. XNOR-Net is integrated in the improved yolov4-tiny network, and by performing binary quantization operations on the weights and inputs, the purpose of reducing storage space and accelerating is achieved;

[0057] 4. The improved yolov4-tiny network can be flexibly deployed on the embedded device Jetson TX2, meeting the real-time requirements of the competition, and can quickly and accurately detect the robot armor plate, laying a solid foundation for the next step of controlling the pan-tilt to aim at the target armor plate and firing bullets at the target. Brief Description of the Drawings

[0058] Figure 1 is the flowchart of the robot armor plate detection method based on deep learning of the present invention;

[0059] Figure 2 is the flowchart of the improved yolov4-tiny target armor plate detection of the present invention;

[0060] Figure 3 is the structural diagram of the improved yolov4-tiny network of the present invention;

[0061] Figure 4 is the structural diagram of the spatial pyramid pooling SPP module of the present invention;

[0062] Figure 5 is the comparison diagram between the convolution of the XNOR-Net network and the conventional CNN convolution of the present invention;

[0063] Figure 6 is the visualization curve graph of loss and mAP during the training process of the improved yolov4-tiny network of the present invention;

[0064] Figure 7 is the comparison diagram of the detection effects of the yolov4-tiny network and the improved yolov4-tiny network of the present invention on the robot armor plate. Detailed Embodiment

[0065] The present invention will be further described below with reference to the drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner. Therefore, it only shows the components related to the present invention.

[0066] As Figure 1 shown, a robot armor plate detection method based on deep learning includes the following steps:

[0067] S1. Obtain the robot armor plate data set;

[0068] Collect robot armor plate pictures in different competition scenarios, construct a robot armor plate data set, and preprocess the data set;

[0069] Furthermore, constructing the robot armor plate data set includes:

[0070] S11. Collect robot armor plate pictures in different competition scenarios, including red robot armor plates, blue robot armor plates, and red and blue armor plates interfered by the competition environment;

[0071] S12. Classify the collected data set samples. A total of 2000 pictures are collected and divided into a training set and a test set according to a ratio of 8:2;

[0072] S13. The resolution of the collected dataset is 1920*1080, which is too large and increases the computational burden. Therefore, a crop operation is performed on the dataset;

[0073] For each object in an image, give the center point of this object a drift of up, down, left, and right within 30 pixels, and then, with this point as the center, crop an image of 400*300;

[0074] S14. Use the YOLO-MARK tool to annotate the robot's red and blue armor plate images, generating a txt file corresponding to each image. The txt file contains the coordinate positions and types of the robot's armor plates;

[0075] S2. Build a robot armor plate detection network. As Figure 3 shown, improve the yolov4-tiny network. Add a path connected to the shallow feature layer of the network, add an output prediction scale with strong small target detection ability, add an spp module to the neck after the backbone network of the network model, and at the same time fuse XNOR-Net in the improved network to build a neural network suitable for real-time detection of robot armor plates;

[0076] Furthermore, build a robot armor plate detection network, including:

[0077] S21. For the yolov4-tiny network, add a path connected to the shallow feature layer of the network, add an output prediction scale of 52*52 with strong small target detection ability, and fuse the shallow features and deep features through upsampling on the original network structure for multi-scale image detection to enhance the detection effect of small targets;

[0078] S22. Introduce the SPP module after the backbone network of yolov4-tiny. As Figure 4 shown, the main part is three max pooling layers with pooling kernel sizes of 5*5, 9*9, and 13*13 respectively. Pooling layers of different sizes can extract different features, fuse multiple receptive fields, and thus obtain more scale information;

[0079] S23. Fuse XNOR-Net in the yolov4-tiny network to achieve the purpose of reducing storage space and accelerating by performing binary quantization operations on both the weights and the inputs simultaneously;

[0080] As Figure 5On the left is the traditional convolution module, and the specific process is Conv+BN+Relu+Pooling. On the right is the fused XNOR network convolution module, and the specific process is BN+binary Relu+XNOR Conv+Pooling. The core of acceleration is to convert the operations between 32-bit floating-point numbers into 1-bit XNOR gate operations;

[0081] The convolution operation uses the convolution kernel to multiply a certain area of the input to obtain the final value. Assume the input is X, the convolution kernel is W, the scaling factors are α and β, the binary activation is H, and the binary weight is B:

[0082] Quantize the activation input: X≈βH

[0083] Quantize the weight: W≈αB

[0084] Then we get:

[0085] X T W≈βH T αB

[0086] In addition:

[0087] H,B∈{+1,-1} n andβ,α∈R +

[0088] Therefore, the following optimization formula is obtained:

[0089] α * ,B * ,β * ,H * =argmin α,B,β,H ‖X⊙W-βαH⊙B‖

[0090] The input and weight calculated through the optimization formula are the solutions of the binary optimal values;

[0091] S3. For the improved yolov4-tiny network, set the network parameters;

[0092] Furthermore, it includes the following steps:

[0093] S31. Use the K-means++ algorithm to re-cluster the robot armor plate target dataset to obtain more accurate and representative anchor boxes;

[0094] Furthermore, using the K-means++ algorithm to re-cluster the robot armor plate target dataset includes the following steps:

[0095] S311. Randomly select a sample point from the dataset as the initial clustering center C1;

[0096] S312. First, calculate the shortest distance between each sample and the currently existing cluster centers (i.e., the distance to the nearest cluster center), denoted as D(x); then calculate the probability that each sample point is selected as the next cluster center. Finally, select the next cluster center according to the roulette method.

[0097] S313. Repeat S311 and S322 until K cluster centers are selected.

[0098] Use the K-means++ algorithm to cluster the dataset in the present invention. Considering that the output scale of the improved yolov4-tiny network in the present invention is three, the number of cluster centers is selected as 3, and the 9 obtained anchor boxes are: (13, 20), (22, 21), (17, 33), (34, 19), (29, 29), (27, 46), (54, 29), (42, 41), (63, 56).

[0099] S32. Adopt CIOU to replace IOU as the regression optimization loss function.

[0100] Furthermore, step S32 includes:

[0101] Compared with the traditional intersection over union (IOU) loss function, CIOU can avoid the problem that the value of the IOU of the loss function is 0 when the predicted box and the ground truth box do not intersect and the problem that the IOU cannot accurately reflect the degree of overlap between the predicted box and the ground truth box; use CIOU to measure the distance and overlap degree between the target box and the predicted box, coordinate the distance, overlap rate, scale, and penalty term between the target and the anchor box, make the regression of the target box more stable, and will not have problems such as divergence during the training process like IOU and generalized intersection over union (GIOU), and use the ratio of the length and width of the predicted box as the penalty term to make the effect of the predicted box more stable.

[0102] The CIOU formula is:

[0103]

[0104]

[0105] Among them, ρ 2 (P, P gt ) represents the Euclidean distance between the center points of the predicted box and the ground truth box, c represents the distance of the diagonal of the overlapping area, w is the width of the predicted box, h is the height of the predicted box, w gt is the width of the ground truth box, h gt is the height of the ground truth box, α is the trade-off parameter, and V is the distance of the aspect ratio between the predicted box and the ground truth box.

[0106] S33. Set the pixels of the network input image, batch size, mini - batch, decay rate of weights, initial learning rate, and number of iterations.

[0107] In this embodiment, the input image pixels are set to 416 * 416, the initial learning rate is 0.00261, the learning rate decay factor is 0.1, saved once every 1000 iterations, and the maximum number of iterations is 25000 times.

[0108] S4. Build a training platform, put the dataset into the improved yolov4 - tiny network for iterative training, as Figure 6 shown;

[0109] Configure the required training environment in the computer. In this experiment, the hardware platform used is GPU NVIDIA GEFORCE RTX 3090 and Inter Core i7 - 10700KF processor, the operating system is ubuntu16.04, and the deep learning framework is Darknet. Put the improved yolov4 - tiny network with set parameters into the training environment. Stop training when reaching the maximum number of iterations of 25000 times and the training loss is completely convergent. The trained yolov4 - tiny network is used as the final robot armor plate detection network.

[0110] S5. Migrate the trained detection network to the robot embedded platform TX2 for armor plate detection testing;

[0111] As Figure 2 shown, migrate the trained improved yolov4 - tiny network to the robot embedded platform TX2 for armor plate detection testing, including:

[0112] S51. Configure the deep learning environment on the embedded platform NVIDIA Jetson TX2. Use the flash package Jetpack4.3 to flash the device, and install CUDA9.0, CUDNN7.6.3, and Opencv3.4 toolkits. Transplant the weight file trained on NVIDIA GEFORCE RTX 3090 to TX2 for detection operations.

[0113] S52. Image pre - processing, implemented through Opencv. The image read by the camera is in BGR format, converted to the RGB format expected by Darknet, scaled to the network input size of 416 * 416, and the aspect ratio of the original image is maintained during the scaling process. Normalize the image pixel values;

[0114] S53. Input the pre - processed image into the improved network for forward calculation to detect the armor plate;

[0115] S54. Non-maximum suppression. First, rank the confidence levels of all predicted bounding boxes on the image, select the predicted bounding box with the highest confidence level as the reference, then calculate the intersection over union (IoU) between the remaining predicted bounding boxes and the reference predicted bounding box. If it is greater than the set threshold, delete the bounding box, and then select the one with the highest confidence level from the remaining bounding boxes as the reference, and repeat the above process until the best predicted bounding boxes that meet the criteria are obtained.

[0116] S55. Output the filtered detection results;

[0117] To evaluate the detection results of the improved YOLOv4-tiny network, the accuracy and real-time performance of the detection network need to be considered; in this experiment, mAP is used as the evaluation index for detection accuracy;

[0118] mAP is related to precision (P, %), recall (R, %), and the formula is as follows:

[0119]

[0120]

[0121]

[0122]

[0123]

[0124] In the formula, TP is the number of samples correctly classified as positive, FP is the number of samples misclassified as positive, FN is the number of samples misclassified as negative, M is the total number of classes, and AP(k) is the AP value for the k-th class. The F1 score is an index used to measure the accuracy of binary classification. The F1 score can be regarded as a weighted average of precision and recall; the detection time is measured by the average time consumed by the object detection network to detect an image, with the unit of ms.

[0125] Table 1 Comparison of algorithm performance indicators before and after improving YOLOv4-tiny

[0126] Map Recall F1 Fps yolov4 94.29％ 0.94 0.92 4.4 yolov4-tiny 88.22％ 0.81 0.86 33.3 Improved yolov4-tiny 93.63％ 0.90 0.92 31

[0127] As analyzed from Table 1, YOLOv4 has the best detection effect, with a precision of up to 94.29%, but the Fps is only 4.4. Although the Fps of YOLOv4-tiny is 33.3, the precision is 88.22%; by improving YOLOv4-tiny, the detection precision can be improved without significantly reducing the detection speed.

[0128] WillFigure 7 Comparing (a) and (b), that is Figure 7 (a) is the yolov4-tiny network Figure 7 (b) is the improved yolov4-tiny network. By comparison, it can be seen that there are cases of missed detection in the yolov4-tiny network, while the improved yolov4-tiny network has a good detection effect on the robot armor plate.

[0129] Inspired by the ideal embodiments of the present invention described above, through the above description, relevant staff can completely make various changes and modifications without departing from the technical idea of the present invention. The technical scope of the present invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for detecting robot armor plates based on deep learning, characterized in that, It includes the following steps: S1. Collect pictures of the robot armor plates in different competition scenarios, construct a dataset of robot armor plates, and preprocess the dataset; S2. Add a path connected to the shallow feature layer of the network in the yolov4-tiny network, add an output prediction scale for detecting small targets, add an SPP module to the neck after the backbone network of the network model, and fuse XNOR-Net in the yolov4-tiny network to construct an improved yolov4-tiny network; The step S2 includes: S21. Add a path connected to the shallow feature layer of the network in yolov4-tiny, add an output prediction scale with strong ability to detect small targets, and fuse the shallow and deep features of the network through upsampling on the original network structure; S22. Introduce the SPP module, obtain global features and local features through kernels of different sizes, fuse receptive fields of different scales in the network model, and enrich the feature information; S23. Fuse XNOR-Net in the yolov4-tiny network, use XNOR and bit counting to replace the multiplication operation in the traditional convolution, and perform binary quantization operations on the weights and inputs at the same time; The convolution operation multiplies the convolution kernel by a certain area of the input. Assume the input is X, the convolution kernel is W, the scaling factors are α and β, the binary activation is H, and the binary weight is B: Quantize the activation input: X≈βH Quantize the weights: W≈αB Then we get: X T W≈βH T αB where \(H, B\in\{+1, - 1\}\) n and \(\beta,\alpha\in\mathbb{R}\) + ; Derive the optimization formula: α * , B * , β * , H * = argmin α,B,β,H ||X ⊙ W - βαH ⊙ B|| The input and weights calculated by the optimization formula are the solutions of the binary optimal values; S3. Set parameters for the improved yolov4-tiny network; S4. Build a training platform, put the dataset into the improved yolov4-tiny network for iterative training; The step S3 includes: S31. Use the K-means++ algorithm to re-cluster the target dataset of the robot armor plates; S32. Use the complete intersection over union to replace the intersection over union as the regression optimization loss function; S33. Set the pixels of the network input image, batch size, mini-batch, decay rate of the weights, initial learning rate, and number of iterations; S5. Transfer the trained improved yolov4-tiny network to the robot embedded platform Jetson TX2 for armor plate detection and testing; The step S5 includes: S51. Configure the NVIDIA Jetson TX2 environment; S52. Perform image preprocessing through opencv. The image read by the camera is in BGR format, convert it to the RGB format expected by Darknet, scale the image to the network input size, keep the aspect ratio of the original image during the scaling process, and perform normalization operations on the image pixel values; S53. Input the preprocessed image into the improved yolov4-tiny network for forward calculation to detect the armor plate; S54. First, rank the confidence levels of all prediction boxes, select the prediction box with the highest confidence level as the benchmark, then calculate the intersection over union (IoU) between the remaining prediction boxes and the benchmark prediction box. If it is greater than the set threshold, delete the prediction box, and then select the prediction box with the highest confidence level from the remaining prediction boxes as the benchmark. Repeat the above process until the best prediction box that meets the criteria is finally obtained. S55. Use the mean average precision (mAP) as the evaluation index for the detection accuracy of the yolov4-tiny network, and output the filtered detection results. S6. The robot receives the target armor plate information through the improved yolov4-tiny network, and uses the PID control method to control the pan-tilt to aim at the target armor plate and fire bullets at the target.

2. The method for detecting robot armor plates based on deep learning according to claim 1, wherein The step S1 includes: S11. Collect pictures of the robot armor plates in different competition scenarios, including red robot armor plates, blue robot armor plates, and red and blue armor plates interfered by the competition environment. S12. Classify the collected dataset samples and divide them into a training set and a test set according to a certain proportion. S13. Perform cropping operations on the dataset. S14. Use the YOLO-MARK tool to annotate the pictures of the robot red and blue armor plates to generate label data corresponding to the pictures.

3. The method for detecting robot armor plates based on deep learning according to claim 1, characterized in that The step S6 includes: The upper computer and the lower computer use serial communication, and the communication content is the pan-tilt attitude. The upper computer obtains the desired pan-tilt attitude by using the conversion between the image pixel coordinate system and the real-world coordinate system, and sends the desired pan-tilt attitude to the lower computer through the serial port for cascade PID control of the pan-tilt attitude.

Images