Design Method of Autonomous Navigation System for Ground Unmanned Platform Based on Intelligent Acceleration Card

Abstract

The present invention relates to a design method for an autonomous navigation system of a ground unmanned platform based on an intelligent acceleration card, belonging to the field of environmental perception of ground unmanned platforms. The autonomous navigation system of the present invention includes a target recognition algorithm, a visualization module, a camera / lidar fusion module, a target area generation module, a navigation module, and a planning module. Among them, the target recognition algorithm and the visualization module run on the Cambrian intelligent acceleration card MLU100, and the camera / lidar fusion module, the target area generation module, the navigation module, and the planning module run on the autonomous navigation computer. The present invention is driven by the autonomous obstacle avoidance of the ground unmanned platform, adaptively transforms the general target detection algorithm YOLOv3, and realizes the deployment and transplantation of the algorithm on domestic intelligent chips. Finally, through the fusion strategy of images and point clouds, the mapping of two-dimensional image positions to three-dimensional space positions is completed, and target detection is realized, so as to complete the corresponding autonomous navigation tasks.

Description

Technical Field

[0001] The present invention belongs to the field of environmental perception of ground unmanned platforms, and particularly relates to a design method for an autonomous navigation system of a ground unmanned platform based on an intelligent acceleration card. Background Art

[0002] In actual combat scenarios, ground unmanned platforms can use their own environmental perception devices, such as lidar, cameras, millimeter-wave radars, and infrared cameras, to perceive armored vehicle or soldier targets on both sides of the enemy and ourselves, and thus conduct autonomous navigation and driving. Accurately and efficiently identifying the positions of targets of interest is the key to improving the capabilities of the autonomous navigation system of ground unmanned platforms. Grishick first applied convolutional neural networks to target detection, replacing traditional manually designed descriptors to extract target features, and improving the speed and accuracy of target detection. Subsequently, foreign scholars proposed models such as the YOLO series and SSD series, which can simultaneously balance the speed and accuracy of target detection and meet the requirements of real-time detection.

[0003] In addition to relying on the improvement of target detection algorithms, the environmental perception ability of ground unmanned platforms also requires hardware support with powerful parallel computing capabilities. The computing power of the edge computing device Xavier of NVIDIA can reach 30 TOPS, and the power in the typical operating state is 30 W; the computing power of the computing device of Tesla can reach 144 TOPS, and the power in the typical operating state is 72 W, which is mainly used to improve the environmental perception ability of mass-produced vehicle models.

[0004] In order to meet the harsh working environment of ground unmanned platforms, physical encapsulation must be carried out on edge computing devices to meet relevant requirements such as shock and vibration, high and low temperatures, weather, and electromagnetic interference. At present, in order to improve the computing power of edge computing devices, the power has also increased accordingly, and the compact layout inside ground unmanned platforms is not conducive to the heat dissipation of the edge computing devices after encapsulation, which correspondingly restricts the further improvement of computing power. At the same time, with the gradual breakthrough of key technologies for ground unmanned platforms, carrying out the research and development of equipment models is the future development trend. Therefore, the present invention is driven by the domestic demand for equipment, conducts research on domestic intelligent acceleration cards and the adaptation transplantation and deployment of target recognition algorithms, realizes the system integration of target recognition algorithms and autonomous navigation system software, and provides an autonomous navigation system integration solution with low power consumption and high computing power. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] The technical problem to be solved by the present invention is how to provide a design method for an autonomous navigation system of a ground unmanned platform based on an intelligent acceleration card to solve the problems of research on domestic intelligent acceleration cards and the adaptation transplantation and deployment of target recognition algorithms.

[0007] (2) Technical Solution

[0008] To solve the above technical problems, the present invention proposes a method for deploying an autonomous navigation system for a ground unmanned platform based on an intelligent acceleration card, and the method includes the following steps:

[0009] S11. Select the Cambrian intelligent acceleration card MLU100 as an external device of the general-purpose processor, complete the information interaction between the processor on the autonomous navigation computer and the acceleration card through the PCIe bus, and realize the system integration of the acceleration card;

[0010] S12. Collect images of armored vehicles under UNREAL hardware-in-the-loop simulation and physical environment, and augment the original data through the OpenCV open-source library;

[0011] S13. Use the Labelme tool to annotate the armored vehicles in the images, make a JSON file according to the item positions and armored vehicle categories of the armored vehicles in the images, and form a self-made armored vehicle dataset; Combine the VOC2007 and VOC2012 image detection public datasets with the self-made armored vehicle dataset to form a multi-class sample database, which constitutes the training and test data of the YOLOv3 recognition model, and 80% of which is used for training and 20% is used for testing;

[0012] S14. Deploy the YOLOv3 recognition algorithm based on the improved Darknet53 framework of the residual neural network on a general-purpose server, and perform iterative training on the above dataset. The initial learning rate is set to 0.001, the forgetting factor is set to 0.9, and the number of iterations is set to 50000. During the training process, adaptively adjust the training parameters through the mean average precision, and use the multi-scale training and multi-label classification methods to obtain the final target recognition model;

[0013] S15. Convert the target recognition model under the Darknet framework of the general-purpose server to a model under the Caffe architecture through the tool darknet2caffe-yolov3.py;

[0014] S16. Adjust the generated YOLOv3 model file; first, modify the input layer, set the output image to [13416416], and add the mean and standard deviation of the RGB three channels as [0000.00392]; then modify the upsampling layer. First, change the type of the two upsampling layers from "Upsample" to "Interp", and then modify the upsampling layer size parameters "upsample_param" {scale: 2} to "interp_param" {height: 26 width: 26} and {height: 52 width: 52} respectively; then modify the convolutional "convolution" layer, change the value of "num_output" to 3×(number of classes 21 + 5) = 78; finally, modify the yolov3 parameter layer, change the recognized classes to 21, and complete the model adaptation work;

[0015] S17. The domestic intelligent computing acceleration card MLU100 adopts a multi-core processing architecture and runs in the way of data parallelism and model parallelism;

[0016] S18. Use the offline model conversion tool to convert the.caffemodel model trained under the Caffe framework, and integrate the basic operators, fused operators, MLU kernel version, weights, input / output data sizes, parameter information, model version, and MLU instructions into the model, so that the target detection algorithm can run completely independently of the machine learning library and the deep learning framework, and directly call the underlying parallel computing library of the Cambricon intelligent acceleration card;

[0017] S19. Develop an application program, including three parts: pre-processing, post-processing, and inference; the pre-processing part is responsible for collecting the image frames in the scene and converting them into a preset size; the inference part is responsible for loading the YOLOv3 algorithm offline model, allocating the input and output data memory, binding the acceleration card device and environment variables, and finally handing it over to the MLU100 intelligent acceleration card for inference recognition through CNRT (Cambricon Neuware Runtime Library) and the driver. The post-processing is responsible for displaying the inference results, and the results are the bounding boxes of the target detection.

[0018] A method for target recognition and positioning of a ground unmanned platform autonomous navigation system based on an intelligent acceleration card. The autonomous navigation system includes a target recognition algorithm, a visualization module, a camera / Lidar fusion module, a target area generation module, a navigation module, and a planning module. The method includes the following steps:

[0019] S21. Deploy the target recognition algorithm and the visualization module on the Cambrian intelligent acceleration card MLU100, and deploy the camera / LiDAR fusion module, the target area generation module, the navigation module, and the planning module on the autonomous navigation computer. The modules of the autonomous navigation system use the inter-process communication method of Dnet;

[0020] S22. Design the hardware interface of the integration scheme, including various sensors, communication devices, and the autonomous navigation computer. The network switch is responsible for data exchange between the autonomous navigation computer, the network port sensor, the video processing device, and the communication device;

[0021] S23. Design the software interface of the integration scheme. The entire software architecture adopts a hierarchical structure, which is the operating system layer, the communication middleware layer, and the application layer from bottom to top;

[0022] S24. Design the camera / LiDAR fusion module. The domestic intelligent acceleration card is responsible for target detection, and sends the pixel coordinates of the output target bounding box to the camera / LiDAR fusion module of the autonomous navigation computer through the Dnet middleware; the camera / LiDAR fusion module of the autonomous navigation computer is responsible for clustering the point cloud data collected by the LiDAR;

[0023] S25. Through the calibration of the LiDAR and the camera, obtain the transformation matrix from the camera point cloud data to the image coordinate system; project the center point coordinates of the clustered target point cloud onto the image frame collected by the camera, that is, convert the three-dimensional space coordinates into two-dimensional pixel coordinates;

[0024] S26. The camera / LiDAR fusion module calculates the Euclidean distance between the two according to the pixel coordinates recognized by the domestic intelligent acceleration card and the projected point cloud coordinates, judges the mapping relationship between the projected point cloud coordinates and the image recognition coordinates based on the shortest distance, and then extracts the three-dimensional position of the target in the scene.

[0025] An autonomous navigation driving method for a ground unmanned platform autonomous navigation system based on an intelligent acceleration card, the method includes the following steps:

[0026] S31. Since the three-dimensional position of the target constructed by the camera / LiDAR fusion module belongs to the LiDAR coordinate system, and the navigation module of the ground unmanned platform is based on the global coordinate system of longitude and latitude, convert the position of the target in the LiDAR coordinate to the global coordinate system;

[0027] S32. According to the result of the camera / LiDAR fusion module, know which type of point cloud the recognized target belongs to, and then obtain the boundary values of the target three-dimensional coordinates, including x_min, x_max, y_min, y_max, z_min, z_max, generate a large amount of point cloud data, and send it to the mapping module of the vehicle-mounted program through the Dnet communication mechanism;

[0028] S33. Add up the passability costs of the map classifier to obtain the final cost map, which serves as the input for the local path planner; dynamically and online generate a cluster of cubic Bezier curve alternative paths, and optimize the currently executable path based on the maximum curvature, cost map, and path offset distance criteria;

[0029] S34. For the optimized optimal executable path, set the target speed of the end point to zero. Since it takes three stages to go from the current speed to zero, namely, the acceleration stage, the constant speed stage, and the deceleration stage, calculate the speed that the ground unmanned platform should reach at each position. Combine the pure tracking algorithm and the position of the target point to generate the curvature that should be reached at each position;

[0030] S35. Send the speed and curvature to the VCU controller of the ground unmanned platform through CAN communication. Based on the motion control algorithm of the vehicle dynamics model, it completes tasks such as obtaining the desired rotational speed, calculating the feedforward torque, and calculating the feedback torque, realizes the dynamic torque distribution of the six-wheel independent drive motors, and further controls the autonomous driving of the ground unmanned platform.

[0031] (III) Beneficial Effects

[0032] The present invention proposes a design method for an autonomous navigation system of a ground unmanned platform based on an intelligent acceleration card. The method of the present invention selects a domestic intelligent acceleration card as the hardware support and constructs an autonomous navigation system for the ground unmanned platform. Driven by the autonomous obstacle avoidance of the ground unmanned platform, the general target detection algorithm YOLOv3 is adaptively modified to realize the deployment and transplantation of the algorithm on domestic intelligent chips. Finally, through the fusion strategy of images and point clouds, the mapping from the two-dimensional image position to the three-dimensional space position is completed to realize target detection, and thus the corresponding autonomous navigation task is completed. Brief Description of the Drawings

[0033] Figure 1 Computer integrating Cambrian intelligent acceleration card of the present invention;

[0034] Figure 2 Computer architecture of the present invention;

[0035] Figure 3 Training and test data sets;

[0036] Figure 4 Structural diagram of YOLOv3;

[0037] Figure 5 Data parallelism;

[0038] Figure 6 Model parallelism;

[0039] Figure 7 Target inference and recognition process;

[0040] Figure 8 is the overall integration solution;

[0041] Figure 9 is the hardware architecture;

[0042] Figure 10 is the software architecture;

[0043] Figure 11 is the schematic diagram of the autonomous navigation system;

[0044] Figure 12 is the laboratory test scenario;

[0045] Figure 13 is the schematic diagram of coordinate transformation. Detailed implementation manners

[0046] To make the objectives, content and advantages of the present invention clearer, the following further describes in detail the detailed implementation manners of the present invention with reference to the accompanying drawings and embodiments.

[0047] The objective of the present invention is to design an autonomous navigation system based on a domestic intelligent acceleration card to achieve target detection in the application scenarios of ground unmanned platforms, so as to complete the autonomous obstacle avoidance task.

[0048] The autonomous navigation system of the present invention is a software system, including a target recognition algorithm, a visualization module, a camera / Lidar fusion module, a target area generation module, a navigation module and a planning module. Among them, the target recognition algorithm and the visualization module run in the Cambrian intelligent acceleration card MLU100, and the camera / Lidar fusion module, the target area generation module, the navigation module and the planning module run on the autonomous navigation computer.

[0049] The design method of the autonomous navigation system for ground unmanned platforms based on an intelligent acceleration card of the present invention includes: an adaptation transplantation and deployment method for a target recognition algorithm, a method for the autonomous navigation system to achieve target recognition and positioning, and an autonomous navigation driving method.

[0050] The steps of the adaptation transplantation and deployment method for the target recognition algorithm are as follows:

[0051] S11. Select the Cambrian intelligent acceleration card MLU100 as an external device of the general-purpose processor, and complete the information interaction between the processor on the autonomous navigation computer and the acceleration card through the PCIe bus to achieve the system integration of the acceleration card, as Figure 1 shown. The entire system can be divided into a hardware layer, a system layer, an API layer, a framework layer and an application layer from bottom to top, as Figure 2 shown.

[0052] S12. Collect images of armored vehicles in UNREAL hardware-in-the-loop simulation and physical environments, generating a total of 2,000 images. Use methods such as cropping, translation, rotation, mirroring, brightness adjustment, and noise addition through the OpenCV open-source library to augment the original data, dynamically generating more than 5,000 images of armored vehicles.

[0053] S13. Use the Labelme tool to annotate the armored vehicles in the images, create a JSON file based on the item positions and armored vehicle categories in the images, and form a self-made armored vehicle dataset. Combine the VOC2007 and VOC2012 image detection public datasets with the self-made armored vehicle dataset to form a sample database of 21 categories including humans, birds, cats, cows, dogs, horses, sheep, airplanes, bicycles, boats, buses, cars, motorcycles, trains, bottles, chairs, dining tables, potted plants, sofas, monitors, and armored vehicles, which constitutes the training and test data for the YOLOv3 recognition model. 80% of them are used for training and 20% for testing, totaling 24,316 pictures, as Figure 3 shown.

[0054] S14. Deploy the YOLOv3 recognition algorithm based on the improved Darknet53 framework of the residual neural network on a general server and perform iterative training on the above dataset. Set the initial learning rate to 0.001, the forgetting factor to 0.9, and the number of iterations to 50,000. During the training process, adaptively adjust the training parameters through the average accuracy rate, and use multi-scale training and multi-label classification methods to obtain the final object recognition model.

[0055] S15. Convert the object recognition model under the Darknet framework of the general server to a model under the Caffe architecture through the tool darknet2caffe-yolov3.py.

[0056] python darknet2caffe-yolov3.py yolov3-voc.cfg yolov3-voc_final.weights yolov3-voc.prototxtyolov3-voc_final.caffemodel

[0057] S16. Adjust the generated YOLOv3 model file. The network structure diagram is as Figure 4As shown below. First, modify the input layer, set the output image to [13416416], and add the mean and standard deviation of the RGB three channels as [0000.00392]; then modify the upsampling layer. First, change the type of the two upsampling layers from "Upsample" to "Interp", and then change the upsampling layer size parameters "upsample_param" {scale: 2} to "interp_param" {height: 26 width: 26} and {height: 52 width: 52} respectively; then modify the convolutional "convolution" layer and change the value of "num_output" to 3×(number of classes 21 + 5) = 78; finally, modify the YOLOv3 parameter layer and change the recognized classes to 21 to complete the model adaptation work.

[0058] S17. The domestic intelligent computing acceleration card MLU100 adopts a multi-core processing architecture, which can be divided into data parallelism and model parallelism, as Figure 5 、 6 shown below. In response to the real-time requirements of object detection, set the maximum model parallelism, split the YOLOv3 model, so as to start multiple computing cores to calculate different input data, complete the parallel operation of the model on different cores, achieve the maximum utilization of the number of computing cores, and thus reduce the latency.

[0059] S18. Use the offline model conversion tool to convert the.caffemodel model trained under the Caffe framework, and integrate basic operators, fusion operators, MLU core version, weights, input / output data sizes, parameter information, model version, and MLU instructions into the model, so that the object detection algorithm can run completely independently of the machine learning library and the deep learning framework, directly call the underlying parallel computing library of the Cambrian intelligent acceleration card, and improve the algorithm execution efficiency.

[0060] . / builf_offline.sh tool / caffe / genoff-model yolov3-voc.prototxt-weights yolov3-voc_final.caffemodel-mcore MLU100-mname yolov3-voc_final_offline-model_parallel2

[0061] S19. Develop an application program, which mainly includes three parts: pre-processing, post-processing, and inference. The pre-processing part is responsible for collecting image frames in the scene and converting them into a preset size; the inference part is responsible for loading the offline model of the YOLOv3 algorithm, allocating input and output data memory, binding the acceleration card device and environment variables, and finally handing it over to the MLU100 intelligent acceleration card for inference recognition through CNRT (CambriconNeuware Runtime Library) and the driver. The post-processing is responsible for displaying the inference results (the bounding boxes of object detection). The process is as Figure 7 shown.

[0062] The method steps for the autonomous navigation system to achieve object recognition and positioning are as follows:

[0063] S21. Replace the vehicle-mounted autonomous navigation computer of the ground unmanned platform with a computer integrated with a domestic intelligent acceleration card, and transplant all the autonomous navigation system software to the computer integrated with the domestic intelligent acceleration card (subsequently referred to as the autonomous navigation computer). All data processing is completed on one computer. The integration scheme is as Figure 8 shown. Each module of the autonomous navigation system uses the inter-process (IPC) communication method of Dnet. Some parameter settings are as follows:

[0064] 1) Set the communication address of the object recognition module:

[0065] #define ADDR_MLU100_IPC "ADDR_MLU100_IPC"

[0066] 2) Set the communication address of the camera / Lidar fusion module:

[0067] #define ADDR_AUTO_CONTROL_IPC "ADDR_AUTO_CONTROL_IPC"

[0068] 3) Send the pixel points of the center of the recognized target box to the camera / Lidar fusion module through the inter-process communication transmission protocol, and set the data format of the coordinates of the center pixel points of the target:

[0069]

[0070] 4) The camera / Lidar fusion module sends the obtained recognized target center position to the visualization module through the inter-process communication transmission protocol to visualize the results of the fusion algorithm, and set the data format of the coordinates of the center point of the target in the Lidar coordinate system:

[0071]

[0072]

[0073] S22. Design the hardware interface of the integration solution, mainly including various sensors, communication devices, and an autonomous navigation computer, as Figure 9 shown. The network switch is responsible for data exchange among the autonomous navigation computer, network port sensors, video processing devices, and communication devices; the autonomous navigation computer is responsible for deploying all operating software; the multi-line lidar and perception cameras are connected to the autonomous navigation computer through network ports to provide measurement information for the environmental perception module; multiple remote control cameras are connected to the video processing device through SDI interfaces, and through image stitching, provide a larger field of view for personnel to perform remote driving operations; the positioning device adopts a GNSS / INS combined positioning solution and is connected to the autonomous navigation computer through an RS422 interface; the chassis control computer adopts an embedded system and is connected to the autonomous navigation computer through a network port, responsible for receiving the motion control instructions of the autonomous navigation computer and the remote control instructions of the remote control system, controlling the chassis system to maneuver, and feeding back relevant status information.

[0074] S23. Design the software interface of the integration solution. The entire software architecture adopts a layered structure, which from bottom to top are the operating system layer, communication middleware layer, and application layer, as Figure 10 shown. Among them, the operating system adopts the Ubuntu16.04 system, the communication middleware adopts Dnet, and the applications include multiple software components that implement specific functions, belonging to different functional modules. The multi-line lidar acquisition component, multi-line lidar data acquisition component, camera data acquisition component, path tracking component, target recognition and positioning component, map navigation component, etc. all run as independent processes, adopting a non-centered topology structure and are all deployed on the autonomous navigation computer.

[0075] S24. Design the camera / lidar fusion module, as Figure 11 shown. The domestic intelligent acceleration card is responsible for target detection and sends the pixel coordinates of the output target bounding box to the camera / lidar fusion module of the autonomous navigation computer through the Dnet middleware; the camera / lidar fusion module of the autonomous navigation computer is responsible for clustering the point cloud data collected by the lidar.

[0076] S25. Through lidar and camera calibration, obtain the transformation matrix from the camera point cloud data to the image coordinate system. Project the center point coordinates of the clustered target point cloud onto the image frame collected by the camera, that is, convert the three-dimensional space coordinates into two-dimensional pixel coordinates.

[0077] S26. The camera / lidar fusion module calculates the Euclidean distance between the two based on the pixel coordinates identified by the domestic intelligent acceleration card and the projected point cloud coordinates. Judge the mapping relationship between the projected point cloud coordinates and the image recognition coordinates based on the shortest distance, and then extract the three-dimensional position of the target in the scene. The laboratory test scene is asFigure 12 as shown

[0078] The steps of the autonomous navigation driving method are as follows:

[0079] S31. Since the target three-dimensional position constructed by the camera / Lidar fusion module belongs to the Lidar coordinate system, while the navigation module of the ground unmanned platform is based on the global coordinate system of longitude and latitude. Therefore, the position of the target in the Lidar coordinate is converted to the global coordinate system through the formula, as Figure 13 shown

[0080]

[0081] In the formula, o l -x l -y l is the Lidar coordinate system; o-x-y is the global coordinate system, that is, the Mercator coordinate system, and the position information is obtained through the positioning device; α is the heading angle of the ground unmanned platform; b is the longitudinal distance from the centroid of the ground unmanned platform to the center of the Lidar; (x l ,y l ) is the position of the target in the Lidar coordinate system; (x v ,y v ) is the centroid coordinate of the ground unmanned platform; (x, y) is the position of the target in the Mercator coordinate system.

[0082] S32. According to the results of the camera / Lidar fusion module, it can be known which type of point cloud the recognized target belongs to, and then the boundary values of the target three-dimensional coordinates (including x_min, x_max, y_min, y_max, z_min, z_max) are obtained. To facilitate the display of target information on the map, it is assumed that all targets are approximately cylindrical. Taking the target center coordinates (a, b, c) as the center, the distance between y_min and y_max represents the diameter of the target, and the distance between z_min and z_max represents the height of the target. A large amount of point cloud data is generated and sent to the mapping module of the vehicle-mounted program through the Dnet communication mechanism.

[0083] S33. Add up the passability costs of the map classifier to obtain the final cost map, which is used as the input of the local path planner. Dynamically generate a cluster of cubic Bezier curve alternative paths online, and optimize the currently executable path according to the maximum curvature, cost map, and path offset distance criteria.

[0084] S34. For the optimized optimal executable path, set the target speed of the end point to zero. Since it takes three stages to go from the current speed to zero, namely, the acceleration stage, the constant-speed stage, and the deceleration stage, calculate the speed that the ground unmanned platform should reach at each position, and combine the pure tracking algorithm and the position of the target point to generate the curvature that should be reached at each position.

[0085] S35. Send the speed and curvature to the VCU controller of the ground unmanned platform through CAN communication. Based on the motion control algorithm of the vehicle dynamics model, it completes the acquisition of the desired speed, the calculation of the feedforward torque, the calculation of the feedback torque, etc., realizes the dynamic torque distribution of the six-wheel independent drive motors, and further controls the autonomous driving of the ground unmanned platform.

[0086] The method of the present invention selects a domestic intelligent acceleration card as the hardware support and constructs an autonomous navigation system for the ground unmanned platform. Taking the autonomous obstacle avoidance of the ground unmanned platform as the traction, the general target detection algorithm YOLOv3 is adaptively modified to realize the deployment and transplantation of the algorithm on the domestic intelligent chip. Finally, through the fusion strategy of the image and the point cloud, the mapping from the two-dimensional image position to the three-dimensional space position is completed to realize target detection, so as to complete the corresponding autonomous navigation task.

[0087] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.

Claims

1. A method for deploying an autonomous navigation system for a ground unmanned platform based on an intelligent acceleration card, characterized in that The method includes the following steps: S11. Select the domestic intelligent computing acceleration card MLU100 as an external device of the general-purpose processor, complete the information interaction between the processor on the autonomous navigation computer and the acceleration card through the PCIe bus, and realize the system integration of the acceleration card; S12. Collect armored vehicle images under UNREAL hardware-in-the-loop simulation and physical environment, and augment the original data through the OpenCV open-source library; S13. Use the Labelme tool to label the armored vehicles in the images, make a JSON file according to the item positions and armored vehicle categories of the armored vehicles in the images, and form a self-made armored vehicle dataset; Combine the VOC2007 and VOC2012 image detection public datasets with the self-made armored vehicle dataset to form a multi-class sample database, which constitutes the training and test data of the YOLOv3 recognition model, and 80% of which is used for training and 20% for testing; S14. Deploy the YOLOv3 recognition algorithm under the Darknet53 framework improved based on the residual neural network on a general-purpose server, and perform iterative training on the above dataset. Set the initial learning rate to 0.001, the forgetting factor to 0.9, and the number of iterations to 50000. During the training process, adaptively adjust the training parameters through the mean average precision, and use the multi-scale training and multi-label classification methods to obtain the final target recognition model; S15. Convert the target recognition model under the Darknet framework of the general-purpose server to a model under the Caffe architecture through the tool darknet2caffe-yolov3.py; S16. Adjust the generated YOLOv3 model file; first, modify the input layer, set the output image to [1 3 416 416], and add the mean and standard deviation of the RGB three channels as [0 0 0 0.00392]; then modify the upsampling layer. First, change the type of the two upsampling layers from "Upsample" to "Interp", and then change the size parameters of the upsampling layers "upsample_param" {scale: 2} to "interp_param" {height: 26 width: 26} and {height: 52 width: 52} respectively; then modify the convolutional "convolution" layer and change the value of "num_output" to 3 (Number of classes 21 + 5) = 78; finally, modify the yolov3 parameter layer, change the recognized classes to 21, and complete the model adaptation work; S17. The domestic intelligent computing acceleration card MLU100 adopts a multi-core processing architecture and runs in the ways of data parallelism and model parallelism; S18. Convert the.caffemodel model trained under the Caffe framework through an offline model conversion tool, and integrate basic operators, fused operators, MLU kernel versions, weights, input / output data sizes, parameter information, model versions, and MLU instructions into the model, so that the target detection algorithm runs completely independently of the machine learning library and deep learning framework, and directly calls the underlying parallel computing library of the domestic intelligent acceleration card; S19. Develop an application program, including three parts: pre-processing, post-processing, and inference; The pre-processing part is responsible for collecting picture frames in the scene and converting them into a preset size; The inference part is responsible for loading the offline model of the YOLOv3 algorithm, allocating input and output data memory, binding the acceleration card device and environment variables, and finally handing it over to the MLU100 intelligent acceleration card for inference recognition through CNRT (Cambricon Neuware Runtime Library) and the driver. The post-processing is responsible for displaying the results of the inference, and the result is the bounding box of the target detection.

2. The method for deploying an autonomous navigation system for a ground unmanned platform based on an intelligent acceleration card according to claim 1, wherein In the step S12, shear, translation, rotation, mirroring, brightness adjustment, and noise addition are performed through the OpenCV open-source library to augment the original data, and more than 5000 armored vehicle images are dynamically generated.

3. The method for deploying an autonomous navigation system for a ground unmanned platform based on an intelligent acceleration card according to claim 1, wherein In the step S13, a sample database of 21 categories including human, bird, cat, cow, dog, horse, sheep, airplane, bicycle, ship, bus, car, motorcycle, train, bottle, chair, dining table, potted plant, sofa, monitor, and armored vehicle is formed.

4. The method for deploying an autonomous navigation system of a ground unmanned platform based on an intelligent acceleration card according to claim 1, wherein In the step S17, according to the real-time requirement of target detection, the maximum model parallelism is set, and the YOLOv3 model is segmented so as to start multiple computing cores to calculate different input data and complete the parallel operation of the model on different cores.

Images