Intelligent car lightweight obstacle target detection method

Abstract

The present application relates to a kind of intelligent car light weight obstacle target detection method, comprising (1) collecting the original image of obstacle on road and making data set;(2) based on improved Yolov4, construct light weight network model;(3) build intelligent car target detection experimental environment, and utilize data set to carry out multiple training to light weight network model, obtain multiple network model training weights;(4) obtain the best network model training weight, and transplant to intelligent car target detection experimental environment;(5) utilize depth camera component to obtain the information outside intelligent car, and utilize the best network model training weight to detect and identify the obstacle in front of intelligent car.The present application enhances the identification efficiency of intelligent car by improving algorithm, aims at solving the problems, such as the real-time of detecting obstacle and low detection accuracy, due to the low algorithmic power of vehicle-mounted chip of intelligent car.

Description

Technical Field

[0001] This invention belongs to the field of target detection, specifically relating to a lightweight obstacle target detection method for intelligent vehicles. Background Technology

[0002] In recent years, with the advancement of science and technology, artificial intelligence (AI) has also developed rapidly. Machine vision, in particular, is one of the main research areas in AI, and it has been widely applied in the fields of computers and machines. In environmental perception, visual recognition using machine vision has become the mainstream research area for target detection. Autonomous driving, as an important branch of AI, has now become a research hotspot both domestically and internationally. The perception component of autonomous vehicles mainly relies on sensors such as cameras and radar to perceive the surrounding environment during operation. Obstacle detection is a crucial aspect of environmental perception for autonomous vehicles, and its effectiveness directly impacts safe driving. Therefore, efficient, accurate, and highly portable target detection algorithms play a vital role in the development of autonomous driving technology.

[0003] Currently, due to the limitations of low computing power in vehicle-mounted chips, the real-time performance and accuracy of obstacle detection are relatively low, which cannot meet the requirements of autonomous driving technology for obstacle detection. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight obstacle target detection method for intelligent vehicles. The method uses a depth camera at the front of the vehicle to acquire real-time images, detect and identify obstacles, and improves the recognition efficiency of the intelligent vehicle by improving the algorithm. This aims to solve the problems of low real-time performance and low detection accuracy of obstacle detection in intelligent vehicles due to the limited computing power of onboard chips.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: a lightweight obstacle target detection method for intelligent vehicles, comprising the following steps:

[0006] Step 1: Collect raw image data of obstacles on the road to create a dataset;

[0007] Step 2: Construct a lightweight network model based on the improved Yolov4; The improvement method of Yolov4 is as follows: (1) Replace the backbone of the original Yolov4 network with the backbone of the Ghostnet network; (2) Add the ECA attention mechanism with residual connections to the three effective feature layers in the output to the neck network; (3) Introduce a residual structure after stacking the 13×13 feature layer in the output to the head and the first downsampling in the PAnet feature fusion network; (4) Replace the 3×3 standard convolution in the neck network with Ghost modules and depthwise separable convolutions;

[0008] Step 3: Set up an experimental environment for target detection in a smart car. Use the dataset created in Step 1 to train the lightweight network model built in Step 2 multiple times to obtain training weights for multiple network models.

[0009] Step 4: Test the training weights of the multiple network models obtained in Step 3 on the validation set data, and select the training weights of the network model with the best performance based on the test results.

[0010] Step 5: Transfer the training weights of the network model with the best performance in Step 4 to the intelligent vehicle target detection experimental environment built in Step 3.

[0011] Step 6: Use the depth camera component to acquire information about the outside world of the smart car. Input the acquired information into the input end of the network. Use the network model with the best performance in Step 4 to train the weights to detect and identify obstacles in front of the smart car. Display the real-time detection results through the built-in visualization software.

[0012] Step 7: If the external information acquired by the depth camera does not contain the trained obstacles, the visualization software will not display the detection results; otherwise, repeat step 6 to perform obstacle detection in real time and display the detection results.

[0013] For step two, in the Ghost module, set s redundant features corresponding to each basic feature, and output... The network identifies several basic features and then performs a linear transformation on each feature to generate corresponding redundant features, thereby reducing the number of network parameters and computational cost.

[0014] In step two, the depthwise separable convolution consists of 3×3 channel-wise convolution and 1×1 pointwise convolution.

[0015] The detection metrics in step four are precision, recall, and average accuracy.

[0016] The depth camera in step six is ​​installed at the front of the smart car.

[0017] In step six, the specific steps for the intelligent vehicle to perform obstacle detection are as follows: First, in the experimental environment, the depth camera collects obstacle information and inputs it into the input port of the lightweight network model. Then, the feature extraction network of the Ghostnet backbone extracts effective feature layers and the feature fusion network of the neck network, and outputs it to the head network. Based on the best authority value of the object category in the training weights of the obtained best network model, the target obstacle is detected, and the obstacle detection result is output.

[0018] Furthermore, the detection results include obstacle category information, location information, and the accuracy of object detection.

[0019] The beneficial effects of this invention are as follows: This invention provides a lightweight obstacle detection method for intelligent vehicles. It uses a depth camera to collect information about predetermined obstacles in front of the intelligent vehicle for real-time obstacle detection. However, due to the low computing power limitations of the vehicle's onboard chip, it cannot meet the real-time requirements for obstacle detection. By improving the target detection algorithm, changing the backbone network and convolutional form of the network model, and adding an ECA attention mechanism, the parameters of the network model are reduced, and the detection accuracy is improved. This ensures the real-time obstacle detection of the intelligent vehicle while improving the accuracy of obstacle category detection and reducing the impact of the low computing power of the onboard chip. Attached Figure Description

[0020] Figure 1 This is a flowchart of the present invention;

[0021] Figure 2 This refers to the existing Yolov4 network model.

[0022] Figure 3 This invention provides a lightweight network model based on an improved Yolov4 architecture.

[0023] Figure 4 This is a schematic diagram of the ECA attention mechanism.

[0024] Figure 5 The schematic diagram of the ECA attention mechanism structure for residual connections provided by this invention;

[0025] Figure 6 This is a schematic diagram of the Ghost module structure.

[0026] Figure 7 A diagram showing the residual block structure for residual connections;

[0027] Figure 4 In this diagram, H represents the height of the image, W represents the width of the image, C represents the number of channels, and Z and X both represent feature information. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the invention in any way.

[0029] A lightweight obstacle target detection method for intelligent vehicles includes the following steps:

[0030] Step 1: Use a camera to randomly collect raw image data of vehicles and obstacles on the road to create a dataset, and use software to label the categories of obstacles;

[0031] Step two: Construct a lightweight network model based on the improved Yolov4; the specific structure code of the modified algorithm is as follows:

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] Step 3: Set up an experimental environment for target detection in a smart car. Use the dataset created in Step 1 to train the lightweight network model built in Step 2 multiple times to obtain training weights for multiple network models.

[0041] Step 4: Test the training weights of the multiple network models obtained in Step 3 on the validation set data, and select the training weights of the network model with the best performance based on the test results.

[0042] Step 5: Transfer the training weights of the network model with the best performance in Step 4 to the intelligent vehicle target detection experimental environment built in Step 3.

[0043] Step 6: Use the depth camera component to acquire information about the outside world of the smart car. Input the acquired information into the input end of the lightweight network model. Use the network model with the best performance in Step 4 to train the weights to detect and identify obstacles in front of the smart car. Display the real-time detection results through the built-in visualization software Vedio.

[0044] Step 7: If the external information acquired by the depth camera does not contain the trained obstacles, the visualization software will not display the detection results; otherwise, repeat step 6 to perform obstacle detection in real time and display the detection results.

[0045] In step one, the photos collected cover common obstacles to define the target detection work of the intelligent vehicle, including pedestrians, cars, buses, electric bicycles, bicycles, etc., and the categories of the photos are labeled.

[0046] The improvement method for Yolov4 in step two is as follows: (1) Replace the backbone of the original Yolov4 network with the backbone of the Ghostnet network; (2) Add the ECA attention mechanism with residual connections to the three effective feature layers in the output to the neck network; (3) Introduce a residual structure after stacking the 13×13 feature layer in the output to the head and the first downsampling in the PAnet feature fusion network; (4) Replace the 3×3 standard convolution in the neck network with Ghost modules and depthwise separable convolutions.

[0047] Furthermore, in the Ghost module, s redundant features are defined for each basic feature, and only the output needs to be... We first identify 1 basic feature, then perform a linear transformation on each feature to generate corresponding redundant features, thus reducing the number of network parameters and computational cost. The number of parameters and computational cost are reduced by a factor of 's' compared to the original convolutional form, as shown in the formula:

[0048]

[0049] In the formula, n represents the number of channels generated after n sets of k×k Kernel convolutions; The size of the feature map is represented by c; the number of channels is represented by s; m = n / s intrinsic feature maps are generated. Linear operations used to compute phantom feature maps; k represents the convolution kernel size.

[0050] Furthermore, depthwise separable convolution consists of 3×3 channel-wise convolution and 1×1 pointwise convolution. The computational cost of depthwise separable convolution compared to the original convolution form is:

[0051]

[0052] In the formula, and Represents the number of channels; This represents the size of the input feature map;

[0053] Furthermore, the residual connection's ECA attention mechanism, based on the residual connection concept, adds a lightweight ECA attention mechanism to the residual edges on the basis of the original output information. The residual edges mainly capture the key feature information, thereby increasing the accuracy of obstacle recognition.

[0054] The detection metrics in step four are precision, recall, and average accuracy.

[0055] Furthermore, the depth camera, mounted on the movable vehicle body, is the core component for obstacle detection. Installed at the front of the intelligent vehicle, it collects environmental information about the vehicle's surroundings and performs real-time obstacle detection. Commercially available products can be used for this depth camera.

[0056] In step six, the specific steps for the intelligent vehicle to perform obstacle detection are as follows: First, in the constructed experimental environment, the depth camera collects obstacle information and inputs it into the input port of the lightweight network model. Then, the feature extraction network of the Ghostnet backbone extracts effective feature layers, and the feature fusion network of the neck network extracts the effective feature layers. The output is then sent to the head network, which detects target obstacles based on the best authority value of the object category in the training weights of the obtained best network model. The detection result of the obstacle is output, which includes the obstacle category information, location information, and the accuracy of the detected object.

[0057] The obstacle target detection method for lightweight intelligent vehicles of the present invention and the obstacle target detection method based on the conventional Yolov4 algorithm in the prior art were tested and compared in the same environment. The results are shown in the table below.

[0058]

[0059] The lightweight obstacle detection method for intelligent vehicles disclosed in this invention greatly reduces the number of parameters and computational load compared to traditional single-stage target detection algorithms, and improves the detection accuracy of the network model. The original Yolov4 network weights are 244MB, while the network model provided by this invention has weights of only 43MB, less than 1 / 5 of the original model. This improves the real-time obstacle detection effect and efficiency of intelligent vehicles, ensures the safety of intelligent vehicles during operation, and meets the goals of real-time and accurate obstacle detection.

[0060] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the specific implementation of the present invention with reference to the above embodiments. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention are within the protection scope of the pending claims.

Claims

1. A lightweight obstacle target detection method for intelligent vehicles, characterized in that, Includes the following steps: Step 1: Collect raw image data of obstacles on the road to create a dataset; Step 2: Construct a lightweight network model based on the improved Yolov4; The improvement method of Yolov4 is as follows: (1) Replace the backbone of the original Yolov4 network with the backbone of the Ghostnet network; (2) Add the ECA attention mechanism with residual connections to the three effective feature layers in the output to the neck network; (3) Introduce a residual structure after stacking the 13×13 feature layer in the output to the head and the first downsampling in the PAnet feature fusion network; (4) Replace the 3×3 standard convolution in the neck network with Ghost modules and depthwise separable convolutions; In the Ghost module, s redundant features are set for each basic feature, the basic features are output, and then a linear transformation is performed on each basic feature to generate the corresponding redundant features, so as to reduce the number of network parameters and computational cost; the depthwise separable convolution consists of 3×3 channel-wise convolution and 1×1 pointwise convolution. Step 3: Set up an experimental environment for target detection in a smart car. Use the dataset created in Step 1 to train the lightweight network model built in Step 2 multiple times to obtain training weights for multiple network models. Step 4: Test the training weights of the multiple network models obtained in Step 3 on the validation set data, and select the training weights of the network model with the best performance based on the test results. Step 5: Transfer the training weights of the network model with the best performance in Step 4 to the intelligent vehicle target detection experimental environment built in Step 3. Step 6: Use the depth camera component to acquire information about the outside world of the smart car. Input the acquired information into the input end of the network. Use the network model with the best performance in Step 4 to train the weights to detect and identify obstacles in front of the smart car. Display the real-time detection results through the built-in visualization software. Step 7: When the external information acquired by the depth camera does not contain the trained obstacles, the detection results are not displayed in the visualization software. Conversely, repeat step six to perform obstacle detection in real time and display the detection results.

2. The lightweight obstacle target detection method for an intelligent vehicle according to claim 1, characterized in that: The detection metrics in step four are precision, recall, and average accuracy.

3. The lightweight obstacle target detection method for an intelligent vehicle according to claim 1, characterized in that: The depth camera in step six is ​​installed at the front of the smart car.

4. The lightweight obstacle target detection method for an intelligent vehicle according to claim 1, characterized in that: In step six, the specific steps for the intelligent vehicle to perform obstacle detection are as follows: First, in the experimental environment, the depth camera collects obstacle information and inputs it into the input port of the lightweight network model. Then, the feature extraction network of the Ghostnet backbone extracts effective feature layers and the feature fusion network of the neck network, and outputs it to the head network. Based on the best authority value of the object category in the training weights of the obtained best network model, the target obstacle is detected, and the obstacle detection result is output.

5. The lightweight obstacle target detection method for an intelligent vehicle according to claim 4, characterized in that: The detection results include obstacle category information, location information, and the accuracy of object detection.

Images