An automatic driving-based light combination network detection system and method and a storage medium

Abstract

The application relates to a light combined network detection system and method based on automatic driving and a storage medium, the system comprising: an input module for acquiring image data information in real time; an image feature extraction module connected with the input module, comprising a Backbone network unit and a Mobileone network unit, and adopting an image data shunting algorithm to allocate the acquired image data information in real time to the Backbone network unit and the Mobileone network unit for feature extraction; a PANet module connected with the image feature extraction module and used for receiving image feature data information and performing image feature fusion; and a prediction module connected with the PANet module. The application can not only reduce the model size, model parameter quantity and related computing power of an original algorithm model by using a backbone network optimization method under the condition that the function type unmanned vehicle storage condition and computing power condition are limited, but also does not reduce the technical indexes of a perception algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a light combined network detection system and method based on automatic driving and a storage medium. BACKGROUND

[0002] In recent years, the research in the field of automatic driving and intelligent transportation has developed rapidly and achieved fruitful results. The traffic scene target detection technology based on deep learning and computer vision has a very rich application field and is gradually replacing traditional algorithms. However, the application of deep learning algorithms often consumes a large amount of computing resources and is not easy to realize real-time detection. If the algorithm can be improved to improve the model detection accuracy and reasoning speed, and then deployed to edge computing devices such as vehicle cameras or electronic monitoring, the cost can be effectively reduced, which is conducive to the large-scale deployment and popularization of deep learning target detection algorithms in the field of transportation.

[0003] In the prior art, the automatic driving industry mainly uses deep learning to detect dynamic and static targets around the vehicle to obtain the position information and category information of the targets. The model trained by the deep learning detection network does not perform well when transplanted to the vehicle for reasoning. The main reason is that the computing power of the vehicle is limited and the environment of the vehicle is complex. Therefore, it is necessary to optimize the deep learning related detection algorithm. From yolov3 to yolov7, different versions are optimized for detection accuracy and performance. The vehicle, especially the functional unmanned vehicle, needs to handle a relatively large number of tasks. In the case of limited computing power, the detection algorithm transplanted to the vehicle will have a big problem. Therefore, it is urgent to optimize the detection type algorithm.

[0004] At present, yolov5 and other series of detection networks are mainly used to solve the video perception task of functional unmanned vehicles. However, there are many targets to be perceived by the vehicle, and there are many complex states. Therefore, optimizing the backbone network structure of such algorithms can integrate models for multiple different detection tasks in limited storage or computing power space. However, due to the limited computing power and storage space of the functional unmanned vehicle, the perception ability of the functional unmanned vehicle will be reduced. The existing methods for reducing computing power, reducing storage space, and increasing algorithm processing speed generally use techniques such as cropping and distillation of deep learning algorithms to reduce computing power and storage space of the model, and use software libraries such as Tensorrt to accelerate. The acceleration driver in tensorrt drives the cuda core in the graphics card to accelerate. However, this acceleration method has a high cost and will increase the consumption of hardware power of the vehicle. SUMMARY

[0005] In view of the shortcomings of the prior art, the present invention provides a lightweight combined network detection system, method and storage medium based on autonomous driving. Not only can it reduce the model size, model parameter quantity and related computing power of the original algorithm model by using backbone network optimization method under the limited storage and computing power conditions of functional unmanned vehicles, but it also does not reduce the technical indicators of the perception algorithm.

[0006] To achieve the above and other related objectives, the present invention provides the following technical solution:

[0007] A lightweight combined network detection system based on autonomous driving, the system comprising:

[0008] The input module is used to acquire image data information in real time;

[0009] An image feature extraction module, connected to the input module, includes a Backbone network unit and a Mobileone network unit. It uses an image data splitting algorithm to distribute real-time acquired image data information to the Backbone network unit and the Mobileone network unit for feature extraction.

[0010] The PANet module is connected to the image feature extraction module and is used to receive image feature data information and perform image feature fusion.

[0011] The prediction module, connected to the PANet module, is used to classify the fused image features and output feature image data information of different sizes.

[0012] Furthermore, the image data splitting algorithm includes:

[0013] K1. Converts real-time acquired image data into matrix data and outputs discretized image matrix data.

[0014] K2. Input the discretized image matrix data information into the entropy function H.

[0015]

[0016] Where n is the number of samples, xi is the i-th discretized image matrix, p is the sample probability function, and the output is the entropy data information corresponding to the image matrix;

[0017] K3. Setting a preset threshold based on the entropy value data information corresponding to the image matrix, if the entropy value data information corresponding to the image matrix is greater than the preset threshold, the discretized image matrix data information corresponding thereto is transmitted to the Backbone network unit for feature extraction, if the entropy value data information corresponding to the image matrix is less than the preset threshold, the discretized image matrix data information corresponding thereto is transmitted to the Mobileone network unit for feature extraction.

[0018] Further, in step K2, the sample probability function p is

[0019]

[0020] Further, the Mobileone network unit includes multi-layer 1*1 and 3*3 convolution, and a fully connected layer.

[0021] Further, the system further includes a detection module connected with the prediction module, for detecting the real-time acquired image data information according to the feature image data information of different sizes, and outputting detection data information.

[0022] In order to achieve the above object and other related purposes, the present application also provides a light combined network detection method based on automatic driving, which comprises:

[0023] U1. The unmanned vehicle drives on the road, and real-time road image data information is acquired based on the vehicle-mounted camera;

[0024] U2. Based on the real-time acquired road image data information, an image data shunting algorithm is used for image data shunting, and shunted image data information is outputted;

[0025] U3. The shunted image data information is inputted into the Backbone network and the Mobileone network for image feature extraction, and image feature matrix data information is outputted;

[0026] U4. The image feature matrix data information is fused and classified, and feature image data information of different sizes is outputted;

[0027] U5. The feature image data information of different sizes is detected, and detection data information is outputted.

[0028] Further, the detection data information includes image classification data information, target positioning data information, semantic segmentation data information and instance segmentation data information.

[0029] Further, the shunted image data information includes target image data information and non-target image data information.

[0030] To achieve the above object and other related objects, the present application also provides a computer readable storage medium, which stores a computer program programmed or configured to perform any one of the automatic driving based light combined network detection methods.

[0031] The present application has the following positive effects:

[0032] 1. The present application optimizes the backbone network after detecting perception through the Backbone network and the Mobileone network, reduces the complexity of the network, the volume of the trained model becomes smaller, the parameter quantity becomes smaller, the inference speed is accelerated, and the computing power usage rate is reduced.

[0033] 2. The present application adopts an image data shunting algorithm to process original image data information in a hierarchical manner, which not only improves the detection effect, but also improves the detection effect.

[0034] 3. The present application increases the capacity of the entire functional unmanned vehicle perception system and improves the perception ability of the perception system. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a schematic diagram of the system framework of the present application;

[0036] Figure 2 is a schematic diagram of the method flow of the present application. DETAILED DESCRIPTION

[0037] The exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0038] Embodiment 1: As shown in the figure, an automatic driving based light combined network detection system, the system comprises: Figure 1 An input module for acquiring image data information in real time;

[0039] An image feature extraction module connected with the input module, comprising a Backbone network unit and a Mobileone network unit, using an image data shunting algorithm to allocate the real-time acquired image data information to the Backbone network unit and the Mobileone network unit for feature extraction;

[0040]

[0041] ​The PANet module is connected with the image feature extraction module and is configured to receive image feature data information and perform image feature fusion.

[0042] The prediction module is connected with the PANet module and is configured to classify the fused image features and output feature image data information of different sizes.

[0043] In the embodiment, the image data shunting algorithm comprises the following steps.

[0044] K1. The image data information acquired in real time is converted into matrix data information, and the discretized image matrix data information is output.

[0045] K2. The discretized image matrix data information is input into an entropy function H,

[0046]

[0047] wherein n is the number of samples, xi is the i-th discretized image matrix, p is a sample probability function, and the entropy value data information corresponding to the image matrix is output.

[0048] K3. Based on the entropy value data information corresponding to the image matrix, a preset threshold is set, if the entropy value data information corresponding to the image matrix is greater than the preset threshold, the discretized image matrix data information corresponding to the image matrix is transmitted to the Backbone network unit for feature extraction, if the entropy value data information corresponding to the image matrix is less than the preset threshold, the discretized image matrix data information corresponding to the image matrix is transmitted to the Mobileone network unit for feature extraction.

[0049] In the embodiment, in step K2, the sample probability function p is

[0050]

[0051] In the embodiment, the Mobileone network unit comprises multiple layers of 1*1 and 3*3 convolution, and a fully connected layer.

[0052] In the embodiment, the system further comprises a detection module connected with the prediction module, configured to detect the image data information acquired in real time according to the feature image data information of different sizes, and output detection data information.

[0053] The mobileone is adopted as a backbone network for detecting the network, and is embedded into the whole network. When the network is trained, the feature extraction of the target can be more sufficient. The backbone network uses multi-layer 1*1 and 3*3 convolution, combines a regularization module, uses a minimum feature extractor, extracts as many features as possible, and the relevant link parameters are not increased. In this way, the volume of the model will not be increased, the parameters will be reduced, and the computing power is reduced. When the model is used for reasoning, a single-branch structure is used, that is, k is set to 1, which greatly improves the reasoning speed, reduces the use of computing power, reduces the link parameters of the model, and makes the model more miniaturized.

[0054] Embodiment 2: Based on the light combined network detection system for autonomous driving in embodiment 1, the present application is further described and illustrated.

[0055] As Figure 2 shown, the present application also provides a light combined network detection method for autonomous driving, which comprises:

[0056] U1. An unmanned vehicle travels on a road, and real-time road image data information is acquired based on a vehicle-mounted camera;

[0057] U2. Based on the real-time road image data information, an image data shunting algorithm is used for image data shunting, and shunted image data information is output;

[0058] U3. The shunted image data information is input into a Backbone network and a Mobileone network for image feature extraction, and image feature matrix data information is output;

[0059] U4. The image feature matrix data information is fused and classified, and different size feature image data information is output;

[0060] U5. The different size feature image data information is detected, and detection data information is output.

[0061] In this embodiment, the detection data information includes image classification data information, target positioning data information, semantic segmentation data information and instance segmentation data information.

[0062] In this embodiment, the shunted image data information includes target image data information and non-target image data information.

[0063] In order to achieve the above-mentioned purposes and other related purposes, the present application also provides a computer readable storage medium, which stores a computer program programmed or configured to execute any one of the light combined network detection methods for autonomous driving.

[0064] Any reference to storage, memory, database or other medium herein can include non-volatile and / or volatile storage. Non-volatile storage can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile storage can include random-access memory (RAM), or external cache memory. By way of illustration, and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus DRAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM).

[0065] In summary, the application can reduce the model size, model parameter quantity, and related computing power of the original algorithm model by using the backbone network optimization method, without reducing the technical indicators of the perception algorithm, in the case of limited storage conditions and computing power conditions of the functional unmanned vehicle.

[0066] The specific embodiments described above do not constitute an exhaustive description of the scope of the disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the disclosure. Any modification, equivalent replacement, and improvement, etc. made within the spirit and principles of the disclosure should be included within the scope of the disclosure.

Claims

1. An automatic driving-based light group network detection system, characterized by, The system comprises an input module for acquiring image data information in real time; An image feature extraction module connected with the input module, comprising a Backbone network unit and a Mobileone network unit, using an image data shunting algorithm to allocate the real-time acquired image data information to the Backbone network unit and the Mobileone network unit for feature extraction; A PANet module connected with the image feature extraction module, for receiving image feature data information and performing image feature fusion; A prediction module connected with the PANet module, for classifying the fused image features and outputting feature image data information of different sizes; The image data shunting algorithm comprises: K1. converting the real-time acquired image data information into matrix data information, and outputting discretized image matrix data information; K2. inputting the discretized image matrix data information into an entropy function H, ， Wherein, n is the sample number, x i is the i-th discretized image matrix, p is the sample probability function, and the output image matrix corresponds to the entropy value data information. K3. based on the entropy value data information corresponding to the image matrix, setting a preset threshold, if the entropy value data information corresponding to the image matrix is greater than the preset threshold, the discretized image matrix data information corresponding thereto is transmitted to the Backbone network unit for feature extraction, if the entropy value data information corresponding to the image matrix is less than the preset threshold, the discretized image matrix data information corresponding thereto is transmitted to the Mobileone network unit for feature extraction; In step K2, the sample probability function p is 。 2. The automatic driving based light group network detection system according to claim 1, wherein: The Mobileone network unit comprises multiple layers of 1*1 and 3*3 convolution, and a fully connected layer.

3. The automated driving based light group network detection system of claim 1, wherein, The system further comprises a detection module connected with the prediction module, for detecting the real-time acquired image data information according to the feature image data information of different sizes, and outputting detection data information.

4. An automatic driving-based light group network detection method, characterized by, The light combined network detection system based on automatic driving according to any one of claims 1-3, the method comprising: U1. The unmanned vehicle drives on the road, and acquires road image data information in real time based on the vehicle-mounted camera; U2. Based on the real-time acquired road image data information, an image data shunting algorithm is used for image data shunting, and shunted image data information is outputted; U3. The shunted image data information is inputted into the Backbone network and the Mobileone network for image feature extraction, and image feature matrix data information is outputted; U4. The image feature matrix data information is fused and classified, and feature image data information of different sizes is outputted; U5. The feature image data information of different sizes is detected, and detection data information is outputted.

5. The automatic driving based light-weight ensemble network detection method according to claim 4, characterized in that: The detection data information comprises image classification data information, target positioning data information, semantic segmentation data information and instance segmentation data information.

6. The automatic driving based light-weight ensemble network detection method according to claim 4, characterized in that: The shunted image data information comprises target image data information and non-target image data information.

7. A computer readable storage medium characterized in that, The computer readable storage medium has stored thereon a computer program programmed or configured to perform the light combined network detection method based on automatic driving according to any one of claims 4-6.

Images