A deep learning-based target detection system

Abstract

The application belongs to the technical field of target detection systems, and specifically relates to a target detection system based on deep learning, which is mainly applied to the aspect of automatic driving of automobiles and comprises a data marking module, a data storage module, a data processing module, a data verification module, a simulation simulation module and a result recording module. The result recording module is used to save and record data that passes simulation, so as to be subsequently imported into an automatic driving system and used to provide target detection for automatic driving. The application stores relevant data in the result recording module through simulation simulation and automatically updates previous data, so that the result recording module always stores the latest stable data, ensures that data calling does not appear in a disorderly state, reduces the occupation of system memory by previous data, reduces the memory occupation of previous data, and reduces the influence of the running efficiency of the target detection system caused by the decrease of the running efficiency of the target detection system.

Description

Technical Field

[0001] This invention belongs to the technical field of target detection systems, specifically a target detection system based on deep learning. Background Technology

[0002] In the field of artificial intelligence, autonomous driving has always been an important research direction. By using intelligent driver assistance systems to avoid dangers caused by human negligence, tragedies can be avoided to the greatest extent possible. Current autonomous driving target detection systems mainly rely on cameras and lidar to detect the types and positions of various objects in the scene in front of the vehicle, and then transmit the relevant information to the target detection system for analysis. When a danger occurs, emergency braking is taken.

[0003] Current object detection systems primarily employ two processes: object category detection and object instance detection. The first step is to locate the object's position, and the second step is to classify the object at that position. Object instance detection offers finer granularity; for example, when a vehicle or person is detected, it will specify the brand and model of the vehicle, or a specific person within the "person" category. Object category detection, on the other hand, focuses on identifying the category of the detected object. For instance, in autonomous driving scenarios, it will only provide the categories of objects captured by the cameras in the current scene, such as people, vehicles, buildings, or traffic lights.

[0004] Existing object detection systems are based on the classic object detection framework Faster R-CNN in deep learning technology. By trying different feature extraction layers, a model with excellent speed and accuracy is trained and applied to the system. It is mainly based on the original VGG16 and ZF models of Faster R-CNN and used for feature extraction, thereby realizing deep learning in the object detection system.

[0005] While the aforementioned learning framework can effectively ensure that the object detection system can perform autonomous learning and optimization and be applied to autonomous driving systems, the object detection method will generate a large amount of redundant data during the detection process. If this redundant data is not filtered out in time, the storage pressure of the system will increase as the running time of the object detection system increases. Furthermore, the increasing amount of data will also affect the reaction efficiency of the object detection system, resulting in a decrease in detection efficiency after long-term operation.

[0006] In view of this, in order to solve the above problems, the present invention proposes a target detection system based on deep learning. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, and considering the problems that existing target detection systems accumulate a large amount of redundant data after long-term operation, affecting the operating efficiency of the target detection system, this invention proposes a target detection system based on deep learning.

[0008] The technical solution adopted by this invention to solve its technical problem is:

[0009] The present invention provides a deep learning-based target detection system, characterized in that it comprises:

[0010] A data tagging module is used to tag objects on the road and assign a unique number to each object.

[0011] A data storage module is used to save the data marked in the data marking module and classify it according to the category number of the marking.

[0012] A data processing module, which is used to analyze and process the stored data in the data storage module;

[0013] The data verification module is used to classify and organize the processed data and verify whether each data item is erroneous.

[0014] The simulation module uses data that has been verified to be complete and correct, and then imports the data into the program for simulation to confirm whether the target detection system can detect and identify the simulated target information.

[0015] The result recording module is used to save and record the data of the simulation, so that it can be imported into the autonomous driving system later to provide target detection for autonomous driving.

[0016] The object detection system is mainly based on convolutional neural networks (CNNs) to achieve autonomous deep learning. A CNN mainly includes convolutional layers, pooling layers, and activation layers, and finally uses fully connected layers for classification, thus enabling autonomous deep learning for object detection. The convolutional layer is the core layer of the CNN, and its parameters can be viewed as a series of learnable filters. During the forward computation, we input data of a certain region size and multiply it by the convolutional kernel to obtain a new two-dimensional data, called a feature map. Sliding the filter across an image yields many feature maps. We can understand this as each convolutional kernel only caring about features within a region the size of the kernel in the image; when learned features appear, the kernel is activated.

[0017] Preferably, the data labeling module performs target detection based on a camera and a lidar. After the camera and lidar collect data, the data labeling module automatically labels regular objects.

[0018] The data tagging module stores unidentifiable objects in a separate folder, and technicians then manually add tags to these objects.

[0019] The data labeling module includes data collection and data annotation. The data labeling module uses a camera and LiDAR to annotate the collected objects, including annotating the object's category and location information, which are used as labels for simulation.

[0020] Preferably, the data storage module is based on both cloud disk storage and local hard disk storage to prevent data loss or damage that would make it impossible to find the original data source for replacement.

[0021] Preferably, the data storage module has two independent storage layers, which are used to store target category detection data and target instance detection data respectively.

[0022] Preferably, the data processing module is used to transform the tagged data in the data storage module into data that the data verification module can recognize.

[0023] The main function of the data processing module is not only to convert the marked data into data that the data verification module can recognize, but also to preprocess the marked image data, including image scaling, cropping, whitening, etc., which can effectively improve the data verification speed of the data verification module.

[0024] Preferably, the data verification module is used to verify data integrity to prevent incomplete data from entering the simulation module, which could lead to simulation failure and require a significant amount of time for troubleshooting.

[0025] Preferably, the simulation module uses the collected data to perform three-dimensional simulation to verify the recognition accuracy of the target detection system, identify problems in a timely manner during the simulation, optimize the efficiency of target recognition, and reduce the memory usage of the target detection system.

[0026] The core function of the simulation module is to use the Faster R-CNN detection framework to train a target detection model suitable for the current scene, verify whether the target detection system performs well in terms of accuracy and speed in the process of recognizing objects based on the data collected above, and then optimize accordingly based on the problems that arise to ensure the stability of the target detection system in actual use.

[0027] Preferably, the result recording module periodically filters the saved data and automatically removes redundant and useless data, thereby improving the operating speed of the target detection system.

[0028] The result recording module includes a data comparison library, which stores the latest data to be compared with redundant and useless data from the past, thereby enabling data filtering.

[0029] The results recording module periodically compares the data and then uses different colors to identify different file categories. For example, if two images completely overlap and have the same color parameters during fitting, they can be identified as the same data. Therefore, the results recording module will automatically delete such data, thereby reducing the proportion of redundant and useless data occupying memory and speeding up the operation of the target detection system.

[0030] The beneficial effects of this invention are as follows:

[0031] 1. The deep learning-based target detection system of this invention, through simulation verification, if the data verification passes and accurate identification can be performed during the simulation process, it indicates that the labeled data is correct. Therefore, the simulation imports the relevant data into the result recording module for storage and automatically updates the previous data, thereby ensuring that the result recording module always stores the latest stable data, ensuring that there will be no disorder when data is retrieved, and reducing the memory occupation of past data, thus reducing the impact of past data on the operating efficiency of the target detection system. Attached Figure Description

[0032] The invention will now be further described with reference to the accompanying drawings.

[0033] Figure 1 This is the overall system block diagram of the present invention;

[0034] Figure 2 This is a flowchart of the simulation module of the present invention;

[0035] Figure 3 This is a flowchart of the result recording module of the present invention. Detailed Implementation

[0036] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0037] like Figure 1-3 As shown, the object detection system based on deep learning according to the present invention includes:

[0038] A data tagging module is used to tag objects on the road and assign a unique number to each object.

[0039] A data storage module is used to save the data marked in the data marking module and classify it according to the category number of the marking.

[0040] A data processing module, which is used to analyze and process the stored data in the data storage module;

[0041] The data verification module is used to classify and organize the processed data and verify whether each data item is erroneous.

[0042] The simulation module uses data that has been verified to be complete and correct, and then imports the data into the program for simulation to confirm whether the target detection system can detect and identify the simulated target information.

[0043] The result recording module is used to save and record the data of the simulation, so that it can be imported into the autonomous driving system later to provide target detection for autonomous driving.

[0044] Example 1:

[0045] First, this object detection system is based on the classic deep learning detection framework—Faster R-CNN. A target detection model with high detection accuracy and running speed is designed on this framework.

[0046] To ensure the target detection system can accurately identify target objects and learn and optimize autonomously, it needs to accumulate a large number of raw data sources in its initial stage. This is achieved by using the data labeling module in the target detection system to identify and label external objects, thereby obtaining raw data sources. The data labeling module is based on the open-source tool LabelImg, which is compiled and installed in a Linux environment. By opening each image or 3D model, technicians first label the first batch of raw data sources so that the system can understand what these data sources specifically refer to. Then, these image data are stored in the data storage module. LabelImg is a fully graphical image annotation tool developed based on Python and Qt, which can directly convert the annotation information required for detection into XML files and save them.

[0047] Meanwhile, the data labeling module will classify the collected data into target category detection and target instance detection, so that the two different types of data are stored in independent storage layers to avoid interference between the two types of data during retrieval. At the same time, the stored data will be sent to the cloud disk for cloud storage to prevent data loss due to local hard drive failure, which would affect the normal operation of the target detection system.

[0048] The simulation module primarily uses the object detection framework Faster R-CNN for model training. Faster R-CNN is developed based on the popular deep learning framework Caffe, which is written in C++. Caffe is a clear and efficient deep learning framework with a pure C++ / CUDA architecture. It supports command line and provides interfaces for Matlab and Python, allowing seamless switching between CPU and GPU. The Caffe framework mainly consists of four major components: Blob, Layer, Net, and Solver. Blob uses the SyncedMemory class to store data. The data member shape_ stores the current dimension information of the blob, data_ points to the memory or GPU memory where the data is stored, and diff_ stores the gradient information of the network during backpropagation. Although blobs used in convolutional neural networks are basically 4-dimensional, blobs are not a fixed-dimensional data structure and are not limited to 4 dimensions. The `diff__at` function in the `Blob` class can be used to obtain the backpropagated loss, the `data_at` function can be used to obtain the data stored in memory, the `Reshape` function is used to change the size of the blob dimensions, and the `BlobProto` class is responsible for serializing the data in the blob into the Caffe model.

[0049] The design pattern used in Caffe is the factory pattern. Both Solver and Layer objects are created using this pattern. The Solver class is used to implement the optimization function in Caffe. That is, when training the model, this class is the first entry point to make some calls. The Solver class provides six optimization methods: SGD, AdaDelta, AdaGrad, Adam, Nesterov, and RMSProp.

[0050] Next, we analyze the Net class operations. This class mainly implements the forward and backward propagation of the convolutional neural network during training. It uses the ForwardBackward function to call the Forward and Backward functions to perform forward and backward propagation of the network. The ForwardFromTo function performs forward propagation from the start layer to the end layer, and the BackwardFromTo function implements backward propagation from the start layer to the end layer. The ToProto function mainly implements the serialization of the neural network to a disk file. Layer is the basic building block of Net, such as ReLU layer, Convolution layer, pooling layer, etc., each corresponding to a specific implementation of the Layer class. Based on the above framework, the autonomous deep learning of the object detection system is realized.

[0051] When the target detection system is simulated and verified, such as Figure 2 As shown, the data in the data storage module is first imported into the data verification module. Then, the data verification module classifies the imported data, using methods such as classifying by labels like cars, trees, and people. The data verification module then verifies each of these categories of data to ensure that the data has not been damaged or compromised by viruses in the storage module. This also ensures that the data has high stability during simulation, making it less likely to cause bugs during the simulation process. This ensures that the simulation module can successfully simulate the stored data, thereby verifying the accuracy and stability of the target detection system.

[0052] The validated data is imported into the simulation module for verification. The simulation module uses the Faster R-CNN object detection framework to simulate the data, maximizing the simulation of the real environment. This ensures that the labeled data can be stably output in practical applications, guaranteeing the normal operation of the object detection system. During simulation verification, if the data passes verification and can be accurately identified during the simulation process, it indicates that the labeled data is problem-free. Therefore, the simulation imports the relevant data into the result recording module for storage and automatically updates previous data. This ensures that the result recording module always stores the latest stable data, preventing data corruption during retrieval and reducing the memory usage of past data, thus minimizing the impact on the efficiency of the object detection system.

[0053] Example 2:

[0054] The results recording module stores the data passed in the simulation module, which can then be applied in the target detection system. At the same time, the target detection system is based on the deep learning framework Caffe and automatically performs deep learning to optimize the data in the results recording module, thereby ensuring the high efficiency of the target detection system.

[0055] For example, the results recording module stores data for more than one month, such as... Figure 3 As shown, the result recording module automatically extracts all data at the current time and then imports the data into the data verification module for preliminary verification to ensure that the data is intact. If data corruption occurs, an early warning will be issued immediately, allowing technicians to update the labeled data in a timely manner. This prevents the labeled data from causing the target detection system to fail to recognize the target properly during actual application, thus preventing automatic driving from malfunctioning.

[0056] If the data is intact and passes verification, the categorized data is then compared sequentially to identify completely duplicate data. Because the target detection system's detection algorithm is complex, a large amount of duplicate backup data is generated during computation to prevent data loss. This duplicate data is useless and redundant, ensuring a stable data source for the target detection system. When the categorized data is compared sequentially in the data comparison database, which contains only the latest labeled data, it can be compared with previously redundant and useless data. This comparison technique is developed using C++. If duplicate data is found, such as... Figure 3 As shown, the data filtering unit will automatically filter out this part of the past duplicate data and then re-import it to wait for the second round of filtering. If duplicates still appear, the data will be completely deleted, thereby reducing the memory usage of this data on the target detection system, improving the system's operating efficiency, and avoiding deleting the data in the first filtering. If a mistake is made, this part cannot be recovered, which will easily increase operating costs.

[0057] After duplicate data filtering is completed, the remaining data is saved again. Each saved data item is marked with a filter tag, so it will not be selected again in the next filtering. This prevents data from being filtered repeatedly, reduces the resource consumption of the target detection system, ensures the stable and fast operation of the target detection system, and improves the operational stability of the target detection system. At the same time, data marked with a filter tag will reduce the filtering frequency, thereby minimizing the occupation and consumption of system resources and ensuring the smoothness of the system.

Claims

1. A deep learning-based object detection system, characterized by, include: A data tagging module is used to tag objects on the road and assign a unique number to each object. A data storage module is used to save the data marked in the data marking module and classify it according to the category number of the marking. A data processing module, which is used to analyze and process the stored data in the data storage module; The data verification module is used to classify and organize the processed data and verify whether each data item is erroneous. The simulation module uses data that has been verified to be complete and correct, and then imports the data into the program for simulation to confirm whether the target detection system can detect and identify the simulated target information. During simulation verification, if the data verification passes and can be accurately identified during the simulation process, it indicates that the labeled data is correct. Therefore, the simulation will import the relevant data into the result recording module for storage. The result recording module is used to save and record the data of the simulation, so that it can be imported into the autonomous driving system later to provide target detection for autonomous driving. The simulation module uses the collected data to perform three-dimensional simulation, verify the recognition accuracy of the target detection system, identify problems in a timely manner during the simulation, optimize the efficiency of target recognition, and reduce the memory usage of the target detection system. The result recording module periodically filters the saved data and automatically removes redundant and useless data, thereby improving the running speed of the target detection system. The results recording module periodically compares the data and then uses different colors to identify different file categories. When two images are fitted, if the two images completely overlap and the color parameters are consistent, they are determined to be the same data. Therefore, the results recording module will automatically delete the data, thereby reducing the proportion of redundant and useless data occupying memory. The result recording module includes a data comparison library, which stores the latest data to be compared with redundant and useless data from the past, thereby enabling data filtering. 2.The deep learning based object detection system of claim 1, wherein: The data labeling module performs target detection based on a camera and a lidar. After the camera and lidar collect data, the data labeling module automatically labels regular objects.

3. The target detection system based on deep learning according to claim 2, characterized in that: The data tagging module stores unidentifiable objects in a separate folder, and technicians then manually add tags to these objects.

4. The target detection system based on deep learning according to claim 1, characterized in that: The data storage module is based on both cloud disk storage and local hard disk storage to prevent data loss or damage from being unable to find the original data source for replacement.

5. The deep learning-based target detection system according to claim 4, characterized in that: The data storage module has two independent storage layers, which are used to store target category detection data and target instance detection data respectively.

6. The target detection system based on deep learning according to claim 1, characterized in that: The data processing module is used to transform the tagged data in the data storage module into data that the data verification module can recognize.

7. The target detection system based on deep learning according to claim 1, characterized in that: The data verification module is used to verify data integrity to prevent incomplete data from entering the simulation module, which could lead to simulation failure and require a significant amount of time for troubleshooting.

Images