A method for detecting electric vehicles and electric vehicle batteries based on TensorRT accelerated reasoning

Abstract

The present invention belongs to the field of artificial intelligence technology and relates to an electric vehicle and electric vehicle battery detection method based on TensorRT accelerated reasoning. First, a sample image is obtained and preprocessed to obtain a target data set; then a YOLOX-Nano model is trained according to the target data set, and after the model is tested, the YOLOX network parameter adjustment is used to obtain an optimal target detection model; then the JetsonNano 2GB development kit is system optimized, and the obtained optimal target detection model is encapsulated into the JetsonNano 2GB development kit; finally, TensorRT accelerated reasoning is used to detect and identify electric bicycles, electric motorcycles or electric vehicle batteries entering an elevator, and the elevator is controlled by Jetson-GPIO; electric vehicles and electric vehicle batteries are detected at the same time, fundamentally solving the problem of charging electric vehicles for high-rise residents, and the adopted training data set can be better applied in various elevator cars, is not affected by environmental changes, has low cost, high recognition rate, and stable equipment operation.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically is a method for restricting high-rise residents from charging their electric vehicles by controlling elevators, and in particular is a method for detecting electric vehicles and electric vehicle batteries based on TensorRT accelerated reasoning. Background Art

[0002] In recent years, the market ownership of electric vehicles has continued to increase. As the number of electric vehicles increases, public charging resources for electric vehicles are tight, and charging problems have also arisen. Due to the lack of outdoor charging piles and uneven charging prices, charging problems at home are relatively common. In order to prevent fires caused by charging electric vehicles at home, it is necessary to strictly control the entry of electric vehicles or electric vehicle batteries into homes.

[0003] With the development of artificial intelligence technology, the application of target detection is becoming more and more extensive. Today, when the Internet of Things technology has been applied to elevators to realize the Internet of Things elevators, artificial intelligence technology is combined with the Internet of Things elevators. By deploying AI models, elevator monitoring can detect electric vehicles and electric vehicle batteries, thereby controlling the operation of elevators and restricting high-rise residents from entering their homes with electric vehicles or charging electric vehicle batteries.

[0004] At present, in practical applications, the electric vehicle identification system in elevators usually uses a special data set for electric vehicles in the elevator car in the technical use of the AI ​​module, and the AI ​​model selects the basic traditional network training. For example, CN202010479416.1 discloses a method and system for elevator car anomaly detection for electric vehicle identification. The data set uses a special training data set for electric vehicles and electric vehicle pictures downloaded from the Baidu search engine. The YoloV3 network is used for model training. The electric vehicle training data set has undergone 1000 rounds of iterative optimization to improve the accuracy of the electric vehicle identification model; CN202011311209.1 discloses an elevator electric vehicle identification and warning method based on AI and HMM, which preprocesses the original image of the elevator monitoring video to obtain the preprocessed image; the electric vehicle selected in the data set of the elevator management method, device and system disclosed in CN201911043677.2 is an electric bicycle; however, there are still the following disadvantages in the selection of the data set: (1) Electric bicycles of different brands and sizes are not available. (1) There are differences in the types of electric vehicles, such as whether there are advertisements in the car or whether there are closed elevators and sightseeing elevators. The data background collected using elevator cars is single, and the anti-interference ability for special types of car environments is weak, which affects the accuracy of electric vehicle recognition; (2) Electric vehicles are complex and can be further divided into electric bicycles and electric motorcycles. Compared with bicycles, the common features of these two data samples are not obvious enough. Without classification training, the model is prone to misidentify bicycles, shopping carts or wheelchairs, etc.; (3) The state of electric vehicles in elevators is not unique. For example, in winter or rainy days, electric vehicles may be covered with windbreakers or raincoats, residents may block part of the electric vehicles, and the electric vehicles captured by surveillance may be from the front or the side. Comprehensive consideration is needed, otherwise the trained model will have a low recognition rate for electric vehicles in the above situations; YoloV3, MobileNet, FastRCNN and other networks are widely used and have relatively complete solutions for applications in different fields, but these models are old, and the trained models have low recognition rates, slow speeds and occupy resources.

[0005] In terms of hardware selection, most of them use open source Raspberry Pi or customized development boards to carry model deployment. CN202010479416.1 discloses a method and system for elevator car abnormality detection for electric vehicle identification. Its deployment device is Raspberry Pi. Although Raspberry Pi and other hardware are widely used in IoT hardware, they do not have CUDA acceleration function. As a target detection device for artificial intelligence applications, it cannot accelerate reasoning. At the same time, high resource usage causes device operation jams, affecting the stable operation of the entire system.

[0006] At present, the biggest hidden danger of electric vehicles entering households is that their batteries can easily catch fire when charging in a faulty state. Most electric vehicles on the market have detachable batteries. Simply identifying electric vehicles cannot solve the fundamental problem of battery charging at home. Therefore, there is an urgent need to design a method that can detect both electric vehicles and electric vehicle batteries. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art, and to provide a method for detecting electric vehicles and electric vehicle batteries based on TensorRT accelerated reasoning. Starting from data collection, an analysis method combining experiments and practice is adopted, and various situations are comprehensively considered to accurately detect electric vehicles and electric vehicle batteries, fundamentally solving the problem of charging electric vehicles and electric vehicle batteries for high-rise residents.

[0008] To achieve the above technical problem, the specific process of detecting electric vehicles and electric vehicle batteries of the present invention includes the following steps:

[0009] (1) Image preprocessing: Obtain sample images, screen and batch process them, and create a dataset in VOC format as the target dataset;

[0010] (2) Model training: Train the YOLOX-Nano model based on the target data set obtained in step (1), test the model, and use the YOLOX network to adjust parameters to obtain the optimal target detection model;

[0011] (3) Hardware optimization: System optimization of the Jetson Nano 2GB development kit. The Jetson Nano 2GB development kit uses the hardware device for artificial intelligence development released by NVIDIA at GTC2020;

[0012] (4) Model deployment: Convert the optimal target detection model obtained in step (2) into an onnx model, then convert it into an .engine model through DeepStream, and encapsulate the model into the JetsonNano 2GB development kit;

[0013] (5) Detection and Identification: Use TensorRT to accelerate reasoning to detect and identify electric bicycles, electric motorcycles, or electric vehicle batteries entering the elevator, and control the elevator through Jetson-GPIO.

[0014] As a further technical solution of the present invention, the sample images in step 1 are videos collected by monitoring in the elevator, pictures collected independently, and network pictures. The target data set construction steps are:

[0015] (1-1) Sample images are collected and screened from multiple aspects. The collection scenes are divided into indoor and outdoor scenes. Outdoor scenes are divided into streets and open-air parking lots. Indoor scenes are divided into elevator monitoring screenshots and corridors. The collection time is divided into morning and night. Night scenes are collected when the lights are on. In terms of target status, the target is collected in eight directions and three angles. The eight directions refer to east, west, south, north, southeast, northeast, southwest and northwest. The three angles refer to 0 degrees, 45 degrees and 90 degrees. For interference items, pedestrian occlusion and clothing occlusion are added.

[0016] (1-2) Filter the open source dataset VOC2012 dataset, retain only the bicycle category in the VOC2012 dataset, and modify the absolute path of the corresponding dataset label file through the xml tree;

[0017] (1-3) For the sample images in step (1-1) and the data set selected in step (1-2), VOC formatting is performed according to the format of the VOC data set to obtain sample images of four types of targets: electric bicycles, electric motorcycles, bicycles, and electric vehicle batteries;

[0018] (1-4) After the sample images of the four types of targets obtained in step (1-3) are arranged in a ratio of 1:1:1:1, they are divided into a training set, a verification set, and a test set in a ratio of 6:2:2.

[0019] As a further technical solution of the present invention, the source of the network pictures is the seller comment pictures under the links of electric bicycles, electric motorcycles, bicycles and electric vehicle batteries on various e-commerce platforms, and the crawler keywords are electric bicycles, electric motorcycles, bicycles, batteries and electric vehicle batteries.

[0020] As a further technical solution of the present invention, the model training steps in step (2) are as follows:

[0021] (2-1) Select pytorch1.8 as the training environment, and rewrite yolox_voc_nano.py according to the nano.py file under the custom file in YOLOX, so that the YOLOX network can train the Nano model;

[0022] (2-2) Import the training set and validation set sample data divided in step (1-4) into the modified YOLOX network for model training to obtain the target detection model in .pth format;

[0023] (2-3) Import the test set sample data divided in step (1-4) into the modified YOLOX network, test the performance of the target detection model obtained in step (2-2), and modify the network parameters according to the training data statistics in step 2-2. Finally, determine the learning rate as:

[0024] ,

[0025] Among them, lr is the learning rate, batch_size is the number of samples captured in one training, and the optimal target detection model is obtained after adjusting the parameters and retraining.

[0026] As a further technical solution of the present invention, the packaging device in step (3) is a Jetson Nano 2GB device with JetPack 4.5.1 installed. Since the memory and video memory of the Jetson Nano device are shared, the virtual memory SWAPFile is first added to increase the performance, the pip3 tool is installed and updated, and unnecessary software such as office and related dependencies are uninstalled. The csi camera script is written and the camera is tested.

[0027] As a further technical solution of the present invention, the specific steps of model deployment in step (4) are as follows:

[0028] (4-1) Convert the optimal target detection model obtained in step (2-3) into .onnx format using the onnx conversion tool;

[0029] (4-2) Create the YOLOX-DeepStream tool based on the DeepStream tool integrated in JetPack 4.5.1, and convert the .onnx model in step (4-1) into a .engine model that supports TensorRT accelerated reasoning through YOLOX-DeepStream;

[0030] (4-3) Use DeepStream to encapsulate the .engine model in step (4-2);

[0031] (4-4) Activate the Jetson-GPIO tool, output high and low levels through GPIO, and connect to the elevator control panel.

[0032] As a further technical solution of the present invention, the specific process of detection and identification in step (5) is:

[0033] (5-1) Use the signal.txt file as a flag file to detect whether the target exists. After JetsonNano 2GB is started, it is initialized first. After deleting the signal.txt file, the CSI camera is started to capture the scene of the elevator car;

[0034] (5-2) When the camera captures an electric bicycle, electric motorcycle or electric vehicle battery, Jetson-GPIO is called, and the high and low level status of the GPIO interface is determined based on whether signal.txt exists;

[0035] (5-3) If the signal.txt file does not exist, that is, the demand target is detected for the first time at the current time point, the GPIO pulls the level high from the 12 pin to control the elevator car, and the elevator door cannot be closed and run. At this time, a warning audio is played to remind the illegal residents in the elevator car;

[0036] (5-4) Continue to monitor the elevator car through the CSI camera. At this time, if the resident moves the electric bicycle, electric motorcycle or electric vehicle battery out of the elevator car, the GPIO level state will return to a low level and the elevator will resume normal operation;

[0037] (5-5) If the resident has not removed the battery of the electric bicycle, electric motorcycle or electric vehicle from the elevator car, the marker file signal.txt will be retained and the GPIO will still maintain a high level output.

[0038] The present invention is equally applicable to automatic doors.

[0039] Compared with the prior art, the present invention has the following advantages: first, it uses TensorRT to accelerate reasoning, which greatly improves the recognition accuracy and recognition speed; second, in addition to detecting electric vehicles, it can also detect electric vehicle batteries, fundamentally solving the problem of charging electric vehicles for high-rise residents; and the training data set uses a general data set rather than a dedicated data set, which can be better applied in various elevator cars and is not affected by environmental changes; the selected JetsonNano 2GB hardware has better performance, lower cost, high recognition rate, and stable equipment operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the working principle of the present invention;

[0041] Figure 2 A diagram of some data samples collected by an embodiment of the present invention;

[0042] Figure 3 This is a flow chart of detection and recognition using TensorRT to accelerate reasoning as described in the present invention. DETAILED DESCRIPTION

[0043] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is described in detail below in conjunction with the accompanying drawings and embodiments. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] Example:

[0045] The electric vehicle and electric vehicle battery detection process based on TensorRT accelerated reasoning described in this embodiment is specifically implemented by the following steps:

[0046] Step 1: In the image preprocessing stage, sample images are obtained through videos collected by elevator monitoring, images collected independently, and images from the Internet. Sample images are screened and batch processed to create a data set in VOC format.

[0047] Step 2: In the model training phase, the YOLOX-Nano model is trained based on the data set obtained in step 1. After testing the model, the YOLOX network is used to adjust the parameters to obtain the optimal target detection model.

[0048] Step 3: Hardware optimization stage, perform system optimization on the JetsonNano 2GB development kit;

[0049] Step 4: In the model deployment phase, the model trained in step 2 is first converted into an onnx model, then converted into an .engine model through DeepStream, and the model is encapsulated into the JetsonNano 2GB development kit;

[0050] Step 5: In the detection and recognition stage, use TensorRT to accelerate reasoning to detect and identify electric bicycles, electric motorcycles, or electric vehicle batteries entering the elevator, and control the elevator through Jetson-GPIO.

[0051] Specifically, the source of the network images in step 1 is the seller's comment images under the links of electric bicycles, electric motorcycles, bicycles and electric vehicle batteries on various e-commerce platforms, and the crawler keywords are electric bicycles, electric motorcycles, bicycles, batteries and electric vehicle batteries; the source and production steps of the data set are as follows:

[0052] Step 1-1: The self-sampled images are divided into indoor and outdoor scenes. Outdoor scenes are divided into streets and open-air parking lots, and indoor scenes are divided into elevator monitoring screenshots and corridors. The collection time is divided into morning and night, and the night is conducted in an environment with lights on. In terms of target state, the target is collected in 8 directions and 3 angles (0 degrees, 45 degrees and 90 degrees). For interference items, additional states such as pedestrian occlusion and clothing occlusion are added;

[0053] Step 1-2: Filter the VOC2012 dataset, retain only the bicycle category in the VOC2012 dataset, and modify the absolute path of the corresponding dataset label file through the xml tree;

[0054] Step 1-3: For the images filtered by 31-1 and the data set filtered by step 1-2, VOC formatting is performed according to the format of the VOC data set to obtain sample images of four types of targets: electric bicycles, electric motorcycles, bicycles, and electric vehicle batteries; some of the images obtained in this embodiment are as follows Figure 2 As shown;

[0055] Step 1-4: Arrange the sample images of the four types of targets obtained in step 1-3 in a ratio of 1:1:1:1, and then divide them into training set, verification set and test set in a ratio of 6:2:2.

[0056] Specifically, the model training steps in step 2 are as follows:

[0057] Step 2-1: The training environment is pytorch1.8, and the YOLOX network is modified to enable it to train the Nano model;

[0058] Step 2-2: Import the training set and validation set sample data divided in step 1-4 into the YOLOX network for model training to obtain the target detection model in .pth format;

[0059] Step 2-3: Import the test set sample data divided in step 1-4 into the network to test the performance of the target detection model obtained in step 2-2. At the same time, modify the YOLOX network parameters according to the training data statistics in step 2-2, and finally determine the learning rate as follows:

[0060] ,

[0061] Among them, lr is the learning rate, batch_size is the number of samples captured in one training, and finally the optimal target detection model is obtained after parameter adjustment and retraining.

[0062] Specifically, in step 3, for the Jetson Nano 2GB device with JetPack 4.5.1 installed, since the Jetson Nano device shares memory and video memory, you need to first increase the virtual memory SWAPFile to increase performance, install and update tools such as pip3, uninstall unnecessary software such as office and related dependencies, write the csi camera script and test the camera.

[0063] Specifically, the model deployment steps in step 4 are as follows:

[0064] Step 4-1: For the optimal target detection model obtained in step 2-3, convert the optimal target detection model into .onnx format using the onnx conversion tool;

[0065] Step 4-2: Create the YOLOX-DeepStream tool based on the DeepStream tool integrated in JetPack 4.5.1, and convert the .onnx model in step 4-1 into a .engine model that supports TensorRT accelerated reasoning through YOLOX-DeepStream;

[0066] Step 4-3: Use DeepStream to encapsulate the .engine model in step 4-2;

[0067] Step 4-4: Activate the Jetson-GPIO tool, output high and low levels through GPIO, and link with the elevator control panel.

[0068] Specifically, the detection and recognition process based on TensorRT accelerated reasoning includes the following steps:

[0069] Step 5-1: signal.txt is a flag file for detecting whether the target exists. After the device is started, it is initialized first, the signal.txt file is deleted, and then the CSI camera is started to capture the scene of the elevator car;

[0070] Step 5-2: When the camera captures an electric bicycle, electric motorcycle or electric vehicle battery, Jetson-GPIO is called, and the high and low level status of the GPIO interface is determined based on whether signal.txt exists;

[0071] Step 5-3: If the signal.txt file does not exist, that is, the demand target is detected for the first time at the current time point, GPIO pulls the level high from pin 12 to control the elevator car, and the elevator door cannot be closed and run. At this time, a warning audio is played to remind the illegal residents in the car;

[0072] Step 5-4: Continue to monitor the elevator car through the CSI camera. At this time, if the resident moves the electric car or battery out of the elevator car, the GPIO level status returns to a low level and the elevator resumes normal operation;

[0073] Step 5-5: If the resident has not moved the electric vehicle or battery out of the elevator car, the marker file signal.txt is retained and the GPIO still maintains a high level output.

[0074] Finally, it should be noted that the functional devices in this example can be integrated into one processing device, or two or more devices can be integrated into one device.

[0075] The functions described in this embodiment may exist in the form of hardware or in the form of software in a computer-readable storage device. Based on such an existence mode, the part of the technical solution of this embodiment that contributes to the prior art or the part of the technical solution of the present invention may be embodied in the form of a software product. The computer software product is stored in a computer-readable storage medium, which includes several instruction sets and can enable a computer device, including a server, an Internet of Things device or a network device, to execute all or part of the steps of the methods described in each example of the present invention.

[0076] This embodiment extends the recognition of electric vehicle batteries based on the recognition of electric vehicles and bicycles, and trains the model with a general data set, so that the hardware equipment or software solutions can be applied to scenarios other than elevator cars. For example, residential buildings with automatic doors can also control the automatic doors through the solution described in this example, thereby completing the function of restricting electric vehicles or electric vehicle batteries from entering residential buildings.

[0077] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any changes and substitutions made by any technician familiar with the art within the technical scope disclosed by the present invention using the contents of the drawings of the present invention specification, or directly or indirectly applied in other related technical fields, should be covered within the patent protection scope of the present invention.

Claims

1. A method for detecting electric vehicles and electric vehicle batteries based on TensorRT accelerated reasoning, characterized in that: The following steps are involved: (1) Image preprocessing: Obtain sample images, screen and batch process them, and create a dataset in VOC format as the target dataset; (2) Model training: Train the YOLOX-Nano model based on the target data set obtained in step (1), and after testing the model, use the YOLOX network to adjust the parameters to obtain the optimal target detection model. Specifically: (2-1) Select pytorch1.8 as the training environment, and rewrite yolox_voc_nano.py according to the nano.py file under the custom file in YOLOX, so that the YOLOX network can train the Nano model; (2-2) Import the training set and validation set sample data divided in step (1-4) into the modified YOLOX network for model training to obtain the target detection model in .pth format; (2-3) Import the test set sample data divided in step (1-4) into the modified YOLOX network, test the performance of the target detection model obtained in step (2-2), and modify the network parameters according to the training data statistics in step 2-2. Finally, determine the learning rate as: ， Among them, lr is the learning rate, batch_size is the number of samples captured in one training, and the optimal target detection model is obtained after retraining after adjusting the parameters; (3) Hardware optimization: System optimization of the JetsonNano 2GB development kit; (4) Model deployment: Convert the optimal target detection model obtained in step (2) into an onnx model, then convert it into an .engine model through DeepStream, and encapsulate the model into the JetsonNano 2GB development kit; (5) Detection and Identification: Use TensorRT to accelerate reasoning to detect and identify electric bicycles, electric motorcycles, or electric vehicle batteries entering the elevator, and control the elevator through Jetson-GPIO.

2. The electric vehicle and electric vehicle battery detection method based on TensorRT accelerated reasoning according to claim 1 is characterized in that: The sample images in step (1) are videos collected by elevator monitoring, self-collected images, and network images. The target dataset is constructed as follows: (1-1) Sample images are collected and screened from multiple aspects. The collection scenes are divided into indoor and outdoor scenes. Outdoor scenes are divided into streets and open-air parking lots. Indoor scenes are divided into elevator monitoring screenshots and corridors. The collection time is divided into morning and night. Night scenes are collected when the lights are on. In terms of target status, the target is collected in eight directions and three angles. The eight directions refer to east, west, south, north, southeast, northeast, southwest and northwest. The three angles refer to 0 degrees, 45 degrees and 90 degrees. For interference items, pedestrian occlusion and clothing occlusion are added. (1-2) Filter the open source dataset VOC2012 dataset, retain only the bicycle category in the VOC2012 dataset, and modify the absolute path of the corresponding dataset label file through the xml tree; (1-3) For the sample images in step (1-1) and the data set selected in step (1-2), VOC formatting is performed according to the format of the VOC data set to obtain sample images of four types of targets: electric bicycles, electric motorcycles, bicycles, and electric vehicle batteries; (1-4) After the sample images of the four types of targets obtained in step (1-3) are arranged in a ratio of 1:1:1:1, they are divided into a training set, a verification set, and a test set in a ratio of 6:2:

2.

3. The electric vehicle and electric vehicle battery detection method based on TensorRT accelerated reasoning according to claim 2 is characterized in that: The JetsonNano 2GB development kit described in step (3) uses hardware devices for artificial intelligence development. First, add virtual memory SWAPFile, install and update the pip3 tool, uninstall the office software and related dependencies, write the csi camera script and test the camera.

4. The electric vehicle and electric vehicle battery detection method based on TensorRT accelerated reasoning according to claim 3 is characterized in that: The specific steps for model deployment in step (4) are as follows: (4-1) Convert the optimal target detection model obtained in step (2-3) into .onnx format using the onnx conversion tool; (4-2) Create the YOLOX-DeepStream tool based on the DeepStream tool integrated in JetPack 4.5.1, and convert the .onnx model in step (4-1) into a .engine model that supports TensorRT accelerated reasoning through YOLOX-DeepStream; (4-3) Use DeepStream to encapsulate the .engine model in step (4-2); (4-4) Activate the Jetson-GPIO tool, output high and low levels through GPIO, and connect to the elevator control panel.

5. The electric vehicle and electric vehicle battery detection method based on TensorRT accelerated reasoning according to claim 4 is characterized in that: The specific process of detection and identification in step (5) is as follows: (5-1) Use the signal.txt file as a flag file to detect whether the target exists. After JetsonNano 2GB is started, it is initialized first. After deleting the signal.txt file, the CSI camera is started to capture the scene of the elevator car; (5-2) When the camera captures an electric bicycle, electric motorcycle or electric vehicle battery, Jetson-GPIO is called, and the high and low level status of the GPIO interface is determined based on whether signal.txt exists; (5-3) If the signal.txt file does not exist, that is, the demand target is detected for the first time at the current time point, the GPIO pulls the level high from the 12 pin to control the elevator car, and the elevator door cannot be closed and run. At this time, a warning audio is played to remind the illegal residents in the elevator car; (5-4) Continue to monitor the elevator car through the CSI camera. At this time, if the resident moves the electric bicycle, electric motorcycle or electric vehicle battery out of the elevator car, the GPIO level state will return to a low level and the elevator will resume normal operation; (5-5) If the resident has not removed the battery of the electric bicycle, electric motorcycle or electric vehicle from the elevator car, the marker file signal.txt will be retained and the GPIO will still maintain a high level output.

Images