Method for improving AI pedestrian recognition efficiency of vehicle-mounted electronic rearview mirror

Abstract

The invention discloses a method for improving AI pedestrian recognition efficiency of a vehicle-mounted electronic rearview mirror, and aims to solve the problems of precision reduction, insufficient real-time performance and resource waste caused by NPU quantification in the prior art. According to the method, the boundary is set in the image acquired by the camera, and the AI pedestrian identification is limited to be only performed in the first area from the vehicle body to the boundary, so that the processing data volume is remarkably reduced, the calculation complexity is reduced, and the identification speed and the real-time performance are improved. Specifically, a YOLOv5 target detection AI algorithm is adopted, and a deep learning model trained for first area image data is preferably used, so that the recognition precision is improved. Meanwhile, an NPU module in the processor is used for accelerating operation. The boundary position can be dynamically adjusted through a remote controller to meet different scene requirements. In addition, the system can also record the boundary adjustment condition and retrain the model to optimize the recognition effect. According to the invention, the accuracy and efficiency of vehicle-mounted electronic rearview mirror AI pedestrian identification are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent vehicle technology, and more specifically, to a method for improving the efficiency of AI pedestrian recognition in a vehicle-mounted electronic rearview mirror. Background Art

[0002] At present, it has become a trend to use electronic rearview mirrors on large vehicles such as buses. In order to improve driving safety, many electronic rearview mirrors integrate artificial intelligence (AI) pedestrian recognition functions to assist drivers in identifying pedestrians around the vehicle.

[0003] In the prior art, AI pedestrian recognition solutions for vehicle-mounted electronic rearview mirrors usually use embedded processors and use their internal neural processing units (NPUs) to accelerate the operation of AI algorithms. Although the use of NPUs can significantly improve the processing speed of AI algorithms, NPUs usually need to perform quantization operations when processing models, which will lead to a loss of model accuracy and thus affect the accuracy of pedestrian recognition. For example, in actual applications, objects such as telephone poles, traffic cones, or trees beside the road may be mistakenly identified as pedestrians, thereby interfering with the driver's judgment.

[0004] In addition, the existing AI pedestrian recognition algorithms also have certain deficiencies in real-time performance. Some systems have difficulty in fully synchronizing the AI ​​recognition frame rate with the camera acquisition frame rate, which can easily lead to missed detections, especially when vehicles are driving fast or pedestrians are moving fast.

[0005] At the same time, traditional AI pedestrian recognition methods usually process the entire image range captured by the camera. However, when driving, drivers usually pay more attention to pedestrians who are closer to the vehicle. Identifying pedestrians who are farther away from the vehicle is relatively small in practical application, but it will take up a lot of computing resources, resulting in a waste of resources.

[0006] Therefore, a method is needed to improve the efficiency of AI pedestrian recognition in vehicle-mounted electronic rearview mirrors, so as to improve the accuracy and real-time performance of recognition and optimize the utilization of computing resources. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a method for improving the AI ​​pedestrian recognition efficiency of a vehicle-mounted electronic rearview mirror, so as to solve the problems existing in the prior art of reduced accuracy, insufficient real-time performance and waste of resources due to NPU quantization.

[0008] In order to achieve the above object, the present invention provides a method for improving the AI ​​pedestrian recognition efficiency of a vehicle-mounted electronic rearview mirror, comprising the following steps:

[0009] Step 1: Collect image data from the rear of the vehicle through a camera; this is the basis for subsequent pedestrian recognition, and the video image stream from the rear of the vehicle is obtained in real time through the camera installed on the vehicle.

[0010] Step 2: Set a dividing line in the image data to divide the image data into a first area from the vehicle body to the dividing line and a second area outside the dividing line; the present invention innovatively introduces the concept of a dividing line, which divides the image captured by the camera into two areas. The first area refers to the close-range area from the vehicle body to the dividing line, which is also the area where the present invention focuses on pedestrian recognition. The second area is the long-range area outside the dividing line, which is not the main processing object of AI pedestrian recognition in the present invention, thereby significantly reducing the amount of data that needs to be processed.

[0011] Step 3: Perform AI pedestrian recognition only on the image data of the first area; The key point of the present invention is that the AI ​​pedestrian recognition algorithm is limited to processing only the image data of the first area. Since the area of ​​this area is smaller than the global image, the amount of data that needs to be processed is greatly reduced, thereby reducing the computational complexity, improving the recognition speed, and reducing the impact of errors caused by factors such as quantization. Specifically, AI pedestrian recognition is performed in this step, specifically using the YOLOv5 target detection AI algorithm. The YOLOv5 algorithm is known for its high efficiency and accuracy, and can quickly and accurately identify pedestrian targets in images.

[0012] Step 4: Superimpose and display the identified pedestrian information on the display screen of the electronic rearview mirror; Superimpose and display the pedestrian information identified in step 3 (for example, the pedestrian's position coordinates, bounding box, etc.) on the display screen of the electronic rearview mirror in a visual manner so that the driver can intuitively understand the situation behind the vehicle and improve driving safety.

[0013] Furthermore, in order to improve the performance of the YOLOv5 target detection AI algorithm, the YOLOv5 target detection AI algorithm uses a deep learning model trained with a pedestrian detection database. The deep learning model is trained with a large amount of annotated pedestrian image data to learn the characteristics of pedestrians, so that pedestrian targets in the image can be more accurately identified. More preferably, the pedestrian detection database is trained with the image data of the first area collected during the test. This training method for a specific area enables the model to focus more on identifying pedestrians in close areas, further improving the accuracy and efficiency of recognition.

[0014] In order to accelerate the operation of AI pedestrian recognition and improve real-time performance, the NPU module inside the processor is used for AI pedestrian recognition in step 3. NPU (neural network processor) is a hardware accelerator specially designed for artificial intelligence algorithms such as deep learning, which can greatly improve the efficiency of AI calculations.

[0015] In order to meet the needs of different scenarios and improve the flexibility of use, the position of the dividing line can be adjusted dynamically in real time through the remote control. In this way, it can adapt to the driver's driving habits. It can also flexibly adjust the position of the dividing line through the remote control according to actual conditions, such as driving speed, surrounding environment, etc., so as to adjust the size of the area that needs to be AI recognized.

[0016] Furthermore, in order to enable the model to adapt to the optimal recognition effect under different dividing line settings, the present invention also provides a mechanism for dynamically updating the model: during multiple vehicle operations, the position of the dividing line adjusted by the remote control each time is recorded, and the position of the dividing line with the largest width in the first area is retained, and the image data with the largest width in the first area is used again to retrain the YOLOv5 target detection AI algorithm. This method determines the usage of different drivers on the vehicle, determines the maximum recognition range required on the corresponding vehicle, retrains the model, and ensures the recognition effect when the width of the first area is the largest. When other drivers adjust the first area to a smaller one, it can also meet the usage needs and even achieve higher and faster recognition efficiency.

[0017] By limiting the processing area of ​​AI pedestrian recognition, the present invention significantly reduces the amount of image data that needs to be processed, reduces the computational complexity, and improves the recognition speed and real-time performance; at the same time, by reducing the interference factors in the recognition area, the influence of quantization errors is alleviated, thereby improving the accuracy of pedestrian recognition. In addition, the function of dynamically adjusting the dividing line enables the present invention to adapt to different application scenarios and has higher practical value. DETAILED DESCRIPTION

[0018] The preferred embodiments of the present invention are described below. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0019] Example 1.

[0020] This embodiment aims to provide a specific method for improving the efficiency of AI pedestrian recognition in an electronic rearview mirror system for a bus. The bus is equipped with an electronic rearview mirror system, which includes: a camera, a processor, a vertically arranged display screen, and a remote control for user interaction. The processor integrates a GPU module and an NPU module. The display screen of the electronic rearview mirror is usually installed near the A-pillar in the cab.

[0021] Step 1: Image data acquisition

[0022] The camera installed at the rearview mirror of the bus collects video image data from the rear of the vehicle in real time. The camera is a high-resolution color camera that can capture clear rear images. The collected image data is transmitted in YUV format to reduce the amount of data. For example, the image resolution collected by the camera is 1080x1920 (width x height, to fit the vertical screen) and the frame rate is 30fps.

[0023] Step 2: Set the dividing line

[0024] The system provides the function of adjusting the dividing line. When the driver enters the dividing line adjustment mode by pressing a specific button on the remote control, a vertical dividing line will be superimposed on the display screen of the electronic rearview mirror. The dividing line is presented on the screen in the form of an adjustable vertical line.

[0025] The driver can adjust the horizontal position of the dividing line on the screen by using the left and right arrow keys of the remote controller. For example, the initial position of the dividing line can be set at the middle position of the horizontal direction of the display screen. The driver can move the dividing line to the left side of the screen by pressing the left arrow key on the remote controller; and move the dividing line to the right side of the screen by pressing the right arrow key.

[0026] The position of the dividing line is determined by the number of pixel columns on the display screen. For example, a display screen with a resolution of 1080x1920 has a horizontal pixel column number of 0 to 1079. The position of the dividing line can be represented by an integer value, for example, the dividing line is located at the 600th pixel column.

[0027] When the driver completes the dividing line adjustment and exits the adjustment mode, the dividing line will no longer be displayed on the screen to avoid affecting the driver's normal observation of the rear situation.

[0028] According to the position of the dividing line on the display screen, the image data collected by the camera is divided into two areas: the first area from the image edge body position to the pixel column where the dividing line is located, and the second area outside the dividing line. For example, if the dividing line is located at the 600th column of pixels and the body is located at the 50th column of pixels, the first area is the 50th to 599th column of pixels in the image.

[0029] Step 3: AI Pedestrian Recognition

[0030] The core of the present invention is to perform AI pedestrian recognition only on the image data of the first area. The specific process is as follows:

[0031] Image preprocessing: The GPU module inside the processor converts the YUV format image data collected by the camera into RGB format for display on the electronic rearview mirror screen. At the same time, the GPU module also crops the original image data and extracts the image data of the first area. The cropping range is determined by the current position of the vehicle body edge and the dividing line on the display screen.

[0032] AI model loading: The NPU module inside the processor loads the parameters of the pre-trained YOLOv5 target detection AI model. This model is a deep learning model trained using a database containing a large number of pedestrian images. It can identify pedestrian targets in the image and give their location information (for example, the coordinates of the bounding box).

[0033] Pedestrian detection: The NPU module inputs the cropped RGB image data of the first area into the loaded YOLOv5 model for inference operation. The YOLOv5 algorithm scans the image in the area, predicts the possible pedestrian targets, and outputs the detection results containing the coordinates of the pedestrian bounding box, confidence level, and other information. Since only part of the image is processed, the computational burden of the NPU is significantly reduced, and the recognition speed is accelerated.

[0034] Result screening: The YOLOv5 model may output multiple detection results. The system will screen them according to the confidence threshold and only retain the detection results with higher confidence to reduce false positives.

[0035] Step 4: Information Overlay Display

[0036] The processor converts the pedestrian detection results output by the NPU module, including information such as the pedestrian's bounding box coordinates, into graphic elements that can be displayed on the electronic rearview mirror screen. For example, a colored rectangular frame is used to select the recognized pedestrian and the rectangular frame is superimposed on the display of the electronic rearview mirror. Importantly, no matter where the dividing line is set, the pedestrian recognition results will be superimposed on the entire display screen for easy observation by the driver.

[0037] The training of the YOLOv5 target detection AI algorithm is completed by the following method.

[0038] Model training: In order to improve the accuracy of pedestrian recognition in the first area under different dividing line settings, we can collect image data of different first areas captured by the camera during the actual operation of the bus, build a special pedestrian detection database, and use this database to retrain the YOLOv5 model.

[0039] NPU acceleration: Utilize the parallel computing capability of the NPU module to accelerate the reasoning process of the YOLOv5 model and ensure the real-time performance of pedestrian recognition.

[0040] Dynamic dividing line adjustment: The driver can enter and exit the dividing line adjustment mode through specific buttons on the remote control, and adjust the horizontal position of the dividing line on the screen in real time through the left and right arrow keys.

[0041] Dynamic model update: The system records the various positions of the dividing line adjusted by the driver during multiple bus trips. The system retains the image data of the dividing line position with the largest width in the first area, and uses this data to regularly retrain the YOLOv5 model in the background. Of course, if circumstances permit, multiple models for different dividing line positions can also be trained based on the usage habits of different drivers.

[0042] It can be seen from the above embodiments that the present invention fully utilizes the computing power of the NPU by limiting the processing area of ​​AI pedestrian recognition (divided by a vertical dividing line on the vertical screen), and combines the dynamically adjustable dividing line and the model dynamic update mechanism to effectively improve the AI ​​pedestrian recognition efficiency and accuracy of the on-board electronic rearview mirror, which is more in line with the needs of actual bus driving scenarios.

[0043] This embodiment is only a preferred implementation mode of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for improving the AI ​​pedestrian recognition efficiency of a vehicle-mounted electronic rearview mirror, characterized in that: The following steps are involved: Step 1: Collect image data from behind the vehicle through a camera; Step 2: setting a dividing line in the image data, dividing the image data into a first area from the vehicle body to the dividing line and a second area outside the dividing line; Step 3: Perform AI pedestrian recognition only on the image data of the first area; Step 4: Overlay and display the identified pedestrian information on the display screen of the electronic rearview mirror; In step 3, AI pedestrian recognition is performed, specifically using the YOLOv5 target detection AI algorithm.

2. The method according to claim 1, characterized in that The YOLOv5 object detection AI algorithm uses a deep learning model trained on a pedestrian detection database.

3. The method according to claim 2, characterized in that The pedestrian detection database is obtained by training the image data of the first area collected during testing.

4. The method according to claim 1, characterized in that In step 3, the NPU module inside the processor is used to perform AI pedestrian recognition.

5. The method according to claim 1, characterized in that The position of the dividing line can be dynamically adjusted in real time via a remote controller.

6. The method according to claim 5, characterized in that During multiple vehicle operations, the position of the dividing line adjusted by the remote control each time is recorded, and the position of the dividing line with the largest width in the first area is retained, and the image data with the largest width in the first area is used again to retrain the YOLOv5 target detection AI algorithm.