A blind spot monitoring method based on 360-degree surround view camera

Abstract

This invention relates to a blind spot monitoring method based on a 360-degree surround-view camera. The method includes steps such as data acquisition, generating a top-down view of a designated area, identifying objects using the dark spot variance method, continuously tracking the object, reporting the object's condition, and determining whether to exit the system. This method belongs to the field of automotive image processing technology. The purpose of this invention is to address the problem that existing products, which mostly use radar and other detection equipment for blind spot monitoring, increase initial costs. To reduce this cost, this solution provides a blind spot monitoring method based on a 360-degree surround-view camera.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile on-board image processing, and in particular to a blind spot monitoring method based on a 360-degree surround view camera. Background Art

[0002] The blind spot monitoring system, also known as the lane change assist system, is a high-tech safety configuration on the car. Its main function is to eliminate the blind spots of the rearview mirrors. It uses microwave radar and other equipment to detect overtaking vehicles in the blind spots of the rearview mirrors on both sides of the vehicle, and remind the driver to avoid accidents due to the blind spots of the rearview mirrors during lane changes.

[0003] Traditional blind spot monitoring systems are implemented through detection equipment such as microwave radar, which has a short detection distance and increases additional hardware costs. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem that the monitoring of blind spots in existing products is mostly achieved by using detection equipment such as radar, which increases the original cost. In order to reduce this part of the cost, this solution provides a blind spot monitoring method based on a 360-degree surround view camera.

[0005] A blind spot monitoring method based on a 360-degree surround view camera, the blind spot monitoring method comprises the following steps:

[0006] 1.1. Data collection: Use the 360-degree surround view camera to collect a frame of real-time image data from the rear left camera, rear right camera, front left camera, and front right camera respectively;

[0007] 1.2. Generate a bird's-eye view of the specified area. According to the camera calibration parameters, convert the physical coordinates of each point in the specified ground area into image coordinates. Then, based on the image coordinates, collect the original image to generate a bird's-eye view of the specified area.

[0008] 1.3、 Dark point variance method to find objects

[0009] 1.3.1. Divide the top view into blocks by row. Assume that the current row is n, and the corresponding block is all pixels in rows nM to n+M, where M is 5.

[0010] 1.3.2. Calculate the dark point variance of each block to form a dark point variance array. The dark point variance refers to the variance of all dark points. Dark points refer to the points remaining after excluding the brightest 1 / 3 points in the block.

[0011] 1.3.3. Based on the dark spot variance array, select continuous blocks with values ​​greater than X as objects. X is set to 0.18. The object length is the distance between the head and tail of the object. If the object length is less than 1 meter, the object is invalid. Otherwise, the object is valid and saved to the temporary object array.

[0012] 1.4. Continuous Object Tracking

[0013] 1.4.1. Initialize the valid object queue upon first entry;

[0014] 1.4.2. For each valid object in the queue, find the temporary object closest to its head and tail. If the distance is less than 2 meters, it is a match. Use Kalman filtering to calculate the head and tail moving speeds. Otherwise, remove the object from the valid object queue.

[0015] 1.4.3. If there is a temporary object that is not matched with a valid object, it is considered a new valid object, created and added to the valid object queue;

[0016] 1.5. Report based on the object's condition. For each valid object, determine whether it has moved forward continuously for N frames and the moving distance is greater than 1 meter based on its first or last Kalman filter moving speed. If it has moved forward continuously for N frames and the moving distance is greater than 1 meter, report it.

[0017] 1.6. Determine whether to exit. If the valid object meets the requirements of continuous forward movement of N frames and the moving distance is greater than 1 meter, then exit data collection. Otherwise, jump to step 1.1 and re-collect data.

[0018] Furthermore, in the step 1.5, in the reporting according to the object situation, N is 5.

[0019] The beneficial effect of adopting this technical solution is that, based on the 360-degree surround view system, without increasing hardware costs, real-time image data collected from the vehicle's rear left, rear right, front left, and front right cameras is processed through software algorithms to simultaneously monitor the vehicle's left and right blind spots. The effective monitoring area is also larger than that of traditional microwave radar, saving the original left and right monitoring hardware equipment and saving costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of the method of the present invention. DETAILED DESCRIPTION

[0021] The following further describes the present invention with reference to the accompanying drawings and preferred embodiments of the present invention. The examples are only used to explain the present invention and are not intended to limit the scope of the present invention.

[0022] like Figure 1 As shown in , a blind spot monitoring method based on a 360-degree surround view camera, the steps of the blind spot monitoring method are as follows:

[0023] 1.1. Data collection: The 360-degree surround view camera collects a frame of real-time image data from the vehicle's rear left camera, rear right camera, front left camera, and front right camera respectively;

[0024] 1.2. Generate a bird's-eye view of the specified area. According to the camera calibration parameters, convert the physical coordinates of each point in the specified ground area into image coordinates. Then, based on the image coordinates, collect the original image to generate a bird's-eye view of the specified area.

[0025] 1.3、 Dark point variance method to find objects

[0026] 1.3.1. Divide the top view into blocks by row. Assume that the current row is n, and the corresponding block is all pixels from rows nM to n+M. M defaults to 5. The relative distance between each block and the actual vehicle is fixed.

[0027] 1.3.2. Calculate the variance of dark points for each block to form a dark point variance array. The dark point variance refers to the variance of all dark points. Dark points are those remaining after excluding the brightest 1 / 3 of the points in the block. Only dark points are included to eliminate the influence of lane lines.

[0028] 1.3.3. Based on the dark spot variance array, select continuous blocks with values ​​greater than X as objects. X is set to 0.18. The object length is the distance between the head and tail of the object. If the object length is less than 1 meter, the object is invalid. Otherwise, the object is valid and saved to the temporary object array.

[0029] 1.4. Continuous Object Tracking

[0030] 1.4.1. Initialize the valid object queue upon first entry;

[0031] 1.4.2. For each valid object in the queue, find the temporary object closest to its head and tail. If the distance is less than 2 meters, it is a match. Use Kalman filtering to calculate the head and tail moving speeds. Otherwise, remove the object from the valid object queue.

[0032] 1.4.3. If there is a temporary object that is not matched with a valid object, it is considered a new valid object, created and added to the valid object queue;

[0033] 1.5. Report based on the object's condition. For each valid object, determine whether it has moved forward continuously for N frames and the moving distance is greater than 1 meter based on its first or last Kalman filter moving speed. If it has moved forward continuously for N frames and the moving distance is greater than 1 meter, report it.

[0034] 1.6. Determine whether to exit. If the valid object meets the requirements of continuous forward movement of N frames and the moving distance is greater than 1 meter, then exit data collection. Otherwise, jump to step 1.1 and re-collect data.

[0035] Specifically, since blind spot monitoring needs to monitor vehicles behind and to the sides, in this solution, the four ground areas of the vehicle's rear left, rear right, left and right are agreed upon as designated areas, among which the rear left and rear right areas of the vehicle extract images from the rear camera, and the left and right areas of the vehicle extract images from the left camera and right camera of the vehicle respectively.

[0036] By capturing a frame of real-time image data from the vehicle's rear left, rear right, front left, and front right cameras and carefully observing the image of the ground area, it can be seen that if there are no vehicles or other objects entering, the brightness of the image block is relatively uniform. If a vehicle or other object enters, the brightness of this image block becomes uneven. Therefore, a vehicle overtaking in the blind spot can be treated as a long, uneven image block that continuously moves forward relative to the vehicle. This allows for blind spot monitoring.

[0037] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A blind spot monitoring method based on a 360-degree surround view camera, characterized in that the steps of the blind spot monitoring method are as follows: 1.

1. Data Collection: The 360-degree surround view camera collects a frame of real-time image data from the vehicle's rear left camera, rear right camera, front left camera, and front right camera respectively; 1.

2. Generate a bird's-eye view of the specified area: According to the camera calibration parameters, the physical coordinates of each point in the specified ground area are converted into image coordinates. Based on the image coordinates, the original image is collected to generate a bird's-eye view of the specified area. 1.3、 Dark point variance method to find objects 1.3.

1. Divide the top view into blocks by row. Assume that the current row is n, and the corresponding block is all pixels in rows nM to n+M, where M is 5. 1.3.

2. Calculate the dark point variance of each block to form a dark point variance array. The dark point variance refers to the variance of all dark points. Dark points refer to the points remaining after excluding the brightest 1 / 3 points in the block. 1.3.

3. Based on the dark spot variance array, select continuous blocks with values ​​greater than X as objects. X is set to 0.

18. The object length is the distance between the head and tail of the object. If the object length is less than 1 meter, the object is invalid. Otherwise, the object is valid and saved to the temporary object array. 1.

4. Continuous Object Tracking 1.4.

1. Initialize the valid object queue upon first entry; 1.4.

2. For each valid object in the queue, find the temporary object closest to its head and tail. If the distance is less than 2 meters, it is a match. Use Kalman filtering to calculate the head and tail moving speeds. Otherwise, remove the object from the valid object queue. 1.4.

3. If there is a temporary object that is not matched with a valid object, it is considered a new valid object, created and added to the valid object queue; 1.

5. Reporting based on object status: For each valid object, determine whether it has moved forward continuously for N frames and the moving distance is greater than 1 meter based on its first or last Kalman filter moving speed. If it has moved forward continuously for N frames and the moving distance is greater than 1 meter, then report it; 1.

6. Determine whether to exit. If the valid object meets the requirements of continuous forward movement of N frames and the moving distance is greater than 1 meter, then exit data collection. Otherwise, jump to step 1.1 and re-collect data.

2. The blind spot monitoring method based on a 360-degree surround view camera according to claim 1, characterized in that: In the step 1.5, the object condition is reported, and N is 5.

Images