A method for obtaining a traffic signal light blocked by a bus

Abstract

The present invention proposes a method for obtaining a traffic light blocked by a bus, comprising: step A, obtaining an image through a front camera and performing traffic light recognition; step B, obtaining the blocked area behind the bus in real time; step C, transmitting the status and information of the traffic light. When the reversing radar detects that a vehicle has entered the blocked area, the vehicle system transmits the status information of the traffic light identified in step A to the on-board display screen behind the bus in real time to prompt the rear vehicle. The present invention has low computational complexity and low requirements for the processor. The built-in chip of the bus's built-in vehicle system can meet the computational requirements; there is no need to install expensive depth, TOF, radar and other sensors; the ranging accuracy is high, and there is no other three-dimensional scene reconstruction or drift and other cumulative errors.

Description

Technical Field

[0001] The invention belongs to the technical field of vehicles, and in particular relates to a solution to a bus blocking a traffic light based on a vehicle computer system. Background Art

[0002] At present, when some large vehicles (such as buses) are driving on the road, due to their high bodies, they will block the sight of small vehicles driving behind them, resulting in the rear vehicles not being able to see the traffic lights. This phenomenon seriously threatens the road driving safety of small vehicles driving behind.

[0003] The invention with application number 201810789400.3 discloses a traffic light recognition and warning device for large vehicles, including a camera, an LED display, a speaker, an ultrasonic vehicle distance probe and a host, which is characterized in that: the camera is installed in the center of the top of the large vehicle, the LED display, the speaker and the ultrasonic vehicle distance probe are installed at the rear of the large vehicle, the host is installed near the driver's seat, and the camera, the LED display, the speaker, and the ultrasonic vehicle distance probe are all electrically connected to the host through cables. The method can identify the status of the traffic light in front of the vehicle and transmit it to the rear, so as to help the owner of the rear vehicle understand the status of the traffic light in front in real time.

[0004] In addition, the invention with application number 201910866296.8 discloses a method and device for displaying traffic lights. The method is applied to a vehicle, and a display module is provided at the rear of the vehicle. The method may include: obtaining status information of the traffic lights in front of the vehicle; displaying the status information on the display module at the rear of the vehicle so that traffic participants behind the vehicle can obtain the status information of the traffic lights through the display module.

[0005] The above methods all use a preset distance to remind the rear vehicle with a signal light, that is, by presetting a constant value, and then judging whether there is an obstruction based on the distance (for example, if the distance between the two vehicles is less than the preset value, it is judged as an obstruction). In fact, the real-time changes in road slope, the height changes of different traffic lights, the height changes of the two vehicles, the real-time changes in the speed of the two vehicles, and the real-time changes in the distance between the two vehicles will affect the preset value. In the prior art, defining the preset value as a constant is bound to affect the detection accuracy. Summary of the invention

[0006] The present invention proposes a method for obtaining traffic lights, which can realize real-time detection of traffic lights, buses in front, and cars behind in motion, and accurately determine whether there is a line of sight obstruction for the vehicle behind, so as to timely transmit traffic light information to the vehicle behind. The scheme is:

[0007] Step A: Acquire an image through the front camera and perform traffic light recognition;

[0008] Step B: obtaining the blocked area behind the bus in real time;

[0009] B1, obtain the height difference h1 between the front camera installation position and the reversing radar position, the height difference h2 from the top of the bus to the reversing radar position, and the horizontal distance S0 between the front camera installation position and the reversing radar position;

[0010] B2. Obtain the elevation angle α between the center point of the front camera and the traffic light, α=arctan(S / f), where f is the imaging focal length of the front camera, and S is the vertical distance from the imaging of the traffic light to the horizontal line of the center of the image;

[0011] B3. After a certain time interval Δt, take step B2 again to obtain the second sampling data. The two samplings are used to calculate the elevation angle α between the center point of the front camera and the traffic light. 1 and α 2 At the same time, the vehicle system is used to obtain the real-time vehicle speed V 车 , and ∫(V 车 *Δt) Get the distance S that the bus traveled during the time interval between two photos taken by the front camera 行；

[0012] Through the above data, the horizontal distance S between the front camera and the traffic light is obtained. 灯 , S 灯 =(S 行

[0013] *tanα 2 ) / (tanα 1 -tanα 2 ); Get the vertical distance h between the traffic light and the front camera 灯 ,h 灯 =tan(α 2 )*(S 行 *tanα 2 ) / (tanα 1 -tanα 2 );

[0014] B4. Get the occluded area S behind the bus 遮挡 , S 遮挡 =(S 灯 +S 0 )*h 2 / (h 1 +h 2 );

[0015] Step C: Transmission of traffic light status and information: When the reversing radar detects that a vehicle has entered the blocked area, the vehicle computer system transmits the traffic light status information identified in step A to the on-board display screen at the rear of the bus in real time to alert the vehicles behind.

[0016] Furthermore, the traffic light recognition in step A includes traffic light color recognition and number recognition.

[0017] Furthermore, the traffic light color recognition includes: first, converting the RGB value into the corresponding hue, saturation and value (HSV) coordinates; second, using the corresponding color feature value, searching the sensitive area to identify the corresponding color area, and completing the construction of the structural element in combination with the appearance characteristics of the traffic light; third, binarizing the constructed structural element and filling the corresponding background color to complete the recognition process.

[0018] Compared with the prior art, the advantages and positive effects of the present invention are as follows: the present invention can complete the traffic light recognition, obtain the blocked area behind the bus in real time and accurately, and make corresponding reminders by relying only on the bus's own onboard system, front camera, reversing radar, rear display screen and other components. No additional components or modules are needed. Without making any hardware improvements to the existing buses, the method of the present invention can enable the bus to have the function of intelligently reminding the traffic lights of the blocked vehicles behind. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a diagram showing the relationship between the elevation angles of the front camera and the traffic light of the present invention;

[0020] Figure 2 is a distance relationship diagram between the front camera and the traffic light of the present invention;

[0021] Figure 3 It is a schematic diagram of the shielding area behind the bus of the present invention. DETAILED DESCRIPTION

[0022] Design process of the present invention:

[0023] Initially, the height of the bus and the height of the traffic light are set to be constant. The rear vehicle is an SUV with a height of 1.5 meters. Assuming that the distance to the bus is 5 meters, the SUV is just in the critical state of being able to see the traffic light. If the vehicle is replaced by a car with a height of 1 meter under the same circumstances, the driver in the car will be unable to see the traffic light due to the lower height and obstruction of vision. Similarly, the height of the bus and the height of the traffic light will also change. Therefore, only when the vehicle height, traffic light height, distance between the two vehicles, etc. are used as independent variables to calculate the occlusion threshold in real time can it be accurately detected whether the rear vehicle is blocked.

[0024] Later, it is assumed that the distance and height between the bus and the small vehicle behind are constant, and the distance between the bus and the traffic light is 30 meters. At this time, the vehicle behind is just in the critical state of being able to see the traffic light; because the angle of sight from the driver of the rear vehicle to the rear roof of the bus is constant, according to the principle of similar triangles, the farther the bus is from the traffic light, the larger the occlusion area. It is further concluded that when the distance between the two vehicles is constant, the occlusion situation will change with the different distances between the two vehicles and the traffic light.

[0025] From the above two examples, it can be seen that setting a fixed preset value (the distance between two vehicles) as the basis for detecting occlusion is very inaccurate, and there are cases of missed detection and false detection. The present invention proposes a method for obtaining traffic lights under bus occlusion, which can improve road driving safety.

[0026] The difficulty faced by the present invention when it was proposed is how to accurately, in real time, and dynamically measure the obstructed area behind the bus when the bus and the traffic light are in relative motion. Currently, common distance measurement methods include radar distance measurement, ultrasonic distance measurement, laser distance measurement, etc. Since the sensors used in the above distance measurement methods cannot identify the traffic lights whose height and angle change in real time, dynamic distance measurement cannot be completed. During the design process, attempts were also made to use stereo cameras, depth sensors, TOF (Time of Flight) cameras and other image sensors to complete real-time three-dimensional reconstruction of the scene, which can technically realize dynamic distance measurement of traffic lights. However, its disadvantages are obvious: First, binocular, depth, and TOF sensors have strict requirements on environmental conditions, including changes in illumination and background complexity, which makes it impossible for this method to complete 3D reconstruction at night or when lighting conditions are poor, resulting in failure of real-time ranging; second, the algorithm complexity of 3D scene reconstruction is high, and it has high requirements on the processor, requiring the installation of an additional high-performance processor to complete; third, the accumulated drift error of real-time dynamic 3D reconstruction cannot be eliminated. As the bus continues to run, its accumulated error will cause the ranging result to be very inaccurate. If auxiliary correction modules such as GPS are added, the error can be reduced to a certain extent, but the algorithm complexity will further increase, which is not worth the loss; fourth, the high cost of depth, TOF, and high-performance processors makes this method less practical.

[0027] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0028] This embodiment proposes a method for obtaining a traffic light blocked by a bus, which uses the bus's own on-board system, front camera (or a driving recorder connected to the on-board system), reversing radar, rear display screen and other equipment to complete the process of traffic light status recognition, rear vehicle blocking detection, traffic light signal transmission and display, etc., so as to realize the real-time prompt of the rear blocking vehicle during the bus driving process, so as to improve the road driving safety of small vehicles behind. The scheme is as follows:

[0029] 1. Obtain images through the front camera and identify traffic lights;

[0030] The front of the bus is equipped with a front camera or a driving recorder for monitoring purposes, which is interconnected with the vehicle computer system to transmit and save video image data. The present invention uses the front camera of the bus to take real-time photos of the road in the direction of the bus's travel. The built-in chip of the vehicle computer system is used as a computing processor to perform real-time analysis of the captured video images through an image processing-based traffic light recognition algorithm. It is specifically divided into two parts: traffic light color recognition (red, green, yellow) and digital recognition (remaining time of the traffic light).

[0031] The specific process of traffic light color recognition is as follows: Taking the recognition of red traffic lights as an example, first convert the RGB value into the corresponding hue, saturation, and value (HSV) coordinates. rgb is a three-dimensional color map array. The size of hsv is the same as the rgb dimension. Secondly, use the red color feature value to search for sensitive areas to identify the red color area; and combine the shape features of the traffic light to complete the construction of the structural element. Thirdly, the constructed structural element is binarized and filled with background color to complete the recognition process.

[0032] Specific process of digital recognition: The previous part completes the recognition of the color of the traffic light, and then the area of ​​the traffic light device is intercepted as the region of interest (ROI). Digital recognition in the ROI area can eliminate the influence of the cluttered background. The present invention uses the KNN (K-Nearest Neighbor) algorithm as an example. The KNN algorithm has high accuracy and can be used to process numerical data. In addition, the training time complexity is O (n), there is no data input assumption, and it is not sensitive to abnormal values. The specific steps are: 1) Use 200 digital images on traffic lights as training data; 2) Calculate the distance between the test data and each training data; 3) Sort according to the increasing relationship of the distance; 4) Select K points with the smallest distance; 5) Determine the frequency of occurrence of the category of the first K points; 6) Return the category with the highest frequency of occurrence in the first K points as the predicted classification of the test data. Furthermore, the digital recognition process of the traffic light device can be completed by the KNN algorithm.

[0033] 2. Obtain the occluded area behind the bus in real time

[0034] 1. Get the height difference h between the front camera installation position and the reversing radar position 1 , the height difference from the top of the bus to the reversing radar position h 2 , the horizontal distance S between the front camera installation position and the reversing radar position 0 The above physical quantities can be obtained during installation or measured later.

[0035] 2. Obtain the elevation angle α between the center point of the front camera and the traffic light

[0036] refer to Figure 1 , α is the elevation angle to be calculated, α=arctan(S / f), where f is the imaging focal length of the front camera, and S is the vertical distance from the imaging of the traffic light to the horizontal line of the center of the image; in the present invention, the elevation angle α is related to the height of the traffic light ( Figure 1 In the actual driving process, the traffic light is not necessarily located in front of the bus, but when the traffic light is located in front of the bus, it is the most serious case that the large truck blocks the vehicle behind it; when the traffic light is offset to the sides, the driver of the following vehicle may see the traffic light. At this time, the large truck reminds the following vehicle, which is a double insurance and safer.

[0037] 3. Get the horizontal distance S between the front camera and the traffic light 灯 , S 灯 =(S 行 *tanα2) / (tanα 1 -tanα 2 ); Get the vertical distance h between the traffic light and the front camera 灯 ,h 灯 =tan(α 2 )*(S 行 *tanα 2 ) / (tanα 1 -tanα 2 ), where S 行 It is the distance the bus travels during the time interval between two photos taken by the front camera.

[0038] According to the method described in 2 above, the elevation angles from the front camera to the traffic light can be calculated in real time as α1 and α2 respectively; the front camera video sampling in this embodiment is 30FPS, that is, 30 frames of images are taken per second, so the time interval between two adjacent images is 1 / 30 second, so it is necessary to calculate the distance traveled by the car within this 1 / 30 second. The car speed can be directly obtained by the driving computer of the car system, recorded as V 车 , where the car travels a distance S 行 It is obtained by integrating the vehicle speed and time, specifically:

[0039] S 行 =∫(V 车 *Δt), where V 车 is the real-time speed of the car, directly obtained by the on-board computer built into the vehicle system, and Δt is the image sampling interval of the camera. For example, if the camera acquires images at 30 frames per second (FPS), then Δt here is 1 / 30 second. Figure 2 , tan(α 1 )=h 灯 / (S 行 +S 灯 ), tan(α 2 )=h 灯 / S 灯

[0040] According to the relationship between trigonometric functions, we can list the system of linear equations of two variables as above, and further calculate:

[0041] S 灯 =(S 行 *tanα 2 ) / (tanα 1 -tanα 2 )

[0042] h 灯 =tan(α 2 )*(S 行 *tanα 2 ) / (tanα 1 -tanα 2 )

[0043] 4. Get the occluded area S behind the bus 遮挡

[0044] S 遮挡 is the radar detection threshold that needs to be calculated. That is, when the bus's rear reversing radar detects that the distance to the rear vehicle is less than S 遮挡 When (i.e. Figure 3 If a vehicle is detected in the middle shaded area), the result is that there is a blocked car behind the bus, and it is necessary to remind it of the real-time signal light status. Based on the relationship between trigonometric functions, we can list the following binary linear equation:

[0045] tanβ=h 2 / S 遮挡

[0046] tanβ=(h 灯 +h 1 ) / (S 灯 +S 0 +S 遮挡 )

[0047] From this we can get S 遮挡 =(S 灯 +S 0 )*h 2 / (h 1 +h 2 ).

[0048] The method of the present invention has the advantages of 1) low computational complexity and low requirements on the processor, and the built-in chip of the bus's built-in vehicle computer system can meet the computational requirements; 2) no expensive depth, TOF, radar and other sensors need to be installed; and 3) high ranging accuracy. It can be seen from the calculation formula that its accuracy is only related to the accuracy of the detected real-time vehicle speed, and there is no other cumulative error such as scene reconstruction or drift.

[0049] 3. Traffic Light Status and Information Transmission

[0050] When the reversing radar detects that a vehicle has entered the blocked area, the vehicle system transmits the status information of the traffic light identified in step A to the on-board display screen at the rear of the bus in real time to alert the vehicles behind.

[0051] The present invention can monitor in real time whether there is an obstructed vehicle behind the bus, and the traffic light reminder function will be activated only when a vehicle enters the obstructed area, which is both intelligent and energy-saving; the current buses only need to update the vehicle system firmware designed according to the method to have the corresponding intelligent obstruction detection and traffic light reminder functions, and the upgrade is convenient, fast and low-cost; small vehicles obstructed behind the bus can effectively reduce violations and unsafe driving after obtaining real-time traffic light status prompts, thereby improving road driving safety.

[0052] The above-described embodiments of the present invention do not constitute a limitation on the protection scope of the present invention. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A method for obtaining a traffic light blocked by a bus, wherein the bus is provided with a vehicle computer system, a front camera, a reversing radar and a rear display screen, characterized in that: include: Step A: Acquire an image through the front camera and perform traffic light recognition; Step B: obtaining the blocked area behind the bus in real time; B1, obtain the height difference h1 between the front camera installation position and the reversing radar position, the height difference h2 from the top of the bus to the reversing radar position, and the horizontal distance S0 between the front camera installation position and the reversing radar position; B2. Obtain the elevation angle α between the center point of the front camera and the traffic light, α=arctan(S / f), where f is the imaging focal length of the front camera, and S is the vertical distance from the imaging of the traffic light to the horizontal line of the center of the image; B3. After a certain time interval Δt, step B2 is taken again to obtain the second sampling data. The two samplings are used to calculate the elevation angles α1 and α2 between the center point of the front camera and the traffic light respectively; at the same time, the real-time speed V of the car is obtained using the vehicle computer system. 车 , and ∫(V 车 *Δt) Get the distance S that the bus traveled during the time interval between two photos taken by the front camera 行； Through the above data, the horizontal distance S between the front camera and the traffic light is obtained. 灯 , S 灯 =(S 行 *tanα2) / (tanα1-tanα2); Get the vertical distance h between the traffic light and the front camera 灯 ,h 灯 =tan(α2)*(S 行 *tanα2) / (tanα1-tanα2); B4. Get the occluded area S behind the bus 遮挡 , S 遮挡 =(S 灯 +S0)*h2 / (h1+h2); Step C: Transmission of traffic light status and information: When the reversing radar detects that a vehicle has entered the blocked area, the vehicle computer system transmits the traffic light status information identified in step A to the on-board display screen at the rear of the bus in real time to alert the vehicles behind.

2. The traffic light acquisition method according to claim 1, characterized in that: The traffic light recognition in step A includes traffic light color recognition and number recognition.

3. The traffic signal light acquisition method according to claim 2, characterized in that: The traffic light color recognition includes: first, converting the RGB value into the corresponding hue, saturation and brightness coordinates; second, using the corresponding color feature value, searching the sensitive area to identify the corresponding color area, and completing the construction of the structural element in combination with the appearance characteristics of the traffic light; third, binarizing the constructed structural element and filling the corresponding background color to complete the recognition process.

Images