A system and method for evaluating functionality of highway traffic signs

Through the system of lidar, camera units and speed sensors, the functionality of highway traffic signs is evaluated, and the problem that the existing technology cannot evaluate the actual use effect of traffic signs is solved, and scientific evaluation and safety guarantee of the functionality of the signs is achieved.

CN117518181BActive Publication Date: 2024-11-05RES INST OF HIGHWAY MINIST OF TRANSPORT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311541304.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-17
Publication Date
2024-11-05
Estimated Expiration
2043-11-17

AI Technical Summary

Technical Problem

The prior art cannot effectively evaluate the functionality of highway traffic signs in actual use, and cannot determine whether the signs play a proper role, which poses safety risks.

Method used

Using a system including lidar, camera unit and speed sensor, through point cloud data, images and vehicle speed information, determine whether the traffic sign is within the driver's effective field of vision, and calculate the straight line visual distance to evaluate the functionality of the traffic sign. .

Benefits of technology

It has achieved scientific and effective evaluation of highway traffic signs, discovered potential problems and hidden dangers, ensured the safety of vehicle traffic, and has the advantages of real-time detection and high efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117518181B_ABST
    Figure CN117518181B_ABST
Patent Text Reader

Abstract

The present invention relates to a system and method for evaluating the functionality of highway traffic signs, the system comprising: a laser radar, installed on the roof directly above the driving position of the evaluation vehicle, facing forward and keeping horizontal; a camera unit, installed at the same position and in the same direction as the laser radar, used to collect images of the signs; a speed sensor, used to collect speed information of the evaluation vehicle; a processing unit, connected to the laser radar, the camera unit and the speed sensor, capable of receiving point cloud data, images and speed information; a trigger unit, connected to the processing unit, used to receive operation instructions, thereby controlling the processing unit to receive point cloud data, images and speed information; the processing unit determines the type of the sign, the straight-line visual distance and the visual angle based on the point cloud data, the image and the speed information, thereby evaluating whether the function of the sign is normal. The present invention can evaluate the functionality of the sign based on the actual use effect of the sign based on the objective environment in which the driver is located.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of road traffic safety facility detection and evaluation, and mainly relates to a system and method suitable for on-site rapid evaluation of the functionality of in-service traffic signs. Background Art

[0002] Highway traffic signs are an important type of traffic safety protection facility. They convey specific information through graphic symbols and text to achieve the purpose of managing traffic, instructing driving, and ensuring smooth roads and driving safety. At present, retro-reflective traffic signs are still the main type of signs used on highways. Their performance is mainly reflected by the retro-reflective coefficient of the reflective film, and the quality inspection standards of their products are also centered around the determination of the retro-reflective coefficient.

[0003] However, in actual use, the factors that determine whether traffic signs can function properly include not only the sign's own parameters such as reflectivity and font size, but also external environmental factors such as the location of the traffic sign, road alignment, obstacle obstruction, and lighting conditions.

[0004] Traffic signs are usually set up outdoors and are prone to dust accumulation. Even if the technical parameters of traffic signs meet the relevant standards, their reflective ability will change over time, which will cause them to fail to function properly. Another unfavorable situation is that traffic signs that meet the relevant standards fail to function properly due to factors such as the location, road alignment, obstacles, and lighting conditions. The above situations are relatively common in actual road use, which will prevent vehicle drivers from identifying the content of the signs in a timely and effective manner, resulting in traffic signs failing to properly play their role in traffic control, posing a great safety hazard.

[0005] The Chinese invention patent application with publication number CN112507902A detects images extracted from traffic surveillance videos, compares the traffic sign detection results with the benchmark traffic sign area, and determines whether the traffic sign is abnormal due to a traffic accident or natural disaster based on the similarity.

[0006] The Chinese utility model patent with the announcement number CN215574640U discloses a sign reverse measuring instrument, which allows the user to clean the dust on the surface of traffic signs, especially high-altitude traffic signs, and detect the reflective film in a handheld manner.

[0007] The Chinese utility model patent with announcement number CN207690301U discloses a vehicle-mounted traffic sign performance detection device, which can measure the retroreflection coefficient of traffic signs in batches through a retroreflection coefficient measuring instrument arranged at the front end of the vehicle.

[0008] It can be seen that the existing solutions all detect the state of the traffic sign itself or the retroreflection coefficient. Such detection cannot reflect the effect of the traffic sign in the specific use process, and cannot be used to determine whether the traffic sign plays a proper role in the actual use process, that is, whether the traffic sign functions normally in actual use.

[0009] Therefore, it is necessary to design an effective system and method for evaluating the functionality of traffic signs, starting from the objective environment in which drivers are located and based on the actual use effects of in-service highway traffic signs to evaluate their functionality. Summary of the invention

[0010] The purpose of the present invention is to provide a system and method for evaluating the functionality of highway traffic signs, so as to make a scientific and effective judgment on the actual use effect of highway traffic signs in service.

[0011] To achieve the above objectives, on the one hand, the present invention provides a system for evaluating the functionality of a highway traffic sign, comprising:

[0012] A laser radar, which is mounted on the roof of the evaluation vehicle just above the driving position, facing the front of the vehicle and kept horizontal;

[0013] A camera unit, which is installed at the same position and in the same direction as the laser radar, and is used to collect an image of a target traffic sign;

[0014] A speed sensor, the speed sensor is arranged on the evaluation vehicle and is used to collect speed information of the evaluation vehicle;

[0015] a processing unit, the processing unit being communicatively connected with the laser radar, the camera unit and the speed sensor, and capable of receiving point cloud data provided by the laser radar, images provided by the camera unit and vehicle speed information provided by the speed sensor;

[0016] a trigger unit, the trigger unit being in communication with the processing unit and configured to receive an operation instruction of a tester, thereby controlling the processing unit to receive the point cloud data from the laser radar, the image from the camera unit, and the vehicle speed information from the speed sensor;

[0017] The processing unit determines the type of the target traffic sign, the straight-line visual distance, and the horizontal angle and vertical angle of the target traffic sign based on the point cloud data, the image, and the vehicle speed information, and evaluates whether the function of the target traffic sign is normal by judging whether the target traffic sign is within the effective field of view of the driver when the straight-line visual distance of the target traffic sign is greater than or equal to its minimum visual distance.

[0018] Preferably, the system further includes a storage unit, which is communicatively connected to the processing unit, the laser radar, the camera unit and the speed sensor, and is used to store the point cloud data, the image and the vehicle speed information.

[0019] Preferably, the trigger unit includes a first trigger, and when the first trigger is triggered, the processing unit receives the point cloud data, the image and the vehicle speed information.

[0020] Preferably, the trigger unit further comprises a second trigger, and when the second trigger is triggered, the processing unit stops receiving the point cloud data, the image and the vehicle speed information.

[0021] In another aspect, the present invention provides a method for evaluating functionality of a highway traffic sign, comprising the following steps:

[0022] Step 1: The system for evaluating the functionality of highway traffic signs is installed on an evaluation vehicle, and the evaluation vehicle is driven toward the target traffic sign and according to the speed limit of the road, with the tester sitting in the co-pilot seat of the evaluation vehicle;

[0023] Step 2: When the tester can clearly see the text and icons of the target traffic sign, he / she immediately operates the trigger unit to control the processing unit to receive the point cloud data from the laser radar, the image from the camera unit, and the vehicle speed information from the speed sensor at this moment;

[0024] Step 3: determining the type of the target traffic sign in the image by a processing unit, and determining the minimum visual recognition distance of the target traffic sign;

[0025] Step 4: determining the straight-line visual distance of the target traffic sign based on the point cloud data of the laser radar, and if the straight-line visual distance is less than the minimum visual distance, determining that the function of the target traffic sign is abnormal;

[0026] Step 5: If the straight line visual distance is greater than or equal to the minimum visual distance, determine whether the target traffic sign is within the effective field of view of the driver at this moment based on the point cloud data of the laser radar; if the target traffic sign is within the effective field of view of the driver, determine that the function of the target traffic sign is normal;

[0027] Step 6: If the target traffic sign is not within the driver's effective field of view, steps 4, 5 and 6 are executed again through the processing unit based on the point cloud data from the lidar at the next moment, the image from the camera unit and the vehicle speed information from the speed sensor.

[0028] Preferably, the processing unit synchronously records the point cloud data of the laser radar and the image of the camera unit at a speed of not less than 10fps.

[0029] Preferably, the process of determining whether the target traffic sign is within the effective field of view of the driver further comprises:

[0030] Step A1: Determine the effective horizontal viewing angle and effective vertical viewing angle of the driver based on the type and speed of the assessed vehicle,

[0031] a 右 =83.08·e -0.014v

[0032]

[0033] a 右 is the effective horizontal viewing angle on the right side of the driver; a 左 is the effective horizontal viewing angle on the left side of the driver; θ1 is the angle between the driver's front direction and the right edge of the left A-pillar when the driver is in the driving position; v is the vehicle speed; v th1 is the critical speed, at which the left boundary of the driver's effective horizontal field of vision coincides with the right edge of the left A-pillar of the vehicle;

[0034]

[0035] b is the effective vertical viewing angle of the driver, θ2 is the angle between the driver's front direction and the upper edge of the front window when the driver is in the driving position, and v is the vehicle speed; v th2 is the critical vehicle speed, at which the upper boundary of the driver's effective vertical field of view coincides with the upper edge of the vehicle's front window, and μ is the proportional coefficient between the upper and lower field of view angles and the left and right field of view angles;

[0036] Step A2: Determine the horizontal angle and vertical angle of the target traffic sign and judge whether it meets the following conditions:

[0037]

[0038] Where α is the horizontal angle of the sign, β is the vertical angle of the sign, a is the effective horizontal angle of the driver, and b is the effective vertical angle of the driver. When the target traffic sign is located on the left side of the longitudinal centerline of the vehicle, a = a 左 , when the traffic sign is on the right side of the longitudinal centerline of the vehicle, a=a 右 ;

[0039] If the horizontal angle and the vertical angle of the target traffic sign meet the conditions, it is determined that the target traffic sign is within the effective field of view of the driver, otherwise it is determined that the target traffic sign is not within the effective field of view of the driver.

[0040] Preferably, the process of determining the straight-line visual distance of the target traffic sign based on the point cloud data of the laser radar further includes:

[0041] Step B1: establishing a mapping relationship between the horizontal and vertical coordinates of each scanning point in the point cloud data and the horizontal and vertical coordinates of each pixel point in the image;

[0042] Step B2: identifying a target traffic sign from the image, and determining a sign scanning point in the point cloud data corresponding to the target traffic sign through the mapping relationship;

[0043] Step B3: Calculate the straight-line visual distance of the target traffic sign using the following formula:

[0044]

[0045] Where n is the number of marker scanning points used to calculate the straight line visual distance, x i is the horizontal coordinate of the i-th marker scanning point, y i is the ordinate of the i-th marker scanning point, z i is the depth coordinate of the i-th marker scanning point.

[0046] Preferably, the horizontal angle α of the target traffic sign satisfies The vertical angle β of the target traffic sign satisfies

[0047] in, For x i The average value of for y i The average value of For z i The average value of .

[0048] Preferably, the n sign scanning points used to calculate the straight-line visual distance of the target traffic sign are points in the first 10% to first 20% range after all the sign scanning points are sorted from largest to smallest according to the straight-line distance.

[0049] The present invention changes the detection object and takes the evaluation of the actual use effect of highway traffic signs as the starting point, breaking through the traditional detection method based on material retroreflective performance indicators and truly realizing a comprehensive evaluation of the overall functionality of traffic signs in real scenarios.

[0050] Through the system and method for evaluating the functionality of highway traffic signs of the present invention, the functionality of traffic signs can be scientifically evaluated according to the actual use effect of the traffic signs and based on the driver's visual perception system during driving, so as to accurately discover the problems and hidden dangers existing in the use of traffic signs, provide a basis for the scientific management and control of highway traffic, and effectively ensure the traffic safety of vehicles; moreover, the system and method of the present invention can perform real-time evaluation of traffic signs during normal vehicle driving and normal driving speed, showing great advantages in detection speed and efficiency; in addition, the evaluation equipment of the present invention is operated by a tester located in the co-pilot seat of the vehicle, without the need for driver participation, effectively ensuring driving safety during the evaluation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 FIG. 4 is a schematic diagram of the structure of a system for evaluating functionality of a highway traffic sign according to a preferred embodiment of the present invention.

[0052] Figure 2 4 is a flow chart of a method for evaluating functionality of a highway traffic sign according to a preferred embodiment of the present invention.

[0053] Figure 3 It is a schematic diagram of a method for a laser radar to measure the straight-line distance to a target traffic sign according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0054] Highway traffic signs are usually set on both sides of the road or above the road, roughly opposite to the direction of vehicle travel. Since vehicles travel on the road at a certain speed, in order to allow drivers to have sufficient time to understand the meaning of the signs and respond accordingly, traffic signs require drivers to see the signs clearly before the distance reaches (or decreases to) a certain distance. This specific distance is the minimum visual distance of traffic signs.

[0055] In the present invention, the minimum visual distance is the lower limit of the visual distance that traffic signs should have when vehicles are using low beam lights. The speed limit values ​​of different highways are different, so the minimum visual distance of their traffic signs is also different. For example, for expressways and first-class highways with speed limits exceeding 100km / h, the minimum visual distance is 120m; the minimum visual distances of expressways, first-class highways, and second-class highways with speed limits of 80km / h are 120m, 90m, and 90m respectively; the minimum visual distances of first-class highways, second-class highways, and third-class highways with speed limits of 60km / h and 40km / h are 70m; for fourth-class and lower-class highways with speed limits not exceeding 30km / h, the minimum visual distance of their traffic signs is 50m. In addition, the minimum visual distance of traffic signs may be different for different types: prohibition and warning signs use the above minimum visual distance, and road signs, instruction signs, and service facility signs can be adjusted by about 10% based on the above values. Generally speaking, the more important and the greater the impact of a traffic sign, the greater its minimum visibility distance, so that the driver has more time to respond.

[0056] Regardless of whether the traffic sign is set on the side of the road or above the road, as the distance between the vehicle and the traffic sign gradually decreases, the angle of the traffic sign (i.e., the angle between the straight line connecting the driver and the traffic sign and the direction directly in front of the driver) will gradually increase. If the traffic sign functions abnormally and visibility deteriorates, the driver can only see or see the content of the sign when the vehicle is very close to the traffic sign, which means that the angle of the traffic sign is relatively large at this time. However, according to the requirements of safe driving, the driver needs to look straight ahead in a natural driving state, and the driver should not be required to make large adjustments to his body posture in order to see the traffic sign clearly. In other words, when the driver can see the content of the sign clearly, the traffic sign should be within the effective field of view of the driver in a normal driving posture.

[0057] The inventors have found that the static field of view of the driver can cover a viewing angle range of about 160° in the left and right directions in front of him, but as the vehicle speed increases, the dynamic effective viewing angle will gradually decrease. When the vehicle speed is 40km / h, the driver's effective viewing angle is reduced to 100°, when the vehicle speed is 60km / h, the effective viewing angle will be reduced to 75°, when the vehicle speed is 80km / h, the effective viewing angle drops to 60°, and when the vehicle speed is 100km / h, the effective viewing angle drops to 40°. Through data fitting, it is found that the vehicle speed and the effective viewing angle are exponentially related.

[0058] At the same time, for the driver of a left-hand drive vehicle, his horizontal viewing angle is also affected by the A-pillar on the left side of the front window. When the vehicle speed does not reach a certain critical value, the driver's effective field of view becomes narrower on the left side of the center line due to the limitation of the A-pillar. At this time, the position corresponding to the left edge of the front window is used as the left boundary of the driver's effective field of view, thereby determining the driver's left effective horizontal viewing angle; when the speed exceeds the critical value, the effective horizontal viewing angle that is exponentially related to the vehicle speed is adopted.

[0059] Taking the above into consideration, the relationship between the effective horizontal viewing angle a of the vehicle driver and the vehicle speed v is:

[0060] a 右 =83.08·e -0.014v

[0061]

[0062] a 右 is the effective horizontal visual angle on the right side of the driver (i.e., the angle between the right boundary of the driver's effective horizontal visual field and the direction directly in front of it); a 左 is the driver's left effective horizontal visual angle (i.e., the angle between the left boundary of the driver's effective horizontal visual field and the direction directly in front of it); θ1 is the angle between the driver's direction directly in front of him and the right edge of the left A-pillar when the driver is in the driving position, which is related to the type of vehicle: when the vehicle is a sedan, θ1 is 15°; when the vehicle is a sports utility vehicle (SUV), θ1 is 20°; when the vehicle is a flat-head vehicle such as a road engineering vehicle, θ1 is 30°; v th1 is the critical speed, at which the left boundary of the driver's effective horizontal field of view coincides with the right edge of the left A-pillar of the vehicle.

[0063] Similarly, the driver's vertical field of view is affected by the height of the front window and the vehicle speed. The upper and lower fields of view of a normal person are narrower than the left and right fields of view. The proportional coefficient μ between the upper and lower field of view angles and the left and right field of view angles is less than 1, and can generally be taken as 2 / 3. Under the same speed conditions, the vertical effective field of view is μ times the horizontal effective field of view. The higher the speed, the smaller the effective viewing angle, and the less likely it is to notice the information at the edge of the field of view. When the vehicle speed does not reach a certain critical value, the driver's effective field of view becomes narrower at the top due to the limitation of the height of the front window. At this time, the position corresponding to the upper edge of the front window is used as the upper boundary of the driver's effective field of view, thereby determining the driver's upper effective vertical viewing angle; when the speed exceeds the critical value, the effective vertical viewing angle that is exponentially related to the vehicle speed is adopted.

[0064] Therefore, the relationship between the effective vertical viewing angle b of the vehicle driver and the vehicle speed v is:

[0065]

[0066] θ2 is the angle between the driver's front direction and the upper edge of the front window when the driver is in the driving position, which is related to the type of vehicle: when the vehicle is a sedan, θ2 is 30°; when the vehicle is an SUV, θ2 is 45°; when the vehicle is a flat-head vehicle such as a road engineering vehicle, θ2 is 60°; v th2 is the critical speed, at which the upper boundary of the driver's effective vertical field of view coincides with the upper edge of the vehicle's front window.

[0067] When the driver maintains a normal driving posture in the driving position, the overall effective field of view is basically elliptical. When the visibility angle of traffic signs is within the driver's effective field of view, the following conditions should be met:

[0068]

[0069] Where α is the horizontal angle of the sign (i.e., the angle between the projection of the straight line connecting the driver and the sign on the horizontal plane and the direction directly in front of the driver); β is the vertical angle of the sign (i.e., the angle between the projection of the straight line connecting the driver and the sign on the vertical plane in front of the driver and the direction directly in front of the driver). a is the effective horizontal angle of view of the driver; b is the effective vertical angle of view of the driver. When the traffic sign is located to the left of the longitudinal centerline of the vehicle, a=a 左 ; When the traffic sign is located on the right side of the longitudinal centerline of the vehicle, a=a 右 .

[0070] Figure 1 FIG. 1 is a schematic diagram showing a structure of a system 100 for evaluating functionality of a highway traffic sign according to a preferred embodiment of the present invention. Figure 1 As shown, the system 100 for evaluating the functionality of a highway traffic sign includes: a laser radar 110 , a camera unit 120 , a speed sensor 130 , a processing unit 140 , a trigger unit 150 , and a storage unit 160 .

[0071] The laser radar 110 is installed on the roof directly above the driving position of the evaluation vehicle, facing the front of the vehicle and kept horizontal, and is used to measure the point-to-point spatial straight-line distance between the target traffic sign board and the radar lens. The point cloud data output by the laser radar 110 can be used to obtain the three-axis coordinates (x, y, z) of the target traffic sign in the spatial rectangular coordinate system, thereby easily calculating the distance of the target traffic sign relative to the radar.

[0072] The camera unit 120 is installed at the same position and in the same direction as the laser radar 110 to collect images of the target traffic sign. The camera unit 120 preferably uses a short-focus wide-angle lens so that a larger field of view can be obtained. Since the position of the camera unit 120 is relatively fixed to the radar lens, the outline of the sign in the video image output by the camera unit 120 can be mapped to the point cloud image of the laser radar 110 at the same time.

[0073] Since the installation positions of the laser radar 110 and the camera unit 120 are very close to the driver's head, the data and information collected by the laser radar 110 and the camera unit 120 can well reflect the actual situation of the traffic signs observed by the driver when driving the vehicle.

[0074] The speed sensor 130 is arranged on the evaluation vehicle to collect the speed information of the vehicle. The speed sensor 130 can be implemented by means of the vehicle's own speed sensor, or a separate speed sensor (such as a wheel encoder). In a preferred embodiment of the present invention, for example, a GPS positioning sensor, a Beidou positioning sensor, etc. can be used to provide vehicle position and speed information at the same time.

[0075] The laser radar 110, the camera unit 120 and the speed sensor 130 are all connected to the processing unit 140 for communication. The laser radar 110 provides point cloud data to the processing unit 140, the camera unit 120 provides the collected images to the processing unit 140, and the speed sensor 130 provides the processing unit 140 with the vehicle speed information.

[0076] The trigger unit 150 is also connected to the processing unit 140 for communication. The trigger unit 150 is operated by the tester to receive the tester's operation instructions, thereby controlling the processing unit 140 to receive the point cloud data from the laser radar 110, the image from the camera unit 120, and the vehicle speed information from the speed sensor 130 and store them in the storage unit 160. The storage unit 160 is also connected to the laser radar 110, the camera unit 120, and the speed sensor 130 for communication.

[0077] The trigger unit 150 further includes a first trigger 151 and a second trigger 152. When the system 100 is used to evaluate the functionality of a highway traffic sign, the evaluation vehicle is driven along the road, and the first trigger 151 and the second trigger 152 are operated by a tester located in the co-pilot position of the evaluation vehicle. When the tester sees the target traffic sign and sees its information clearly, the first trigger 151 is operated, thereby controlling the processing unit 140 to receive data and information from the laser radar 110, the camera unit 120, and the speed sensor 130 and store them in the storage unit 160; when the vehicle drives to the same vertical section of the road where the target traffic sign and the tester are located, the tester operates the second trigger 152 to end the previous storage process. In other words, the system 100 records the point cloud data of the laser radar 110, the image of the camera unit 120, and the vehicle speed information of the speed sensor 130 during the period from when the first trigger 151 is triggered to when the second trigger 152 is triggered, and evaluates the functionality of the target traffic sign on this basis.

[0078] In a preferred embodiment of the present invention, both the first trigger 151 and the second trigger 152 are reset-type buttons, which pop up immediately after being pressed, supporting a faster operating frequency. In particular, when the system 100 of the present invention is implemented based on a laptop computer, a specific button on the laptop computer keyboard can be used as a trigger, and the button on the keyboard is the reset-type button.

[0079] Since the evaluation vehicle is traveling at a relatively high speed, in order to avoid missing key point cloud data and image frames, the speed at which the laser radar 110 provides point cloud data and the camera unit 120 provides images is preferably not less than 10fps. If the evaluation vehicle is traveling at a speed of 90km / h, 10fps means that the time interval between adjacent point cloud data and image frames is 100ms, and the vehicle can travel 2.5m during this period; but if the time interval between adjacent point cloud data and image frames is 200ms (corresponding to 5fps), the vehicle can travel a distance of 5m, and key point cloud data and image frames may be missed, resulting in inaccurate evaluation results.

[0080] In a preferred embodiment of the present invention, the laser radar 110, the camera unit 120 and the speed sensor 130 communicate with the processing unit 140 and the storage unit 160 through a high-speed transmission method to reduce the time difference caused by the transmission of the point cloud data, the image frame and the speed information. In order to avoid errors caused by the differences in the time of the laser radar 110, the camera unit 120 and the speed sensor 130, the same clock can be used to provide unified time for each device, for example, it can be achieved through the NTP protocol (network time protocol) or the PTP protocol (precise time protocol) that supports more accurate time synchronization.

[0081] When evaluating the functionality of highway traffic signs, the system 100 according to the preferred embodiment of the present invention is set on the evaluation vehicle, and the evaluation is carried out as much as possible at night when the traffic volume is small, so as to avoid the influence of other vehicle lights on the evaluation results; the vehicle low beam is used for lighting during the evaluation; the evaluation vehicle is driven according to the speed limit of the road to be inspected, so as to reflect the actual use environment of the traffic sign as much as possible. When there are multiple lanes in the driving direction, the evaluation vehicle should be driven along the lane far from the traffic sign (usually the first lane on the left side of the road); the tester should have normal vision, sit in the co-pilot position of the evaluation vehicle, and observe the road traffic signs in a natural state by looking straight ahead of the road surface during the vehicle's movement. Although there are differences between the co-pilot position and the driving position, the difference in distance between the two and the target traffic sign is very small and can be ignored; and in terms of field of view, the two are very different, the field of view of the driver's position is narrow on the left and large on the right, and the field of view of the co-pilot position is narrow on the right and large on the left, but through the evaluation method of the present invention described in detail later, the difference in field of view between the two will not affect the functional evaluation results of highway traffic signs.

[0082] Figure 2 The flowchart of the method for evaluating the functionality of highway traffic signs according to a preferred embodiment of the present invention is shown. During the evaluation of the vehicle, when the tester can clearly see the text and icons of the target traffic sign, the first trigger is pressed, thereby instructing the processing unit to record the starting time and receive the point cloud data from the laser radar and the image from the camera unit at this time. The tester only needs to see the sign clearly to press the first trigger, without having to understand the specific meaning of the sign information.

[0083] The processing unit determines the type of the target traffic sign in the image through an image recognition algorithm, and determines the minimum visual recognition distance that the target traffic sign should meet in combination with the type information of the road. Preferably, the minimum visual recognition distances of different types of traffic signs at different highway grades and different speed limits are recorded in a lookup table, and the minimum visual recognition distance corresponding to the highway grade can be determined by querying the type of the traffic sign board.

[0084] The processing unit determines the straight-line visual distance of the target traffic sign through the point cloud data of the laser radar, that is, the straight-line distance between the laser radar and the sign when the tester can see the target traffic sign clearly, which is also equivalent to the straight-line distance between the driver and the sign. To this end, it is necessary to find the point coordinates corresponding to the target traffic sign from the point cloud data.

[0085] In a preferred embodiment of the present invention, the point cloud coordinates within the target traffic sign image range are determined by positionally fusing the point cloud data of the laser radar with the image of the camera unit. The specific process is as follows:

[0086] After the relative positions of the laser radar and the camera unit are fixed in the physical structure, the mapping relationship between the horizontal and vertical coordinates of the scanning point on the radar point cloud map and the horizontal and vertical coordinates of the pixel point in the image can be calibrated. r ,y r ) represents the horizontal and vertical coordinates of a scanning point on the radar point cloud (respectively representing the distance of the scanning point in the x-axis direction and the y-axis direction relative to the coordinate origin of the radar lens), and (x c ,y c ) represents the coordinates of the pixel points on the video image collected by the camera unit. Taking a 1920×1080 resolution image as an example, the coordinates of the upper left pixel point are (0, 0), and the coordinates of the lower right pixel point are (1919, 1079). The mapping relationship between the coordinates of the scanning point on the radar point cloud and the coordinates of the speed limit point in the video image can be expressed as:

[0087]

[0088] Once the pixel coordinates of the target traffic sign in the video image are determined, its position coordinates on the point cloud map can be determined.

[0089] During the evaluation process, the pixel coordinates of the landmark area identified by the image can be expressed in matrix form as follows:

[0090] The horizontal axis pixel matrix is

[0091] The vertical coordinate pixel matrix is

[0092] Through the inverse function of the mapping relationship, the horizontal coordinate of the mark in the point cloud area can be obtained as The vertical coordinate of the mark in the point cloud area is

[0093] Then, the depth coordinates corresponding to the horizontal and vertical coordinates of the above point cloud are traversed in the radar point cloud data. According to the horizontal and vertical coordinates and the depth coordinates, a spatial right triangle is constructed to calculate the straight-line distance between the scanning point and the radar lens.

[0094] Considering that the existing image recognition algorithms cannot accurately identify points on the edge of the traffic sign outline, in order to avoid errors, when calculating the actual straight-line visual distance, a portion of the scanning points with the farthest and closest straight-line distances to the radar lens can be excluded, and the arithmetic mean of the straight-line distances of the remaining scanning points is used as the actual straight-line visual distance. For example, after sorting the straight-line distances of each scanning point from large to small, the first 10% and the last 10% are excluded, and the actual straight-line visual distance is calculated based on the middle 10%-90% of the scanning points. In a preferred embodiment of the present invention, the points within the first 10%-20% range after the straight-line distances are sorted from large to small are taken for calculation. In this way, the interference of points with unreliable individual distance data is avoided, and the execution speed of the algorithm is also improved.

[0095] Figure 3 A schematic diagram showing a method for laser radar to measure the straight-line distance to a target traffic sign according to a preferred embodiment of the present invention is shown. In the figure, x, y, and z are the coordinates of the target traffic sign in the rectangular coordinate system of the laser radar, and D is the straight-line distance between the target traffic sign and the laser radar.

[0096] At the moment when the first trigger is triggered, after the aforementioned position fusion process, n radar scanning points (i.e., sign scanning points) are selected from the target traffic sign area (e.g., points within the first 10%-20% range after the straight-line distance is sorted from large to small), and the coordinates of the i-th sign scanning point are marked as (x i ,y i , z i ), 1≤i≤n, at this time, the straight-line distance D between the target traffic sign and the laser radar (i.e., the actual visual distance of the sign) can be expressed as:

[0097]

[0098] For x i The average value of the horizontal distance between the target traffic sign and the radar lens, for y i The average value of the longitudinal distance between the target traffic sign and the radar lens, For z i The average value of the depth distance between the target traffic sign and the radar lens.

[0099] Therefore, the horizontal angle α of the target traffic sign satisfies The vertical angle β satisfies α and β can be obtained through simple calculation.

[0100] After determining the straight-line visibility distance of the target traffic sign, compare it with the minimum visibility distance of the sign. If the straight-line visibility distance is less than the minimum visibility distance, it means that the target traffic sign cannot be seen clearly by the driver in time, thereby determining that the function of the target traffic sign is abnormal; if the straight-line visibility distance is greater than or equal to the minimum visibility distance, it is necessary to further determine whether the target traffic sign is within the driver's effective field of view.

[0101] As mentioned above, based on the evaluation of the vehicle speed and vehicle type at the time when the first trigger is triggered, the driver's effective horizontal viewing angle a and effective vertical viewing angle b can be obtained if the horizontal angle α and vertical angle β of the target traffic sign meet the following conditions:

[0102]

[0103] It can be determined that the target traffic sign is within the effective field of view of the driver. It can be seen that when the first trigger is triggered, the driver can see the target traffic sign clearly when the distance from the target traffic sign is greater than its minimum visual recognition distance, and the sign is within the effective field of view of the driver, and the driver does not need to make a large adjustment of his body posture in order to see the sign clearly. It can be determined that the target traffic sign functions normally.

[0104] If the horizontal angle α and vertical angle β of the target traffic sign meet the following conditions:

[0105]

[0106] It can be determined that the target traffic sign is outside the driver's effective field of view or at the boundary of the effective field of view. This means that at the moment when the first trigger is triggered, the position of the target traffic sign relative to the driver does not make it easy for the driver to notice the target traffic sign. In this case, the function of the target traffic sign should not be considered abnormal. Because the distance between the driver and the sign is greater than the minimum visual distance of the sign at this time, before the vehicle is evaluated to the point where the distance between the driver and the sign is less than the minimum visual distance, as the relative positions of the two change, the target traffic sign may enter the driver's effective field of view. Therefore, it is necessary to continuously collect the point cloud data of the lidar, the image of the camera unit, and the vehicle speed information provided by the speed sensor.

[0107] After the first trigger is pressed, the processing unit synchronously records the point cloud data of the laser radar and the image of the camera unit at a specific speed (for example, not less than 10fps) until the evaluation vehicle continues to drive to the same vertical section of the road as the target traffic sign and the tester, and the tester presses the second trigger. After the tester presses the second trigger, the processing unit stops recording the point cloud data from the laser radar, the image from the camera unit, and the speed information from the speed sensor.

[0108] During the period between when the tester presses the first trigger and when the tester presses the second trigger, the point cloud data, the camera image, and the speed information of the vehicle are stored in the storage unit in a format such as shown in Table 1.

[0109]

[0110] Table 1

[0111] Among them, the "Trigger" item in the first row is marked as "1", indicating that the first trigger is pressed, the "Trigger" item in the last row is marked as "2", indicating that the second trigger is pressed, and the "Trigger" items in each row in between are marked as "0", indicating that no trigger is pressed.

[0112] In the case where the straight-line visual distance is greater than or equal to the minimum visual distance of the target traffic sign, but the sign is outside the effective field of view of the driver, the processing unit re-determines the straight-line visual distance D, horizontal angle α and vertical angle β of the sign based on the next frame of point cloud data, camera image and vehicle speed information, and repeats the previous evaluation process. Before the straight-line visual distance D decreases to less than the minimum visual distance, if the sign is within the effective field of view of the driver, it can be determined that the function of the sign is normal.

[0113] If the sign is not within the driver's effective field of view until the straight-line distance between the laser radar and the target traffic sign is less than the minimum visual recognition distance of the sign, it can be determined that the sign is malfunctioning.

[0114] If a sign is deemed to be malfunctioning because the actual viewing distance is less than the minimum viewing distance, it can usually be inferred that the sign's reflective performance does not meet the requirements, perhaps because the retroreflection coefficient of the reflective material needs to be improved or the orientation of the sign is improperly set; if the actual viewing distance of a sign is greater than the minimum viewing distance but it has never entered the driver's effective field of view, it can usually be inferred that the sign's reflective performance meets the requirements but the setting position or angle is inappropriate.

[0115] When applying the system and method of the present invention to evaluate the functionality of highway traffic signs, there is no need to analyze all the data output by the lidar and camera unit. The trigger unit receives the operator's instructions, and the processing unit only needs to process the data within the time period from the triggering of the first trigger to the triggering of the second trigger, so as to achieve the functional evaluation of the target traffic sign, thereby effectively reducing the requirements of the evaluation process on the data processing and computing capabilities of the processing unit and the data transmission and storage capabilities of the system.

[0116] The main purpose of evaluating the functionality of highway traffic signs is to determine whether the signs can convey information to the drivers of passing vehicles in a timely and accurate manner. However, for the sake of driving safety, the driver is not allowed to participate in the evaluation process while driving the vehicle. The present invention simulates the driver's perspective by arranging a laser radar and a camera unit on the roof just above the driver, and starts the evaluation process by a tester in the co-pilot position, so that the effect of the sign in the actual use scenario can be effectively tested without affecting safe driving. In addition, by calculating the horizontal angle and vertical angle of the sign relative to the driving position, the deviation caused by the difference in the field of vision between the tester and the driver can be effectively avoided, and the errors that may be caused by the tester's unconscious actions such as turning his head, raising his head, leaning over, and sideways to actively look for the sign can also be eliminated. After all, in the actual natural driving state, the driver needs to look straight ahead according to the requirements of safe driving and should not make these actions that will seriously affect driving safety. Moreover, especially for situations where the sign board is hung above the road at a high height, the sign is located on the outside of a one-way multi-lane road, the radius of the flat curve of the road is very small or the longitudinal slope is large, when the straight-line visibility distance of the sign meets the functional evaluation indicators, it is usually necessary to detect its visibility angle to ensure that the driver can effectively obtain the information presented by the sign under natural driving conditions.

[0117] In a preferred embodiment of the present invention, the second trigger can be triggered when the vehicle is evaluated to be in the same vertical section of the road as the target traffic sign, and an indication signal is provided to cause the processing unit to stop recording data and information from the laser radar, the camera unit and the speed sensor. However, it will be appreciated by those skilled in the art that the second trigger can also be omitted. For example, by evaluating the positional relationship of the vehicle relative to the sign when the first trigger is triggered and combining it with evaluating the speed of the vehicle, the time required for the vehicle to pass the sign can be calculated, so that it can be set to automatically stop recording data and information from the laser radar, the camera unit and the speed sensor after a specific time has passed after the first trigger is triggered.

[0118] Those skilled in the art will appreciate that the processing unit may use any image recognition algorithm currently known or available in the future, as long as it can effectively identify the type of the target traffic sign. The corresponding relationship between the parameters affecting the minimum visual recognition distance, such as the highway grade, the sign type, etc., and the minimum visual recognition distance may be preset in the program executed by the processing unit, so that the minimum visual recognition distance corresponding to the target traffic sign may be directly determined based on these parameters.

[0119] The embodiment of the present invention is further described by taking the evaluation of the functionality of the outer side speed limit sign of a one-way three-lane highway as an example. The speed limit of the inner lane of the three-lane highway is 100 km / h, the road plane line shape is a circular curve, and the evaluation vehicle used is a sports utility vehicle (SUV).

[0120] When the tester can clearly see the text and icons of the sign, the first trigger is pressed immediately to capture the video image at that moment. After image recognition, it is determined that the speed limit sign is a type of prohibition sign, and thus its minimum visual distance should be 120m. When the first trigger is triggered, the vehicle is traveling at a speed of 98km / h. The horizontal distance x1 of the sign is a positive number, indicating that the sign is located to the right of the center line of the evaluated vehicle, and the effective horizontal viewing angle a1=a is calculated. 右 =21°; μ=2 / 3, the effective vertical viewing angle b1=14° is calculated, which does not reach the vertical viewing angle of the vehicle under critical state. The landmark point cloud data collected by the laser radar is:

[0121]

[0122] The actual visual distance D1 = 133.23m is calculated, which meets the requirement that the actual visual distance is greater than the minimum visual distance. The sign visual angle is further calculated to obtain α1 = 21.2°; β1 = 2.03°. It can be seen that:

[0123]

[0124] This indicates that at the moment when the first trigger is triggered, the sign is outside the driver's effective field of vision and is difficult to see.

[0125] Continue to calculate the data at the next moment. The vehicle speed is 98km / h. The effective horizontal and vertical viewing angles have not changed. a2 = 21° and b2 = 14°. The landmark point cloud data collected by the laser radar is:

[0126]

[0127] The straight-line distance between the laser radar and the sign at this moment is calculated to be D2 = 125.08m, which is greater than the minimum visual distance. The visual angle of the sign is further calculated to obtain α2 = 20.78°; β2 = 1.75°. It can be seen that:

[0128]

[0129] This indicates that the sign is within the driver's effective field of view at this moment, which means that the sign is functioning properly.

[0130] In a preferred embodiment of the present invention, the processing unit completes the functional evaluation of the target traffic sign in real time and records the evaluation result corresponding to the sign. Since the point cloud data of the laser radar, the image of the camera unit and the speed information of the vehicle have been stored in the storage unit between the time when the tester triggers the first trigger and the time when the tester triggers the second trigger, the information can be processed uniformly after the information of all target traffic signs in a specific road section is collected.

[0131] It should be appreciated that the embodiments described above can be adjusted within a reasonable range according to the specific application conditions and are not limited to the specific situations in the above embodiments. In other words, those skilled in the art can make various changes and / or modifications to the specific embodiments without departing from the concept of the present invention.

Claims

1. A system for evaluating the functionality of a highway traffic sign, wherein the highway traffic sign is arranged on both sides of a road or above the road and is substantially opposite to the direction in which a vehicle travels, the system comprising: A laser radar, which is mounted on the roof of the evaluation vehicle just above the driving position, facing the front of the vehicle and kept horizontal; A camera unit, which is installed at the same position and in the same direction as the laser radar, and is used to collect an image of a target traffic sign; A speed sensor, the speed sensor is arranged on the evaluation vehicle and is used to collect speed information of the evaluation vehicle; a processing unit, the processing unit being communicatively connected with the laser radar, the camera unit and the speed sensor, and capable of receiving point cloud data provided by the laser radar, images provided by the camera unit and vehicle speed information provided by the speed sensor; a trigger unit, the trigger unit being in communication with the processing unit and configured to receive an operation instruction of a tester, thereby controlling the processing unit to receive the point cloud data from the laser radar, the image from the camera unit, and the vehicle speed information from the speed sensor; The processing unit determines the type of the target traffic sign, the straight-line visual distance, and the horizontal angle and the vertical angle of the target traffic sign based on the point cloud data, the image, and the vehicle speed information, and evaluates whether the function of the target traffic sign is normal by judging whether the target traffic sign is within the effective field of vision of the driver when the straight-line visual distance of the target traffic sign is greater than or equal to its minimum visual distance. The process of determining whether the target traffic sign is within the effective field of view of the driver includes: Determine the driver's effective horizontal viewing angle a and effective vertical viewing angle b based on the type and speed of the assessed vehicle; and Determine the horizontal angle and vertical angle of the target traffic sign and judge whether it meets the following conditions: Where α is the horizontal angle of the sign, β is the vertical angle of the sign, If the horizontal angle and the vertical angle of the target traffic sign meet the conditions, it is determined that the target traffic sign is within the effective field of view of the driver, otherwise it is determined that the target traffic sign is not within the effective field of view of the driver.

2. The system according to claim 1, wherein: The system further includes a storage unit, which is communicatively connected to the processing unit, the laser radar, the camera unit and the speed sensor, and is used to store the point cloud data, the image and the vehicle speed information.

3. The system according to claim 1 or 2, wherein: The trigger unit includes a first trigger. When the first trigger is triggered, the processing unit receives the point cloud data, the image, and the vehicle speed information.

4. The system according to claim 3, wherein: The trigger unit further includes a second trigger, and when the second trigger is triggered, the processing unit stops receiving the point cloud data, the image, and the vehicle speed information.

5. A method for evaluating the functionality of a highway traffic sign, wherein the highway traffic sign is arranged on both sides of a road or above the road and is substantially opposite to the direction in which vehicles travel, the method comprising the following steps: Step 1: installing the system according to any one of claims 1 to 4 on an evaluation vehicle, driving the evaluation vehicle toward a target traffic sign and according to the speed limit of the road, with the tester sitting in the co-pilot seat of the evaluation vehicle; Step 2: When the tester can clearly see the content of the target traffic sign, he / she immediately operates the trigger unit to control the processing unit to receive the point cloud data from the laser radar, the image from the camera unit, and the vehicle speed information from the speed sensor at this moment; Step 3: determining the type of the target traffic sign in the image by a processing unit, and determining the minimum visual recognition distance of the target traffic sign; Step 4: determining the straight-line visual distance of the target traffic sign based on the point cloud data of the laser radar, and if the straight-line visual distance is less than the minimum visual distance, determining that the function of the target traffic sign is abnormal; Step 5: If the straight line visual distance is greater than or equal to the minimum visual distance, determine whether the target traffic sign is within the effective field of view of the driver at this moment based on the point cloud data of the laser radar; if the target traffic sign is within the effective field of view of the driver, determine that the function of the target traffic sign is normal; Step 6: If the target traffic sign is not within the effective field of view of the driver, the processing unit executes steps 4, 5 and 6 again based on the point cloud data from the laser radar, the image from the camera unit and the vehicle speed information from the speed sensor at the next moment; The process of determining whether the target traffic sign is within the effective field of view of the driver includes: Determine the driver's effective horizontal viewing angle a and effective vertical viewing angle b based on the type and speed of the assessed vehicle; and Determine the horizontal angle and vertical angle of the target traffic sign and judge whether it meets the following conditions: Where α is the horizontal angle of the sign, β is the vertical angle of the sign, If the horizontal angle and the vertical angle of the target traffic sign meet the conditions, it is determined that the target traffic sign is within the effective field of view of the driver, otherwise it is determined that the target traffic sign is not within the effective field of view of the driver.

6. The method according to claim 5, wherein: The processing unit synchronously records the point cloud data of the laser radar and the image of the camera unit at a speed of not less than 10fps.

7. The method according to claim 5, wherein: The driver's effective horizontal and vertical viewing angles are: to 右 =83.08·e -0.014v a 右 is the effective horizontal viewing angle on the right side of the driver; a 左 is the effective horizontal viewing angle on the left side of the driver; θ1 is the angle between the driver's front direction and the right edge of the left A-pillar when the driver is in the driving position; v is the speed of the vehicle; v th1 is the critical speed, at which the left boundary of the driver's effective horizontal field of view coincides with the right edge of the left A-pillar of the vehicle; θ2 is the angle between the driver's front direction and the upper edge of the front window when the driver is in the driving position, v is the speed of the vehicle; v th2 is the critical vehicle speed, at which the upper boundary of the driver's effective vertical field of view coincides with the upper edge of the vehicle's front window, and μ is the proportional coefficient between the upper and lower field of view angles and the left and right field of view angles; When the target traffic sign is on the left side of the longitudinal centerline of the vehicle, a = a 左 , when the traffic sign is on the right side of the longitudinal centerline of the vehicle, a=a 右 .

8. The method according to claim 5, wherein: The process of determining the straight-line visual distance of the target traffic sign based on the laser radar point cloud data further includes: Step B1: establishing a mapping relationship between the horizontal and vertical coordinates of each scanning point in the point cloud data and the horizontal and vertical coordinates of each pixel point in the image; Step B2: identifying a target traffic sign from the image, and determining a sign scanning point in the point cloud data corresponding to the target traffic sign through the mapping relationship; Step B3: Calculate the straight-line visual distance of the target traffic sign using the following formula: Where n is the number of marker scanning points used to calculate the straight line visual distance, x i is the horizontal coordinate of the i-th marker scanning point, y i is the ordinate of the i-th marker scanning point, z i is the depth coordinate of the i-th marker scanning point.

9. The method according to claim 8, wherein: The horizontal angle α of the target traffic sign satisfies The vertical angle β of the target traffic sign satisfies in, For x i The average value of for y i The average value of For z i The average value of .

10. The method according to claim 8, wherein: The n sign scanning points used to calculate the straight-line visual distance of the target traffic sign are points within the range of the first 10% to the first 20% after all the sign scanning points are sorted from largest to smallest according to the straight-line distance.