A road sign recognition system and detection, positioning and evaluation method

Abstract

The present invention relates to a road sign recognition system and a detection, positioning, and evaluation method, which belongs to the technical field of road sign recognition, and includes: an industrial computer, an edge computing box, and an industrial camera installed on an unmanned vehicle, wherein the industrial computer is communicatively connected to the edge computing box and the industrial camera respectively. The industrial camera is used to capture an image of the road ahead of the unmanned vehicle, and the image of the road ahead includes road signs. The industrial computer is used to forward the image of the road ahead to the edge computing box. The edge computing box includes a target detection module, which is used to detect the image of the road ahead using a target detection model to obtain the category of the road sign and the location area of the road sign in the image of the road ahead. The edge computing box is used to detect the road sign, which has good real-time performance, and the target detection model is used to detect the road sign with high accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of road sign recognition, and in particular to a road sign recognition system for unmanned vehicles and a detection, positioning and evaluation method. Background Art

[0002] Road signs play a key role in traffic safety and are an important factor in guiding autonomous vehicles. When autonomous vehicles are driving on the road, they need to consider the road sign information in their decision-making and planning, and follow the information provided by the road signs.

[0003] Currently, road sign detection and positioning technologies mostly rely on high-performance computers, and cannot meet the real-time and accuracy requirements of unmanned vehicles.

[0004] Based on this, there is an urgent need for a high-precision and real-time road sign recognition technology. Summary of the Invention

[0005] The purpose of the present invention is to provide a road sign recognition system and a detection, positioning and evaluation method, which can realize the detection and positioning of road signs with high accuracy and good real-time performance, and can evaluate the positioning accuracy.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] In a first aspect, the present invention is used to provide a road sign recognition system, comprising: an industrial computer, an edge computing box, and an industrial camera installed on an unmanned vehicle; the industrial computer is communicatively connected to the edge computing box and the industrial camera respectively;

[0008] The industrial camera is used to capture an image of the road ahead of the unmanned vehicle; the image of the road ahead includes road signs;

[0009] The industrial computer is used to forward the front road image to the edge computing box;

[0010] The edge computing box includes a target detection module; the target detection module is used to detect the road image ahead using a target detection model to obtain the category of the road sign and the location area of the road sign in the road image ahead.

[0011] In some embodiments, the invention further comprises: a laser radar installed on the unmanned vehicle and communicatively connected to the industrial computer;

[0012] The laser radar is used to scan and obtain laser radar point cloud data in front of the unmanned vehicle;

[0013] The edge computing box also includes a target positioning module; the target positioning module is used to filter the lidar point cloud data according to the location area to obtain filtered point cloud data, and calculate the first relative position of the road sign relative to the unmanned vehicle based on the filtered point cloud data.

[0014] In some embodiments, the invention further includes: a global positioning system and an inertial navigation system installed on the unmanned vehicle and communicatively connected to the industrial computer;

[0015] The global positioning system is used to locate the position of the unmanned vehicle to obtain the first position coordinates of the unmanned vehicle;

[0016] The inertial navigation system is used to perform angular positioning on the unmanned vehicle to obtain a vehicle heading angle of the unmanned vehicle;

[0017] The edge computing box also includes a positioning accuracy assessment module; the positioning accuracy assessment module is used to convert the first position coordinate into a vehicle coordinate system based on the vehicle heading angle to obtain a second position coordinate, and calculate the second relative position of the road sign relative to the unmanned vehicle based on the second position coordinate; calculate the difference between the second relative position and the first relative position to obtain the positioning accuracy.

[0018] In some embodiments, the industrial camera transmits the front road image to the industrial computer in the format of a ROS topic; the lidar transmits the lidar point cloud data to the industrial computer in the format of a ROS topic; the global positioning system transmits the first position coordinates to the industrial computer in the format of a ROS topic; and the inertial navigation system transmits the vehicle heading angle to the industrial computer in the format of a ROS topic.

[0019] In a second aspect, the present invention provides a road sign detection method, comprising:

[0020] Acquire a road image in front of the unmanned vehicle; the road image in front includes road signs;

[0021] The target detection model is used to detect the road ahead image to obtain the category of the road sign and the location area of the road sign in the road ahead image.

[0022] In a third aspect, the present invention provides a method for locating a road sign, comprising:

[0023] Acquire a road image in front of the unmanned vehicle and laser radar point cloud data in front of the unmanned vehicle; the road image in front includes road signs;

[0024] Detecting the road ahead image using a target detection model to obtain a category of the road sign and a location area of the road sign in the road ahead image;

[0025] The laser radar point cloud data is filtered according to the position area to obtain filtered point cloud data, and a first relative position of the road sign relative to the unmanned vehicle is calculated based on the filtered point cloud data.

[0026] In some embodiments, filtering the laser radar point cloud data according to the location area to obtain the filtered point cloud data specifically includes:

[0027] Using a projection matrix between the laser radar coordinate system and the camera pixel coordinate system, the laser radar point cloud data is projected into the camera pixel coordinate system to obtain projection data of each radar point in the laser radar point cloud data;

[0028] The radar points whose projection data are not within the position area are removed from the laser radar point cloud data to obtain filtered point cloud data.

[0029] In some embodiments, before projecting the lidar point cloud data to the camera pixel coordinate system using the projection matrix between the lidar coordinate system and the camera pixel coordinate system, it also includes: performing preliminary screening on the lidar point cloud data to screen out radar points above the ground in front of the unmanned vehicle to form preliminary screened point cloud data, and using the preliminary screened point cloud data as new lidar point cloud data.

[0030] In some embodiments, calculating the first relative position of the road sign relative to the unmanned vehicle based on the filtered point cloud data specifically includes:

[0031] Performing plane fitting on the filtered point cloud data using the RANSAC method to obtain a fitting plane;

[0032] Calculating the distance between each radar point in the filtered point cloud data and the fitting plane, and removing radar points whose distance is greater than a preset threshold, to obtain point cloud data for calculation;

[0033] The centroid coordinates of the calculation point cloud data are calculated to obtain a first relative position of the road sign relative to the unmanned vehicle.

[0034] In a fourth aspect, the present invention provides a road sign evaluation method, comprising:

[0035] Acquire a road image in front of an unmanned vehicle, laser radar point cloud data in front of the unmanned vehicle, and a first position coordinate and a vehicle heading angle of the unmanned vehicle; the road image in front includes a road sign;

[0036] Detecting the road ahead image using a target detection model to obtain a category of the road sign and a location area of the road sign in the road ahead image;

[0037] Filtering the laser radar point cloud data according to the position area to obtain filtered point cloud data, and calculating a first relative position of the road sign relative to the unmanned vehicle based on the filtered point cloud data;

[0038] Based on the vehicle heading angle, the first position coordinate is converted into a vehicle coordinate system to obtain a second position coordinate, and a second relative position of the road sign relative to the unmanned vehicle is calculated based on the second position coordinate; the difference between the second relative position and the first relative position is calculated to obtain the positioning accuracy.

[0039] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0040] The present invention is used to provide a road sign recognition system and a detection, positioning, and evaluation method, including: an industrial computer, an edge computing box, and an industrial camera installed on an unmanned vehicle, and the industrial computer is respectively connected to the edge computing box and the industrial camera. The industrial camera is used to capture an image of the road ahead of the unmanned vehicle, and the image of the road ahead includes road signs. The industrial computer is used to forward the image of the road ahead to the edge computing box. The edge computing box includes a target detection module, and the target detection module is used to detect the image of the road ahead using a target detection model to obtain the category of the road sign and the location area of the road sign in the image of the road ahead. The edge computing box is used to detect the road sign, which has good real-time performance, and the target detection model is used to detect the road sign with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a schematic diagram of the structure of the identification system provided in Example 1 of the present invention;

[0043] Figure 2A flow chart of the detection method provided in Example 2 of the present invention;

[0044] Figure 3 A flowchart of the positioning method provided in Example 3 of the present invention;

[0045] Figure 4 This is a principle block diagram of the positioning method provided in Example 3 of the present invention;

[0046] Figure 5 A flowchart of the evaluation method provided in Example 4 of the present invention;

[0047] Figure 6 This is a principle block diagram of the evaluation method provided in Example 4 of the present invention;

[0048] Figure 7 This is a schematic diagram of coordinate system conversion provided by Example 4 of the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0050] The purpose of the present invention is to provide a road sign recognition system and a detection, positioning and evaluation method, which can realize the detection and positioning of road signs with high accuracy and good real-time performance, and can evaluate the positioning accuracy.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Example 1:

[0053] This embodiment is used to provide a road sign recognition system for detecting, locating and evaluating the positioning accuracy of road signs. Figure 1 As shown, it includes: an industrial computer, an edge computing box, and an industrial camera installed on an unmanned vehicle. The industrial computer is communicated with the edge computing box and the industrial camera respectively.

[0054] There can be two industrial cameras, which can be arranged at the front end of the vehicle body to shoot the road ahead of the unmanned vehicle, and obtain the image of the road ahead of the unmanned vehicle, which includes road signs.

[0055] The industrial computer forwards images of the road ahead to the edge computing box, which can be an NVIDIA Jetson AGX Xavier, offering excellent real-time performance and flexibility.

[0056] The edge computing box includes a target detection module, which is used to detect the road image ahead using a target detection model to obtain the category of the road sign and the location area of the road sign in the road image ahead.

[0057] The object detection module collects a large amount of image data containing various road sign types and manually annotates it to create a dataset. This dataset includes multiple images and the road sign category corresponding to each image. This dataset is used to train the object detection network. After training, a set of network weight parameters is obtained, resulting in an object detection model. Using this model, the object detection module receives real-time road images from an industrial camera and outputs road sign detection results at a high frame rate.

[0058] It should be noted that the categories of road signs referred to in this embodiment are obtained according to the national standard GB5768.2-2009 for traffic sign classification, specifically each subcategory under the major categories such as warning signs, prohibition signs, instruction signs, direction signs and auxiliary signs. For example, the two subcategories of no passage and no entry under the major category of prohibition signs are two different categories.

[0059] In this embodiment, the location area of the road sign in the front road image can be described by a 2D rectangular bounding box.

[0060] Road sign detection can be achieved through industrial cameras and target detection modules. In order to further locate road sign locations, the recognition system of this embodiment also includes: a laser radar installed on the unmanned vehicle and connected to the industrial computer for communication.

[0061] LiDAR can be mounted on the roof of an autonomous vehicle to scan the road ahead, generating LiDAR point cloud data. This data consists of multiple radar points, each with information such as 3D coordinates.

[0062] The edge computing box also includes a target positioning module, which is used to filter the lidar point cloud data according to the location area to obtain filtered point cloud data, and calculate the first relative position of the road sign relative to the unmanned vehicle based on the filtered point cloud data.

[0063] The positioning of road signs can be achieved through the laser radar and target positioning module. In order to further evaluate the positioning accuracy, the recognition system of this embodiment also includes: a global positioning system and an inertial navigation system installed on the unmanned vehicle and connected to the industrial computer.

[0064] The global positioning system is used to locate the position of the unmanned vehicle and obtain the first position coordinates of the unmanned vehicle.

[0065] The inertial navigation system is used to perform angular positioning of the unmanned vehicle and obtain the vehicle heading angle of the unmanned vehicle.

[0066] The edge computing box also includes a positioning accuracy assessment module, which is used to convert the first position coordinate into the vehicle coordinate system based on the vehicle heading angle to obtain a second position coordinate, and calculate the second relative position of the road sign relative to the unmanned vehicle based on the second position coordinate, calculate the difference between the second relative position and the first relative position, and obtain the positioning accuracy.

[0067] In this embodiment, the hardware devices installed on the unmanned vehicle (including industrial cameras, lidar, global positioning system, and inertial navigation system) all communicate with the edge computing box through the ROS system on the industrial computer, enabling data exchange. Specifically, the industrial camera transmits images of the road ahead to the industrial computer in the format of a ROS topic, which is then transmitted to the target detection module. The lidar transmits lidar point cloud data to the industrial computer in the format of a ROS topic, which is then transmitted to the target positioning module. The global positioning system transmits first position coordinates to the industrial computer in the format of a ROS topic, which is then transmitted to the positioning accuracy assessment module. The inertial navigation system transmits vehicle heading angles to the industrial computer in the format of a ROS topic, which is then transmitted to the positioning accuracy assessment module. The target detection module implements its detection function by integrating a target detection model into the ROS system. The target detection model can be a YOLOV5 model. The target positioning module receives the real-time output of the target detection module and the lidar point cloud data at the same time. Time synchronization of the input data is achieved through time synchronization of the ROS system.

[0068] ROS topics are a one-to-many, asynchronous communication mechanism. Publishers publish messages to topics, and subscribers subscribe to messages from them. One-to-many means a topic can have multiple subscribers, and asynchronous communication means publishers only publish topics without worrying about the number of subscribers or whether they successfully subscribe to messages. In this embodiment, topics for industrial cameras, lidar, global positioning systems, and inertial navigation systems are each provided by a node. The industrial computer receives and processes these four topics. Within this program framework, adding new functionality simply requires adding a node to receive messages, process data, and publish results, without modifying the existing program.

[0069] The target detection module in this embodiment uses the YOLOV5 model, a network with high recognition accuracy. Loaded on the NVIDIA Jetson AGX Xavier, it can achieve target detection at a high frame rate, meeting real-time requirements. The target localization module performs multiple screenings, ultimately refining the selection by fitting the road sign plane to locate the road sign. It also provides road sign normal vector information, providing more information for the unmanned platform's decision-making and planning layers. The positioning accuracy assessment module provides a more intuitive understanding and quantification of the accuracy of positioning results.

[0070] Example 2:

[0071] This embodiment is used to provide a road sign detection method, such as Figure 2 Shown, including:

[0072] S1: Acquire a road image in front of the unmanned vehicle; the road image in front includes road signs;

[0073] S2: Detecting the road ahead image using a target detection model to obtain the category of the road sign and the location area of the road sign in the road ahead image.

[0074] This embodiment collects a large amount of image data containing various types of road signs, creates a data set through manual annotation, and uses the data set to train a target detection network. After training, a set of network weight parameters can be obtained, and a target detection model can be obtained. The target detection model can be used to receive the road ahead image transmitted by the industrial camera in real time and output the detection results of the road sign at a high frame rate. The detection results include the category of the road sign and the location information of the road sign in the road ahead image. The location information can be described by a 2D rectangular bounding box, which constitutes the location area of the road sign in the road ahead image.

[0075] The target detection model of this embodiment can adopt the YOLOV5 model, which has a high recognition accuracy. Moreover, this method is loaded on the NVIDIA Jetson AGX Xavier and can achieve target detection at a high frame rate, meeting the real-time requirements. Therefore, the detection method of this embodiment can detect road signs with high precision and good real-time performance.

[0076] Example 3:

[0077] This embodiment is used to provide a method for locating road signs. Figure 3 Shown, including:

[0078] T1: Acquire a road image in front of the unmanned vehicle and laser radar point cloud data in front of the unmanned vehicle; the road image in front includes road signs;

[0079] T2: Detecting the road ahead image using a target detection model to obtain the category of the road sign and the location of the road sign in the road ahead image;

[0080] T2 in this embodiment is the same as S2 in Example 2, and will not be described again here.

[0081] T3: Filtering the laser radar point cloud data according to the position area to obtain filtered point cloud data, and calculating a first relative position of the road sign relative to the unmanned vehicle based on the filtered point cloud data.

[0082] like Figure 4 As shown, in T3, the lidar point cloud data is filtered according to the position area to obtain the filtered point cloud data, which may include: using the projection matrix between the lidar coordinate system and the camera pixel coordinate system to project the lidar point cloud data to the camera pixel coordinate system to obtain the projection data of each radar point in the lidar point cloud data; and removing the radar points whose projection data are not within the position area from the lidar point cloud data to obtain the filtered point cloud data.

[0083] This embodiment can pre-calibrate the projection matrix between the lidar coordinate system and the camera pixel coordinate system. Specifically, data from the industrial camera and lidar are collected at the same time, and the rotation transformation relationship between the lidar coordinate system and the camera pixel coordinate system is obtained through a joint calibration method of the camera and the lidar. The relationship is expressed as a projection matrix. The projection matrix includes a rotation matrix and a translation matrix. The projection matrix can describe the mapping relationship between the radar point in the lidar coordinate system and the pixel in the camera pixel coordinate system. The radar point in the lidar coordinate system can be uniquely mapped to a pixel in the camera pixel coordinate system.

[0084] It should be noted that the above-mentioned joint calibration method of the camera and the lidar adopts the existing calibration method, which will not be repeated here.

[0085] Preferably, before projecting the lidar point cloud data to the camera pixel coordinate system using the projection matrix between the lidar coordinate system and the camera pixel coordinate system, the positioning method of this embodiment also includes: performing preliminary screening of the lidar point cloud data, screening out radar points above the ground in front of the unmanned vehicle, forming preliminary screened point cloud data, and using the preliminary screened point cloud data as new lidar point cloud data, thereby reducing the number of radar points for the above-mentioned screening process, improving screening efficiency, and further improving positioning speed.

[0086] In T3, calculating the first relative position of the road sign relative to the unmanned vehicle based on the filtered point cloud data may include: performing plane fitting on the filtered point cloud data using a RANSAC method to obtain a fitting plane, where the fitting plane is the plane of the road sign in the vehicle coordinate system; calculating the distance between each radar point in the filtered point cloud data and the fitting plane, and removing radar points with a distance greater than a preset threshold, so as to filter out radar points that are too far from the fitting plane through the preset threshold, and treating these radar points as noise points and removing them to obtain calculation point cloud data; calculating the center of mass coordinates of the calculation point cloud data in the lidar coordinate system, converting the center of mass coordinates to the vehicle coordinate system, calculating the distance between the origin of the vehicle coordinate system and the center of mass coordinates of the road sign in the vehicle coordinate system, and obtaining the first relative position of the road sign relative to the unmanned vehicle in the vehicle coordinate system.

[0087] This embodiment can also calculate the plane equation and normal vector of the road sign based on the plane fitting result.

[0088] In this embodiment, the point cloud data is filtered multiple times. The format of the initial lidar point cloud data can be the ROS standard radar topic format sensor_msg::Pointcloud2. The filtering process is as follows:

[0089] (1) Because the field of view of the laser radar is directly in front of the vehicle, during the calculation process, the laser radar point cloud data is converted to the vehicle coordinate system, and all points with z < -A are filtered out to perform preliminary screening of the laser radar point cloud data to obtain the point cloud data after preliminary screening. The vehicle coordinate system uses the width direction of the unmanned vehicle as the x-axis, the length direction as the y-axis, and the height direction as the z-axis. The installation position of the inertial navigation system on the unmanned vehicle is used as the origin, and A is the installation height of the inertial navigation system on the unmanned vehicle. Generally, the inertial navigation system is installed at the midpoint of the rear seat of the unmanned vehicle. It should be noted that the coordinate axes of the vehicle coordinate system and the laser radar coordinate system in this embodiment are completely parallel, but the origin position is different. Therefore, the vehicle coordinate system and the laser radar coordinate system only have a translation relationship. By measuring the offset of the origin of the two coordinate systems on the x-axis, y-axis, and z-axis, the coordinate conversion can be easily achieved.

[0090] (2) The point cloud data after preliminary screening is projected to the camera pixel coordinate system through the projection matrix. The radar points outside the position area after projection are filtered out through the position area output by the target detection model (i.e., the target detection box) to obtain the filtered point cloud data.

[0091] (3) A plane is fitted using the RANSAC method, which is considered to be the fitting plane of the road sign in the vehicle coordinate system. A preset threshold of distance is set, and radar points whose distance from the fitting plane is greater than the preset threshold are filtered out to obtain point cloud data for calculation. The radar points in the point cloud data for calculation are considered to be the points where the laser radar hits the road sign, and the road sign is located based on the point cloud data for calculation.

[0092] The positioning method of this embodiment performs multiple screenings and finally performs refined screening by fitting the plane of the road sign. At the same time, it can provide normal vector information of the road sign, providing more information for the decision-making and planning layer of the unmanned platform.

[0093] Example 4:

[0094] This embodiment is used to provide a road sign evaluation method, such as Figure 5 Shown, including:

[0095] M1: Acquire a road image in front of the unmanned vehicle, laser radar point cloud data in front of the unmanned vehicle, and a first position coordinate and vehicle heading angle of the unmanned vehicle; the road image in front includes road signs;

[0096] M2: Detecting the road ahead image using a target detection model to obtain the category of the road sign and the location of the road sign in the road ahead image;

[0097] M2 of this embodiment is the same as S2 of Example 2, and will not be described again here.

[0098] M3: Filtering the laser radar point cloud data according to the position area to obtain filtered point cloud data, and calculating a first relative position of the road sign relative to the unmanned vehicle based on the filtered point cloud data;

[0099] M3 of this embodiment is the same as T3 of Example 3, and will not be described again here.

[0100] M4: Based on the vehicle heading angle, the first position coordinate is converted into the vehicle coordinate system to obtain a second position coordinate, and the second relative position of the road sign relative to the unmanned vehicle is calculated based on the second position coordinate; the difference between the second relative position and the first relative position is calculated to obtain the positioning accuracy.

[0101] The vehicle heading angle measured by the inertial navigation system is equal to the angle between the x-axis of the vehicle coordinate system and the x-axis of the UTM coordinate system. It is also equal to the angle between the y-axis of the vehicle coordinate system and the y-axis of the UTM coordinate system. That is, there is an angle between the coordinate axes of these two coordinate systems that is equal to the vehicle heading angle. Based on this angle, the data in the UTM coordinate system can be converted to the vehicle coordinate system.

[0102] like Figure 6 As shown, the first position coordinates of the unmanned vehicle obtained by the global positioning system are coordinates in the WGS1984 geographic coordinate system. The first position coordinates are first converted to the UTM coordinate system, and then the coordinates in the UTM coordinate system are converted to the vehicle coordinate system based on the vehicle heading angle to obtain the second position coordinates. At the same time, the coordinates of the road sign in the WGS1984 geographic coordinate system are pre-collected and converted to the UTM coordinate system, and then converted to the vehicle coordinate system based on the vehicle heading angle to obtain the true position coordinates of the road sign. The true position coordinates can be expressed as (x, y, z), which are the x-axis, y-axis, and z-axis coordinate values, respectively. That is, in this embodiment, the GPS coordinates of the road sign and the unmanned vehicle are first converted to the UTM coordinate system, and then the UTM coordinates are converted to the vehicle coordinate system based on the vehicle heading angle. The distance between the second position coordinates and the true position coordinates of the road sign is calculated to obtain a second relative position. This second relative position is used as the true value of the coordinate difference between the unmanned vehicle and the road sign. The first relative position output by M3 is used as the measurement value for evaluating positioning accuracy.

[0103] Coordinate transformation according to vehicle heading angle is shown in Figure 7 The coordinate transformation formula is as follows:

[0104] ;

[0105] ;

[0106] Among them, O W is the origin of the UTM coordinate system; O V is the origin of the vehicle coordinate system; (x0, y0) is the coordinate of the origin of the vehicle coordinate system in the UTM coordinate system; the coordinates of the road sign in the UTM coordinate system are (x1, y1), and the coordinates in the vehicle coordinate system are (x2, y2), and the vehicle heading angle is .

[0107] Tests have shown that this embodiment has high positioning accuracy for road signs when traveling at a speed of 30 km / h, with an absolute error of no more than 1 m. It can also access GPS and IMU data in real time to perform accuracy evaluation on real-time positioning results, allowing for a more intuitive understanding and quantification of the accuracy of positioning results.

[0108] This embodiment loads the target detection module, target positioning module, and positioning accuracy assessment module into the NVIDIA Jetson AGX Xavier, which is lightweight and easy to install on a vehicle. Based on this, a lightweight road sign recognition method for unmanned vehicles is proposed, which has the functions of detection, positioning, and positioning accuracy assessment. This method takes road signs into consideration when unmanned vehicles are driving on roads. It can detect the type of road signs within the vehicle's field of view and their relative position relative to the vehicle itself, and can also calculate the plane equation and normal vector of the road sign in the vehicle coordinate system. This method has been used in actual vehicles and has been proven to be effective and feasible.

[0109] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0110] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A road sign recognition system, characterized in that: include: Industrial computers, edge computing boxes, and industrial cameras installed on unmanned vehicles; The industrial computer is communicatively connected to the edge computing box and the industrial camera respectively; The industrial camera is used to capture an image of the road ahead of the unmanned vehicle; the image of the road ahead includes road signs; The industrial computer is used to forward the front road image to the edge computing box; The edge computing box includes a target detection module; the target detection module is used to detect the road image ahead using a target detection model to obtain the category of the road sign and the location area of the road sign in the road image ahead; It also includes: a laser radar installed on the unmanned vehicle and communicatively connected to the industrial computer; The laser radar is used to scan and obtain laser radar point cloud data in front of the unmanned vehicle; The edge computing box also includes a target positioning module; the target positioning module is used to perform a preliminary screening of the laser radar point cloud data, screen out radar points above the ground in front of the unmanned vehicle, specifically convert the laser radar point cloud data into a vehicle coordinate system, filter out all radar points with z<-A, where z is the z-axis coordinate and A is the installation height of the inertial navigation system on the unmanned vehicle, to form preliminary screened point cloud data, screen the preliminary screened point cloud data according to the position area to obtain screened point cloud data, and calculate the first relative position of the road sign relative to the unmanned vehicle based on the screened point cloud data, specifically using the RANSAC method to perform plane fitting on the screened point cloud data to obtain a fitting plane; calculate the distance between each radar point in the screened point cloud data and the fitting plane, and remove the radar points whose distance is greater than a preset threshold to obtain point cloud data for calculation; calculate the centroid coordinates of the calculation point cloud data to obtain the first relative position of the road sign relative to the unmanned vehicle; Also included: a global positioning system and an inertial navigation system installed on the unmanned vehicle and communicatively connected to the industrial computer; The global positioning system is used to locate the position of the unmanned vehicle to obtain the first position coordinates of the unmanned vehicle; The inertial navigation system is used to perform angular positioning on the unmanned vehicle to obtain a vehicle heading angle of the unmanned vehicle; The edge computing box also includes a positioning accuracy evaluation module; the positioning accuracy evaluation module is used to convert the first position coordinate to the vehicle coordinate system based on the vehicle heading angle to obtain a second position coordinate, and calculate the second relative position of the road sign relative to the unmanned vehicle based on the second position coordinate. Specifically, the first position coordinate is converted to the UTM coordinate system, and then converted to the vehicle coordinate system based on the vehicle heading angle to obtain the second position coordinate. At the same time, the coordinates of the road sign in the WGS1984 geographic coordinate system are pre-collected, and the coordinates are converted to the UTM coordinate system, and then converted to the vehicle coordinate system based on the vehicle heading angle to obtain the real position coordinates of the road sign, and the distance between the second position coordinate and the real position coordinate of the road sign is calculated to obtain the second relative position; the difference between the second relative position and the first relative position is calculated to obtain the positioning accuracy.

2. The road sign recognition system according to claim 1, characterized in that: The industrial camera transmits the front road image to the industrial computer in the format of a ROS topic; the lidar transmits the lidar point cloud data to the industrial computer in the format of a ROS topic; the global positioning system transmits the first position coordinates to the industrial computer in the format of a ROS topic; the inertial navigation system transmits the vehicle heading angle to the industrial computer in the format of a ROS topic.

3. A road sign evaluation method, characterized in that: include: Acquire a road image in front of an unmanned vehicle, laser radar point cloud data in front of the unmanned vehicle, and a first position coordinate and a vehicle heading angle of the unmanned vehicle; the road image in front includes a road sign; Detecting the road ahead image using a target detection model to obtain a category of the road sign and a location area of the road sign in the road ahead image; Performing a preliminary screening on the laser radar point cloud data to screen out radar points above the ground in front of the unmanned vehicle, specifically converting the laser radar point cloud data into a vehicle coordinate system, filtering out all radar points with z<-A, where z is the z-axis coordinate and A is the installation height of the inertial navigation system on the unmanned vehicle, to form preliminary screened point cloud data, screening the preliminary screened point cloud data according to the position area to obtain screened point cloud data, and calculating a first relative position of the road sign relative to the unmanned vehicle based on the screened point cloud data, specifically performing plane fitting on the screened point cloud data using a RANSAC method to obtain a fitting plane; calculating a distance between each radar point in the screened point cloud data and the fitting plane, and removing radar points whose distance is greater than a preset threshold to obtain point cloud data for calculation; Calculating the centroid coordinates of the calculation point cloud data to obtain a first relative position of the road sign relative to the unmanned vehicle; Based on the vehicle heading angle, the first position coordinate is converted into the vehicle coordinate system to obtain the second position coordinate, and the second relative position of the road sign relative to the unmanned vehicle is calculated based on the second position coordinate. Specifically, the first position coordinate is converted into the UTM coordinate system, and then converted into the vehicle coordinate system based on the vehicle heading angle to obtain the second position coordinate. At the same time, the coordinates of the road sign in the WGS1984 geographic coordinate system are pre-collected, and the coordinates are converted into the UTM coordinate system, and then converted into the vehicle coordinate system based on the vehicle heading angle to obtain the real position coordinates of the road sign. The distance between the second position coordinate and the real position coordinate of the road sign is calculated to obtain the second relative position; the difference between the second relative position and the first relative position is calculated to obtain the positioning accuracy.

4. The road sign evaluation method according to claim 3, characterized in that: The filtering of the initially filtered point cloud data according to the location area to obtain the filtered point cloud data specifically includes: Projecting the preliminarily screened point cloud data onto the camera pixel coordinate system using a projection matrix between the laser radar coordinate system and the camera pixel coordinate system to obtain projection data for each radar point in the preliminarily screened point cloud data; The radar points whose projection data are not within the position area are removed from the preliminarily filtered point cloud data to obtain filtered point cloud data.

Images