A road sign intelligent recognition assisted fusion navigation positioning method and device

Abstract

The present invention discloses a fusion navigation positioning method and device assisted by intelligent identification of road signs, which is applied to the field of navigation and positioning technology. In the method, the state of the satellite navigation system GNSS is first evaluated, and GNSS / INS combined navigation is performed when the GNSS state is good, otherwise INS / DR combined navigation is performed, and high-precision navigation positioning is achieved through road sign information processing and road sign information assisted navigation correction during the INS / DR combined navigation process. The present invention utilizes the uniqueness, easy identification, accurate location information and easy acquisition characteristics of most road signs. In scenarios where the availability of satellite navigation systems is insufficient, the accurate location information provided by road signs is used to provide location correction for the INS / DR combined navigation system, thereby achieving high-precision positioning in scenarios where GNSS is unavailable.

Description

Technical Field

[0001] The present invention relates to the field of navigation and positioning technology, and in particular to a road sign intelligent recognition assisted fusion navigation and positioning method and device. Background Art

[0002] High-precision positioning has important applications in many fields. For example, with the rapid development of autonomous driving technology, navigation systems need to meet higher requirements, including: high precision, high resolution, high reliability, low cost, low power consumption and miniaturization. However, the current main navigation and positioning technologies cannot meet these requirements alone.

[0003] The Global Navigation Satellite System (GNSS) is currently the main positioning technology. It has the advantages of global coverage, all-weather operation, and high positioning accuracy, making it the most widely used positioning and navigation method. However, in some cases, GNSS availability will be impaired and accuracy will decrease. For example, in urban canyon areas, the GNSS positioning accuracy will drop sharply or even fail due to the obstruction of satellite signals, multipath reflection, etc. For another example, GNSS cannot be used in scenarios such as tunnels and underground garages. Moreover, if the GNSS signal is weak, it is easy to be overwhelmed by intentional or unintentional interference signals and cannot be used.

[0004] Inertial Navigation System (INS) is an autonomous navigation system that does not rely on external information. Its working principle is based on Newton's laws of motion. It is based on the carrier's motion acceleration measured by the accelerometer and the carrier's rotational angular velocity measured by the gyroscope. These measurements are processed by a computer to obtain the carrier's attitude, velocity and position. INS has the advantages of high data update rate, short-term accuracy and good stability. With the maturity of micro-electromechanical system (MEMS) processing technology, MEMS inertial devices (accelerometers and gyroscopes) have achieved unprecedented development and are widely used in integrated navigation systems with their advantages of low price, light weight, small size and low power consumption. However, the existing MEMS devices have low accuracy and cannot maintain positioning accuracy for a long time and long distance. Therefore, INS usually requires other information assistance to correct errors in a timely manner.

[0005] Since GNSS and INS have good complementarity, they are generally combined to form a GNSS / INS integrated navigation system, which enables the navigation system to provide stable and reliable navigation information.

[0006] In addition, for vehicle navigation systems, the combination of inertial navigation and odometer, i.e. INS / DR combined navigation, is also a technical solution. In this way, when the availability of GNSS decreases, the odometer provides the INS with the movement speed of the vehicle system, forming a dead reckoning (DR) mode, which can partially correct the rapid divergence of errors during pure INS navigation. However, the accuracy of this method is limited by factors such as the accuracy of MEMS devices and the accuracy of the odometer, and 1NS / DR combined navigation cannot provide high-precision positioning for a long time and long distance.

[0007] In a multi-source fusion high-precision positioning method and device patent with publication number CN110779521A, on the basis of using satellite navigation and inertial navigation combined navigation, a combination of odometer and inertial navigation is used to provide high-precision position information when satellite navigation is not available. In particular, in view of several factors that affect the accuracy of the combination of odometer and inertial navigation, corresponding technical solutions are proposed to improve the positioning accuracy of the combination of odometer and inertial navigation. However, this patent only provides high-precision position information with the assistance of odometer when satellite navigation is not available, and still cannot provide high-precision positioning for a long time and long distance. Summary of the invention

[0008] Purpose of the invention: In view of the defect in the prior art that high-precision positioning cannot be obtained when the satellite navigation system is not available, the present invention discloses a fusion navigation positioning method and device assisted by intelligent recognition of road signs, which achieves high-precision positioning in various scenarios by integrating the intelligent recognition technology of road signs into navigation.

[0009] Technical solution: In order to achieve the above technical objectives, the present invention adopts the following technical solution.

[0010] A road sign intelligent recognition assisted fusion navigation positioning method comprises the following steps:

[0011] S1. Satellite navigation system status assessment: The satellite navigation system is GNSS. Check whether the GNSS signal is stably received and the system status is good. If the GNSS status is good, perform GNSS / INS combined navigation and execute step S2. Otherwise, perform INS / DR combined navigation and road sign information assisted navigation correction and execute step S3.

[0012] S2. Performing combined navigation of satellite navigation system and inertial navigation system: Inertial navigation system is INS, and GNSS / INS combined navigation is realized by GNSS and INS. The state of carrier motion is measured by INS, the position of the moving carrier is calculated, and the real-time position and attitude information of the moving carrier is output by combining the GNSS positioning and orientation results with the INS positioning and orientation results, so as to realize navigation positioning, and then return to step S1 to enter the next processing cycle;

[0013] S3. Perform inertial navigation system and dead reckoning combined navigation: Dead reckoning is DR. INS and DR implement INS / DR combined navigation system. INS measures the state of the moving carrier, calculates the position of the moving carrier, combines DR positioning and orientation results with 1NS positioning and orientation results, and outputs real-time moving carrier position and attitude information;

[0014] S4, road sign information processing: collecting a road image on a moving carrier, extracting a number of road sign areas in the road image, identifying road sign information in all extracted road sign areas, searching the identification results in a road sign database, and executing step S5 if the road sign information can be obtained during the search process, otherwise returning to step S1 and entering the next processing cycle;

[0015] S5. Perform auxiliary navigation correction on road sign information: obtain a number of real-time road sign image information based on the road image, and determine in turn whether the moving carrier passes through the location of the road sign. After a successful determination, when the moving carrier passes through the location of the road sign, correct the moving carrier position output in step S3, output real-time moving carrier position and posture information, implement auxiliary navigation positioning, return to step S1, and wait to enter the next processing cycle; otherwise, return to step S4 and re-collect the road image.

[0016] Preferably, the specific process of processing the road sign information in S4 includes:

[0017] S41, acquiring road images: a camera fixed in front of the moving carrier captures road images in the moving carrier's forward direction in real time, and retains the moment of each image acquisition for subsequent calculations;

[0018] S42, identifying a number of road sign areas: processing the acquired road image to identify an image area containing road signs; if the identified road sign area is a rectangle, it indicates that the road sign is located directly in front of the driving direction, and then executing step S43; otherwise, it indicates that there is no road sign area in the road image, or the road sign is not in front of the driving direction, and then returning to step S1 to enter the next processing cycle;

[0019] S43, intelligent recognition of road signs: intelligently recognize road signs in the road sign area, including the text, traffic signs and relative positional relationship between the parts of the road signs; if the recognition is successful, execute step S44; otherwise, return to step S1 and enter the next processing cycle;

[0020] S44, road sign information retrieval: according to the identification information in S43, the corresponding road sign information is retrieved from the road sign database; if the corresponding road sign information is not retrieved, then return to step S1 and enter the next processing cycle; otherwise, execute step S45;

[0021] S45, output the retrieved road sign information: the road sign information includes the road sign number, the Chinese characters on the road sign, the geographical coordinates of the road sign, and the size of the road sign; output the retrieved road sign information and end the processing, and execute step S5.

[0022] Preferably, the road sign geographical coordinates include the longitude, latitude and altitude of the road sign.

[0023] Preferably, the specific process of performing road sign information assisted navigation correction in S5 includes:

[0024] S51, obtaining a number of real-time road sign image information according to the road image, and determining whether the current road sign information can be used for navigation and positioning: obtaining a number of real-time road sign image information, the road sign image information including the length and width pixel numbers of the road sign, and then calculating the long side distance estimation value and the short side distance estimation value; comparing the difference between the long side distance estimation value and the short side distance estimation value, if the difference is greater than a first threshold value, it is determined that the current road sign information has a large error and is not suitable for navigation and positioning, and the processing process ends, returning to step S1, and entering the next processing cycle; otherwise, executing step S52;

[0025] S52, calculating the distance between the moving carrier and the road sign according to the long side distance estimation value and the short side distance estimation value, and judging whether the moving carrier passes the location of the road sign: the distance between the moving carrier and the road sign is the average value of the long side distance estimation value and the short side distance estimation value, and comparing the average value with the second threshold value, if the average value is less than the second threshold value, the moving carrier passes the location of the road sign, and executing step S53; otherwise, it cannot be determined whether the moving carrier passes the location of the road sign, and returning to step S51;

[0026] S53, road sign information assisted navigation correction: obtain road sign information in the road sign database, and output the motion carrier position information based on the precise positioning calculation of the road sign in the road sign information.

[0027] Preferably, the specific process of calculating the long side distance estimation value and the short side distance estimation value in step S51 includes: obtaining a plurality of real-time road sign image information, the road sign image information including the length and width pixel numbers of the road sign, and then calculating the distance between the moving carrier and the road sign:

[0028]

[0029]

[0030] where w 1 、p 1 and d 1 are the actual length of the long side of the road sign, the number of long pixels and the estimated value of the long side distance, respectively. 2 、p 2 and d 2 are the actual length of the short side of the road sign, the number of wide pixels and the estimated distance of the short side, respectively, and f is the focal length of the camera.

[0031] Preferably, the specific process of combining the GNSS positioning and orientation results with the INS positioning and orientation results in S2 or combining the DR positioning and orientation results with the INS positioning and orientation results in S3 includes: combining the GNSS positioning and orientation results or the DR positioning and orientation results with the INS positioning and orientation results using the Kalman filtering method.

[0032] A fusion navigation positioning device assisted by intelligent recognition of road signs, used to implement any of the above-mentioned fusion navigation positioning methods assisted by intelligent recognition of road signs, comprising: a fusion navigation processing module, a road sign information processing module and a road sign information database connected in sequence;

[0033] The fusion navigation processing module includes a fusion navigation processing computer and an inertial measurement device connected to the fusion navigation processing computer, a GNSS satellite navigation module, an odometer, a camera, a wireless communication module and an antenna, and a power module;

[0034] The memory of the fusion navigation processing computer stores a road sign database; the road sign information processing module includes a camera connected to the fusion navigation processing computer.

[0035] Preferably, the inertial measurement device is an accelerometer or a gyroscope.

[0036] Preferably, the fusion navigation positioning device also includes a network server or a cloud computing server, and the network server or cloud computing server is used to store a road sign database, and the network server or cloud computing server is connected to the fusion navigation processing computer via a wireless communication module and an antenna.

[0037] Beneficial effects:

[0038] 1. In scenarios where satellite navigation systems are unavailable and INS / DR combined navigation systems are used for navigation and positioning, such as urban canyons, tunnels, underground garages, etc., the present invention uses the precise location information provided by road signs to provide auxiliary position correction for the INS / DR combined navigation system, fundamentally suppressing the trend of INS / DR errors accumulating over time and continuously diverging, eliminating navigation and positioning errors, and achieving high-precision positioning;

[0039] 2. The present invention uses technologies such as intelligent image processing and text recognition to process road sign information. Road signs have the advantages of uniqueness, easy identification, accurate location information and easy acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a general method flow chart of the present invention;

[0041] Figure 2 It is a module composition diagram of the present invention;

[0042] Figure 3 Correction process for INS / DR navigation;

[0043] Figure 4 This is a flowchart for processing road sign information;

[0044] Figure 5 It is a block diagram of the device composition of the present invention;

[0045] Figure 6 This is an example diagram of a road sign that can be used for auxiliary positioning in an urban road of the present invention;

[0046] Figure 7 This is an example diagram of a road sign that can be used for auxiliary positioning in the highway of the present invention;

[0047] Figure 8 This is an example diagram of a road sign that can be used for auxiliary positioning in a county-level highway of the present invention;

[0048] Fig. 9 This is an example diagram of a road sign that cannot be used for auxiliary positioning according to the present invention;

[0049] Fig.10 Extract example images for road sign image regions;

[0050] Fig.11 This is an example diagram of the process of road sign distance decreasing from far to near. DETAILED DESCRIPTION

[0051] The present invention is further described and explained below in conjunction with the accompanying drawings.

[0052] There are a large number of various signs and signs placed in and around urban roads. For example, road traffic signs use text or graphic symbols to convey instructions, prohibitions and other signals to vehicles and pedestrians, or indicate road directions, road names, etc. For another example, there are some signs for publicity and advertising in places near the road. Some of the information provided by such signs can be used for positioning. Such signs have the following characteristics:

[0053] 1) The content of these signs is easy to identify. For example, traffic signs are usually white text on a blue background. The content of the signs is mainly text, without complex patterns, and is easy for computers to identify.

[0054] 2) These signs are unique within a small area. First, the road names within a region are generally unique; second, even if there are signs with the same road names and content in different regions (such as different cities and counties), the correct sign can be easily identified by the current approximate location.

[0055] 3) The precise location information of these signs is known. In particular, if we regard a road sign as a plane, its precise location can be defined as the precise location of its center point, including the longitude, latitude, and altitude of the point. In addition, the height of the center point of the road sign from the ground is also easy to obtain in advance.

[0056] 4) The sizes of these road signs are known, including the length in the left-right direction, the width or height in the up-down direction, etc.

[0057] The attached figure shows some examples of road signs that can be used for auxiliary positioning and those that cannot be used for auxiliary positioning. Figure 6 This is an example diagram of a road sign that can be used for auxiliary positioning in an urban road of the present invention; Figure 7 This is an example diagram of a road sign that can be used for auxiliary positioning in the highway of the present invention; Figure 8 This is an example of a road sign that can be used for auxiliary positioning in a county-level highway of the present invention. The recognition accuracy of such road signs may be difficult to guarantee. Such road signs with insufficient recognition accuracy should not be included in the road sign information database. Fig. 9 This is an example diagram of a road sign that cannot be used for auxiliary positioning in the present invention. This type of road sign is not unique on the road.

[0058] Based on the above road sign features, the present invention utilizes the information provided by these road signs to achieve high-precision combined navigation.

[0059] The following is an explanation of the letter definitions:

[0060] GNSS: Global Navigation Satellite System; GNSS is a radio navigation technology. The errors of GNSS are all random errors, such as atmospheric error, ionospheric delay error and multipath effect error. The advantages of GNSS are high accuracy, no error accumulation and cheap receivers; while the disadvantages are discontinuous output, inability to output attitude information and high cost.

[0061] INS: Inertial Navigation System, which uses inertial instruments for navigation and positioning; inertial instruments include accelerometers and inertial sensitive devices such as traditional mechanical gyroscopes, vibration gyroscopes, optical gyroscopes, MEMS / MOMES gyroscopes, superconducting magnetic levitation gyroscopes, etc. The errors of INS include position error, velocity error and attitude error. All errors include deterministic errors such as errors related to acceleration and random errors such as first-order Markov processes, and the errors accumulate over time; its advantages are strong autonomy, high short-term accuracy, and the ability to continuously provide position, velocity and attitude; but its disadvantage is that the navigation error accumulates over time, and the higher the navigation accuracy, the more expensive it is.

[0062] DR: Dead Reckoning, which is dead reckoning, realizes navigation and positioning through a speed meter or odometer. Its advantages are autonomous positioning, simple structure and low cost. Its disadvantages are large error accumulation, which is limited to occasions with low requirements.

[0063] In integrated navigation, loop feedback method or optimal estimation method is used to achieve performance complementarity of the combination of various systems. The optimal estimation method mainly uses the Kalman filter algorithm to estimate and eliminate the system error from the perspective of probability statistics optimality.

[0064] The present invention discloses a road sign intelligent recognition assisted fusion navigation positioning method and device, which is used for high-precision positioning of moving carriers such as vehicle-mounted systems and other objects; wherein the device includes three parts, and the relationship between them is as shown in the attached figure. Figure 2 As shown:

[0065] Module 1: Fusion navigation processing module is the main control module, which is used to process the fusion of satellite navigation, odometer and road image position information. It evaluates the combined navigation status according to the status of satellite navigation, odometer, etc., and feeds it back to module 2. At the same time, it receives the road sign position information output by module 2 and performs fusion navigation processing.

[0066] Module 2: Road sign image processing module, used for road image information collection and analysis. It retrieves road sign information from the road sign information database based on the results of road sign image recognition and obtains the retrieval results from the database; at the same time, it receives the fusion status information fed back by module 1 and outputs the road sign location information to module 1 under certain conditions, and module 1 performs fusion navigation processing.

[0067] Module 3: Road sign information database, used to store road sign information. It accepts the search request from module 2 and feeds back the search result to module 2.

[0068] Module 1: Fusion Navigation Processing Module

[0069] The fusion navigation processing module is the main control module. In each processing cycle, it executes GNSS / INS integrated navigation or INS / DR integrated navigation according to the GNSS status. During INS / DR integrated navigation, the function of executing module 2 is to process road sign information and assist in navigation correction.

[0070] As attached Figure 1 As shown, the specific steps include:

[0071] Step 1.1: GNSS status assessment. Generally, the GNSS receiver will output some status to declare the availability, accuracy level and other information of the current positioning and orientation output. This information can be used to describe the status of the GNSS. When the GNSS can provide effective positioning and orientation output, the GNSS status is good; otherwise, the GNSS status is considered to be bad. The GNSS status can be determined based on the status flag output by the GNSS module. If the GNSS status is good, GNSS / INS combined navigation is performed, i.e., step 1.2; otherwise, when the GNSS status is not good, such as in an underground garage, tunnel or other area where the GNSS signal is completely shielded, or in an area where the GNSS positioning error is large on a road with tall buildings in the city, INS / DR combined navigation and road sign information assisted navigation correction are performed, i.e., step 1.3 and subsequent steps.

[0072] Step 1.2: Execute GNSS / INS integrated navigation processing. GNSS / INS integrated navigation is based on inertial measurement components, such as accelerometers, gyroscopes, etc., to measure the motion state of the carrier, calculate the carrier position, and use the Kalman filter method to combine the GNSS positioning and orientation results with the INS positioning and orientation results to output real-time carrier position, attitude and other information. GNSS / INS integrated navigation is a relatively mature technology with many implementation schemes. After executing this step, return to step 1.1 and enter the next processing cycle.

[0073] Step 1.3: INS / DR integrated navigation processing. INS / DR integrated navigation is similar to GNSS. It is also based on the measurement of the carrier's motion state by inertial measurement components (accelerometers, gyroscopes, etc.) to estimate the carrier's position. The difference is that it uses the Kalman filter method to combine the DR positioning and orientation results with the INS positioning and orientation results. INS / DR integrated navigation is a relatively mature technology with many implementation solutions.

[0074] Step 1.4: Road sign information processing. Execute the processing of module 2, including road image acquisition, road sign area extraction, road sign information recognition, road sign database retrieval and other steps. If the road sign information processing returns a successful result, execute step 1.5; otherwise, return to step 1.1 and enter the next processing cycle.

[0075] Step 1.5: INS / DR navigation correction processing. By estimating and tracking the distance to the road sign, determine whether the vehicle system has passed the location of the road sign. When the vehicle system passes the location of the road sign, correct the position of the INS / DR combined navigation. After the correction is completed, return to step 1.1 and enter the next processing cycle.

[0076] The calculation of the carrier posture can refer to Yan Gongmin's doctoral dissertation "Research on Vehicle-mounted Autonomous Positioning and Orientation System" published by Northwestern Polytechnical University in 2006 and Yan Gongmin and Weng Jun's "Strapdown Inertial Navigation Algorithm and Integrated Navigation Principle" published by Northwestern Polytechnical University Press in 2019; the Kalman filtering method estimates and eliminates the system error of the combined navigation from the perspective of probabilistic statistical optimality.

[0077] The process of INS / DR navigation correction processing is as follows: Figure 3 As shown, the specific steps include the following steps.

[0078] Step 1.5.1: Estimate the distance between the moving carrier and the road sign based on the number of pixels of the length and width of the road sign.

[0079] The method for estimating the distance d between the road sign and the shooting point (vehicle system) from the road sign image is based on the focal length f (unit: meter) of the camera and the size w (unit: meter) of the target object and the number of pixels p of the target object in the image, and is estimated according to the following formula:

[0080]

[0081] The focal length f of the camera is given by the working parameters of the camera; the actual length w of the long side or short side of the road sign is provided by the road sign information processing module.

[0082] Let the actual length of the long side of the road sign be w 1 , the number of pixels is p 1 , the actual length of the short side of the road sign is w 2 , the number of pixels is p 2 , then according to formula (2) and formula (3), two estimated distances, namely the estimated value of the long side distance d, are estimated respectively. 1 and the short side distance estimate d 2 ;

[0083]

[0084]

[0085] Then, compare d 1 and d 2 If the difference between the two values ​​is large, that is, |d 1 -d 2 |>d 0, where d 0 is the first threshold value, i.e., a preset threshold value, such as d 0 If it is set to 2 meters, it means that the current road sign information has a large error and is not suitable for positioning, and the processing process ends. Otherwise, it means that the current road sign image information is stable and can be used for positioning correction, that is, execute step 1.5.2.

[0086] Step 1.5.2: Calculate the distance between the moving carrier and the road sign based on the long side distance estimate and the short side distance estimate.

[0087] The distance d between the moving carrier and the road sign can be taken as the average value of the two.

[0088] d=(d 1 +d 1 ) / twenty four)

[0089] Then, it is determined whether the vehicle-mounted system has reached the location of the road sign.

[0090] The distance d between the moving carrier and the road sign can be compared with the preset threshold value d th That is, the second threshold value comparison, if d<d th , it means that the vehicle system is passing the location of the road sign, and then execute the next step. th The value can be 0.5 meters. Otherwise, it means that the vehicle system has not yet reached the location of the road sign, return to step 1.5.1, continue to monitor the distance to the road sign, and Fig.11 This is an example of the process of road signs changing from far to near.

[0091] Step 1.5.3: Correct the positioning output of the integrated navigation according to the location information of the road sign.

[0092] When the vehicle-mounted system passes the location of the road sign, the position of the INS / DR combined navigation is corrected, that is, the precise position of the road sign is used as the position output of the combined navigation. However, the height of the output position needs to be the height of the ground below the road sign obtained by subtracting the height of the road sign from the height of the road sign from the ground, so as to obtain an accurate position output.

[0093] Module 2: Road sign information processing module

[0094] The road sign information processing module is used to collect and analyze road image information and retrieve the road sign information database. Figure 4 This is a flowchart of road sign information processing. Figure 4 As shown, the specific steps include the following steps.

[0095] Step 2.1: Obtain road images. The camera fixed in front of the vehicle captures the road image in the vehicle's forward direction in real time. The moment of image acquisition is retained for subsequent estimation of driving distance.

[0096] Step 2.2: Identify road sign areas. Use a deep learning-based target detection algorithm to process the acquired road image. Preferably, a regression-based target detection algorithm, such as an SSD-based algorithm, a ResNet algorithm, or a RefineDet algorithm, is used to quickly complete the road sign area identification. If there is a complete road sign in the image, the image of the road sign area is output, otherwise the sign of unsuccessful road sign identification is output. Fig.10 As shown, the acquired road image is processed to identify the image area containing the road sign. If the road sign area is identified and the road sign area is rectangular, it means that the road sign is located directly in front of the driving direction, and step 2.3 is executed; otherwise, it means that there is no road sign in the image area, or the road sign is not in front of the driving direction, and the processing ends.

[0097] Step 2.3: Intelligent recognition of road sign information. Use an intelligent algorithm based on deep learning to intelligently recognize information in the image area containing the road sign, including the text, traffic signs, and the relative position relationship between the various parts of the road sign. Preferably, a fully convolutional semantic segmentation module is used to process the road sign image area to accurately obtain multiple sub-areas such as different Chinese characters, pinyin, and numbers in the road sign; the text recognition module and the number recognition module use the SSD algorithm, R-CNN algorithm, etc. to recognize text or numbers in each sub-area respectively; then, the recognized content is used to generate the feature information of the road sign. If the above information is successfully recognized, execute step 2.4; otherwise, end the processing.

[0098] Step 2.4: Retrieve road sign information. Retrieve the corresponding road sign information from the database using the road sign features identified in the previous step. If necessary, information to disambiguate road signs should be provided during the retrieval, such as the current region (city, county, or district). If no road sign that meets the conditions can be retrieved, the process ends. If a road sign that meets the conditions is retrieved, proceed to step 2.5.

[0099] Step 2.5: Output the retrieved road sign information, which may include the road sign number, road sign geographic coordinates (longitude, latitude and altitude), and road sign size (length, width or height). End the process.

[0100] Module 3: Road sign information database, storing road sign information.

[0101] The road sign information database is used to store road sign information, and respond to the road sign information retrieval request of module 2 and return the corresponding road sign information. Figure 6 As shown, the road sign information includes the following:

[0102] Road sign number: a globally unique number assigned to each road sign in the database;

[0103] Text on road signs: including Chinese characters, pinyin and English, used to indicate road names, directions, traffic instructions, etc.;

[0104] Symbols on road signs: including the indication symbols on road signs, such as left turn, right turn, straight ahead or a combination of multiple symbols;

[0105] Layout of road sign content: including the relative positions of road characters;

[0106] Size of road signs: length, width (or height), etc.

[0107] The location of the road sign refers to the geographical coordinates of the center point of the road sign, which generally includes three components: longitude, latitude and altitude. In addition, in order to eliminate the ambiguity of road signs caused by the same name of roads in different regions, the road sign information center also includes information on the province, city and district to which the road sign belongs.

[0108] The road sign information database of the present invention can be stored in the memory of the navigation device or in a network server. If it is stored in a network server or a cloud computing server, the database server is accessed through a wireless communication network.

[0109] The present invention utilizes the uniqueness, easy identification, accurate location information and easy acquisition characteristics of most road signs, and with the help of technologies such as intelligent image processing and text recognition, in scenarios where satellite navigation systems are not available and INS / DR combined navigation systems are used for navigation and positioning, such as urban canyons, tunnels, underground garages and other scenarios, the present invention utilizes the accurate location information provided by road signs to provide auxiliary position correction for the INS / DR combined navigation and positioning system, fundamentally suppressing the trend of 1NS / DR errors accumulating over time and continuously diverging, eliminating navigation and positioning errors, and achieving high-precision positioning.

[0110] As attached Figure 5 As shown, the present invention also provides a high-precision fusion positioning device assisted by road signs, including: a fusion navigation processing computer, and an inertial measurement device (including a gyroscope and an accelerometer, etc.) connected to the integrated navigation computer, a GNSS satellite navigation module, an odometer, a camera, a wireless communication module and an antenna, and a power supply module. The fusion navigation computer is used to execute a fusion navigation positioning method assisted by intelligent recognition of road signs of the present invention.

[0111] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A fusion navigation positioning method assisted by intelligent recognition of road signs, It is characterized in that The following steps are involved: S1. Satellite navigation system status assessment: The satellite navigation system is GNSS. Check whether the GNSS signal is stably received and the system status is good. If the GNSS status is good, GNSS / INS integrated navigation is performed and step S2 is executed; otherwise, INS / DR integrated navigation and road sign information assisted navigation correction are performed and step S3 is executed; S2. Performing combined navigation of satellite navigation system and inertial navigation system: Inertial navigation system is INS, and GNSS / INS combined navigation is realized by GNSS and INS. The state of carrier motion is measured by INS, the position of the moving carrier is calculated, and the real-time position and attitude information of the moving carrier is output by combining the GNSS positioning and orientation results with the INS positioning and orientation results, so as to realize navigation positioning, and then return to step S1 to enter the next processing cycle; S3. Perform inertial navigation system and dead reckoning combined navigation: Dead reckoning is DR. INS and DR implement INS / DR combined navigation system. INS measures the state of the moving carrier, calculates the position of the moving carrier, combines the DR positioning and orientation results with the INS positioning and orientation results, and outputs the real-time position and attitude information of the moving carrier. S4, road sign information processing: collecting a road image on a moving carrier, extracting a number of road sign areas in the road image, identifying road sign information in all extracted road sign areas, searching the identification results in a road sign database, and executing step S5 if the road sign information can be obtained during the search process, otherwise returning to step S1 and entering the next processing cycle; S5, perform road sign information auxiliary navigation correction: obtain a number of real-time road sign information according to the road image, and judge in turn whether the moving carrier passes through the location of the road sign. After the judgment is successful, when the moving carrier passes through the location of the road sign, correct the moving carrier position output in step S3, output the real-time moving carrier position and posture information, realize auxiliary navigation positioning, return to step S1, and wait to enter the next processing cycle; Otherwise, return to step S4 to re-collect the road image; The specific process of performing road sign information auxiliary navigation correction in S5 includes: S51, obtaining some real-time road sign information according to the road image, and determining whether the current road sign information can be used for navigation and positioning: obtaining some real-time road sign information, the road sign information includes the length and width pixel numbers of the road sign, and then calculating the long side distance estimation value and the short side distance estimation value; comparing the difference between the long side distance estimation value and the short side distance estimation value, if the difference is greater than a first threshold value, it is determined that the current road sign information has a large error and is not suitable for navigation and positioning, and the processing process ends, returning to step S1, and entering the next processing cycle; otherwise, executing step S52; S52, calculating the distance between the moving carrier and the road sign according to the long side distance estimation value and the short side distance estimation value, and judging whether the moving carrier passes the location of the road sign: the distance between the moving carrier and the road sign is the average value of the long side distance estimation value and the short side distance estimation value, and comparing the average value with the second threshold value, if the average value is less than the second threshold value, the moving carrier passes the location of the road sign, and executing step S53; otherwise, it cannot be determined whether the moving carrier passes the location of the road sign, and returning to step S51; S53, road sign information assisted navigation correction: obtain road sign information in the road sign database, and output the motion carrier position information based on the precise positioning calculation of the road sign in the road sign information.

2. According to claim 1, a road sign intelligent recognition assisted fusion navigation positioning method, It is characterized in that The specific process of road sign information processing in S4 includes: S41, acquiring road images: a camera fixed in front of the moving carrier captures road images in the moving carrier's forward direction in real time, and retains the moment of each image acquisition for subsequent calculations; S42, identifying a number of road sign areas: processing the acquired road image to identify an image area containing road signs; if the identified road sign area is a rectangle, it indicates that the road sign is located directly in front of the driving direction, and then executing step S43; otherwise, it indicates that there is no road sign area in the road image, or the road sign is not in front of the driving direction, and then returning to step S1 to enter the next processing cycle; S43, intelligent recognition of road signs: intelligently recognize road signs in the road sign area, including the text, traffic signs and relative positional relationship between the parts of the road signs; if the recognition is successful, execute step S44; otherwise, return to step S1 and enter the next processing cycle; S44, road sign information retrieval: according to the identification information in S43, the corresponding road sign information is retrieved from the road sign database; if the corresponding road sign information is not retrieved, then return to step S1 and enter the next processing cycle; otherwise, execute step S45; S45, output the retrieved road sign information: the road sign information includes the road sign number, the Chinese characters on the road sign, the geographical coordinates of the road sign, and the size of the road sign; output the retrieved road sign information and end the processing, and execute step S5.

3. According to claim 2, a road sign intelligent recognition assisted fusion navigation positioning method, Features: The road sign geographic coordinates include the longitude, latitude and altitude of the road sign.

4. A road sign intelligent recognition assisted fusion navigation positioning method according to claim 1, It is characterized in that The specific process of calculating the long side distance estimation value and the short side distance estimation value in step S51 includes: obtaining a number of real-time road sign information, the road sign information includes the length and width pixel numbers of the road sign, and then calculating the distance between the moving carrier and the road sign: where w 1 、p 1 and d 1 are the actual length of the long side of the road sign, the number of long pixels and the estimated value of the long side distance, respectively. 2 、p 2 and d 2 are the actual length of the short side of the road sign, the number of wide pixels and the estimated distance of the short side, respectively, and f is the focal length of the camera.

5. A road sign intelligent recognition assisted fusion navigation positioning method according to claim 1, It is characterized in that The specific process of combining the GNSS positioning and orientation results with the INS positioning and orientation results in S2 or combining the DR positioning and orientation results with the INS positioning and orientation results in S3 includes: combining the GNSS positioning and orientation results or the DR positioning and orientation results with the INS positioning and orientation results using the Kalman filtering method.

6. A fusion navigation positioning device assisted by intelligent recognition of road signs, used to implement a fusion navigation positioning method assisted by intelligent recognition of road signs as described in any one of claims 1 to 5, It is characterized in that include: A fusion navigation processing module, a road sign information processing module and a road sign information database connected in sequence; The fusion navigation processing module includes a fusion navigation processing computer and an inertial measurement device connected to the fusion navigation processing computer, a GNSS satellite navigation module, an odometer, a camera, a wireless communication module and an antenna, and a power module; The memory of the fusion navigation processing computer stores a road sign database; the road sign information processing module includes a camera connected to the fusion navigation processing computer.

7. A road sign intelligent recognition assisted fusion navigation and positioning device according to claim 6, Features: The inertial measurement device is an accelerometer or a gyroscope.

8. A road sign intelligent recognition assisted fusion navigation and positioning device according to claim 6, Features: It also includes a network server or a cloud computing server, which is used to store a road sign database, and the network server or the cloud computing server is connected to the fusion navigation processing computer through a wireless communication module and an antenna.

Images