Highway tunnel information issuing system based on vehicle-mounted AR-HUD and Beidou navigation positioning

Abstract

The invention relates to the technical field of data processing, in particular to a highway tunnel information issuing system based on vehicle-mounted AR-HUD and Beidou navigation positioning, which comprises the following steps: fusing an IMU positioning result and a UWB positioning result by using a Kalman filtering algorithm to obtain a vehicle positioning result; iMU error features and UBW error features are obtained according to the first variable quantity of the tunnel positioning result and the second variable quantity of the vehicle positioning result; when the IMU error feature is smaller than or equal to the UBW error feature, re-fusing the IMU positioning result and the UWB positioning result according to the difference between the IMU error feature and the UBW error feature to obtain an updated vehicle positioning result; and when the IMU error feature is greater than the UBW error feature, resetting the positioning process of the inertial navigation technology, and issuing information according to the updated vehicle positioning result. According to the invention, the accuracy of vehicle positioning is further ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning. BACKGROUND

[0002] In order to ensure the driving safety of vehicles and deal with emergency situations in the highway tunnel, tunnel information needs to be published to vehicles in the tunnel, such as the location of fire-fighting equipment, the entrance position of emergency escape passages, etc. When vehicles receive these tunnel information, they will display or mark them in the AR scene of the vehicle-mounted AR-HUD. When marking the tunnel information in the AR scene of the vehicle-mounted AR-HUD, it needs to be based on the positioning of the vehicle. When the positioning of the vehicle is inaccurate, it may cause the marking position of the tunnel information in the AR scene of the vehicle-mounted AR-HUD to deviate.

[0003] The conventional positioning of the vehicle is obtained by the Beidou navigation positioning technology. However, the Beidou signal will be weakened or even not received in the tunnel, resulting in a large error in the positioning result obtained by the Beidou navigation positioning technology. The usual way is to fuse the inertial navigation technology and the UWB base station positioning technology by using the Kalman filtering algorithm when the Beidou signal is weakened or lost, and then to realize positioning. However, since the errors in the positioning results obtained by the inertial navigation technology and the UWB base station positioning technology are accumulated with time and change with the environment, and the Kalman filtering algorithm uses Gaussian distributed errors for filtering fusion, the error in the positioning result obtained by the Kalman filtering algorithm cannot be ignored for the errors changing with time and environment, that is, the error in the positioning result obtained by the Kalman filtering algorithm will cause the marking position of the tunnel information in the AR scene of the vehicle-mounted AR-HUD to deviate significantly. SUMMARY

[0004] In order to solve the problem that the errors in the positioning results obtained by the inertial navigation technology and the UWB base station positioning technology have the characteristics of changing with time and environment, and the error in the positioning result obtained by the Kalman filtering algorithm cannot be ignored, the present application provides a highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning.

[0005] The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning of the present application adopts the following technical scheme: An embodiment of the present application provides a highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning, which comprises the following modules: A tunnel positioning module is used to project and transform the vehicle image into the tunnel model to obtain the positions of all vehicles in the tunnel, denoted as tunnel positioning results, and publish the tunnel information containing the tunnel positioning results of all vehicles to all vehicles; The vehicle positioning module is configured to position each vehicle in the tunnel by using inertial navigation technology and UWB base stations respectively, to obtain an IMU positioning result and a UWB positioning result respectively, and to fuse the IMU positioning result and the UWB positioning result by using a Kalman filtering algorithm to obtain a vehicle positioning result. The positioning updating module is configured to, for the same vehicle, obtain a first change amount of the tunnel positioning result and a second change amount of the vehicle positioning result within a preset time, and record a difference between the second change amount and the first change amount as an error change feature. The growth trend and the fluctuation trend of all error change features obtained before the current time are recorded as an IMU error feature and a UBW error feature respectively; when the IMU error feature is less than or equal to the UBW error feature, the IMU positioning result and the UWB positioning result are re-fused according to the difference between the IMU error feature and the UBW error feature to obtain an updated vehicle positioning result at the current time; and when the IMU error feature is greater than the UBW error feature, the positioning process of the inertial navigation technology is reset. The information publishing module is configured to visualize the tunnel information on the AR-HUD according to the updated vehicle positioning result at the current time.

[0006] Preferably, the vehicle image is projected and transformed into the tunnel model to obtain the positions of all vehicles in the tunnel, recorded as tunnel positioning results, and the specific steps include the following: Each camera in the tunnel collects a vehicle image, and after affine transformation and image fusion of all vehicle images collected by all cameras, a panoramic view of all vehicles in the tunnel is obtained; the panoramic view is pasted on the road in the tunnel model; the pixel point position of each vehicle in the panoramic view is identified, and the positioning result of the pixel point position in the tunnel model is recorded as the tunnel positioning result of each vehicle.

[0007] Preferably, the specific steps of obtaining the first change amount of the tunnel positioning result and the second change amount of the vehicle positioning result within the preset time include the following: For the i-th time when the Beidou signal strength is less than the preset strength, the tunnel positioning result of each vehicle at the i-th time is recorded as The tunnel positioning result at the i-p-th time is recorded as The distance between and is recorded as the first change amount; the vehicle positioning result of each vehicle at the i-th time is recorded as The vehicle positioning result at the i-p-th time is recorded as The distance between and is recorded as the second change amount; and p is the preset time.

[0008] Preferably, the growth trend and fluctuation trend of all error change characteristics obtained before the current moment are recorded as IMU error characteristics and UBW error characteristics respectively, and the specific steps include the following: Mark the moment when the Beidou signal strength is less than the preset strength as the initial moment; for the current moment after the Beidou signal strength is less than the preset strength, the time sequence composed of error change characteristics obtained before the current moment and after the initial moment is recorded as the error change characteristic sequence; filter and normalize the error change characteristic sequence, and use the STL algorithm to obtain the trend component of the normalized error change characteristic sequence; the difference between the normalized error change characteristic sequence and the trend component is recorded as the fluctuation component; The mean value of the current moment and the mean value of several element values before the current moment in the trend component are recorded as the IMU error characteristics; the mean value of the absolute values of all element values in the fluctuation component is recorded as the UBW error characteristics.

[0009] Preferably, according to the difference between the IMU error characteristics and the UBW error characteristics, the IMU positioning result and the UWB positioning result are re-fused to obtain the updated vehicle positioning result at the current moment, and the specific steps include the following: Record the IMU error characteristics as a1, record the UBW error characteristics as a2, and record a1 / (a2+0.1) as the IMU error correction strength; Use the IMU error correction strength to update the fusion weight of the Kalman filter algorithm when fusing the IMU positioning result and the UWB positioning result, and the updated fusion weight is negatively correlated with the IMU error correction strength; Re-fuse the IMU positioning result and the UWB positioning result using the updated fusion weight to obtain the updated vehicle positioning result at the current moment.

[0010] Preferably, the tunnel information is visualized onto the vehicle-mounted AR-HUD according to the updated vehicle positioning result at the current moment, and the specific steps include the following: The tunnel information includes a plurality of reference positions, and the reference positions include fire-fighting equipment positions, emergency escape passage entrance positions, fire occurrence positions, and roadblock positions; Record the difference between the reference position and the vehicle positioning result as the marked position, and display the warning icon at the marked position in the AR scene of the vehicle-mounted AR-HUD.

[0011] Preferably, the IMU error correction strength is used to update the fusion weight of the Kalman filter algorithm when fusing the IMU positioning result and the UWB positioning result, and the specific formula includes the following: ; Wherein represents the updated fusion weight, a represents the IMU error correction strength, and x represents the updated fusion weight.

[0012] Preferably, the updated vehicle positioning result , wherein represents the updated fusion weight, P1 represents the UWB positioning result, and P2 represents the IMU positioning result.

[0013] Preferably, the updated fusion weight ; wherein P1 represents the UWB positioning result, P2 represents the IMU positioning result, and PO represents the vehicle positioning result obtained by the Kalman filtering algorithm fusion.

[0014] Preferably, the time point of resetting the positioning process of the inertial navigation technology is re-marked as the initial time point.

[0015] The technical scheme of the present application has the following beneficial effects: The present application uses the inertial navigation technology and the UWB base station to respectively position each vehicle in the tunnel, respectively obtains the IMU positioning result and the UWB positioning result, and uses the Kalman filtering algorithm to fuse the IMU positioning result and the UWB positioning result to obtain the vehicle positioning result. The vehicle positioning result obtained in this process has smaller error compared with the IMU positioning result and the UWB positioning result, which ensures the accuracy of vehicle positioning.

[0016] Further, the present application obtains the first change amount of the tunnel positioning result and the second change amount of the vehicle positioning result within a preset time, obtains the IMU error feature and the UBW error feature, when the IMU error feature is less than or equal to the UBW error feature, re-fuses the IMU positioning result and the UWB positioning result according to the difference between the IMU error feature and the UBW error feature to obtain the updated vehicle positioning result at the current time, and resets the positioning process of the inertial navigation technology when the IMU error feature is greater than the UBW error feature.

[0017] This process, on the one hand, resets the integral process of the inertial navigation technology based on the IMU error feature and the UBW error feature, so that the integral process resetting time is appropriate, that is, it avoids the problem that the error of the UWB positioning result is relatively large when resetting too early, resulting in the error of the vehicle positioning result becoming large, and also avoids the problem that the error of the IMU positioning result is accumulated for a long time when resetting too late, resulting in the error of the subsequent vehicle positioning result becoming large; on the other hand, the fusion process of the Kalman filtering algorithm (i.e. the vehicle positioning result) is updated, so that the updated vehicle positioning result further considers the interference of other errors changing with time and environment in addition to the Gaussian distribution error on the basis of the Kalman filtering algorithm fusion.

[0018] In summary, the application resets the inertial navigation technology or updates the vehicle positioning result according to the error distribution of the IMU positioning result and the UWB positioning result with time or environmental changes, further ensures the accuracy of the vehicle positioning at the current time, and helps to avoid the position deviation of the icon on the AR-HUD. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 The framework structure diagram of the highway tunnel information publishing system based on vehicle AR-HUD and Beidou navigation positioning provided by an embodiment of the present application is shown in Figure 2 The step flowchart of all modules in the highway tunnel information publishing system based on vehicle AR-HUD and Beidou navigation positioning provided by an embodiment of the present application is shown in DETAILED DESCRIPTION

[0021] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined purpose, the highway tunnel information publishing system based on vehicle AR-HUD and Beidou navigation positioning according to the present application, its specific implementation, structure, features and effects, are described in detail as follows in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0023] The specific scheme of the highway tunnel information publishing system based on vehicle AR-HUD and Beidou navigation positioning provided by the present application is described in detail below in combination with the drawings.

[0024] Embodiment one:

[0025] Please refer to Figure 1 which shows the highway tunnel information publishing system based on vehicle AR-HUD and Beidou navigation positioning provided by the embodiment one of the present application, which includes the following modules: tunnel positioning module, vehicle positioning module, positioning update module, information publishing module. Figure 2The steps contained in all modules are shown.

[0026] A tunnel positioning module is used to collect vehicle driving information in the tunnel and project the information on a three-dimensional tunnel model in real time.

[0027] A plurality of industrial cameras (hereinafter referred to as cameras for short) are installed in the tunnel, each camera has a downward-looking perspective, the fields of view of all cameras cover the entire road in the tunnel, each camera simultaneously collects video images at the same frame rate (10 frames per second in this embodiment), each frame of image is a color (RGB) image (the image size in this embodiment is 512*512), and each frame of image collected by all cameras contains driving information of all vehicles in the tunnel (for example, the position of the vehicle, the license plate number of the vehicle, and the like).

[0028] Wireless broadcasting technology is used in the tunnel to broadcast and publish vehicle driving information and tunnel-related information in the tunnel, and when a vehicle receives the broadcast, it can learn the traffic operation in the tunnel.

[0029] Specifically, the module is used to implement the following steps: Step S101, vehicle images are collected by the cameras installed in the tunnel and projected and transformed into the tunnel model to obtain the positions of all vehicles in the tunnel, denoted as tunnel positioning results, and tunnel information containing the tunnel positioning results of all vehicles is published to all vehicles.

[0030] In the vehicle positioning module, each vehicle is positioned by using Beidou navigation technology, inertial navigation technology, and UWB base station.

[0031] Specifically, this embodiment takes into account that the Beidou satellite signal in the tunnel is weak or even cannot be received, so it is necessary to fuse other positioning technologies, for example, fuse the positioning results of the inertial navigation technology and the positioning results of the UWB base station positioning technology.

[0032] The inertial navigation technology fuses the vehicle-mounted inertial measurement unit (IMU) to perform positioning, the IMU integrates a 3-axis accelerometer and a 3-axis gyroscope, which are respectively used to measure the three-axis linear acceleration of the vehicle and the three-axis angular velocity of the vehicle. The inertial navigation technology is based on Newton's law of mechanics, and the vehicle positioning is realized by integrating the acceleration and angular velocity data.

[0033] The inertial navigation technology is known, and the specific principle thereof will not be described herein.

[0034] The UWB (Ultra-Wideband) base station positioning technology uses the time of flight of signals between a plurality of positioning base stations deployed in the tunnel and the vehicle to perform positioning, and in this embodiment, a UWB base station is installed every 800 meters. This technology is also known, and will not be described in detail herein.

[0035] The module specifically implements the following steps: In step S102, the inertial navigation technology and the UWB base station are used to respectively position each vehicle in the tunnel, and IMU positioning results and UWB positioning results are obtained. The Kalman filtering algorithm is used to fuse the IMU positioning results and the UWB positioning results to obtain vehicle positioning results.

[0036] The positioning updating module is used to update the vehicle positioning results obtained above.

[0037] Specifically, on the one hand, the core problem of the inertial navigation technology is that the position and the speed are obtained by twice integration of the acceleration, and the attitude is obtained by integration of the angular velocity. Any small error will be amplified in the integration process, and eventually lead to exponential growth of the error with time. On the other hand, when the UWB signal encounters obstacles (such as tunnel walls and vehicles), it will be reflected and refracted, forming multiple propagation paths, resulting in detection of multiple delay signals, which interfere with the time delay measurement of the direct signal wave. Especially when there are obstacles (such as other vehicles) between the base station and the vehicle, the signal needs to detour or penetrate the obstacles, and the propagation time is prolonged. That is, the error of the UWB positioning result is affected by the environment.

[0038] In the above module, when the Kalman filtering algorithm is used to fuse the IMU positioning results and the UWB positioning results, it is considered that the Kalman filtering algorithm is a fast and efficient known filtering method based on Gaussian noise error. It can fuse multiple source data (i.e. IMU positioning results and UWB positioning results), so that the error of the obtained vehicle positioning results is reduced. However, in this embodiment, the error of the IMU positioning result grows exponentially with time, and the error of the UWB positioning result is affected by the environment (i.e. the IMU positioning result and the UWB positioning result not only have Gaussian distributed errors, but also have errors that change with time and environment). The error of the vehicle positioning result obtained by the Kalman filtering algorithm cannot be effectively suppressed.

[0039] In this module, the error of the vehicle positioning result is further suppressed (i.e. the vehicle positioning result obtained by the Kalman filtering algorithm is updated) through the following two steps: In step S103, for the same vehicle, after a preset time, a first change amount of the tunnel positioning result and a second change amount of the vehicle positioning result in the preset time are obtained. The difference between the second change amount and the first change amount is recorded as an error change feature. The growth trend and the fluctuation trend of all error change features obtained before the current time are recorded as IMU error features and UBW error features, respectively.

[0040] Step S104, when the IMU error feature is less than or equal to the UBW error feature, according to the difference between the IMU error feature and the UBW error feature, the IMU positioning result and the UWB positioning result are re-fused to obtain the updated vehicle positioning result at the current time; when the IMU error feature is greater than the UBW error feature, the positioning process of the inertial navigation technology is reset.

[0041] An information publishing module is configured to implement step S105, and visualize the tunnel information on the AR-HUD according to the vehicle positioning result at the current time.

[0042] The vehicle positioning result in the module has smaller error, and according to the vehicle positioning result, the vehicle driving information in the tunnel and the tunnel related information can be accurately displayed on the AR-HUD, for example, some warning or prompt information is marked on the AR-HUD, so as to avoid the error of the marking position of the warning or prompt information.

[0043] Embodiment two:

[0044] Step S101, a camera installed in the tunnel is used to collect vehicle images and project and transform the vehicle images into a tunnel model to obtain the positions of all vehicles in the tunnel, denoted as tunnel positioning results, and tunnel information including the tunnel positioning results of all vehicles is published to all vehicles, and the specific implementation method includes: A tunnel three-dimensional model (referred to as a tunnel model) is constructed or placed in a three-dimensional modeling or simulation software (such as Maya, CAD modeling software or UE5 game engine), and the three-dimensional model in the embodiment one-to-one restores the tunnel (or the three-dimensional model one-to-one restores the construction drawing of the tunnel). The three-dimensional model includes the road, fire-fighting equipment, emergency escape passage entrance and the like in the tunnel.

[0045] In addition, a plurality of monitoring points are manually selected outside the tunnel entrance, and the positioning results of the monitoring points are obtained by using Beidou positioning technology; and the positioning results are marked in the three-dimensional model, and then the positioning results of any position in the three-dimensional model (such as the position of any grid point in the three-dimensional model) are obtained by using a linear interpolation algorithm.

[0046] When the tunnel is put into use, for each frame of image (denoted as vehicle image) collected by each camera in the tunnel, affine transformation is performed on each frame of vehicle image by using the calibrated homography matrix, and the purpose is to transform the image collected by the camera perspective into the overhead perspective (that is, the overhead perspective image obtained by affine transformation). Transforming the vehicle image collected by each camera into an overhead perspective image, using Gaussian pyramid fusion technology to splice and fuse the overhead perspective images of all cameras to obtain an overhead panoramic view of all vehicles in the tunnel. The panoramic view is pasted on the road in the three-dimensional model, and each pixel point on the panoramic view in the three-dimensional model corresponds to a positioning result.

[0047] Among them, the affine transformation and Gaussian pyramid fusion technology are all known image processing technologies, and the embodiment will not be described in detail.

[0048] In this embodiment, YOLOV5 is used to identify the rectangular bounding box of each vehicle in the panoramic view, and the positioning result corresponding to the center pixel point of each bounding box represents the position of the vehicle in the tunnel, denoted as the tunnel positioning result of each vehicle.

[0049] It should be noted that when YOLOV5 is used to identify the rectangular bounding box of each vehicle in the panoramic view, the panoramic view is equally divided into a plurality of image blocks, and YOLOV5 is used to identify the rectangular bounding box of each vehicle in each image block, and then the bounding box of all vehicles is obtained; in this embodiment, the size of each image block is 512x512, and if the image block is not enough after equal division, it is filled into 512x512 by filling 0.

[0050] At this point, this step uses all cameras in the tunnel to collect image data of vehicles in the tunnel, and projects and transforms it into a three-dimensional model in real time, so that the three-dimensional model can display the positions of all vehicles (i.e., the tunnel positioning result of each vehicle) in real time.

[0051] Further, the tunnel information including the tunnel positioning result of all vehicles is broadcasted.

[0052] Among them, the tunnel information includes the tunnel positioning result of all vehicles and the license plate number, and also includes the position of the fire-fighting equipment in the tunnel, the position of the emergency escape entrance, etc.

[0053] In step S102, each vehicle in the tunnel is positioned by using inertial navigation technology and UWB base station respectively, and IMU positioning result and UWB positioning result are obtained respectively, and Kalman filtering algorithm is used to fuse the IMU positioning result and the UWB positioning result to obtain the vehicle positioning result, and the specific implementation method includes: When the vehicle detects that the Beidou signal strength (specifically represented by the carrier-to-noise ratio) is less than the preset strength after the vehicle enters the tunnel, the Beidou navigation technology is no longer used for positioning, and the inertial navigation technology and the UWB base station are used for positioning. When the vehicle detects that the Beidou signal strength is greater than or equal to the preset strength, the Beidou navigation technology is continued to be used for positioning. The preset strength in the embodiment is 42 dBHz, and in other embodiments, it can be set to other values, which are not limited in the embodiment.

[0054] The moment when the Beidou signal strength is less than the preset strength is recorded as the positioning switching moment, and the specific method of using the inertial navigation technology and the UWB base station for positioning after the positioning switching moment is as follows: The positioning result obtained by using the inertial navigation technology for positioning the vehicle is recorded as the IMU positioning result, and the positioning result obtained by using the UWB base station for positioning the vehicle is recorded as the UWB positioning result. Both the IMU positioning result and the UWB positioning result can represent the position of the vehicle in the tunnel, but the two positioning results alone have a large error.

[0055] In the embodiment, the Kalman filtering algorithm is used to fuse the IMU positioning result and the UWB positioning result to obtain the vehicle positioning result, which has a smaller error than the IMU positioning result or the UWB positioning result. The IMU positioning result is used as the state quantity of the Kalman filtering algorithm, and the UWB positioning result is used as the observation value of the Kalman filtering algorithm. The specific process of the Kalman filtering algorithm is known, and the embodiment will not be described in detail.

[0056] In step S103, every preset time, the first change of the tunnel positioning result and the second change of the vehicle positioning result within the preset time are obtained, and the difference between the second change and the first change is recorded as the error change feature. The growth trend and the fluctuation trend of all error change features obtained before the current moment are recorded as the IMU error feature and the UWB error feature, respectively, and the specific implementation scheme includes: It should be noted that after the positioning switching moment, although the Kalman filtering algorithm is used to fuse the IMU positioning result and the UWB positioning result to obtain the vehicle positioning result, the positioning error is reduced. However, the IMU positioning result and the UWB positioning result not only have Gaussian distribution errors, but also have other types of errors, such as the error of the IMU positioning result showing an exponential growth trend with time, and the error of the UWB positioning result being affected by the tunnel environment (such as the curve of the tunnel, the tunnel wall, and the vehicle in the tunnel). However, all errors are treated as Gaussian noise errors in the Kalman filtering algorithm, which leads to the error of the obtained vehicle positioning result cannot be further suppressed, that is, the error of the vehicle positioning result can be further optimized in the embodiment.

[0057] It is further needed to be explained that any one vehicle in the tunnel is recorded as a target vehicle, for the target vehicle, after the target vehicle receives the tunnel information broadcast by the tunnel, the target vehicle first obtains the tunnel positioning results of all vehicles contained in the tunnel information, and in the tunnel positioning results of all vehicles, the tunnel positioning result corresponding to the target vehicle is obtained by matching the license plate number of the target vehicle.

[0058] For the tunnel positioning result of the target vehicle, the error comes from the measurement error of the positioning result of the monitoring point in step S101 and the error existing in the affine transformation process (such as Gaussian noise when the camera images, error of the homography matrix calibrated, etc.), which presents the characteristics of Gaussian distribution, which is independent of the environment and time; this embodiment uses the tunnel positioning result of the target vehicle to suppress the error of the IMU positioning result and the UWB positioning result which changes with time and environment.

[0059] Specifically, the target vehicle is every 0.5 seconds as a time after the positioning switching time, the positioning switching time is regarded as the 0th time, and is marked as the initial time, and the tunnel positioning result of the ith time after the initial time is recorded as , and the tunnel positioning result of the i-pth time is recorded as The distance between and is represented as , which represents the first change amount of the tunnel positioning result in the time period p, and this embodiment takes p=4 as an example for description, and other embodiments can set p to other integer values greater than 0, and preferably p is less than or equal to 6.

[0060] The vehicle positioning result of the ith time is recorded as , and the vehicle positioning result of the i-pth time is recorded as The distance between and is represented as , which represents the second change amount of the vehicle positioning result in the time period p.

[0061] Specifically, when i-p is less than 0, the i-pth time represents a time before the positioning switching time, and the vehicle positioning result at this time is the positioning result obtained by using the Beidou navigation technology.

[0062] The absolute value of the difference between the second change amount and the first change amount is recorded as the error change feature of the ith time.

[0063] The first change amount in the preset time p represents the movement (i.e., displacement) of the target vehicle in the preset time p. The errors of the tunnel positioning results at different times have the same distribution (i.e., the errors of the tunnel positioning results at different times have the same Gaussian distribution, which can offset each other to a certain extent), so the first change amount at different times can relatively accurately represent the movement of the target vehicle in the preset time p.

[0064] The second change amount in the preset time p also represents the movement of the target vehicle in the preset time p. However, the error distribution of the vehicle positioning results at different times is different (because the IMU positioning results and the UWB positioning results contain errors that change with time and environment), so the second change amount at different times has certain differences when describing the movement of the target vehicle in the preset time p.

[0065] The error change feature represents the difference between the movement of the target vehicle described by the second change amount in the preset time p and the movement of the target vehicle described by the first change amount in the preset time p, which can reflect the error change of the vehicle positioning results.

[0066] Further, the time sequence sequence formed by the error change features of all times before the current time and after the initial time (including the current time and the initial time) is denoted as an error change feature sequence. The growth trend existing in the sequence can reflect the error change of the IMU, and the fluctuation trend existing in the sequence can reflect the error change of the UBW. In this embodiment, the growth trend and the fluctuation trend of the error change feature sequence are denoted as an IMU error feature and a UBW error feature, respectively.

[0067] As an example, the method for obtaining the IMU error feature and the UBW error feature is as follows: First, a Gaussian filter with a length of 3 is used to filter the error change feature sequence, which aims to further eliminate the interference of the errors of the first change amount that have not been offset when the first change amount changes over time.

[0068] The filtered error change feature sequence is linearly normalized, and the trend component (a time sequence sequence with the same length as the error change feature sequence) of the normalized error change feature sequence is obtained by using the STL algorithm (time sequence decomposition algorithm), which represents the growth trend contained in the error change feature sequence.

[0069] The difference between the normalized error change feature sequence and the trend component (i.e., the difference between the elements at the same time in the time sequence sequences represented by the two) is obtained, and a fluctuation component (also a time sequence sequence with the same length as the error change feature sequence) is obtained.

[0070] The mean value of the absolute values of all element values in the fluctuation component is recorded as the UBW error feature.

[0071] The mean value of the absolute values of all element values in the fluctuation component is recorded as the UBW error feature.

[0072] Step S104, when the IMU error feature is less than or equal to the UBW error feature, the IMU positioning result and the UWB positioning result are re-fused according to the difference between the IMU error feature and the UBW error feature to obtain the updated vehicle positioning result at the current time; when the IMU error feature is greater than the UBW error feature, the positioning process of the inertial navigation technology is reset, including the specific implementation method: When the IMU error feature is greater than the UBW error feature, it means that the error interference of the IMU positioning result is relatively large compared to the error interference of the UWB positioning result, that is, the vehicle positioning result obtained by the target vehicle at the current time will be obviously interfered by the error interference of the IMU positioning result, at this time, the positioning process of the inertial navigation technology is reset in the embodiment.

[0073] The reset of the positioning process of the inertial navigation technology means that the three-axis linear acceleration of the vehicle and the three-axis angular velocity of the vehicle measured by the IMU are re-used to integrate the acceleration and the angular velocity from the current time to obtain the IMU positioning result; the specific integration process is a known content in the inertial navigation technology, and the embodiment will not be described here.

[0074] It should be noted that the vehicle positioning result at the current time is still the vehicle positioning result obtained in step S102, and the IMU positioning result is obtained using the reset inertial navigation technology after the time after the current time, and the vehicle positioning result after the time is obtained based on the IMU positioning result using step S102. At the same time, the current time is marked as the initial time, and then steps S103 and S104 are used to determine whether the IMU error feature is greater than the UBW error feature at the subsequent time, if so, the positioning process of the inertial navigation technology is continued to be reset.

[0075] The above process avoids the situation that the error of the IMU positioning result is accumulated for a long time, resulting in the error of the subsequent vehicle positioning result becoming large; further, the integration process of the inertial navigation technology is reset based on the UBW error feature, so that the reset time of the integration process is appropriate, that is, the problem that the error of the vehicle positioning result becomes large due to the relatively large error of the UWB positioning result when the reset is too early, and the problem that the error of the subsequent vehicle positioning result becomes large due to the long-time accumulation of the error of the IMU positioning result when the reset is too late.

[0076] It should be noted that in step S103, it is recorded that when there are less than 5 element values before the current time, the subsequent step S104 is not executed; one of the purposes is to avoid frequent resetting of the inertial navigation technology in step S104, for example, when the error of the UWB positioning result is always small or does not exist, the inertial navigation technology can be avoided from being frequently reset to cause the inertial navigation technology to be invalid.

[0077] When the IMU error feature is less than or equal to the UWB error feature, the IMU error feature is recorded as a1, the UWB error feature is recorded as a2, and a1 / (a2+0.1) is recorded as the IMU error correction strength a; the purpose of adding 0.1 to the denominator is to avoid the denominator being 0.

[0078] The IMU error correction strength represents the relative size of the cumulative error of the IMU positioning result changing over time, or in other words, the difference between the IMU error feature and the UWB error feature. The smaller the value, the smaller the IMU error feature relative to the UWB error feature (the interference of the cumulative error of the IMU positioning result changing over time is relatively small).

[0079] As known from the Kalman filtering algorithm, when fusing the IMU positioning result and the UWB positioning result, it is based on the error size (or error variance) of the IMU positioning result and the error size of the UWB positioning result, wherein the larger the error size (such as the variance of the error of the IMU positioning result), the smaller the weight when fusing, and the smaller the error size (such as the variance of the error of the IMU positioning result), the larger the weight when fusing. The weight when fusing is essentially represented by the gain coefficient of the Kalman filtering.

[0080] In addition, all errors (such as the error of the IMU positioning result) in the Kalman filtering algorithm are considered as Gaussian distributed errors, without considering other errors including errors changing over time and environment; the IMU error correction strength is used to re-fuse on the basis of the fusion process of the Kalman filtering algorithm in this embodiment, to update the fusion result (i.e. the vehicle positioning result) of the Kalman filtering algorithm, so that it can further consider other error conditions (such as errors changing over time and environment).

[0081] Specifically, the fusion process of the Kalman filtering algorithm is represented as: ; Wherein, P1 represents the UWB positioning result, P2 represents the IMU positioning result, PO represents the vehicle positioning result obtained by fusion, and x represents the weight when fusing (referred to as fusion weight); after transforming the above formula, we get: .

[0082] In particular, when P2 is equal to P1, it means that the UWB positioning result and the IMU positioning result are the same, at this time x can take any value in the interval [0, 1], and the embodiment sets x = 0.5, and other embodiments can also not execute the subsequent steps but directly execute step S105.

[0083] Further, the fusion weight is updated and corrected by using the IMU error correction strength, and the following is obtained: ; Wherein represents the updated fusion weight, and a represents the IMU error correction strength. The purpose of the formula is to update the fusion weight of the fusion process on the basis of the Kalman filtering algorithm.

[0084] At this time, the updated vehicle positioning result .

[0085] The updated vehicle positioning result Compared with the vehicle positioning result P before updating, the former further considers the interference of other errors changing with time and environment in addition to the Gaussian distribution error on the basis of the Kalman filtering algorithm fusion. Specifically, the greater the IMU error correction strength (at this time is smaller), the greater the error interference of the IMU positioning result changing with time, and at this time the updated vehicle positioning result does not pay attention to the IMU positioning result, but pays more attention to the UWB positioning result. The smaller the IMU error correction strength (at this time is greater), the greater the error interference of the IMU positioning result changing with time, and at this time the updated vehicle positioning result pays more attention to the IMU positioning result.

[0086] In particular, when is less than or equal to 0.1, let ; when is greater than or equal to 0.9, let ; the purpose is to avoid being too large or too small to cause the IMU positioning result or the UWB positioning result not to participate in the fusion.

[0087] So far, in the current time, when the IMU error feature is less than or equal to the UWB error feature, the vehicle positioning result is updated to suppress the interference of errors changing with time or environment.

[0088] In summary, the step resets the inertial navigation technology or updates the vehicle positioning result according to the error distribution of the IMU positioning result and the UWB positioning result, which changes with time or environment, at the current time, to further ensure the accuracy of the vehicle positioning at the current time.

[0089] In step S105, the tunnel information is visualized on the AR-HUD according to the vehicle positioning result at the current time, and the specific implementation method includes: In addition to the tunnel positioning result of all vehicles, the tunnel information also includes the positions of fire-fighting equipment and emergency escape passage entrances, which are referred to as reference positions. The reference positions represent the positioning results in the three-dimensional tunnel model.

[0090] For the vehicle positioning result at the current time obtained above (if the vehicle positioning result is updated, it refers to the updated vehicle positioning result). The difference between the reference position and the vehicle positioning result is obtained, which is referred to as the marked position, representing the relative position of the reference position relative to the vehicle body.

[0091] The AR-HUD is a display device based on AR technology, which can project a virtual AR scene onto the windshield. The display positions of various icons (or UI elements) with warning functions in the AR scene are all relative positions relative to the vehicle positioning result. In the AR scene, some preset icons are displayed at each marked position (for example, a fire icon is displayed at the marked position corresponding to the fire-fighting equipment, and an emergency escape passage icon is displayed at the marked position corresponding to the emergency escape passage entrance). The process of displaying icons (or UI elements) in the AR scene according to the vehicle positioning result is a known technology, and this embodiment will not be described in detail.

[0092] Embodiment Three:

[0093] In step S101, the method for obtaining the calibrated homography matrix is: After each camera in the tunnel is fixed, a checkerboard is placed on the tunnel road in the field of view of the camera. An image sample is collected by the camera, and the image of the checkerboard taken from the overhead perspective is referred to as the checkerboard sample. The SIFT corner point algorithm is used to detect the corner points in the image sample and the checkerboard sample, and the corner point matching algorithm (such as the normalized cross-correlation matching algorithm) is used to match the corner points in the image sample with the corner points in the checkerboard sample to obtain all the matched corner point pairs. The RANSAC algorithm is used to obtain the homography matrix based on the matched corner point pairs, and the homography matrix can perform affine transformation on the image sample to the checkerboard sample, so that the image sample becomes an overhead perspective. The homography matrix is referred to as the calibrated homography matrix.

[0094] The methods used in the above process are all known image processing techniques, and thus will not be described in detail.

[0095] In step S102, when the Beidou signal strength detected by the vehicle is greater than or equal to the preset strength (i.e., before the positioning switching time), in this embodiment, the positioning result of the Beidou navigation technology (abbreviated as Beidou positioning result) is obtained at the same time, as well as the IMU positioning result and the UWB positioning result.

[0096] The Beidou positioning result, the IMU positioning result and the UWB positioning result are fused by using the Kalman filtering algorithm to obtain the vehicle positioning result, wherein the IMU positioning result and the UWB positioning result are used as the state quantity of the Kalman filtering algorithm, and the Beidou positioning result is used as the observation value of the Kalman filtering algorithm.

[0097] In addition, before the positioning switching time, steps S103 and S104 in embodiment one are no longer implemented, and step S105 is directly executed.

[0098] The positioning result obtained by using the Beidou positioning technology in step S101 and the positioning result obtained by positioning the vehicle by the UWB base station in step S102 are both the positioning result determined by the longitude, latitude and height (altitude) three dimensions, which is the spherical coordinate in the geocentric coordinate system. In this embodiment, the Mercator projection method is used to map the spherical coordinate into the plane coordinate (i.e., the coordinate in the vehicle navigation map). Therefore, the tunnel positioning result, the IMU positioning result and the UWB positioning result in embodiment two are all unified into the plane coordinate. In addition, the distance described in step S103 is the Euclidean distance.

[0099] It should be noted that in this embodiment, the height difference of the road can be ignored when driving in the tunnel, and therefore the height is not considered. In other embodiments, the height can also be used as the third dimension of the vertical plane coordinate.

[0100] In step S105, the tunnel information further includes: fire occurrence position, roadblock position, etc.

[0101] The method for obtaining the fire occurrence position and the roadblock position is as follows: The center point of the bounding box of the fire area and the roadblock area in the panoramic image is detected by using the YOLOV5 neural network, and the center point is used as the fire occurrence position and the roadblock position.

[0102] The fire occurrence position and the roadblock position are displayed on the vehicle AR-HUD, so as to remind the vehicle to drive in the correct lane.

[0103] Other improved embodiments based on the present application also include: replacing the Kalman filtering algorithm with an extended Kalman filtering algorithm or an ion filtering algorithm.

[0104] It should be noted that although the ion filtering algorithm is free of Gaussian constraints, the limited number of particles will limit the algorithm's suppression of errors that change over time or environment (too many particles will greatly reduce the calculation speed). The above embodiments can still further improve the accuracy of the vehicle positioning result when using the ion filtering algorithm.

[0105] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning, characterized in that, The system includes the following modules: The tunnel positioning module is used to project and transform vehicle images into a tunnel model to obtain the positions of all vehicles in the tunnel, which are recorded as tunnel positioning results. The tunnel information, including the tunnel positioning results of all vehicles, is then published to all vehicles. The vehicle positioning module is used to locate each vehicle in the tunnel using inertial navigation technology and UWB base stations, obtaining IMU positioning results and UWB positioning results respectively. The Kalman filter algorithm is used to fuse the IMU positioning results and UWB positioning results to obtain the vehicle positioning result. The positioning update module is used to obtain the first change in the tunnel positioning result and the second change in the vehicle positioning result within the preset time period for the same vehicle, and the difference between the second change and the first change is recorded as the error change feature. The growth trend and fluctuation trend of all error change characteristics obtained before the current moment are denoted as IMU error characteristics and UBW error characteristics, respectively. When the IMU error feature is less than or equal to the UBW error feature, the IMU positioning result and the UWB positioning result are re-fused based on the difference between the IMU error feature and the UBW error feature to obtain the updated vehicle positioning result at the current time. When the IMU error characteristics are greater than the UBW error characteristics, the positioning process of the inertial navigation technology is reset; The information publishing module is used to visualize the tunnel information on the vehicle-mounted AR-HUD based on the updated vehicle positioning results at the current moment.

2. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 1, characterized in that, The specific steps involved in projecting and transforming the vehicle images onto the tunnel model to obtain the positions of all vehicles in the tunnel, denoted as the tunnel positioning result, are as follows: Each camera inside the tunnel captures vehicle images. After affine transformation and image fusion, all vehicle images captured by the cameras are used to obtain a panoramic view of all vehicles inside the tunnel. The panoramic view is then overlaid on the roads in the tunnel model. Identify the pixel position of each vehicle in the panoramic image, and record the positioning result of the pixel position in the tunnel model as the tunnel positioning result of each vehicle.

3. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 1, characterized in that, The specific steps for obtaining the first change in tunnel positioning results and the second change in vehicle positioning results within a preset time period are as follows: For the i-th time after the BeiDou signal strength falls below the preset strength, the tunnel positioning result for each vehicle at the i-th time is denoted as... The tunnel positioning result at time ip is denoted as ,Will and The distance is denoted as the first change; the vehicle positioning result of each vehicle at time i is denoted as... The vehicle location result at time ip is denoted as ,Will and The distance is denoted as the second change; p is the preset time.

4. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 1, characterized in that, The growth trend and fluctuation trend of all error change characteristics obtained before the current time are denoted as IMU error characteristics and UBW error characteristics, respectively, and the specific steps are as follows: The moment when the BeiDou signal strength is less than the preset strength is marked as the initial moment; for the current moment after the BeiDou signal strength is less than the preset strength, the time series composed of the error change characteristics obtained before the current moment and after the initial moment is denoted as the error change characteristic sequence; the error change characteristic sequence is filtered and normalized, and the trend component of the normalized error change characteristic sequence is obtained using the STL algorithm; the difference between the normalized error change characteristic sequence and the trend component is denoted as the fluctuation component; The mean of the current element value and several element values ​​before the current time in the trend component is denoted as the IMU error characteristic; the mean of the absolute values ​​of all element values ​​in the fluctuation component is denoted as the UBW error characteristic.

5. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 1, characterized in that, The specific steps involved in re-fusing the IMU positioning results and UWB positioning results based on the differences between the IMU error characteristics and the UBWB error characteristics to obtain the updated vehicle positioning results at the current moment are as follows: The IMU error characteristic is denoted as a1, the UBW error characteristic is denoted as a2, and a1 / (a2+0.1) is denoted as the IMU error correction strength; The fusion weights of the Kalman filter algorithm when fusing IMU positioning results and UWB positioning results are updated using the IMU error correction strength. The updated fusion weights are negatively correlated with the IMU error correction strength. The IMU positioning results and UWB positioning results are re-fused using the updated fusion weights to obtain the updated vehicle positioning results at the current time.

6. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 1, characterized in that, The specific steps for visualizing the tunnel information onto the in-vehicle AR-HUD based on the updated vehicle positioning results at the current moment are as follows: The tunnel information includes several reference locations, including the location of fire-fighting equipment, the location of emergency escape route entrances, the location of the fire, and the location of roadblocks; The difference between the reference position and the vehicle positioning result is recorded as the marked position, and a warning icon is displayed at the marked position in the AR scene of the vehicle AR-HUD.

7. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 5, characterized in that, The specific formula for updating the fusion weights of the Kalman filter algorithm when fusing IMU positioning results and UWB positioning results by utilizing the IMU error correction strength is as follows: ； in This represents the updated fusion weights, where 'a' represents the IMU error correction strength, and 'x' represents the fusion weights before the update.

8. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 5, characterized in that, The updated vehicle location results ,in The updated fusion weights are represented by P1, P2, and P1 represents the UWB localization result.

9. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 7, characterized in that, The fusion weights before the update Where P1 represents the UWB positioning result, P2 represents the IMU positioning result, and PO represents the vehicle positioning result obtained by the Kalman filter algorithm fusion.

10. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation and positioning as described in claim 4, characterized in that, The moment of the positioning process of the reset inertial navigation technology is remarked as the initial moment.

Images