Highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning

Abstract

The present application relates to the technical field of data processing, in particular to a highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning, comprising: fusing IMU positioning results and UWB positioning results by using Kalman filtering algorithm to obtain vehicle positioning results; obtaining IMU error characteristics and UBW error characteristics according to a first change of tunnel positioning results and a second change of vehicle positioning results; when the IMU error characteristics are less than or equal to the UBW error characteristics, re-fusing the IMU positioning results and the UWB positioning results according to the difference between the IMU error characteristics and the UBW error characteristics to obtain updated vehicle positioning results; when the IMU error characteristics are greater than the UBW error characteristics, resetting the positioning process of inertial navigation technology, and publishing information according to the updated vehicle positioning results. The present application further ensures the accuracy of vehicle positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning. BACKGROUND

[0002] In order to ensure the driving safety of vehicles and deal with emergency situations in highway tunnels, tunnel information needs to be published to vehicles in the tunnel, such as the location of fire-fighting equipment, the entrance position of emergency escape passages, etc. When vehicles receive this tunnel information, they will display or mark it in the AR scene of the vehicle-mounted AR-HUD. When marking the tunnel information in the AR scene of the vehicle-mounted AR-HUD, it needs to be based on the positioning of the vehicle. When the positioning of the vehicle is inaccurate, it may cause the marking position of the tunnel information in the AR scene of the vehicle-mounted AR-HUD to deviate.

[0003] The conventional positioning of vehicles is obtained by Beidou navigation positioning technology. However, the Beidou signal will weaken in tunnels, and even the Beidou signal cannot be received, resulting in a large error in the positioning result obtained by the Beidou navigation positioning technology. The usual practice is to fuse the inertial navigation technology and the UWB base station positioning technology by using the Kalman filtering algorithm when the Beidou signal weakens or is lost, and then realize positioning. However, since the errors in the positioning results obtained by the inertial navigation technology and the UWB base station positioning technology are accumulated with time and change with the environment, and the Kalman filtering algorithm uses Gaussian distributed errors for filtering fusion, the error in the positioning result obtained by the Kalman filtering algorithm cannot be ignored for errors that change with time and environment, that is, the error in the positioning result obtained by the Kalman filtering algorithm will cause the marking position of the tunnel information in the AR scene of the vehicle-mounted AR-HUD to deviate significantly. SUMMARY

[0004] In order to solve the problem that the errors in the positioning results obtained by the inertial navigation technology and the UWB base station positioning technology have the characteristics of changing with time and environment, and the error in the positioning result obtained by the Kalman filtering algorithm cannot be ignored, the present application provides a highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning.

[0005] The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning of the present application adopts the following technical scheme:

[0006] An embodiment of the present application provides a highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning, which comprises the following modules:

[0007] a tunnel positioning module, configured to project and transform the vehicle images into the tunnel model to obtain positions of all the vehicles in the tunnel, denoted as tunnel positioning results, and publish tunnel information including the tunnel positioning results of all the vehicles to all the vehicles;

[0008] a vehicle positioning module, configured to position each vehicle in the tunnel by using inertial navigation technology and UWB base stations respectively to obtain IMU positioning results and UWB positioning results respectively, and fuse the IMU positioning results and the UWB positioning results by using a Kalman filtering algorithm to obtain vehicle positioning results;

[0009] a positioning updating module, configured to, for a same vehicle, obtain a first change amount of the tunnel positioning results and a second change amount of the vehicle positioning results within a preset time, and record a difference between the second change amount and the first change amount as an error change feature;

[0010] a growth trend and a fluctuation trend of all the error change features obtained before a current time are recorded as IMU error features and UBW error features respectively; when the IMU error features are less than or equal to the UBW error features, the IMU positioning results and the UWB positioning results are re-fused according to a difference between the IMU error features and the UBW error features to obtain updated vehicle positioning results at the current time; and when the IMU error features are greater than the UBW error features, a positioning process of the inertial navigation technology is reset;

[0011] an information publishing module, configured to visualize the tunnel information to the AR-HUD according to the updated vehicle positioning results at the current time.

[0012] Preferably, the projection and transformation of the vehicle images into the tunnel model to obtain the positions of all the vehicles in the tunnel, denoted as the tunnel positioning results, include the following specific steps:

[0013] each camera in the tunnel collects vehicle images, and after affine transformation and image fusion of all the vehicle images collected by all the cameras, a panoramic view of all the vehicles in the tunnel is obtained; the panoramic view is pasted on a road in the tunnel model; pixel point positions of each vehicle in the panoramic view are identified, and a positioning result of the pixel point positions in the tunnel model is denoted as a tunnel positioning result of each vehicle.

[0014] Preferably, the obtaining of the first change amount of the tunnel positioning results and the second change amount of the vehicle positioning results within the preset time includes the following specific steps:

[0015] for an i-th time point after the strength of the Beidou signal is less than a preset strength, a tunnel positioning result of each vehicle at the i-th time point is denoted as a tunnel positioning result at an i-p-th time point is denoted as is obtained by and the distance between the two is recorded as a first change amount; the vehicle positioning result of each vehicle at the i-th moment is recorded as the vehicle positioning result at the i-p-th moment is recorded as the distance between the two is recorded as a second change amount; p is a preset time.

[0016] Preferably, the growth trend and fluctuation trend of all error change characteristics obtained before the current moment are recorded as IMU error characteristics and UBW error characteristics respectively, and the specific steps include the following:

[0017] The moment when the Beidou signal strength is less than the preset strength is marked as the initial moment; for the current moment after the Beidou signal strength is less than the preset strength, the error change characteristics obtained before the current moment and after the initial moment form a time sequence, which is recorded as an error change characteristic sequence; the error change characteristic sequence is filtered and normalized, and the trend component of the normalized error change characteristic sequence is obtained by using the STL algorithm; the difference between the normalized error change characteristic sequence and the trend component is recorded as a fluctuation component.

[0018] The mean value of the current moment and the mean value of several element values before the current moment in the trend component are recorded as IMU error characteristics; the mean value of the absolute values of all element values in the fluctuation component is recorded as UBW error characteristics.

[0019] Preferably, the IMU positioning result and the UWB positioning result are re-fused according to the difference between the IMU error characteristics and the UBW error characteristics to obtain the updated vehicle positioning result at the current moment, and the specific steps include the following:

[0020] The IMU error characteristics are recorded as a1, the UBW error characteristics are recorded as a2, and a1 / (a2+0.1) is recorded as the IMU error correction strength.

[0021] The fusion weight of the Kalman filter algorithm in fusing the IMU positioning result and the UWB positioning result is updated by using the IMU error correction strength, and the updated fusion weight is negatively correlated with the IMU error correction strength.

[0022] The IMU positioning result and the UWB positioning result are re-fused by using the updated fusion weight to obtain the updated vehicle positioning result at the current moment.

[0023] Preferably, the tunnel information is visualized onto the AR-HUD on the vehicle according to the updated vehicle positioning result at the current moment, and the specific steps include the following:

[0024] ​​The tunnel information includes a plurality of reference positions, and the reference positions include a fire-fighting equipment position, an emergency escape passage entrance position, a fire occurrence position, and a roadblock position.

[0025] The difference between the reference position and the vehicle positioning result is recorded as a marked position, and the marked position in the AR scene of the vehicle-mounted AR-HUD displays a warning icon.

[0026] Preferably, the IMU error correction intensity is used to update the fusion weight of the Kalman filtering algorithm when fusing the IMU positioning result and the UWB positioning result, and the specific formula includes:

[0027] ;

[0028] wherein represents the updated fusion weight, a represents the IMU error correction intensity, and x represents the fusion weight before updating.

[0029] Preferably, the updated vehicle positioning result , wherein represents the updated fusion weight, P1 represents the UWB positioning result, and P2 represents the IMU positioning result.

[0030] Preferably, the fusion weight before updating ; wherein P1 represents the UWB positioning result, P2 represents the IMU positioning result, and PO represents the vehicle positioning result obtained by fusing the Kalman filtering algorithm.

[0031] Preferably, the time point of resetting the positioning process of the inertial navigation technology is re-marked as the initial time point.

[0032] The technical scheme of the present application has the following beneficial effects:

[0033] The present application uses the inertial navigation technology and the UWB base station to respectively position each vehicle in the tunnel, respectively obtains the IMU positioning result and the UWB positioning result, and uses the Kalman filtering algorithm to fuse the IMU positioning result and the UWB positioning result to obtain the vehicle positioning result. The vehicle positioning result obtained in this process has smaller error compared with the IMU positioning result and the UWB positioning result, thereby ensuring the accuracy of vehicle positioning.

[0034] Further, the first change amount of the tunnel positioning result and the second change amount of the vehicle positioning result within a preset time are obtained, the IMU error feature and the UBW error feature are obtained; when the IMU error feature is less than or equal to the UBW error feature, the IMU positioning result and the UWB positioning result are re-fused according to the difference between the IMU error feature and the UBW error feature, and the updated vehicle positioning result at the current time is obtained; and when the IMU error feature is greater than the UBW error feature, the positioning process of the inertial navigation technology is reset.

[0035] The process resets the integral process of the inertial navigation technology based on the IMU error feature and the UBW error feature, so that the integral process resetting time is appropriate, that is, the problem that the error of the UWB positioning result is relatively large when the integral process is reset too early, resulting in the error of the vehicle positioning result being large, is avoided, and the problem that the error of the IMU positioning result is accumulated for a long time when the integral process is reset too late, resulting in the error of the subsequent vehicle positioning result being large, is also avoided; on the other hand, the fusion process (that is, the vehicle positioning result) of the Kalman filtering algorithm is updated, so that the updated vehicle positioning result further considers the interference of other errors changing with time and environment in addition to the Gaussian distribution error on the basis of the Kalman filtering algorithm fusion.

[0036] In summary, the inertial navigation technology is reset or the vehicle positioning result is updated according to the error distribution of the IMU positioning result and the UWB positioning result changing with time or environment, which further ensures the accuracy of the vehicle positioning at the current time, and helps to avoid the position deviation of the icon on the vehicle-mounted AR-HUD. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0038] Figure 1 a framework structure diagram of a highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning provided by an embodiment of the present application;

[0039] Figure 2 a step flowchart of all modules implemented in the highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to further clarify the technical means and effects taken by the present application to achieve the predetermined inventive purpose, the specific implementation, structure, features and effects of the highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning according to the present application are described in detail as follows in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0042] The specific scheme of the highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning provided by the present application is described in detail below in combination with the drawings.

[0043] Embodiment one:

[0044] Please refer to Figure 1 which shows the highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning provided by the present application, which includes the following modules: tunnel positioning module, vehicle positioning module, positioning update module, information publishing module. Figure 2 The steps contained in all modules are shown.

[0045] The tunnel positioning module is used to collect vehicle driving information in the tunnel and project these information in real time on the three-dimensional model of the tunnel.

[0046] A number of industrial cameras (hereinafter referred to as cameras) are installed in the tunnel, each camera has a downward-looking view, and the field of view of all cameras covers the entire highway in the tunnel, each camera simultaneously collects video images at the same frame rate (10 frames per second in this embodiment), each frame is a color (RGB) image (the image size in this embodiment is 512x512), and each frame of image collected by all cameras contains driving information of all vehicles in the tunnel (such as vehicle position, vehicle license plate number, etc.).

[0047] Wireless broadcasting technology is used in the tunnel to broadcast and publish vehicle driving information and tunnel-related information in the tunnel, and when the vehicle receives the broadcast, it can know the traffic running condition in the tunnel.

[0048] Specifically, this module is used to implement the following steps:

[0049] Step S101, collect vehicle images by using cameras installed in the tunnel and project and transform into the tunnel model to obtain the positions of all vehicles in the tunnel, denoted as tunnel positioning results, and publish the tunnel information including the tunnel positioning results of all vehicles to all vehicles.

[0050] In the vehicle positioning module, each vehicle is positioned by using Beidou navigation technology, inertial navigation technology and UWB base station.

[0051] Specifically, the embodiment takes into account that the Beidou satellite signal in the tunnel is weak, and even no Beidou signal can be received, so it is necessary to fuse other positioning technologies, such as fusing the positioning results of inertial navigation technology and the positioning results of UWB base station positioning technology.

[0052] The inertial navigation technology fuses the vehicle-mounted inertial measurement unit (IMU) to perform positioning, and the IMU integrates 3-axis accelerometer + 3-axis gyroscope, which are respectively used to measure the three-axis linear acceleration of the vehicle and the three-axis angular velocity of the vehicle. The inertial navigation technology is based on Newton's law of mechanics, and the vehicle positioning is realized by integrating the acceleration and angular velocity data.

[0053] The inertial navigation technology is known, and the specific principle thereof will not be described herein.

[0054] The UWB (Ultra-Wideband) base station positioning technology uses the time of flight of signals between the positioning base stations deployed in the tunnel and the vehicle to perform positioning, and one UWB base station is installed every 800 meters in the embodiment. The technology is also known, and will not be described in detail herein.

[0055] The module specifically realizes the following steps:

[0056] Step S102, position each vehicle in the tunnel by using the inertial navigation technology and the UWB base station respectively to obtain the IMU positioning result and the UWB positioning result, and fuse the IMU positioning result and the UWB positioning result by using Kalman filtering algorithm to obtain the vehicle positioning result.

[0057] The positioning updating module is used to update the vehicle positioning result obtained above.

[0058] Specifically, on the one hand, the core problem of inertial navigation technology is that the position and velocity need to be obtained by double integration of acceleration, and the attitude needs to be integrated with angular velocity. Any small error will be amplified in the integration process, and eventually lead to exponential growth of error with time. On the other hand, the UWB signal encounters obstacles (such as tunnel walls, vehicles) and produces reflection and refraction, forming multiple propagation paths, resulting in the detection of multiple delay signals, which interfere with the time delay measurement of the direct signal wave; especially when there are obstacles (such as other vehicles) between the base station and the vehicle, the signal needs to detour or penetrate the obstacle, and the propagation time is prolonged; that is, the error of the UWB positioning result is affected by the environment.

[0059] In the above module, when the Kalman filtering algorithm is used to fuse the IMU positioning result and the UWB positioning result, it is considered that the Kalman filtering algorithm is a fast and efficient known filtering method based on Gaussian noise error. It can fuse multiple source data (i.e. IMU positioning result and UWB positioning result), so that the error of the obtained vehicle positioning result is reduced. However, in this embodiment, the error of the IMU positioning result grows exponentially with time, and the error of the UWB positioning result is affected by the environment (i.e. the IMU positioning result and the UWB positioning result not only have Gaussian distribution error, but also have error that changes with time and environment), resulting in that the error of the vehicle positioning result obtained by the Kalman filtering algorithm cannot be effectively suppressed.

[0060] In this module, the error of the vehicle positioning result is further suppressed (i.e. the vehicle positioning result obtained by the Kalman filtering algorithm is updated) by the following two steps:

[0061] Step S103, for the same vehicle, after a preset time, obtain a first change amount of the tunnel positioning result and a second change amount of the vehicle positioning result within the preset time, and record the difference between the second change amount and the first change amount as an error change feature. The growth trend and fluctuation trend of all error change features obtained before the current time are recorded as IMU error features and UBW error features, respectively.

[0062] Step S104, when the IMU error feature is less than or equal to the UBW error feature, the IMU positioning result and the UWB positioning result are re-fused according to the difference between the IMU error feature and the UBW error feature, to obtain the updated vehicle positioning result at the current time; when the IMU error feature is greater than the UBW error feature, reset the positioning process of the inertial navigation technology.

[0063] The information publishing module is used to realize step S105, visualizing the tunnel information on the AR-HUD according to the vehicle positioning result at the current time.

[0064] The vehicle positioning result in the module has smaller error, and based on the vehicle positioning result, vehicle driving information and tunnel related information in the tunnel can be accurately displayed on the vehicle AR-HUD, for example, some warning or prompt information is marked on the vehicle AR-HUD, so that the marking position of the warning or prompt information is avoided from being wrong.

[0065] Embodiment two:

[0066] In step S101, a vehicle image is collected by using a camera installed in the tunnel and is projected and transformed into a tunnel model to obtain positions of all vehicles in the tunnel, denoted as tunnel positioning results, and tunnel information including the tunnel positioning results of all vehicles is published to all vehicles, and a specific implementation method includes:

[0067] A tunnel three-dimensional model (referred to as a tunnel model) is constructed or placed in a three-dimensional modeling or simulation software (for example, a modeling software such as Maya, CAD, or a UE5 game engine), and the three-dimensional model in this embodiment one-to-one restores the tunnel (or the three-dimensional model one-to-one restores the construction drawing of the tunnel). The three-dimensional model includes a road, fire-fighting equipment, and an emergency escape passage entrance in the tunnel.

[0068] In addition, a plurality of monitoring points outside the tunnel entrance are manually selected, and the positioning results of the monitoring points are obtained by using Beidou positioning technology; and the positioning results are marked in the three-dimensional model, and then the positioning results of any position in the three-dimensional model (for example, the position of any grid point in the three-dimensional model) are obtained by using a linear interpolation algorithm.

[0069] When the tunnel is put into use, for each frame of image (denoted as a vehicle image) collected by each camera in the tunnel, an affine transformation is performed on each frame of vehicle image by using a calibrated homography matrix, and the purpose is to transform the image collected by the camera perspective into a top-down perspective (that is, the affine transformation obtains a top-down perspective image). All vehicle images collected by each camera are transformed into top-down perspective images, and all top-down perspective images of the cameras are spliced and fused by using a Gaussian pyramid fusion technology to obtain a top-down panorama of all vehicles in the tunnel. The panorama is pasted on the road in the three-dimensional model, and each pixel point on the panorama in the three-dimensional model corresponds to a positioning result.

[0070] The affine transformation and the Gaussian pyramid fusion technology are both known image processing technologies, and the embodiment will not be described in detail.

[0071] In this embodiment, YOLOV5 is used to identify a rectangular bounding box of each vehicle in the panorama, and a positioning result corresponding to a center pixel point of each bounding box represents a position of the vehicle in the tunnel, denoted as a tunnel positioning result of each vehicle.

[0072] It should be noted that when the rectangular bounding box of each vehicle in the panoramic image is recognized by YOLOV5, the panoramic image is equally divided into a plurality of image blocks, and the rectangular bounding box of the vehicle in each image block is recognized by YOLOV5, and then the bounding box of all vehicles is obtained; in the embodiment, the size of each image block is 512*512, and if the image block is less than 512*512 after equal division, it is filled into 512*512 by filling 0.

[0073] At this point, the image data of the vehicles in the tunnel is collected by all cameras in the tunnel, and is projected and transformed into the three-dimensional model in real time, so that the three-dimensional model can display the positions of all vehicles (i.e., the tunnel positioning results of each vehicle) in real time.

[0074] Further, the tunnel information including the tunnel positioning results of all vehicles is broadcasted.

[0075] The tunnel information includes the tunnel positioning results of all vehicles and license plate numbers, and also includes the positions of fire-fighting equipment and emergency escape entrances in the tunnel.

[0076] In step S102, each vehicle in the tunnel is positioned by using the inertial navigation technology and the UWB base station respectively to obtain the IMU positioning result and the UWB positioning result respectively, and the IMU positioning result and the UWB positioning result are fused by using the Kalman filtering algorithm to obtain the vehicle positioning result, and the specific implementation method includes:

[0077] After the vehicle enters the tunnel, when the vehicle detects that the Beidou signal strength (specifically represented by carrier-to-noise ratio) is less than a preset strength, the Beidou navigation technology is no longer used for positioning, but the inertial navigation technology and the UWB base station are used for positioning. When the vehicle detects that the Beidou signal strength is greater than or equal to the preset strength, the Beidou navigation technology is continued to be used for positioning. In the embodiment, the preset strength is 42 dBHz, and in other embodiments, it can be set to other values, which are not limited in the embodiment.

[0078] The moment when the Beidou signal strength is less than the preset strength is recorded as the positioning switching moment, and the specific method of positioning by using the inertial navigation technology and the UWB base station after the positioning switching moment is:

[0079] The positioning result obtained by using the inertial navigation technology to position the vehicle is recorded as the IMU positioning result, and the positioning result obtained by using the UWB base station to position the vehicle is recorded as the UWB positioning result; both the IMU positioning result and the UWB positioning result can represent the position of the vehicle in the tunnel, but these two positioning results have a large error alone.

[0080] The embodiment fuses the IMU positioning result and the UWB positioning result by using a Kalman filtering algorithm to obtain a vehicle positioning result, which has a smaller error compared with the IMU positioning result or the UWB positioning result. The IMU positioning result is taken as a state quantity of the Kalman filtering algorithm, and the UWB positioning result is taken as an observation value of the Kalman filtering algorithm. The specific process of the Kalman filtering algorithm is known, and will not be described in detail in the embodiment.

[0081] In step S103, every preset time, the first change amount of the tunnel positioning result and the second change amount of the vehicle positioning result within the preset time are obtained, and the difference between the second change amount and the first change amount is taken as an error change feature. The growth trend and the fluctuation trend of all the error change features obtained before the current time are taken as an IMU error feature and a UWB error feature, respectively. The specific implementation scheme includes:

[0082] It should be noted that after the positioning switching time, although the vehicle positioning result is obtained by fusing the IMU positioning result and the UWB positioning result by using the Kalman filtering algorithm, the positioning error is reduced. However, the IMU positioning result and the UWB positioning result not only have Gaussian distribution errors, but also have other types of errors, for example, the error of the IMU positioning result presents an exponential growth trend with time, and the error of the UWB positioning result is affected by the tunnel environment (for example, the curve of the tunnel, the tunnel wall, the vehicle in the tunnel, etc.). However, all the errors are regarded as Gaussian noise errors in the Kalman filtering algorithm, so that the error of the obtained vehicle positioning result cannot be further suppressed, that is, the error of the vehicle positioning result can be further optimized in the embodiment.

[0083] It should be further noted that any vehicle in the tunnel is taken as a target vehicle. For the target vehicle, after the target vehicle receives the tunnel information broadcast by the tunnel, the tunnel positioning result of all the vehicles contained in the tunnel information is first obtained, and among all the tunnel positioning results of the vehicles, the tunnel positioning result corresponding to the target vehicle is obtained by matching the license plate number of the target vehicle.

[0084] For the tunnel positioning result of the target vehicle, the error comes from the measurement error of the positioning result of the monitoring point in step S101 and the error existing in the affine transformation process (for example, Gaussian noise when the camera images, error of the homography matrix in the calibration, etc.), which presents the characteristics of Gaussian distribution and is irrelevant to the environment and time. The tunnel positioning result of the target vehicle is used to suppress the error in the IMU positioning result and the UWB positioning result which changes with time and environment in the embodiment.

[0085] Specifically, the target vehicle is at a positioning switching time, every 0.5 seconds is a time, the positioning switching time is regarded as the 0th time, and is marked as an initial time, and the tunnel positioning result of the ith time after the initial time is recorded as , the tunnel positioning result of the i-pth time is recorded as , the distance between and is represented as , which represents the first change amount of the tunnel positioning result within the time period p. In this embodiment, p=4 is taken as an example for description, and in other embodiments, p can be set to other integer values greater than 0, and preferably p is less than or equal to 6.

[0086] The vehicle positioning result of the ith time is recorded as , and the vehicle positioning result of the i-pth time is recorded as , the distance between and is represented as , which represents the second change amount of the vehicle positioning result within the time period p.

[0087] Specifically, when i-p is less than 0, the i-pth time represents a time before the positioning switching time, and the vehicle positioning result at this time is the positioning result obtained by using the Beidou navigation technology.

[0088] The absolute value of the difference between the second change amount and the first change amount is recorded as the error change feature of the ith time.

[0089] The first change amount within the preset time p represents the movement (i.e., displacement) of the target vehicle within the preset time p, wherein the errors of the tunnel positioning results at different times have the same distribution (i.e., the errors of the tunnel positioning results at different times have the same Gaussian distribution, which can offset each other to a certain extent), and thus the first change amount at different times can relatively accurately represent the movement of the target vehicle within the preset time p.

[0090] The second change amount within the preset time p also represents the movement of the target vehicle within the preset time p, but the error distribution of the vehicle positioning results at different times is different (because the IMU positioning result and the UWB positioning result contain errors that change with time and environment), and thus the second change amount at different times has certain differences when describing the movement of the target vehicle within the preset time p.

[0091] The error change feature represents the difference between the movement of the target vehicle described by the second change amount within the preset time p and the movement of the target vehicle described by the first change amount, which can reflect the error change of the vehicle positioning result.

[0092] Further, the time sequence composed of the error change characteristics of all time points before the current time point and after the initial time point (including the current time point and the initial time point) is recorded as an error change characteristic sequence; the growth trend existing in the sequence can reflect the error change of the IMU, and the fluctuation trend existing in the sequence can reflect the error change of the UBW, and the growth trend and the fluctuation trend of the error change characteristic sequence are recorded as an IMU error characteristic and a UBW error characteristic respectively in the embodiment.

[0093] As an example, the method for obtaining the IMU error characteristic and the UBW error characteristic is as follows:

[0094] First, a Gaussian filter kernel with a length of 3 is used to perform Gaussian filtering on the error change characteristic sequence, and the purpose is to further exclude the interference of the error that has not been mutually canceled when the first change amount changes over time.

[0095] The filtered error change characteristic sequence is linearly normalized, and the STL algorithm (time series decomposition algorithm) is used to obtain the trend component of the normalized error change characteristic sequence (a time sequence with the same length as the error change characteristic sequence), which represents the growth trend contained in the error change characteristic sequence.

[0096] The difference between the normalized error change characteristic sequence and the trend component (that is, the difference between the element values of the same time sequence represented by the two) is obtained, and the fluctuation component (also a time sequence with the same length as the error change characteristic sequence) is obtained.

[0097] The mean of the current time point and a plurality of (for example, 5) element values before the current time point in the trend component is recorded as the IMU error characteristic; in particular, when there are less than 5 element values before the current time point, the subsequent step S104 is not performed, and the subsequent step S105 is directly performed.

[0098] The mean of the absolute values of all element values in the fluctuation component is recorded as the UBW error characteristic.

[0099] Step S104, when the IMU error characteristic is less than or equal to the UBW error characteristic, the IMU positioning result and the UWB positioning result are re-fused according to the difference between the IMU error characteristic and the UBW error characteristic to obtain the updated vehicle positioning result at the current time point; when the IMU error characteristic is greater than the UBW error characteristic, the positioning process of the inertial navigation technology is reset, and the specific implementation method includes:

[0100] When the IMU error feature is greater than the UBW error feature, it indicates that the error interference of the IMU positioning result is relatively large in terms of the error interference of the relative UWB positioning result, that is, the vehicle positioning result obtained by the target vehicle at the current time will be obviously interfered by the error of the IMU positioning result. At this time, the embodiment resets the positioning process of the inertial navigation technology.

[0101] The reset positioning process of the inertial navigation technology refers to resetting the integration process of acceleration and angular velocity at the current time, and reusing the IMU to measure the three-axis linear acceleration of the vehicle and the three-axis angular velocity of the vehicle to obtain the IMU positioning result by integrating the acceleration and angular velocity from the current time. The integration process is a known content in the inertial navigation technology, and will not be described herein.

[0102] It should be noted that the vehicle positioning result at the current time is still the vehicle positioning result obtained in step S102. The IMU positioning result is obtained by using the reset inertial navigation technology after the time after the current time, and the vehicle positioning result after the time is obtained based on the IMU positioning result by using step S102. At the same time, the current time is marked as the initial time, and then steps S103 and S104 are used to judge again whether the IMU error feature is greater than the UBW error feature at the subsequent time. If yes, the positioning process of the inertial navigation technology is continued to be reset.

[0103] The above process avoids the case that the error of the IMU positioning result is accumulated for a long time to cause the error of the subsequent vehicle positioning result to become large. Further, the integration process of the inertial navigation technology is reset based on the UBW error feature, so that the reset timing of the integration process is appropriate, that is, the problem that the error of the vehicle positioning result becomes large due to the relatively large error of the UWB positioning result when the reset is too early is avoided, and the problem that the error of the IMU positioning result is accumulated for a long time to cause the error of the subsequent vehicle positioning result to become large when the reset is too late is also avoided.

[0104] It should be noted that step S103 records that when there are less than 5 element values before the current time, the subsequent step S104 is not executed. One of the purposes is to avoid frequent resetting of the inertial navigation technology in step S104, for example, when the error of the UWB positioning result is always small or does not exist, the inertial navigation technology can be avoided from being frequently reset to cause the inertial navigation technology to be invalid.

[0105] When the IMU error feature is less than or equal to the UBW error feature, the IMU error feature is recorded as a1, the UBW error feature is recorded as a2, and a1 / (a2+0.1) is recorded as the IMU error correction degree a. The purpose of adding 0.1 to the denominator is to avoid the denominator being 0.

[0106] The IMU error correction strength represents the relative size of the cumulative error of the IMU positioning result changing over time, or in other words, the difference between the IMU error characteristics and the UWB error characteristics. The smaller the value, the smaller the IMU error characteristics relative to the UWB error characteristics (the interference of the cumulative error of the IMU positioning result changing over time is relatively small).

[0107] As can be known from the Kalman filtering algorithm, when fusing the IMU positioning result and the UWB positioning result, the fusion is based on the error size (or the variance of the error) of the IMU positioning result and the error size of the UWB positioning result, wherein the larger the error size (for example, the variance of the error of the IMU positioning result), the smaller the weight when fusing, and the smaller the error size (for example, the variance of the error of the IMU positioning result), the larger the weight when fusing. The weight when fusing is essentially represented by the gain coefficient of the Kalman filtering.

[0108] In addition, all errors (for example, the error of the IMU positioning result) in the Kalman filtering algorithm are considered as Gaussian distributed errors, without considering other errors including errors changing over time and environment; the embodiment re-fuses on the basis of the fusion process of the Kalman filtering algorithm by using the IMU error correction strength, to realize the update of the fusion result (that is, the vehicle positioning result) of the Kalman filtering algorithm, so as to further consider other error conditions (for example, errors changing over time and environment).

[0109] Specifically, the fusion process of the Kalman filtering algorithm is represented as:

[0110] ;

[0111] Wherein, P1 represents the UWB positioning result, P2 represents the IMU positioning result, PO represents the vehicle positioning result obtained by fusing, and x represents the weight when fusing (referred to as the fusion weight); after transforming the above formula, the following formula is obtained: .

[0112] Specifically, when P2 is equal to P1, it means that the UWB positioning result and the IMU positioning result are the same, at this time, x can take any value in the interval [0, 1], and in the embodiment, x=0.5, and in other embodiments, the subsequent step can not be executed but directly executed step S105.

[0113] Further, the fusion weight is updated and corrected by using the IMU error correction strength, and the following formula is obtained:

[0114] ;

[0115] Wherein , a represents the IMU error correction strength. The purpose of this formula is to update the fusion weight of the fusion process based on the Kalman filter algorithm.

[0116] The updated vehicle positioning result .

[0117] The updated vehicle positioning result Compared with the vehicle positioning result P before the update, the former further considers the interference of other errors changing with time and environment in addition to the Gaussian distributed error based on the fusion of the Kalman filter algorithm. Specifically, the greater the IMU error correction strength (at this time is smaller), the greater the error interference of the IMU positioning result changing with time, and at this time the updated vehicle positioning result does not focus on the IMU positioning result, but focuses more on the UWB positioning result. The smaller the IMU error correction strength (at this time is greater), the greater the error interference of the IMU positioning result changing with time, and at this time the updated vehicle positioning result pays more attention to the IMU positioning result.

[0118] Specifically, when is less than or equal to 0.1, let ; when is greater than or equal to 0.9, let ; the purpose is to avoid being too large or too small, resulting in the IMU positioning result or the UWB positioning result not participating in the fusion.

[0119] So far, in the current time, when the IMU error feature is less than or equal to the UWB error feature, the vehicle positioning result is updated to suppress the interference of errors changing with time or environment.

[0120] In summary, in the current time, according to the error distribution of the IMU positioning result and the UWB positioning result changing with time or environment, the inertial navigation technology is reset or the vehicle positioning result is updated, further ensuring the accuracy of the vehicle positioning in the current time.

[0121] Step S105, visualizing the tunnel information to the AR-HUD according to the vehicle positioning result in the current time, including the specific implementation method:

[0122] In addition to containing the tunnel positioning results of all vehicles, the tunnel information also includes the positions of fire-fighting equipment, emergency escape passage entrances, etc. These positions are referred to as reference positions, and these reference positions represent the positioning results in the tunnel three-dimensional model.

[0123] For the above-mentioned obtained current time vehicle positioning result (if the vehicle positioning result is updated, it refers to the updated vehicle positioning result). The difference between the reference position and the vehicle positioning result is obtained, denoted as the marked position, which represents the relative position of the reference position relative to the vehicle body.

[0124] The vehicle-mounted AR-HUD is a display device based on AR technology, which can project a virtual AR scene onto the windshield, wherein the display positions of various icons (or UI elements) with warning functions in the AR scene are all relative positions relative to the vehicle positioning result. In the AR scene, some preset icons are displayed at each marked position (for example, a fire icon is displayed at the marked position corresponding to a fire-fighting device, and an emergency escape passage icon is displayed at the marked position corresponding to the entrance of an emergency escape passage). The process of displaying icons (or UI elements) in the AR scene according to the vehicle positioning result is a known technology, and the present embodiment will not be described in detail.

[0125] Embodiment three:

[0126] In step S101, the method for obtaining the calibrated homography matrix is:

[0127] When each camera in the tunnel is fixed, a chessboard is placed on the tunnel road in the field of view of the camera. An image sample is collected by the camera, and the image of the chessboard taken from the overhead perspective is denoted as a chessboard sample. The SIFT corner point algorithm is used to detect the corner points in the image sample and the chessboard sample, and the corner point matching algorithm (such as the normalized cross-correlation matching algorithm) is used to match the corner points in the image sample with the corner points in the chessboard sample, to obtain all the matched corner point pairs. Based on these matched corner point pairs, the RANSAC algorithm is used to obtain a homography matrix, which can perform affine transformation on the image sample to the chessboard sample, so that the image sample becomes an overhead perspective. This homography matrix is denoted as the calibrated homography matrix.

[0128] The methods used in the above process are all known image processing techniques, and the present embodiment will not be described in detail.

[0129] In step S102, when the Beidou signal strength detected by the vehicle is greater than or equal to the preset strength (i.e. before the positioning switching time), the positioning result of the Beidou navigation technology (denoted as Beidou positioning result) and the IMU positioning result and the UWB positioning result are obtained at the same time in the present embodiment.

[0130] The Beidou positioning result, the IMU positioning result and the UWB positioning result are fused by using a Kalman filtering algorithm to obtain the vehicle positioning result, wherein the IMU positioning result and the UWB positioning result are state variables of the Kalman filtering algorithm, and the Beidou positioning result is an observation value of the Kalman filtering algorithm.

[0131] In addition, before the positioning switching time, steps S103 and S104 in Embodiment 1 are no longer implemented, and step S105 is directly executed.

[0132] The positioning result obtained by using the Beidou positioning technology in step S101 and the positioning result obtained by positioning the vehicle by the UWB base station in step S102 are both positioning results determined by longitude, latitude and height (elevation) three dimensions, and the positioning result is a spherical coordinate in the geocentric coordinate system. In the present embodiment, the Mercator projection method is used to map the spherical coordinate into a plane coordinate (i.e. the coordinate in the vehicle navigation map). Therefore, the tunnel positioning result, the IMU positioning result and the UWB positioning result in Embodiment 2 are all unified into plane coordinates. In addition, the distance described in step S103 is all the Euclidean distance.

[0133] It should be noted that in the present embodiment, the height difference of the road is negligible when driving in the tunnel, so the height is not considered. In other embodiments, the height can also be used as the third dimension of the vertical plane coordinate.

[0134] In step S105, the tunnel information further includes: fire occurrence position, roadblock position, etc.

[0135] The method for obtaining the fire occurrence position and the roadblock position is as follows:

[0136] The center point of the bounding box of the fire area and the roadblock area in the panoramic image is detected by using the YOLOV5 neural network, and the center point is taken as the fire occurrence position and the roadblock position.

[0137] The fire occurrence position and the roadblock position are displayed on the vehicle AR-HUD to remind the vehicle to drive in the correct lane.

[0138] Other improved embodiments based on the present application also include: replacing the Kalman filtering algorithm with an extended Kalman filtering algorithm or an ion filtering algorithm.

[0139] It should be noted that although the ion filtering algorithm is free from Gaussian constraints, the limited number of particles will also limit the suppression of errors that change over time or environment (too many particles will greatly reduce the calculation speed). The ion filtering algorithm can still further improve the accuracy of the vehicle positioning result when used in the above embodiments.

[0140] The above merely provides the preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the principle of the present application should be included in the protection scope of the present application.

Claims

1. A highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning, characterized in that, The system comprises the following modules: A tunnel positioning module, configured to project and transform vehicle images into a tunnel model to obtain positions of all vehicles in the tunnel, denoted as tunnel positioning results, and publish tunnel information comprising the tunnel positioning results to all vehicles; A vehicle positioning module, configured to position each vehicle in the tunnel by using inertial navigation technology and UWB base stations respectively to obtain IMU positioning results and UWB positioning results respectively, and fuse the IMU positioning results and the UWB positioning results by using a Kalman filtering algorithm to obtain vehicle positioning results; A positioning updating module, configured to obtain a first change amount of the tunnel positioning results and a second change amount of the vehicle positioning results within a preset time interval after each vehicle has passed the preset time interval, and record a difference between the second change amount and the first change amount as an error change feature; A growth trend and a fluctuation trend of all error change features obtained before the current time are denoted as IMU error features and UBW error features respectively; When the IMU error features are less than or equal to the UBW error features, the IMU positioning results and the UWB positioning results are re-fused according to a difference between the IMU error features and the UBW error features to obtain updated vehicle positioning results at the current time; When the IMU error features are greater than the UBW error features, a positioning process of the inertial navigation technology is reset; An information publishing module, configured to visualize the tunnel information to an AR-HUD on a vehicle according to the updated vehicle positioning results at the current time.

2. The highway tunnel information publishing system based on the vehicle-mounted AR-HUD and the Beidou navigation positioning according to claim 1, characterized in that, The projection and transformation of the vehicle images into the tunnel model to obtain the positions of all vehicles in the tunnel, denoted as the tunnel positioning results, comprise the following specific steps: Each camera in the tunnel collects vehicle images, and after affine transformation and image fusion of all vehicle images collected by all cameras, a panoramic view of all vehicles in the tunnel is obtained; and the panoramic view is pasted on a road in the tunnel model; Pixel point positions of each vehicle in the panoramic view are identified, and a positioning result of the pixel point positions in the tunnel model is denoted as a tunnel positioning result of each vehicle. 3.The highway tunnel information publishing system based on the vehicle-mounted AR-HUD and Beidou navigation positioning of claim 1, characterized in that, The obtaining of the first change amount of the tunnel positioning results and the second change amount of the vehicle positioning results within the preset time interval comprises the following specific steps: For the i-th moment after the Beidou signal strength is less than the preset strength, the tunnel positioning result of each vehicle at the i-th moment is recorded as The tunnel positioning result at the i-p-th moment is recorded as The distance between and is recorded as the first change amount; the vehicle positioning result of each vehicle at the i-th moment is recorded as The vehicle positioning result at the i-p-th moment is recorded as The distance between and is recorded as the second change amount; p is a preset time.

4. The highway tunnel information publishing system based on the vehicle-mounted AR-HUD and the Beidou navigation positioning of claim 1, characterized in that, The growth trend and the fluctuation trend of all error change features obtained before the current time are denoted as IMU error features and UBW error features respectively, which comprise the following specific steps: A time point at which the Beidou signal strength is less than a preset strength is marked as an initial time point; for a current time point after the Beidou signal strength is less than the preset strength, a time sequence composed of error change features obtained before the current time point and after the initial time point is denoted as an error change feature sequence; the error change feature sequence is filtered and normalized, and a trend component of the normalized error change feature sequence is obtained by using an STL algorithm; a difference between the normalized error change feature sequence and the trend component is denoted as a fluctuation component; A mean value of current time point elements and a mean value of a plurality of elements before the current time point in the trend component are denoted as IMU error features; and a mean value of absolute values of all elements in the fluctuation component is denoted as UBW error features.

5. The highway tunnel information publishing system based on vehicle-mounted AR-HUD and Beidou navigation positioning according to claim 1, characterized in that, The specific steps of the current time updated vehicle positioning result obtained by re-fusing the IMU positioning result and the UWB positioning result according to the difference between the IMU error feature and the UWB error feature include the following: The IMU error feature is denoted as a1, the UWB error feature is denoted as a2, and a1 / (a2+0.1) is denoted as the IMU error correction strength; The fusion weight of the Kalman filtering algorithm in fusing the IMU positioning result and the UWB positioning result is updated by using the IMU error correction strength, and the updated fusion weight is negatively correlated with the IMU error correction strength; The IMU positioning result and the UWB positioning result are re-fused by using the updated fusion weight to obtain the current time updated vehicle positioning result. 6.The highway tunnel information publishing system based on the vehicle-mounted AR-HUD and Beidou navigation positioning according to claim 1, characterized in that, The specific steps of visualizing the tunnel information to the vehicle-mounted AR-HUD according to the current time updated vehicle positioning result include the following: The tunnel information includes a plurality of reference positions, and the reference positions include fire-fighting equipment positions, emergency escape passage entrance positions, fire occurrence positions, and roadblock positions; The difference between the reference position and the vehicle positioning result is denoted as a marked position, and a warning icon is displayed at the marked position in the AR scene of the vehicle-mounted AR-HUD.

7. The highway tunnel information publishing system based on the vehicle-mounted AR-HUD and the Beidou navigation positioning of claim 5, characterized in that, The specific formula of updating the fusion weight of the Kalman filtering algorithm in fusing the IMU positioning result and the UWB positioning result by using the IMU error correction strength includes the following: ； wherein represents the updated fusion weight, a represents the IMU error correction strength, and x represents the fusion weight before updating. 8.The highway tunnel information publishing system based on the vehicle-mounted AR-HUD and Beidou navigation positioning of claim 5, characterized in that, The updated vehicle positioning result wherein denotes the updated fusion weight, P1 denotes the UWB positioning result, and P2 denotes the IMU positioning result. 9.The highway tunnel information publishing system based on the vehicle-mounted AR-HUD and Beidou navigation positioning of claim 7, characterized in that, The fusion weight before the update ; wherein P1 represents a UWB positioning result, P2 represents an IMU positioning result, and PO represents a vehicle positioning result obtained by Kalman filtering algorithm fusion.

10. The highway tunnel information publishing system based on the vehicle-mounted AR-HUD and Beidou navigation positioning according to claim 4, characterized in that, The time of resetting the positioning process of the inertial navigation technology is re-marked as the initial time.

Images