Method, apparatus, system, and computer program product for locating a moving vehicle

Abstract

The application discloses a kind of sports vehicle positioning method, device, system and computer program product, belong to positioning technical field.The method comprises: obtaining the motion video information of vehicle;According to the current time camera observation value z of vehicle's motion video information t ;According to the prior training completed bidirectional long short time memory network model and the correction value x of previous time t‑1 , the current time priori estimation value is obtained t‑1 According to the covariance matrix P of the correction value x of previous time t‑1 Current time kalman gain K t ;Based on kalman filtering algorithm, according to the current time priori estimation value current time camera observation value z t And current time kalman gain K t The correction value x of current time is obtained t Initial time correction value x0 is the second coordinate information of vehicle in initial time based on ultra-wideband UWB positioning technology. Through the above technical scheme, the dynamic correction of UWB positioning data can be realized, the positioning accuracy is improved, the noise and error are significantly reduced, and the vehicle motion trajectory is more smooth.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of positioning, and particularly relates to a moving vehicle positioning method, device, system and computer program product. BACKGROUND

[0002] Test testing is a key link to improve the safety and reliability of automatic driving systems. Compared with non-quantitative typical scene testing, accurate measurement of vehicle motion parameters is particularly important for quantitative evaluation and technical iteration of automatic driving systems, and accurate real-time positioning is the basis for ensuring the accuracy of test data. Global Navigation Satellite System (GNSS) can meet the high-precision positioning requirements in open environments, but its reliability will decrease significantly in sheltered environments such as tunnels.

[0003] Ultra-Wideband (UWB) positioning technology has significant advantages in complex and sheltered environments due to its wide frequency band, good penetration ability and decimeter-level positioning accuracy. Through flexible base station deployment, it can achieve positioning signal coverage in specific areas and effectively solve the test problems in tunnels or complex urban scenes. However, due to the influence of multipath effect, Non-Line-of-Sight (NLOS) and system hardware performance, UWB still faces challenges in high-speed vehicle positioning. SUMMARY

[0004] To solve the above problems, the present application provides a moving vehicle positioning method, device, system and computer program product.

[0005] In one aspect, the present application provides a moving vehicle positioning method, which comprises:

[0006] obtaining motion video information of a vehicle; obtaining a current time camera observation value z t of the vehicle based on the motion video information of the vehicle; obtaining a previous time correction value x t and a covariance matrix P t-1 of the previous time correction value x t-1 ; obtaining a current time prior estimate value from a previously trained bidirectional long short-term memory network model and the previous time correction value x t-1 The bidirectional long short-term memory network model is used to predict the current time prior estimate value t-1 from the previous time correction value x t-1 The current time prior estimate value from the previous time correction value x t-1The covariance matrix P t-1 Get the current Kalman gain K t Based on the Kalman filter algorithm, according to the current moment prior estimate The current camera observation value z t and the current Kalman gain K t Get the current correction value x t ; Among them, the initial moment correction value x0 is the second coordinate information of the vehicle obtained based on ultra-wideband UWB positioning technology.

[0007] Another aspect of the present invention provides a moving vehicle positioning device, comprising:

[0008] The first acquisition module is used to acquire the motion video information of the vehicle; the observation value acquisition module is used to obtain the camera observation value z of the vehicle at the current moment according to the motion video information of the vehicle t , the camera observation value z t represents the first coordinate information of the vehicle obtained based on the motion video information; the second acquisition module is used to obtain the correction value x at the previous moment t-1 and the correction value x at the previous moment t-1 The covariance matrix P t-1 ; Prior estimation value acquisition module, used to obtain the prior estimation value based on the pre-trained bidirectional long short-term memory network model and the previous moment correction value x t-1 , get the current time prior estimate The bidirectional long short-term memory network model is used to correct the value x at the previous moment t-1 Predict the prior estimate of the current moment Kalman gain module is used to correct the value x according to the previous moment t-1 The covariance matrix P t-1 Get the current Kalman gain K t ; Correction value module, used for based on the Kalman filter algorithm, according to the current moment prior estimate The current camera observation value z t and the current Kalman gain K t Get the current correction value x t ; Among them, the initial moment correction value x0 is the second coordinate information of the vehicle obtained based on ultra-wideband UWB positioning technology.

[0009] Yet another aspect of the present invention provides an electronic device, comprising: a processor and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned moving vehicle positioning method.

[0010] On the other hand, the present invention provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the above-mentioned moving vehicle positioning method.

[0011] In yet another aspect, the present invention provides a computer program product comprising instructions, which, when run on a computer, enables the computer to execute the steps of the aforementioned moving vehicle positioning method and various possible implementations.

[0012] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0013] Get the vehicle's motion video information; get the vehicle's current camera observation value z based on the vehicle's motion video information t , camera observation value z t Represents the first coordinate information of the vehicle obtained based on the motion video information; obtains the correction value x at the previous moment t-1 and the correction value x at the previous moment t-1 The covariance matrix P t-1 ; According to the pre-trained bidirectional long short-term memory network model and the previous moment correction value x t-1 , get the current time prior estimate The bidirectional long short-term memory network model is used to correct the value x according to the previous moment t-1 Predict the current moment prior estimate According to the correction value x at the previous moment t-1 The covariance matrix P t-1 Get the current Kalman gain K t ; Based on the Kalman filter algorithm, according to the current moment prior estimate The camera observation value z at the current moment t and the current Kalman gain K t Get the current correction value x t; Wherein, the initial time correction value x0 is the second coordinate information of the vehicle obtained based on the ultra-wideband (UWB) positioning technology, so that the moving vehicle positioning method becomes a UWB positioning vehicle motion trajectory calibration method of fusion imaging technology. The method takes the vehicle dynamic position data (i.e. camera observation value) recognized by the high-speed camera as the reference value, and corrects and predicts the positioning data of UWB. At the same time, based on Kalman filtering algorithm, the bidirectional long short-term memory network is fused to learn the state transition function and related parameters of the moving vehicle. The motion state of the target is learned from the trajectory data of the moving vehicle, the moving vehicle state transition function is modeled as a learnable network, and then the Kalman filtering algorithm is used to dynamically correct the target state vector. The positioning accuracy of UWB can be effectively improved, the positioning noise and error can be reduced, and the reliability of UWB positioning technology in moving vehicle positioning and trajectory detection can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A flowchart of a moving vehicle positioning method provided by an embodiment of the application is shown in the figure.

[0015] Figure 2 A side view of a camera space model provided by an embodiment of the application is shown in the figure.

[0016] Figure 3 A top view of a camera space model provided by an embodiment of the application is shown in the figure.

[0017] Figure 4 A measurable line segment in the world coordinate system in a single vanishing point calibration model provided by an embodiment of the application is shown in the figure.

[0018] Figure 5 A corresponding line segment in the image coordinate system in a single vanishing point calibration model provided by an embodiment of the application is shown in the figure.

[0019] Figure 6 A schematic diagram of the overall framework of a vehicle motion trajectory calibration model provided by an embodiment of the application is shown in the figure.

[0020] Figure 7 A schematic diagram of the internal structure of a BILSTM-KF model provided by an embodiment of the application is shown in the figure.

[0021] Figure 8 A schematic diagram of the internal structure of a BILSTM-KF model provided by an embodiment of the application is shown in the figure.

[0022] Figure 9 A schematic diagram of a test site provided by an embodiment of the application is shown in the figure.

[0023] Figure 10 A detection process diagram of an AprilTag algorithm provided by an embodiment of the application is shown in the figure.

[0024] Figure 11 A state transition function training and prediction schematic diagram provided by the embodiment of the present application;

[0025] Figure 12 Another state transition function training and prediction schematic diagram provided by the embodiment of the present application;

[0026] Figure 13 A UWB positioning after correction and reference value comparison diagram provided by the embodiment of the present application for data sequence;

[0027] Figure 14 Another UWB positioning after correction and reference value comparison diagram provided by the embodiment of the present application;

[0028] Figure 15 Still another UWB positioning after correction and reference value comparison diagram provided by the embodiment of the present application;

[0029] Figure 16 Still another UWB positioning after correction and reference value comparison diagram provided by the embodiment of the present application;

[0030] Figure 17 A structure schematic diagram of a moving vehicle positioning device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0031] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0032] Referring to Figure 1 The embodiment of the present application provides a moving vehicle positioning method, which comprises the following steps:

[0033] Step 101: acquiring moving video information of a vehicle.

[0034] When the vehicle is driving on the road, the vehicle is photographed by a pre-set image acquisition device, and the moving video information of the vehicle can be acquired. The image acquisition device can be a video camera, a high-speed camera or the like. When the parameters of the high-speed camera meet the following requirements, the shooting effect is better: the camera resolution reaches 4096x 2048 (4K level resolution), and the highest reaches 500 frames per second (FPS, Frames Per Second) under full resolution. The position of the image acquisition device can be set in the middle of the gantry crossing the road, and the lens is directed downward to shoot.

[0035] In order to better identify, track and locate the vehicle, a visual positioning mark is arranged on the vehicle, such as the top of the vehicle. When applied, the visual positioning mark is captured in real time by using an image acquisition device, that is, the motion video information obtained contains the visual positioning mark, so that the state and position of the vehicle can be monitored. The visual positioning mark can be an AprilTag mark, which is a two-dimensional pattern, or an ARTag mark, or an ArUco mark, and the embodiment is not limited thereto, and the AprilTag mark is preferred.

[0036] In step 102, the current time camera observation value z of the vehicle is obtained according to the motion video information of the vehicle t The camera observation value z t represents the first coordinate information of the vehicle obtained based on the motion video information.

[0037] Since the visual positioning mark is arranged on the vehicle, the visual positioning mark in the motion video information of the vehicle is identified, detected and positioned, so that the current time camera measurement value of the vehicle, that is, the pixel coordinate value of the vehicle, can be obtained. Then, by using a single vanishing point camera calibration method, the current time camera measurement value is converted into the current time camera observation value z of the vehicle in the world coordinate t The current time camera observation value z t represents the first coordinate information of the vehicle at the current time t obtained based on the motion video information.

[0038] The algorithm for identifying the visual positioning mark can use the method in the prior art, and the embodiment is not limited thereto. When the visual positioning mark is an AprilTag mark, the algorithm for identifying the mark can be called an AprilTag detection algorithm. The algorithm is a visual mark system applied in machine vision tasks, and is commonly used in the fields of object recognition, target tracking, visual positioning, simultaneous localization and mapping (SLAM) and pose estimation. Due to its high-precision detection capability and robustness under different light and partial occlusion, the algorithm is suitable for detection and identification of moving objects. In other words, the algorithm has good robustness to changes in viewing angle and partial occlusion, and can reliably detect and track vehicles even in complex traffic environments, fast vehicle movement or poor light conditions.

[0039] Before the visual positioning mark is identified, the original video information collected by the image collection device is subjected to image preprocessing operation, and the embodiment does not limit the specific operation content of the preprocessing operation, which can include image graying, Gaussian filtering and binarization, etc. The identification process can be divided into straight line extraction, quadrilateral detection and decoding of the detected quadrilateral. First, the gradient direction and amplitude of the pixels are calculated, and the straight line in the image is fitted by weighted least squares; then, the quadrilateral composed of the straight line is detected by searching tree method; finally, the detected quadrilateral is subjected to homography transformation and parameter estimation, and the quadrilateral is decoded by the threshold value of space change.

[0040] The current time camera measurement value is converted into the current time camera observation value z of the vehicle in the world coordinate t Before that, the calibration of the image collection device needs to be performed. The embodiment does not specifically limit the calibration method, and the method in the prior art can be adopted.

[0041] The projection relationship between the two-dimensional image coordinates (u, v) and the three-dimensional world coordinates (x, y, z) is established: the schematic diagram of the camera space model is shown in Figures 2-3 The origin of the world coordinate system is the vertical projection point O w of the camera on the road plane, the X w axis is perpendicular to the road to the right, the Y w axis is parallel to the road, and the Z w axis is perpendicular to the ground plane upward. The origin of the camera coordinate system is at the camera position, the X c axis is parallel to the X w axis, the Z c axis points to the ground along the optical axis, and the Y c axis is perpendicular to the X C OZ c plane. The origin of the image coordinate system is at the image principal point r point, that is, the image coordinate system projection of the intersection of the Z c axis and the ground. Under this condition, the camera calibration parameters can be simplified as the camera focal length f, the pitch angle φ, the deflection angle θ and the camera height h.

[0042] The unfolded camera calibration projection relationship can be obtained as:

[0043]

[0044] In the formula, λ is a scale factor, and λ≠0.

[0045] Referring to Figures 4-5 , the vanishing point (Vanish point, VP) contains a large amount of information in the camera calibration, and the calibration parameters φ and θ can be solved according to the VP principle. The infinite point in the road direction is represented by the three-dimensional homogeneous coordinates as X1=[-tanθ 1 0 0] T, which projects to the image plane to form vanishing point (u1, v1), as shown in Figure 5 Thus, we can get:

[0046]

[0047] Besides VP, we can usually construct constraint relations to solve the camera height by known physical line segments in the scene. As shown in Figures 4-5 , let the length of the road dashed line (blue line segment) be l, and the world coordinates of the two end points of the dashed line be (*, y a , 0) and (*, y b , 0), and the corresponding image coordinates be (*, v a ) and (*, v b ), where * means that the value is irrelevant to the calculation; let the projection distance of the road width w Figure 4 (line segment on the left side of w) on the X w axis of the world coordinate system be Δx, and the corresponding pixel difference Δu on the image along the u axis. In Figure 4 and Figure 5 , (x3, y3), (x4, y4) represent any two points with the same y coordinate in the world coordinate system, and u3, u4 represent the coordinates of the two points on the u axis of the image plane, and the physical distance Δx of the two points in the world coordinate system and the pixel difference Δu corresponding to the u axis on the image are related, and h can be solved by the relationship. Δx has a geometric relationship with the road width, i.e. Δx = w * secθ.

[0048] From the related content, h can be indirectly derived from w and l as:

[0049]

[0050] In the formula, τ = (v a -v1)(v b -v1) / (v a -v b ), and δ is the horizontal length along the v = 0 line in the image.

[0051] For the same camera, the camera height obtained by calculating along the road direction or the line segment perpendicular to the road direction should be equal, so equations (9) and (10) are combined to get an equation about f:

[0052]

[0053] In the formula, k v = δτl / wv1.

[0054] When f is uniquely determined, according to formulas (2) and (3), φ and θ can be solved, and according to formula (4) or (5), h can be solved, all unknown calibration parameters are solved, and the calibration projection relationship between the two-dimensional image coordinates and the three-dimensional world coordinates can be established through formula (1).

[0055] In step 103, the previous time correction value x t-1 and the covariance matrix P t-1 of the previous time correction value x t-1 are obtained.

[0056] The initial time correction value x0 is the second coordinate information of the vehicle at the initial time obtained based on the ultra-wideband (UWB) positioning technology. The UWB positioning technology can collect vehicle motion coordinate information, and the embodiment does not limit the UWB positioning technology, which can be prior art. In application, a UWB positioning base station and a UWB positioning tag need to be arranged.

[0057] For specific implementation process of this step, refer to the related description of steps 104-106.

[0058] In step 104, the bidirectional long short-term memory network model trained in advance and the previous time correction value x t-1 are used to obtain the current time prior estimate value The bidirectional long short-term memory network model is used to predict the current time prior estimate value t-1 from the previous time correction value x

[0059] Although the Kalman filter (KF) algorithm is widely used in state estimation of dynamic systems, it has limited performance in nonlinear complex scenarios, and is often based on constant acceleration assumption, which limits the prediction accuracy. In this embodiment, the KF algorithm and the bidirectional long short-term memory (BILSTM) neural network are combined to calibrate and predict the vehicle target trajectory. Specifically, according to the optimal result x t-1 of the previous time, the prior estimate value of the current time is predicted. t The observation value z of the current time is used to correct the prior estimate value t of the current time, so as to obtain the optimal result x t of the current time.

[0060] The bidirectional long short-term memory (BILSTM) model will be described below. The model is used to predict the current time prior estimate value t-1 from the previous time correction value x The network model can be implemented based on a Keras framework. Specifically, the network model comprises, in sequence, an input layer, a first bidirectional long short-term memory (LSTM) layer, a first dropout layer, a second bidirectional LSTM layer, a second dropout layer, a full connection layer, and an output layer. The two bidirectional LSTM layers can be configured with 120 neurons. To balance the fitting ability, the calculation efficiency, and the control of overfitting of the model, a dropout layer with a dropout rate of 0.5 is attached after each bidirectional LSTM layer. A full connection layer containing 120 neurons is arranged before the output layer, and an activation function is used, which can be ReLU.

[0061] In the model initialization, all weights can be initialized by the Xavier method, and the bias value is initialized to 0. The initial motion state of the vehicle is set as the starting point x0, and the covariance matrix P0 of the starting state is set as the unit matrix I. The learning rate of the model is set to 0.001, the Adam optimizer is used for parameter optimization, and the training batch size is 16. The network model learns the parameters adaptively through the gradient descent method on the training set, which is made according to the continuous trajectory values collected by the UWB. The sequence data set of the n+1 step is predicted by the previous n steps, which is used to learn the process equation of the motion. After the model training is completed, the validation set is verified, and the model can be put into use.

[0062] obtained at the previous time (or the last time) is taken as the input of the bidirectional long short-term memory network model, and the current time prior estimate value t-1 is predicted.

[0063]

[0064] In the formula, the mapping function f(·) is fitted and learned by the BILSTM neural network, the powerful nonlinear fitting ability of the neural network is used to learn the complex motion trajectory of the vehicle target, so as to replace the state transition matrix in the KF algorithm; w t is the process noise.

[0065] In step 105, the covariance matrix P t-1 of the correction value x t-1 at the previous time is obtained. t .

[0066] Specifically, the covariance matrix P t-1 of the correction value x t-1 at the previous time is obtained, and the covariance matrix P t-1 of the correction value x t-1 at the previous time and the Jacobian matrix F t-1Get the current prior estimate The covariance matrix of According to the covariance matrix and the current camera observation value z t The observation matrix H t Get the current Kalman gain K t .

[0067] Among them, the covariance matrix of the prior estimate at the current moment can be obtained by the following formula,

[0068]

[0069] Where, F t-1 is the correction value x at the previous moment t-1 The Jacobian matrix of ; Q is the covariance matrix of process noise; is the prior estimate at the current moment The covariance matrix of P t-1 is the correction value x at the previous moment t-1 The covariance matrix of .

[0070]

[0071] Where: k t is the Kalman gain at the current moment; H is the observation matrix at the current moment, which is based on the camera observation value z at the current moment. t =R is the observation noise covariance matrix, which can be estimated from the variance of historical observation data. Q is the process noise covariance matrix, which can be manually adjusted according to the size of the observation noise.

[0072] Step 106: Based on the Kalman filter algorithm, the a priori estimate value at the current moment The camera observation value z at the current moment t and the current Kalman gain K t Get the current correction value x t .

[0073] Specifically, according to the current Kalman gain K t , the camera observation value z at the current moment t The observation matrix H t and the current prior estimate Get the intermediate value; based on the intermediate value and the current moment prior estimate Get the current correction value x t . Get the current correction value x t The formula is as follows:

[0074]

[0075] Where: z t is the observation value of the vehicle target at the current moment;

[0076] The previous moment correction value x in step 103 t-1 The method of obtaining the current moment correction value x can be found in t The methods of obtaining them are not described here one by one.

[0077] According to the identity matrix, the current Kalman gain K t , the camera observation value z at the current moment t The observation matrix H t , the current prior estimate The covariance matrix of Get the current correction value x t The covariance matrix P t . Get the current correction value x t The covariance matrix P t The formula is as follows:

[0078]

[0079] Where, P t is the optimal value x at the current moment t The covariance matrix of ; I is the identity matrix.

[0080] The previous moment correction value x in step 103 t-1 The covariance matrix P t-1 The method of obtaining the current moment correction value x can be found in t The covariance matrix P t The methods of obtaining them are not described here one by one.

[0081] This embodiment combines the KF algorithm with the BILSTM (Bidirectional Long Short-Term Memory) neural network, which is called the BILSTM-KF model. The vehicle motion trajectory calibration model is constructed by the BILSTM-KF model. The overall framework diagram is shown in the figure below. Figure 6 As shown, the optimal value x of the model at the previous moment (t-1) is t-1 and the observation value z at the current time t t As input, get the optimal value x at the current moment t , and iteratively update the target value vector in turn. The internal structure diagram of the BILSTM-KF model is as follows Figure 7 As shown in Figure 2, the model first uses a BILSTM neural network to learn the state transition function f(·), and then uses the prior estimate x output by f(·) t-1 and the observed value z tInput the KF algorithm to finally solve the optimal correction value at the current moment. Figure 6 In the figure, x0 represents the second coordinate information of the vehicle at the initial moment obtained based on ultra-wideband UWB positioning technology, z1 represents the first coordinate information of the vehicle at time 1 obtained based on motion video information, x1 represents the correction value at time 1, and z t represents the first coordinate information of the vehicle at time t obtained based on the motion video information, x t Indicates the correction value at time t.

[0082] See also Figure 8 From the perspective of algorithm development, in each time step t, the optimal value x of the previous moment (i.e., moment t-1) is t-1 Enter the BILSTM neural network model as input to obtain the prior estimate of the current time t Then use the covariance matrix P at time t-1 t-1 , Q, calculate the prior covariance matrix at the current time t according to formula (8) Then use R, according to formula (9) we get the Kalman gain K t ; Then use the output of the BILSTM neural network Combined K t and the input value z at the current time t t , according to formula (10), the optimal value x at the final time t is obtained t ; while using and K t According to formula (11), the optimal covariance matrix P at the final time t is obtained t , and update iteratively in sequence.

[0083] It should be noted that the process of obtaining the prior estimate value at the current moment and the covariance matrix of the prior estimate value at the current moment can be called the prediction step; the process of obtaining the Kalman gain at the current moment and the optimal value x at the current moment can be called the prediction step. t , the process step of obtaining the covariance matrix of the optimal value at the current moment is called the update step.

[0084] The embodiment proposes a moving vehicle positioning method, and a UWB positioning vehicle motion trajectory calibration model integrating high-speed imaging technology is used in the implementation process. The high-speed camera captures the dynamic position of the vehicle, provides high spatio-temporal resolution and reliable reference position information, and can be used to calibrate and compensate the positioning data of the UWB, so as to improve the overall positioning accuracy of the UWB positioning device in a high dynamic environment. To realize the above calibration model, the calibration model is a UWB positioning trajectory correction and prediction algorithm based on Kalman filtering and bidirectional long short-term memory network (BILSTM-KF). By combining the KF algorithm and the BILSTM network, the UWB positioning data can be corrected, and the positioning noise and error can be reduced.

[0085] The following experiments and result analysis are performed on the positioning method of the moving vehicle provided in the embodiment:

[0086] The experiment scene is built as follows:

[0087] The experiment site is set up on a 700-meter-long test road of the highway traffic test field, and 24 UWB positioning base stations (or UWB base stations) are arranged on both sides of the road. The first 6 positioning base stations are spaced 33 meters apart, and the remaining positioning base stations are spaced 20 meters apart, with a total coverage range of about 240 meters. A high-speed camera is installed in the middle of the gantry at the starting point, which can be a 5F08-M type. The camera resolution is 4096x2048 (4K level resolution), and the maximum is 500 frames per second (FPS) at full resolution. The experimental site is shown in the following figure Figure 9

[0088] First, a camera calibration model is established, then the AprilTag algorithm is used to detect the vehicle position of the camera output video, and the UWB vehicle positioning data is synchronously collected, and finally the vehicle motion trajectory is corrected by using the two kinds of synchronous positioning data and the calibration model and method of the embodiment. The experiment mainly includes the following three aspects of verification: (1) verifying the accuracy of the camera calibration model; (2) verifying the recognition success rate of the AprilTag detection algorithm for the target vehicle; (3) verifying the accuracy of the moving vehicle positioning device of the embodiment, i.e. the accuracy of the vehicle motion trajectory correction.

[0089] Explanation of camera calibration results:

[0090] To verify the accuracy of the camera calibration method, multiple verification points are selected in all directions within the camera field of view to comprehensively evaluate the calibration method. In addition, to obtain the accurate true value coordinates of the verification points, a metrologically calibrated total station is used to measure the coordinates of the verification points, as shown in the following table (x, y). At the same time, the selected verification points are converted from image coordinates to world coordinates by using the above established camera calibration projection relationship, and the corresponding calibration results are shown in Table 1.

[0091] ​Table 1 Camera calibration results and errors

[0092] Validation point (x, y) / m (X, Y) / m RMSE / m 1 (-4.6877，13.7562) (-4.6252，13.8095) 0.0821 2 (-4.6871，17.7386) (-4.6701，17.7858) 0.0502 3 (-4.6956，21.7294) (-4.7343，21.8129) 0.0920 4 (-4.6931，25.7244) (-4.7514，25.7704) 0.0743 5 (-4.6850，29.7122) (-4.6587，29.8162) 0.1073 6 (-4.6890，33.6920) (-4.6748，33.7443) 0.0542 7 (-4.6816，37.6813) (-4.7070，37.7771) 0.0991 8 (-4.6688，41.6823) (-4.7206，41.6382) 0.0680 9 (3.3971，15.1215) (3.46474，15.0036) 0.1359 10 (3.4010，19.0658) (3.43212，18.9978) 0.0748 11 (3.4025，25.1088) (3.37491，25.0121) 0.1006 12 (1.6378，29.4158) (1.7041，29.3757) 0.0775 13 (1.6285，35.3751) (1.6458，35.3460) 0.0339 14 (1.6416，41.3380) (1.5990，41.2298) 0.1163 15 (3.4027，28.9251) (3.3542，29.0009) 0.0900

[0093] The above table shows the verification point coordinates (x, y) and the calibration coordinates (X, Y). To evaluate the calibration accuracy, the root mean square error (RMSE) is used as the main evaluation index. The calculation formula of RMSE is as follows:

[0094]

[0095] Through equation (12), the RMSE of each verification point can be obtained, as shown in the above table. And the average RMSE of all verification points is 0.0837, which can meet the demand of the calibration model.

[0096] Explanation of AprilTag detection results:

[0097] The original video data used in the experiment was obtained by a 5F08-M high-speed camera. The sampling frequency of the camera was set to 200 Hz, and the image resolution was 4096x2048 pixels. Four segments of vehicle motion videos at different speeds along a straight line and a curve were used for vehicle recognition and detection. The total number of frames of the four videos was 640 frames, 540 frames, 907 frames and 1272 frames, respectively.

[0098] During the video recognition process, 585 frames were successfully recognized in the first video, corresponding to a detection accuracy of 91.40%; 501 frames were successfully recognized in the second video, with a detection accuracy of 92.77%; 816 frames were successfully recognized in the third video, with a detection accuracy of 89.96%; and 1216 frames were successfully recognized in the fourth video, with a detection accuracy of 95.59%. Figure 10 The vehicle detection process of the AprilTag algorithm applied to two of the videos is shown in the above table.

[0099] Explanation of vehicle motion trajectory correction result analysis:

[0100] To verify the effectiveness of the calibration model proposed in this embodiment in optimizing the positioning accuracy of UWB vehicle motion trajectories, high-speed camera and UWB positioning were used to collect vehicle positioning data at different speeds along a straight line and a curve, and the obtained motion trajectories were analyzed.

[0101] Since the KF algorithm is used for correction and prediction, the state transition function needs to be set in advance to obtain the prior estimate However, the vehicle motion is affected by factors such as the environment, making it difficult to obtain the motion state function or the function is not accurate, which affects the final result. Therefore, this embodiment uses a bidirectional long short-term memory network to learn the target state transition function and related parameters of the sequence data, thereby obtaining the prior estimate required for KF algorithm correction and prediction The training and prediction process of the moving target state transfer function is as follows: Figures 11-12 As shown, Figure 11 and Figure 12 They are for different data columns. The purple, gray, and blue balls in the figure represent the prior estimates obtained by training the BILSTM network. ; The green, red and yellow balls represent the To show the coordinates more clearly, the x and y coordinates are shown separately.

[0102] In the trajectory correction and prediction model of this embodiment, the BILSTM-KF algorithm is used for data fusion and correction. In the specific operation, the UWB positioning estimate at time t+1 is converted into As the prior value of the Kalman filter, and based on the camera observation value at the current time t, the UWB positioning data at time t+1 is predicted and corrected. Figures 13-16 The corrected UWB positioning data, UWB positioning raw data, and the correction reference value observed by the camera are displayed for different data columns. Figures 13-14 The data point sequence in represents the trajectory of the vehicle traveling along a straight line at different speeds. Figures 15-16 The sequence of data points in the middle represents the trajectory of a vehicle traveling along a curve at different speeds. The green markers indicate the x and y coordinates of the trajectory after UWB positioning correction; the blue markers represent the x and y coordinates of the UWB correction baseline obtained through camera synchronization; and the black markers show the raw coordinate data of UWB positioning.

[0103] from Figures 13-16 It can be clearly seen that the stability of the UWB positioning data after correction has been significantly improved. Compared with before correction, the motion trajectory is smoother and the fluctuation is significantly reduced. This shows that the method of integrating high-speed camera and UWB positioning using the UWB calibration model and BILSTM-KF algorithm of this embodiment can effectively reduce the noise and error in UWB positioning data, thereby improving positioning accuracy and the smoothness of the vehicle motion trajectory. Figures 13-16 The data of the four trajectories are calculated using formula (12). The RMSE between each corrected UWB positioning data and the reference value are 0.0178m, 0.0127m, 0.0110m and 0.0171m respectively. The RMSE results of each trajectory after correction are shown in Table 2 below.

[0104] Table 2 Error between UWB positioning and camera reference value after correction

[0105] Motion trajectory RMSEx(m) RMSEy(m) RMSE(m) 1 0.0128 0.0227 0.0178 2 0.0100 0.0154 0.0127 3 0.0070 0.0149 0.0110 4 0.0169 0.0173 0.0171

[0106] In addition, further experimental analysis shows that the calibration model proposed in this embodiment exhibits good robustness and adaptability under different motion patterns and environmental conditions. Through repeated experimental verification, the method of this embodiment can maintain high positioning accuracy in complex environments, indicating its wide applicability and potential advantages.

[0107] This embodiment proposes a vehicle motion trajectory calibration model that fuses high-speed image technology with UWB positioning, and introduces an improved Kalman filter algorithm to fuse bidirectional long short-term memory networks for vehicle motion trajectory correction and prediction. By using the dynamic position data of the vehicle recognized by the high-speed camera as a reference, this embodiment successfully realizes dynamic correction of UWB positioning data, improves positioning accuracy, significantly reduces noise and errors, and makes the vehicle motion trajectory smoother. Experimental results verify the effectiveness of the model and algorithm, showing its potential in high-precision positioning.

[0108] This research has important theoretical and practical significance. Theoretically, it expands the application range of traditional Kalman filter algorithms, enabling them to more effectively handle nonlinear motion models and thus improve the accuracy of UWB positioning. Practically, the calibration model provided in this research can be applied to vehicle positioning systems in occluded environments, and has broad application prospects in fields such as intelligent transportation systems, autonomous driving, and unmanned driving.

[0109] The UWB positioning vehicle trajectory calibration model and BILSTM-KF algorithm proposed in this embodiment provide new ideas and methods for the development of high-precision positioning technology. Future work can verify the effectiveness of this method in more extensive application scenarios and explore more efficient calculation methods to meet the real-time requirements of practical applications. At the same time, this technology can be combined with other advanced positioning and perception technologies to provide a more comprehensive solution for precise positioning and trajectory prediction of vehicles.

[0110] Referring to Figure 17 , the embodiment of the present application provides a moving vehicle positioning device for executing the moving vehicle positioning method provided by the above-mentioned embodiments, which comprises a first acquisition module 201, an observation value obtaining module 202, a second acquisition module 203, a priori estimate value obtaining module 204, Kalman gain obtaining module 205 and correction value obtaining module 206.

[0111] The first acquisition module 201 is used to acquire the motion video information of the vehicle. The observation value obtaining module 202 is used to obtain the camera observation value z t of the vehicle at the current time based on the motion video information of the vehicle. The camera observation value z t represents the coordinate information of the vehicle obtained based on the motion video information. The second acquisition module 203 is used to acquire the correction value x t-1and the correction value x at the previous moment t-1 The covariance matrix P t-1 The prior estimation value obtaining module 204 is used to obtain the prior estimation value based on the pre-trained bidirectional long short-term memory network model and the previous moment correction value x t-1 , get the current time prior estimate The bidirectional long short-term memory network model is used to correct the value x according to the previous moment t-1 Predict the current moment prior estimate The Kalman gain obtaining module 205 is used to obtain the Kalman gain according to the correction value x at the previous moment. t-1 The covariance matrix P t-1 Get the current Kalman gain K t The correction value obtaining module 206 is used to obtain the current prior estimation value based on the Kalman filter algorithm. The camera observation value z at the current moment t and the current Kalman gain K t Get the current correction value x t , where the initial correction value x0 is the coordinate information of the vehicle at the initial moment obtained based on ultra-wideband UWB positioning technology.

[0112] Optionally, the Kalman gain obtaining module 205 is specifically configured to:

[0113] Get the correction value x at the previous moment t-1 The covariance matrix P t-1 ; According to the correction value x at the previous moment t-1 The covariance matrix P t-1 and the Jacobian matrix F t-1 Get the current time prior estimate The covariance matrix of According to the covariance matrix and the camera observation value z at the current moment t The observation matrix H t Get the current Kalman gain K t .

[0114] Optionally, the correction value obtaining module 206 is specifically configured to:

[0115] According to the current Kalman gain K t , the camera observation value z at the current moment t The observation matrix H t and the current prior estimate Get the intermediate value; based on the intermediate value and the current moment prior estimate Get the current correction value x t .

[0116] Optionally, the second acquisition module 203 acquires the correction value x at the previous moment t-1 The covariance matrix P t-1 When, specifically used for:

[0117] According to the identity matrix, the Kalman gain K at the previous moment t-1 , the camera observation value z at the previous moment t-1 The observation matrix H t-1 , the prior estimate at the previous moment The covariance matrix of Get the correction value x at the previous moment t-1 The covariance matrix P t-1 .

[0118] Optionally, the vehicle is equipped with a visual positioning marker, and the observation value obtaining module 202 is specifically used to:

[0119] The visual positioning markers in the vehicle's motion video information are identified, detected and positioned to obtain the vehicle's current moment camera measurement value; the current moment camera measurement value is converted into the vehicle's current moment camera observation value z in world coordinates using the single vanishing point camera calibration method. t .

[0120] Optionally, the visual positioning mark is an AprilTag mark.

[0121] Optionally, the bidirectional long short-term memory network model includes: an input layer, a first bidirectional long short-term memory network layer, a first random inactivation layer, a second bidirectional long short-term memory network layer, a second random inactivation layer, a fully connected layer and an output layer connected in sequence.

[0122] It should be noted that the aforementioned embodiments of the moving vehicle positioning device are merely illustrative of the division of the aforementioned functional modules when locating a moving vehicle. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, i.e., the internal structure of the device can be divided into different functional modules to perform all or part of the aforementioned functions. Furthermore, the moving vehicle positioning device and the moving vehicle positioning method embodiments of the aforementioned embodiments share the same concept. The specific implementation process is detailed in the method embodiments and will not be further elaborated here.

[0123] An embodiment of the present invention provides a moving vehicle positioning system, which includes: a vehicle motion measurement subsystem and a moving vehicle positioning device.

[0124] The vehicle motion measurement subsystem has a visual positioning marker, a camera and a UWB positioning device, the visual positioning marker is installed on the vehicle, the camera is erected above the road on which the vehicle travels, and the UWB positioning device is used to position the vehicle based on the UWB positioning technology to obtain coordinate information.

[0125] That is, the system is mainly divided into a vehicle motion measurement system and a UWB calibration system, as shown in Figure 1 The measurement system is composed of a high-speed camera and a UWB positioning base station, and is used to synchronously collect the motion state information of the vehicle, wherein the UWB mainly collects the motion coordinate information of the vehicle, and the high-speed camera collects the motion video information of the vehicle. The resolution of the camera is 4096x2048 (4K level resolution), and the maximum is 500 frames per second (FPS) under full resolution. The main function of the UWB correction system is to correct the motion coordinate information output by the UWB by using the proposed BILSTM-KF algorithm. At the same time, after the collected vehicle motion video information is recognized and positioned by the AprilTag detection algorithm, the single vanishing point camera calibration method is used to convert the vehicle coordinate information into the world coordinate information.

[0126] An embodiment of the present application provides an electronic device, which comprises a memory and a processor. The processor is connected with the memory and is configured to execute the above-mentioned motion vehicle positioning method based on instructions stored in the memory. The number of processors can be one or more, and the processor can be single-core or multi-core. The memory can include non-permanent memory in a computer readable medium, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip. The memory can be an example of the following computer readable medium.

[0127] An embodiment of the present application provides a computer readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the above-mentioned moving vehicle positioning method. The computer readable storage medium includes permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc-read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0128] It can be understood by those skilled in the art that the present application can be implemented by other embodiments without departing from the spirit or essential characteristics thereof. Therefore, the above-mentioned disclosed embodiments are only examples and are not the only ones. All changes within the scope of the present application or within the scope equivalent to the present application are included in the present application.

Claims

1. A method of locating a moving vehicle, characterized by, The method comprises: acquiring motion video information of a vehicle; obtaining a camera observation value z of the vehicle at a current moment according to motion video information of the vehicle t , the camera observation value z t represents first coordinate information of the vehicle obtained based on the motion video information; Obtaining a previous time correction value x t-1 and a previous time correction value x t-1 covariance matrix P t-1 ; According to a pre-trained bidirectional long short-term memory network model and a previous time correction value x t-1 , a current time prior estimation value is obtained The bidirectional long short-term memory network model is used to predict the current time prior estimation value t-1 according to the previous time correction value x According to the correction value x of the previous time t-1 covariance matrix P of the previous time t-1 Obtain the Kalman gain K of the current time t ; based on a Kalman filter algorithm, according to the current time prior estimate value The current time camera observation value z t And the current time Kalman gain K t Get the current time correction value x t ; wherein the initial time correction value x0 is second coordinate information of the vehicle at an initial time obtained based on an ultra-wideband (UWB) positioning technology.

2. The method of claim 1, wherein, The current time Kalman gain K is obtained according to the covariance matrix of the previous time correction value x t-1 t comprises:​ Obtaining a correction value x at a previous time t-1 Covariance matrix P of the correction value x t-1 ; According to the correction value x of the previous time t-1 Covariance matrix P of the previous time t-1 and Jacobian matrix F t-1 Get the prior estimate value of the current time Covariance matrix According to the covariance matrix P t - and the current time camera observation z t observation matrix H t get the current time Kalman gain K t .

3. The method of claim 1, wherein, The Kalman filtering algorithm is based on the current time priori estimation value The current time camera observation value z t And the current time Kalman gain K t Get the current time correction value x t Comprise: According to the current time Kalman gain K t , the current time camera observation value z t , the observation matrix H t of the current time prior estimate value , an intermediate value is obtained; According to the intermediate value and the current time prior estimate value x t - Obtain the current time correction value x t .

4. The method of claim 2, wherein, The acquisition of the previous time correction value x t-1 Covariance matrix P t-1 Comprise: According to a unit matrix, a Kalman gain K t-1 of a previous time, an observation matrix H t-1 of a camera observation z t-1 of the previous time, a covariance matrix Q of a priori estimation value of the previous time, a covariance matrix P t-1 of a correction value x t-1 of the previous time is obtained.

5. The method of claim 1, wherein, The vehicle is equipped with a visual positioning marker, and the camera observation value z of the vehicle at the current moment is obtained based on the motion video information of the vehicle. t include: detecting and positioning a visual positioning marker in the motion video information of the vehicle to obtain a current time camera measurement value of the vehicle. The current time camera measurement is converted into a current time camera observation of the vehicle in world coordinates z using a single vanishing point camera calibration method t .

6. The method of claim 5, wherein, The visual positioning marker is an AprilTag marker.

7. The method of claim 1, wherein, The bidirectional long short-term memory network model comprises, in sequence, an input layer, a first bidirectional long short-term memory network layer, a first random inactivation layer, a second bidirectional long short-term memory network layer, a second random inactivation layer, a full connection layer, and an output layer.

8. A sports vehicle positioning device, characterized by, The device comprises: a first acquisition module configured to acquire motion video information of a vehicle; An observation value obtaining module is configured to obtain a camera observation value z of the vehicle at a current time according to motion video information of the vehicle t , wherein the camera observation value z t represents first coordinate information of the vehicle at an initial time obtained based on the motion video information. A second obtaining module is configured to obtain a correction value x at a previous moment t-1 and a covariance matrix P of the correction value x at the previous moment t-1 and the correction value x at the previous moment t-1 ​ The module for obtaining the prior estimation value is used to obtain the prior estimation value based on the pre-trained bidirectional long short-term memory network model and the previous moment correction value x t-1 , get the current time prior estimate The bidirectional long short-term memory network model is used to correct the value x at the previous moment t-1 Predict the prior estimate of the current moment Kalman gain module is used to correct the value x according to the previous moment t-1 The covariance matrix P t-1 Get the current Kalman gain K t ; a correction value obtaining module, configured to obtain a current time correction value x based on a Kalman filtering algorithm according to the current time prior estimation value x the current time camera observation value z t and the current time Kalman gain K t obtain the current time correction value x t ; wherein the initial time correction value x0 is second coordinate information of the vehicle at an initial time obtained based on an ultra-wideband (UWB) positioning technology.

9. A sports vehicle positioning system, characterized by The system comprises: a vehicle motion measurement subsystem having a visual positioning marker, a camera, and a UWB positioning device, the visual positioning marker being installed on a vehicle, the camera being arranged above a road on which the vehicle travels, and the UWB positioning device being configured to position the vehicle based on a UWB positioning technology to obtain second coordinate information; a moving vehicle positioning device as claimed in claim 8.

10. A computer program product comprising instructions, characterized in that, When the computer program product is run on a computer, the computer is caused to perform the method as claimed in any one of claims 1-7.

Images