Vision and inertia combined positioning method

Abstract

The invention relates to a vision and inertia combined positioning method, and relates to the technical field of positioning. According to the vision and inertia combined positioning method, the problems that a single data source is easily influenced by an external environment and the data is unstable in a traditional positioning system are solved by combining vision and inertia data, particularly in an environment with a poor GPS signal, continuous and accurate positioning can be effectively provided, and the positioning accuracy is improved through dynamic data fusion and error correction. The adaptive capacity of the system to a complex environment is enhanced, the smoothness and accuracy of a positioning result are ensured, the reliability of the navigation system and the safety of vehicle driving are improved, the vehicle position can be updated in real time, and potential risks caused by positioning errors are reduced, so that more stable and reliable navigation support is provided for the vehicle; the operation risk is reduced, the use effect of the system under adverse conditions is enhanced, and the overall operation efficiency and safety are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of positioning technology, and in particular to a vision and inertia combined positioning method. Background Art

[0002] In the field of modern computer vision and image processing, target positioning is a key and widely used important research topic. Traditional target positioning methods mainly rely on multi-viewing vision, such as stereo vision or lidar sensors. However, these methods usually have disadvantages such as expensive equipment, complex calculations, and strong dependence on the environment. In many practical applications, monocular vision systems have become a very attractive choice due to their low cost, easy deployment and maintenance. Although monocular vision systems have the above advantages, they face many challenges in target positioning. For example, monocular vision systems cannot directly obtain depth information in the scene, which makes it difficult to accurately determine the three-dimensional position of an object based on a single frame image. The target object in the monocular image may be affected by multiple factors such as lighting, occlusion, and background complexity, which increases the difficulty of feature extraction and recognition. Traditional methods based on image processing and feature matching often perform poorly in complex environments and are difficult to meet the requirements of high-precision positioning. Summary of the invention

[0003] The present invention aims at the technical problems existing in the prior art and provides a visual and inertial joint positioning method, which solves the problem that when using a single sensor system (such as relying only on visual positioning or inertial measurement), due to the inherent limitations of the technology, especially in high dynamics, low visibility or strong interference from environmental factors, traditional methods are difficult to maintain positioning accuracy and stability, resulting in positioning failure or error accumulation, thereby reducing the overall performance and reliability of the system. Secondly, traditional technologies often have slow response speeds when dealing with complex or dynamic environments and cannot effectively adapt to rapidly changing environmental conditions, which can easily lead to reduced navigation accuracy in practical applications and affect the operating efficiency and safety of mobile devices. In addition, the prior art often lacks effective verification and fusion processing of the consistency between visual and inertial data, which can easily lead to difficulty in providing reliable positioning information when the visual system is obstructed by environmental factors or the accumulated error of the inertial system is too large, thereby affecting the operating performance and decision-making quality of the entire system.

[0004] The technical solution of the present invention to solve the above technical problems is as follows: a visual and inertial joint positioning method, comprising the following steps: continuously acquiring a plurality of frames of image data and inertial data at a plurality of time points before a vehicle is about to enter a tunnel; identifying and analyzing a plurality of frames of image data before a vehicle is about to enter a tunnel based on Canny edge detection and Hough transform, obtaining driving lane line information before the vehicle is about to enter the tunnel, and obtaining corresponding driving lane line data in the tunnel; comprehensively analyzing the inertial data at a plurality of time points before the vehicle is about to enter the tunnel, obtaining an average vehicle driving speed and an average vehicle acceleration; predicting the vehicle predicted coordinate information of the vehicle at each future time point in the tunnel based on the corresponding driving lane line data in the tunnel and the average vehicle driving speed and the average vehicle acceleration.

[0005] Furthermore, each frame of image data includes the pixel value of each pixel point in the image in front of the vehicle, the corresponding driving lane line data in the tunnel is specifically the longitude and latitude coordinates of each position point of the corresponding driving lane line in the tunnel, and the inertial data is specifically the vehicle driving speed value.

[0006] Furthermore, the specific steps for obtaining the lane line information before the vehicle is about to enter the tunnel are as follows: reading several frames of image data before the vehicle is about to enter the tunnel and performing preprocessing; calculating the gradients of the several frames of image data before the vehicle is about to enter the tunnel after preprocessing in the horizontal and vertical directions, respectively, to obtain the horizontal gradient value and the vertical gradient value of each pixel point of each frame image; combining the horizontal gradient value and the vertical gradient value of each pixel point of each frame image for comprehensive analysis, respectively, to obtain the gradient amplitude value and direction value of each pixel point of each frame image; performing edge analysis on each frame image based on the gradient amplitude value and direction value of each pixel point, respectively, to obtain a binarized image of each frame image; performing Hough transform processing on the binarized image of each frame image, to obtain the lane line information before the vehicle is about to enter the tunnel.

[0007] Furthermore, preprocessing is performed on a plurality of frames of image data before the vehicle is about to enter the tunnel, wherein the preprocessing includes performing grayscale processing and Gaussian filtering processing on the plurality of frames of image data before the vehicle is about to enter the tunnel.

[0008] Furthermore, the specific formula for calculating the gradient amplitude value and direction value of each pixel point in each frame of image is as follows: Among them, G ij is the gradient amplitude value of the jth pixel point of the i-th frame image, Sp ij is the horizontal gradient value of the jth pixel of the i-th frame image, Cz ij is the vertical gradient value of the jth pixel of the i-th frame image, θ ijis the value of the jth pixel in the i-th frame image, i=1,2,3,…,i0, i0 is the image frame number, j=1,2,3,…,j0, j0 is the number of pixels in the image.

[0009] Furthermore, the specific steps of performing Hough transform processing on the binary image of each frame are as follows: determining the step size of the distance resolution and the angle resolution for each frame; initializing the accumulator for each frame, and performing voting processing on each edge pixel; identifying the straight line of each frame from the accumulator, and performing comparative analysis to obtain the driving lane line information before the vehicle enters the tunnel.

[0010] Furthermore, the specific steps of predicting the vehicle's predicted coordinate information at each future time point in the tunnel are as follows: read the vehicle's speed value at each time point before the vehicle is about to enter the tunnel and perform a comprehensive analysis to obtain the vehicle's average speed and the vehicle's acceleration; use the vehicle's average speed as the initial speed value when the vehicle enters the tunnel; based on the vehicle's initial speed value when the vehicle enters the tunnel and the vehicle's average acceleration, analyze and predict the predicted driving distance value of the vehicle at each future time point in the tunnel, and analyze the vehicle's predicted coordinate information based on the latitude and longitude coordinates of each position point of the corresponding driving lane line in the tunnel.

[0011] Furthermore, the specific steps for obtaining the mean vehicle speed and the mean vehicle acceleration are as follows: reading the vehicle speed value at each time point before the vehicle is about to enter the tunnel, and obtaining the mean vehicle speed value by combining the mean method; performing acceleration analysis on the vehicle speed value at each time point before the vehicle is about to enter the tunnel, and obtaining several groups of vehicle acceleration values; performing mean analysis on several groups of vehicle acceleration values ​​by combining the moving exponential average method, and obtaining the mean vehicle acceleration value; wherein, the specific formulas for calculating the mean vehicle speed and the mean vehicle acceleration value are as follows: Where V′ is the mean vehicle speed, V q is the vehicle speed at the qth time point before the vehicle enters the tunnel, q = 1, 2, 3, ..., q0, q0 is the number of time points, A′ is the mean vehicle acceleration, a w is the acceleration value of the wth group of vehicles, α w1 is the weighting coefficient of the acceleration value of the wth group of vehicles, a w+1 is the acceleration value of the w+1th group of vehicles, α w2 is the weighting coefficient of the acceleration value of the w+1th group of vehicles, α w1 +α w2 =1, w=1,2,3,…,w0, w0 is the number of vehicle acceleration value groups.

[0012] Furthermore, the specific formula for predicting the actual driving distance of a vehicle in a tunnel at a certain point in the future is as follows: Among them, X q′0 is the predicted driving distance of the vehicle at the q′0th time point in the future in the tunnel, V′ is the mean driving speed of the vehicle, that is, the initial speed value when the vehicle enters the tunnel, A′ is the mean acceleration of the vehicle, T q′0 is the time value of the vehicle at the future q′0th time point in the tunnel, and β is the error factor.

[0013] Furthermore, the specific calculation process of the error factor is as follows: obtain the actual driving distance value at each time point before the vehicle is about to enter the tunnel; analyze the estimated driving distance value at each time point before the vehicle is about to enter the tunnel based on the mean vehicle acceleration and the vehicle driving speed value at each time point before the vehicle is about to enter the tunnel; analyze the distance error value between the actual driving distance value at each time point before the vehicle is about to enter the tunnel and the estimated driving distance value, and perform mean analysis to obtain the error factor.

[0014] The beneficial effects of the present invention are as follows: by combining visual and inertial data, the problem of single data source being susceptible to external environmental influences and data instability in traditional positioning systems is solved, especially in environments with poor GPS signals (such as tunnels), continuous and accurate positioning can be effectively provided. In addition, through dynamic data fusion and error correction, the system's adaptability to complex environments is enhanced, the smoothness and accuracy of positioning results are guaranteed, and the reliability of the navigation system and the safety of vehicle driving are significantly improved. The system can update the vehicle position in real time, reduce potential risks caused by positioning errors, and optimize the vehicle's driving strategy in tunnels and other complex sections, thereby providing the vehicle with more stable and reliable navigation support, which not only reduces operational risks, but also enhances the use effect of the system under adverse conditions, greatly improving the overall operating efficiency and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The present invention is a flow chart of a vision and inertial joint positioning method.

[0016] Figure 2 The present invention is a flowchart of the specific steps of obtaining the lane line information of a vehicle before entering a tunnel in a vision and inertial joint positioning method. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0018] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0019] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0020] The overall idea of ​​the problem in the embodiment of this application is as follows:

[0021] First, the image data and inertial data before the vehicle enters the tunnel are continuously collected, which provides basic information for subsequent analysis. The collected image data are preprocessed (including grayscale conversion and Gaussian filtering), and then the horizontal and vertical gradients of the image are calculated to obtain the gradient amplitude and direction. This information is used to identify the lane line through Canny edge detection and Hough transform to provide a reference for the vehicle's navigation in the tunnel. The inertial data is analyzed to obtain the vehicle's average speed and acceleration. These statistical data reflect the vehicle's driving status and provide parameters for predicting the vehicle's future position. The lane line data and inertial data (average speed and acceleration) are combined to predict the vehicle's position coordinates at each time point in the tunnel, thereby improving the accuracy and real-time performance of navigation. By comparing the actual driving distance with the predicted driving distance, the error factor is calculated to adjust and optimize the system prediction to ensure higher positioning accuracy.

[0022] See also Figure 1The embodiment of the present invention provides a technical solution: a vision and inertial joint positioning method, comprising the following steps: continuously acquiring a number of frames of image data before the vehicle is about to enter the tunnel (assuming that the camera captures at a speed of 30 frames / second, about 900 frames of images 30 seconds before entering the tunnel can be collected. These images contain important information about lanes, roadside signs and other important navigation marks, which can be used to track the movement and position of the vehicle) and inertial data at a number of time points; based on Canny edge detection and Hough transform, identifying and analyzing a number of frames of image data before the vehicle is about to enter the tunnel, obtaining the driving lane line information before the vehicle is about to enter the tunnel, and obtaining the corresponding driving lane line data in the tunnel; comprehensively analyzing the inertial data at a number of time points before the vehicle is about to enter the tunnel, obtaining the average vehicle speed and the average vehicle acceleration; based on the corresponding driving lane line data in the tunnel and the average vehicle speed and the average vehicle acceleration, predicting the vehicle predicted coordinate information of the vehicle at each future time point in the tunnel.

[0023] Each frame of image data includes the pixel value of each pixel point in the image in front of the vehicle, the corresponding driving lane line data in the tunnel is specifically the latitude and longitude coordinates of each position point of the corresponding driving lane line in the tunnel, and the inertial data is specifically the vehicle driving speed value.

[0024] Among them, the camera is used to capture the front scene of the vehicle in real time during driving, and the vehicle's GPS data is used to match the pre-loaded high-precision map to determine the exact position of the lane line. In areas without GPS signals (such as tunnels), the map data and the last known position of the vehicle before entering the tunnel are used to estimate the position of the lane line through interpolation or prediction models. Vehicles are usually equipped with speed sensors, such as wheel speed sensors, which are used to measure the rotation speed of the wheels and then calculate the speed of the vehicle.

[0025] Specifically, Figure 2 As shown in FIG. 1 , the specific steps of obtaining the driving lane line information before the vehicle is about to enter the tunnel are as follows: read several frames of image data before the vehicle is about to enter the tunnel, and perform preprocessing; calculate the gradients of the several frames of image data before the vehicle is about to enter the tunnel after preprocessing in the horizontal and vertical directions, and obtain the horizontal direction gradient value and the vertical direction gradient value of each pixel point of each frame image, which are specifically calculated using the Sobel operator, wherein the kernels of the Sobel operator in the x direction and the y direction are as follows: Among them, S x is the kernel of the Sobel operator in the x direction, S yis the kernel of the Sobel operator in the y direction, which is used to convolve with the pixels of the image to calculate the gradient of each point; edge analysis is performed on each frame of the image based on the gradient amplitude value and direction value of each pixel to obtain a binary image of each frame of the image (in these images, edge pixels are marked as white, usually with a value of 255, and non-edge pixels are marked as black, with a value of 0), which specifically includes non-maximum suppression, double threshold processing, edge tracking and connection; among them, non-maximum suppression is intended to refine the edges obtained by the previous steps. For each pixel in the image, the gradient direction of each pixel will be used to determine the edge direction of the pixel. According to the direction, the gradient amplitude of the current pixel is compared with the gradient amplitude of the adjacent pixels along the direction. If the gradient amplitude of the current pixel is not the local maximum value along the gradient direction in its neighborhood, the amplitude of the pixel is set to zero (which means that the pixel is not an edge).

[0026] Double threshold processing aims to determine the true edge by presetting two thresholds (high threshold and low threshold):

[0027] Strong edge: All pixels with gradient magnitude greater than a high threshold are considered strong edge pixels.

[0028] Weak edge: Pixels with gradient magnitude between the high threshold and the low threshold are considered weak edge pixels.

[0029] Non-edge: Pixels with gradient magnitude below a low threshold are considered non-edge pixels.

[0030] Edge Tracking and Connection Through edge tracking, we further determine which weak edges should be included as real edges:

[0031] Tracking: Starting from the strong edge pixel, track all the weak edge pixels connected to it. If a weak edge pixel is connected to any strong edge pixel, then this weak edge is also confirmed as a true edge.

[0032] Confirmed edges: Ultimately, all weak edge pixels that have not been confirmed as true edges will be suppressed, leaving only strong edges and confirmed weak edges.

[0033] The binary image of each frame is processed by Hough transform to obtain the lane line information before the vehicle enters the tunnel.

[0034] In this implementation scheme, several frames of image data are first read before the vehicle is about to enter the tunnel. These images contain information about the road conditions in front of the vehicle, which is crucial for lane line detection. Each frame of the image is converted into a grayscale image, which simplifies the subsequent processing steps. Because the grayscale image eliminates the interference of color information, the algorithm can focus more on brightness changes, making it easier to identify structures such as lane lines. Gaussian filtering is applied to smooth the image to reduce noise interference. This step is a key preparation before edge detection because it helps remove random noise in the image and reduce false detections. Sobe is used on the preprocessed image. l operator calculates the gradient in the horizontal and vertical directions respectively. This step is the key to identifying the edge position in the image, because the edge usually appears as a significant change in the image brightness. The non-maximum suppression step refines the edge by retaining only the pixels with the largest local gradient and eliminating non-edges or weak edges, making the edge more refined and prominent. The double threshold processing distinguishes strong edges from weak edges by setting two high and low thresholds. Strong edges are directly accepted, and weak edges need to be further verified whether they are connected to strong edges to decide whether to retain them. The edge tracking and connection step ensures the integrity of the edge through edge tracking. If the weak edge is connected to the strong edge, it is regarded as a true edge, which ensures the continuity of the lane line. The lane line is extracted from the binary image using Hough transform. This transform can identify and extract the straight line part in the image, which is very effective for building a vehicle's driving trajectory model. By converting the color image into a grayscale image and removing noise through Gaussian filtering, the complexity of data processing is reduced and the subsequent edge detection and lane line recognition steps are accelerated. Sobe The l operator combined with non-maximum suppression can accurately identify the real edges in the image. The dual threshold and edge tracking technology further improve the reliability of edge detection and ensure the correct identification of lane lines. The application of Hough transform makes it possible to effectively detect and reconstruct lane lines even when the lane lines in the image are partially blocked or incomplete, providing accurate navigation information for the vehicle.

[0035] Specifically, several frames of image data before the vehicle is about to enter the tunnel are preprocessed, wherein the preprocessing includes grayscale processing and Gaussian filtering processing on the several frames of image data before the vehicle is about to enter the tunnel (i.e., each frame of the image is converted into grayscale to reduce data complexity and focus on brightness information, and Gaussian filtering is applied to reduce noise in each frame of the image to improve the accuracy of subsequent edge detection. Gaussian filtering smoothes the image by weighted averaging the surrounding pixels of each pixel in the image, and the weight is determined by a Gaussian function, and the closer the point to the center pixel, the greater the weight).

[0036] In this embodiment, first, continuous image frames before entering the tunnel are read from the vehicle's camera device. These image frames provide initial visual data for the processing flow. Each frame of the image is converted from color to grayscale. The color image contains three color channels (red, green, and blue), while the grayscale image contains only one brightness channel, which greatly reduces the amount of data and simplifies the processing flow. Gaussian filtering is applied to the grayscale image. The filtering smoothes the image by calculating the weighted average of the center pixel and its neighboring pixels. The weight is determined by the Gaussian distribution, that is, the weight of the center pixel is the largest, and the weight gradually decreases with increasing distance. By converting the image to grayscale, the complexity of processing data is reduced, making subsequent image processing steps (such as edge detection) faster and more efficient. Gaussian filtering effectively reduces random noise in the image and improves image quality, which is particularly important for improving the performance of the edge detection algorithm, because noise can lead to misjudgment or omission of important edge information. Through smoothing, Gaussian filtering helps to highlight the real edge features and reduce the false edges caused by image noise, thereby improving the accuracy and reliability of edge detection technology. The combination of grayscale processing and Gaussian filtering is an effective method for edge detection technology (such as Sobe l operator or Canny edge detection) creates clearer and more precise input conditions. Reducing noise and irrelevant color information helps these algorithms more effectively identify real edges, such as lane lines. Before entering low-light or complex environments such as tunnels, the vehicle's visual system may be severely disturbed. Gaussian filtering smoothes the image and reduces the impact of noise under these conditions, ensuring that the lane line detection algorithm can maintain high performance in non-ideal visual environments. By providing clearer and more accurate image data, the preprocessing step directly improves the vehicle's autonomous driving system's responsiveness to the environment, which is critical to achieving safe autonomous driving, especially under changing road conditions, such as at tunnel entrances.

[0037] Specifically, the specific formula for calculating the gradient amplitude value and direction value of each pixel point in each frame of image is as follows: Among them, G ij is the gradient amplitude value of the jth pixel point of the i-th frame image, Sp ij is the horizontal gradient value of the jth pixel of the i-th frame image, Cz ij is the vertical gradient value of the jth pixel of the i-th frame image, θ ij is the value of the jth pixel in the i-th frame image, i=1,2,3,…,i0, i0 is the image frame number, j=1,2,3,…,j0, j0 is the number of pixels in the image.

[0038] It should be explained that atan2 is used here instead of a simple inverse tan function because atan2 can return the correct angle range (from -π to π) according to the sign of the horizontal gradient value and the vertical gradient value, which is necessary to determine the actual direction of the edge.

[0039] In this implementation, atan2 is used because the traditional inverse tan function can only return an angle in the range of -π / 2 to π / 2, which limits its application in image processing. Because ignoring the signs of the horizontal and vertical gradient values ​​may result in a 180-degree directional error, the atan2 function can provide a complete 360-degree angle range (actually -π to π) based on the signs of the horizontal and vertical gradient values, which is critical for accurately determining the actual direction of the pixel edge. By combining the square root of the sum of the squares of the horizontal and vertical gradients, the gradient amplitude calculation can provide a more comprehensive indication of the edge strength. This calculation method can more effectively highlight the edges in the image than the gradient in a single direction, especially in complex scenes with variable edge directions. This gradient calculation method is very important for achieving high efficiency. Accurate edge detection is crucial, as it directly affects subsequent image analysis steps such as edge tracking and lane line recognition: the correct gradient direction and accurate amplitude calculation enable gradient-based edge detection algorithms (such as Canny edge detection) to more accurately identify and track real edges in images. This is especially important for lane line detection, because accurate lane line information directly affects the vehicle's navigation and safety systems. In autonomous driving and advanced driver assistance systems, accurate image edge information can help the system better understand and respond to the surrounding environment, especially in complex or low-visibility driving conditions such as tunnels, at night or in bad weather conditions. By effectively calculating the gradient amplitude and direction, it is possible to reduce the consumption of computing resources while maintaining high accuracy, thereby improving the overall efficiency and response speed of the system.

[0040] Specifically, the specific steps of performing Hough transform processing on the binary image of each frame of image are as follows: determine the step size of distance resolution and angle resolution for each frame of image, generally speaking, the step size of distance resolution is 1 pixel, and the step size of angle resolution is 1 degree (π / 180 radians); initialize the accumulator for each frame of image, and perform voting processing on each edge pixel, specifically: establish a two-dimensional accumulator array, that is, record the number of votes for the parameter combination of distance resolution and angle resolution of each frame of image, traverse each pixel in the image, and for edge pixels with white values, calculate their votes for the distance resolution corresponding to all possible angle resolution values. , increase the number of parameter combinations of the corresponding distance resolution and angle resolution in the accumulator; identify the straight line in each frame from the accumulator, and perform comparative analysis to obtain the lane line information of the vehicle before entering the tunnel. The specific steps are as follows: find entries whose votes exceed the preset threshold from the accumulator. These entries represent the straight lines detected in the image. Extract the parameter combination of the distance resolution and angle resolution of the straight line from the accumulator. These parameters describe the lane line in the image. Repeat the above steps for each frame to continuously track the changes of the lane line. For each straight line detected in the sequence, accumulate evidence by comparing adjacent frames to confirm that it is a lane line.

[0041] In this implementation scheme, the resolution parameters are set: distance resolution: usually set to 1 pixel, which means that every unit of pixel change will be taken into account in the distance calculation, angular resolution: set to 1 degree (or π / 180 radians), allowing the accumulator to cover 360 degrees of possible directions, ensuring that straight lines in all directions can be captured, accumulator creation: create a two-dimensional array for each possible combination of distance resolution and angular resolution values. This accumulator will record the number of votes for each parameter combination and is used to determine the straight lines in the image, edge pixel identification: traverse each pixel in the binary image, and for pixels with a value of 255 (white, indicating an edge), calculate its vote for all possible distance resolutions and angular resolutions, accumulate votes: whenever an edge pixel votes for a particular angle, the corresponding unit of the relevant distance resolution value in the accumulator is increased, and entries exceeding a preset threshold are searched from the accumulator. These entries correspond to the straight lines detected in the image. For each exceeding threshold, the accumulator counts the number of votes. The accumulator entries that pass the threshold are used to extract their distance resolution and angle resolution parameters, which define the position and direction of the detected line. By comparing the lines detected in adjacent frames, evidence is accumulated to confirm that the detected line is a lane line. This step ensures the continuity and stability of the lane line. The fine-grained resolution setting of the accumulator enables the system to identify the direction and position of the lane line with high accuracy, which is crucial for the vehicle's navigation system, especially under complex or curvy road conditions. By systematically considering all angles, the Hough transform can effectively identify and track lines when visual information is incomplete or partially occluded, which enhances the robustness of lane line detection and enables the system to maintain high performance under various visual conditions. By analyzing consecutive frames and accumulating evidence of lines, the system can dynamically track changes in lane lines, which is particularly important for high-speed driving and responding to sudden changes in road conditions. This continuous analysis helps the driver assistance system predict and adapt to road changes more accurately.

[0042] Specifically, the specific steps for predicting the vehicle's predicted coordinate information at each future time point in the tunnel are as follows: read the vehicle's speed value at each time point before the vehicle is about to enter the tunnel and perform a comprehensive analysis to obtain the vehicle's average speed and the vehicle's acceleration; use the vehicle's average speed as the initial speed value when the vehicle enters the tunnel; analyze and predict the predicted driving distance value of the vehicle at each future time point in the tunnel based on the vehicle's initial speed value and the vehicle's acceleration average when the vehicle enters the tunnel, and analyze the vehicle's predicted coordinate information based on the longitude and latitude coordinates of each position point of the corresponding driving lane line in the tunnel (read the longitude and latitude coordinates when the vehicle enters the tunnel, and analyze the predicted position point in the tunnel where the vehicle will arrive based on the predicted driving distance value corresponding to the number of time points, and regard the corresponding longitude and latitude coordinates as the vehicle's predicted coordinate information).

[0043] In this embodiment, the driving speed at each time point before entering the tunnel is read from the vehicle's sensor, the collected speed data is comprehensively analyzed, and the average driving speed and acceleration of the vehicle are calculated. These averages will be used to predict the future position of the vehicle. The calculated average driving speed is set as the initial speed of the vehicle when entering the tunnel. Based on the initial speed and the average acceleration of the vehicle, the driving distance of the vehicle at each subsequent time point in the tunnel is predicted. The longitude and latitude coordinates of the vehicle when entering the tunnel are read, and the position coordinates of the vehicle at each future time point are calculated in combination with the predicted driving distance. This involves converting the driving distance into changes in longitude and latitude coordinates to accurately locate the position of the vehicle in the tunnel. By using the statistical mean of speed and acceleration, the prediction is more accurate because it takes into account the average conditions of vehicle driving rather than instantaneous or abnormal data points. In the tunnel, the GPS signal may be lost. The use of the prediction method can ensure that the vehicle position can be continuously tracked even when the GPS signal is unavailable. By calculating the position coordinates at each time point, the driving assistance system can be helped to predict potential collisions and other safety issues and adjust the vehicle's driving path or speed in a timely manner. In emergency situations, such as vehicle failures or accidents, accurate location information can be quickly transmitted to the rescue team to shorten the response time.

[0044] Specifically, the specific steps for obtaining the mean vehicle speed and the mean vehicle acceleration are as follows: read the vehicle speed value at each time point before the vehicle is about to enter the tunnel, and obtain the mean vehicle speed value by combining the mean method; perform acceleration analysis on the vehicle speed value at each time point before the vehicle is about to enter the tunnel, and obtain several groups of vehicle acceleration values; perform mean analysis on several groups of vehicle acceleration values ​​by combining the moving exponential average method, and obtain the mean vehicle acceleration value; wherein, the specific formulas for calculating the mean vehicle speed and the mean vehicle acceleration value are as follows: Where V′ is the mean vehicle speed, V q is the vehicle speed at the qth time point before the vehicle enters the tunnel, q = 1, 2, 3, ..., q0, q0 is the number of time points, A′ is the mean vehicle acceleration, a w is the acceleration value of the wth group of vehicles, α w1 is the weighting coefficient of the acceleration value of the wth group of vehicles, a w+1 is the acceleration value of the w+1th group of vehicles, α w2 is the weighting coefficient of the acceleration value of the w+1th group of vehicles, α w1 +α w2 =1, w=1, 2, 3, ..., w0, w0 is the number of vehicle acceleration value groups, and w0=q0-1.

[0045] It needs to be explained that α w1 , α w2The specific calculation process is: summing up the acceleration values ​​of the wth and w+1th groups of vehicles to obtain the acceleration sum value, and then performing proportion analysis on the acceleration values ​​of the wth and w+1th groups of vehicles and the acceleration sum value respectively, and the analysis result is the weighting coefficient.

[0046] In this implementation scheme, the vehicle's speed at each time point before entering the tunnel is read from the vehicle's sensor, and a simple mean method is used to calculate the average of all collected speed data. This average represents the typical speed of the vehicle before entering the tunnel. The acceleration is calculated by analyzing the speed change at two consecutive time points, and the acceleration value calculated each time is organized into several groups of data. The acceleration values ​​of all groups are summed, and the acceleration value of each group is compared with the total, and the proportion of the acceleration value of each group in the total is calculated. These proportions are used as weighting coefficients, and the moving exponential average method is used to calculate the weighted average of the acceleration according to the weighting coefficient. This method provides the average speed of the vehicle before entering the tunnel, which helps to set more accurate initial conditions and is very important for predicting the behavior of the vehicle in the tunnel. By accurately calculating the acceleration and using the weighted average method, the acceleration value representing the real dynamic changes of the vehicle can be obtained, which enhances the accuracy and representativeness of the data. The moving exponential average method provides a smoothing process for the acceleration data, reduces the impact of random fluctuations and abnormal data, and makes the acceleration measurement more stable and reliable. Through accurate initial speed and acceleration data, the prediction model can more accurately predict the position of the vehicle at each time point in the tunnel.

[0047] Specifically, the specific formula for predicting the actual driving distance of a vehicle in a tunnel at a certain point in the future is as follows: Among them, X q′0 is the predicted driving distance of the vehicle at the q′0th time point in the future in the tunnel, V′ is the mean driving speed of the vehicle, that is, the initial speed value when the vehicle enters the tunnel, A′ is the mean acceleration of the vehicle, T q′0 is the time value of the vehicle at the future q′0 time point in the tunnel (usually in seconds), and β is the error factor.

[0048] In this implementation scheme, the specific formula for predicting the actual driving distance value of the vehicle at a certain point in the future in the tunnel makes the position prediction more accurate by considering the influence of acceleration on the vehicle's driving distance. This method is particularly suitable for situations where the vehicle speed changes greatly, such as acceleration or deceleration. This method is not only suitable for scenarios of straight-line driving, but can also adapt to the changing trajectory of the vehicle under the influence of different accelerations, providing a general solution to handle various driving situations. By continuously calculating the position at each point in time, this method supports dynamic tracking of vehicle movement, which is particularly important for navigation and location services in dynamic environments. The prediction model can be directly integrated into the vehicle's navigation system to help improve path planning and tracking accuracy, especially in environments where GPS signals are unstable (such as tunnels, urban high-rise buildings, etc.), providing more accurate dynamic position data for advanced driver assistance systems, and can optimize the response time and accuracy of safety functions such as automatic braking and lane keeping. This physical model-based position prediction method provides a reliable mathematical foundation for autonomous driving technology and helps to achieve safer and more autonomous driving strategies.

[0049] Specifically, the specific calculation process of the error factor is as follows: obtain the actual driving distance value of the vehicle at each time point before entering the tunnel (calculate the driving distance through GPS positioning data); analyze the estimated driving distance value of the vehicle at each time point before entering the tunnel based on the mean vehicle acceleration and the vehicle driving speed value at each time point before entering the tunnel; analyze the distance error value between the actual driving distance value and the estimated driving distance value at each time point before entering the tunnel, and perform mean analysis to obtain the error factor.

[0050] In this implementation scheme, the GPS positioning data is used to calculate the actual driving distance at each time point before the vehicle is about to enter the tunnel. This data provides accurate geographic location information and ensures the accuracy of distance measurement. The predicted driving distance is calculated based on the average acceleration of the vehicle and the speed value at each time point. This step involves physical kinematic formulas. The moving distance is estimated by integrating speed and acceleration data. The actual driving distance and the estimated driving distance are compared, and the difference between the two, i.e., the distance error value, is calculated. The distance error values ​​at all time points are statistically analyzed (such as averaging) to obtain an error factor. The error factor represents the average degree of deviation between the estimated value and the actual value. By calculating the error factor, the accuracy of the navigation system in predicting the vehicle position can be accurately evaluated, which is crucial for adjusting and optimizing the system algorithm, helping to reduce prediction errors and improve navigation accuracy. The analysis results of the error factor can be used to adjust the vehicle control system, especially in autonomous driving and advanced driver assistance systems (ADAS). By reducing prediction errors, the responsiveness and reliability of the system can be improved. By regularly calculating and publishing the error factor, the performance and reliability of the system can be demonstrated to users, which not only increases user trust in the system, but also provides higher safety guarantees in actual driving.

[0051] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0052] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0054] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0055] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0056] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0057] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A visual and inertial joint positioning method, characterized in that: The following steps are involved: Continuously obtain several frames of image data and inertial data at several time points before the vehicle enters the tunnel; Based on Canny edge detection and Hough transform, several frames of image data before the vehicle enters the tunnel are identified and analyzed to obtain the lane line information before the vehicle enters the tunnel, and obtain the corresponding lane line data in the tunnel; Comprehensively analyze the inertial data of several time points before the vehicle enters the tunnel to obtain the average vehicle speed and acceleration; Based on the corresponding lane line data in the tunnel and the average vehicle speed and acceleration, the vehicle coordinate information at each future time point in the tunnel is predicted.

2. The visual and inertial combined positioning method according to claim 1, characterized in that: Each frame of image data includes the pixel value of each pixel point in the image in front of the vehicle, the corresponding driving lane line data in the tunnel is specifically the latitude and longitude coordinates of each position point of the corresponding driving lane line in the tunnel, and the inertial data is specifically the vehicle driving speed value.

3. The visual and inertial combined positioning method according to claim 2, characterized in that: The specific steps for obtaining the lane line information before the vehicle enters the tunnel are as follows: Read several frames of image data before the vehicle enters the tunnel and perform preprocessing; Calculate the gradients in the horizontal and vertical directions for several frames of image data before the vehicle enters the tunnel after preprocessing, and obtain the horizontal and vertical gradient values ​​of each pixel point in each frame of the image; The horizontal gradient value and the vertical gradient value of each pixel point of each frame image are combined and analyzed respectively to obtain the gradient amplitude value and direction value of each pixel point of each frame image; Perform edge analysis on each frame of image based on the gradient amplitude value and direction value of each pixel point to obtain a binary image of each frame of image; The binary image of each frame is processed by Hough transform to obtain the lane line information before the vehicle enters the tunnel.

4. The visual and inertial combined positioning method according to claim 3, characterized in that: Preprocessing is performed on a plurality of frames of image data before the vehicle is about to enter the tunnel, wherein the preprocessing includes performing grayscale processing and Gaussian filtering processing on the plurality of frames of image data before the vehicle is about to enter the tunnel.

5. The visual and inertial combined positioning method according to claim 3, characterized in that: The specific formula for calculating the gradient amplitude and direction value of each pixel point in each frame of image is as follows: Among them, G ij is the gradient amplitude value of the jth pixel point of the i-th frame image, Sp ij is the horizontal gradient value of the jth pixel of the i-th frame image, Cz ij is the vertical gradient value of the jth pixel of the i-th frame image, θ ij is the value of the jth pixel in the i-th frame image, i=1,2,3,…,i0, i0 is the image frame number, j=1,2,3,…,j0, j0 is the number of pixels in the image.

6. The visual and inertial combined positioning method according to claim 3, characterized in that: The specific steps of performing Hough transform on the binary image of each frame are as follows: Determine the step size of distance resolution and angle resolution for each frame of image; Initialize the accumulator for each frame of image and perform voting on each edge pixel; The straight lines in each frame of the image are identified from the accumulator and compared and analyzed to obtain the lane line information before the vehicle enters the tunnel.

7. The visual and inertial combined positioning method according to claim 2, characterized in that: The specific steps for predicting the vehicle's predicted coordinate information at each future time point in the tunnel are as follows: Read the vehicle speed value at each time point before the vehicle enters the tunnel and conduct a comprehensive analysis to obtain the average vehicle speed and the average vehicle acceleration; The average speed of vehicles is taken as the initial speed value when the vehicles enter the tunnel; The predicted driving distance of the vehicle at each future time point in the tunnel is predicted based on the initial speed value of the vehicle when the vehicle enters the tunnel and the average value of the vehicle acceleration, and the predicted coordinate information of the vehicle is analyzed based on the latitude and longitude coordinates of each position point of the corresponding driving lane line in the tunnel.

8. The visual and inertial combined positioning method according to claim 7, characterized in that: The specific steps for obtaining the mean value of vehicle speed and vehicle acceleration are as follows: Read the vehicle speed value at each time point before the vehicle enters the tunnel, and obtain the average vehicle speed value by combining the average method; Perform acceleration analysis on the vehicle speed value at each time point before the vehicle enters the tunnel, and obtain several groups of vehicle acceleration values; Perform mean analysis on several groups of vehicle acceleration values ​​in combination with the moving exponential average method to obtain the mean value of vehicle acceleration; The specific formulas for calculating the mean vehicle speed and the mean vehicle acceleration are as follows: Where V′ is the mean vehicle speed, V q is the vehicle speed at the qth time point before the vehicle enters the tunnel, q = 1, 2, 3, ..., q0, q0 is the number of time points, A′ is the mean vehicle acceleration, a w is the acceleration value of the wth group of vehicles, α w1 is the weighting coefficient of the acceleration value of the wth group of vehicles, a w+1 is the acceleration value of the w+1th group of vehicles, α w2 is the weighting coefficient of the acceleration value of the w+1th group of vehicles, α w1 +α w2 =1, w=1,2,3,…,w0, w0 is the number of vehicle acceleration value groups.

9. The visual and inertial combined positioning method according to claim 7, characterized in that: The specific formula for predicting the actual driving distance of a vehicle at a certain point in the future in the tunnel is as follows: in, is the predicted driving distance of the vehicle at the q′0th time point in the future in the tunnel, V′ is the mean driving speed of the vehicle, that is, the initial speed value when the vehicle enters the tunnel, A′ is the mean acceleration of the vehicle, is the time value of the vehicle at the future q′0 time point in the tunnel, and β is the error factor.

10. The visual and inertial combined positioning method according to claim 9, characterized in that: The specific calculation process of the error factor is as follows: Get the actual driving distance value of the vehicle at each time point before entering the tunnel; Analyzing the estimated value of the travel distance at each time point before the vehicle is about to enter the tunnel based on the mean value of the vehicle acceleration and the vehicle travel speed value at each time point before the vehicle is about to enter the tunnel; The distance error between the actual driving distance value and the estimated driving distance value at each time point before the vehicle is about to enter the tunnel is analyzed, and a mean analysis is performed to obtain the error factor.

Images