Synchronous positioning and three-dimensional path real-time generation method for panoramic image of driving environment

Abstract

The invention belongs to the technical field of intelligent driving, and particularly relates to a synchronous positioning and three-dimensional path real-time generation method for a panoramic image of a driving environment, which improves the accuracy of feature point extraction by enhancing the contrast ratio of images acquired under a low-light condition, and remarkably improves the vehicle positioning precision by using an improved feature point matching algorithm. Feature point distribution in the panoramic image is updated in combination with the predicted position, and real-time generation and dynamic correction of the three-dimensional path diagram are realized, so that the adaptability and reliability of the system in complex and changing environments are effectively enhanced. Therefore, the problem that the performance of a traditional method is reduced in a low-light environment is solved, the overall performance of the intelligent driving system is greatly improved, and the driving safety and the path planning accuracy are ensured.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology, and specifically relates to a method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path. Background Art

[0002] Existing driving environment perception systems usually rely on sensors such as cameras and LiDAR to obtain information about the vehicle's surroundings and process it through computer vision algorithms. The main steps of these systems include: Image acquisition: Use multiple cameras or sensors to capture images of the environment around the vehicle from different angles.

[0003] Image stitching: stitch the collected multi-view images to form a panoramic image.

[0004] Feature point extraction and matching: Extract feature points from the panoramic image and use feature point matching between adjacent frames to estimate the vehicle motion state.

[0005] Path planning: Based on the feature point matching results, the vehicle's driving direction and speed are calculated, and the corresponding path planning is generated.

[0006] Although existing technologies can meet basic needs to a certain extent, insufficient image brightness in low-light conditions can lead to inaccurate feature point extraction, which in turn affects the positioning accuracy of the vehicle and the reliability of path planning. In addition, existing systems often lack a dynamic update and correction mechanism for the three-dimensional path map, making it difficult to adapt to complex dynamic environmental changes. Summary of the invention

[0007] The purpose of the present invention is to provide a method for synchronous positioning of panoramic images of a driving environment and real-time generation of a three-dimensional path, which not only improves the vehicle positioning accuracy under low-light conditions, but also realizes the accurate real-time generation of a three-dimensional path, effectively improving the reliability and safety of the intelligent driving system, so as to solve the problems raised in the above-mentioned background technology.

[0008] To achieve the above object, the present invention adopts the following technical solution: a method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path, comprising the following steps: Collect multi-view low-light images of the vehicle's surroundings; based on the multi-view low-light images, adjust the brightness of each view image to enhance the contrast, and stitch the adjusted images to form a panoramic image; mark feature points in the panoramic image and record their position information, and use the position information of the feature points to calculate the vehicle's driving direction and speed; predict the vehicle's position at the next moment based on the driving direction and speed, and update the feature point distribution in the panoramic image based on the predicted position; construct a three-dimensional path map based on the updated feature point distribution, and continuously monitor and correct the three-dimensional path map.

[0009] Preferably, the collecting of multi-view low-light images of the vehicle's surroundings includes: Activate multiple cameras set up around the vehicle, each camera is configured with enhanced night vision mode; Based on the night vision enhancement mode, adjust the exposure time of each camera , where ISO is the sensitivity and EV is the ambient brightness value; Take images using the adjusted exposure time and record the timestamp of each frame , where t_start is the start time, n is the number of frames, and TI is the time interval.

[0010] Preferably, in the multi-view low-light image, adjusting the brightness of each view image to enhance the contrast includes: Receives a timestamped image and calculates the average brightness value of each frame , where B(x,y) is the brightness value of the pixel, W and H are the image width and height respectively; According to the average brightness value L_avg, determine the gain coefficient G=G_max-(L_avg / R) of each image, where G_max is the maximum gain value and R is the reference brightness value; The gain coefficient G is used to adjust the brightness of each frame of the image, and the updated brightness value ; Remap the adjusted image brightness value to the range of 0 to 255. The mapping formula is: , where MIN_B_new and MAX_B_new are the minimum and maximum brightness values ​​of the adjusted image, respectively, and B_final is the final brightness value.

[0011] Preferably, stitching the adjusted images to form a panoramic image includes: Receive the adjusted image corresponding to the mapped brightness value B_final(x,y), and calculate the feature point set F={p1,p2,...,pn} of the overlapping area for each frame of the image; Based on the feature point set F, the transformation matrix M=[abc;def] is determined using the matching point pairs between adjacent images, where a to f are transformation parameters, by minimizing the error To solve, I1 and I2 are two adjacent images; Apply the transformation matrix M to each image, convert it to the global coordinate system, and get the new position , ensure that all images are aligned in the same coordinate system, P_transformed is the aligned image; The aligned images P_transformed(x,y) are fused and the final pixel value is calculated by weighted average method: , where w_i is the weight of the i-th image and P(x,y) is the panoramic image.

[0012] Preferably, marking feature points in the panoramic image and recording their position information includes: Receive the fused panoramic image P(x,y) and calculate the gradient amplitude GRAD=sqrt(dx^2+dy^2) of each frame, where dx and dy are the gradients in the x and y directions respectively; Based on the gradient amplitude GRAD, determine the local maximum point as a feature point set F={f1,f2,...,fm}, where the position of each feature point fi is represented by the coordinates (X_i,Y_i); For each feature point fi in the feature point set F, calculate its descriptor D_i=SUM((P(x+x_i,y+y_i)-AVG_P)^2), where AVG_P is the average value of the pixel values ​​in the surrounding area, and (X_i,Y_i) is the coordinate of the center point; The feature points and their descriptors D_i are stored in the database together with the location information (X_i, Y_i) to form a feature point list: L={(f1,D_1,X_1,Y_1),(f2,D_2,X_2,Y_2),...,(fm,D_m,X_m,Y_m)}.

[0013] Preferably, the method of calculating the driving direction and speed of the vehicle using the position information of the feature points includes: Receive the feature points and their position information (X_i, Y_i) in the feature point list L, and match feature point pairs P between consecutive frames: P={(f1_t1,f1_t2),(f2_t1,f2_t2),...,(fn_t1,fn_t2)}, where t1 and t2 represent different time points; Based on the feature point pair P, calculate the displacement vector V=(X_2-X_1,Y_2-Y_1) of each feature point between the two frames, where (X_1,Y_1) and (X_2,Y_2) are the position coordinates of the same feature point at two time points respectively; The vehicle travel direction A=atan2(V_y,V_x) is calculated using the displacement vector V, where V_x and V_y are the components of the displacement vector on the x-axis and y-axis respectively, and atan2 is an inverse tangent function; Combining the displacement vector V and the time interval TI, the vehicle speed S=sqrt(V_x^2+V_y^2) / TI is calculated, where TI is the time difference between the capture of two frames of images.

[0014] Preferably, predicting the vehicle position at the next moment according to the driving direction and speed includes: Receive the vehicle driving direction A and the vehicle driving speed S, and determine the current vehicle position coordinates C=(X_cur, Y_cur), where (X_cur, Y_cur) is the position of the vehicle in the current frame; Based on the vehicle driving direction A, calculate the direction vector DIR=(cos(A), sin(A)), where cos and sin are cosine and sine operations respectively; Using the vehicle speed S and time interval TI, calculate the displacement distance , combined with the direction vector DIR, we get the displacement vector ; The displacement vector DELTA is added to the current vehicle position coordinate C to obtain the predicted vehicle position at the next moment N=(X_cur+DELTA_X,Y_cur+DELTA_Y), where DELTA_X and DELTA_Y are the components of the displacement vector on the x-axis and y-axis respectively.

[0015] Preferably, the updating of the distribution of feature points in the panoramic image in combination with the predicted position includes: Receive the predicted vehicle position N=(X_next, Y_next) at the next moment, and obtain the feature point list L in the current panoramic image: L={(f1,D_1,X_1,Y_1),(f2,D_2,X_2,Y_2),...,(fm,D_m,X_m,Y_m)}; Based on the predicted vehicle position N at the next moment, calculate the position change POS_CHANGE of the vehicle relative to each feature point = (X_next-X_i, Y_next-Y_i), where (X_i, Y_i) is the position coordinate of the feature point fi in the current frame; Use the position change POS_CHANGE to update the new position coordinates (X'_i, Y'_i) of each feature point. The new position coordinate calculation formula is X'_i=X_i+POS_CHANGE_X, Y'_i=Y_i+POS_CHANGE_Y, where POS_CHANGE_X and POS_CHANGE_Y are the components of POS_CHANGE on the x-axis and y-axis respectively; Remap the updated feature point position coordinates (X'_i, Y'_i) to the panoramic image to form an updated feature point list L_updated: L_updated={(f1,D_1,X'_1,Y'_1),(f2,D_2,X'_2,Y'_2),...,(fm,D_m,X'_m,Y'_m)}.

[0016] Preferably, constructing a three-dimensional path map based on the updated feature point distribution includes: Receive the updated feature point list L_updated, and calculate the height value Z_i of each feature point on the z-axis; Based on the position coordinates (X'_i, Y'_i, Z_i) of the feature points, a three-dimensional point cloud PC is established in the local coordinate system: PC={(X'_1,Y'_1,Z_1),(X'_2,Y'_2,Z_2),...,(X'_m,Y'_m,Z_m)}; Using the points in the three-dimensional point cloud PC, calculate the distance between adjacent feature points ED_ij=sqrt((X'_j-X'_i)^2+(Y'_j-Y'_i)^2+(Z_j-Z_i)^2), where (i, j) is an adjacent feature point pair; According to the distance ED_ij, edges connecting adjacent feature points are constructed to form a three-dimensional path graph G_path={(f1,f2,ED_12),(f2,f3,ED_23),...,(fn-1,fn,ED_n-1n)}, where ED_ij represents the distance between feature points fi and fj.

[0017] Preferably, the continuously monitoring and correcting the three-dimensional path map comprises: Based on the three-dimensional path graph G_path, an initial threshold TH=TH_0 is set for detecting path changes, and for each edge e_ij in the path graph belonging to the G_path set, its edge length ED(e_ij)=dist_ij is calculated; Update the position coordinates (X'_i, Y'_i, Z_i) of the feature points through sensor data, and recalculate the new length NEW_ED (e_ij) of each side: NEW_ED(e_ij)=sqrt((X'_j-X'_i)^2+(Y'_j-Y'_i)^2+(Z_j-Z_i)^2); Compare the new length NEW_ED(e_ij) with the original length ED(e_ij). If |NEW_ED(e_ij)-ED(e_ij)|>TH, mark it as an exception and record the exception information in the exception information list AN: AN={(e_1,|NEW_ED(e_1)-ED(e_1)|),...,(e_n,|NEW_ED(e_n)-ED(e_n)|)}; According to the abnormal information in the abnormal information list AN, the marked edges are corrected and the edge length is updated using the weighted average method. The formula is: , where W_old and W_new are the old weight and new weight respectively, and satisfy W_old+W_new=1, CORRECTED_ED is the corrected edge length, and W is the weight.

[0018] Technical effects and advantages of the present invention: Compared with the prior art, the method for synchronous positioning of panoramic images of a driving environment and real-time generation of a three-dimensional path proposed by the present invention has the following advantages: The present invention improves the accuracy of feature point extraction by enhancing the contrast of images collected under low-light conditions, and significantly improves the vehicle positioning accuracy by using an improved feature point matching algorithm. By updating the distribution of feature points in the panoramic image in combination with the predicted position, the real-time generation and dynamic correction of the three-dimensional path map is realized, thereby effectively enhancing the adaptability and reliability of the system in complex and changing environments. This not only solves the problem of performance degradation of traditional methods in low-light environments, but also greatly improves the overall performance of the intelligent driving system, ensuring driving safety and the accuracy of path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention is a flow chart of a method for synchronously positioning a panoramic image of a driving environment and generating a three-dimensional path in real time. DETAILED DESCRIPTION

[0020] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0021] The present invention provides Figure 1 The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path includes the following steps: Step 1: Collect multi-view low-light images of the vehicle's surroundings; specifically including: Activate multiple cameras around the vehicle, each with enhanced night vision mode, to ensure high-quality images of the environment can be captured in low-light conditions. Enhanced night vision mode increases the sensitivity of the camera in low-light environments, making the image clearer and more detailed.

[0022] Based on the night vision enhancement mode, adjust the exposure time of each camera , where ISO is the sensitivity and EV is the ambient brightness value; the principle of the formula is to calculate the appropriate exposure time based on the ambient brightness and sensitivity. A lower EV value indicates a darker environment, which requires a longer exposure time to capture enough light; a higher ISO value indicates a higher sensitivity, which can capture enough light in a shorter time. By dynamically adjusting the exposure time, the image quality can be optimized under different lighting conditions. Reasonable adjustment of the exposure time can avoid overexposure or underexposure of the image, thereby improving the accuracy of feature point extraction and subsequent processing.

[0023] Take images using the adjusted exposure time and record the timestamp of each frame , where t_start is the start time, n is the number of frames, and TI is the time interval. The principle of this formula is to calculate the specific shooting time of each frame based on the start time and the number of frames. For example, if the start time is 0 seconds and the time interval is 0.033 seconds (i.e. 30 frames per second), then the timestamp of the 10th frame image is Second.

[0024] By recording the timestamp of each frame, the shooting time of each frame can be accurately tracked, which is crucial for subsequent image matching, feature point tracking, and motion estimation. Accurate recording of timestamps helps improve the accuracy of positioning and path planning.

[0025] Step 2: Based on the multi-view low-light image, adjust the brightness of each view image to enhance the contrast; specifically including: Receives a timestamped image and calculates the average brightness of each frame (the average brightness of the entire image) , where B(x,y) is the brightness value of the pixel, W and H are the image width and height respectively; The formula The principle is to obtain the average brightness value of the entire image by summing the brightness values ​​of all pixels and dividing it by the total number of pixels. This value is used to evaluate the overall brightness level of the image and provide a basis for subsequent gain adjustment. By calculating the average brightness value of each frame of the image, the overall brightness level of the image can be quantified. This step provides a benchmark reference for subsequent brightness adjustment and ensures the rationality of the brightness adjustment.

[0026] According to the average brightness value L_avg, the gain coefficient G=G_max-(L_avg / R) of each image is determined, where G_max is the maximum gain value and R is the reference brightness value; the principle of this formula is to calculate the gain coefficient based on the difference between the average brightness of the image and the reference brightness. When the image is darker, L_avg is smaller and the G value is larger, thereby increasing the image brightness; conversely, when the image is brighter, L_avg is larger and the G value is smaller, thereby reducing the image brightness.

[0027] By calculating the gain coefficient, the gain can be dynamically adjusted according to the average brightness of the image, thereby optimizing the image brightness under different lighting conditions. The introduction of the gain coefficient makes the image brightness more balanced and improves the accuracy of feature point extraction.

[0028] The gain coefficient G is used to adjust the brightness of each frame of the image, and the updated brightness value ; The principle of this formula is to adjust the brightness value of each pixel by multiplying it by the gain coefficient. When the gain coefficient is greater than 1, the brightness is increased, and when it is less than 1, the brightness is reduced, thereby achieving overall brightness adjustment. By applying the gain coefficient to adjust the brightness of each frame of the image, the brightness and contrast of the image can be effectively improved, especially in low light conditions, which can significantly improve the image quality and facilitate subsequent processing.

[0029] Remap the adjusted image brightness value to the range of 0 to 255. The mapping formula is: , where MIN_B_new and MAX_B_new are the minimum and maximum brightness values ​​of the adjusted image, respectively, and B_final is the final brightness value.

[0030] The principle of this formula is to normalize the adjusted brightness value and remap it to the standard range of 0 to 255. The specific steps are as follows: Calculates the minimum and maximum adjusted brightness values.

[0031] Use a linear transformation to map the adjusted brightness values ​​to between 0 and 255, ensuring that the brightness distribution of the image is uniform and meets display standards.

[0032] By remapping the adjusted brightness value, ensuring that the image brightness value falls within the standard range of 0 to 255, the contrast and visual effect of the image are enhanced, which facilitates the subsequent feature point extraction and matching.

[0033] Step 3: stitch the adjusted images together to form a panoramic image; specifically including: Receive the adjusted image corresponding to the mapped brightness value B_final(x,y), and calculate the feature point set F={p1,p2,...,pn} of the overlapping area for each frame of image; by calculating the feature point set in each frame of image, the overlapping area between adjacent images can be identified. These feature points provide key information for subsequent image alignment and stitching, ensuring the accuracy and stability of the stitching process.

[0034] Based on the feature point set F, the transformation matrix M=[abc;def] is determined using the matching point pairs between adjacent images, where a, b, c, d, e, f are transformation parameters, which represent the specific values ​​of the transformation matrix. To solve, I1 and I2 are two adjacent images; The principle of this formula is to solve the transformation matrix M by minimizing the difference between the feature points in the two images. The specific steps are as follows: Calculate the corresponding position of each feature point p_i in the feature point set F in the two images.

[0035] Use the transformation matrix M to map the feature points p_i in one image to another image.

[0036] The transformation matrix M is optimized by minimizing the brightness difference of corresponding feature points in the two images.

[0037] By calculating the transformation matrix, adjacent images can be aligned to the same coordinate system, ensuring that the stitched panoramic image is seamless and distortion-free. The accuracy of the transformation matrix directly affects the stitching quality.

[0038] Apply the transformation matrix M to each image, convert it to the global coordinate system, and get the new position , ensuring that all images are aligned in the same coordinate system, P_transformed is the aligned image; the principle of this formula is to apply the transformation matrix M to each pixel in the image, transforming it from the original coordinate system to the global coordinate system. The specific steps are as follows: For each pixel (x, y), calculate its new position in the global coordinate system.

[0039] Assign the transformed pixel values ​​to the new locations, ensuring that all images are aligned in the same coordinate system.

[0040] By applying a transformation matrix, all images are transformed into a global coordinate system, ensuring alignment and smooth transitions between images. This step is critical to achieving seamless stitching.

[0041] The aligned images P_transformed(x,y) are fused and the final pixel value is calculated by weighted average method: , where w_i is the weight of the i-th image, and P(x,y) is the panoramic image. The principle of this formula is to generate the final panoramic image by weighted averaging the aligned images. The specific steps are as follows: For each pixel (x, y), calculate its weighted average in all aligned images.

[0042] The weights w_i are assigned according to the importance of the images or the size of the overlapping area, ensuring a smooth transition in the overlapping area.

[0043] Assign the weighted average value to the corresponding position in the final panoramic image.

[0044] The aligned images are fused by weighted averaging to generate high-quality panoramic images. This method can smooth the overlapping areas between images, reduce stitching traces, and improve the overall visual effect.

[0045] Step 4: Marking feature points in the panoramic image and recording their location information; specifically including: Receive the fused panoramic image P(x,y) and calculate the gradient amplitude of each frame (used to measure the edge strength of each pixel in the image) GRAD=sqrt(dx^2+dy^2), where dx and dy represent the gradient values ​​of the image in the x and y directions, respectively, reflecting the rate of change of the image brightness along these two directions; the principle of this formula is to determine the edge in the image based on the magnitude of the gradient. The specific steps are as follows: For each pixel, calculate its gradient in the x direction (dx) and y direction (dy).

[0046] The gradient magnitude is calculated using the above formula to quantify the edge strength at that point.

[0047] By calculating the gradient of an image in the x-direction and the y-direction, edges and details in the image can be identified.

[0048] Based on the gradient magnitude GRAD, the local maximum point is determined as the feature point set F={f1,f2,...,fm}, where the position of each feature point fi is represented by the coordinates (X_i,Y_i); by finding the local maximum point of the gradient magnitude, the significant feature points in the image can be effectively identified. These feature points provide key information for subsequent descriptor calculation and matching.

[0049] For each feature point fi in the feature point set F, calculate its descriptor D_i=SUM((P(x+x_i,y+y_i)-AVG_P)^2), where AVG_P is the average value of the pixel values ​​in the surrounding area, and (X_i,Y_i) is the coordinate of the center point; the principle of this formula is to perform statistical analysis on the pixel values ​​in the area around the feature point and calculate the sum of squares of the differences relative to the average value to characterize the local structure of the point. The specific steps are as follows: For each feature point, a fixed-size window is defined, including the point and its surrounding pixels.

[0050] Calculate the average AVG_P of all pixel values ​​within the window.

[0051] For each pixel in the window, the square of the difference between it and the average value is calculated, and the sum is taken to obtain the descriptor D_i.

[0052] By calculating the descriptor of each feature point, the local structural information around the point can be effectively represented. The similarity of the descriptors can be used to match feature points, thereby achieving image splicing or recognition tasks.

[0053] The feature points and their descriptors D_i are stored in the database together with the location information (X_i, Y_i) to form a feature point list: L={(f1,D_1,X_1,Y_1),(f2,D_2,X_2,Y_2),...,(fm,D_m,X_m,Y_m)}, f_i,D_i,X_i,Y_i: the ith feature point, its descriptor and position coordinates respectively. By storing the feature points, descriptors and their position information, subsequent retrieval and matching operations are facilitated.

[0054] Step 5: Calculate the vehicle's driving direction and speed using the location information of the feature points; specifically including: Receive the feature points and their position information (X_i, Y_i) in the feature point list L, and match feature point pairs P between consecutive frames: P={(f1_t1,f1_t2),(f2_t1,f2_t2),...,(fn_t1,fn_t2)}, where t1 and t2 represent different time points; by matching feature point pairs between consecutive frames, the motion trajectory of feature points can be tracked. These matching results provide basic data for the subsequent calculation of displacement vectors, driving direction and speed.

[0055] Based on the feature point pair P, calculate the displacement vector V=(X_2-X_1,Y_2-Y_1) of each feature point between the two frames, where (X_1,Y_1) and (X_2,Y_2) are the position coordinates of the same feature point at two time points respectively; the principle of this formula is to determine its displacement vector by calculating the coordinate difference between the two frames at the same position of the feature point. The specific steps are as follows: For each matched feature point pair, calculate its displacement components on the x-axis and y-axis.

[0056] Combine these components into a two-dimensional vector V.

[0057] By calculating the displacement vectors of feature points between two frames, the movement of feature points can be quantified. These displacement vectors provide key data for the subsequent calculation of the vehicle's driving direction and speed.

[0058] The displacement vector V is used to calculate the vehicle's driving direction A=atan2(V_y,V_x), where V_x and V_y are the components of the displacement vector on the x-axis and y-axis respectively, and atan2 is the inverse tangent function used to calculate the angle; the principle of this formula is to calculate the direction angle of the displacement vector through the inverse tangent function. The specific steps are as follows: Use the atan2 function to calculate the angle between the x- and y-components of the displacement vector.

[0059] This angle indicates the direction in which the feature point moves, and thus reflects the direction in which the vehicle is traveling.

[0060] Combine the displacement vector V and the time interval (the time difference between two frames of images) TI to calculate the vehicle speed S = sqrt (V_x^2 + V_y^2) / TI, where TI is the time difference between two frames of images. The principle of this formula is to determine the speed by calculating the modulus of the displacement vector and dividing it by the time interval. The specific steps are as follows: Calculate the modulus of the displacement vector, which is sqrt(V_x^2+V_y^2).

[0061] Divide the module length by the time interval TI to obtain the speed S of the vehicle.

[0062] Through the above steps, the present invention realizes the function of calculating the driving direction and speed of the vehicle by using the position information of the feature points.

[0063] Step 6: predicting the vehicle position at the next moment according to the driving direction and speed; specifically including: Receive the vehicle's driving direction A and vehicle's driving speed S, and determine the current vehicle position coordinates C=(X_cur, Y_cur), where (X_cur, Y_cur) is the vehicle's position in the current frame; by receiving the vehicle's driving direction and speed as well as the current position coordinates, basic data is provided for subsequent calculations.

[0064] Based on the vehicle driving direction A, calculate the direction vector DIR=(cos(A), sin(A)), where cos and sin are cosine and sine operations respectively; the principle of this formula is to use trigonometric functions to convert angles into two-dimensional plane vectors. The specific steps are as follows: Calculate the cosine value cos(A) of the driving direction angle A, which represents the component of the direction vector on the x-axis.

[0065] Calculate the sine value sin(A) of the driving direction angle A, which represents the component of the direction vector on the y-axis.

[0066] By calculating the direction vector, the direction of travel can be converted to a unit vector on a two-dimensional plane.

[0067] Using the vehicle speed S and time interval TI, calculate the displacement distance , combined with the direction vector DIR, we get the displacement vector The principle of this formula is to calculate the displacement distance by multiplying the vehicle's speed S by the time interval TI. Combined with the direction vector DIR, the displacement vector DELTA can be further calculated. The specific steps are as follows: Calculate displacement distance .

[0068] Multiply the displacement distance by the direction vector to get the displacement vector .

[0069] By calculating the displacement distance and combining it with the direction vector, the displacement vector of the vehicle over a period of time can be determined.

[0070] Add the displacement vector DELTA to the current vehicle position coordinate C to obtain the predicted vehicle position at the next moment N=(X_cur+DELTA_X,Y_cur+DELTA_Y), where DELTA_X and DELTA_Y are the components of the displacement vector on the x-axis and y-axis respectively. The principle of this formula is to obtain the position coordinate at the next moment by adding each component of the displacement vector to the corresponding component of the current position coordinate. The specific steps are as follows: Calculate the x-axis component of the displacement vector .

[0071] Calculate the component of the displacement vector on the y-axis .

[0072] Add these components to the current position coordinates to obtain the position coordinates of the next moment N=(X_cur+DELTA_X,Y_cur+DELTA_Y).

[0073] Step 7: Update the distribution of feature points in the panoramic image in combination with the predicted position; specifically including: Receive the predicted vehicle position N=(X_next, Y_next) at the next moment, and obtain the feature point list L in the current panoramic image: L={(f1,D_1,X_1,Y_1),(f2,D_2,X_2,Y_2),...,(fm,D_m,X_m,Y_m)}; provides basic data for subsequent calculations by receiving the predicted vehicle position at the next moment and the list of feature points in the current panoramic image.

[0074] Based on the predicted vehicle position N at the next moment, calculate the position change of the vehicle relative to each feature point POS_CHANGE=(X_next-X_i,Y_next-Y_i), where (X_i,Y_i) is the position coordinate of the feature point fi in the current frame; the principle of this formula is to determine the position change by calculating the difference between the new position of the vehicle and the current position of the feature point. The specific steps are as follows: For each feature point fi, calculate its position change relative to the new position of the vehicle.

[0075] The component of the position change on the x-axis is POS_CHANGE_X=X_next-X_i.

[0076] The component of position change on the y-axis is POS_CHANGE_Y=Y_next-Y_i.

[0077] By calculating the position change of the vehicle relative to each feature point, the new position of the feature point in the global coordinate system can be determined.

[0078] Use the position change POS_CHANGE to update the new position coordinates (X'_i, Y'_i) of each feature point. The new position coordinate calculation formula is X'_i=X_i+POS_CHANGE_X, Y'_i=Y_i+POS_CHANGE_Y, where POS_CHANGE_X and POS_CHANGE_Y are the components of POS_CHANGE on the x-axis and y-axis respectively; the principle of this formula is to update the position of the feature point by adding the components of the position change to the current position coordinates of the feature point. The specific steps are as follows: Calculate the new position coordinate of the updated feature point on the x-axis X'_i=X_i+POS_CHANGE_X.

[0079] Calculate the new position coordinate of the updated feature point on the y-axis Y'_i=Y_i+POS_CHANGE_Y.

[0080] By adding the position change to the current position coordinates of the feature point, the new position of the feature point in the global coordinate system can be updated.

[0081] Remap the updated feature point position coordinates (X'_i, Y'_i) to the panoramic image to form an updated feature point list L_updated: L_updated={(f1,D_1,X'_1,Y'_1),(f2,D_2,X'_2,Y'_2),...,(fm,D_m,X'_m,Y'_m)}. By remapping the updated feature point position coordinates to the panoramic image, the correct distribution of the feature points in the new panoramic image can be ensured.

[0082] Step 8: Construct a three-dimensional path map based on the updated feature point distribution; specifically including: The updated feature point list L_updated is received, and the height value Z_i on the z-axis is calculated for each feature point; by calculating the height value of each feature point, the two-dimensional feature point can be expanded into the three-dimensional space.

[0083] Based on the position coordinates (X'_i, Y'_i, Z_i) of the feature points, a three-dimensional point cloud PC is established in the local coordinate system: PC={(X'_1,Y'_1,Z_1),(X'_2,Y'_2,Z_2),...,(X'_m,Y'_m,Z_m)}; Use a stereo vision algorithm (such as a binocular camera) or a depth sensor (such as LiDAR) to obtain the height value Z_i of each feature point. Combine the height value Z_i with the position coordinates (X'_i,Y'_i) of the feature point to form a three-dimensional coordinate (X'_i,Y'_i,Z_i).

[0084] Using the points in the three-dimensional point cloud PC, calculate the distance between adjacent feature points ED_ij=sqrt((X'_j-X'_i)^2+(Y'_j-Y'_i)^2+(Z_j-Z_i)^2), where (i,j) is a pair of adjacent feature points; the principle of this formula is based on the Euclidean distance formula in three-dimensional space, which is used to calculate the straight-line distance between two points. The specific steps are as follows: For each pair of adjacent feature points (i, j), calculate their coordinate differences on the x-axis, y-axis, and z-axis.

[0085] Use the above formula to calculate the three-dimensional distance ED_ij between two points.

[0086] According to the distance ED_ij, construct the edge connecting the adjacent feature points to form a three-dimensional path graph G_path={(f1,f2,ED_12),(f2,f3,ED_23),...,(fn-1,fn,ED_n-1n)}, where ED_ij represents the distance between the feature points fi and fj. For each pair of adjacent feature points, record the distance ED_ij between them. Combine these feature points and their distances into a tuple (f_i,f_j,ED_ij), which represents the connection and distance between the feature points fi and fj. Collect all tuples to form a three-dimensional path graph G_path. By constructing a three-dimensional path graph, the feature points and their distance relationship can be visualized.

[0087] Step 9: Continuously monitor and correct the three-dimensional path map; specifically including: Based on the three-dimensional path graph G_path, an initial threshold TH=TH_0 is set for detecting path changes. For each edge e_ij in the path graph belonging to the G_path set, its edge length ED(e_ij)=dist_ij is calculated; by setting the initial threshold and calculating the initial length of each edge, a benchmark can be provided for subsequent path change detection.

[0088] Update the position coordinates (X'_i, Y'_i, Z_i) of the feature points through sensor data, and recalculate the new length NEW_ED (e_ij) of each side: NEW_ED(e_ij)=sqrt((X'_j-X'_i)^2+(Y'_j-Y'_i)^2+(Z_j-Z_i)^2); By using sensor data to update the position coordinates of the feature points and recalculating the new lengths of each edge, the actual position changes of the feature points can be dynamically reflected. The principle of this formula is based on the Euclidean distance formula in three-dimensional space, which is used to calculate the straight-line distance between two points. The specific steps are as follows: Update the 3D coordinates (X'_i, Y'_i, Z_i) of each feature point using the sensor data.

[0089] For each edge e_ij, recalculate its new length NEW_ED(e_ij) using the above formula.

[0090] Compare the new length NEW_ED(e_ij) with the original length ED(e_ij). If |NEW_ED(e_ij)-ED(e_ij)|>TH, mark it as an exception and record the exception information in the exception information list AN: AN={(e_1,|NEW_ED(e_1)-ED(e_1)|),...,(e_n,|NEW_ED(e_n)-ED(e_n)|)}; By comparing the old and new edge lengths and detecting abnormal changes, abnormal situations in the path graph can be discovered in a timely manner.

[0091] The process of data analysis and anomaly detection is as follows: For each edge e_ij, calculate its edge length difference |NEW_ED(e_ij)-ED(e_ij)|.

[0092] If the difference is greater than the set threshold TH, the edge is marked as abnormal and its difference is recorded in the abnormal information list AN.

[0093] According to the abnormal information in the abnormal information list AN, the marked edges are corrected and the edge length is updated using the weighted average method. The formula is: , where W_old and W_new are the old weight and the new weight respectively, and W_old+W_new=1, CORRECTED_ED is the corrected edge length, W is the weight, W_old: old weight (OldWeight), indicating the weight of the original edge length. W_new: new weight (NewWeight), indicating the weight of the new edge length. W_old+W_new=1: The sum of the weights is 1, ensuring the effectiveness of the weighted average.

[0094] The principle of this formula is to calculate the corrected side length by combining the old side length and the new side length through the weighted average method. The specific steps are as follows: According to the abnormal information in the abnormal information list AN, the edge that needs to be corrected is determined.

[0095] For each edge e_ij marked as abnormal, the corrected edge length CORRECTED_ED(e_ij) is calculated using the above formula.

[0096] Make sure the weights W_old and W_new satisfy W_old + W_new = 1 to ensure the validity of the weighted average.

[0097] In summary, the present invention improves the accuracy of feature point extraction by enhancing the contrast of images collected under low-light conditions, and significantly improves the vehicle positioning accuracy by using an improved feature point matching algorithm. By updating the distribution of feature points in the panoramic image in combination with the predicted position, the real-time generation and dynamic correction of the three-dimensional path map is realized, thereby effectively enhancing the adaptability and reliability of the system in complex and changing environments. This not only solves the problem of performance degradation of traditional methods in low-light environments, but also greatly improves the overall performance of the intelligent driving system, ensuring driving safety and the accuracy of path planning.

[0098] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path, characterized in that: The following steps are involved: Collect multi-view low-light images of the vehicle's surroundings; Based on the multi-view low-light images, adjusting the brightness of each view image to enhance contrast, and stitching the adjusted images to form a panoramic image; Marking feature points in the panoramic image and recording their position information, and calculating the driving direction and speed of the vehicle using the position information of the feature points; Predicting the vehicle position at the next moment according to the driving direction and speed, and updating the distribution of feature points in the panoramic image in combination with the predicted position; A three-dimensional path map is constructed based on the updated feature point distribution, and the three-dimensional path map is continuously monitored and corrected.

2. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 1, characterized in that: The collecting of multi-view low-light images of the vehicle's surrounding environment includes: Activate multiple cameras set up around the vehicle, each camera is configured with enhanced night vision mode; Based on the night vision enhancement mode, adjust the exposure time of each camera , where ISO is the sensitivity and EV is the ambient brightness value; Take images using the adjusted exposure time and record the timestamp of each frame , where t_start is the start time, n is the number of frames, and TI is the time interval.

3. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 2, characterized in that: The step of adjusting the brightness of each viewing angle image to enhance the contrast of the multi-view low-light image includes: Receives a timestamped image and calculates the average brightness value of each frame , where B(x,y) is the brightness value of the pixel, W and H are the image width and height respectively; According to the average brightness value L_avg, determine the gain coefficient G=G_max-(L_avg / R) of each image, where G_max is the maximum gain value and R is the reference brightness value; The gain coefficient G is used to adjust the brightness of each frame of the image, and the updated brightness value ; Remap the adjusted image brightness value to the range of 0 to 255. The mapping formula is: , where MIN_B_new and MAX_B_new are the minimum and maximum brightness values ​​of the adjusted image, respectively, and B_final is the final brightness value.

4. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 3, characterized in that: The step of stitching the adjusted images to form a panoramic image includes: Receive the adjusted image corresponding to the mapped brightness value B_final(x,y), and calculate the feature point set F={p1,p2,...,pn} of the overlapping area for each frame of the image; Based on the feature point set F, the transformation matrix M=[abc;def] is determined using the matching point pairs between adjacent images, where a to f are transformation parameters, by minimizing the error To solve, I1 and I2 are two adjacent images; Apply the transformation matrix M to each image, convert it to the global coordinate system, and get the new position , ensure that all images are aligned in the same coordinate system, P_transformed is the aligned image; The aligned images P_transformed(x,y) are fused and the final pixel value is calculated by weighted average method: , where w_i is the weight of the i-th image and P(x,y) is the panoramic image.

5. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 4, characterized in that: The marking of feature points in the panoramic image and recording of their position information includes: Receive the fused panoramic image P(x,y) and calculate the gradient amplitude GRAD=sqrt(dx^2+dy^2) of each frame, where dx and dy are the gradients in the x and y directions respectively; Based on the gradient amplitude GRAD, determine the local maximum point as a feature point set F={f1,f2,...,fm}, where the position of each feature point fi is represented by the coordinates (X_i,Y_i); For each feature point fi in the feature point set F, calculate its descriptor D_i=SUM((P(x+x_i,y+y_i)-AVG_P)^2), where AVG_P is the average value of the pixel values ​​in the surrounding area, and (X_i,Y_i) is the coordinate of the center point; The feature points and their descriptors D_i are stored in the database together with the location information (X_i, Y_i) to form a feature point list: L={(f1,D_1,X_1,Y_1),(f2,D_2,X_2,Y_2),...,(fm,D_m,X_m,Y_m)}.

6. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 5, characterized in that: The method of calculating the driving direction and speed of the vehicle by using the position information of the feature points includes: Receive the feature points and their position information (X_i, Y_i) in the feature point list L, and match feature point pairs P between consecutive frames: P={(f1_t1,f1_t2),(f2_t1,f2_t2),...,(fn_t1,fn_t2)}, where t1 and t2 represent different time points; Based on the feature point pair P, calculate the displacement vector V=(X_2-X_1,Y_2-Y_1) of each feature point between the two frames, where (X_1,Y_1) and (X_2,Y_2) are the position coordinates of the same feature point at two time points respectively; The vehicle travel direction A=atan2(V_y,V_x) is calculated using the displacement vector V, where V_x and V_y are the components of the displacement vector on the x-axis and y-axis respectively, and atan2 is an inverse tangent function; Combining the displacement vector V and the time interval TI, the vehicle speed S=sqrt(V_x^2+V_y^2) / TI is calculated, where TI is the time difference between the capture of two frames of images.

7. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 6, characterized in that: The predicting the vehicle position at the next moment according to the driving direction and speed includes: Receive the vehicle driving direction A and the vehicle driving speed S, and determine the current vehicle position coordinates C=(X_cur, Y_cur), where (X_cur, Y_cur) is the position of the vehicle in the current frame; Based on the vehicle driving direction A, calculate the direction vector DIR=(cos(A), sin(A)), where cos and sin are cosine and sine operations respectively; Using the vehicle speed S and time interval TI, calculate the displacement distance , combined with the direction vector DIR, we get the displacement vector ; The displacement vector DELTA is added to the current vehicle position coordinate C to obtain the predicted vehicle position at the next moment N=(X_cur+DELTA_X,Y_cur+DELTA_Y), where DELTA_X and DELTA_Y are the components of the displacement vector on the x-axis and y-axis respectively.

8. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 7, characterized in that: The updating of the distribution of feature points in the panoramic image in combination with the predicted position includes: Receive the predicted vehicle position N=(X_next, Y_next) at the next moment, and obtain the feature point list L in the current panoramic image: L={(f1,D_1,X_1,Y_1),(f2,D_2,X_2,Y_2),...,(fm,D_m,X_m,Y_m)}; Based on the predicted vehicle position N at the next moment, calculate the position change POS_CHANGE of the vehicle relative to each feature point = (X_next-X_i, Y_next-Y_i), where (X_i, Y_i) is the position coordinate of the feature point fi in the current frame; Use the position change POS_CHANGE to update the new position coordinates (X'_i, Y'_i) of each feature point. The new position coordinate calculation formula is X'_i=X_i+POS_CHANGE_X, Y'_i=Y_i+POS_CHANGE_Y, where POS_CHANGE_X and POS_CHANGE_Y are the components of POS_CHANGE on the x-axis and y-axis respectively; Remap the updated feature point position coordinates (X'_i, Y'_i) to the panoramic image to form an updated feature point list L_updated: L_updated={(f1,D_1,X'_1,Y'_1),(f2,D_2,X'_2,Y'_2),...,(fm,D_m,X'_m,Y'_m)}.

9. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 8, characterized in that: The constructing of a three-dimensional path map according to the updated feature point distribution includes: Receive the updated feature point list L_updated, and calculate the height value Z_i of each feature point on the z-axis; Based on the position coordinates (X'_i, Y'_i, Z_i) of the feature points, a three-dimensional point cloud PC is established in the local coordinate system: PC={(X'_1,Y'_1,Z_1),(X'_2,Y'_2,Z_2),...,(X'_m,Y'_m,Z_m)}; Using the points in the three-dimensional point cloud PC, calculate the distance between adjacent feature points ED_ij=sqrt((X'_j-X'_i)^2+(Y'_j-Y'_i)^2+(Z_j-Z_i)^2), where (i, j) is an adjacent feature point pair; According to the distance ED_ij, edges connecting adjacent feature points are constructed to form a three-dimensional path graph G_path={(f1,f2,ED_12),(f2,f3,ED_23),...,(fn-1,fn,ED_n-1n)}, where ED_ij represents the distance between feature points fi and fj.

10. The method for synchronous positioning of a panoramic image of a driving environment and real-time generation of a three-dimensional path according to claim 9, characterized in that: The continuously monitoring and correcting the three-dimensional path map comprises: Based on the three-dimensional path graph G_path, an initial threshold TH=TH_0 is set for detecting path changes, and for each edge e_ij in the path graph belonging to the G_path set, its edge length ED(e_ij)=dist_ij is calculated; Update the position coordinates (X'_i, Y'_i, Z_i) of the feature points through sensor data, and recalculate the new length NEW_ED (e_ij) of each side: NEW_ED(e_ij)=sqrt((X'_j-X'_i)^2+(Y'_j-Y'_i)^2+(Z_j-Z_i)^2); Compare the new length NEW_ED(e_ij) with the original length ED(e_ij). If |NEW_ED(e_ij)-ED(e_ij)|>TH, mark it as an exception and record the exception information in the exception information list AN: AN={(e_1,|NEW_ED(e_1)-ED(e_1)|),...,(e_n,|NEW_ED(e_n)-ED(e_n)|)}; According to the abnormal information in the abnormal information list AN, the marked edges are corrected and the edge length is updated using the weighted average method. The formula is: , where W_old and W_new are the old weight and new weight respectively, and satisfy W_old+W_new=1, CORRECTED_ED is the corrected edge length, and W is the weight.

Images