Parking positioning optimization method and device, equipment and storage medium

Abstract

The invention discloses a parking positioning optimization method and device, equipment and a storage medium, and relates to the technical field of vehicle parking, and the method comprises the steps: obtaining vehicle environment image information, generating a depth map according to the vehicle environment image information, carrying out the optical flow tracking processing according to the depth map, and obtaining an optical flow tracking result, a preset algorithm is adopted for correction based on the optical flow tracking result, a parking positioning result is obtained, an environment image is obtained through a camera, a depth map is generated, optical flow tracking is executed, the tracking result is corrected through the preset algorithm, the accuracy of parking positioning is improved, and accurate parking positioning is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle parking, and in particular to a parking positioning optimization method, device, equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, autonomous driving and robotics have become hot research areas. In these fields, accurate positioning technology is crucial for autonomous vehicles or robots because it determines whether they can accurately perceive the surrounding environment and make correct decisions. Binocular camera technology has become a commonly used positioning technology because it can calculate depth information and the movement of feature points through image analysis.

[0003] Existing parking positioning algorithms based on optical flow tracking usually use a camera to capture images of the vehicle's surroundings, perform optical flow tracking on feature points in the images, estimate their motion between consecutive frames, analyze the optical flow tracking results, identify occluded feature points, and use a depth estimation algorithm to make corrections to compensate for the effects of occlusion.

[0004] However, there are still some problems with the existing parking positioning algorithms based on the parallax and optical flow tracking of binocular cameras. First, due to the influence of factors such as light and weather, the images captured by the camera may be noisy, which will affect the accuracy of parallax calculation and optical flow tracking. Secondly, in complex environments, such as dense buildings or vegetation, feature points may be blocked, which will cause inaccurate results of optical flow tracking and thus affect the accuracy of positioning. In addition, the existing algorithms often require a lot of computing resources when processing blocked feature points, which may cause real-time problems in practical applications. There is an urgent need to improve the accuracy of parking positioning by optimizing the algorithm. Summary of the invention

[0005] The main purpose of this application is to provide a parking positioning optimization method, device, equipment and storage medium, aiming to solve the technical problem of how to improve the accuracy of parking positioning.

[0006] To achieve the above objectives, the present application proposes a parking positioning optimization method, the method comprising:

[0007] Acquire vehicle environment image information;

[0008] Generate a depth map according to the vehicle environment image information;

[0009] Performing optical flow tracking processing according to the depth map to obtain an optical flow tracking result;

[0010] Based on the optical flow tracking result, a preset algorithm is used to perform correction to obtain a parking positioning result.

[0011] In one embodiment, the step of performing optical flow tracing processing according to the depth map to obtain an optical flow tracing result includes:

[0012] Based on the depth map, obtaining image feature points;

[0013] The optical flow tracing result is obtained according to the image feature points through a preset optical flow tracing energy model.

[0014] In one embodiment, before the step of obtaining the optical flow tracing result according to the image feature points by using a preset optical flow tracing energy model, the step includes:

[0015] Establish an initial optical flow tracing energy model;

[0016] The initial optical flow tracing energy model is trained according to the vehicle environment image information to obtain a preset optical flow tracing energy model.

[0017] In one embodiment, the step of obtaining the optical flow tracing result according to the image feature points by using a preset optical flow tracing energy model comprises:

[0018] According to the image feature points, get the current frame and the nearest key frame;

[0019] Based on the current frame and the most recent key frame, a first image, a first time, a second image, a second time, a weight, and a photometric affine parameter are obtained, wherein the first image is an image of the most recent key frame, the first time is an exposure time of the most recent key frame, the second image is an image of the current frame, and the second time is an exposure time of the current frame;

[0020] The first image, the first time, the second image, the second time, the weight and the photometric affine parameter are input into the preset optical flow tracing energy model to obtain the optical flow tracing result.

[0021] In one embodiment, the step of correcting the optical flow tracking result using a preset algorithm to obtain a parking positioning result includes:

[0022] Identify the optical flow tracking result to obtain blocked feature points;

[0023] Using a depth estimation algorithm to correct the blocked feature points to obtain corrected feature points;

[0024] The parking positioning result is obtained by performing calculation based on the corrected feature points according to the synchronous positioning algorithm.

[0025] In one embodiment, before the step of correcting the blocked feature points using a depth estimation algorithm to obtain a parking positioning result, the step includes:

[0026] The confidence propagation method is used to iteratively optimize the depth estimation algorithm;

[0027] The method of using confidence propagation to iteratively optimize the depth estimation algorithm includes:

[0028] Set the maximum number of iterations and relaxation factor;

[0029] Determine the positional relationship between the nearest key frame and the non-key frame of the occluded feature point by using the relaxation factor;

[0030] Use grayscale block matching to estimate the rough position of non-key frames and give the uncertainty;

[0031] Comparing the positions and uncertainties of the key frame and the non-key frame to obtain depth uncertainty;

[0032] Until the depth uncertainty is less than a preset threshold or the number of iterations reaches the maximum number of iterations, the depth estimation algorithm is obtained.

[0033] In one embodiment, the step of calculating based on the corrected feature points according to a synchronous positioning algorithm to obtain a parking positioning result includes:

[0034] Based on the corrected feature points, the depth information and uncertainty of the updated key frame are obtained;

[0035] The depth information and the uncertainty are filtered through a preset filter to obtain a parking positioning result.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a parking positioning optimization device, which includes:

[0037] An acquisition module, used for acquiring vehicle environment image information;

[0038] A processing module, used for generating a depth map according to the vehicle environment image information;

[0039] The processing module is further used to perform optical flow tracking processing according to the depth map to obtain an optical flow tracking result;

[0040] The result module is used to perform correction based on the optical flow tracking result by using a preset algorithm to obtain a parking positioning result.

[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes a parking positioning optimization device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the parking positioning optimization method described above.

[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the parking positioning optimization method described above are implemented.

[0043] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the parking positioning optimization method as described above are implemented.

[0044] The present application obtains vehicle environment image information, generates a depth map based on the vehicle environment image information, performs optical flow tracking based on the depth map, obtains optical flow tracking results, uses a preset algorithm to perform corrections based on the optical flow tracking results, obtains parking positioning results, obtains environmental images through a camera, generates a depth map, performs optical flow tracking, and uses a preset algorithm to correct the tracking results, thereby improving the accuracy of parking positioning and achieving precise parking positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0046] Figure 1 A schematic diagram of the process of the first embodiment of the parking positioning optimization method of the present application;

[0047] Figure 2 A schematic diagram of the flow chart of the second embodiment of the parking positioning optimization method of the present application;

[0048] Figure 3 A schematic diagram of the flow chart of the third embodiment of the parking positioning optimization method of the present application;

[0049] Figure 4 This is a schematic diagram of the module structure of the parking positioning optimization device according to an embodiment of the present application;

[0050] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the parking positioning optimization method in the embodiment of the present application.

[0051] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0053] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0054] The main solution of the embodiment of the present application is: by acquiring vehicle environment image information, generating a depth map based on the vehicle environment image information, performing optical flow tracking processing based on the depth map to obtain optical flow tracking results, and using a preset algorithm to perform corrections based on the optical flow tracking results to obtain parking positioning results.

[0055] Based on this, the present application embodiment provides a parking positioning optimization method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the parking positioning optimization method of the present application.

[0056] In this embodiment, the parking location optimization method includes steps S10 to S40:

[0057] Step S10, obtaining vehicle environment image information.

[0058] It should be noted that the binocular camera installed on the vehicle is used to capture the dynamic and static environment around the vehicle with high resolution and high frame rate. In this embodiment, two CMOS cameras with a resolution of 1280*720 are used on the left and right, the shooting frequency is 30fps, the focal length is 2.8mm, and the shooting distance is 2-5 meters to capture images of the vehicle's surrounding environment. These cameras need to have good dynamic range and low noise performance to ensure that clear images can be obtained under different lighting and weather conditions, and then the captured original images are subjected to denoising, contrast enhancement, white balance adjustment and other operations to improve image quality and reduce the complexity of subsequent processing. Through image processing algorithms, key features in the image, such as road signs, lane lines, traffic signals, pedestrians, vehicles, etc., are identified and extracted. These features are essential for understanding the environment in which the vehicle is located. In addition to visual information, modern vehicles may also be equipped with other sensors, such as radar, laser radar (LiDAR), ultrasonic sensors, etc. By fusing the data of these sensors, more comprehensive and reliable environmental information can be obtained.

[0059] Step S20: generating a depth map according to the vehicle environment image information.

[0060] It should be noted that the binocular camera on the vehicle will capture two images of the same scene, and these images have parallax due to the different viewing angles of the camera. Parallax refers to the horizontal pixel difference of the same object in two images, which is the basis for calculating depth. By calculating these parallaxes, a corresponding depth value can be generated for each pixel in the image, thereby creating a depth map. In this embodiment, the matching window size is set to 5*5 pixels, the matching step size is 1 pixel, the threshold is 0.05, and the maximum parallax is 192 pixels.

[0061] In the process of generating depth maps, advanced disparity calculation algorithms such as semi-global matching (SGM) are used. These algorithms can effectively extract disparity information and generate accurate depth maps under different ambient lighting and complex scenes. Depth maps can not only reflect the distance of objects, but also provide important information about the geometric structure of the scene. In addition, the generated depth map will be further used for subsequent image processing and analysis tasks. For example, in optical flow tracking, depth maps can help more accurately estimate the motion trajectory of feature points in consecutive frames. In obstacle detection and avoidance, depth information can be used to determine the specific position and distance of obstacles, so as to carry out effective path planning. The generation of depth maps also needs to take into account the computational efficiency and real-time performance of the algorithm to ensure that the system can still respond quickly and make accurate decisions when the vehicle is driving at high speed. At the same time, the robustness of the system is also very important, and it needs to be able to handle uncertainties and errors caused by occlusion, lighting changes or dynamic scenes.

[0062] Step S30, performing optical flow tracking processing according to the depth map to obtain an optical flow tracking result.

[0063] It should be noted that optical flow tracking is a key technology for dynamic analysis of feature points in the vehicle's surroundings based on depth maps. By tracking feature points in the depth map, the system can estimate the movement of these points in consecutive video frames, thereby obtaining information about the vehicle's motion relative to the surrounding environment. This process involves not only the detection of feature points, but also the analysis of their continuity in the time series. When performing optical flow tracking, the system uses image processing algorithms, such as the Lucas-Kanade method or more advanced optical flow algorithms, to predict the position of feature points in the next frame. These algorithms take into account the principle of brightness constancy in image sequences, that is, if the brightness of an object's feature points remains unchanged in consecutive frames, then its direction and speed of movement in the image can be estimated by comparing the difference between the current frame and the next frame. The accuracy of optical flow tracking directly affects the accuracy of vehicle positioning and navigation. Therefore, the system needs to be able to handle a variety of complex situations, such as lighting changes, occlusion, fast motion, etc. In order to improve robustness, modern systems may use multi-scale analysis, feature point optimization selection, or machine learning techniques to enhance the performance of optical flow tracking. In addition, the analysis of optical flow tracking results can also be used for more advanced applications, such as scene understanding, behavior prediction, and interaction recognition. For example, in autonomous driving, by analyzing the optical flow of pedestrians or vehicles, the system can predict their movement trajectories and make safer driving decisions. The system combines other sensor data, such as radar or lidar, to provide additional depth and motion information, thereby improving the overall tracking accuracy and reliability. Through this multi-sensor fusion approach, optical flow tracking can become a powerful and reliable component in autonomous driving systems.

[0064] Step S40: Correct the optical flow tracking result using a preset algorithm to obtain a parking positioning result.

[0065] It should be noted that correction based on optical flow tracking results is an important part of ensuring the accuracy of parking positioning. Optical flow tracking provides motion information of feature points in continuous frames, but may be affected by occlusion, noise and illumination changes, resulting in tracking errors. In order to improve the accuracy of positioning, the system uses a preset algorithm to finely correct these results. The correction algorithm includes occlusion detection and processing mechanisms. For example, by analyzing the motion trajectory and depth information of feature points, the system can identify occluded feature points and adopt specific strategies to handle these points. For example, the possible position and motion of occluded points can be estimated by interpolation or extrapolation methods in space and time using the information of surrounding unoccluded feature points. In addition, the correction algorithm also integrates machine learning techniques, such as support vector machines (SVM) or neural networks, to automatically identify and adapt to different tracking scenarios and error modes. Through training data sets, these models can learn how to most effectively correct optical flow tracking results in specific situations. The correction process also includes uncertainty assessment. The system assigns a confidence score to each corrected feature point to reflect the reliability of the correction result. This score can be used to weight subsequent positioning calculations to ensure that only high-quality data is used for parking positioning. Finally, the corrected feature point information will be integrated into the simultaneous localization and mapping (SLAM) algorithm to obtain the final parking positioning result to achieve accurate vehicle positioning. In this way, the system can provide accurate parking navigation even in complex parking environments, thereby improving parking efficiency and safety.

[0066] The present embodiment provides a parking positioning optimization method, which obtains vehicle environment image information, generates a depth map based on the vehicle environment image information, performs optical flow tracking processing based on the depth map to obtain an optical flow tracking result, and uses a preset algorithm to perform correction based on the optical flow tracking result to obtain a parking positioning result. The environment image is obtained through a binocular camera, a depth map is generated, optical flow tracking is performed, and the tracking result is corrected using a preset algorithm, thereby improving the accuracy of parking positioning and achieving precise parking positioning.

[0067] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2 The parking location optimization method step S30 further includes steps S301 to S302:

[0068] Step S301, obtaining image feature points based on the depth map.

[0069] It should be noted that image feature points based on depth maps are a key component in autonomous driving and computer vision. Depth maps provide depth information of each pixel in the scene relative to the camera, which enables the system to distinguish between foreground and background and identify the outline and shape of objects. By analyzing significant changes in the depth map, image feature points can be accurately extracted, which are usually edges, corners, or areas with obvious texture changes.

[0070] Specifically, feature point extraction is usually implemented using algorithms such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Features), or ORB (Oriented FAST and Rotated BRIEF). These algorithms can detect key points in an image, calculate their orientation and scale, and generate descriptors that are robust to image rotation, scaling, and partial occlusion. The extracted feature points are not only used to enhance the recognizability of the image, but also play an important role in subsequent tasks such as image matching, target tracking, and 3D reconstruction. For example, in stereo vision systems, feature point matching can be used to calculate the 3D structure of the scene; in optical flow estimation, they can provide information about the motion of objects; in robot navigation, feature points can be used for environmental mapping and path planning. In addition, the stability and repeatability of feature points are crucial for long-term operation of systems. Therefore, feature point extraction algorithms need to be able to adapt to different environmental conditions, including lighting changes, dynamic scenes, and noise interference. With the development of machine learning and deep learning technologies, feature point extraction methods are also constantly improving. The feature representation learned through training data sets can further improve the recognition accuracy of feature points and the robustness of the system.

[0071] Step S302, obtaining an optical flow tracing result according to the image feature points through a preset optical flow tracing energy model.

[0072] It should be noted that before using the preset optical flow tracking energy model, an initial optical flow tracking energy model is first established, and the initial optical flow tracking energy model is trained according to the vehicle environment image information to obtain the preset optical flow tracking energy model. Specifically, establishing the initial optical flow tracking energy model is the first step to achieve accurate optical flow estimation. The model defines how to quantify the motion of feature points in the image sequence. This model is usually based on physical and geometric principles, taking into account the consistency of the brightness of feature points in the image sequence, as well as their continuity in space and time. Based on the initial model, it is crucial to train the model using the vehicle environment image information. This process involves collecting a large amount of image data and using supervised learning or unsupervised learning algorithms to adjust the model parameters so that it can more accurately reflect the optical flow in the real world. During the training process, the model will learn how to extract the motion information of feature points from the image sequence and optimize the robustness of occlusion, illumination changes and other interference factors. The obtained preset optical flow tracking energy model will have high adaptability and accuracy. This model can handle complex dynamic scenes, such as fast-moving vehicles, pedestrians or other obstacles, while maintaining sensitivity to subtle movements. Through training, the model is also able to recognize and adapt to different environmental conditions, such as different light levels and weather conditions, thereby improving performance in a variety of real-world driving situations.

[0073] Furthermore, according to the image feature points, the current frame and the nearest key frame are obtained. By extracting and matching the image feature points, the correspondence between the current frame and the nearest key frame can be determined. In this embodiment, the optical flow tracking window size is set to 7*7 pixels, the search range is ±5 pixels, the minimum matching cost is 10, and the maximum number of iterations is 100. Key frames usually contain significant environmental changes or motion information, and feature point matching provides an accurate measure of the motion of objects between two frames. This process is crucial for estimating vehicle motion, environmental mapping, and event detection, and provides basic data for autonomous driving and video analysis.

[0074] Further, based on the current frame and the nearest key frame, the first image, the first time, the second image, the second time, the weights and the photometric affine parameters are obtained, wherein the first image is the image of the nearest key frame, the first time is the exposure time of the nearest key frame, the second image is the image of the current frame, and the second time is the exposure time of the current frame. Based on the analysis of the current frame and the nearest key frame, the first image and the first time, as well as the second image and the second time, are obtained in order to establish an accurate time series model in optical flow tracking. The first image and the first time represent the visual information and its exposure conditions of the nearest key frame, while the second image and the second time represent the corresponding information of the current frame. The weights play a role in balancing the importance of different image information in this process, ensuring that the brightness changes and exposure differences in the image sequence are considered when matching feature points and estimating optical flow. Photometric affine parameters are mathematical models that describe the photometric changes between two images, and they can capture the changes in pixel intensity caused by changes in lighting conditions or object motion. These parameters are crucial for accurately estimating the motion of feature points between two frames at different exposure times, because they allow the model to correct for visual differences caused by these factors. By combining these elements, the system is able to more accurately predict the movement of feature points over time, providing autonomous vehicles with detailed information about the dynamic changes in the surrounding environment.

[0075] Furthermore, the first image, the first time, the second image, the second time, the weight and the photometric affine parameter are input into the preset optical flow tracking energy model to obtain the optical flow tracking result. The dynamic photometric error energy is obtained according to the following formula:

[0076]

[0077] The point set p represents a series of map points where the environment image can be observed in the current frame, and the point set N p represents the eight-point Pattern defined in , and p~ represents the projection point in the Pattern. i represents the image of the most recent key frame, I represents the image of the current frame, τ i and τ j Represent the exposure time of the most recent key frame and the current frame respectively, and ω represents the weight for downgrading high image gradients.

[0078] Based on the binocular camera, two pairs of photometric affine parameters a are introduced L , b L and a R 、a L , the static residual energy of the left and right cameras is obtained based on the following formula:

[0079]

[0080] in Indicates that the right camera takes an image. Represents the image taken by the left camera. The energy function for obtaining the final optical flow tracking is:

[0081]

[0082] Combining the above and below equations, we get the final optical flow tracking energy.

[0083] This embodiment obtains image feature points based on the depth map, obtains optical flow tracking results based on the image feature points through a preset optical flow tracking energy model, and optimizes parking positioning and navigation.

[0084] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 3 The parking location optimization method step S40 further includes steps S401 to S404:

[0085] Step S401, identifying the optical flow tracking result to obtain the blocked feature points.

[0086] It should be noted that by comparing the photometric changes of feature points in consecutive frames, we detect brightness inconsistencies or sudden changes, which may be an indication of occlusion. Combined with the depth map information, we analyze the depth changes of feature points. If the depth changes suddenly or does not match the surrounding environment, it indicates that the feature point is occluded, and the position of the occluded feature point is obtained.

[0087] Specifically, before calculating the photometric error of each pair of corresponding pixels, it is first checked whether the 3D point is occluded by the observed key frame. To this end, for the pixel with inverse depth in the current frame, it is first mapped to the three-dimensional space according to the camera intrinsic parameter to obtain its three-dimensional point. Then, according to the camera pose transformation between the current frame and the reference frame, the point is projected to the camera coordinates of the reference frame to obtain. Then it is further projected to the reference frame to obtain the 2D pixel on the reference frame. In the binocular camera system, the inverse depth extracted from the corresponding position in the depth image of the reference frame obtained after binocular matching, the occlusion judgment threshold is set to 0.5, if p, [z] (that is, the z-axis coordinate of the 3D point p) is greater than the occlusion judgment threshold, it means that there is another point closer to the camera than p, that is, p, is occluded by another point, which is regarded as the feature point occluded. This method can effectively identify the occluded feature points for subsequent processing.

[0088] Step S402: Correct the blocked feature points using a depth estimation algorithm to obtain corrected feature points.

[0089] It should be noted that the correction of occluded feature points is achieved through depth estimation algorithms, which is crucial to maintaining scene continuity and accuracy. Depth estimation algorithms use multi-frame image information combined with occlusion detection mechanisms to infer the potential position of occluded feature points in three-dimensional space. These algorithms may be based on physical models, use statistical methods, or apply machine learning techniques, especially deep learning methods, to improve the accuracy of estimation. During the correction process, the system considers the persistence and spatial distribution of occlusions, and predicts the occluded area by analyzing the surrounding unoccluded feature points and the geometric structure of the scene. In addition, the fusion of data from other sensors, such as lidar or stereo vision data, can further enhance the robustness of depth estimation. The corrected feature points are reintegrated into the optical flow model to optimize motion estimation and scene understanding. This process is crucial for environmental perception of autonomous vehicles, path planning for robot navigation, and scene rendering in augmented reality applications, ensuring high navigation accuracy and environmental adaptability even when vision is limited.

[0090] Furthermore, the confidence propagation method is used to iteratively optimize the depth estimation algorithm. Confidence propagation is an iterative optimization technique that improves the accuracy of depth estimation by exchanging information between feature points. This method considers the relationship and consistency between feature points and gradually adjusts the depth estimation value of each point through an iterative process until the depth map of the entire scene reaches a stable state, thereby enhancing the reliability of depth information and the overall consistency of the scene.

[0091] Further, the maximum number of iterations and relaxation factor are set, and the positional relationship between the nearest key frame and the non-key frame of the occluded feature point is determined by the relaxation factor. The rough position of the non-key frame is estimated by grayscale block matching, and the uncertainty is given. The position and uncertainty of the key frame and the non-key frame are compared to obtain the depth uncertainty, until the depth uncertainty is less than the preset threshold or the number of iterations reaches the maximum number of iterations, and the depth estimation algorithm is obtained. In the depth estimation algorithm, setting the maximum number of iterations and relaxation factors is an important parameter to ensure the convergence and efficiency of the algorithm. The maximum number of iterations limits the maximum number of loops that the algorithm attempts to optimize the problem, prevents infinite loops and improves computational efficiency. The relaxation factor is used to control the amplitude of the update step in each round of iteration to avoid excessive updates that cause the algorithm to be unstable or miss the optimal solution. In this embodiment, the maximum number of iterations of confidence propagation is set to 100, and the relaxation factor is 0.95. Through the relaxation factor, the algorithm can smoothly adjust the positional relationship between the nearest key frame and the non-key frame of the feature point to achieve fine adjustment of the position. Using grayscale block matching, the algorithm can estimate the rough position of the feature point in the non-key frame, and at the same time calculate an uncertainty to reflect the credibility of the estimate. The algorithm then compares the positions and uncertainties of feature points in keyframes and non-keyframes, and uses this information to comprehensively evaluate the uncertainty of depth. During the iteration process, the depth estimate is continuously updated until a preset uncertainty threshold is reached, indicating that the estimate is accurate enough, or the number of iterations reaches an upper limit, ensuring that the algorithm is completed within a limited number of steps. This approach not only improves the accuracy of depth estimation, but also enhances the algorithm's robustness to occlusion and noise.

[0092] Step S403: Calculate based on the corrected feature points according to the synchronous positioning algorithm to obtain a parking positioning result.

[0093] Furthermore, based on the corrected feature points, the depth information and uncertainty of the updated keyframe are obtained. During the update process, the algorithm considers the consistency of the new feature point position with the previous estimate, and integrates multiple frames of data through weighted averaging or more complex fusion strategies to reduce random errors and improve the accuracy of depth estimation. At the same time, the uncertainty assessment takes into account the reliability of observations and the stability of predictions to ensure that only high-quality data is used for keyframe updates. In addition, the update mechanism also needs to be able to handle rapid changes in dynamic scenes, such as the movement of vehicles or pedestrians, to ensure that the depth information of the keyframe can reflect the latest state of the scene in a timely manner. This continuous update process is crucial to achieving real-time and dynamic three-dimensional environment perception, especially in applications such as autonomous driving and robotic vision, where accurate depth information and reasonable uncertainty assessment are the basis for safe navigation and decision-making.

[0094] Further, the depth information and uncertainty are passed through a preset filter to obtain a parking positioning result. The depth information and uncertainty are input into a preset filter, such as a Kalman filter or a particle filter. In this embodiment, the initial covariance matrix of the Kalman filter is set to a unit matrix, the process noise covariance matrix is ​​0.01I, and the observation noise covariance matrix is ​​0.01I. The filter uses the previous state estimate and the new observation data to continuously optimize the estimation of the vehicle position and posture through prediction and update steps. In the prediction stage, the filter predicts the state at the next moment based on the vehicle's motion model, while considering the dynamic characteristics and process noise of the system. When the new depth information and uncertainty are obtained, the filter enters the update stage, combines the observation data with the predicted state through the measurement model, corrects the prediction error, and improves the positioning accuracy. The design of the filter needs to consider the stability and computational efficiency of the algorithm to meet the real-time requirements. In addition, the filter should also have good robustness and be able to handle sensor noise, environmental interference and nonlinear factors. By reasonably setting the parameters of the filter, such as the process noise covariance and the observation noise covariance, the uncertainty of the prediction and the credibility of the observation data can be balanced. Ultimately, the parking positioning result output by the filter will provide the vehicle with accurate position information, which is crucial for autonomous vehicles to navigate, avoid obstacles, and park accurately in complex parking environments. Accurate positioning results can help improve parking efficiency, reduce collision risks, and enhance user experience.

[0095] This embodiment identifies the optical flow tracking results to obtain occluded feature points, uses a depth estimation algorithm to perform corrections based on the occluded feature points to obtain corrected feature points, and performs calculations based on the corrected feature points according to the synchronous positioning algorithm to obtain parking positioning results, thereby improving the accuracy and robustness of parking positioning.

[0096] This application also provides a parking positioning optimization device, please refer to Figure 4 , the device comprises:

[0097] An acquisition module 10 is used to acquire vehicle environment image information;

[0098] A processing module 20, configured to generate a depth map according to the vehicle environment image information;

[0099] The processing module 20 is further used to perform optical flow tracking processing according to the depth map to obtain an optical flow tracking result;

[0100] The result module 30 is used to perform correction based on the optical flow tracking result by using a preset algorithm to obtain a parking positioning result.

[0101] The parking positioning optimization device provided by the present application adopts the parking positioning optimization method in the above embodiment, which can solve the technical problem of how to improve the accuracy of parking positioning. Compared with the prior art, the parking positioning optimization device provided by the present application has the same beneficial effects as the parking positioning optimization method provided by the above embodiment, and the other technical features in the parking positioning optimization device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0102] In one embodiment, the processing module 20 is further used to obtain image feature points based on the depth map; and obtain optical flow tracing results according to the image feature points by using a preset optical flow tracing energy model.

[0103] In one embodiment, the processing module 20 is further used to establish an initial optical flow tracing energy model; the initial optical flow tracing energy model is trained according to the vehicle environment image information to obtain a preset optical flow tracing energy model.

[0104] In one embodiment, the processing module 20 is also used to obtain the current frame and the most recent key frame based on the image feature points; based on the current frame and the most recent key frame, obtain the first image, the first time, the second image, the second time, the weight and the photometric affine parameters, wherein the first image is the image of the most recent key frame, the first time is the exposure time of the most recent key frame, the second image is the image of the current frame, and the second time is the exposure time of the current frame; input the first image, the first time, the second image, the second time, the weight and the photometric affine parameters into a preset optical flow tracking energy model to obtain an optical flow tracking result.

[0105] In one embodiment, the result module 30 is also used to identify the optical flow tracking results to obtain the obscured feature points; use the depth estimation algorithm to correct the obscured feature points to obtain the corrected feature points; and calculate based on the corrected feature points according to the synchronous positioning algorithm to obtain the parking positioning result.

[0106] In one embodiment, the result module 30 is also used to iteratively optimize the depth estimation algorithm by using the confidence propagation method; the depth estimation algorithm is iteratively optimized by using the confidence propagation method, including: setting the maximum number of iterations and the relaxation factor; determining the position relationship between the nearest key frame and the non-key frame of the occluded feature point through the relaxation factor; estimating the rough position of the non-key frame by using grayscale block matching, and giving the uncertainty; comparing the position and uncertainty of the key frame and the non-key frame to obtain the depth uncertainty; until the depth uncertainty is less than a preset threshold or the number of iterations reaches the maximum number of iterations, a depth estimation algorithm is obtained.

[0107] In one embodiment, the result module 30 is further used to obtain depth information and uncertainty of the updated key frame based on the corrected feature points; and obtain the parking positioning result by passing the depth information and uncertainty through a preset filter.

[0108] The present application provides a parking positioning optimization device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the parking positioning optimization method in the above-mentioned embodiment 1.

[0109] Reference below Figure 5 , which shows a schematic diagram of the structure of a parking location optimization device suitable for implementing the embodiment of the present application. The parking location optimization device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The parking positioning optimization device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0110] like Figure 5As shown, the parking positioning optimization device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the xxx device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the parking location optimization device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a parking location optimization device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0111] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0112] The parking positioning optimization device provided by the present application adopts the parking positioning optimization method in the above embodiment, which can solve the technical problem of how to improve the accuracy of parking positioning. Compared with the prior art, the parking positioning optimization device provided by the present application has the same beneficial effects as the parking positioning optimization method provided by the above embodiment, and the other technical features in the parking positioning optimization device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0113] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0115] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the parking positioning optimization method in the above-mentioned embodiment.

[0116] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0117] The computer-readable storage medium may be included in the parking positioning optimization device; or may exist independently without being assembled into the parking positioning optimization device.

[0118] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the parking location optimization device, the parking location optimization device can be written in one or more programming languages ​​or a combination thereof to perform computer program codes for the operation of the present application. The programming languages ​​include object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0119] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0120] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0121] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned parking positioning optimization method, and can solve the technical problem of how to improve the accuracy of parking positioning. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the parking positioning optimization method provided in the above-mentioned embodiment, and will not be repeated here.

[0122] The present application also provides a computer program product, including a computer program, which implements the steps of the parking positioning optimization method as described above when executed by a processor.

[0123] The computer program product provided in this application can solve the technical problem of how to improve the accuracy of parking positioning. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the parking positioning optimization method provided in the above embodiment, which will not be repeated here.

[0124] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A parking positioning optimization method, characterized in that: The method comprises: Acquire vehicle environment image information; Generate a depth map according to the vehicle environment image information; Performing optical flow tracking processing according to the depth map to obtain an optical flow tracking result; Based on the optical flow tracking result, a preset algorithm is used to perform correction to obtain a parking positioning result.

2. The method according to claim 1, characterized in that The step of performing optical flow tracking processing according to the depth map to obtain an optical flow tracking result includes: Based on the depth map, obtaining image feature points; The optical flow tracing result is obtained according to the image feature points through a preset optical flow tracing energy model.

3. The method according to claim 2, characterized in that Before the step of obtaining the optical flow tracing result according to the image feature points by using a preset optical flow tracing energy model, the method includes: Establish an initial optical flow tracing energy model; The initial optical flow tracing energy model is trained according to the vehicle environment image information to obtain a preset optical flow tracing energy model.

4. The method according to claim 2, characterized in that The step of obtaining the optical flow tracing result according to the image feature points by using a preset optical flow tracing energy model comprises: According to the image feature points, get the current frame and the nearest key frame; Based on the current frame and the most recent key frame, a first image, a first time, a second image, a second time, a weight, and a photometric affine parameter are obtained, wherein the first image is an image of the most recent key frame, the first time is an exposure time of the most recent key frame, the second image is an image of the current frame, and the second time is an exposure time of the current frame; The first image, the first time, the second image, the second time, the weight and the photometric affine parameter are input into the preset optical flow tracing energy model to obtain the optical flow tracing result.

5. The method according to claim 1, characterized in that The step of using a preset algorithm to perform correction based on the optical flow tracking result to obtain a parking positioning result includes: Identify the optical flow tracking result to obtain blocked feature points; Using a depth estimation algorithm to correct the blocked feature points to obtain corrected feature points; The parking positioning result is obtained by performing calculation based on the corrected feature points according to the synchronous positioning algorithm.

6. The method according to claim 5, characterized in that Before the step of correcting the blocked feature points by using a depth estimation algorithm to obtain the corrected feature points, the method includes: The confidence propagation method is used to iteratively optimize the depth estimation algorithm; The method of using confidence propagation to iteratively optimize the depth estimation algorithm includes: Set the maximum number of iterations and relaxation factor; Determine the positional relationship between the nearest key frame and the non-key frame of the occluded feature point by using the relaxation factor; Use grayscale block matching to estimate the rough position of non-key frames and give the uncertainty; Comparing the positions and uncertainties of the key frame and the non-key frame to obtain depth uncertainty; Until the depth uncertainty is less than a preset threshold or the number of iterations reaches the maximum number of iterations, the depth estimation algorithm is obtained.

7. The method according to claim 5, characterized in that The step of calculating based on the corrected feature points according to a synchronous positioning algorithm to obtain a parking positioning result includes: Based on the corrected feature points, the depth information and uncertainty of the updated key frame are obtained; The depth information and the uncertainty are filtered through a preset filter to obtain a parking positioning result.

8. A parking positioning optimization device, characterized in that: The device comprises: An acquisition module, used for acquiring vehicle environment image information; A processing module, used for generating a depth map according to the vehicle environment image information; The processing module is further used to perform optical flow tracking processing according to the depth map to obtain an optical flow tracking result; The result module is used to perform correction based on the optical flow tracking result by using a preset algorithm to obtain a parking positioning result.

9. A parking positioning optimization device, characterized in that: The device comprises: a memory, a processor, and a parking positioning optimization program stored in the memory and executable on the processor, wherein the parking positioning optimization program is configured to implement the steps of the parking positioning optimization method according to any one of claims 1 to 7.

10. A medium, characterized in that The medium stores a parking positioning optimization program, and when the parking positioning optimization program is executed by the processor, the steps of the parking positioning optimization method according to any one of claims 1 to 7 are implemented.

Images